Hierarchical reinforcement learning method and device based on strategy options

Abstract

The invention discloses a hierarchical reinforcement learning method and device based on strategy options. The method comprises the following steps: constructing a high-level strategy network, a low-level strategy network and an evaluation network; acquiring a state track from the simulation environment; updating the parameters of the high-level strategy network, the low-level strategy network andthe evaluation network based on the state trajectory and the learning process of the strategy online algorithm; generating an updated strategy model according to the high-level strategy network, thelow-level strategy network and the evaluation network of which the parameters are updated, and testing the updated strategy model. According to the hierarchical reinforcement learning method based onthe strategy option, the action and the high-level strategy can be learned from zero in the simulation environment, the performance is stable, and the data use efficiency is high.

Description

technical field [0001] The invention relates to the technical field of machine learning, in particular to a strategy option-based layered reinforcement learning method and a strategy option-based layered reinforcement learning device. Background technique [0002] In recent years, deep learning has made great breakthroughs in the fields of pattern recognition, computer vision, and natural language processing. Combining the perception ability of deep learning and the decision-making ability of reinforcement learning produces deep reinforcement learning, which can be directly controlled according to the input image, which is an artificial intelligence method of thinking mode. For example, deep reinforcement learning has achieved superhuman performance in specific scenarios such as Go and some e-sports games, and has gradually been applied to nonlinear system control, confrontational artificial intelligence design and other fields. In traditional reinforcement learning, becaus...

AI Technical Summary

Problems solved by technology

[0003] Hierarchical reinforcement learning based on policy options is a branch of hierarchical reinforcement learning. Although existing learning algorithms based on policy options can enable agents to learn relatively complex high-level strategies, they also have unstable performance and low data usage efficiency. And other issues

Images