An Efficient Value Function Iterative Reinforcement Learning Method for Shared Recurrent Neural Networks

Abstract

The invention discloses a high-efficiency value function iterative reinforcement learning method for a shared cyclic neural network. The method includes: obtaining sample data through the interaction between an agent and the environment, and adding the sample data to a sample pool; randomly selecting the sample data from the sample pool The sample data is used as the training sample data; the output of the Critic network is normalized according to the training sample data, and its MLP network and shared LSTM parameters are updated; The parameters of the MLP part of the Critic network are updated; the third Critic network and the fourth Critic network in the Critic network, and the second Actor network parameters of the Actor network are updated. The method combines recurrent neural network with value function iteration to improve algorithm training efficiency and shorten algorithm training time.

Description

technical field [0001] The invention relates to the technical field of reinforcement learning, in particular to a high-efficiency value function iterative reinforcement learning method of a shared cyclic neural network. Background technique [0002] Reinforcement learning is based on the theoretical framework of Markov decision process, which models the sequential decision-making task as a trial-and-error learning problem of the interaction between the agent and the system environment. Two types of model-free reinforcement learning algorithms, value function iteration methods and policy optimization methods, are widely used to solve various decision-making problems. Compared with the strategy optimization method, the value function iteration method can use the data generated by the historical strategy to update the algorithm, so it requires fewer interactions with the environment, has a higher utilization rate of samples, and is more capable of solving real environment decis...

AI Technical Summary

Problems solved by technology

Although the policy search method using the recurrent neural network has the ability to solve some observable problems in the environment, due to the fact that this type of method faces the problems of long time consumption and high sampling cost for the interaction between the agent and the environment in actual tasks

Images