Strategy collaborative selection method based on deep reinforcement learning DDPG algorithm framework

Abstract

The invention discloses a strategy cooperation selection method based on a deep reinforcement learning DDPG algorithm framework. According to the strategy cooperation selection method, an output action is selected by adopting a strategy cooperation mode, a pair of strategy network output actions are used for evaluation, a Q value obtained by evaluation is used as a weight, and the action is selected by using a probability. Policy cooperation can reduce the possibility of local optimum, improve overfitting, reduce strategy fluctuation and increase stability. And in addition, dropout is added into the actor network, so that the coupling is reduced, the generalization is improved, and the training speed is increased. Meanwhile, according to the thought of a TD3 algorithm, noise is added after the action is selected by the actor target network so as to reduce the size of errors, the network structure of the DDPG is changed, the problem of over-estimation of the DDPG is effectively solved, and the problem of overlarge fluctuation of a strategy network is avoided.

Description

technical field [0001] The present invention relates to the technical field of reinforcement learning, in particular to a method for strategic cooperation selection based on a deep reinforcement learning DDPG algorithm framework. Background technique [0002] The problem discussed in reinforcement learning is how an agent finds a strategy to maximize the reward it can obtain in a complex and uncertain environment. Lillicrap et al. proposed the DDPG (deep deterministic policy gradient) algorithm in 2015, which is a deep reinforcement learning algorithm on the actor-critic framework (Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez , T., Tassa, Y., Silver, D., & Wierstra, D. (2015). Continuous control with deep reinforcement learning.). DDPG is the first reinforcement learning algorithm to efficiently solve many high-dimensional continuous control tasks. It is also a deterministic policy gradient algorithm based on actor-critic architecture. Contains actor current ...

AI Technical Summary

Problems solved by technology

[0003] The purpose of the present invention is to provide a strategy collaborative selection method based on the deep reinforcement learning DDPG algorithm framework, which changes the network structure of DDPG, effectively improves the overestimation problem of DDPG, and avoids the problem of excessive fluctuation of the strategy network

Images