A deep deterministic strategy gradient learning method based on a reviewer and double experience pools

Abstract

The invention provides a deep deterministic strategy gradient reinforcement learning method and device based on a reviewer and double experience pools for an intelligent unmanned system. The inventiondiscloses an intelligent unmanned system, which belongs to the technical field of artificial intelligence application and mainly comprises the following steps: determining the observation space and action space size of an intelligent body of the intelligent unmanned system, and constructing an actor module and a reviewer critic module; Creating a plurality of reviewer sub-modules in the critic module; Creating an annular array data structure of the double experience pools; And performing a parameter gradient updating and training process of the actor module and the critic module, and ending the training process when the maximum number of iterations is achieved or a termination condition is met. According to the invention, a reinforcement learning method with higher stability and higher performance can be provided, so that the performance of the intelligent agent is effectively improved.

Description

technical field [0001] The invention belongs to the technical field of computer artificial intelligence, and in particular relates to a deep deterministic strategy gradient reinforcement learning method with multiple critics and double experience pools. Background technique [0002] In recent years, artificial intelligence technology has set off a huge wave, and various related intelligent information technologies have emerged one after another. Deep reinforcement learning (Deep Reinforcement Learning) combines the perception ability of deep learning and the decision-making ability of reinforcement learning itself, and has become a The focus of attention (see literature [1,2]). Not long ago, DeepMind's AlphaGo Go program based on the reinforcement learning method defeated the top professional Go player Li Shishi (see literature [3]), and then proposed an upgraded version of AlphaGo Zero in the following year to defeat human players in Go, chess, Japanese shogi, etc. And its...

AI Technical Summary

Problems solved by technology

[0015] Aiming at the deficiencies in the existing technology, the present invention proposes a deep deterministic policy gradient reinforcement learning method based on critics and dual experience pools for intelligent unmanned systems. By designing the deep deterministic policy gradients of multiple critics, Aiming at solving the training stability problem of the existing technology and improving the performance of the method, the convergence speed of the training process is improved by designing a double experience pool, so as to realize a reinforcement learning method with higher performance and higher stability

Images