Actor-Critic neural network continuous control-based fast learning algorithm

Abstract

The invention relates to an Actor-Critic neural network continuous control-based fast learning algorithm. The algorithm comprises the following steps of initializing an experience pool, initializing aneural network, constructing output interference, accumulating the experience tool, and sampling and training a deep reinforcement learning neural network according to a priority number prop. According to the algorithm, sampling is optimized according to the priority number prop calculated by TD-diff, sigmoid-TD and a using trace UT, and a convergence speed of TD-error is improved, so that the learning speed of the algorithm is improved.

Description

technical field [0001] The invention belongs to the technical field of reinforcement learning algorithms, in particular to a fast learning algorithm based on Actor-Critic neural network continuous control. Background technique [0002] In recent years, deep reinforcement learning has been brilliant. The Go program AlphaGo developed by Google successfully defeated the world's top chess player Lee Sedol, setting off a wave of artificial intelligence worldwide. The success of AlphaGo is attributed to the deep reinforcement learning algorithm. Most of the current deep reinforcement learning algorithms use the method of memory replay. The concept of memory playback was proposed in 1993. By 2013, with the introduction of the DQN algorithm, memory playback has been widely used in various aspects of deep reinforcement learning. However, because memory playback often adopts random sampling, the neural network repeatedly learns some states, and the priority state cannot be learned, a...

AI Technical Summary

Problems solved by technology

The method based only on the size of the time difference error TD_error sometimes performs worse than random sampling, resulting in low learning efficiency

Images