A multi-agent deep reinforcement learning method, system and application

Abstract

The invention discloses a multi-agent deep reinforcement learning algorithm based on partition experience and multi-thread interaction. First, the algorithm uses the experience replay form of the partitioned buffer area to distinguish positive experience, negative experience and neutral experience by dividing the reward space, and uses stratified random sampling to extract these experience data during training. Secondly, the algorithm uses multi-threaded interaction to promote the trial-and-error process between the agent and the environment. Multiple clones of the agent learn in parallel and integrate their learning experience to train the parameters of the network model. The advantages are: the multi-agent deep reinforcement learning algorithm based on buffer replay and multi-thread interaction proposed by the present invention, combined with the advantages of partitioned experience buffer and multi-thread interaction mode, is introduced into the multi-agent deep reinforcement learning algorithm; It is superior to existing models in terms of convergence speed and training efficiency, and has higher usability in a multi-agent environment, and can be used to solve the problem of multi-agent cooperative tracking targets.

Description

technical field [0001] The invention relates to a multi-agent deep reinforcement learning method, system and application, and belongs to the field of multi-agent technology. Background technique [0002] Deep reinforcement learning is an efficient strategy search algorithm that combines deep learning (Deep Learning, DL) and reinforcement learning (Reinforcement Learning, RL). Extract data features in the dimensional state space and search for the optimal behavior strategy. At present, the research results of DRL can be applied to multi-agent systems, in order to realize complex combat tasks such as mutual cooperation and competition among multi-agents. However, DRL is difficult to adapt to the dynamic and changeable environment, and faces many problems in the research: First, the learning efficiency is low: the essence of DRL is a trial-and-error learning process, and the learning experience is generated through the continuous interaction between the agent and the environme...

AI Technical Summary

Problems solved by technology

However, DRL is difficult to adapt to the dynamic and changeable environment, and faces many problems in the research: First, the learning efficiency is low: the essence of DRL is a trial-and-error learning process, and the learning experience is generated through the continuous interaction between the agent and the environment and stored in it into the cache

Due to the uneven quality of experience, this will make it difficult for the network model to learn effective sample data; second, the training time is long: with the increase of the number of agents, the action space shows an exponential growth trend, and the dimension of decision output will become more and more bigger

Moreover, the behavior decision of each agent not only needs to consider the state of the environment in which it is located, but also consider the impact of the decisions taken by other agents on its own strategy, which will lead to slow neural network training and even difficult convergence.

Images