Multi-agent deep reinforcement learning method, system and application

Abstract

The invention discloses a multi-agent deep reinforcement learning algorithm based on partition experience and multi-thread interaction. Firstly, the algorithm can be used for distinguishing positive experience, negative experience and neutral experience by dividing a reward space by using an experience replay form of a partitioned cache region, and extracting experience data by using a layered random sampling mode during training; and secondly, the algorithm promotes the trial and error process of the intelligent agent and the environment by applying a multi-thread interaction mode, and parameters of a network model are trained through parallel learning of multiple clone bodies of the intelligent agent and integration of learning experience of the clone bodies. The method has the advantages that the multi-agent deep reinforcement learning algorithm based on cache region replay and multi-thread interaction is introduced into the multi-agent deep reinforcement learning algorithm by combining the advantages of a partitioned experience cache region and a multi-thread interaction mode; and the method is superior to an existing model in convergence speed and training efficiency, has higher availability in a multi-agent environment, and can be used for solving the problem of cooperative target tracking of multiple agents.

Description

Technical field[0001]The present invention relates to a multi-intelligent depth strengthening learning method, system and application, belonging to the field of intelligent body.Background technique[0002]Deep strengthening learning is a high-efficiency policy search algorithm that combines deep learning (Reinforcement Learning, RL). It uses powerful characteristics of artificial neural network to enable enhanced learning to be complex. Data characteristics are extracted in the dimensional space and perform a search for the best behavior strategy. At present, DRL's research results can be applied to multi-smart systems, in order to achieve multi-smart, complex combat tasks such as competitance. However, DRL is difficult to adapt to dynamic changes, and there are many problems in research: First, the learning efficiency is low: the essence of DRL is a test error study process, and learning experience through intelligent interaction and environment, and departments Go to the cache area...

AI Technical Summary

Problems solved by technology

However, DRL is difficult to adapt to the dynamic and changeable environment, and faces many problems in the research: First, the learning efficiency is low: the essence of DRL is a trial-and-error learning process, and the learning experience is generated through the continuous interaction between the agent and the environment and stored in it into the cache

Due to the uneven quality of experience, this will make it difficult for the network model to learn effective sample data; second, the training time is long: with the increase of the number of agents, the action space shows an exponential growth trend, and the dimension of decision output will become more and more bigger

Moreover, the behavior decision of each agent not only needs to consider the state of the environment in which it is located, but also consider the impact of the decisions taken by other agents on its own strategy, which will lead to slow neural network training and even difficult convergence.

Images