Multi-agent cooperation model based on deep reinforcement learning

Abstract

The invention discloses a multi-agent cooperation model based on deep reinforcement learning, which comprises a centralized Critic network, a plurality of distributed Actor networks and a Q value decomposition network. Each Actor network interacts with an environment to generate state-action information, the state-action information is stored in an empirical buffer area, the Critic network samples from the empirical buffer area, all state-action information serves as input, a global reward R is designed with the task of the cooperation multi-agent system as a target, and a global action value Qtot is obtained through learning in a TD error mode; the Q value decomposition network decomposes the global action value Qtot into action value Qi based on a single agent, and gradient update of each Actor network depends on the action value Qi of the corresponding single agent after decomposition. According to the method, the consistency of the global optimal action and the local optimal action is ensured, so that the exploration efficiency of multiple agents in a continuous action space is improved.

Description

technical field [0001] The invention relates to the technical field of multi-agent reinforcement learning, in particular to a multi-agent cooperation model based on deep reinforcement learning. Background technique [0002] MAS is a distributed decision-making system composed of multiple agents interacting with the environment. Since the 1970s, MAS has carried out numerous researches, the purpose of which is to establish a swarm intelligence system with a specific autonomous level and autonomous learning ability. The characteristics of MAS information sharing, distributed computing and collaborative execution have a very wide range of application requirements in real life, especially in many fields such as military, industry, and transportation. In decision-making optimization problems, reinforcement learning shows a huge advantage in online learning, and it is more in line with the learning mechanism of biological groups. With the upsurge of reinforcement learning led by ...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the problems of low efficiency, slow convergence speed and poor stability of the existing multi-agent reinforcement learning methods, this application provides a multi-agent cooperation model based on deep reinforcement learning, which ensures the global optimal action and local optimal action. Consistency, thereby improving the efficiency of multi-agent exploration in continuous action spaces

Images