Collaborative multi-agent reinforcement learning method

Abstract

The invention discloses a collaborative multi-agent reinforcement learning method. The method comprises the following steps: obtaining observation information of each agent and a global state of a system; transmitting the obtained observation information of each intelligent agent into a deep neural network to calculate and obtain state action values of all actions of the intelligent agent; performing action selection by utilizing a greedy rule; transmitting the state action value corresponding to the adopted action and the global observation information into a reward highway network; rewardingthe highway network to perform information fusion and inputting a combined state action value; and performing gradient back transmission by utilizing a reward signal given by the environment and updating parameters of the neural network so as to obtain a strategy model of each intelligent agent. The data volume required in the training process of the multi-agent system can be reduced, and the invention is suitable for being popularized to large-scale multi-agent systems.

Description

technical field [0001] The invention belongs to the field of automatic control, in particular to a collaborative multi-agent reinforcement learning method. Background technique [0002] Many problems in the real world can be regarded as cooperative multi-agent problems, such as: UAV swarm control, multi-player video games, control of sensor networks, etc. The use of reinforcement learning algorithms to control multi-agent systems has also been widely used in the above fields. However, collaborative multi-agent reinforcement learning algorithms face three main challenges that make it difficult to achieve good results in complex scenarios. The first is that the joint action space of agents explodes exponentially with the increase of the number of agents, which makes the direct application of single-agent control algorithms to multi-agent systems bring the curse of dimensionality. Second, the environment of a multi-agent system often only gives a global feedback signal corres...

AI Technical Summary

Problems solved by technology

In addition, the reward distribution method in the above algorithm is completely unsupervised, and the expressive ability of the centralized structure greatly affects the effectiveness of the global reward distribution mechanism

Images