Value decomposition multi-agent reinforcement learning training method using attention network

Abstract

The invention provides a value decomposition multi-agent reinforcement learning training method using an attention network. The method comprises experience collection using an exploration strategy and joint action value function calculation. When the joint action value function is calculated, two methods are used to improve the effect: an attention network and a separated value decomposition structure are used. The attention network uses the global state as input, so that the network can pay attention to important parts in the global state, and the weight is calculated more accurately. The separated value decomposition structure adopts two layers of weighted networks, and each layer of network uses different global state codes, so that the difficulty of super network learning is reduced. And the independent action value function of each agent is fused by using a super network, and finally a joint action value function is output. The attention network and the separated value decomposition structure used in the invention effectively improve the learning efficiency, accelerate the model convergence, and improve the effect of the multi-agent cooperation strategy.

Description

technical field [0001] The invention relates to a reinforcement learning and training method, in particular to a multi-agent reinforcement learning and training method using a value decomposition of an attention network. Background technique [0002] Reinforcement learning (RL) is a subdivision of machine learning, which is mainly used to solve sequential decision-making problems. Through continuous interaction with the environment, the agent continuously explores and optimizes the strategy, and finally achieves the maximization of the return or the established goal. In multi-agent reinforcement learning, multiple agents need to work together to accomplish the same goal. Because of this, multi-agent reinforcement learning is closer to real-world problems and has more practical significance. [0003] In the multi-agent environment, there are some problems that do not exist in the single-agent environment, such as environmental instability, dimensional explosion and credit a...

AI Technical Summary

Problems solved by technology

[0003] In the multi-agent environment, there are some problems that do not exist in the single-agent environment, such as unstable environment, dimension explosion and credit allocation problems, etc.

Images