A Reinforcement Learning Approach for Agent Contribution Assignment under Multi-Agent Cooperation Tasks

Abstract

The invention discloses a method for assigning agent contribution under the multi-agent cooperation task of reinforcement learning, which includes: each agent independently observes the environment state, inputs its own strategy network, and obtains its own action decision; executes the action of each agent in a simulated environment , the global reward of environmental feedback; model the interaction between agents as an undirected graph, use this graph to calculate the contribution weight of each agent; use the contribution weight of each agent to calculate the local reward of each agent, Each agent's respective policy network is trained using this local reward. This method can contribute to the results (rewards) of the interaction between multiple agents and the environment, which can play the role of credibility assignment, provide a more accurate reward description for the training algorithm, and help the multi-agent system learn from cooperative tasks. better strategy.

Description

technical field [0001] The invention belongs to the field of artificial intelligence automation, and particularly relates to a method for assigning agent contributions under a multi-agent reinforcement learning cooperation task. Background technique [0002] The use of reinforcement learning to train agents to perform tasks is a common solution in the modern field of AI automation, where many scenarios rely on multiple agents cooperating to achieve a common goal. In the multi-agent cooperation system, the global reward information can be used as the training signal of the global value function and the value function of each agent, or the contribution degree of each agent can be allocated separately, the local reward of each agent can be estimated, and it can be used as the value of each agent. The learning signal provides gradients for training the policy or value network of each agent. [0003] The engineering implementation of using all the rewards of each agent is relati...

AI Technical Summary

Problems solved by technology

This method is easy to implement and control when the number of agents is small, but when the number of agents is large, the parameters of the model will increase exponentially, making the model difficult to train

In this case, considering the workload and difficulty of engineering implementation, the method of distributed and independent training of each agent model is usually used to reduce the difficulty of model training. However, this method requires local rewards for each agent to provide independent training signal, in the case where the environment only provides a global reward signal, the distribution of contribution to estimate the contribution of each independent agent becomes an urgent problem to be solved

Images