Multi-agent reinforcement learning method and system based on hierarchical attention mechanism

Abstract

The invention provides a multi-agent reinforcement learning method and system based on a hierarchical attention mechanism, and the method comprises the steps: building a learning environment which comprises a plurality of agents, enabling a Critic network to calculate estimated values based on the observation values and action values, obtained by the hierarchical attention mechanism of each agent,of other agents in the plurality of agents, optimizing through a minimum joint loss function until the minimum joint loss function converges, calculating an action value function based on the observed value, the action value and the trained estimated value in combination with an estimated Actor network, optimizing through a maximized dominance function until an optimal action value function is obtained, and calculating an optimal action-value function based on the optimal action value function. and performing a deterministic action; A hierarchical attention mechanism is combined with an Actor-Critic network framework, so that the expandability of the intelligent agent in an environment to which the intelligent agent belongs can be enhanced for learning.

Description

technical field [0001] The invention relates to a multi-agent, in particular to a multi-agent reinforcement learning method and system based on a layered attention mechanism. Background technique [0002] Deep reinforcement learning has made remarkable progress in many domains, such as Atari games, Go, and complex continuous control tasks related to locomotion. We usually refer to the robot that learns and implements decisions as the agent, and everything that interacts with it outside the agent is called the environment. The agent chooses actions, the environment responds to these actions, and presents a new state to the agent. At the same time, the environment will also generate a benefit (ie reward), which is the goal that the agent wants to maximize in the process of selecting actions. This series of decision-making processes can be modeled as a Markov decision process (Markov Decision Process, MDP), which is an idealized form of reinforcement learning problems in math...

AI Technical Summary

Problems solved by technology

This idea is not scalable, and the size of its state space and action space grows exponentially with the increase in the number of agents.

Moreover, since the central controller must collect the information of each agent and distribute the decision to each agent during the decision-making process, there are very high communication requirements for the agents that execute the decision, which is difficult to achieve in the real world.

Images