Visual analysis method for maddpg multi-agent reinforcement learning model

Abstract

The invention belongs to the field of information technology, and discloses a visual analysis method for MADDPG multi-agent reinforcement learning model, comprising the following steps: Step 1: Select a cooperative game as the running environment of the MADDPG model, and define relevant parameters Collection; Step 2: Train the MADDPG model, save and compute important intermediate data; Step 3: Design a label board to identify agents and landmarks; Step 4: Design a statistical view; Step 5: Design a critic behavior view for evaluating the model Learn the performance of the obtained critics; Step 6: Design the interactive view. The invention proposes a new visual analysis method, which can support the interactive analysis of the work flow and internal principle of the MADDPG model in the cooperative environment. The invention designs multiple synergistic views to reveal the internal execution mechanism of the MADDPG model from different perspectives.

Description

technical field [0001] The invention belongs to the field of information technology, and in particular relates to a visual analysis method for MADDPG multi-agent reinforcement learning model. Background technique [0002] Deep reinforcement learning is a very hot research field today, and it has been used to solve various challenging application problems such as autonomous driving, traffic control, and robotic system control. Despite the superior performance of deep reinforcement learning in these applications, researchers still know very little about their intrinsic execution mechanisms. In recent years, researchers have proposed various visual analysis methods to improve the interpretability of deep reinforcement learning models. For Q-Network (DQN), a visual analysis system DQNViz is designed to reveal the experience space of agents from different levels. For competing deep Q-network (dueling DQN) and Asynchronous Advantage Actor-Critic models, saliency maps are generat...

AI Technical Summary

Problems solved by technology

Compared with single-agent deep reinforcement learning, analyzing multi-agent deep reinforcement learning models is more challenging, mainly because: 1) The increase in the number of agents leads to an exponential growth of the state space, how to visualize the experience generated by multiple agents spaces and reveal potential connections between them? 2) Multiple agents are constantly interacting with different environmental objects (landmarks), how to intuitively visualize the interaction process over time? Existing research lacks interpretability research on multi-agent deep reinforcement learning models

Images