Multi-Agent RL Disrupted Agent Detection with Action Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-agent reinforcement learning environments, detecting abnormal actions of agents disrupted by external attacks is challenging, which can degrade system performance, and existing methods do not efficiently address this issue.
Innovation Solution
A method and apparatus for calculating an action score for an agent's actions based on observation and action space information, determining abnormal actions by comparing scores against thresholds, and adjusting scores based on overlapping observation ranges, to identify disrupted agents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If agents exchange observation information and action space information to improve detection accuracy, then the reliability of disrupted agent detection is improved, but the communication bandwidth requirement increases
Solution Approach 1:
The patent extracts only the essential information needed for disrupted agent detection from the full observation and action space data. By calculating action scores based on selected features rather than transmitting complete information, the system achieves reliable detection while minimizing communication bandwidth usage between agents.
Solution Approach 2:
The patent applies local quality by having each agent independently calculate action scores for specific actions in the action space of other agents based on their local observation information. This distributed calculation approach improves detection reliability without requiring centralized processing or excessive communication of raw data.
2Measurement precision
If all action information is transmitted between agents for accurate detection, then the measurement precision of agent behavior is improved, but the loss of time for information processing increases
Solution Approach 1:
The patent applies preliminary action by pre-calculating action scores for actions in the action space before actual detection is needed. Each agent maintains pre-computed value calculations according to learned policies, enabling rapid disruption detection without time-consuming real-time analysis of complete action information.
Solution Approach 2:
The patent uses partial action by calculating action scores only for relevant actions in the action space rather than analyzing all possible actions. This selective calculation approach maintains measurement precision for critical behaviors while reducing overall processing time through focused analysis.
Data Source
AI summary
A method and an apparatus for detecting a disrupted agent in multi-agent reinforcement learning environment. An embodiment of the present disclosure provides a method for detecting disrupted agent in multi-agent reinforcement learning environment, including: calculating, by the first agent, an action score for one or more of the actions included in the action space of the second agent, based on one or more of observation information and action space information received from one or more other agents; and determining, based on the action score, whether the second agent is the disrupted agent, wherein the action score is a value calculated based on a value calculated according to a learned policy for each action, and is an index having a higher value for a relatively important action among actions that may be performed by the agent.