Task deduction and capability evolution method and system based on multi-agent antagonistic mutual feeding

Abstract

The application discloses a kind of task deduction and capability evolution method and system based on multi-agent confrontation mutual feed, method is: immersion situation is generated and red and blue confrontation agent is initialized;Multi-agent confrontation deduction is executed and tactical layer online evolution is realized;Quantitative index is constructed based on ability digital image and short board is diagnosed;According to short board, directional stress injection is completed;Strategy layer offline evolution is realized using multi-agent reinforcement learning;Closed-loop iteration drives capability autonomous evolution.The system includes five modules of situation generation, multi-agent engine, double-layer evolution control, capability evaluation and stress injection, data storage and model training.The application solves the problem of traditional simulation training static, one-way, extensive evaluation, realizes scene dynamic realistic, accurate quantization of ability, continuous evolution of confrontation, and can be widely applied to intelligent deduction and capability training of high-risk tasks such as emergency rescue and military confrontation.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, computer simulation and simulation training technology, and in particular relates to a method and system for task inference and capability evolution based on multi-agent adversarial feedback. Background Technology

[0002] In high-risk, high-pressure, and highly uncertain tasks such as earthquakes, emergencies, and major accidents, pre-event simulations and training are core components for improving response capabilities and optimizing decision-making. Traditional simulation training systems generally adopt scripted, fixed-process simulation modes, which have three insurmountable drawbacks:

[0003] 1. Static nature: The opponent's behavior patterns are fixed, the environmental parameters are preset, and the training content is highly repetitive, making it impossible to adapt to the complex situation of dynamic evolution in the real world;

[0004] 2. One-way nature: The system cannot make intelligent adjustments based on the trainee's real-time performance, and cannot conduct targeted reinforcement training for weaknesses in ability;

[0005] 3. The assessment is crude: it relies on manual review after the fact and lacks a real-time, quantitative, and multi-dimensional capability measurement system and a self-evolutionary guidance mechanism.

[0006] Existing simulation techniques based on multi-agent systems and reinforcement learning are mostly applied to games or simple environments. When simulating high-risk tasks, they generally suffer from technical problems such as insufficient scenario realism, low credibility of agent behavior, uncontrollable evolutionary direction, and inability to quantify and evaluate. Therefore, there is an urgent need for a technical solution that can generate highly realistic dynamic scenarios, realize red-blue multi-party adversarial feedback and co-evolution, and accurately drive the systematic improvement of trainees' abilities. Summary of the Invention

[0007] This invention aims to overcome the shortcomings of existing high-risk task simulation training systems, which are static, unidirectional, and have crude evaluation methods. It provides an intelligent inference method that features dynamic adversarial, bidirectional feedback, and capability-oriented evolution. This method enables high-fidelity dynamic generation of training scenarios, accurate quantitative evaluation of trainees' capabilities, and adaptive adjustment of adversarial intensity, thereby significantly improving the realism, relevance, and effectiveness of simulation training.

[0008] To achieve the above objectives, this invention provides a method and system for task deduction and capability evolution based on multi-agent adversarial feedback. The core is a two-layer evolutionary architecture. In a generative immersive scenario, a targeted pressure injection mechanism drives the adversarial agents to achieve online adjustment at the tactical level and offline evolution at the strategy level, thereby forming a standardized and quantifiable digital profile of capabilities.

[0009] The present invention provides a task deduction and capability evolution method based on multi-agent adversarial feedback, comprising the following steps:

[0010] Step 1: Construct a generative immersive scenario and initialize a multi-agent system that includes adversarial red and blue teams.

[0011] Step 2: Perform multi-agent adversarial simulation in the scenario, realize online evolution of the tactical layer during the process, and record complete simulation trajectory data.

[0012] Step 3: Based on the simulated trajectory data, construct a multi-dimensional digital profile of the participants' capabilities in real time, and diagnose capability shortcomings through indicator deviation.

[0013] Step 4: Based on the shortcomings in capabilities, inject quantitative pressure into the simulation by adjusting the Red Team's strategy, environmental parameters, or mission objectives.

[0014] Step 5: Based on the trajectory data accumulated through inference, use multi-agent reinforcement learning to train the policy models of each party offline, so as to achieve co-evolution of the policy layer.

[0015] Step 6: Repeat steps 2 to 5 to form a closed-loop iteration of deduction-evaluation-pressure-evolution, driving the continuous improvement of the trainee's capabilities.

[0016] Furthermore, step 2 of the deduction process adopts a two-layer evolutionary model:

[0017] Tactical layer online evolution: Within a single simulation round, the agent makes decisions through a policy network based on real-time environmental conditions and opponent behavior data; the built-in meta-learning module quickly fine-tunes tactical choices based on short-term gains, achieving real-time adaptation within the round.

[0018] Mutual feedback observation: The system collects and records the environmental state sequence, the action sequence of each party, key events, and task completion indicators in full, forming complete inference trajectory data.

[0019] Furthermore, in step 3, based on the projected trajectory data, the system calculates multi-dimensional capability indicators in real time to form a normalized digital capability profile, formally represented as: These correspond to five quantitative indicators: situational awareness, decision response, resource scheduling, collaborative cooperation, and adaptive innovation, with a value range of [0,1].

[0020] The system diagnoses weaknesses by calculating the deviation in capabilities: ;in As a preset capability benchmark threshold, When the i-th capability is determined to be a weakness, δ is the system-defined judgment coefficient.

[0021] Furthermore, in step 4, based on the capability deficiency diagnosis results, the system initiates a targeted pressure injection mechanism, and the pressure intensity quantification formula is as follows: P represents the overall pressure intensity. The weights are adaptive to the weakest link; pressure directly affects the red team's strategy weights, environmental disturbance coefficients, and task difficulty coefficients.

[0022] Furthermore, after step 5 completes one or more rounds of deduction, the offline evolution phase begins:

[0023] (1) Experience pool construction: Store the inferred trajectory data into the hierarchical experience playback pool.

[0024] (2) Reward function design:

[0025] Blue team reward function: ;

[0026] Red team reward function: ;

[0027] Where R_task represents task completion rate, R_time represents time efficiency, R_coop represents collaboration benefits, R_cost represents resource consumption, D_blue represents the blue team's weakness exposure rate, and U_env represents environmental uncertainty gain.

[0028] (3) Co-evolutionary training: The MADDPG and MAPPO multi-agent reinforcement learning algorithms are used to construct a joint loss function:

[0029] ;

[0030] The red and blue strategy networks are trained in a centralized manner and executed in a distributed manner.

[0031] (4) Model update: Update the policy parameters according to the gradient descent rule:

[0032] ;

[0033] ;

[0034] η is the learning rate, and θ is the policy network parameter.

[0035] The present invention provides a task extrapolation and capability evolution system, comprising:

[0036] Context Generation and Management Module: Used to build and maintain generative immersive contexts.

[0037] Multi-agent engine module: Used to manage the operation, decision-making, and interaction of the red and blue agents.

[0038] Dual-layer evolution control module: used to achieve unified scheduling of online evolution at the tactical layer and offline evolution at the strategy layer.

[0039] Capability Assessment and Stress Injection Module: Used to quantitatively build a digital profile of capabilities, diagnose capability gaps, and perform targeted stress injection.

[0040] Data storage and model training module: used to store the inferred trajectory data and perform multi-agent reinforcement learning training.

[0041] Furthermore, the capability assessment and stress injection module includes a weakness diagnosis unit and a stress strategy generator. The weakness diagnosis unit identifies weaknesses based on indicator deviation, and the stress strategy generator maps weaknesses into executable environmental adjustment instructions and red team strategy adjustment weights.

[0042] The beneficial technical effects of this invention compared to the prior art are as follows:

[0043] 1. High dynamic realism: The combination of generative scenarios and multi-agent adversarial approaches makes the inference process dynamic and unpredictable, greatly improving the realism of training.

[0044] 2. Precise and efficient capability evolution: Through quantitative capability digital profiling and targeted pressure injection, personalized and targeted reinforcement of weaknesses can be achieved.

[0045] 3. The level of confrontation continues to rise: the red and blue sides evolve together to avoid the intelligent degradation of traditional system opponents and continuously provide high-level confrontation pressure.

[0046] 4. Reproducible, quantifiable, and examineable: The entire process adopts formal definitions, mathematical formulas, and quantitative indicators, and can be executed automatically by computers, meeting the requirements for full disclosure of invention patents.

[0047] 5. Scientific review and evaluation: The complete data chain records the evolution trajectory of capabilities, supporting in-depth review, solution optimization, and quantitative evaluation of effects. Attached Figure Description

[0048] Figure 1 This is a flowchart of the task deduction and capability evolution method based on multi-agent adversarial feedback of the present invention.

[0049] Figure 2 This is a schematic diagram of the two-layer evolutionary architecture of the present invention.

[0050] Figure 3 This is a schematic diagram illustrating the working principle of the directional pressure injection mechanism of the present invention.

[0051] Figure 4 This is a schematic diagram of the module composition of the task deduction and capability evolution system of the present invention. Detailed Implementation

[0052] The present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0053] The flowchart of a task deduction and capability evolution method based on multi-agent adversarial feedback according to the present invention is shown in Figure 1, and specifically includes the following steps:

[0054] Step 1: Construct a generative immersive scenario and initialize a multi-agent system that includes adversarial red and blue teams.

[0055] Based on geographic information data, social behavior data, and disaster / event mechanism models, generative artificial intelligence technology is used to dynamically construct a high-fidelity interactive virtual task space that includes the physical environment, virtual social actors, and red-blue adversarial intelligent agents; initial strategy models are loaded for both red and blue intelligent agents to complete the unified initialization of situation parameters, resource configuration, and adversarial rules.

[0056] Step 2: Perform multi-agent adversarial simulation in the scenario, realize online evolution of the tactical layer during the process, and record complete simulation trajectory data.

[0057] In a virtual scenario, red and blue intelligent agents engage in adversarial simulations based on task objectives. The blue team's objective is to handle, rescue, or control the situation, while the red team's objective is to interfere, disrupt, or escape. The simulation process employs a two-layer evolutionary mode (e.g., Figure 2 (as shown)

[0058] Tactical layer online evolution: Within a single simulation round, the agent makes decisions through a policy network based on real-time environmental conditions and opponent behavior data; the built-in meta-learning module quickly fine-tunes tactical choices based on short-term gains, achieving real-time adaptation within the round;

[0059] Mutual feedback observation: The system collects and records the environmental state sequence, the action sequence of each party, key events, and task completion indicators in full, forming complete inference trajectory data.

[0060] Step 3: Based on the simulated trajectory data, construct a multi-dimensional digital profile of the participants' capabilities in real time, and diagnose capability shortcomings through indicator deviation.

[0061] Based on the projected trajectory data, the system calculates multi-dimensional capability indicators in real time, forming a normalized digital capability profile, which is formally represented as: These correspond to five quantitative indicators: situational awareness, decision-making response, resource allocation, collaborative cooperation, and adaptive innovation. The value range of these indicators is [0,1]. Specific definitions:

[0062] Situational awareness capability: =Number of effectively perceived targets / Total number of targets in the scene.

[0063] Decision-making and responsiveness: = Actual response time / Maximum allowable time.

[0064] Resource scheduling efficiency: =Standard resource consumption / Actual resource consumption.

[0065] Coordination and cooperation level: = The average of the number of collaborative actions by a single agent / the total number of actions.

[0066] Adaptability and innovation capability: =Benefit from using unconventional strategies / Maximum theoretical benefit.

[0067] The system diagnoses weaknesses by calculating the deviation in capabilities: ;in As a preset capability benchmark threshold, When the i-th capability is determined to be a weakness, δ is the system-defined judgment coefficient.

[0068] Step 4: Based on the shortcomings in capabilities, inject quantitative pressure into the simulation by adjusting the Red Team's strategy, environmental parameters, or mission objectives.

[0069] Based on the capability deficiency diagnosis results, the system initiates a targeted pressure injection mechanism (such as...). Figure 3 As shown), the formula for quantifying pressure intensity is: P represents the overall pressure intensity. The system adapts weights to address weaknesses; pressure is injected into the simulation scenario in three ways:

[0070] Red strategy optimization: Strengthen countermeasures related to weaknesses;

[0071] Environmental parameter disturbances: Adjust disaster response rate, communication strength, and resource availability;

[0072] Task objective changes: Dynamically add unexpected sub-tasks to increase the challenges of response and resource reallocation.

[0073] Step 5: Based on the trajectory data accumulated through inference, use multi-agent reinforcement learning to train the policy models of each party offline, so as to achieve co-evolution of the policy layer.

[0074] After one or more rounds of simulation, it enters the offline evolution phase:

[0075] (1) Experience pool construction: Store the inferred trajectory data into the hierarchical experience playback pool;

[0076] (2) Reward function design:

[0077] Blue team reward function: ;

[0078] Red team reward function: ;

[0079] Where R_task is the task completion rate, R_time is the time efficiency, R_coop is the collaboration benefit, R_cost is the resource consumption, D_blue is the blue team's weakness exposure, and U_env is the environmental uncertainty gain.

[0080] (3) Co-evolutionary training: The MADDPG and MAPPO multi-agent reinforcement learning algorithms are used to construct a joint loss function:

[0081] ;

[0082] The red and blue strategy networks are trained centrally and executed in a distributed manner.

[0083] (4) Model update: Update the policy parameters according to the gradient descent rule:

[0084] ;

[0085] d;

[0086] η is the learning rate, and θ is the policy network parameter.

[0087] Step 6: Repeat steps 2 to 5 to form a closed-loop iteration of deduction-evaluation-pressure-evolution. By comparing the digital profiles of capabilities from multiple rounds with the results of benchmark tests at fixed difficulty, the effectiveness of capability evolution can be quantitatively output.

[0088] A task extrapolation and capability evolution system of the present invention, such as Figure 4 As shown, it specifically includes:

[0089] Context Generation and Management Module: Used to build and maintain generative immersive contexts.

[0090] Multi-agent engine module: Used to manage the operation, decision-making, and interaction of the red and blue agents.

[0091] Dual-layer evolution control module: used to achieve unified scheduling of online evolution at the tactical layer and offline evolution at the strategy layer.

[0092] Capability Assessment and Stress Injection Module: Used to quantitatively build a digital profile of capabilities, diagnose capability gaps, and perform targeted stress injection.

[0093] Data storage and model training module: used to store the inferred trajectory data and perform multi-agent reinforcement learning training.

[0094] Furthermore, the capability assessment and stress injection module includes a weakness diagnosis unit and a stress strategy generator. The weakness diagnosis unit identifies weaknesses based on indicator deviation, and the stress strategy generator maps weaknesses into executable environmental adjustment instructions and red team strategy adjustment weights.

[0095] Example:

[0096] Urban emergency response task simulation

[0097] 1. Scenario initialization: The system generates a virtual city environment including commercial areas, underground passages, and dense crowds. The blue team is a special police intelligent agent with the initial strategy of standard handling procedures; the red team is a dangerous agent with the initial strategy of holding the hijacking point.

[0098] 2. Confrontational simulation and capability diagnosis: In the first round of simulation, the Red team maneuvered through underground passages and carried out a second hijacking. The Blue team experienced coordination chaos due to communication delays and intelligence lag. The system diagnosed that dynamic intelligence fusion and cross-group coordination capabilities were weaknesses.

[0099] 3. Targeted pressure injection: The system increases the weight of the Red team's maneuver dispersion and information interference strategies, strengthens communication environment disturbances, and adaptively increases the difficulty of the simulation.

[0100] 4. Offline strategy evolution: The simulation data is stored in the experience pool, and the system uses reinforcement learning to train the blue team's local autonomous decision-making and non-communication collaborative strategies, while simultaneously optimizing the red team's concealment maneuvering and deception strategies.

[0101] 5. Closed-loop iterative evolution: After multiple rounds of iteration, the Blue Team's collaborative capability index has increased from 0.32 to 0.86, the baseline task success rate has increased from 40% to 85%, and the Red Team's strategy diversity and confrontation intensity have been improved simultaneously.

[0102] This invention achieves dynamic simulation, quantitative evaluation, and autonomous capability evolution of high-risk mission simulation through multi-agent adversarial feedback and a two-layer evolution mechanism, providing an engineeringable intelligent training technology solution for fields such as emergency rescue, handling of emergencies, and military confrontation.

Claims

1. A method for task deduction and capability evolution based on multi-agent adversarial feedback, characterized in that, Includes the following steps: Step 1: Construct a generative immersive scenario and initialize a multi-agent system that includes adversarial red and blue teams; Step 2: Perform multi-agent adversarial simulation in the scenario, realize online evolution of the tactical layer during the process, and record complete simulation trajectory data; Step 3: Based on the simulated trajectory data, construct a multi-dimensional digital profile of the participants' capabilities in real time, and diagnose capability shortcomings through indicator deviation. Step 4: Based on the shortcomings in capabilities, inject quantitative pressure into the simulation by adjusting the Red Team's strategy, environmental parameters, or mission objectives; Step 5: Based on the trajectory data accumulated through inference, use multi-agent reinforcement learning to train the policy models of each party offline, so as to achieve co-evolution of the policy layer; Step 6: Repeat steps 2 to 5 to form a closed-loop iteration of deduction-evaluation-pressure-evolution, driving the continuous improvement of the trainee's capabilities.

2. The method for task deduction and capability evolution based on multi-agent adversarial feedback as described in claim 1, characterized in that, The deduction process in step 2 adopts a two-layer evolution mode: Tactical layer online evolution: Within a single simulation round, the agent makes decisions through a policy network based on real-time environmental conditions and opponent behavior data; The built-in meta-learning module quickly fine-tunes tactical choices based on short-term gains, enabling real-time adaptation within rounds. Mutual feedback observation: The system collects and records the environmental state sequence, the action sequence of each party, key events, and task completion indicators in full, forming complete inference trajectory data.

3. The method for task deduction and capability evolution based on multi-agent adversarial feedback as described in claim 1, characterized in that, Step 3, based on the projected trajectory data, involves the system calculating multi-dimensional capability indicators in real time to form a normalized digital capability profile, formally represented as follows: These correspond to five quantitative indicators: situational awareness, decision-making response, resource scheduling, collaborative cooperation, and adaptive innovation, with a value range of [0,1]. The system diagnoses weaknesses by calculating the deviation in capabilities: ;in As a preset capability benchmark threshold, When the i-th capability is determined to be a weakness, δ is the system-defined judgment coefficient.

4. The method for task deduction and capability evolution based on multi-agent adversarial feedback as described in claim 1, characterized in that, Step 4, based on the capability deficiency diagnosis results, involves the system initiating a targeted pressure injection mechanism. The pressure intensity quantification formula is as follows: P represents the overall pressure intensity. The weights are adaptive to the weakest link; pressure directly affects the red team's strategy weights, environmental disturbance coefficients, and task difficulty coefficients.

5. The method for task deduction and capability evolution based on multi-agent adversarial feedback as described in claim 1, characterized in that, After one or more rounds of deduction in step 5, the offline evolution phase begins. (1) Experience pool construction: Store the inferred trajectory data into the hierarchical experience playback pool; (2) Reward function design: Blue team reward function: ; Red team reward function: ; Where R_task is the task completion rate, R_time is the time efficiency, R_coop is the collaboration benefit, R_cost is the resource consumption, D_blue is the blue team's weakness exposure, and U_env is the environmental uncertainty gain. (3) Co-evolutionary training: The MADDPG and MAPPO multi-agent reinforcement learning algorithms are used to construct a joint loss function: ； The red and blue strategy networks are trained centrally and executed in a distributed manner. (4) Model update: Update the policy parameters according to the gradient descent rule: ； ； η is the learning rate, and θ is the policy network parameter.

6. A task deduction and capability evolution system for implementing the method of any one of claims 1-5, characterized in that, include: Context Generation and Management Module: Used to build and maintain generative immersive contexts; Multi-agent engine module: Used to manage the operation, decision-making, and interaction of the red and blue agents; Dual-layer evolution control module: used to achieve unified scheduling of online evolution at the tactical layer and offline evolution at the strategy layer; Capability Assessment and Stress Injection Module: Used to quantitatively construct a digital profile of capabilities, diagnose capability gaps, and perform targeted stress injection; Data storage and model training module: used to store the inferred trajectory data and perform multi-agent reinforcement learning training.

7. The task deduction and capability evolution system according to claim 6, characterized in that, The capability assessment and stress injection module includes a shortcoming diagnosis unit and a stress strategy generator. The shortcoming diagnosis unit identifies shortcomings based on the deviation of indicators, and the stress strategy generator maps the shortcomings into executable environmental adjustment instructions and red team strategy adjustment weights.

Images