Competitive multi-agent reinforcement learning decision optimization method

Abstract

The invention discloses a competitive multi-agent reinforcement learning decision optimization method and system, and the method improves the response speed of agents in a dense confrontation scene through optimizing an information interaction mechanism, screening key state information of opponents, shielding redundant data and reducing decision delay. In addition, a competition-oriented reward function is constructed, individual confrontation contributions are accurately quantified, the problem that global rewards are difficult to distribute to individuals is solved, the training oscillation amplitude is reduced, and the intelligent agent is prevented from falling into local optimum. Moreover, a confrontation difficulty adaptive training method is designed, the opponent strength is dynamically adjusted according to the competition level of the intelligent agent, and the adaptive ability of the model in competition scenes with different intensities is improved. And finally, based on competition scene characteristics, transforming a CTDE normal form, a value decomposition logic and an information interaction module, realizing unification of a local confrontation decision and a global competition target, and improving exclusive suitability of the technical scheme.

Description

Technical Field

[0001] This invention relates to the technical field of artificial intelligence and multi-agent systems, and in particular to a competitive multi-agent reinforcement learning decision optimization method and system, which is applicable to multi-agent adversarial application scenarios such as intelligent robot confrontation, drone swarm competition, virtual scene confrontation, and distributed resource competition. Background Technology

[0002] In recent years, Multi-Agent Reinforcement Learning (MARL), as a core technology for achieving complex autonomous decision-making, has made significant progress in fields such as cooperative control, intelligent transportation, and military simulation. Mainstream methods can be divided into two main categories: those based on value function decomposition (such as QMIX) and those based on policy gradients (such as MAPPO), and both commonly employ the Centralized Training with Decentralized Execution (CTDE) paradigm.

[0003] However, existing MARL methods are primarily designed for cooperative tasks, and their core mechanisms—such as full information exchange, collaborative value decomposition, and global reward sharing—are ill-suited to the essential needs of competitive scenarios. In competitive environments such as resource competition, objective suppression, and adversarial games, agents need to quickly identify high-threat opponents, quantify individual adversarial contributions, and dynamically adapt to changes in opponent strategies under local observation constraints.

[0004] Current technology has the following key shortcomings:

[0005] 1. Redundant information exchange: The use of the full communication mode in the cooperation scenario does not distinguish between key threat information and redundant status, resulting in high decision-making delay (>20%) and easy to miss the opportunity to counterattack;

[0006] 2. Unreasonable distribution of competitive benefits: Rewards depend solely on the final outcome, failing to quantify intermediate adversarial behaviors such as "suppression actions" and "seizing key points," leading to training oscillations (amplitude > 30%) and getting stuck in local optima;

[0007] 3. Poor training generalization: With a fixed opponent strategy strength, the model's win rate fluctuates by more than 40% when facing opponents of different skill levels, making it difficult to adapt to dynamic adversarial environments;

[0008] 4. Incompatible technical architecture: The collaborative logic of cooperative algorithms is in conflict with the exclusivity and adversarial nature of competitive scenarios, and there is a lack of dedicated optimization mechanisms.

[0009] Although existing solutions have attempted to directly migrate QMIX or MAPPO to competitive scenarios, or adopt independent DQN, MADDPG and other basic frameworks, none of them have fundamentally solved the above problems, limiting their practicality and stability in highly dynamic and highly adversarial tasks. Summary of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of the prior art and provide a competitive multi-agent reinforcement learning decision optimization method, thereby improving the decision-making efficiency, training stability and cross-scenario adaptability of agents in adversarial environments.

[0011] To achieve the above objectives, the technical solution provided by this invention is as follows:

[0012] A competitive multi-agent reinforcement learning decision optimization method includes:

[0013] Construct a competitive multi-agent scenario and set the parameters, environmental parameters, and basic training parameters for our agent and the opponent's agent.

[0014] The competitive attention dynamic communication mechanism is implemented, in which the agent calculates the threat level of the opponent agent based on local observations, filters the key information of the opponent agents with the highest threat level through the attention weight matrix, and inputs the key information obtained into the competitive decision model, which then determines the action of the agent.

[0015] Based on the execution results of our intelligent agent's actions, we quantify the adversarial contribution and resource gain, and combine the adversarial contribution and resource gain into a total reward through dynamic weighting, which is then fed back to the competitive decision-making model.

[0016] The competitive decision-making model is trained with adaptive adversarial difficulty to optimize it and obtain the final competitive decision-making model for agent decision-making.

[0017] Furthermore, the execution process of the competitive attention dynamic communication mechanism is as follows:

[0018] Calculate the threat level of the adversary agent:

[0019] Threat level = Resource possession rate × 0.6 + Aggressiveness of actions × 0.4;

[0020] Wherein, resource occupancy rate = current resource holdings of the opposing intelligent agent / total resource holdings in the scenario;

[0021] Aggressiveness of actions is quantified as "Attack = 1.0, Seize = 0.8, Defense = 0.3, Movement = 0.1", reflecting the intensity of the opponent's actions.

[0022] Calculate the attention weight matrix based on threat level:

[0023] (Threat level × 0.8);

[0024] in, The attention weight matrix has a dimension equal to the number of friendly agents multiplied by the number of adversary agents. The function ensures that the sum of the weights is 1, and 0.8 is an adjustment coefficient to increase the weight ratio of high-threat opponents;

[0025] The system uses an attention weight matrix to filter key information from adversary agents with high threat levels for interaction, while filtering out redundant data with a threat level <0.4.

[0026] Furthermore, the process of obtaining the total reward includes:

[0027] Quantifying the contribution of adversarial efforts:

[0028] Using counterfactual reasoning, fixing the actions of other agents, and setting our agent's actions as "default no-attack actions," we calculate the change in the opponent's agent's gains and combine this with KL divergence to quantify policy differences:

[0029] Contribution to resistance = ;

[0030] in, = Opponent's gain after our agent performs an action - Opponent's gain when our agent does not perform an action; ΔR_opponent < 0 indicates effective suppression, take the absolute value; Let KL divergence be the KL divergence. The distribution of opponent policies after the agent's actions are executed. The opponent's strategy distribution when no action is performed is given by the following formula:

[0031] ;

[0032] Quantifying resource returns:

[0033] Resource gains = Resource acquisition amount × 0.7 + Key area control duration × 0.3;

[0034] The total reward is calculated by combining the contribution of the opponent and the resource gain through dynamic weighting. :

[0035] ;

[0036] in, , For dynamic weights, + =1, updated in real time through environmental feedback.

[0037] Furthermore, the competitive decision-making model undergoes adaptive training to increase adversarial difficulty, including:

[0038] Win rate statistics: Calculate the win rate of our agent based on the results of the most recent N games;

[0039] Dynamically adjust the strength of the opponent's AI agent based on the win rate:

[0040] If the win rate is greater than 70%, the attack priority of the opponent's AI will be increased by 0.2, increasing the probability of the opponent's AI seizing resources.

[0041] If the win rate is less than 30%, the opponent's defense threshold will be reduced by 0.15 to reduce the opponent's invalid actions.

[0042] If the win rate is 30% ≤ win rate ≤ 70%, then the strength of the opponent's agent remains unchanged;

[0043] Parameter iterative update:

[0044] The Dirichlet distribution parameters are updated every 100 iterations, using the following formula:

[0045] ;

[0046] in, The attenuation coefficient is... This represents the current number of training games. This represents the total number of training games.

[0047] Furthermore, the parameters that our intelligent agent and the adversary intelligent agent need to set include:

[0048] Observe spatial parameters, including the state of our intelligent agent, the state of the opponent's intelligent agent, and the state of the environment;

[0049] Among them, the status of our intelligent agent includes our position, our resource holdings, and remaining energy; the status of the opponent's intelligent agent includes the opponent's position, the opponent's resource holdings, and the opponent's action intentions; the status of the environment includes the remaining amount of resources and the control rate of key areas.

[0050] Action space parameters, including discrete actions such as movement, capture, attack, and defense;

[0051] Capability parameters include movement speed, attack priority, defense threshold, and communication filtering threshold;

[0052] Initial state parameters include initial position, initial resource holdings, initial energy, and opponent's initial strategy;

[0053] Environmental parameters include environment size, number of resource points, initial remaining resource points, control rate of key areas, and scene termination conditions.

[0054] The scenarios that terminate include termination of the confrontation, termination of resources, and termination of the time limit;

[0055] Basic training parameters include total training time steps, network configuration parameters, experience replay parameters, and target network update parameters;

[0056] The network configuration parameters include the learning rate of the activist network, the learning rate of the critic network, the activation function, and the discount factor.

[0057] The parameters for experience replay include the experience pool size and the batch sampling size.

[0058] Furthermore, to achieve the above objectives, the present invention also provides a competitive multi-agent reinforcement learning decision optimization system for implementing the above-described competitive multi-agent reinforcement learning decision optimization method, comprising:

[0059] The competitive multi-agent scenario construction module is used to set the parameters, environmental parameters, and basic training parameters of our agent and the opponent's agent;

[0060] A competitive attention dynamic communication module is used to intelligently filter key information of adversary agents in competitive scenarios;

[0061] A competitive decision-making model is used to determine the actions of our intelligent agent;

[0062] The competition-oriented causal composite reward module is used to obtain the total reward based on the execution results of our agent's actions;

[0063] The adversarial difficulty adaptive training module is used to train the competitive decision model and optimize it to obtain the final competitive decision model used for agent decision-making.

[0064] Furthermore, the competitive attention dynamic communication module includes an adversary agent threat calculation unit, an attention weight matrix calculation unit, and a key information filtering unit.

[0065] Furthermore, the competition-oriented causal composite reward module includes a confrontation contribution quantification unit, a resource benefit quantification unit, and a total reward composite unit.

[0066] Compared with existing technologies, the principles and advantages of this technical solution are as follows:

[0067] 1. Improve competitive decision-making efficiency: By optimizing the information exchange mechanism, filtering key opponent status information, shielding redundant data, reducing decision delay, and improving the response speed of intelligent agents in intensive confrontation scenarios.

[0068] 2. Optimize the distribution of competitive benefits: Construct a competition-oriented reward function to accurately quantify individual adversarial contributions, solve the problem of "difficulty in allocating global rewards to individuals", reduce training oscillations, and prevent agents from getting trapped in local optima.

[0069] 3. Enhance training generalization: Design an adaptive training method for adversarial difficulty, dynamically adjust the strength of the opponent according to the competitive level of the agent, and improve the model's adaptability in competitive scenarios with different intensities.

[0070] 4. Enhance technology adaptability: Based on the characteristics of competitive scenarios, modify the CTDE paradigm (Centralized Training and Decentralized Execution, which refers to the competitive adaptation of the traditional CTDE architecture to the characteristics of competitive scenarios, transforming it from "centralized for collaboration" to "centralized for confrontation"), value decomposition logic, and information interaction module to achieve the unification of "local confrontation decision-making" and "global competitive goals", thereby enhancing the specific adaptability of the technical solution. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1 This invention provides a competitive multi-agent reinforcement learning decision optimization method.

[0073] Figure 2 This is a flowchart illustrating the principle of the competitive attention dynamic communication mechanism in a competitive multi-agent reinforcement learning decision optimization method according to an embodiment of the present invention.

[0074] Figure 3 This is a connection block diagram of a competitive multi-agent reinforcement learning decision optimization system according to an embodiment of the present invention;

[0075] Figure 4 A connection diagram of the internal units of the competitive attention dynamic communication module;

[0076] Figure 5 A connection diagram of the various units within the competition-oriented causal compound reward module. Detailed Implementation

[0077] The present invention will be further described below with reference to specific embodiments:

[0078] like Figure 1 As shown in this embodiment, a competitive multi-agent reinforcement learning decision optimization method includes the following steps:

[0079] S1. Construct a competitive multi-agent scenario and set the parameters, environmental parameters, and basic training parameters for our agent and the opponent's agent.

[0080] In this step, the parameters for setting our own intelligent agents (6 agents) and the opponent's intelligent agents (3 agents) include:

[0081] Observe spatial parameters, including the state of our intelligent agent, the state of the opponent's intelligent agent, and the state of the environment;

[0082] Among them, the status of our intelligent agent includes our position, our resource holdings, and remaining energy; the status of the opponent's intelligent agent includes the opponent's position, the opponent's resource holdings, and the opponent's action intentions; the status of the environment includes the remaining amount of resources and the control rate of key areas.

[0083] Action space parameters, including discrete actions such as movement, capture, attack, and defense;

[0084] Capability parameters include movement speed, attack priority, defense threshold, and communication filtering threshold (which can be dynamically adjusted according to the difficulty of the scenario, such as our agent's movement speed of 2.7 units / time step and the opponent's initial movement speed of 3.0 units / time step).

[0085] Initial state parameters include initial position, initial resource holdings (0), initial energy, and the opponent agent's initial policy (set as the basic patrol policy).

[0086] Environmental parameters include environment size (1500×1500 units), number of resource points, initial remaining resource points (100 units), key area control rate (quantified by agent occupation time), and scene termination conditions.

[0087] The scenarios can be terminated under the following conditions: confrontation termination (e.g., one side's agents are completely suppressed (survival count ≤ 1)), resource termination (e.g., one side completely occupies key resource points (control rate ≥ 80%)), and time limit termination (e.g., a single game reaches the maximum number of steps (900 steps)).

[0088] Basic training parameters include total training time steps (3 Step 1), network configuration parameters, experience replay parameters, target network update parameters;

[0089] Among them, the network configuration parameters include the ActionNetwork Learning Rate (8) ), critic network learning rate (8) ), activation function ( ), discount factor (0.9997);

[0090] The parameters for experience replay include the experience pool size (5000) and the batch sampling size (32).

[0091] S2, Execute a competitive attention-based dynamic communication mechanism, such as Figure 2 As shown, our agent calculates the threat level of the opponent's agent based on local observations, filters key information of the opponent's agent with the highest threat level through the attention weight matrix, and inputs the key information obtained from the filter into the competitive decision model, which then determines the action of our agent.

[0092] The specific calculation process for this step is as follows:

[0093] Calculate the threat level of the adversary agent:

[0094] Threat level = Resource possession rate × 0.6 + Aggressiveness of actions × 0.4;

[0095] Wherein, resource occupancy rate = current resource holdings of the opposing intelligent agent / total resource holdings in the scenario;

[0096] Aggressiveness of actions is quantified as "Attack = 1.0, Seize = 0.8, Defense = 0.3, Movement = 0.1", reflecting the intensity of the opponent's actions.

[0097] Calculate the attention weight matrix based on threat level:

[0098] (Threat level × 0.8);

[0099] in, The attention weight matrix has a dimension equal to the number of friendly agents multiplied by the number of adversary agents. The function ensures that the sum of the weights is 1, and 0.8 is an adjustment coefficient to increase the weight ratio of high-threat opponents;

[0100] The system uses an attention weight matrix to filter key information from adversary agents with high threat levels for interaction, while filtering out redundant data with a threat level <0.4.

[0101] S3. Based on the execution results of our intelligent agent's actions, quantify the adversarial contribution and resource gain, and combine the adversarial contribution and resource gain into a total reward through dynamic weighting, and feed it back to the competitive decision-making model.

[0102] The specific calculation process for this step is as follows:

[0103] Quantifying the contribution of adversarial efforts:

[0104] Using counterfactual reasoning, fixing the actions of other agents, and setting our agent's actions as "default no-attack actions," we calculate the change in the opponent's agent's gains and combine this with KL divergence to quantify policy differences:

[0105] Contribution to resistance = ;

[0106] in, = Opponent's gain after our agent performs an action - Opponent's gain when our agent does not perform an action; ΔR_opponent < 0 indicates effective suppression, take the absolute value; Let KL divergence be the KL divergence. The distribution of opponent policies after the agent's actions are executed. The opponent's strategy distribution when no action is performed is given by the following formula:

[0107] ;

[0108] Quantifying resource returns:

[0109] Resource gains = Resource acquisition amount × 0.7 + Key area control duration × 0.3;

[0110] The total reward is calculated by combining the contribution of the opponent and the resource gain through dynamic weighting. :

[0111] ;

[0112] in, , For dynamic weights, + =1, updated in real time through environmental feedback.

[0113] S4. Perform adaptive training on the competitive decision-making model to optimize it and obtain the final competitive decision-making model used for agent decision-making.

[0114] In this step, the competitive decision-making model undergoes adaptive training to improve adversarial difficulty, including:

[0115] Win rate statistics: Calculate the win rate of our agent based on the results of the most recent N games;

[0116] Dynamically adjust the strength of the opponent's AI agent based on the win rate:

[0117] If the win rate is greater than 70%, the attack priority of the opponent's AI will be increased by 0.2, increasing the probability of the opponent's AI seizing resources.

[0118] If the win rate is less than 30%, the opponent's defense threshold will be reduced by 0.15 to reduce the opponent's invalid actions.

[0119] If the win rate is 30% ≤ win rate ≤ 70%, then the strength of the opponent's agent remains unchanged;

[0120] Parameter iterative update:

[0121] The Dirichlet distribution parameters are updated every 100 iterations, using the following formula:

[0122] ;

[0123] in, The attenuation coefficient is... This represents the current number of training games. This represents the total number of training games.

[0124] In addition, this embodiment also includes a competitive multi-agent reinforcement learning decision optimization system for implementing the above-described competitive multi-agent reinforcement learning decision optimization method, such as... Figure 3 As shown, it includes:

[0125] The competitive multi-agent scenario construction module is used to set the parameters, environmental parameters, and basic training parameters of our agent and the opponent's agent;

[0126] A competitive attention dynamic communication module is used to intelligently filter key information of adversary agents in competitive scenarios;

[0127] A competitive decision-making model is used to determine the actions of our intelligent agent;

[0128] The competition-oriented causal composite reward module is used to obtain the total reward based on the execution results of our agent's actions;

[0129] The adversarial difficulty adaptive training module is used to train the competitive decision model and optimize it to obtain the final competitive decision model used for agent decision-making.

[0130] Specifically, such as Figure 4 As shown, the competitive attention dynamic communication module includes an adversary agent threat calculation unit, an attention weight matrix calculation unit, and a key information filtering unit. Figure 5 As shown, the competition-oriented causal composite reward module includes a quantification unit for competitive contribution, a quantification unit for resource gains, and a composite unit for total rewards.

[0131] This invention employs a competitive attention-based dynamic communication mechanism to shield redundant information, reducing decision latency to less than 0.1 seconds, a reduction of over 40% compared to traditional methods (0.17-0.21 seconds), significantly improving response speed in intensive adversarial scenarios. Furthermore, the competition-oriented causal composite reward function accurately quantifies individual adversarial contributions, controlling training oscillations to within 15%, a reduction of over 53% compared to existing methods (32%-38%), preventing the agent from getting trapped in local optima. Moreover, the adversarial difficulty adaptive training method ensures that the model's win rate fluctuation does not exceed 20% in high, medium, and low-intensity competitive scenarios, with a generalization rate ≥80%, an improvement of over 45% compared to traditional fixed-intensity training methods (generalization rate 48%-55%). The information interaction, reward calculation, and training mechanisms are specifically designed for adversarial scenarios, avoiding the "incompatibility" of cooperative algorithm architectures, resulting in significantly better targeting and effectiveness in competitive decision-making than existing adaptation schemes. It is adaptable to various scenarios such as resource competition, adversarial hunting, and distributed competitive decision-making. It can be directly integrated into products such as intelligent robot combat and drone swarm competition, making it highly practical. Moreover, the parameters can be flexibly adjusted according to scenario requirements. Finally, the incremental training logic avoids full retraining and reduces data transmission and computation by combining key information filtering, reducing computational resource consumption by more than 30% compared to traditional methods.

[0132] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any changes made in accordance with the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A competitive multi-agent reinforcement learning decision optimization method, characterized in that, include: Construct a competitive multi-agent scenario and set the parameters, environmental parameters, and basic training parameters for our agent and the opponent's agent. The competitive attention dynamic communication mechanism is implemented, in which the agent calculates the threat level of the opponent agent based on local observations, filters the key information of the opponent agents with the highest threat level through the attention weight matrix, and inputs the key information obtained into the competitive decision model, which then determines the action of the agent. Based on the execution results of our intelligent agent's actions, we quantify the adversarial contribution and resource gain, and combine the adversarial contribution and resource gain into a total reward through dynamic weighting, which is then fed back to the competitive decision-making model. The competitive decision-making model is trained with adaptive adversarial difficulty to optimize it and obtain the final competitive decision-making model for agent decision-making.

2. The competitive multi-agent reinforcement learning decision optimization method according to claim 1, characterized in that, The execution process of the competitive attention dynamic communication mechanism is as follows: Calculate the threat level of the adversary agent: Threat level = Resource possession rate × 0.6 + Aggressiveness of actions × 0.4; Wherein, resource occupancy rate = current resource holdings of the opposing intelligent agent / total resource holdings in the scenario; Aggressiveness of actions is quantified as "Attack = 1.0, Seize = 0.8, Defense = 0.3, Movement = 0.1", reflecting the intensity of the opponent's actions. Calculate the attention weight matrix based on threat level: (Threat level × 0.8); in, The attention weight matrix has a dimension equal to the number of friendly agents multiplied by the number of adversary agents. The function ensures that the sum of the weights is 1, and 0.8 is an adjustment coefficient to increase the weight ratio of high-threat opponents; The system uses an attention weight matrix to filter key information from adversary agents with high threat levels for interaction, while filtering out redundant data with a threat level <0.

4.

3. The competitive multi-agent reinforcement learning decision optimization method according to claim 1, characterized in that, The process of obtaining the total reward includes: Quantifying the contribution of adversarial efforts: Using counterfactual reasoning, fixing the actions of other agents, and setting our agent's actions as "default no-attack actions," we calculate the change in the opponent's agent's gains and combine this with KL divergence to quantify policy differences: Contribution to resistance = ; in, = Opponent's gain after our agent performs an action - Opponent's gain when our agent does not perform an action; ΔR_opponent < 0 indicates effective suppression, take the absolute value; Let KL divergence be the KL divergence. The distribution of opponent policies after the agent's actions are executed. The opponent's strategy distribution when no action is performed is given by the following formula: ； Quantifying resource returns: Resource gains = Resource acquisition amount × 0.7 + Key area control duration × 0.3; The total reward is calculated by combining the contribution of the opponent and the resource gain through dynamic weighting. : ； in, , For dynamic weights, + =1, updated in real time through environmental feedback.

4. The competitive multi-agent reinforcement learning decision optimization method according to claim 1, characterized in that, Adaptive training of the competitive decision-making model to increase adversarial difficulty includes: Win rate statistics: Calculate the win rate of our agent based on the results of the most recent N games; Dynamically adjust the strength of the opponent's AI agent based on the win rate: If the win rate is greater than 70%, the attack priority of the opponent's AI will be increased by 0.2, increasing the probability of the opponent's AI seizing resources. If the win rate is less than 30%, the opponent's defense threshold will be reduced by 0.15 to reduce the opponent's invalid actions. If the win rate is 30% ≤ win rate ≤ 70%, then the strength of the opponent's agent remains unchanged; Parameter iterative update: The Dirichlet distribution parameters are updated every 100 iterations, using the following formula: ； in, The attenuation coefficient is... This represents the current number of training games. This represents the total number of training games.

5. A competitive multi-agent reinforcement learning decision optimization method according to any one of claims 1-4, characterized in that, The parameters that need to be set for our intelligent agent and the adversary intelligent agent include: Observe spatial parameters, including the state of our intelligent agent, the state of the opponent's intelligent agent, and the state of the environment; Among them, the status of our intelligent agent includes our position, our resource holdings, and remaining energy; the status of the opponent's intelligent agent includes the opponent's position, the opponent's resource holdings, and the opponent's action intentions; the status of the environment includes the remaining amount of resources and the control rate of key areas. Action space parameters, including discrete actions such as movement, capture, attack, and defense; Capability parameters include movement speed, attack priority, defense threshold, and communication filtering threshold; Initial state parameters include initial position, initial resource holdings, initial energy, and the opponent agent's initial strategy; Environmental parameters include environment size, number of resource points, initial remaining resource points, control rate of key areas, and scene termination conditions. The scenarios that terminate include termination of the confrontation, termination of resources, and termination of the time limit; Basic training parameters include total training time steps, network configuration parameters, experience replay parameters, and target network update parameters; The network configuration parameters include the learning rate of the activist network, the learning rate of the critic network, the activation function, and the discount factor. The parameters for experience replay include the experience pool size and the batch sampling size.

6. A competitive multi-agent reinforcement learning decision optimization system, used to implement the competitive multi-agent reinforcement learning decision optimization method of claim 1, characterized in that, include: The competitive multi-agent scenario construction module is used to set the parameters, environmental parameters, and basic training parameters of our agent and the opponent's agent; A competitive attention dynamic communication module is used to intelligently filter key information of adversary agents in competitive scenarios; A competitive decision-making model is used to determine the actions of our intelligent agent; The competition-oriented causal composite reward module is used to obtain the total reward based on the execution results of our agent's actions; The adversarial difficulty adaptive training module is used to train the competitive decision model and optimize it to obtain the final competitive decision model used for agent decision-making.

7. The competitive multi-agent reinforcement learning decision optimization system according to claim 6, characterized in that, The competitive attention dynamic communication module includes a threat calculation unit for adversary agents, an attention weight matrix calculation unit, and a key information filtering unit.

8. The competitive multi-agent reinforcement learning decision optimization system according to claim 6, characterized in that, The competition-oriented causal composite reward module includes a counter-contribution quantification unit, a resource benefit quantification unit, and a total reward composite unit.

Images