An online evolutionary algorithm and training platform for intelligent agents based on self-game

Abstract

The present invention discloses an online evolutionary algorithm and training platform for intelligent agents based on self-game, including: a task and environment module, an intelligent agent training module, a strategy evaluation module, a data management module and a user interaction module. The task and environment module dynamically defines training tasks and environmental parameters through a task generation unit and an environment simulation unit. The intelligent agent training module includes an initial strategy generation unit, a course learning unit and a self-game optimization unit to work together to strengthen the learning framework. The strategy evaluation module tests the advantages and disadvantages of intelligent agent strategies through performance evaluation, diversity analysis and comparative analysis. The data management module records and analyzes training data to manage the historical strategy library. The user interaction module provides task configuration, training monitoring and result visualization functions. The present invention improves the adaptability and robustness of intelligent agents by introducing a self-game mechanism and dynamic opponent generation, and is suitable for strategy optimization and training in complex collaboration and confrontation scenarios.

Description

Technical Field

[0001] The present invention relates to the field of intelligent agent intelligent game technology, and in particular to an intelligent agent online evolution algorithm and a training platform based on self-game. Background Art

[0002] With the rapid development of artificial intelligence technology, agent-based learning algorithms are being used more and more widely in complex task scenarios. In multi-agent environments, the ability to optimize collaborative and adversarial tasks has become an important criterion for measuring agent performance.

[0003] However, traditional agent training methods face the following challenges: traditional reinforcement learning algorithms are usually trained for fixed tasks, have limited performance in dynamic task environments or confrontation scenarios, and cannot flexibly adapt to dynamic changes in the environment; the complexity of multi-agent collaboration and confrontation. In a multi-agent environment, agents need to deal with collaborative and confrontation tasks at the same time, and existing algorithms are insufficient in optimizing the collaborative capabilities and confrontation strategies of agents at the same time; problems with training efficiency and strategy diversity. Traditional training platforms usually rely on static opponents or fixed reward mechanisms, which make it difficult to effectively motivate agents to explore diverse strategies, and easily lead to the simplification and overfitting of strategies; insufficient data management and evaluation capabilities. Complex multi-agent task generation and training processes require efficient data management and strategy evaluation mechanisms, and existing systems still have deficiencies in data storage, analysis, and result visualization.

[0004] In response to the above problems, the present invention provides an online evolutionary algorithm and training platform for intelligent agents based on self-game. This invention effectively improves the adaptability, collaboration efficiency and strategy diversity of intelligent agents in complex dynamic environments through task classification, reinforcement learning framework, self-game mechanism and dynamic adjustment of rewards, combined with a multi-module collaborative system architecture, and provides a complete intelligent agent training and optimization solution. Summary of the invention

[0005] In response to the above problems, the present invention provides an online evolutionary algorithm and training platform for intelligent agents based on self-game, so as to solve the problems that the existing technology lacks effective integration with multi-agent training and lacks a complete integrated training platform for task generation, strategy optimization, data management and user interaction.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: The present invention provides an online evolutionary training platform for intelligent agents based on self-game, comprising: Task and environment module, agent training module, strategy evaluation module, data management module and user interaction module; The task and environment module is used to define and simulate the training environment of the intelligent agent. The task and environment module includes a task generation unit and an environment simulation unit. The task generation unit is used to dynamically generate task objectives of different difficulties. The task objectives include collaborative tasks and confrontation tasks. The environment simulation unit supports multi-agent environment simulation to dynamically adjust environment parameters. The environment parameters include reward mechanisms and opponent strategies. The agent training module is the core of the system and is used to realize the functions of strategy optimization, curriculum learning, and self-game mechanism. The agent training module includes an initial strategy generation unit, a curriculum learning unit, and a self-game optimization unit. The initial strategy generation unit generates an initial strategy based on random initialization or rules. The curriculum learning unit trains the agent strategy in stages according to the complexity of the task. The self-game optimization unit generates a dynamic opponent strategy to confront the agent. The strategy evaluation module is used to test and evaluate the performance of the trained agent strategy; the strategy evaluation module includes a performance evaluation unit, a diversity analysis unit and a comparative analysis unit, the performance evaluation unit evaluates the strategy by the winning rate and resource efficiency indicators, the diversity analysis unit analyzes the innovation and diversity of the strategy, and the comparative analysis unit compares with the benchmark algorithm to verify the improvement effect; The data management module is responsible for storing, updating and analyzing training data and strategy information. The data management module includes a training data storage unit, a strategy pool management unit and a data analysis unit. The training data storage unit stores the state, action and reward data generated by the interaction between the agent and the environment. The strategy pool management unit stores the historical optimal strategy for subsequent training and evaluation. The data analysis unit performs statistics and optimization analysis on the stored data. The user interaction module provides users with an interface for system parameter configuration, task definition and result display. The user interaction module includes a task configuration unit, a training monitoring unit and a result visualization unit. The task configuration unit supports users to define task objectives and environmental parameters. The training monitoring unit displays the training progress and performance curve of the intelligent agent in real time. The result visualization unit displays the strategy evaluation results, which include a win rate curve and a strategy convergence diagram.

[0007] The task generation unit and the environment simulation unit in the task and environment module jointly complete the task modeling, which is the basis of the entire system and is used to define the mathematical model of the interaction between the intelligent agent and the environment; The collaborative tasks include resource allocation and goal achievement scenarios, the confrontational tasks include defeating opponents and area control scenarios, the task generation unit dynamically adjusts the task difficulty according to user needs or training progress, and the task difficulty is divided into simple tasks and complex tasks. The simple tasks are single-goal tasks for training basic behaviors of intelligent agents, and the complex tasks are multi-goal mixed tasks involving collaboration and confrontation; The environment simulation unit simulates the dynamic interaction between the agent and the environment based on task generation, the dynamic interaction includes multi-agent environment simulation and dynamic parameter adjustment, the multi-agent environment simulation supports the interactive behavior of multiple agents in cooperation and confrontation scenarios, the dynamic parameter adjustment includes a reward mechanism and an opponent strategy, the reward mechanism dynamically allocates reward values ​​according to the task type, and the opponent strategy optimizes the opponent behavior according to the self-game mechanism; The task generation unit is the input source of task modeling, which directly affects the definition of state space and action space. The environment simulation unit provides operation rules for task modeling and affects the definition of reward function and transition probability. The state space defines a set of environmental information perceived by the agent. The state space includes its own state, environmental state, and other agent states. The own state includes position, speed, and resource status. The environmental state includes target position, resource distribution, and obstacle information. The other agent states include collaborative tasks and confrontational tasks. The collaborative tasks represent the position information and behavior of teammates, and the confrontational tasks represent the strategy history and current state of the opponent. The state space is expressed as: ; in, Represents the local state of a single agent at a certain moment; The action space defines the operations that the agent can perform in each state. The operations include collaborative task actions and confrontation task actions. The collaborative task actions include resource allocation and path planning. The resource allocation is to allocate resources to different targets. The path planning is to select the optimal path to reach the target. The confrontation task includes attack behavior and avoidance behavior. The attack behavior is to select the target opponent to attack. The avoidance behavior is to avoid the threat of the opponent to optimize its own survivability. The action space is expressed as: ; in, Represents an action that the agent can choose; The reward function is distributed in real time by the environment simulation unit to evaluate the effect of the action performed by the agent in a certain state. The action effect includes the reward design of the collaborative task and the reward design of the confrontation task. The reward design of the collaborative task includes achieving the goal, failing to achieve the goal and failure due to resource conflict. The reward design of the confrontation task includes defeating the opponent, being defeated and continuous victory. The reward mechanism is dynamically adjusted based on the task completion and the performance of the agent. The transition probability is determined by the environment simulation unit to describe the change of the environment state after the agent performs an action. The state change includes collaborative tasks and confrontation tasks. The collaborative task is that the transition probability is affected by the teammates' behavior and environmental rules, and the confrontation task is that the transition probability is determined by the opponent's strategy and dynamic environmental factors. The transition probability is expressed as: ; in, is the current state, is the current action of the agent.

[0008] The initial strategy generation unit, the course learning unit and the self-game optimization unit in the agent training module work together to form a reinforcement learning framework. The reinforcement learning framework adopts a combination of centralized training and distributed execution, and optimizes the agent strategy based on the Actor-Critic architecture design. The Actor-Critic architecture, i.e., the Actor-Critic network, is a common framework in reinforcement learning and is widely used in strategy optimization and state evaluation tasks. The Actor is responsible for generating the agent's behavior strategy in the framework and maximizing the cumulative reward through the policy gradient method. The Critic is responsible for evaluating the strategy selected by the Actor and providing feedback through the estimated value function to help optimize the strategy. The initial strategy generation unit is responsible for providing the starting point of the initial behavior strategy reinforcement learning training. The course learning unit gradually increases the complexity of the agent strategy through phased task design. The phased task design divides the task into primary stage, intermediate stage and advanced stage. The primary stage is a single-target task, the intermediate stage is a multi-agent collaborative task, and the advanced stage is a complex confrontation task. The complex confrontation task gradually improves the performance of the agent based on the strategy optimization of the low-level task. The course learning unit provides a clear learning path for the agent based on the initial strategy generation; The self-game optimization unit improves the adaptability of the agent in complex confrontation tasks by introducing dynamic opponent strategies, generates opponent strategies based on the current agent strategies, introduces random disturbances to increase confrontation diversity, and dynamically adjusts the strength of the opponent strategies according to the agent winning rate, increasing the strength of the opponent strategies when the winning rate is high, and reducing the strength of the opponent strategies when the winning rate is low; The centralized training improves teamwork ability by optimizing agent strategies from a global perspective during the centralized training phase. The global Critic network is used to evaluate the overall state and strategy, and the Actor network of each agent generates individual strategies to adapt to different task objectives. The distributed execution, in which each agent makes independent decisions based on local states to adapt to real-time environmental requirements, and the Actor network generates actions based on local observations to reduce dependence on global information. In collaborative tasks, agents achieve real-time collaboration by sharing some strategy parameters; The Actor network generates an action probability distribution based on the input state to guide the agent's behavior decision-making and optimize the cumulative reward through the policy gradient method, which is expressed as: ; The Critic network evaluates the value of the current state to provide optimization direction for strategy update and minimize the valuation error, which is expressed as: ; The Actor network generates strategies and the Critic network evaluates the pros and cons of strategies, and the two are combined through centralized training and distributed execution to form a closed-loop optimization.

[0009] The self-game optimization unit in the agent training module implements the self-game mechanism. The self-game optimization unit is the core module of the self-game mechanism and is responsible for the dynamic opponent generation and strategy adjustment in the agent confrontation task. The specific work includes dynamic opponent strategy generation, dynamic opponent strength adjustment and multiple rounds of confrontation training. The dynamic opponent strategy generation generates a dynamic opponent based on the current agent strategy, introduces random perturbations and historical strategies, the opponent strength dynamic adjustment dynamically adjusts the complexity of the opponent strategy based on the agent's performance, and the multiple rounds of adversarial training optimize the agent strategy through multiple interactions and confrontations; The dynamic opponent strategy generation is the basic function of the self-game optimization unit to create a diverse confrontation environment for the agent, copy and generate the opponent strategy from the current agent strategy, introduce random perturbations into the opponent strategy to increase the uncertainty of the strategy and avoid confrontation solidification, and use the strategy pool management unit of the training platform to select historical strategies to provide multi-difficulty and diverse opponents; The dynamic adjustment of the opponent's strength is a further optimization after the dynamic opponent is generated, which is used to ensure that the difficulty of the opponent matches the ability of the agent. The opponent's strength is dynamically evaluated based on the winning rate of the agent. If the winning rate is higher than the preset threshold, the complexity of the opponent's strategy is increased. If the winning rate is lower than the preset threshold, the opponent's strength is reduced. The difficulty factor is used to dynamically control the core parameters of the opponent's strategy. The multi-round adversarial training is an important stage of the self-game mechanism. The evolution of the agent strategy is promoted through multiple interactions. In each round of training, the agent competes with the dynamically generated opponent, and the interaction data is stored in the training data storage unit. Based on the confrontation results, the agent optimizes its strategy using the Actor-Critic network in the reinforcement learning framework. The opponent strategy is updated after each round of training, and the complexity of the confrontation is gradually increased. The self-game mechanism improves the performance of the agent in resource allocation and path planning tasks by simulating the cooperative behavior of the opponent in the collaborative task. In the multi-agent environment, simulated teammates are generated for comparison with the target teammates, and the team collaboration is optimized through the reward mechanism. The agent learns how to divide the work and cooperate in the self-game to complete the resource allocation task; In the confrontation task, the self-game mechanism generates complex opponents through self-game, simulates the real competition environment to optimize the agent strategy, uses the Elo scoring mechanism to grade the opponent's strength, dynamically selects the opponent's strategy, and the agent improves its survivability in the dynamic environment by optimizing the attack behavior and avoidance strategy.

[0010] The course learning unit and the self-game optimization unit in the agent training module work together to complete the training process. The training process is divided into two main stages: course learning and self-game. Combined with dynamic task generation and phased optimization strategy, the agent evolution from simple tasks to complex tasks is gradually realized. The course learning unit is the basis of the training process. It gradually improves the strategy complexity and ability of the agent through task classification, and provides a three-stage training environment with task complexity from low to high, namely the primary stage, the intermediate stage and the advanced stage. The primary stage is to train the agent to complete simple tasks, the intermediate stage introduces multi-agent collaborative tasks, and the advanced stage further adds confrontational tasks. The agent needs to face dynamic opponents and optimize the collaboration and confrontation strategies; Based on course learning, the self-game optimization unit promotes the strategy evolution of the intelligent agent through dynamic opponent strategy generation and multiple rounds of adversarial training, generates dynamic opponent strategies in an adversarial training environment, prompts the intelligent agent to continuously optimize, uses a dynamic opponent strategy generation mechanism to provide the intelligent agent with confrontations of different difficulties, dynamically adjusts the opponent strength according to the winning rate in multiple rounds of confrontation, and maintains the effectiveness of the intelligent agent training; The training process is divided into an initialization phase, a course learning phase, a self-game phase and a comprehensive training phase by combining course learning and self-game mechanisms. The initialization phase provides the agent with a random or rule-generated initial strategy through an initial strategy generation unit. The course learning phase gradually increases the task complexity and optimizes the agent's collaborative ability through dynamic task classification. The self-game phase optimizes the agent's confrontation ability by generating dynamic opponent strategies. The comprehensive training phase combines collaborative and confrontation tasks to test the agent's global ability. Each stage in the training process relies on the support of the data management module and the user interaction module. The training data storage unit in the data management module records the interaction data between the agent and the environment in real time. The interaction data includes state, action and reward. The strategy pool management unit in the data management module stores the optimal strategy of each stage to provide reference for dynamic opponent generation. The data analysis unit in the data management module analyzes the training results to adjust the task parameters and reward mechanism. The task configuration unit in the user interaction module supports the user to adjust the training parameters in real time, and the training monitoring unit in the user interaction module provides visual feedback, which includes the task completion rate and the strategy optimization curve.

[0011] The environment simulation unit in the task and environment module and the data analysis unit in the data management module work together to realize the reward design, which is the core driver of the reinforcement learning framework and the self-game mechanism. The reward mechanism provides optimization goals for the learning of the intelligent agent, and ensures that the intelligent agent can effectively evolve at different task stages through the design of the task-level reward table and the dynamic adjustment mechanism; The environment simulation unit is responsible for real-time reward distribution during the training process, guiding the strategy optimization of the agent. In the collaborative task, the reward mechanism evaluates the overall performance of the team and distributes rewards based on resource utilization and path efficiency. In the confrontation task, the reward mechanism evaluates the attack efficiency and survivability of the agent and distributes targeted reward values. When the agent achieves the task goal, a positive reward is given, and a negative reward is given for the task failure or resource waste. An incremental reward mechanism is designed for the continuous successful behavior to encourage the continuous optimization of the strategy. The data analysis unit adjusts the reward mechanism in real time by analyzing the training results to ensure that the reward design can adapt to different task stages and the learning progress of the agent, adjusts the reward weight of the task goal according to the performance curve of the agent, dynamically balances the reward values ​​of different goals in collaborative and confrontational tasks, prevents the agent from being biased towards a certain strategy, uses the win rate curve and task completion rate to evaluate the reward distribution effect, and adjusts the values ​​in the reward table through data feedback; The reward table design is the core of the reward mechanism, which realizes the refined design of task classification and reward value, sets reward value according to the classification and complexity of the task, guides the agent to gradually optimize the strategy from low level to high level, designs corresponding reward and penalty rules for different behaviors, rewards collaboration efficiency in collaborative tasks, and rewards attack success rate in confrontation tasks; The reward mechanism focuses on guiding the teamwork and resource optimization capabilities of the agents in collaborative tasks, rewards the agents for completing collaborative behaviors, including resource sharing, target allocation, and path planning, punishes behaviors that cause resource conflicts or inefficiencies, allocates global rewards for team completion of tasks, and allocates individual rewards based on the contribution ratio of the agents, designs path efficiency rewards, and gives additional rewards to agents that complete path planning; In confrontation tasks, the reward mechanism emphasizes the confrontation ability and adaptability of the agent, rewards the agent's attacking behavior, avoidance behavior and victory results, punishes unnecessary losses or inefficient strategy choices, distributes winning and losing rewards based on the confrontation results, punishes being defeated, introduces innovative strategy rewards, and encourages the agent to explore diverse confrontation strategies.

[0012] The online evolutionary algorithm of intelligent agents based on self-game includes the following steps: Step S1, completing task modeling through the task generation unit and the environment simulation unit in the task and environment module; Step S2, strengthening the learning framework through the collaborative work of the initial strategy generation unit, the course learning unit and the self-game optimization unit in the agent training module; Step S3, realizing the self-game mechanism through the self-game optimization unit in the agent training module; Step S4, completing the training process through the collaborative work of the course learning unit and the self-game optimization unit in the agent training module; Step S5, reward design is implemented through the collaboration of the environment simulation unit in the task and environment module and the data analysis unit in the data management module.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention realizes dynamic task generation and real-time adjustment of environmental parameters through task and environment modules, so that the intelligent agent can adapt to complex and changeable collaboration and confrontation scenarios, overcoming the deficiency that traditional training methods are difficult to cope with dynamic environments.

[0014] The present invention introduces a self-game mechanism, which enables the intelligent agent to continuously optimize its strategy in multiple rounds of confrontation through dynamic opponent generation and opponent strength adjustment, thereby improving its adaptability and robustness. It can provide the intelligent agent with a diversified and dynamic training environment to avoid a single strategy.

[0015] The intelligent agent training module of the present invention gradually improves the performance of the intelligent agent in collaborative and confrontational tasks through course learning and phased training. Combined with the multi-agent service set and strategy optimization mechanism, the intelligent agent can efficiently complete complex tasks, breaking through the limitations of the existing technology in the collaborative and confrontation optimization capabilities in multi-agent tasks.

[0016] The present invention adopts a reinforcement learning framework, realizes global strategy optimization through centralized training, and improves real-time decision-making ability through distributed execution, effectively combines the advantages of the two training methods, and improves the decision-making efficiency of the intelligent agent in practical applications.

[0017] The present invention dynamically adjusts the reward mechanism through the data analysis module and combines it with the task-level reward design to ensure that the intelligent agent maintains the incentive effect during the training of different stages, effectively avoiding the strategy overfitting problem caused by the traditional fixed reward mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It is understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a schematic diagram of an online evolutionary algorithm and training platform for intelligent agents based on self-game; Figure 2 It is a schematic diagram of the reward table design of the present invention; Figure 3 It is a schematic diagram of the structure of the present invention; DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but is only for selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0021] Please refer to Figure 1-Figure 3 Schematic diagram of an online evolutionary training platform for an intelligent agent based on self-game provided by an embodiment of the present invention, including: Task and environment module, agent training module, strategy evaluation module, data management module and user interaction module; The task and environment module is used to define and simulate the training environment of the intelligent agent. The task and environment module includes a task generation unit and an environment simulation unit. The task generation unit is used to dynamically generate task objectives of different difficulties. The task objectives include collaborative tasks and confrontation tasks. The environment simulation unit supports multi-agent environment simulation to dynamically adjust environment parameters. The environment parameters include reward mechanisms and opponent strategies. The agent training module is the core of the system and is used to realize the functions of strategy optimization, curriculum learning, and self-game mechanism. The agent training module includes an initial strategy generation unit, a curriculum learning unit, and a self-game optimization unit. The initial strategy generation unit generates an initial strategy based on random initialization or rules. The curriculum learning unit trains the agent strategy in stages according to the complexity of the task. The self-game optimization unit generates a dynamic opponent strategy to confront the agent. The strategy evaluation module is used to test and evaluate the performance of the trained agent strategy; the strategy evaluation module includes a performance evaluation unit, a diversity analysis unit and a comparative analysis unit, the performance evaluation unit evaluates the strategy by the winning rate and resource efficiency indicators, the diversity analysis unit analyzes the innovation and diversity of the strategy, and the comparative analysis unit compares with the benchmark algorithm to verify the improvement effect; The data management module is responsible for storing, updating and analyzing training data and strategy information. The data management module includes a training data storage unit, a strategy pool management unit and a data analysis unit. The training data storage unit stores the state, action and reward data generated by the interaction between the agent and the environment. The strategy pool management unit stores the historical optimal strategy for subsequent training and evaluation. The data analysis unit performs statistics and optimization analysis on the stored data. The user interaction module provides users with an interface for system parameter configuration, task definition and result display. The user interaction module includes a task configuration unit, a training monitoring unit and a result visualization unit. The task configuration unit supports users to define task objectives and environmental parameters. The training monitoring unit displays the training progress and performance curve of the intelligent agent in real time. The result visualization unit displays the strategy evaluation results, which include a win rate curve and a strategy convergence diagram.

[0022] The task generation unit and the environment simulation unit in the task and environment module jointly complete the task modeling; the task modeling is the basis of the entire system and is used to define the mathematical model of the interaction between the intelligent agent and the environment; The collaborative tasks include resource allocation and goal achievement scenarios, the confrontational tasks include defeating opponents and area control scenarios, the task generation unit dynamically adjusts the task difficulty according to user needs or training progress, and the task difficulty is divided into simple tasks and complex tasks. The simple tasks are single-goal tasks for training basic behaviors of intelligent agents, and the complex tasks are multi-goal mixed tasks involving collaboration and confrontation; The environment simulation unit simulates the dynamic interaction between the agent and the environment based on task generation, the dynamic interaction includes multi-agent environment simulation and dynamic parameter adjustment, the multi-agent environment simulation supports the interactive behavior of multiple agents in cooperation and confrontation scenarios, the dynamic parameter adjustment includes a reward mechanism and an opponent strategy, the reward mechanism dynamically allocates reward values ​​according to the task type, and the opponent strategy optimizes the opponent behavior according to the self-game mechanism; The task generation unit is the input source of task modeling, which directly affects the definition of state space and action space. The environment simulation unit provides operation rules for task modeling and affects the definition of reward function and transition probability. The state space defines a set of environmental information perceived by the agent. The state space includes its own state, environmental state, and other agent states. The own state includes position, speed, and resource status. The environmental state includes target position, resource distribution, and obstacle information. The other agent states include collaborative tasks and confrontational tasks. The collaborative tasks represent the position information and behavior of teammates, and the confrontational tasks represent the strategy history and current state of the opponent. The state space is expressed as: ; in, Represents the local state of a single agent at a certain moment; The action space defines the operations that the agent can perform in each state. The operations include collaborative task actions and confrontation task actions. The collaborative task actions include resource allocation and path planning. The resource allocation is to allocate resources to different targets. The path planning is to select the optimal path to reach the target. The confrontation task includes attack behavior and avoidance behavior. The attack behavior is to select the target opponent to attack. The avoidance behavior is to avoid the threat of the opponent to optimize its own survivability. The action space is expressed as: ; in, Represents an action that the agent can choose; The reward function is distributed in real time by the environment simulation unit to evaluate the effect of the action performed by the agent in a certain state. The action effect includes the reward design of the collaborative task and the reward design of the confrontation task. The reward design of the collaborative task includes achieving the goal, failing to achieve the goal and failure due to resource conflict. The reward design of the confrontation task includes defeating the opponent, being defeated and continuous victory. The reward mechanism is dynamically adjusted based on the task completion and the performance of the agent. The transition probability is determined by the environment simulation unit to describe the change of the environment state after the agent performs an action. The state change includes collaborative tasks and confrontation tasks. The collaborative task is that the transition probability is affected by the teammates' behavior and environmental rules, and the confrontation task is that the transition probability is determined by the opponent's strategy and dynamic environmental factors. The transition probability is expressed as: ; in, is the current state, is the current action of the agent.

[0023] The initial strategy generation unit, the course learning unit and the self-game optimization unit in the agent training module work together to form a reinforcement learning framework; the reinforcement learning framework adopts a combination of centralized training and distributed execution, and optimizes the agent strategy based on the Actor-Critic architecture design. The Actor-Critic architecture, i.e., the Actor-Critic network, is a commonly used framework in reinforcement learning and is widely used in strategy optimization and state evaluation tasks. The Actor in the framework is responsible for generating the agent's behavior strategy and maximizing the cumulative reward through the policy gradient method. The Critic is responsible for evaluating the strategy selected by the Actor and providing feedback through the estimated value function to help optimize the strategy; The initial strategy generation unit is responsible for providing the starting point of the initial behavior strategy reinforcement learning training. The course learning unit gradually increases the complexity of the agent strategy through phased task design. The phased task design divides the task into primary stage, intermediate stage and advanced stage. The primary stage is a single-target task, the intermediate stage is a multi-agent collaborative task, and the advanced stage is a complex confrontation task. The complex confrontation task gradually improves the performance of the agent based on the strategy optimization of the low-level task. The course learning unit provides a clear learning path for the agent based on the initial strategy generation; The self-game optimization unit improves the adaptability of the agent in complex confrontation tasks by introducing dynamic opponent strategies, generates opponent strategies based on the current agent strategies, introduces random disturbances to increase confrontation diversity, and dynamically adjusts the strength of the opponent strategies according to the agent winning rate, increasing the strength of the opponent strategies when the winning rate is high, and reducing the strength of the opponent strategies when the winning rate is low; The centralized training improves teamwork ability by optimizing agent strategies from a global perspective during the centralized training phase. The global Critic network is used to evaluate the overall state and strategy, and the Actor network of each agent generates individual strategies to adapt to different task objectives. The distributed execution, in which each agent makes independent decisions based on local states to adapt to real-time environmental requirements, and the Actor network generates actions based on local observations to reduce dependence on global information. In collaborative tasks, agents achieve real-time collaboration by sharing some strategy parameters; The Actor network generates action probability distribution based on the input state to guide the agent's behavior decision-making, and optimizes the cumulative reward through the policy gradient method, which is expressed as: ; The Critic network evaluates the value of the current state to provide optimization direction for strategy update and minimize the valuation error, which is expressed as: ; The Actor network generates strategies and the Critic network evaluates the pros and cons of strategies, and the two are combined through centralized training and distributed execution to form a closed-loop optimization.

[0024] It should be noted that the combination of centralized training and distributed execution can be understood as an architectural design for training and execution. Centralized training means that during the training phase, the system uses global information to optimize the overall strategy. This method is suitable for learning complex collaboration and confrontation strategies because it has a global perspective. Distributed execution means that during the actual operation execution phase, the agent makes independent decisions based on the environmental state of its local observations. This method is more suitable for application scenarios with high real-time requirements and reduces dependence on global information. High-quality collaboration and confrontation strategies are obtained through centralized training, but during the execution phase, each agent makes independent decisions to adapt to the dynamic real-time environment requirements.

[0025] The self-game optimization unit in the agent training module implements the self-game mechanism; the self-game optimization unit is the core module of the self-game mechanism and is responsible for the dynamic opponent generation and strategy adjustment in the agent confrontation task. The specific work includes dynamic opponent strategy generation, dynamic opponent strength adjustment and multiple rounds of confrontation training; The dynamic opponent strategy generation generates a dynamic opponent based on the current agent strategy, introduces random perturbations and historical strategies, the opponent strength dynamic adjustment dynamically adjusts the complexity of the opponent strategy based on the agent's performance, and the multiple rounds of adversarial training optimize the agent strategy through multiple interactions and confrontations; The dynamic opponent strategy generation is the basic function of the self-game optimization unit to create a diverse confrontation environment for the agent, copy and generate the opponent strategy from the current agent strategy, introduce random perturbations into the opponent strategy to increase the uncertainty of the strategy and avoid confrontation solidification, and use the strategy pool management unit of the training platform to select historical strategies to provide multi-difficulty and diverse opponents; The dynamic adjustment of the opponent's strength is a further optimization after the dynamic opponent is generated, which is used to ensure that the difficulty of the opponent matches the ability of the agent. The opponent's strength is dynamically evaluated based on the winning rate of the agent. If the winning rate is higher than the preset threshold, the complexity of the opponent's strategy is increased. If the winning rate is lower than the preset threshold, the opponent's strength is reduced. The difficulty factor is used to dynamically control the core parameters of the opponent's strategy. The multi-round adversarial training is an important stage of the self-game mechanism. The evolution of the agent strategy is promoted through multiple interactions. In each round of training, the agent competes with the dynamically generated opponent, and the interaction data is stored in the training data storage unit. Based on the confrontation results, the agent optimizes its strategy using the Actor-Critic network in the reinforcement learning framework. The opponent strategy is updated after each round of training, and the complexity of the confrontation is gradually increased. The self-game mechanism improves the performance of the agent in resource allocation and path planning tasks by simulating the cooperative behavior of the opponent in the collaborative task. In the multi-agent environment, simulated teammates are generated for comparison with the target teammates, and the team collaboration is optimized through the reward mechanism. The agent learns how to divide the work and cooperate in the self-game to complete the resource allocation task; In the confrontation task, the self-game mechanism generates complex opponents through self-game, simulates the real competition environment to optimize the agent strategy, uses the Elo scoring mechanism to grade the opponent's strength, dynamically selects the opponent's strategy, and the agent improves its survivability in the dynamic environment by optimizing the attack behavior and avoidance strategy.

[0026] The course learning unit and the self-game optimization unit in the agent training module work together to complete the training process; the training process is divided into two main stages: course learning and self-game, combining dynamic task generation and phased optimization strategy to gradually realize the evolution of agents from simple tasks to complex tasks; The course learning unit is the basis of the training process. It gradually improves the strategy complexity and ability of the agent through task classification, and provides a three-stage training environment with task complexity from low to high, namely the primary stage, the intermediate stage and the advanced stage. The primary stage is to train the agent to complete simple tasks, the intermediate stage introduces multi-agent collaborative tasks, and the advanced stage further adds confrontational tasks. The agent needs to face dynamic opponents and optimize the collaboration and confrontation strategies; Based on course learning, the self-game optimization unit promotes the strategy evolution of the intelligent agent through dynamic opponent strategy generation and multiple rounds of adversarial training, generates dynamic opponent strategies in an adversarial training environment, prompts the intelligent agent to continuously optimize, uses a dynamic opponent strategy generation mechanism to provide the intelligent agent with confrontations of different difficulties, dynamically adjusts the opponent strength according to the winning rate in multiple rounds of confrontation, and maintains the effectiveness of the intelligent agent training; The training process is divided into an initialization phase, a course learning phase, a self-game phase and a comprehensive training phase by combining course learning and self-game mechanisms. The initialization phase provides the agent with a random or rule-generated initial strategy through an initial strategy generation unit. The course learning phase gradually increases the task complexity and optimizes the agent's collaborative ability through dynamic task classification. The self-game phase optimizes the agent's confrontation ability by generating dynamic opponent strategies. The comprehensive training phase combines collaborative and confrontation tasks to test the agent's global ability. Each stage in the training process relies on the support of the data management module and the user interaction module. The training data storage unit in the data management module records the interaction data between the agent and the environment in real time. The interaction data includes state, action and reward. The strategy pool management unit in the data management module stores the optimal strategy of each stage to provide reference for dynamic opponent generation. The data analysis unit in the data management module analyzes the training results to adjust the task parameters and reward mechanism. The task configuration unit in the user interaction module supports the user to adjust the training parameters in real time, and the training monitoring unit in the user interaction module provides visual feedback, which includes the task completion rate and the strategy optimization curve.

[0027] The environment simulation unit in the task and environment module and the data analysis unit in the data management module work together to realize the reward design; the reward design is the core driver of the reinforcement learning framework and the self-game mechanism. The reward mechanism provides optimization goals for the learning of the intelligent agent, and through the task-level reward table design and dynamic adjustment mechanism, it ensures that the intelligent agent can effectively evolve at different task stages; The environment simulation unit is responsible for real-time reward distribution during the training process, guiding the strategy optimization of the agent. In the collaborative task, the reward mechanism evaluates the overall performance of the team and distributes rewards based on resource utilization and path efficiency. In the confrontation task, the reward mechanism evaluates the attack efficiency and survivability of the agent and distributes targeted reward values. When the agent achieves the task goal, a positive reward is given, and a negative reward is given for the task failure or resource waste. An incremental reward mechanism is designed for the continuous successful behavior to encourage the continuous optimization of the strategy. The data analysis unit adjusts the reward mechanism in real time by analyzing the training results to ensure that the reward design can adapt to different task stages and the learning progress of the agent, adjusts the reward weight of the task goal according to the performance curve of the agent, dynamically balances the reward values ​​of different goals in collaborative and confrontational tasks, prevents the agent from being biased towards a certain strategy, uses the win rate curve and task completion rate to evaluate the reward distribution effect, and adjusts the values ​​in the reward table through data feedback; The reward table design is the core of the reward mechanism, which realizes the refined design of task classification and reward value, sets reward value according to the classification and complexity of the task, guides the agent to gradually optimize the strategy from low level to high level, designs corresponding reward and penalty rules for different behaviors, rewards collaboration efficiency in collaborative tasks, and rewards attack success rate in confrontation tasks; The reward mechanism focuses on guiding the teamwork and resource optimization capabilities of the agents in collaborative tasks, rewards the agents for completing collaborative behaviors, including resource sharing, target allocation, and path planning, punishes behaviors that cause resource conflicts or inefficiencies, allocates global rewards for team completion of tasks, and allocates individual rewards based on the contribution ratio of the agents, designs path efficiency rewards, and gives additional rewards to agents that complete path planning; In confrontation tasks, the reward mechanism emphasizes the confrontation ability and adaptability of the agent, rewards the agent's attacking behavior, avoidance behavior and victory results, punishes unnecessary losses or inefficient strategy choices, distributes winning and losing rewards based on the confrontation results, punishes being defeated, introduces innovative strategy rewards, and encourages the agent to explore diverse confrontation strategies.

[0028] Please refer to Figure 1 Schematic diagram of an online evolutionary algorithm of an intelligent agent based on self-game provided by an embodiment of the present invention, including: Step S1, completing task modeling through the task generation unit and the environment simulation unit in the task and environment module; Step S2, strengthening the learning framework through the collaborative work of the initial strategy generation unit, the course learning unit and the self-game optimization unit in the agent training module; Step S3, realizing the self-game mechanism through the self-game optimization unit in the agent training module; Step S4, completing the training process through the collaborative work of the course learning unit and the self-game optimization unit in the agent training module; Step S5, reward design is implemented through the collaboration of the environment simulation unit in the task and environment module and the data analysis unit in the data management module.

[0029] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention has various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An online evolutionary training platform for intelligent agents based on self-game, characterized by: include: Task and environment module, agent training module, strategy evaluation module, data management module and user interaction module; The task and environment module is used to define and simulate the training environment of the intelligent agent. The task and environment module includes a task generation unit and an environment simulation unit. The task generation unit is used to dynamically generate task objectives of different difficulties. The task objectives include collaborative tasks and confrontation tasks. The environment simulation unit supports multi-agent environment simulation to dynamically adjust environment parameters. The environment parameters include reward mechanisms and opponent strategies. The agent training module is the core of the system and is used to realize the functions of strategy optimization, curriculum learning, and self-game mechanism. The agent training module includes an initial strategy generation unit, a curriculum learning unit, and a self-game optimization unit. The initial strategy generation unit generates an initial strategy based on random initialization or rules. The curriculum learning unit trains the agent strategy in stages according to the complexity of the task. The self-game optimization unit generates a dynamic opponent strategy to confront the agent. The strategy evaluation module is used to test and evaluate the performance of the trained agent strategy; the strategy evaluation module includes a performance evaluation unit, a diversity analysis unit and a comparative analysis unit, the performance evaluation unit evaluates the strategy quality through the winning rate and resource efficiency indicators, the diversity analysis unit analyzes the innovation and diversity of the strategy, and the comparative analysis unit compares with the benchmark algorithm to verify the improvement effect; The data management module is responsible for storing, updating and analyzing training data and strategy information. The data management module includes a training data storage unit, a strategy pool management unit and a data analysis unit. The training data storage unit stores the state, action and reward data generated by the interaction between the agent and the environment. The strategy pool management unit stores the historical optimal strategy for subsequent training and evaluation. The data analysis unit performs statistics and optimization analysis on the stored data. The user interaction module provides users with an interface for system parameter configuration, task definition and result display. The user interaction module includes a task configuration unit, a training monitoring unit and a result visualization unit. The task configuration unit supports users to define task objectives and environmental parameters. The training monitoring unit displays the training progress and performance curve of the intelligent agent in real time. The result visualization unit displays the strategy evaluation results, which include a win rate curve and a strategy convergence diagram.

2. The self-game-based intelligent agent online evolution training platform according to claim 1 is characterized in that: The task generation unit and the environment simulation unit in the task and environment module jointly complete the task modeling; wherein: The task modeling is the basis of the whole system and is used to define the mathematical model of the interaction between the agent and the environment; The collaborative tasks include resource allocation and goal achievement scenarios, the confrontational tasks include defeating opponents and area control scenarios, the task generation unit dynamically adjusts the task difficulty according to user needs or training progress, and the task difficulty is divided into simple tasks and complex tasks. The simple tasks are single-goal tasks for training basic behaviors of intelligent agents, and the complex tasks are multi-goal mixed tasks involving collaboration and confrontation; The environment simulation unit simulates the dynamic interaction between the agent and the environment based on task generation, the dynamic interaction includes multi-agent environment simulation and dynamic parameter adjustment, the multi-agent environment simulation supports the interactive behavior of multiple agents in cooperation and confrontation scenarios, the dynamic parameter adjustment includes a reward mechanism and an opponent strategy, the reward mechanism dynamically allocates reward values ​​according to the task type, and the opponent strategy optimizes the opponent behavior according to the self-game mechanism; The task generation unit is the input source of task modeling, which directly affects the definition of state space and action space. The environment simulation unit provides operation rules for task modeling and affects the definition of reward function and transition probability. The state space defines a set of environmental information perceived by the agent. The state space includes its own state, environmental state, and other agent states. The own state includes position, speed, and resource status. The environmental state includes target position, resource distribution, and obstacle information. The other agent states include collaborative tasks and confrontational tasks. The collaborative tasks represent the position information and behavior of teammates, and the confrontational tasks represent the strategy history and current state of the opponent. The state space is expressed as: ； in, Represents the local state of a single agent at a certain moment; The action space defines the operations that the agent can perform in each state. The operations include collaborative task actions and confrontation task actions. The collaborative task actions include resource allocation and path planning. The resource allocation is to allocate resources to different targets. The path planning is to select the optimal path to reach the target. The confrontation task includes attack behavior and avoidance behavior. The attack behavior is to select the target opponent to attack. The avoidance behavior is to avoid the threat of the opponent to optimize its own survivability. The action space is expressed as: ； in, Represents an action that the agent can choose; The reward function is distributed in real time by the environment simulation unit to evaluate the effect of the action performed by the agent in a certain state. The action effect includes the reward design of the collaborative task and the reward design of the confrontation task. The reward design of the collaborative task includes achieving the goal, failing to achieve the goal and failure due to resource conflict. The reward design of the confrontation task includes defeating the opponent, being defeated and continuous victory. The reward mechanism is dynamically adjusted based on the task completion and the performance of the agent. The transition probability is determined by the environment simulation unit to describe the change of the environment state after the agent performs an action. The state change includes collaborative tasks and confrontation tasks. The collaborative task is that the transition probability is affected by the teammates' behavior and environmental rules, and the confrontation task is that the transition probability is determined by the opponent's strategy and dynamic environmental factors. The transition probability is expressed as: ； in, is the current state, is the current action of the agent.

3. The self-game-based intelligent agent online evolution training platform according to claim 1 is characterized in that: The initial strategy generation unit, the course learning unit and the self-game optimization unit in the agent training module work together to strengthen the learning framework; wherein: The reinforcement learning framework adopts a combination of centralized training and distributed execution, and optimizes the agent strategy based on the Actor-Critic architecture design. The Actor-Critic architecture, namely the Actor-Critic network, is a common framework in reinforcement learning and is widely used in strategy optimization and state evaluation tasks. The Actor is responsible for generating the agent's behavior strategy in the framework and maximizing the cumulative reward through the policy gradient method. The Critic is responsible for evaluating the strategy selected by the Actor and providing feedback through the estimated value function to help optimize the strategy. The initial strategy generation unit is responsible for providing the starting point of the initial behavior strategy reinforcement learning training. The course learning unit gradually increases the complexity of the agent strategy through phased task design. The phased task design divides the task into primary stage, intermediate stage and advanced stage. The primary stage is a single-target task, the intermediate stage is a multi-agent collaborative task, and the advanced stage is a complex confrontation task. The complex confrontation task gradually improves the performance of the agent based on the strategy optimization of the low-level task. The course learning unit provides a clear learning path for the agent based on the initial strategy generation; The self-game optimization unit improves the adaptability of the agent in complex confrontation tasks by introducing dynamic opponent strategies, generates opponent strategies based on the current agent strategies, introduces random disturbances to increase confrontation diversity, and dynamically adjusts the strength of the opponent strategies according to the agent winning rate, increasing the strength of the opponent strategies when the winning rate is high, and reducing the strength of the opponent strategies when the winning rate is low; The centralized training improves teamwork ability by optimizing agent strategies from a global perspective during the centralized training phase. The global Critic network is used to evaluate the overall state and strategy, and the Actor network of each agent generates individual strategies to adapt to different task objectives. The distributed execution, in which each agent makes independent decisions based on local states to adapt to real-time environmental requirements, and the Actor network generates actions based on local observations to reduce dependence on global information. In collaborative tasks, agents achieve real-time collaboration by sharing some strategy parameters; The Actor network generates action probability distribution based on the input state to guide the agent's behavior decision-making, and optimizes the cumulative reward through the policy gradient method, which is expressed as: ； The Critic network evaluates the value of the current state to provide optimization direction for strategy update and minimize the valuation error, which is expressed as: ； The Actor network generates strategies and the Critic network evaluates the pros and cons of strategies, and the two are combined through centralized training and distributed execution to form a closed-loop optimization.

4. The self-game-based intelligent agent online evolution training platform according to claim 1, characterized in that: The self-game optimization unit in the agent training module implements a self-game mechanism; wherein: The self-game optimization unit is the core module of the self-game mechanism and is responsible for the dynamic opponent generation and strategy adjustment in the intelligent agent confrontation task. The specific work includes dynamic opponent strategy generation, dynamic opponent strength adjustment and multiple rounds of confrontation training. The dynamic opponent strategy generation generates a dynamic opponent based on the current agent strategy, introduces random perturbations and historical strategies, the opponent strength dynamic adjustment dynamically adjusts the complexity of the opponent strategy based on the agent's performance, and the multiple rounds of adversarial training optimize the agent strategy through multiple interactions and confrontations; The dynamic opponent strategy generation is the basic function of the self-game optimization unit to create a diverse confrontation environment for the agent, copy and generate the opponent strategy from the current agent strategy, introduce random perturbations into the opponent strategy to increase the uncertainty of the strategy and avoid confrontation solidification, and use the strategy pool management unit of the training platform to select historical strategies to provide multi-difficulty and diverse opponents; The dynamic adjustment of the opponent's strength is a further optimization after the dynamic opponent is generated, which is used to ensure that the difficulty of the opponent matches the ability of the agent. The opponent's strength is dynamically evaluated based on the winning rate of the agent. If the winning rate is higher than the preset threshold, the complexity of the opponent's strategy is increased. If the winning rate is lower than the preset threshold, the opponent's strength is reduced. The difficulty factor is used to dynamically control the core parameters of the opponent's strategy. The multi-round adversarial training is an important stage of the self-game mechanism. The evolution of the agent strategy is promoted through multiple interactions. In each round of training, the agent competes with the dynamically generated opponent, and the interaction data is stored in the training data storage unit. Based on the confrontation results, the agent optimizes its strategy using the Actor-Critic network in the reinforcement learning framework. The opponent strategy is updated after each round of training, and the complexity of the confrontation is gradually increased. The self-game mechanism improves the performance of the agent in resource allocation and path planning tasks by simulating the cooperative behavior of the opponent in the collaborative task. In the multi-agent environment, simulated teammates are generated for comparison with the target teammates, and the team collaboration is optimized through the reward mechanism. The agent learns how to divide the work and cooperate in the self-game to complete the resource allocation task; In the confrontation task, the self-game mechanism generates complex opponents through self-game, simulates the real competition environment to optimize the agent strategy, uses the Elo scoring mechanism to grade the opponent's strength, dynamically selects the opponent's strategy, and the agent improves its survivability in the dynamic environment by optimizing the attack behavior and avoidance strategy.

5. The self-game-based intelligent agent online evolution training platform according to claim 1, characterized in that: The course learning unit and the self-game optimization unit in the agent training module work together to complete the training process; wherein: The training process is divided into two main stages: curriculum learning and self-game. It combines dynamic task generation with phased optimization strategy to gradually realize the evolution of intelligent agents from simple tasks to complex tasks. The course learning unit is the basis of the training process. It gradually improves the strategy complexity and ability of the agent through task classification, and provides a three-stage training environment with task complexity from low to high, namely the primary stage, the intermediate stage and the advanced stage. The primary stage is to train the agent to complete simple tasks, the intermediate stage introduces multi-agent collaborative tasks, and the advanced stage further adds confrontational tasks. The agent needs to face dynamic opponents and optimize the collaboration and confrontation strategies; Based on course learning, the self-game optimization unit promotes the strategy evolution of the intelligent agent through dynamic opponent strategy generation and multiple rounds of adversarial training, generates dynamic opponent strategies in an adversarial training environment, prompts the intelligent agent to continuously optimize, uses a dynamic opponent strategy generation mechanism to provide the intelligent agent with confrontations of different difficulties, dynamically adjusts the opponent strength according to the winning rate in multiple rounds of confrontation, and maintains the effectiveness of the intelligent agent training; The training process is divided into an initialization phase, a course learning phase, a self-game phase and a comprehensive training phase by combining course learning and self-game mechanisms. The initialization phase provides the agent with a random or rule-generated initial strategy through an initial strategy generation unit. The course learning phase gradually increases the task complexity and optimizes the agent's collaborative ability through dynamic task classification. The self-game phase optimizes the agent's confrontation ability by generating dynamic opponent strategies. The comprehensive training phase combines collaborative and confrontation tasks to test the agent's global ability. Each stage in the training process relies on the support of the data management module and the user interaction module. The training data storage unit in the data management module records the interaction data between the agent and the environment in real time. The interaction data includes state, action and reward. The strategy pool management unit in the data management module stores the optimal strategy of each stage to provide reference for dynamic opponent generation. The data analysis unit in the data management module analyzes the training results to adjust the task parameters and reward mechanism. The task configuration unit in the user interaction module supports the user to adjust the training parameters in real time, and the training monitoring unit in the user interaction module provides visual feedback, which includes the task completion rate and the strategy optimization curve.

6. The self-game-based intelligent agent online evolution training platform according to claim 1, characterized in that: The task and environment simulation unit in the environment module and the data analysis unit in the data management module cooperate to realize the reward design; wherein: The reward design is the core driver of the reinforcement learning framework and self-game mechanism. The reward mechanism provides optimization goals for the learning of the agent, and ensures that the agent can effectively evolve at different task stages through the design of task-level reward tables and dynamic adjustment mechanisms. The environment simulation unit is responsible for real-time reward distribution during the training process, guiding the strategy optimization of the agent. In the collaborative task, the reward mechanism evaluates the overall performance of the team and distributes rewards based on resource utilization and path efficiency. In the confrontation task, the reward mechanism evaluates the attack efficiency and survivability of the agent and distributes targeted reward values. When the agent achieves the task goal, a positive reward is given, and a negative reward is given for the task failure or resource waste. An incremental reward mechanism is designed for the continuous successful behavior to encourage the continuous optimization of the strategy. The data analysis unit adjusts the reward mechanism in real time by analyzing the training results to ensure that the reward design can adapt to different task stages and the learning progress of the agent, adjusts the reward weight of the task goal according to the performance curve of the agent, dynamically balances the reward values ​​of different goals in collaborative and confrontational tasks, prevents the agent from being biased towards a certain strategy, uses the win rate curve and task completion rate to evaluate the reward distribution effect, and adjusts the values ​​in the reward table through data feedback; The reward table design is the core of the reward mechanism, which realizes the refined design of task classification and reward value, sets reward value according to the classification and complexity of the task, guides the agent to gradually optimize the strategy from low level to high level, designs corresponding reward and penalty rules for different behaviors, rewards collaboration efficiency in collaborative tasks, and rewards attack success rate in confrontation tasks; The reward mechanism focuses on guiding the teamwork and resource optimization capabilities of the agents in collaborative tasks, rewards the agents for completing collaborative behaviors, including resource sharing, target allocation, and path planning, punishes behaviors that cause resource conflicts or inefficiencies, allocates global rewards for team completion of tasks, and allocates individual rewards based on the contribution ratio of the agents, designs path efficiency rewards, and gives additional rewards to agents that complete path planning; In confrontation tasks, the reward mechanism emphasizes the confrontation ability and adaptability of the agent, rewards the agent's attacking behavior, avoidance behavior and victory results, punishes unnecessary losses or inefficient strategy choices, distributes winning and losing rewards based on the confrontation results, punishes being defeated, introduces innovative strategy rewards, and encourages the agent to explore diverse confrontation strategies.

7. An online evolutionary algorithm of intelligent agents based on self-game, applied to the training platform described in any one of claims 1 to 6, characterized in that: The following steps are involved: Step S1, completing task modeling through the task generation unit and the environment simulation unit in the task and environment module; Step S2, strengthening the learning framework through the collaborative work of the initial strategy generation unit, the course learning unit and the self-game optimization unit in the agent training module; Step S3, realizing the self-game mechanism through the self-game optimization unit in the agent training module; Step S4, completing the training process through the collaborative work of the course learning unit and the self-game optimization unit in the agent training module; Step S5, reward design is implemented through the collaboration of the environment simulation unit in the task and environment module and the data analysis unit in the data management module.

Images