Crowdsourcing enhanced training method and system

Abstract

The present application relates to the technical field of electronic games, and particularly relates to a crowd wisdom enhancement training method and system, the crowd wisdom enhancement training method comprises the following steps: setting a perception layer module, a decision layer module, an execution layer module, a group relative policy optimization (GRPO) module and a direct preference optimization (DPO) module, using the GRPO to pre-train a bottom execution strategy in a self-game environment; collecting human expert game data, and fine-tuning the strategy through the DPO module; and a large language model of the Agentic-Core adapting high-level instruction generation through context learning or light weight fine-tuning (LoRA), so that the crowd wisdom enhancement training system of the present application can support real-time development, live interaction and other core scenes of pixel wind games.

Description

Technical Field

[0001] This invention relates to the field of video game technology, and in particular to a method and system for enhancing collective intelligence training. Background Technology

[0002] Existing reinforcement learning methods and products for multi-agent cooperative-competitive scenarios have the following significant shortcomings: (1) Limited scalability: Existing MARL benchmark environments (such as SMAC and MPE) are mostly small-scale adversarial modes of 2×N, which cannot effectively simulate the multi-party fragmentation (M≥3), dynamic alliance and betrayal behaviors in real online games or complex e-sports. When the number of agents exceeds dozens, training stability drops sharply.

[0003] (2) Insufficient depth of collaboration: In environments such as SMAC, agents can often achieve high scores through simple local observations (such as the number of friendly and enemy agents within their field of vision), lacking incentives for deep team collaborative behaviors such as "sacrifice-gain" and "quality-oriented delivery." Traditional value decomposition methods (such as QMIX) struggle to express non-monotonic collaborative gains.

[0004] (3) Bottleneck of black-box rewards: Hand-designed reward functions (such as kill score and survival time) cannot capture the psychological factors of human players in the game: such as deceptive movement, feints, bait strategies, and the tactical beauty of "skillful teamwork", resulting in mechanical and predictable AI behavior.

[0005] (4) Huge computational cost: Traditional Actor-Critic architectures (such as MAPPO and COMA) require the maintenance of a centralized Critic network in multi-agent scenarios. The input dimension increases exponentially with the number of agents M×N, resulting in unacceptable training memory and time costs, and making it difficult to deploy in real-time battle environments.

[0006] (5) Lack of hierarchical decision-making ability: Most existing methods employ a flat action space, which fails to effectively distinguish between millisecond-level micro-operations and second-level macro-strategies, making it difficult to learn long-range dependent strategies (such as "encirclement in two minutes"). These shortcomings prevent current technologies from supporting core scenarios such as real-time development and live interactive streaming of pixel-style games, necessitating groundbreaking innovation.

[0007] Therefore, it is necessary to provide a collective intelligence enhancement training method and system to support core scenarios such as real-time development and live interactive streaming of pixel-style games. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for collective intelligence enhancement training to support core scenarios such as real-time development and live interactive streaming of pixel-style games.

[0009] To address the problems existing in the prior art, this invention provides a method for enhancing collective intelligence training, comprising the following steps: S1: Set up a perception layer module and adopt a hierarchical observation mechanism to divide environmental information into two levels: global state and player state. The global state features and player state features are concatenated and then input into the decision layer. S2: Set up the decision-making layer module, introduce a large language model as a high-level inference engine, and perform the following steps; S21: Convert the numerical state output by the perception layer into text or symbolic representation; S22: The large language model generates a high-level instruction set c, c∈C, where C is a finite macro action space; S23: Higher-level instructions are passed to the execution layer in the form of embedded vectors to guide the selection of lower-level strategies; S3: Set up the execution layer module. The execution layer is responsible for generating specific atomic actions. The execution layer is trained using a group-relative strategy optimization. S4: Set up a group-relative policy optimization module. For G agents within the same team, calculate the advantage function based on the average reward within the group. ; in, Let be the average reward advantage value within the group after agent i takes a certain action at a specific time t. Let be the cumulative reward of agent i at time t. Let be the cumulative reward of agent j at time t. Let G be the standard deviation of the rewards for the G agents in the group. It is a small constant; S5: Set up a direct preference optimization module, use human game recordings to build a preference dataset, and fine-tune the strategy directly through the DPO loss function.

[0010] Optionally, in the aforementioned crowd intelligence enhancement training method, Global state: Contains macro-level information for all teams. The macro-level information for all teams is extracted into global features by the Transformer encoder. The macro-level information for all teams includes quality distribution, spiky location, and total spore resources. Player Status: Includes each agent's own mass, speed, cooldown time, and details of enemy and friendly units within its field of vision.

[0011] Optionally, in the aforementioned collective intelligence enhancement training method, the atomic action in S3 is a:=(x,y,t), where (x,y) represents the direction of movement, splitting, or spraying, (x,y) is a continuous value, and t∈{0,1} indicates whether to perform the splitting or spraying operation.

[0012] Optionally, in the aforementioned crowd intelligence enhancement training method, The preference dataset in S5 is ,in, In game state, The winning side; The losing side; Fine-tuning the strategy directly using the DPO loss function : ; in, for For a policy network with θ as a parameter, For reference strategy (initial or frozen copy). For preference-based datasets (x, , The mathematical expectation of the distribution. It is the Sigmoid activation function. Hyperparameters used to control the degree of deviation.

[0013] The present invention also provides a crowd intelligence enhancement training system, comprising: Set up the perception layer module and configure it to adopt a hierarchical observation mechanism to divide environmental information into two levels: global state and player state. The global state features and player state features are concatenated and then input into the decision layer. The decision layer module is configured to introduce a large language model as a high-level inference engine and perform the following steps: convert the numerical state output by the perception layer into text or symbolic representation; the large language model generates a high-level instruction set c, c∈C, where C is a finite macro action space; the high-level instructions are passed to the execution layer in the form of embedded vectors to guide the selection of the underlying strategy. The execution layer module is configured to be responsible for generating specific atomic actions. The execution layer is trained using a group-relative strategy optimization. The group-based relative strategy optimization module is configured to calculate the advantage function based on the average reward within the group for G agents in the same team. ; in, Let t be the average reward advantage value within the group after agent i takes a certain action at a specific time t. Let be the cumulative reward of agent i at time t. Let be the cumulative reward of agent j at time t. Let G be the standard deviation of the rewards for the G agents in the group. It is a small constant; The Direct Preference Optimization module is configured to construct a preference dataset using human match recordings and directly fine-tune the policy using the DPO loss function.

[0014] Optionally, in the aforementioned crowd intelligence enhancement training system, Global state: Contains macro-level information for all teams. The macro-level information for all teams is extracted into global features by the Transformer encoder. The macro-level information for all teams includes quality distribution, spiky location, and total spore resources. Player Status: Includes each agent's own mass, speed, cooldown time, and details of enemy and friendly units within its field of vision.

[0015] Optionally, in the aforementioned collective intelligence enhancement training system, the atomic action in the execution layer module is a:=(x,y,t), where (x,y) represents the direction of movement, splitting, or spraying, (x,y) is a continuous value, and t∈{0,1} indicates whether to perform a splitting or spraying operation.

[0016] Optionally, in the aforementioned crowd intelligence enhancement training system, The preference dataset in the direct preference optimization module is ,in, In game state, The winning side; The losing side; Fine-tuning the strategy directly using the DPO loss function : ; in, for For a policy network with θ as a parameter, For reference strategy (initial or frozen copy). For preference-based datasets (x, , The mathematical expectation of the distribution. It is the Sigmoid activation function. Hyperparameters used to control the degree of deviation.

[0017] Compared with the prior art, the present invention has the following advantages: (1) Improved collaboration efficiency: Through the intra-group relative evaluation mechanism of GRPO, agents can spontaneously generate advanced cooperative behaviors, including: "Sacrificing individual quality for team vision": Edge agents actively split to expand their vision, even if their own quality decreases; "Targeted mass delivery": The intelligent agent autonomously identifies the core units in the team and sprays mass toward them to form a super-large "team pillar".

[0018] Experiments show that in a 4×8 scale environment, the team's overall phasing efficiency is improved by 35%.

[0019] (2) Reduced computational costs: By abandoning the centralized Critic network, the training overhead of this invention is significantly reduced in large-scale M×N scenarios: The computational complexity of each step is O(M). N)2) decreased to O(M) N); The actual training speed is 40% to 60% faster than MAPPO, and the memory usage is reduced by more than 50%.

[0020] (3) Humanized strategy: By combining Agentic-Core's symbolic reasoning with DPO's preference alignment, the agent exhibits "human-like" tactical logic that is difficult to achieve with traditional reinforcement learning: Deceptive movement: Intentionally moving in the wrong direction to lure the opponent; Feint and cover: Some agents launch feint attacks to draw fire, allowing teammates to seize resources. Risk trade-off: When at a disadvantage, choose "fake retreat, real counterattack" rather than mechanical conservatism.

[0021] Human reviewers improved their ratings of the "tactical aesthetics" of AI behavior by 2.1 times (out of 1-5).

[0022] (4) Scalability and generalization ability: The system supports dynamically expanding the number of teams M≥2 and the number of agents per team N≥1 without redesigning the architecture. Trained agents can be directly transferred to environments with different map sizes and resource distributions, exhibiting better generalization performance than traditional MARL methods.

[0023] (5) Value of esports training and balance testing: By adjusting DPO preference data (such as injecting aggressive or conservative expert data), AI practice opponents with different styles can be generated, providing targeted training for professional teams. Simultaneously, multi-style AI self-play can be used to quickly assess the impact of game version updates on tactical diversity. Attached Figure Description

[0024] Figure 1 The flowchart illustrates the training process of the multi-agent GRPO reinforcement learning algorithm provided in this embodiment of the invention. Detailed Implementation

[0025] The specific embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. The advantages and features of the present invention will become clearer from the following description. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0026] In the following, if the methods described herein include a series of steps, the order of these steps presented herein is not necessarily the only order in which these steps can be performed, and some of the steps described may be omitted and / or some other steps not described herein may be added to the method.

[0027] 1. The application scenarios of this invention are as follows: This invention is primarily applied to the field of complex, multi-dimensional interactive video games, and is particularly suitable for real-time strategy (RTS), multiplayer online battle arena (MOBA), and "ball-evolving" games (such as GoBigger, Agar.io variants, etc.) with the following technical characteristics: (1) Large-scale multi-team adversarial environment: It supports MM competing teams, each team containing NN controllable intelligent agents, forming an M×N swarm intelligence interaction pattern. Teams are in competition with each other, but must cooperate within a team, allowing dynamic alliances and betrayals.

[0028] (2) Highly dynamic evolutionary game units: Basic entities in the game (such as clone spheres, spore spheres, and spiked spheres) undergo real-time physical state changes, including: Split: An agent splits into multiple sub-units; Merge: The re-merging of multiple sub-units; Eject: Actively release mass to alter terrain or accelerate; Eat: Determines the winner by comparing quality.

[0029] These operations both constitute part of the action space and alter the state of the environment.

[0030] (3) Hierarchical decision-making needs: The agent must make decisions simultaneously at the following two levels: Macro-strategic level: team encirclement, resource blockade, luring the enemy deep into our territory, and collective retreat; Microscopic operation layer: precise splitting angle, dodging spiked balls, mass transfer, instantaneous merging.

[0031] Traditional flat motion spaces struggle to capture these two types of decisions with vastly different time scales simultaneously.

[0032] (4) Esports training and game balance test: It is used to produce competitive AI that surpasses the level of human professional players, providing high-intensity practice; The game's character / item / map balance was evaluated by simulating different styles of human players (aggressive, conservative, deceptive). Support preference alignment: Make the AI's behavior style conform to the "tactical aesthetics" advocated by the game community.

[0033] Typical application example: In a GoBigger-like environment with M=4 teams and N=8 agents per team, the method described in this invention can realize real-time game between hundreds of agents, with each agent having a state dimension >50 and an action space consisting of continuous two-dimensional directions plus discrete split / spray commands.

[0034] 2. This invention aims to address the following core shortcomings of traditional reinforcement learning algorithms in existing large-scale cooperative-competitive multi-agent systems: (1) Reward sparsity and delay: In an M×N highly dynamic environment, the causal relationship between individual actions and the final outcome is extremely sparse, resulting in large policy gradient variance and slow convergence.

[0035] (2) Low collaboration efficiency: Traditional methods (such as PPO and QMIX) rarely spontaneously generate advanced collaborative behaviors such as "sacrificing individual quality for team vision" or "directly delivering quality to core teammates," and often fall into simple collaboration with local optima.

[0036] (3) The strategy is seriously out of sync with human behavior: AI driven purely by reward maximization often produces mechanical, non-human-like actions (such as meaningless jittering and extreme conservatism), lacking the psychological game, deceptive movement, and tactical aesthetics unique to human players, and is not suitable for esports training and balance testing.

[0037] (4) The scalability bottleneck of centralized critics: In traditional Actor-Critic architectures, the input dimension of the critic network increases exponentially with the number of agents in multi-agent scenarios, making the training overhead unacceptable.

[0038] To this end, this invention proposes a fusion framework of "Agentic Hierarchical Reinforcement Learning with GRPO and DPO (AGD)", with the following specific objectives: Introducing Group Relative Policy Optimization (GRPO): This approach eliminates the centralized commentator and utilizes the relative rewards of agents within a team to estimate the advantage function, thereby reducing computational complexity and enhancing collaborative emergence capabilities.

[0039] Combined with Direct Preference Optimization (DPO): Preference pairs are constructed using game data from human experts, and strategies are directly fine-tuned so that AI can exhibit behaviors that conform to human aesthetics and tactical logic while winning.

[0040] A hierarchical intelligent agent architecture is adopted: a large language model (LLM) is used as the high-level inference engine to generate macro-level instructions, and atomic actions are executed at the lower level, thereby decoupling strategy and operation and improving long-term decision-making capabilities.

[0041] By integrating the above technologies, this invention ultimately achieves an intelligent agent training method and system that features efficient collective intelligence collaboration, humanized strategies, and scalability to large-scale multi-team adversarial environments.

[0042] Specifically, this invention provides a method for enhancing collective intelligence training, such as... Figure 1 As shown, the crowd intelligence enhancement training method includes the following steps: S1: Set up a perception layer module and adopt a hierarchical observation mechanism to divide environmental information into two levels: global state and player state. The global state features and player state features are concatenated and then input into the decision layer. Furthermore, the Global State contains macroscopic information about all teams. The macroscopic information of all teams is extracted with global features by the Transformer encoder. The macroscopic information of all teams includes mass distribution, spike ball position, and total spore resources. The Player State contains the mass, speed, cooldown time, and details of enemy and friendly units within the field of vision of each agent.

[0043] S2: Set up the decision-making module (Agentic-Core), introduce a large language model as a high-level inference engine, and perform the following steps; S21: Convert the numerical state output by the perception layer into text or symbolic representation (e.g., "Our team is superior in quality, but the right wing is weak"). S22: The large language model generates a high-level instruction set c, c∈C, where C is a finite macro action space; For example: TEAM_SPLIT_ATTACK (Collective Split Attack); FALLBACK_DEFENSE (shrink defense); FEED_CORE (delivers mass to core teammates); S23: Higher-level instructions are passed to the execution layer in the form of embedded vectors to guide the selection of underlying strategies.

[0044] S3: Set up the execution layer module. The execution layer is responsible for generating specific atomic actions. The execution layer is trained using a group-relative strategy optimization. Furthermore, the atomic action is a:=(x,y,t), where (x,y) represents the direction of movement, splitting, or ejection, (x,y) are continuous values, and t∈{0,1} indicates whether to perform a splitting or ejection operation. The execution layer is trained using Group Relative Policy Optimization (GRPO) and is independent of centralized commentators.

[0045] S4: Set up the Group Relative Policy Optimization (GRPO) module. For G agents within the same team, calculate the advantage function based on the average reward within the group. ; in, Let t be the average reward advantage value within the group after agent i takes a certain action at a specific time t. Let be the cumulative reward of agent i at time t (considering the discount factor). Let be the cumulative reward of agent j at time t. Let G be the standard deviation of the rewards for the G agents in the group. Use a small constant to prevent division by zero. This step allows the agent to focus on outperforming its teammates in a relative way, thus spontaneously generating cooperative behaviors such as "sacrifice-gain".

[0046] S5: Set up the Direct Preference Optimization (DPO) module, construct a preference dataset using video recordings of top human players, and fine-tune the strategy directly through the DPO loss function; Furthermore, the preference dataset is ,in, In game state, The winning side; The losing side; Fine-tuning the strategy directly using the DPO loss function : ; in, for For a policy network with θ as a parameter, For reference strategy (initial or frozen copy). For preference-based datasets (x, , The mathematical expectation of the distribution. It is the Sigmoid activation function. Hyperparameters are used to control the degree of deviation. This module ensures that the agent not only pursues victory but also behaves in accordance with the tactical aesthetics of human experts; Furthermore, fine-tuning refers to the process in deep learning of further training a pre-trained model to adapt it to a specific task or dataset. Fine-tuning enables the model to perform better on a specific task by training it on a smaller, newer dataset, building upon the pre-trained model, without needing to train the entire model from scratch. This approach not only saves time and computational resources but also effectively improves the model's performance on specific tasks.

[0047] Preferably, the present invention also provides a crowd intelligence enhancement training system, comprising: (1) Set up the perception layer module and configure it to adopt a hierarchical observation mechanism to divide the environmental information into two levels, namely the global state and the player state. The global state features and the player state features are concatenated and then input into the decision layer. Furthermore, the Global State contains macroscopic information about all teams. The macroscopic information of all teams is extracted with global features by the Transformer encoder. The macroscopic information of all teams includes mass distribution, spike ball position, and total spore resources. The Player State contains the mass, speed, cooldown time, and details of enemy and friendly units within the field of vision of each agent.

[0048] (2) Decision-making module (Agentic-Core), configured to introduce a large language model as a high-level inference engine, executes the following steps; 1) Convert the numerical state output by the perception layer into text or symbolic representation (e.g., "Our team is superior in quality, but the right wing is weak"). 2) The large language model generates a high-level instruction set c, c∈C, where C is a finite macro action space; For example: EAM_SPLIT_ATTACK (Collective Split Attack); FALLBACK_DEFENSE (shrink defense); FEED_CORE (delivers mass to core teammates); 3) High-level instructions are passed to the execution layer in the form of embedded vectors to guide the selection of the underlying strategy. (3) The execution layer module is configured to be responsible for generating specific atomic actions. The execution layer is trained using a group-relative strategy optimization. Furthermore, the atomic action is a:=(x,y,t), where (x,y) represents the direction of movement, splitting, or ejection, (x,y) are continuous values, and t∈{0,1} indicates whether to perform a splitting or ejection operation. The execution layer is trained using Group Relative Policy Optimization (GRPO) and is independent of centralized commentators.

[0049] (4) Group Relative Policy Optimization (GRPO) module, configured to calculate the advantage function based on the average reward within the group for G agents in the same team: ; in, Let t be the average reward advantage value within the group after agent i takes a certain action at a specific time t. Let be the cumulative reward of agent i at time t (considering the discount factor). Let be the cumulative reward of agent j at time t. Let G be the standard deviation of the rewards for the G agents in the group. Use a small constant to prevent division by zero. This step allows the agent to focus on outperforming its teammates in a relative way, thus spontaneously generating cooperative behaviors such as "sacrifice-gain".

[0050] (5) Direct Preference Optimization (DPO) module, configured to construct a preference dataset using video recordings of matches by top human players, and to directly fine-tune the policy through the DPO loss function; further, the preference dataset is... ,in, In game state, The winning side; The losing side; Fine-tuning the strategy directly using the DPO loss function : ; in, for For a policy network with θ as a parameter, For reference strategy (initial or frozen copy). For preference-based datasets (x, , The mathematical expectation of the distribution. It is the Sigmoid activation function. Hyperparameters are used to control the degree of deviation. This module ensures that the agent not only pursues victory but also behaves in accordance with the tactical aesthetics of human experts.

[0051] Compared with the prior art, the present invention has the following advantages: (1) Improved collaboration efficiency: Through the intra-group relative evaluation mechanism of GRPO, agents can spontaneously generate advanced cooperative behaviors, including: "Sacrificing individual quality for team vision": Edge agents actively split to expand their vision, even if their own quality decreases; "Targeted mass delivery": The intelligent agent autonomously identifies the core units in the team and sprays mass toward them to form a super-large "team pillar".

[0052] Experiments show that in a 4×8 scale environment, the team's overall phasing efficiency is improved by 35%.

[0053] (2) Reduced computational costs: By abandoning the centralized Critic network, the training overhead of this invention is significantly reduced in large-scale M×N scenarios: The computational complexity of each step is O(M). N)2) decreased to O(M) N); The actual training speed is 40% to 60% faster than MAPPO, and the memory usage is reduced by more than 50%.

[0054] (3) Humanized strategy: By combining Agentic-Core's symbolic reasoning with DPO's preference alignment, the agent exhibits "human-like" tactical logic that is difficult to achieve with traditional reinforcement learning: Deceptive movement: Intentionally moving in the wrong direction to lure the opponent; Feint and cover: Some agents launch feint attacks to draw fire, allowing teammates to seize resources. Risk trade-off: When at a disadvantage, choose "fake retreat, real counterattack" rather than mechanical conservatism.

[0055] Human reviewers improved their ratings of the "tactical aesthetics" of AI behavior by 2.1 times (out of 1-5).

[0056] (4) Scalability and generalization ability: The system supports dynamically expanding the number of teams M≥2 and the number of agents per team N≥1 without redesigning the architecture. Trained agents can be directly transferred to environments with different map sizes and resource distributions, exhibiting better generalization performance than traditional MARL methods.

[0057] (5) Value of esports training and balance testing: By adjusting DPO preference data (such as injecting aggressive or conservative expert data), AI practice opponents with different styles can be generated, providing targeted training for professional teams. Simultaneously, multi-style AI self-play can be used to quickly assess the impact of game version updates on tactical diversity.

[0058] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. A method for enhancing collective intelligence training, characterized in that, Includes the following steps: S1: Set up a perception layer module and adopt a hierarchical observation mechanism to divide environmental information into two levels: global state and player state. The global state features and player state features are concatenated and then input into the decision layer. S2: Set up the decision-making layer module, introduce a large language model as a high-level inference engine, and perform the following steps; S21: Convert the numerical state output by the perception layer into text or symbolic representation; S22: The large language model generates a high-level instruction set c, c∈C, where C is a finite macro action space; S23: Higher-level instructions are passed to the execution layer in the form of embedded vectors to guide the selection of lower-level strategies; S3: Set up the execution layer module. The execution layer is responsible for generating specific atomic actions. The execution layer is trained using a group-relative strategy optimization. S4: Set up a group-relative policy optimization module. For G agents within the same team, calculate the advantage function based on the average reward within the group. ； in, Let i be the average reward advantage value within the group after agent i takes an action at a specific time t. Let be the cumulative reward of agent i at time t. Let be the cumulative reward of agent j at time t. Let G be the standard deviation of the rewards for the G agents in the group. It is a small constant; S5: Set up a direct preference optimization module, use human game recordings to build a preference dataset, and fine-tune the strategy directly through the DPO loss function.

2. The crowd intelligence enhancement training method as described in claim 1, characterized in that, Global state: Contains macro-level information for all teams. The macro-level information for all teams is extracted into global features by the Transformer encoder. The macro-level information for all teams includes quality distribution, spiky location, and total spore resources. Player Status: Includes each agent's own mass, speed, cooldown time, and details of enemy and friendly units within its field of vision.

3. The crowd intelligence enhancement training method as described in claim 1, characterized in that, In S3, the atomic action is a:=(x,y,t), where (x,y) represents the direction of movement, splitting, or ejection, (x,y) is a continuous value, and t∈{0,1} indicates whether to perform the splitting or ejection operation.

4. The crowd intelligence enhancement training method as described in claim 1, characterized in that, The preference dataset in S5 is ,in, In game state, The winning side; The losing side; Fine-tuning the strategy directly using the DPO loss function : ； in, for For a policy network with θ as a parameter, For reference strategy, For preference-based datasets (x, , The mathematical expectation of the distribution. It is the Sigmoid activation function. Hyperparameters used to control the degree of deviation.

5. A crowd intelligence enhancement training system, characterized in that, include: Set up the perception layer module and configure it to adopt a hierarchical observation mechanism to divide environmental information into two levels: global state and player state. The global state features and player state features are concatenated and then input into the decision layer. The decision layer module is configured to introduce a large language model as a high-level inference engine and perform the following steps: convert the numerical state output by the perception layer into text or symbolic representation; the large language model generates a high-level instruction set c, c∈C, where C is a finite macro action space; the high-level instructions are passed to the execution layer in the form of embedded vectors to guide the selection of the underlying strategy. The execution layer module is configured to be responsible for generating specific atomic actions. The execution layer is trained using a group-relative strategy optimization. The group-based relative strategy optimization module is configured to calculate the advantage function based on the average reward within the group for G agents in the same team. ； in, Let i be the average reward advantage value within the group after agent i takes an action at a specific time t. Let be the cumulative reward of agent i at time t. Let be the cumulative reward of agent j at time t. Let G be the standard deviation of the rewards for the G agents in the group. It is a small constant; The Direct Preference Optimization module is configured to construct a preference dataset using human match recordings and directly fine-tune the policy using the DPO loss function.

6. The crowd intelligence enhancement training system as described in claim 5, characterized in that, Global state: Contains macro-level information for all teams. The macro-level information for all teams is extracted into global features by the Transformer encoder. The macro-level information for all teams includes quality distribution, spiky location, and total spore resources. Player Status: Includes each agent's own mass, speed, cooldown time, and details of enemy and friendly units within its field of vision.

7. The crowd intelligence enhancement training system as described in claim 5, characterized in that, In the execution layer module, the atomic action is a:=(x,y,t), where (x,y) represents the direction of movement, splitting, or spraying, (x,y) is a continuous value, and t∈{0,1} indicates whether to perform the splitting or spraying operation.

8. The crowd intelligence enhancement training system as described in claim 5, characterized in that, The preference dataset in the direct preference optimization module is ,in, In game state, The winning side; The losing side; Fine-tuning the strategy directly using the DPO loss function : ； in, for For a policy network with parameters θ, For reference strategy, For preference-based datasets (x, , The mathematical expectation of the distribution. It is the Sigmoid activation function. Hyperparameters used to control the degree of deviation.

Images