Intelligent agent facing complex game decision-making scene and construction method thereof

Abstract

The invention provides a complex game decision scene-oriented agent and a construction method thereof, and aims to solve the problem of weak reasoning ability of an existing large language model agent in a complex game decision environment and improve the performance of the agent. The intelligent agent comprises a state adapter, a reasoning decision unit, an action adapter and an autonomous evolution unit; the state adapter is used for extracting original state information from the complex game decision-making environment and arranging the original state information into structured text observation; the reasoning decision-making unit adopts a three-layer closed-loop structure of a planner, an actuator and a verifier to decouple high-layer strategic planning and low-layer action execution, and adopts the verifier to perform self-error correction; the action adapter converts the standardized executable action into an operation instruction which can be recognized in the complex game decision-making environment, and executes the operation instruction in the complex game decision-making environment; and the autonomous evolution unit screens out high-score data by performing quantitative evaluation on the standardized executable actions, and supervises and finely adjusts the reasoning decision unit according to the high-score data.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an intelligent agent for complex game decision-making scenarios and its construction method. Background Technology

[0002] Game theory decision-making is a core technology for handling the mutual influence and competition among multiple decision-makers. By constructing mathematical models that include participants, policy space, and payoff functions, it provides collaborative and competitive decision support for intelligent agents and is widely used in multi-agent systems, resource allocation, and other fields. With the development of Large Language Models (LLM), Large Language Model Agents (LLM-Agents) integrate the generalization of language understanding with the goal-driven characteristics of reinforcement learning, becoming a key carrier for decision-making in complex tasks. They achieve autonomous decision-making through a perception-thinking-action closed loop.

[0003] However, existing complex game decision-making agents suffer from several technical shortcomings: First, they lack a hierarchical decision-making mechanism, mixing high-level strategies with low-level tactics, resulting in heavy reasoning burdens, chaotic strategies, and difficulty in balancing global planning with local operations; second, they lack self-correction and self-evolution capabilities, relying on static training data or fixed prompt templates, and are unable to continuously learn and improve from execution results; third, the quality of training data is uncontrollable, lacking an effective sample selection mechanism, resulting in a low proportion of high-value samples, leading to poor training efficiency and generalization ability; fourth, they lack decision consistency and interpretability, with strategies and tactics not explicitly decoupled, making them prone to decision drift, and lacking a verification module to support rationality assessment.

[0004] In existing technologies, COA-GPT generates military action plans through a large language model, but its action space is simplified and does not involve complex strategies; TextStarCraftII constructs a text-based game environment, but its interface design is non-standardized, highly coupled with specific methods, and its action space is constrained; SwarmBrain adopts a layered architecture, but its underlying state machine design is complex, limiting the decision space and cross-scenario transfer capabilities. None of these technologies solve the aforementioned core defects and are difficult to meet the high requirements of complex game scenarios. Summary of the Invention

[0005] This invention provides an intelligent agent and its construction method for complex game decision-making scenarios, aiming to solve the problem of weak reasoning ability of existing large language model intelligent agents in complex game decision-making environments and improve the performance of intelligent agents.

[0006] The intelligent agent for complex game decision-making scenarios according to embodiments of the present invention includes: a state adapter, a reasoning decision unit, an action adapter, and an autonomous evolution unit; the reasoning decision unit includes a planner, a plan verifier, an executor, and an execution verifier; The state adapter is used to extract raw state information from a complex game decision-making environment and convert the raw state information into structured text observations that can be understood by a large language model and transmit them to the planner. The planner is a large language model. Based on the input structured text observations, the planner outputs high-level natural language strategic instructions to the planning validator. The planning validator performs compliance checks on the input high-level natural language strategic instructions according to a preset rule base. If the check fails, the planner iteratively corrects the high-level natural language strategic instructions and inputs the corrected instructions into the executor. The executor generates standardized executable actions based on the input high-level natural language strategic instructions and transmits them to the execution validator. The execution validator performs legality checks on the input standardized executable actions. If the check fails, the executor iteratively corrects the standardized executable actions and sends the corrected actions to the action adapter and the autonomous evolution unit. The action adapter is used to convert the input standardized executable actions into operation instructions that can be recognized by the complex game decision-making environment, and to execute the operation instructions in the complex game decision-making environment; The autonomous evolution unit is used to quantitatively evaluate the standardized executable actions input, filter out high-scoring data, and perform supervised fine-tuning of the reasoning decision unit based on the high-scoring data.

[0007] According to some embodiments of the present invention, the state adapter is used to sort units in a complex game decision-making environment by spatial location using a proximity sorting algorithm, and to aggregate units performing similar tasks into a single description using a unit aggregation algorithm.

[0008] According to some embodiments of the present invention, the planner uses a structured prompt template that embeds game rules and expert knowledge to generate the high-level natural language strategic instructions.

[0009] According to some embodiments of the present invention, the executor has built-in priority rules.

[0010] According to some embodiments of the present invention, the execution verifier is used to verify the syntax format of standardized executable actions, the existence of target units, the sufficiency of resources, whether the action is in a preset skill list, and whether there are conflicts between instructions.

[0011] According to some embodiments of the present invention, the autonomous evolutionary unit is used to quantify and score each action in the decision trajectory data using a discount gain function; The formula for calculating the discount gain function is as follows: in, For measuring sets; Indicates the index value. This represents the metric, where t represents time. Indicates the measurement step size. Indicates the discount factor. Indicates the observation at time t. This indicates the observation at time t+k.

[0012] According to some embodiments of the present invention, the autonomous evolutionary unit adopts a full-parameter fine-tuning method, the optimizer is AdamW, and the learning rate is set to 5e. -5 .

[0013] The agent construction method for complex game decision-making scenarios according to embodiments of the present invention includes: The state adapter extracts raw state information from complex game decision-making environments and converts the raw state information into structured text observations that can be understood by large language models; Based on the structured text observations, the planner outputs high-level natural language strategic instructions; The planning validator performs compliance verification on the high-level natural language strategic instructions based on a preset rule base. If the verification fails, the planner is driven to iteratively revise the high-level natural language strategic instructions. The executor generates standardized executable actions based on the verified high-level natural language strategic instructions; The execution verifier performs a legality check on the standardized executable action. If the check fails, the executioner is driven to iteratively correct the standardized executable action. The action adapter converts the validated standardized executable actions into operation instructions that can be recognized by the complex game decision-making environment, and executes the operation instructions in the complex game decision-making environment; The autonomous evolution unit performs quantitative evaluation on the standardized executable actions that have passed the verification, selects high-scoring data, and performs supervised fine-tuning on the planner, the plan verifier, the executor, and the execution verifier based on the high-scoring data.

[0014] According to some embodiments of the present invention, the step of converting the original state information into structured text observations that can be understood by a large language model includes: The nearest neighbor sorting algorithm is used to sort units in a complex game decision-making environment according to their spatial location, and the unit aggregation algorithm is used to aggregate units that perform similar tasks into a single description. The planner outputs high-level natural language strategy instructions based on the structured text observations, including: The planner uses a structured prompt template that embeds game rules and expert knowledge to generate the high-level natural language strategic instructions.

[0015] According to some embodiments of the present invention, the autonomous evolutionary unit is used to quantify and score each action in the decision trajectory data using a discount gain function; The formula for calculating the discount gain function is as follows: in, For measuring sets; Indicates the index value. This represents the metric, where t represents time. Indicates the measurement step size. Indicates the discount factor. Indicates the observation at time t. This indicates the observation at time t+k.

[0016] By employing the embodiments of this invention, strategic planning and tactical execution are explicitly decoupled, reducing the burden of model inference, avoiding interference from multi-level tasks, and improving decision consistency and response speed. Dual validators verify the strategy and actions respectively, and multi-round self-correction is achieved through structured feedback, significantly reducing invalid instructions and execution interruptions, and improving system stability. The closed-loop self-evolution mechanism enables the agent to continuously learn from its own experience, and its performance dynamically improves with the number of interactions. High-value samples and rejection of sampling-based fine-tuning reduce noise interference, achieving better training results with less data and reducing resource consumption.

[0017] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of the embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. In the drawings: Figure 1 This is a schematic diagram of the intelligent agent structure according to an embodiment of the present invention. Detailed Implementation

[0019] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art. Furthermore, in some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0020] Game-theoretic decision making is a class of decision modeling and optimization methods based on game theory, primarily used to handle situations where multiple agents have mutual influence and competition relationships. This method describes the behavioral selection mechanisms of different agents under conditions of incomplete information or conflicting objectives by constructing a mathematical model that includes a set of participants, a strategy space, and a payoff function.

[0021] Under the assumption of rational decision-making, the system achieves optimal response strategies for each participant by solving equilibrium solutions in the form of Nash equilibrium, Bayesian equilibrium, or evolutionarily stable strategy (ESS). Game theory decision-making is widely used in multi-agent systems, distributed control, resource allocation, and adversarial learning. It can effectively characterize the mechanism of "influence of others' behavior," providing agents with interpretable cooperative and competitive decision-making strategies, thereby improving the stability and optimality of the overall system.

[0022] Large Language Models (LLMs) are a general-purpose language modeling technique based on deep learning. Their core structure is a Transformer Network, which models the contextual dependencies of sequential data through an attention mechanism. The model is pre-trained on a large-scale corpus using autoregressive or autoencoder methods to learn the statistical patterns and semantic distributions of language, thereby gaining powerful language understanding and generation capabilities.

[0023] LLM achieves cross-task semantic transfer and knowledge generalization through parameter scaling (scaling law) and multi-stage training (pre-training and fine-tuning), enabling it to model conditional probability distributions under the given input context. It generates logically coherent and semantically consistent natural language output. As a core foundational module of general intelligence, the large language model can be integrated into systems such as decision-making, reasoning, and planning, providing high-level semantic understanding and reasoning support for complex tasks.

[0024] Reinforcement Learning (RL) is a machine learning paradigm that optimizes policies through interaction with the environment. Its basic structure consists of an agent and an environment. At each time step, the agent observes the current state, selects an action, and the environment provides feedback with a new state and reward. The agent then updates its policy accordingly to maximize the expected long-term cumulative reward. .

[0025] Reinforcement learning algorithms can be categorized into value-based, policy-based, and actor-critic methods (a combination of both). Through exploration-exploitation trade-off mechanisms, agents can achieve adaptive optimization in nondeterministic environments. Reinforcement learning demonstrates powerful self-learning and self-evolution capabilities in areas such as robot control, path planning, dynamic resource allocation, and game theory decision-making.

[0026] The Large Language Model Agent (LLM-Agent) is an autonomous intelligent system that combines language reasoning ability with environmental interaction ability. This system uses a large language model as its core reasoning and decision-making engine, and achieves closed-loop control of perception-reasoning-action through external tools, memory modules, and an action interface.

[0027] LLM-Agents can perform operations such as decision planning, information retrieval, and policy optimization within a task space described by natural language. Through mechanisms such as Chain-of-Thought (CoT), Reflection, and Reinforcement Learning (RLHF or RLAIF), the agent can continuously improve its behavioral strategies, achieving goal-driven and self-correcting behavior. Unlike traditional static language models, LLM-Agents can not only generate language but also reason and execute based on it; they combine the generalization of language understanding with the goal-driven characteristics of reinforcement learning, making them a key intermediate form on the path to Artificial General Intelligence (AGI).

[0028] The core of this invention lies in constructing an intelligent agent capable of interpretable hierarchical decision-making, self-correction, and evolution based on the pre-trained knowledge and contextual reasoning ability of a large language model, thus demonstrating good decision-making ability in complex game decision-making environments.

[0029] Reference Figure 1 As shown in the figure, the intelligent agent for complex game decision-making scenarios in this embodiment of the invention includes: a state adapter, a reasoning decision unit, an action adapter, and an autonomous evolution unit; the reasoning decision unit includes a planner, a plan verifier, an executor, and an execution verifier; The state adapter is used to extract raw state information from a complex game decision-making environment and convert the raw state information into structured text observations that can be understood by a large language model and transmit them to the planner. The planner is a large language model. Based on the input structured text observations, the planner outputs high-level natural language strategic instructions to the planning validator. The planning validator performs compliance checks on the input high-level natural language strategic instructions according to a preset rule base. If the check fails, the planner iteratively corrects the high-level natural language strategic instructions and inputs the corrected instructions into the executor. The executor generates standardized executable actions based on the input high-level natural language strategic instructions and transmits them to the execution validator. The execution validator performs legality checks on the input standardized executable actions. If the check fails, the executor iteratively corrects the standardized executable actions and sends the corrected actions to the action adapter and the autonomous evolution unit. The action adapter is used to convert the input standardized executable actions into operation instructions that can be recognized by the complex game decision-making environment, and to execute the operation instructions in the complex game decision-making environment; The autonomous evolution unit is used to quantitatively evaluate the standardized executable actions input, filter out high-scoring data, and perform supervised fine-tuning of the reasoning decision unit based on the high-scoring data.

[0030] Based on the above embodiments, further variant embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in each variant embodiment.

[0031] By employing the embodiments of this invention, strategic planning and tactical execution are explicitly decoupled, reducing the burden of model inference, avoiding interference from multi-level tasks, and improving decision consistency and response speed. Dual validators verify the strategy and actions respectively, and multi-round self-correction is achieved through structured feedback, significantly reducing invalid instructions and execution interruptions, and improving system stability. The closed-loop self-evolution mechanism enables the agent to continuously learn from its own experience, and its performance dynamically improves with the number of interactions. High-value samples and rejection of sampling-based fine-tuning reduce noise interference, achieving better training results with less data and reducing resource consumption.

[0032] According to some embodiments of the present invention, the state adapter is used to sort units in a complex game decision-making environment by spatial location using a proximity sorting algorithm, and to aggregate units performing similar tasks into a single description using a unit aggregation algorithm. Here, a unit can be understood as the smallest executor in the game decision-making scenario.

[0033] According to some embodiments of the present invention, the planner uses a structured prompt template that embeds game rules and expert knowledge to generate the high-level natural language strategic instructions.

[0034] According to some embodiments of the present invention, the executor has built-in priority rules.

[0035] According to some embodiments of the present invention, the execution verifier is used to verify the syntax format of standardized executable actions, the existence of target units, the sufficiency of resources, whether the action is in a preset skill list, and whether there are conflicts between instructions.

[0036] According to some embodiments of the present invention, the autonomous evolutionary unit is used to quantify and score each action in the decision trajectory data using a discount gain function; The formula for calculating the discount gain function is as follows: in, For measuring sets; Indicates the index value. This represents the metric, where t represents time. Indicates the measurement step size. Indicates the discount factor. Indicates the observation at time t. This indicates the observation at time t+k.

[0037] According to some embodiments of the present invention, the autonomous evolutionary unit adopts a full-parameter fine-tuning method, the optimizer is AdamW, and the learning rate is set to 5e. -5 .

[0038] The following describes in detail, with reference to the accompanying drawings, an intelligent agent for complex game decision-making scenarios according to an embodiment of the present invention. It is to be understood that the following description is merely exemplary and should not be construed as a specific limitation of the present invention.

[0039] Reference Figure 1 As shown, the intelligent agent for complex game decision-making scenarios in this embodiment of the invention includes a state adapter, a reasoning decision unit, an action adapter, and an autonomous evolution unit. Each module works together to achieve a complete closed loop from environmental perception to decision execution and then to self-optimization.

[0040] The state adapter extracts state information from the decision-making environment and constructs a state text. The agent uses this state text as input and calls modules such as the planner to perform logical reasoning and decision-making. First, the planner performs high-level strategic planning, obtaining a natural language planning scheme. The planning validator evaluates and verifies this scheme; if it fails, feedback is provided and the planner improves it; if it passes, it is handed over to the executor for low-level action execution. This low-level action is in JSON format and is validated by the execution validator. If it fails, feedback is provided and the executor improves it; if it passes, the action adapter performs the actual execution. The decision-making chain in the above closed-loop decision-making process is organized into formatted data, scored using a reinforcement learning-inspired scoring module, and the filtered data is used for batch aggregation and supervised fine-tuning of the agent model.

[0041] The state adapter is responsible for extracting raw state information from the complex game environment and converting it into structured text observations that can be understood by a large language model. In implementation, the module first reads state information such as units, buildings, resources, and visible enemy units through the environment interface. Then, it optimizes the input structure using a proximity-based unit ordering algorithm and a unit aggregation algorithm. The former, based on the nearest neighbor heuristic, sorts units by spatial location, enabling the agent to explicitly reason about spatial relationships; the latter aggregates worker units performing similar tasks into a single description, thereby reducing invalid information input and lowering the cognitive load on the model. The final output text observations include elements such as resource status, friendly unit and building status, technology progress, visible enemy information, and historical actions.

[0042] The reasoning and decision-making unit is the core decision-making module of the intelligent agent, including the planner, plan verifier, executor, and execution verifier, realizing a hierarchical decision-making process of "strategy generation - compliance verification - action conversion - legality verification".

[0043] The planner, implemented using a large language model, is a high-level strategy generation module for the intelligent agent. Its task is to generate high-level natural language strategic instructions (such as "build barracks", "produce attack units", "attack the enemy's main base") based on text observation information provided by the state adaptation module.

[0044] The planner employs a structured prompt template, embedding game rules and expert knowledge to ensure the logic and constraint of strategic decisions. The planner infers the next strategic action based on current resources, production status, enemy position, and historical actions.

[0045] To avoid generating unreasonable or rule-breaking instructions, each instruction output by the planner will be sent to the planner validator for compliance checks, enabling self-reflection and improvement within the decision-making chain.

[0046] The planning validator is responsible for verifying the rules and feasibility of the high-level instructions output by the planner. This module is also implemented using a large language model and has a built-in dynamic expert rule base. Different rule sets correspond to different races (Terran, Zerg, Protoss), such as resource constraints, building prerequisites, and unit production limits. The validator compares the planner's output with the rule base one by one, automatically identifying logical errors (such as insufficient resources, redundant construction, and violation of timing), and generating structured error feedback. The planner iteratively corrects itself based on the feedback until the instruction passes verification or the maximum number of verifications is reached. This "generate-verify-reflect" loop mechanism significantly reduces erroneous decisions and reasoning illusions, enabling the agent to form a highly reliable decision chain in complex strategic spaces.

[0047] The executor is responsible for translating high-level natural language strategic instructions into low-level action sequences executable by the environment. This module, also based on a large language model, maps natural language tasks to a standardized JSON action list through a templated prompt structure. The system first parses the planner's output, identifies the unit type and target object corresponding to the task, and then generates a set of execution instructions containing parameters such as action type, unit ID, target location, or target unit (e.g., "action": "GATEWAYTRAIN_STALKER" or "action": "ATTACK_ATTACK"). The executor internally sets priority rules to ensure that critical tasks (such as production or defense) are completed first when resources are insufficient. Furthermore, to prevent syntax errors or resource overruns, the actions generated by the executor are further reviewed by an execution verifier.

[0048] The execution validator employs a combined programmatic and rule-based validation mechanism to check the legality of the action sequences output by the executor. Specifically, the validator checks whether the syntax of the JSON action is correct, whether the target unit exists, whether resources are sufficient, whether the action is in the allowed skill list, and whether there are any conflicts between instructions. If an error is found, the validator generates structured feedback information to drive the executor to automatically correct itself, iterating up to three times. Through this process, the agent can complete action-level self-correction without human intervention, effectively improving the stability of action execution and resource utilization efficiency.

[0049] The data collection and scoring module is responsible for automatically collecting the decision trajectory of the agent during the game and quantifying and scoring each action.

[0050] To select high-quality samples, the module introduces a discount gain function based on reinforcement learning to comprehensively evaluate the short-term and long-term benefits of each action. The scoring function calculation formula is as follows: in For measuring sets (including minerals, gases, unit quantities, building quantities, etc.). Indicates time The index value. This represents the metric, where t represents time. Indicates the measurement step size. Indicates the discount factor. Indicates the observation at time t. This indicates the observation at time t+k.

[0051] Through this mechanism, the system can assess the contribution of actions to strategic development based on state changes, thereby filtering out decision data that can bring significant benefits.

[0052] The supervised fine-tuning module continuously optimizes the core modules of the agent (planner, executor, and plan validator) using high-scoring samples, forming a self-evolutionary closed loop. Specifically, this module uses the "final output that passes verification" as the training target and masks the intermediate error correction steps, which is equivalent to implementing rejection sampling at the data level.

[0053] In the fine-tuning of the planner and executor, input pairs <observed state, verified high-level instruction> and <observed state + high-level instruction, verified action sequence> are used respectively; in the fine-tuning of the verifier, a contrastive learning method is adopted to construct "incorrect command + feedback" and "correct command + feedback" pairs to enhance its ability to distinguish boundaries.

[0054] Training employed a full-parameter fine-tuning approach, with the optimizer based on AdamW and a learning rate set to 5e. -5 The training batch size was 32, using cosine decay scheduling and a 0.1 warm-up ratio. After multiple iterations, the agent achieved significant improvements in strategic planning, action generation, and verification efficiency.

[0055] In actual operation, the intelligent agent of this invention initiates a decision-making loop at fixed time intervals or resource triggering conditions. In each loop, the state adaptation module generates the latest observations, the planner formulates high-level decisions, the executor completes the action transition, and the execution results are verified by a dual verifier.

[0056] All game trajectories and verification data are recorded in real time and fed back to the data collection module, providing training material for the next round of supervised fine-tuning. Through this closed-loop mechanism, the agent possesses the ability to continuously self-correct and improve performance, realizing the transformation from static strategy to dynamic learning.

[0057] The action adapter is responsible for extracting raw state information from complex game environments and converting it into structured text observations and standardized action interfaces that can be understood by large language models.

[0058] In terms of implementation, this invention designs a low-level action protocol that is friendly to large language models, as shown below. The action interface is defined in a unified JSON format and supports parameters such as instruction type, target unit, and target location to accurately generate executable operations.

[0059] This invention employs a three-layer closed-loop structure of "Planner – Executor – Verifier," decoupling high-level strategic planning from low-level action execution. Each layer operates independently and interacts through a unified data interface, enabling collaborative division of labor between strategic decision-making, tactical translation, and self-correction at the verification layer, thus resolving the issues of strategy confusion and action conflict in existing technologies. Independent verifier modules are set up at both the planning and execution layers, employing rule-based verification and structured feedback to check syntactic correctness, resource feasibility, and logical consistency. The verifier output includes error types and correction suggestions, driving iterative correction by upstream modules, achieving automatic reflection and multi-round self-correction. When verification fails, rejection sampling optimization is automatically executed, retaining only verified instructions or actions for execution, significantly reducing the invalid instruction rate. A closed-loop self-evolving system is constructed, encompassing "game interaction → trajectory acquisition → action scoring → sample selection → supervised fine-tuning → model redeployment." Each action is quantitatively evaluated using a reinforcement learning-inspired discount gain scoring function, selecting high-value samples for retraining. A rejection-sampling supervised fine-tuning strategy is employed, using only validated final outputs for training to continuously improve the model's policy quality and adaptability. A temporal discount cumulative gain function based on multiple metrics (unit resource value, supply blockage rate, resource utilization, etc.) is designed to quantify action quality and standardize it within a time window, retaining only data samples above a threshold. Compared to random sampling or traditional reinforcement learning, this mechanism significantly reduces training costs and improves data efficiency and model performance stability.

[0060] This invention effectively addresses the lack of hierarchical decision-making mechanisms in complex game scenarios for existing large language model agents by constructing a three-layer hierarchical decision-making architecture consisting of a planner, executor, and verifier. This structure achieves explicit decoupling between strategic planning and tactical execution: the planner focuses on generating high-level semantic strategies, the executor is responsible for translating natural language instructions into specific actions, and the dual verifiers perform structured verification of the rationality of the strategy and the legality of the actions, respectively. This division of labor and collaboration significantly reduces the model's reasoning burden, avoids logical confusion and illusory output caused by mixing multiple levels of tasks, thereby improving the consistency, interpretability, and execution efficiency of decisions.

[0061] Furthermore, this invention introduces a self-correction and self-evolution mechanism based on closed-loop feedback, specifically overcoming the shortcomings of existing methods, such as lack of continuous learning ability and low efficiency in utilizing high-quality data. Through structured error feedback provided by the validator, the agent can achieve multiple rounds of self-correction in a single decision; simultaneously, combined with a reinforcement learning-inspired discounted gain scoring function, the system can automatically filter high-value samples from massive interaction trajectories, using only validated high-quality decisions as supervisory signals for fine-tuning. This mechanism not only significantly reduces the rate of invalid actions and training noise, but also enables the model performance to dynamically improve with experience accumulation, significantly enhancing the agent's adaptability and long-term competitiveness in complex, dynamic game environments.

[0062] The following uses StarCraft as an example to explain in detail the specific implementation process of this invention: (a) Environmental configuration The game environment is a Terran vs. Zerg scenario from StarCraft 2. The environment interface supports the extraction of raw states such as unit status, resource data, building information, and battle results, while the action interface supports the parsing and execution of JSON format commands.

[0063] (II) Module Deployment and Operation State Adapter Deployment: Configure a proximity sorting algorithm to sort friendly units by their spatial distance from the main base; configure a unit aggregation algorithm to aggregate 12 mining SCVs (Terran worker units) into a single description of "12 mining SCVs". An example of extracted structured text observation is: "Resources: 800 minerals, 300 gas; Friendly: 1 main base, 2 barracks, 12 SCVs (mining), 8 Marines; Technology; Enemy: 1 visible Zerg base, 6 Zerglings, 50 coordinates from our main base; Historical Action: 1 barracks built in the previous round."

[0064] Deployment of reasoning and decision-making units: Planner: Using a structured prompt template "Based on the current resources {resource status}, friendly status {friendly information}, and enemy status {enemy information}, following the Terran building rules, output the next round of high-level strategic commands", generating the command "Produce 5 Marines to prepare to defend against enemy Zergling raids"; Planning Verifier: Calls the Terran rule base to verify the prerequisites for "producing Marines" (Barracks has been built, which meets the requirements) and resource requirements (500 minerals, currently 800, which is sufficient). Verification passes. Executor: Generates a JSON-formatted action based on high-level instructions: [{"action":"BARRACKSTRAIN_MARINE","units":[7,8,9,10,11]}] (Barracks training machine gunners, specifying 5 barracks unit IDs); Execution verifier: Verify that the JSON format is correct, the target unit (Barracks 7-11) exists, resources are sufficient (500 minerals are sufficient), the action is in the Terran skill list, there are no command conflicts, and the verification passes.

[0065] Action Adapter Deployment: Converts JSON actions into operation commands recognizable by the StarCraft 2 environment, driving barracks to perform Marine production operations.

[0066] Deployment of autonomous evolutionary units: Rating: The rating of this action is calculated using a discount gain function. Assuming that the machine gunner successfully defends against the enemy's attack within the next 20 steps, the unit quantity index is improved, and the resource consumption is reasonable, the rating is 8.5 (out of 10), which is higher than the threshold of 7.0 and is judged as a high-rated sample. Fine-tuning: The input pair of this sample <the above structured text observation, "Produce 5 Marines to prepare for defense against enemy dog ​​attacks"> is used for planner fine-tuning, and the input pair <structured text observation + high-level command, the above JSON action> is used for executor fine-tuning. At the same time, a sample pair of "error command (produce 10 Marines, insufficient resources) + feedback (resources can only support 5)" is constructed for comparative learning of the planner validator.

[0067] (III) Iterative Evolution Each time the agent completes a decision-making cycle (approximately 30-second intervals), it automatically records the decision trajectory and verification data. The autonomous evolution unit initiates supervised fine-tuning every 100 high-scoring samples. After 10 rounds of iterative fine-tuning, the probability of the planner generating unreasonable strategies decreases by 60%, the executor's action error rate decreases by 75%, and the agent's strategy level in areas such as defense and counterattack, and resource utilization is significantly improved.

[0068] This invention, through its hierarchical decision-making structure and closed-loop evolutionary mechanism, possesses the following core advantages: Clear decision-making hierarchy and high efficiency: Explicitly decouple strategic planning from tactical execution, allowing each module to focus on its own task, reducing the burden of model reasoning, avoiding interference from multiple levels of tasks, and improving decision consistency and response speed; Highly executable and with a low error rate: The dual validator verifies the strategy and actions separately, and achieves multi-round self-correction through structured feedback, significantly reducing invalid instructions and execution interruptions, and improving system stability; It has the ability to continuously evolve: the closed-loop self-evolution mechanism realizes a virtuous cycle of "interaction - collection - scoring - screening - fine-tuning", and the agent can continuously learn from its own experience, and its performance dynamically improves with the number of interactions; Low training cost and high data utilization: The discount gain function accurately selects high-value samples, rejects sampling-based fine-tuning to reduce noise interference, achieves better training results with less data, and reduces resource consumption. It should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0069] The agent construction method for complex game decision-making scenarios according to embodiments of the present invention includes: The state adapter extracts raw state information from complex game decision-making environments and converts the raw state information into structured text observations that can be understood by large language models; Based on the structured text observations, the planner outputs high-level natural language strategic instructions; The planning validator performs compliance verification on the high-level natural language strategic instructions based on a preset rule base. If the verification fails, the planner is driven to iteratively revise the high-level natural language strategic instructions. The executor generates standardized executable actions based on the verified high-level natural language strategic instructions; The execution verifier performs a legality check on the standardized executable action. If the check fails, the executioner is driven to iteratively correct the standardized executable action. The action adapter converts the validated standardized executable actions into operation instructions that can be recognized by the complex game decision-making environment, and executes the operation instructions in the complex game decision-making environment; The autonomous evolution unit performs quantitative evaluation on the standardized executable actions that have passed the verification, selects high-scoring data, and performs supervised fine-tuning on the planner, the plan verifier, the executor, and the execution verifier based on the high-scoring data.

[0070] Based on the above embodiments, further variant embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in each variant embodiment.

[0071] According to some embodiments of the present invention, the step of converting the original state information into structured text observations that can be understood by a large language model includes: The nearest neighbor sorting algorithm is used to sort units in a complex game decision-making environment according to their spatial location, and the unit aggregation algorithm is used to aggregate units that perform similar tasks into a single description.

[0072] According to some embodiments of the present invention, the planner outputs high-level natural language strategic instructions based on the structured text observations, including: The planner uses a structured prompt template that embeds game rules and expert knowledge to generate the high-level natural language strategic instructions.

[0073] According to some embodiments of the present invention, the autonomous evolutionary unit is used to quantify and score each action in the decision trajectory data using a discount gain function; The formula for calculating the discount gain function is as follows: in, For measuring sets; Indicates the index value. This represents the metric, where t represents time. Indicates the measurement step size. Indicates the discount factor. Indicates the observation at time t. This indicates the observation at time t+k.

[0074] According to some embodiments of the present invention, the actuator generates standardized executable actions based on verified high-level natural language strategic instructions, including: High-level natural language strategic instructions are processed sequentially based on task priority.

[0075] According to some embodiments of the present invention, the execution verifier performs a legality check on the standardized executable action, including: The execution verifier checks the syntax of standardized executable actions, whether the target unit exists, whether resources are sufficient, whether the action is in the preset skill list, and whether there are conflicts between instructions.

[0076] It should be noted that the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the present invention and form different embodiments. The specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. For example, in the claims, any of the claimed embodiments can be used in any combination.

Claims

1. An intelligent agent for complex game decision-making scenarios, characterized in that, include: State adapter, reasoning and decision-making unit, action adapter, autonomous evolution unit; The reasoning and decision-making unit includes a planner, a plan validator, an executor, and an execution validator; The state adapter is used to extract raw state information from a complex game decision-making environment and convert the raw state information into structured text observations that can be understood by a large language model and transmit them to the planner. The planner is a large language model. Based on the input structured text observations, the planner outputs high-level natural language strategic instructions to the planning validator. The planning validator performs compliance checks on the input high-level natural language strategic instructions according to a preset rule base. If the check fails, the planner iteratively corrects the high-level natural language strategic instructions and inputs the corrected instructions into the executor. The executor generates standardized executable actions based on the input high-level natural language strategic instructions and transmits them to the execution validator. The execution validator performs legality checks on the input standardized executable actions. If the check fails, the executor iteratively corrects the standardized executable actions and sends the corrected actions to the action adapter and the autonomous evolution unit. The action adapter is used to convert the input standardized executable actions into operation instructions that can be recognized by the complex game decision-making environment, and to execute the operation instructions in the complex game decision-making environment; The autonomous evolution unit is used to quantitatively evaluate the standardized executable actions input, filter out high-scoring data, and perform supervised fine-tuning of the reasoning decision unit based on the high-scoring data.

2. The intelligent agent for complex game decision-making scenarios according to claim 1, characterized in that, The state adapter is used to sort units in a complex game decision-making environment by spatial location using a proximity sorting algorithm, and to aggregate units performing similar tasks into a single description using a unit aggregation algorithm.

3. The intelligent agent for complex game decision-making scenarios according to claim 1, characterized in that, The planner uses a structured prompt template that embeds game rules and expert knowledge to generate the high-level natural language strategic instructions.

4. The intelligent agent for complex game decision-making scenarios according to claim 1, characterized in that, The executor has built-in priority rules.

5. The intelligent agent for complex game decision-making scenarios according to claim 1, characterized in that, The execution verifier is used to verify the syntax format of standardized executable actions, whether the target unit exists, whether resources are sufficient, whether the action is in the preset skill list, and whether there are conflicts between instructions.

6. The intelligent agent for complex game decision-making scenarios according to claim 1, characterized in that, The autonomous evolutionary unit is used to quantify and score each action in the decision trajectory data using a discount gain function. The formula for calculating the discount gain function is as follows: in, For measuring sets; Indicates the index value. This represents the metric, where t represents time. Indicates the measurement step size. Indicates the discount factor. Indicates the observation at time t. This indicates the observation at time t+k.

7. The intelligent agent for complex game decision-making scenarios according to claim 1, characterized in that, The self-evolving unit employs a full-parameter fine-tuning method, with AdamW selected as the optimizer and a learning rate set to 5e. −5 .

8. A method for constructing intelligent agents for complex game decision-making scenarios, characterized in that, include: The state adapter extracts raw state information from complex game decision-making environments and converts the raw state information into structured text observations that can be understood by large language models; Based on the structured text observations, the planner outputs high-level natural language strategic instructions; The planning validator performs compliance verification on the high-level natural language strategic instructions based on a preset rule base. If the verification fails, the planner is driven to iteratively revise the high-level natural language strategic instructions. The executor generates standardized executable actions based on the verified high-level natural language strategic instructions; The execution verifier performs a legality check on the standardized executable action. If the check fails, the executioner is driven to iteratively correct the standardized executable action. The action adapter converts the validated standardized executable actions into operation instructions that can be recognized by the complex game decision-making environment, and executes the operation instructions in the complex game decision-making environment; The autonomous evolution unit performs quantitative evaluation on the standardized executable actions that have passed the verification, selects high-scoring data, and performs supervised fine-tuning on the planner, the plan verifier, the executor, and the execution verifier based on the high-scoring data.

9. The method for constructing an intelligent agent for complex game decision-making scenarios according to claim 8, characterized in that, The process of converting the original state information into structured text observations that can be understood by a large language model includes: The nearest neighbor sorting algorithm is used to sort units in a complex game decision-making environment according to their spatial location, and the unit aggregation algorithm is used to aggregate units that perform similar tasks into a single description. The planner outputs high-level natural language strategy instructions based on the structured text observations, including: The planner uses a structured prompt template that embeds game rules and expert knowledge to generate the high-level natural language strategic instructions.

10. The method for constructing an intelligent agent for complex game decision-making scenarios according to claim 8, characterized in that, The autonomous evolutionary unit is used to quantify and score each action in the decision trajectory data using a discount gain function. The formula for calculating the discount gain function is as follows: in, For measuring sets; Indicates the index value. This represents the metric, where t represents time. Indicates the measurement step size. Indicates the discount factor. Indicates the observation at time t. This indicates the observation at time t+k.

Images