A large language model dynamic evaluation and optimization method based on game confrontation

Abstract

The application relates to a large language model dynamic evaluation and optimization method based on game confrontation, which comprises the following steps: obtaining data of a game environment and providing a state input interface and an action execution interface to a large language model agent; analyzing the game environment to obtain a game feature vector; matching a prompt word template, generating a prompt word; inputting the prompt word into the large language model agent, obtaining an action after obtaining a model response and analysis; forming a global battle log database based on the action; obtaining a multi-dimensional evaluation result based on the global battle log database; and forming an evaluation, optimization and re-evaluation closed loop based on the multi-dimensional evaluation result and the global battle log database. The large language model dynamic evaluation and optimization method based on game confrontation can solve the problems that the evaluation process cannot reflect the decision-making performance of the model, the evaluation index lacks linkage investigation, the available range of the evaluation result is small, and the use of the evaluation data is small.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence testing and evaluation technology, specifically to a method for dynamic evaluation and optimization of large language models based on game-based adversarial competition. Background Technology

[0002] As the application of large language models in question-answering generation, code processing, and reasoning tasks continues to expand, how to objectively evaluate their performance in continuous decision-making scenarios has become a fundamental issue in artificial intelligence testing and model iteration. Existing evaluation methods mainly rely on static question sets, offline question answering, or single-round reasoning tasks. These methods facilitate standardized scoring, but they typically lack unified and stable evaluation methods for a model's ability to understand states in dynamic environments, enforce rules, plan long-term, control resources, and self-correct errors.

[0003] The existing technologies have the following shortcomings: First, the evaluation process is usually detached from the real interactive environment, making it difficult to reflect the model's decision-making performance under continuous state changes; second, the evaluation indicators focus on the correctness of the results, lacking a coordinated assessment of rule understanding, process consistency, response costs, and performance fluctuations; third, the evaluation results mostly remain at the level of scores or accuracy rates, which is not conducive to locating the causes of errors and forming subsequent optimization paths; fourth, there is a lack of a closed-loop mechanism to directly feed the evaluation data back into the prompting strategy and model parameter adjustment, making it difficult to form a reusable and iterative model optimization process.

[0004] Therefore, existing technologies have shortcomings and need to be improved and developed. Summary of the Invention

[0005] The present invention provides a dynamic evaluation and optimization method for a large language model based on game-based adversarial competition, which addresses the following problems in the prior art: First, the evaluation process is usually detached from the real interactive environment, making it difficult to reflect the model's decision-making performance under continuous state changes; second, the evaluation indicators focus on the correctness of the results, lacking a coordinated assessment of rule understanding, process consistency, response costs, and performance fluctuations; third, the evaluation results mostly remain at the level of scores or accuracy rates, which is not conducive to locating the causes of errors and forming subsequent optimization paths; fourth, there is a lack of a closed-loop mechanism to directly feed the evaluation data back into the prompt strategy and model parameter adjustment, making it difficult to form a reusable and iterative model optimization process.

[0006] This invention provides a method for dynamic evaluation and optimization of a large language model based on game-based adversarial competition, comprising:

[0007] Acquire rule data, state data, and win / loss determination rules for at least one game environment, and provide state input interface and action execution interface to at least one large language model agent through a standardized game application programming interface;

[0008] Perform feature quantization analysis on the game environment to obtain the game feature vector. ,in, As a game genre, The logarithm of the state space size is used to represent the state complexity. The average number of legal actions is used to represent the decision branching factor. The time limit for each round is used to indicate time pressure;

[0009] Get the current game status Game history information and the game feature vector Match prompt templates from the prompt template library and generate corresponding prompts under the constraints of the context window. ;

[0010] The prompt word Input the corresponding large language model agent and obtain the model response. and respond to the model Perform action parsing to obtain actions that can be executed by the standardized game application programming interface. ;

[0011] Based on the action Update the game state of the game environment and record the state during the game. Prompt words Model response ,action The results of the battles, reaction time, token usage, and decision-making criteria are used to form a global battle log database.

[0012] Based on the global battle log database, the effectiveness of strategies, rule compliance, resource efficiency and stability are quantitatively evaluated to obtain multi-dimensional evaluation results;

[0013] Based on the multi-dimensional evaluation results and the global battle log database, error type diagnosis is performed, and at least one of the following is executed according to the diagnosis results: prompt word optimization, model optimization, and prompt word template library optimization, so as to form a closed loop of evaluation, optimization, and re-evaluation.

[0014] Furthermore, the game environment includes at least one of board games, card games, and other games with clearly defined rules;

[0015] The board games include at least one of Go, chess, backgammon, and six-card poker; the card games include at least one of Guandan, Dou Dizhu, Kuhn Poker, Leduc Poker, bridge, Goofspiel, and Liar's Dice; and the other games include at least one of Tic-Tac-Toe, Connect Four, and checkers.

[0016] The selection criteria for the game environment include at least one of the following: large language model agent relationships, information structure, and action patterns, wherein the information structure includes perfect information structure and imperfect information structure, and the action patterns include turn-based action patterns and simultaneous action patterns.

[0017] Furthermore, the step of obtaining the current game state Game history information and the game feature vector Match prompt templates from the prompt template library and generate corresponding prompts under the constraints of the context window. ,include:

[0018] Obtain game information including at least the game type, board layout, and current turn state;

[0019] Based on the obtained game information, match the prompt word template from the prompt word template library;

[0020] Current game status After being integrated with game history information, the information is placed into the model's context window, and prompts are generated while ensuring that the context token limit is not exceeded. .

[0021] Furthermore, the prompt word template library is a dynamically evolving template library, based on the game feature vector. Automatically generate initial prompt word template The initial prompt word template It includes at least an objective function for role identity and rules and boundary constraints, wherein the objective function is expressed as follows: ,in, Indicates state, Indicates the action; the rules and boundary constraints are output through hard or soft constraint models, whereby the hard or soft constraints are expressed as: if Then a penalty item will be applied. ,in, Representing state The set of legal actions below;

[0022] For the initial prompt word template Iterative updates, satisfying: ,in, Indicates the first The prompt word template after round of iterations, Indicates the first Interaction logs and diagnostic data generated from round-robin assessments This represents the template optimization function;

[0023] The template optimization function analyzes the model's weaknesses based on the interaction logs and diagnostic data, and adjusts the instruction strength, number of examples, or constraints in the current template to generate subsequent prompt word templates. ;

[0024] The prompt word template library is based on game feature vectors. Indexing with iterative versions and achieving template matching and loading by calculating the similarity between the current game features and templates in the library, including: extracting the current game feature vector for the current game environment. And for each candidate prompt word template in the prompt word template library, obtain the corresponding template feature vector. By calculating the current game feature vector With each of the template feature vectors Template matching is achieved using cosine similarity between the two, wherein the cosine similarity satisfies: ,in, Represents the vector dot product. and These represent the vector magnitudes; the candidate prompt word template with the highest cosine similarity is determined as the target prompt word template, and loaded according to the iterative version corresponding to the target prompt word template.

[0025] Furthermore, the prompt word Input the corresponding large language model agent and obtain the model response. and respond to the model Perform action parsing to obtain actions that can be executed by the standardized game application programming interface. ,include:

[0026] The model response is obtained through rule-matching algorithms or small artificial intelligence models. The natural language descriptions in the code are mapped to legal action instructions that the game engine can execute;

[0027] When the model response is detected to be invalid or semantically ambiguous, the model regeneration mechanism is triggered.

[0028] If the model regeneration result still does not meet the legal action conditions, a preset conservative action is executed; wherein, the conservative action is a preset action that meets the legal action constraints of the current state and can maintain the continuous execution of the game.

[0029] Furthermore, the global battle log database records at least the following data:

[0030] The results of a single game and the points at which a game is decided by a timeout;

[0031] Large language models from recorded states To model response The time difference is used as the reaction time.

[0032] Input prompt token count, output response token count, total token consumption in a single round, and average token consumption per step;

[0033] Dynamically generated complete prompt text, model response text containing inference content, and model response. Rules, strategies, or probabilistic reasoning within;

[0034] Among them, state The prompt indicates the game state at a certain point in the game; Indicates the state Generated input prompts, model response This indicates that the large language model is based on prompt words. The generated output content, actions Indicates the model response The actions that are parsed and executed in the game environment;

[0035] The global battle log database is The quadruple is the basic recording unit used to reproduce the decision chain.

[0036] Furthermore, the multi-dimensional evaluation results are calculated using the following formula: ;in, Let represent the weights corresponding to strategy effectiveness, rule compliance, resource efficiency, and stability, respectively, and satisfy the following: ; The score represents the strategy effectiveness sub-score. This indicates that the rule follows the sub-score. This represents the resource efficiency sub-score. The stability sub-scores are represented, with each sub-score ranging from zero to one hundred.

[0037] Furthermore, the strategy effectiveness sub-score It is determined at least based on win / loss ratio, decision quality score, and planning consistency, among which:

[0038] Win / loss ratio satisfies: ;

[0039] The decision quality score satisfies: Wherein, the LLM decision score represents the actual decision score of the large language model at each decision step, and the optimal decision score represents the decision score of the optimal solution in the corresponding state;

[0040] Planning consistency is satisfied: Among them, the actual decision target represents the immediate target corresponding to each step in the decision sequence, the expected long-term target represents the long-term planning target corresponding to the game stage, and the target deviation threshold represents the threshold for judging whether the actual decision deviates from the long-term target.

[0041] The rules follow sub-scores. It should be determined at least based on compliance rate, misunderstanding rate, and type of violation, among which:

[0042] Compliance rate met: ;

[0043] The misunderstanding rate satisfies: ;

[0044] The types of violations are used as statistical categories to identify the most frequent types of violations after the distribution histogram is generated, thus identifying the weak points in the rule understanding of the large language model.

[0045] The resource efficiency sub-score It is determined at least based on average execution time, average token consumption, and resource usage, where:

[0046] Average time consumption meets: ;

[0047] Average token consumption satisfies: ;

[0048] Resource usage includes at least hardware metrics such as GPU utilization, CPU utilization, and memory usage.

[0049] The stability sub-score Determined at least based on performance variance, robustness, and error mode, where:

[0050] The performance variance satisfies: ,in, Indicates the first The bureau's decision-making quality score This represents the average score of decision-making quality across multiple rounds. Indicates the number of statistical bureaus;

[0051] The anti-interference rate meets the following requirements: ;

[0052] Error patterns are clustered based on historical errors to identify recurring error types.

[0053] Furthermore, the step of diagnosing error types based on the multi-dimensional evaluation results and the global battle log database, and performing at least one of the following optimization methods based on the diagnosis results: prompt word optimization, model fine-tuning optimization, and prompt word template library optimization, to form a closed loop of evaluation, optimization, and re-evaluation, includes:

[0054] Monitor the global battle log database and classify error types into at least one of the following: syntax error, logical error, rule misunderstanding, and strategy mistake;

[0055] Based on the error types that occur in the current game, update the subsequent prompts to optimize real-time prompts;

[0056] The error states and the expected corrected actions are paired and then fed into a large model for optimization.

[0057] Based on the global battle log database, we summarize the prompting strategies for specific games or specific opponents, and update the prompting word template library to optimize the prompting word template library.

[0058] After completing at least one of the following: prompt word optimization, model optimization, and prompt word template library optimization, adversarial evaluation is re-executed using the optimized prompt word template or the optimized large language model to verify the effectiveness of the optimization measures.

[0059] Beneficial effects:

[0060] As can be seen from the above technical solutions, the present invention provides a method for dynamic evaluation and optimization of a large language model based on game-based competition, which has the following beneficial effects:

[0061] 1. By placing the large language model in a game-like adversarial environment with clear state evolution and win / loss determination rules, the evaluation of model capabilities is expanded from static single-round questions to a dynamic continuous decision-making process, thus more closely resembling the model's behavior in real interactive tasks.

[0062] 2. By using a unified state input interface, action execution interface, and action parsing mechanism, different game environments and different large language models can interact within the same framework, thereby forming engineering test conditions that can be compared horizontally and executed repeatedly.

[0063] 3. By recording the state, prompt words, model response and action quadruple, as well as additional information such as time consumption, token consumption, and rule basis, each step of the model's decision can be traced. Therefore, it is possible not only to evaluate the results, but also to locate the specific process that produced the results.

[0064] 4. By breaking down the comprehensive score into four primary dimensions—strategy effectiveness, rule compliance, resource efficiency, and stability—and further refining them into calculable sub-indicators, the model's shortcomings can be quantitatively identified. Therefore, subsequent optimization no longer relies on empirical judgment but has a clear objective.

[0065] 5. By feeding back the error diagnosis results into the prompt word optimization, template library optimization, and model optimization stages, the evaluation data can directly participate in subsequent improvements, thus forming a sustainable iterative capability improvement mechanism, rather than remaining at the level of a one-time evaluation result.

[0066] 6. By introducing resource efficiency and stability dimensions, the judgment of model quality is not limited to the outcome of winning or losing, but also considers response costs and performance fluctuations. Therefore, it is more suitable for comprehensive selection and continuous optimization in engineering deployment scenarios.

[0067] It should be understood that all combinations of the foregoing concepts and the additional concepts described in more detail below can be considered part of the inventive subject matter of this disclosure, provided that such concepts do not contradict each other.

[0068] The foregoing and other aspects, embodiments, and features of the teachings of the present invention will be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the invention, such as features and / or beneficial effects of exemplary embodiments, will become apparent from the following description or may be learned through practice of specific embodiments according to the teachings of the present invention. Attached Figure Description

[0069] The accompanying drawings are not drawn to scale. In the drawings, each identical or nearly identical component shown in the various figures may be denoted by the same reference numeral. For clarity, not every component is labeled in each figure. Embodiments of various aspects of the invention will now be described by way of example and with reference to the accompanying drawings, wherein:

[0070] Figure 1 This is a flowchart illustrating the overall process of a game-based dynamic evaluation and optimization method for a large language model in this application.

[0071] Figure 2 This is a flowchart of step S106 of a game-based adversarial dynamic evaluation and optimization method for a large language model in an embodiment of this application.

[0072] Figure 3 This is a flowchart of step S108 of a game-based adversarial large language model dynamic evaluation and optimization method in an embodiment of this application.

[0073] Figure 4 This is a flowchart of step S112 of a game-based adversarial dynamic evaluation and optimization method for a large language model in an embodiment of this application.

[0074] Figure 5 This is an architecture diagram of a game-based adversarial dynamic evaluation and optimization method for a large language model, as described in an embodiment of this application.

[0075] Figure 6This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art to which this invention pertains.

[0077] The terms "first," "second," and similar words used in the specification and claims of this patent application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular forms of "an," "a," or "the," etc., do not indicate a quantity limitation, but rather indicate the presence of at least one. Terms such as "comprising" or "including" mean that the element or object preceding "comprising" encompasses the features, integrals, steps, operations, elements, and / or components listed following "comprising" or "including," and do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; these relative positional relationships may change accordingly when the absolute position of the described object changes.

[0078] Existing AI testing and model iteration technologies suffer from the following shortcomings: First, the evaluation process is often detached from real-world interactive environments, making it difficult to reflect the model's decision-making performance under conditions of continuous state changes. Second, evaluation metrics tend to focus on the correctness of results, lacking a coordinated assessment of rule understanding, process consistency, response costs, and performance fluctuations. Third, evaluation results often remain at the level of scores or accuracy rates, which is not conducive to identifying the causes of errors and forming subsequent optimization paths. Fourth, there is a lack of a closed-loop mechanism to directly feed evaluation data back into the prompting strategies and model parameter adjustments, making it difficult to form a reusable and iterative model optimization process.

[0079] Therefore, embodiments of the present invention provide a method for dynamic evaluation and optimization of a large language model based on game-based adversarial interaction. This method is built upon an interactive closed loop between the game environment and the large language model agent, referring to... Figure 5Overall, it comprises a game engine layer, a general model interaction adaptation layer, a dynamic adversarial evaluation layer, and a diagnostic optimization feedback layer. The game engine layer provides a game environment with clear rules and win / loss criteria; the general model interaction adaptation layer converts game states into prompts that can be processed by a large language model and converts model outputs into actions that can be executed in the game environment; the dynamic adversarial evaluation layer records the game process and generates multi-dimensional quantitative evaluation results; and the diagnostic optimization feedback layer generates optimization measures based on the evaluation results, thus forming a closed loop of evaluation, optimization, and re-evaluation.

[0080] Specifically, embodiments of the present invention provide a method for dynamic evaluation and optimization of a large language model based on game-based adversarial competition, referring to... Figure 1 ,include:

[0081] Step S102: Obtain rule data, state data, and win / loss determination rules for at least one game environment, and provide state input interface and action execution interface to at least one large language model agent through a standardized game application programming interface.

[0082] Step S104: Perform feature quantization analysis on the game environment to obtain the game feature vector. ,in, As a game genre, Representing state complexity, Indicates the decision branching factor. This indicates time pressure.

[0083] Specifically, game type can be quantified using enumeration; state complexity can be quantified using the logarithm of state space size; decision branch factor can be represented by the average number of legal actions; and time pressure can be represented by the time limit per round. By introducing game feature vectors, the differences in interaction difficulty, search space, and time constraints of different game environments can be uniformly mapped into structured inputs that can be used for template matching and subsequent evaluation.

[0084] Step S106: Obtain the current game state Game history information and game feature vectors Match prompt templates from the prompt template library and generate corresponding prompts under the constraints of the context window. .

[0085] When actually generating prompts, the current game state, key historical information, and rule constraints from the template are all placed into the context window to generate prompts without exceeding the context length limit. .

[0086] Step S108: Set the prompt word Input the corresponding large language model agent and obtain the model response. and the model response Perform motion analysis to obtain actions that can be executed by a standardized game application programming interface. .

[0087] Specifically, rule-matching algorithms or small AI models can be used to map the natural language descriptions in the model response into precise action commands. When an invalid or semantically ambiguous action command corresponding to the model output is detected, a model regeneration mechanism is triggered; if the regeneration result still does not meet the legal action conditions, a preset conservative action is executed to maintain the continuity of the game and prevent the system from being interrupted due to illegal operations.

[0088] Step S110: Based on action Update the game environment's match status and record the status during the match. Prompt words Model response ,action The results of the battles, reaction times, token usage, and decision-making criteria are used to form a global battle log database.

[0089] Step S112: Based on the global battle log database, perform quantitative evaluation of strategy effectiveness, rule compliance, resource efficiency and stability to obtain multi-dimensional evaluation results.

[0090] Step S114: Based on the multi-dimensional evaluation results and the global battle log database, perform error type diagnosis, and perform at least one of the following based on the diagnosis results: prompt word optimization, model optimization, and prompt word template library optimization, to form a closed loop of evaluation, optimization, and re-evaluation.

[0091] First, the system connects to at least one game environment with clearly defined rules and win / loss determination rules, and provides state input and action execution capabilities to the large language model agent through a standardized interface. Then, the game environment is feature-quantified to form a game feature vector representing game type, state complexity, decision branching factors, and time pressure. Next, based on the current game state, historical information, and game features, prompt words are generated, and the large language model outputs a response, which, after action parsing, drives the game environment to continue evolving. Simultaneously, the system records state, prompt words, model responses, actions, win / loss results, time consumption, token consumption, and decision-making basis, forming a global battle log database. Finally, based on the log database, the system performs multi-dimensional evaluation, error diagnosis, and at least one optimization measure, thus forming a closed loop of evaluation, optimization, and re-evaluation.

[0092] Instead of simply using large language models for game decision-making, this approach constructs a technical chain that connects environmental interaction, process recording, quantitative evaluation, error attribution, and closed-loop optimization. This allows the model's final win / loss result and intermediate decision-making processes to serve as evaluation criteria, transforming traditional static evaluation into a continuous, replayable, and interventionist engineering evaluation process. It enables the integration of different rule games under a unified interface and allows for horizontal comparison of different large language models using the same evaluation framework. Furthermore, because each decision step is recorded and incorporated into quantitative analysis, it is possible to pinpoint specific shortcomings of the model in rule understanding, strategy selection, resource consumption, and performance fluctuations. This provides a direct basis for subsequent prompt word adjustments, template iterations, and model training.

[0093] In some embodiments, the game environment includes at least one of board games, card games, and other games with clearly defined rules; board games include at least one of Go, chess, backgammon, and six-card poker; card games include at least one of Guandan, Dou Dizhu, Kuhn poker, Leduc poker, bridge, Goofspiel, and Liar's Dice; other games include at least one of Tic-Tac-Toe, Connect Four, and checkers; the selection of the game environment is based on at least one of the following: large language model agent relationships, information structure, and action patterns, wherein the information structure includes perfect information structure and imperfect information structure, and the action patterns include turn-based action patterns and simultaneous action patterns.

[0094] By mapping the capabilities of the large language model used for evaluation to the game's game-like attributes, the evaluation scenarios are no longer arbitrarily selected, but rather structured and configured around cognitive requirements such as perfect information, imperfect information, turn-based decision-making, and simultaneous decision-making. Thus, the game environment itself becomes part of the capability testing dimension, rather than merely an external carrier of the model's output. This limitation allows for a clear correspondence between the evaluation objectives and the testing environment. For example, rule comprehension ability can be evaluated in a rule-dense environment, long-term planning ability in a long-range game environment, and strategy judgment ability under imperfect information in an incomplete information game environment, thereby improving the consistency between test results and actual capabilities.

[0095] In some embodiments, the current game state is obtained. Game history information and game feature vectors Match prompt templates from the prompt template library and generate corresponding prompts under the constraints of the context window. , refer to Figure 2 ,include:

[0096] Step S1061: Obtain game information including at least the game type, board layout, and current turn state.

[0097] Step S1062: Match a prompt word template from the prompt word template library based on the obtained game information.

[0098] Step S1063: Set the current game state After being integrated with game history information, the information is placed into the model's context window, and prompts are generated while ensuring that the context token limit is not exceeded. .

[0099] First, obtain game information including at least the game type, board layout, and current turn status. Then, based on this game information, match the best template from the prompt word template library. Finally, integrate the current game status and historical information into the model context window to form the final prompt word while ensuring that the token limit is not exceeded.

[0100] By incorporating game state, historical information, and template matching into the prompt generation process, prompt words become dynamic inputs coupled with the current situation. This allows the large language model to obtain the necessary information filtered for the current situation before each decision, thereby improving the relevance of state understanding and action generation, reducing the problem of irrelevant historical information occupying contextual resources, and enabling the model to prioritize the information most relevant to the current situation within a limited context window. This improves the effectiveness of prompts and reduces the increase in response costs and decision bias caused by contextual redundancy.

[0101] In some embodiments, the prompt word template library is a dynamically evolving template library, based on game feature vectors. Automatically generate initial prompt word template Initial prompt word template It should be generated at least according to the following sections: role definition, rules and boundary constraints, state description, output format, and example, and should include at least the objective function for the role and the rules and boundary constraints. The role definition section specifies the current agent's role in the game. The objective function is expressed as follows: ,in, Indicates state, Actions are defined to specify the goal to be maximized or minimized in the current game. Rules and boundary constraints constrain the model output through hard or soft constraints, preventing the model from outputting content that does not conform to the current rules; hard or soft constraints are expressed as: if Then a penalty item will be applied. ,in, Representing state The set of legal actions is defined below; the penalty is implemented by reducing the score of candidate actions during the re-ranking stage, or by adding illegal action warning information to the model regeneration prompt.

[0102] For the initial prompt word template Iterative updates, satisfying: ,in, Indicates the first The prompt word template after round of iterations, Indicates the first Interaction logs and diagnostic data generated from round-robin assessments This represents the template optimization function. All templates are indexed and organized according to their corresponding game feature vectors and iterative versions of prompt word templates. When a new game or new match is introduced, the system optimizes the templates based on the feature vectors of the current game. The similarity between the current template and existing templates in the template library is calculated, and the template with the highest similarity or that meets a preset threshold condition is selected as the current template. This improves the template reuse capability between different games and facilitates the transfer of existing optimization results between games of similar types.

[0103] The template optimization function analyzes model weaknesses based on interaction logs and diagnostic data, and adjusts the instruction strength, number of examples, or constraints in the current template to generate subsequent prompt word templates. .

[0104] The prompt word template library is based on game feature vectors Indexing with iterative versions and achieving template matching and loading by calculating the similarity between the current game features and templates in the library, including: extracting the current game feature vector for the current game environment. And for each candidate prompt word template in the prompt word template library, obtain the corresponding template feature vector. By calculating the current game feature vector With each template feature vector Template matching is achieved using cosine similarity between the two pairs of templates, where cosine similarity satisfies: ,in, Represents the vector dot product. and represents the vector magnitude; the candidate prompt word template with the highest cosine similarity is determined as the target prompt word template, and loaded according to the iterative version corresponding to the target prompt word template.

[0105] Initial prompt word templates are automatically generated based on game feature vectors. The initial templates include at least the objective function corresponding to the character identity, as well as rules and boundary constraints. Subsequently, the templates are updated through the template optimization function under the combined effect of iteration rounds, interaction logs, and diagnostic data. Finally, the template library is organized according to game feature vectors and iteration versions, and template matching and loading are achieved through similarity calculation.

[0106] The tooltip templates are elevated from static text to indexable, measurable, and iteratively optimized structured objects, and a template update mechanism based on diagnostic results is introduced. As a result, tooltips are no longer merely manually configured based on experience, but rather technical components that can continuously adapt to game characteristics and model weaknesses as adversarial data accumulates. Template resources can be reused and optimized across different games, match stages, and model states. This reduces the cost of repeatedly constructing tooltips and, through continuous iteration, improves the adaptability between tooltip content and the target task, thereby enhancing the stability and rule consistency of the model output.

[0107] In some embodiments, the prompt word Input the corresponding large language model agent and obtain the model response. and the model response Perform motion analysis to obtain actions that can be executed by a standardized game application programming interface. , refer to Figure 3 ,include:

[0108] Step S1081: Utilize a rule-matching algorithm or a small AI model, such as a parser based on regular expression rules, a lightweight model based on action label classification, or a cascaded combination of both, to process the model response. The natural language descriptions in the code are mapped to legal action instructions that the game engine can execute.

[0109] Step S1082: When an invalid or semantically ambiguous action instruction corresponding to a model response is detected, the model regeneration mechanism is triggered. An action is deemed invalid or semantically ambiguous when it is not in the set of valid actions, lacks necessary parameters, has ambiguous mappings in the action object, or the resolution confidence is below a threshold.

[0110] Step S1083: When the model regeneration result still does not meet the legal action conditions, execute the preset conservative action; wherein, the conservative action is a preset action that meets the legal action constraints of the current state and can maintain the continuous execution of the game, such as selecting the default action from the set of legal actions that does not cause immediate failure, meets the rule constraints and maintains the continuity of the game.

[0111] By using rule matching algorithms or small models, the natural language descriptions output by the large language model are mapped into legal action instructions that can be executed by the game engine; when an invalid or semantically ambiguous action instruction is detected, a regeneration mechanism is triggered; when the regeneration result still does not meet the legal action conditions, a preset conservative action is executed.

[0112] To address the issue that large language model outputs possess natural language features, are highly interpretable, but exhibit structural instability, a hierarchical processing mechanism is proposed, encompassing action parsing, anomaly detection, regeneration, and conservative execution. This mechanism transforms free text output into stable actions within a rule-based environment, thereby resolving the issue of seamless integration between the model's language output and the game engine's execution interface.

[0113] By setting up a legal action parsing and failure fallback mechanism, the system can reduce the risk of game interruption caused by text ambiguity, illegal actions, and inconsistent formats, ensure the continuous operation of the evaluation platform, and enable the evaluation results to more realistically reflect the model's strategy capabilities, rather than being interfered with by interface anomalies.

[0114] In some embodiments, the global battle log database records at least the following data:

[0115] The single-game win / loss results and win / loss milestones, where single-game win / loss results include win, loss, draw, and loss due to timeout; the large language model records the state. To model response The time difference is used as the reaction time; the number of input prompt tokens, the number of output response tokens, the total token consumption per game, and the average token consumption per step; the dynamically generated complete prompt text; the model response text containing inference content; and the model response. Rules, strategies, or probabilistic reasoning within. Among these, states... The prompt indicates the game state at a certain point in the game; Indicates the state Generated input prompts, model response This indicates that the large language model is based on prompt words. The generated output content, actions Indicates the model response The actions that are parsed and executed in the game environment.

[0116] Global battle log database The quadruple is the basic recording unit used to reproduce the decision chain.

[0117] By unifying process data, resource data, and result data from adversarial evaluation into a log database, the log database serves both as a review tool and a data source for subsequent evaluation, diagnosis, and optimization. This recording method ensures the traceability of each step of the model's output. Simultaneously, it allows evaluation to move beyond solely relying on the final outcome, enabling the tracking and analysis of intermediate decisions. This facilitates identifying the location and cause of errors, as well as the sources of resource consumption, providing a high-quality data foundation for subsequent template optimization and model training.

[0118] In some embodiments, the multi-dimensional evaluation results are calculated using the following formula: ;in, These represent the weights corresponding to strategy effectiveness, rule compliance, resource efficiency, and stability, respectively. Each weight is pre-set based on the game category, evaluation purpose, or deployment requirements, or determined using a normalized learning method based on target performance in historical samples, and satisfies the following conditions: ; The score represents the strategy effectiveness sub-score. This indicates that the rule follows the sub-score. This represents the resource efficiency sub-score. The stability sub-scores are represented, with each sub-score ranging from zero to one hundred.

[0119] Instead of using a single win rate or accuracy rate as the criterion for judging model performance, a unified quantitative framework has been established that takes into account result quality, rule enforcement, resource costs, and performance fluctuations. This framework maps different types of capabilities to the same scoring system, facilitating systematic analysis. Through this comprehensive scoring mechanism, comparable unified indicators can be formed across different models, different games, and even different evaluation periods, avoiding biased judgments based on a single result and enabling objective verification of improvements before and after optimization.

[0120] In some embodiments, policy effectiveness sub-score It is determined at least based on win / loss ratio, decision quality score, and planning consistency, among which:

[0121] Win / loss ratio satisfies: ;

[0122] The decision quality score satisfies: Wherein, the LLM decision score represents the actual decision score of the large language model at each decision step, and the optimal decision score represents the decision score of the optimal solution in the corresponding state;

[0123] Planning consistency is satisfied: In this context, the actual decision objective represents the immediate objective corresponding to each step in the decision sequence, the expected long-term objective represents the long-term planning objective corresponding to the game phase, and the objective deviation threshold represents the threshold for determining whether the actual decision deviates from the long-term objective.

[0124] The rules follow the sub-scores It should be determined at least based on compliance rate, misunderstanding rate, and type of violation, among which:

[0125] Compliance rate met: ;

[0126] The misunderstanding rate satisfies: ;

[0127] The types of violations are used as statistical categories to identify the most frequent types of violations after the distribution histogram is generated, thus identifying the weak points in rule understanding of the large language model.

[0128] Resource efficiency sub-score It is determined at least based on average execution time, average token consumption, and resource usage, where:

[0129] Average time consumption meets: ;

[0130] Average token consumption satisfies: ;

[0131] Resource usage includes at least hardware metrics such as GPU utilization, CPU utilization, and memory usage.

[0132] Stability sub-score Determined at least based on performance variance, robustness, and error mode, where:

[0133] The performance variance satisfies: ,in, Indicates the first The bureau's decision-making quality score This represents the average score of decision-making quality across multiple rounds. Indicates the number of statistical bureaus;

[0134] The anti-interference rate meets the following requirements: ;

[0135] Error patterns are clustered based on historical errors to identify recurring error types.

[0136] The evaluation of the decision-making ability of large language models is refined from a single-layer scoring system into a decomposable cluster of indicators. This allows the evaluation results to differentiate whether the model fails due to rule understanding, poor local decision-making quality, insufficient long-term planning, excessive resource consumption, or unstable performance. This multi-indicator decomposition provides clear targets for subsequent diagnostics. By refining the evaluation indicators, the overall score can be further broken down into actionable optimization directions. For example, when the rule compliance sub-score is low, priority can be given to strengthening rule constraints and action verification; when the resource efficiency sub-score is low, priority can be given to compressing context and response content; when stability is low, priority can be given to analyzing error patterns and anti-perturbation strategies, thereby improving the targeting of optimization.

[0137] In some embodiments, error type diagnosis is performed based on multi-dimensional evaluation results and a global battle log database. Based on the diagnosis results, at least one of the following is executed: prompt word optimization, model fine-tuning optimization, and prompt word template library optimization, to form a closed loop of evaluation, optimization, and re-evaluation. Figure 4 ,include:

[0138] Step S1121: Monitor the global battle log database and classify the error types into at least one of the following: syntax error, logic error, rule misunderstanding, and strategy mistake.

[0139] After each match, a replay analysis is performed on the global match log database to identify state fragments corresponding to low-quality decisions or illegal actions, and these fragments are categorized into preset error categories. For syntax errors, the main adjustments are made to output format constraints and action field constraints; for logical errors, the main adjustments are made to add key state summaries or situation analysis requirements; for rule misunderstandings, the main adjustments are made to strengthen rule boundaries and legal action examples in the prompt word template; and for strategy errors, the main adjustments are made to add expected action samples for the corresponding state to the fine-tuning dataset.

[0140] Step S1122: Update the subsequent prompts based on the error types that occur in the current game, in order to optimize the real-time prompts.

[0141] Step S1123: Construct paired data of error states and corrected expected actions and input them into the large model to optimize the model.

[0142] Model optimization is achieved through construction Pairing data is performed. Among them, the status... The game state at the moment the error occurred is used to determine the expected actions, which should originate from the rule engine, expert strategies, or high-quality actions from winning matches. This paired data can be stored in a supervised fine-tuning dataset for subsequent targeted retraining of the large language model.

[0143] Step S1124: Based on the global battle log database, summarize the hint strategies for specific games or specific opponents, and update the hint template library to optimize the hint template library.

[0144] The prompt word template library optimization is based on summarizing better prompting methods for specific games, specific situation stages, or specific opponent types according to the results of multiple rounds of competition, and feeding these results back into the template library to ensure that the iterative relationship of the template library satisfies the aforementioned requirements. This leads to the formation of continuous optimization capabilities.

[0145] Step S1125: After completing at least one of the prompt word optimization, model optimization, and prompt word template library optimization, re-execute the adversarial evaluation using the optimized prompt word template or the optimized large language model to verify the effectiveness of the optimization measures.

[0146] Continuously monitor the global battle log database and classify errors into at least one of the following: syntax errors, logical errors, rule misunderstandings, and strategic mistakes. Update subsequent prompts based on the error type in the current game to achieve real-time prompt word optimization. Simultaneously, construct paired data between error states and expected actions after correction for model optimization. Then, summarize prompt strategies for specific games or specific opponents based on the global logs to update the prompt word template library. After optimization, re-execute adversarial evaluation using the optimized prompt word templates or large language model to verify the effectiveness of the optimization.

[0147] The evaluation output is directly transformed into optimization input, which is then applied to the prompt word content, template library structure, and model parameter layer, forming a multi-layered feedback loop. This structure avoids the problem of evaluation and training being isolated, enabling the system to continuously correct model behavior based on real adversarial logs. It transforms the system from a simple evaluation tool into a continuous iteration platform. Through the re-evaluation mechanism, it can also be verified whether the optimization measures truly improve rule compliance, decision quality, and resource efficiency, rather than just producing accidental improvements in local scenarios.

[0148] The following description, using a chess match evaluation scenario, further illustrates the method of this invention:

[0149] In this embodiment, the chess game engine is connected to a standardized game application programming interface (API). Status It should include at least the current board layout, the current player's move, the historical move sequence, and the remaining time. Model A and Model B are encapsulated as two large language model agents, each loaded with an initial prompt word template matched to the chess environment. The initial prompt word template should include at least: the current player's identity, the objective function, chess rule constraints, a description of legal moves, and an output format description.

[0150] After the game begins, the system executes the following loop in each round of decision-making: First, it reads the current board state from the chess engine to form the state. Subsequently, based on the status Historical move information and corresponding game feature vectors for chess. Generate prompt words Next, the prompt words will be displayed. Send the data to Model A and Model B respectively, and obtain the corresponding model responses. Then, the model response is parsed into move actions using an action parser. Finally, the chess engine executes the moves and updates the board state until the game ends. Each interaction is written to the global game log database and... Stored in quadruplet format.

[0151] After completing multiple rounds of play, the system calls the multi-dimensional evaluation module to calculate the comprehensive evaluation scores for Model A and Model B respectively. Regarding strategy effectiveness, the win-loss ratios of both models are calculated, and the decision quality of each move is scored using the chess engine. For rule compliance, the number of illegal moves and the rule misunderstanding rate are calculated. Regarding resource efficiency, the average single-move decision time and average token consumption are calculated. Regarding stability, the performance variance of the decision quality scores across multiple games is calculated, and the robustness rate is calculated under the condition of introducing perturbation input. Based on the aforementioned comprehensive scoring formula, the system outputs the multi-dimensional evaluation results for Model A and Model B and generates a comparison report.

[0152] In a preferred scenario, the evaluation results showed that Model A's strategy effectiveness score was low in the mid-game phase, and there were instances of timeouts resulting in losses. The system further reviewed the global battle log database corresponding to Model A, identifying that its mid-game responses were excessively long, had too many candidate moves, and lacked sufficient identification of key threats. Based on this diagnosis, the system performed two types of optimizations on Model A: first, prompt word optimization, which added constraints to the mid-game template to prioritize the calculation of direct threats and responses, and limited the number of output candidate moves; second, model fine-tuning optimization, which extracted the expected actions corresponding to key mid-game states from high-quality winning games, and constructed... Fine-tune the samples. After optimization, re-execute the adversarial evaluation using the optimized prompt word template or the fine-tuned model A, and compare the multi-dimensional evaluation results before and after optimization to verify the effectiveness of the optimization measures.

[0153] Another embodiment of the present invention provides a dynamic evaluation and optimization device for a large language model based on game-based adversarial competition, comprising:

[0154] The data acquisition module is used to acquire rule data, state data, and win / loss determination rules of at least one game environment, and to provide state input interface and action execution interface to at least one large language model agent through a standardized game application programming interface.

[0155] The analysis module is used to perform feature quantification analysis on the game environment to obtain game feature vectors. ,in, As a game genre, The logarithm of the state space size is used to represent the state complexity. The average number of legal actions is used to represent the decision branching factor. The time limit for each round is used to indicate time pressure.

[0156] The matchmaking module is used to obtain the current game status. Game history information and game feature vectors Match prompt templates from the prompt template library and generate corresponding prompts under the constraints of the context window. .

[0157] The parsing module is used to analyze the prompt words. Input the corresponding large language model agent and obtain the model response. and the model response Perform motion analysis to obtain actions that can be executed by a standardized game application programming interface. .

[0158] Update module, used for action-based updates Update the game environment's match status and record the status during the match. Prompt words Model response ,action The results of the battles, reaction times, token usage, and decision-making criteria are used to form a global battle log database.

[0159] The evaluation module is used to quantitatively evaluate strategy effectiveness, rule compliance, resource efficiency, and stability based on the global battle log database, and obtain multi-dimensional evaluation results.

[0160] The optimization module is used to diagnose error types based on multi-dimensional evaluation results and the global battle log database, and to perform at least one of the following optimizations based on the diagnosis results: prompt word optimization, model optimization, and prompt word template library optimization, so as to form a closed loop of evaluation, optimization, and re-evaluation.

[0161] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0162] Based on the same inventive concept as the above method embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it enables the electronic device to implement the control method described in the above embodiments.

[0163] In one embodiment, the electronic device may be a server, and in this embodiment, the structure of the electronic device may be as follows: Figure 6 As shown, it includes a memory, a communication module, and one or more processors.

[0164] Memory is used to store computer programs executed by the processor. Memory can be mainly divided into a program storage area and a data storage area. The program storage area can store the operating system and programs required to run instant messaging functions, etc.; the data storage area can store various instant messaging information and operation instruction sets, etc.

[0165] Memory can be volatile memory, such as random access memory (RAM); memory can also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory can be any other medium capable of carrying or storing a desired computer program having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory can be a combination of the above-mentioned types of memory.

[0166] A processor may include one or more central processing units (CPUs) or digital processing units, etc. The processor is used to implement the aforementioned audio data processing methods when it invokes computer programs stored in memory.

[0167] The communication module is used to communicate with terminal devices and other servers.

[0168] This application embodiment does not limit the specific connection medium between the above-described memory, communication module, and processor. This application embodiment... Figure 6 The memory and processor are connected via a bus, and the bus is in... Figure 6 The connections between other components are illustrated with arrows and are for illustrative purposes only, not as limiting information. Buses can be categorized as address buses, data buses, control buses, etc. For ease of description, Figure 6 The text uses only one arrow to describe it, but does not indicate that there is only one bus or one type of bus.

[0169] Based on the same inventive concept as the above-described method embodiments, embodiments of the present invention also provide a computer-readable storage medium for storing a computer program. When the computer program is run on a computer, it enables an electronic device to implement the control methods described in the above embodiments. The computer-readable storage medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0170] Based on the same inventive concept as the above-described method embodiments, embodiments of the present invention also provide a computer program product. The computer program product includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of the control methods described above according to various exemplary embodiments of this application. The program product may take the form of any combination of one or more readable media. These computer program commands can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the commands executed by the processor of the computer or other programmable data processing device generate a process for implementing... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0171] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A method for dynamic evaluation and optimization of a large language model based on game-based adversarial competition, characterized in that, include: Acquire rule data, state data, and win / loss determination rules for at least one game environment, and provide state input interface and action execution interface to at least one large language model agent through a standardized game application programming interface; Perform feature quantization analysis on the game environment to obtain the game feature vector. , include ,in, As a game genre, The logarithm of the state space size is used to represent the state complexity. The average number of legal actions is used to represent the decision branching factor. The time limit for each round is used to indicate time pressure; Get the current game status Game history information and the game feature vector Match prompt templates from the prompt template library and generate corresponding prompts under the constraints of the context window. ; The prompt word Input the corresponding large language model agent and obtain the model response. and respond to the model Perform action parsing to obtain actions that can be executed by the standardized game application programming interface. ; Based on the action Update the game state of the game environment and record the state during the game. Prompt words Model response ,action The results of the battles, reaction time, token usage, and decision-making criteria are used to form a global battle log database. Based on the global battle log database, strategy effectiveness, rule compliance, resource efficiency, and stability are quantitatively evaluated to obtain multi-dimensional evaluation results. Among them, strategy effectiveness refers to the strategy effectiveness sub-score and is determined at least based on win rate, decision quality score, and planning consistency; rule compliance refers to the rule compliance sub-score and is determined at least based on compliance rate, misunderstanding rate, and violation action type; resource efficiency refers to the resource efficiency sub-score and is determined at least based on average time consumption, average token consumption, and resource usage; and stability refers to the stability sub-score and is determined at least based on performance variance, anti-interference rate, and error pattern. Based on the multi-dimensional evaluation results and the global battle log database, error type diagnosis is performed, and at least one of prompt word optimization, model optimization, and prompt word template library optimization is executed according to the diagnosis results to form a closed loop of evaluation, optimization, and re-evaluation. This includes: monitoring the global battle log database and classifying error types into at least one of syntax errors, logical errors, rule misunderstandings, and strategy mistakes; updating subsequent prompt content for the error types appearing in the current game to perform real-time prompt word optimization; constructing paired data of error states and corrected expected actions and inputting it into a large model to optimize the model; summarizing prompt strategies for specific games or specific opponents based on the global battle log database to update the prompt word template library and achieve prompt word template library optimization; after completing at least one of prompt word optimization, model optimization, and prompt word template library optimization, re-executing the adversarial evaluation using the optimized prompt word template or the optimized large language model to verify the effectiveness of the optimization measures.

2. The method for dynamic evaluation and optimization of a large language model based on game-based adversarial competition as described in claim 1, characterized in that, The game environment includes at least one of board games, card games, and other games with clearly defined rules; The board games include at least one of Go, chess, backgammon, and six-card poker; the card games include at least one of Guandan, Dou Dizhu, Kuhn Poker, Leduc Poker, bridge, Goofspiel, and Liar's Dice; and the other games include at least one of Tic-Tac-Toe, Connect Four, and checkers. The selection criteria for the game environment include at least one of the following: large language model agent relationships, information structure, and action patterns, wherein the information structure includes perfect information structure and imperfect information structure, and the action patterns include turn-based action patterns and simultaneous action patterns.

3. The method for dynamic evaluation and optimization of a large language model based on game-based adversarial competition as described in claim 1, characterized in that, The step of obtaining the current game status Game history information and the game feature vector Match prompt templates from the prompt template library and generate corresponding prompts under the constraints of the context window. ,include: Obtain game information including at least the game type, board layout, and current turn state; Based on the obtained game information, match the prompt word template from the prompt word template library; Current game status After being integrated with game history information, the information is placed into the model's context window, and prompts are generated while ensuring that the context token limit is not exceeded. .

4. The method for dynamic evaluation and optimization of a large language model based on game-based adversarial competition according to claim 3, wherein the prompt word template library is a dynamically evolving template library, characterized in that, Based on the game feature vector Automatically generate initial prompt word template The initial prompt word template It includes at least an objective function for role identity and rules and boundary constraints, wherein the objective function is expressed as follows: ,in, Indicates state, Indicates the action; the rules and boundary constraints are output through hard or soft constraint models, whereby the hard or soft constraints are expressed as: if Then a penalty item will be applied. ,in, Representing state The set of legal actions below; For the initial prompt word template Iterative updates, satisfying: ,in, Indicates the first The prompt word template after round of iterations, Indicates the first Interaction logs and diagnostic data generated from round-robin assessments This represents the template optimization function; The template optimization function analyzes the model's weaknesses based on the interaction logs and diagnostic data, and adjusts the instruction strength, number of examples, or constraints in the current template to generate subsequent prompt word templates. ; The prompt word template library is based on game feature vectors. Indexing with iterative versions and achieving template matching and loading by calculating the cosine similarity between the current game features and templates in the library, including: extracting the current game feature vector for the current game environment. And for each candidate prompt word template in the prompt word template library, obtain the corresponding template feature vector. By calculating the current game feature vector With each of the template feature vectors Template matching is achieved using cosine similarity between the two, wherein the cosine similarity satisfies: ,in, Represents the vector dot product. and These represent the vector magnitudes; the candidate prompt word template with the highest cosine similarity is determined as the target prompt word template, and loaded according to the iterative version corresponding to the target prompt word template.

5. The method for dynamic evaluation and optimization of a large language model based on game-based adversarial competition as described in claim 1, characterized in that, The prompt word Input the corresponding large language model agent and obtain the model response. and respond to the model Perform action parsing to obtain actions that can be executed by the standardized game application programming interface. ,include: The model response is obtained through rule-matching algorithms or small artificial intelligence models. The natural language descriptions in the code are mapped to legal action instructions that the game engine can execute; When the model response is detected to be invalid or semantically ambiguous, the model regeneration mechanism is triggered. If the model regeneration result still does not meet the legal action conditions, a preset conservative action is executed; wherein, the conservative action is a preset action that meets the legal action constraints of the current state and can maintain the continuous execution of the game.

6. The method for dynamic evaluation and optimization of a large language model based on game-based adversarial competition as described in claim 1, characterized in that, The global battle log database records at least the following data: The results of a single game and the points at which a game is decided by a timeout; Large language models from recorded states To model response The time difference is used as the reaction time. Input prompt token count, output response token count, total token consumption in a single round, and average token consumption per step; Dynamically generated complete prompt text, model response text containing inference content, and model response. Rules, strategies, or probabilistic reasoning within; Among them, state The prompt indicates the game state at a certain point in the game; Indicates the state Generated input prompts, model response This indicates that the large language model is based on prompt words. The generated output content, actions Indicates the model response The actions that are parsed and executed in the game environment; The global battle log database is The quadruple is the basic recording unit used to reproduce the decision chain.

7. The method for dynamic evaluation and optimization of a large language model based on game-based adversarial competition as described in claim 1, characterized in that, The multi-dimensional evaluation results are calculated using the following formula: ;in, Let represent the weights corresponding to strategy effectiveness, rule compliance, resource efficiency, and stability, respectively, and satisfy the following: ; The score represents the strategy effectiveness sub-score. This indicates that the rule follows the sub-score. This represents the resource efficiency sub-score. The stability sub-scores are represented, with each sub-score ranging from zero to one hundred.

8. The method for dynamic evaluation and optimization of a large language model based on game-based adversarial competition as described in claim 7, characterized in that, The strategy effectiveness sub-score It is determined at least based on win / loss ratio, decision quality score, and planning consistency, among which: Win / loss ratio satisfies: ; The decision quality score satisfies: Wherein, the LLM decision score represents the actual decision score of the large language model at each decision step, and the optimal decision score represents the decision score of the optimal solution in the corresponding state; Planning consistency is satisfied: Among them, the actual decision target represents the immediate target corresponding to each step in the decision sequence, the expected long-term target represents the long-term planning target corresponding to the game stage, and the target deviation threshold represents the threshold for judging whether the actual decision deviates from the long-term target. The rules follow sub-scores. It should be determined at least based on compliance rate, misunderstanding rate, and type of violation, among which: Compliance rate met: ; The misunderstanding rate satisfies: ; The types of violations are used as statistical categories to identify the most frequent types of violations after the distribution histogram is generated, thus identifying the weak points in the rule understanding of the large language model. The resource efficiency sub-score It is determined at least based on average execution time, average token consumption, and resource usage, where: Average time consumption meets: ; Average token consumption satisfies: ; Resource usage includes at least hardware metrics such as GPU utilization, CPU utilization, and memory usage. The stability sub-score Determined at least based on performance variance, robustness, and error mode, where: The performance variance satisfies: ,in, Indicates the first The bureau's decision-making quality score This represents the average score of decision-making quality across multiple rounds. Indicates the number of statistical bureaus; The anti-interference rate meets the following requirements: ; Error patterns are clustered based on historical errors to identify recurring error types.

Images