A large language model multi-agent strategy evaluation method based on negotiation-action-reflection
By adopting an evaluation method based on the Negotiation-Action-Reflection (NAR) cycle, the strategic game process of multi-agents is explicitly decomposed, which solves the problems of untraceable evaluation process, incomparability across models, and poor reproducibility in existing technologies. It realizes comprehensive strategic evaluation of large language models and provides interpretable process-based scoring.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHANGCHUN UNIV OF TECH
- Filing Date
- 2026-04-16
- Publication Date
- 2026-07-14
AI Technical Summary
Existing multi-agent strategy evaluation methods for large language models lack process traceability, cross-model comparability, reproducibility and auditability, and have incomplete indicator systems, making it difficult to comprehensively evaluate the model's capabilities in multiple rounds of strategic interaction.
The evaluation method based on the Negotiation-Action-Reflection (NAR) cycle is adopted. Through the construction of a strategic game environment, prompt adaptation and interface alignment, structured process recording and comprehensive score calculation, each round of interaction is explicitly divided into negotiation, action and reflection stages. Process logs are recorded and consistency verification is performed, and rationality bias and process indicators are introduced.
It achieves traceability of model behavior, fair comparison across models, reproducibility and auditability of evaluation, provides more comprehensive evaluation indicators, and can explain the model's strategy formation and improvement capabilities.
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and large language model evaluation technology, and in particular to an evaluation method for multi-agent strategic game scenarios. Specifically, it is a multi-agent strategic evaluation method for large language models based on the "Negotiate-Act-Reflect (NAR)" cycle, involving game environment construction, prompt adaptation and interface alignment, structured process recording, consistency verification and comprehensive score calculation. Background Technology
[0002] As the application of large language models in complex decision-making, interactive collaboration, and multi-turn dialogues continues to expand, evaluating the capabilities of large language models in multi-turn strategic interactions has become a key issue. Existing evaluation methods commonly include static question answering, single-round multiple-choice questions, or game-style evaluations that only focus on the final outcome. These methods have at least the following shortcomings:
[0003] 1. Lack of process traceability: Simply statistically analyzing the final results makes it difficult to explain the reasons for the formation of strategies, negotiation and communication and action choices in each round of the model, and makes it difficult to locate the source of errors.
[0004] 2. Insufficient cross-model comparability: Different models have significant differences in input / output interfaces, prompt formats, and action expression methods, making it difficult to fairly align evaluation results.
[0005] 3. Insufficient reproducibility and auditability: The lack of unified process log specifications and consistency verification makes it difficult to achieve game replay, error review, alignment analysis and data reuse.
[0006] 4. Incomplete indicator system: Simply focusing on returns may mask issues such as inconsistent strategies, rational biases, and short-sighted behavior, leading to one-sided evaluation conclusions.
[0007] Therefore, it is necessary to propose a multi-agent strategy evaluation method for large language models that is procedural, structured, reproducible, and comparable in multiple rounds of strategic interaction. Summary of the Invention
[0008] This invention aims to address the problems of "untraceable process, difficulty in cross-model alignment, difficulty in reproducing results, and incomplete indicators" in existing multi-agent strategy evaluation. It provides a technical solution for unified evaluation of a model's negotiation ability, action decision-making ability, and reflective update ability in multi-round strategic games. To achieve the above objectives, this invention provides a multi-agent strategy evaluation method based on a large language model of negotiation-action-reflection, comprising at least the following steps:
[0009] Step 1: Construction of Strategic Game Environment and Unified Configuration of Parameters. Construct a strategic game environment and configure environment parameters. The environment includes at least: state representation, action set, payoff function and observable social information; set multiple rounds of interaction and the number of participating agents; the environment is used to receive the actions of each agent in each round and update the state and payoff. In a preferred embodiment, the strategic game environment is provided in the form of a scenario suite, which includes at least four types of progressive strategic task scenarios: (1) individual resource optimization; (2) long-term social cooperation; (3) shared resource management; (4) incomplete information inference; each scenario follows a unified "state-action-payoff" interface protocol to support cross-scenario comparable evaluation.
[0010] Step 2: Prompt adaptation and model interface alignment. Input the state of the environment and social information into the prompt adaptation module, generate stage prompts and complete the interface alignment of different large language models. Dynamically inject the current round of information at runtime and perform structured constraints on the output. The stage prompts include: Description, Negotiation, Request, Report and Reflection.
[0011] Step 3: Negotiate Phase Execution. Input negotiation phase prompts to each agent to output structured negotiation phase data. The structured data includes at least: contextual understanding information, policy intent information, and negotiation message information to obtain the negotiation result.
[0012] Step 4, Action Phase Execution (Act): Input request phase prompts to each agent to make it output structured action phase data. The structured data includes at least: planned behavior information and actual action information. The actual action information is mapped to environmental actions and the result feedback information is obtained by the environment execution.
[0013] Step 5: Reflection Phase Execution and Anomaly Handling Closed Loop. Reports and reflection phase prompts are input to each agent, enabling them to output structured reflection phase data based on the feedback information. This structured data includes at least: learning summary information, policy adjustment information, and confidence information, used to update the next round's policy or memory. If the structured data is missing, unstructured, or fails to parse, error information is recorded, and execution continues using a preset default policy. The process returns to Step 2 to regenerate the phase prompts, achieving sustainable operation and auditable error tracking.
[0014] Step Six: Process Recording and Comprehensive Score Calculation. The inputs and outputs of each round of negotiation, action, and reflection phases are recorded process-wise, generating game-level and agent-level process logs. Consistency checks are performed, and an evaluation score is calculated after the game concludes. Efficiency is calculated for each scenario. deviation from rationality ,in This indicates normalized returns or target achievement. The score for a single scenario represents the deviation from a rational benchmark (equilibrium / optimal / sustainable strategy, etc.) for that scenario. , The deviation penalty coefficient is used, and the final total score is the average of the scores in the six scenarios.
[0015] Beneficial effects
[0016] Compared with the prior art, the present invention has at least the following technical effects:
[0017] 1. Process traceability: Each round of interaction is explicitly broken down into three stages: negotiation, action, and reflection, forming an auditable strategy chain, which is conducive to explaining and diagnosing model behavior.
[0018] 2. Cross-model alignment: Through the prompt adaptation and interface alignment module, the input and output protocols and action mappings are unified, enabling different large language models to be compared fairly under the same evaluation framework.
[0019] 3. Reproducible and replayable: Through two-layer logging and consistency verification at the game level and agent level, it supports game replay, error review, indicator recalculation and data reuse.
[0020] 4. More comprehensive evaluation indicators: In addition to returns, the introduction of rationality bias and process indicators can more accurately reflect the rationality of the model, the stability of the strategy, and the ability to reflect and improve.
[0021] 5. Enhanced robustness: By introducing anomaly handling and default strategies, the evaluation continuity can be maintained when the model output is unstable, and anomalies are explicitly labeled for easy statistics and troubleshooting. Attached Figure Description
[0022] Figure 1 This is a schematic diagram of the overall architecture of the multi-agent strategy evaluation method based on the Negotiation-Action-Reflection (NAR) big language model proposed in this invention.
[0023] Figure 2 This is a schematic diagram of the three-stage cyclical mechanism of Negotiation-Action-Reflection (NAR) proposed in this invention.
[0024] Figure 3 This is a schematic diagram of the structured execution and output process of the multi-agent strategy evaluation process proposed in this invention. Detailed Implementation
[0025] This paper proposes a multi-agent strategy evaluation method based on a large language model of negotiation, action, and reflection. This method explicitly divides the interaction process of multiple agents in strategic games into negotiation, action, and reflection phases, enabling the large language model to form a closed-loop evaluation process of "strategy formation - action execution - result feedback - strategy update" in multiple rounds of interaction. Unlike traditional evaluation methods that only focus on single-round decisions or final gains, this method not only normalizes the final gain score, but also records and quantifies the consistency of negotiation intentions, actual actions, and reflection updates. Thus, it can simultaneously evaluate the model's negotiation ability, action decision-making ability, and reflection improvement ability, and provide an auditable process explanation for the final result.
[0026] Figure 1 The paper presents the overall framework of a multi-agent strategy evaluation method based on a large language model of negotiation, action, and reflection. The evaluation process centers on a strategic game environment, with a prompting adaptation module driving each agent to sequentially complete the three stages of negotiation, action, and reflection in each round, forming a closed-loop iterative process to continuously optimize decision-making strategies. Figure 2 As shown. The unified structured execution and output process in the prompt adaptation module includes steps such as environmental observation, prompt generation, structured decision output, and parsing verification. The entire process is logged to support evaluation and analysis. Figure 3 As shown, the structured process data generated during the evaluation process is uniformly recorded as a process log, and used to calculate efficiency, rationality bias, and process indicators after the game ends, ultimately forming a comprehensive score and process diagnostic conclusion.
[0027] The detailed process of the multi-agent strategy evaluation method based on the big language model of negotiation-action-reflection is as follows:
[0028] Step 1: Overall Evaluation Closed Loop and Symbol Definition
[0029] Let the scene be The number of rounds is The agent number is The environmental condition is as follows: Observable social information is The output of the negotiation phase is The output of the action phase is The output of the reflection phase is The environment transfer function is The profit function is The NAR single-round interaction process of this invention can be formally represented as:
[0030] (1)
[0031] in, Indicates the first Inference calls of a large language model; A historical summary; For each agent in the 1st The set of environmental actions of the wheel; Let be the reward vector for each agent in this round. As can be seen from formula (1), this method divides each round of interaction in a game into three stages that can be recorded and verified separately, thus providing a unified data foundation for subsequent consistency measurement and reflection gain calculation.
[0032] Step 2: Prompt adaptation, structured output, and parsing verification
[0033] To achieve cross-model comparability, this method applies structured JSON constraints to the output of each stage and provides a unified parsing function. With verification function Let the original output text of the model be... The parsed structured object is ,but:
[0034] (2)
[0035] The validation function outputs a validity indicator. :
[0036] (3)
[0037] in, Constraints for stage fields; Constraints on the set of environmental actions. In a preferred implementation: Negotiation phase At a minimum, field awareness, strategy, and messaging are required; request phase. At a minimum, the field must involve actual action (or actual decision); reflection phase At a minimum, it requires field learning, strategy adjustment, and confidence level. When... An exception handling loop is triggered in a timely manner to ensure that the evaluation process is not interrupted and that exceptions are auditable.
[0038] Step 3: Exception Handling and Default Strategy
[0039] When parsing fails, fields are missing, or actions are invalid, this method uses the default strategy function. Generate a rollback action And record the reason for the abnormality. :
[0040] (4)
[0041] Preferably, Select according to the following priority: (1) The most recent legal action in the previous round. (2) Pre-set conservative actions (3) Rule baseline policy action (4) Random legal actions within the action set. And mark the "trigger stage, exception type, and source of strategy" in the log, so as to maintain the continuity of evaluation and not cover up the impact of unstable model output.
[0042] Step 4: Definition of the Consultation-Action Consistency Indicator
[0043] To measure "consistency between words and actions," this method extracts planned actions from the negotiation phase. (For example, given by the actual intentions and actions in the reflection phase or mapped from the strategic intentions), and obtain the actual actions from the action phase. Let the distance function be... (Discrete actions use 0 / 1 mismatch rate, continuous actions use...) or Consistency is defined as follows:
[0044] (5)
[0045] in Let be the normalization constant, so that Match-level consistency can be achieved by considering the average number of rounds played and the average number of players.
[0046] (6)
[0047] As can be seen from formulas (5) and (6), this method can directly quantify the degree of deviation between negotiation intentions and actual actions without relying on the final outcome, thereby revealing hidden behavioral patterns such as "restrained negotiation but aggressive action".
[0048] Step 5: Reflect on Gain and Learning Convergence Trajectory
[0049] Let the first The normalized efficiency of the wheel is (Can be defined by player average or social efficiency), the reflection gain is defined as the amount of efficiency improvement in the next round:
[0050] (7)
[0051] The convergence trajectory can be learned before Characterized by the average slope of the wheel efficiency curve:
[0052] (8)
[0053] Step Six: Efficiency deviation from rationality Unified scoring definition
[0054] In the scene In this context, the final (or cumulative) profit of a game is... The upper and lower bounds of scene revenue are , Efficiency is then defined as:
[0055] (9)
[0056] A rational benchmark can be an equilibrium strategy, an optimal strategy, a sustainable strategy, or a near-optimal benchmark, denoted as . or its corresponding action / benefit , Rational bias can manifest as either action deviation or payoff deviation:
[0057] (10)
[0058] in For scene normalization constant, To prevent extremely small positive numbers with a denominator of zero, the single-scene score is defined as: (11)
[0059] in The weighting coefficient is preferably within the range of values. Alternatively, the default value of 1 can be used. Additional output can be provided when process metrics need to be included in the explanatory output. , , When it is necessary to include it in the overall score, an extended format can be set:
[0060] (12)
[0061] in It can be configured to 0 (not included) or positive weight (included), and the total score for multiple scenarios is:
[0062] (13)
[0063] in For the number of scenes, For scene weights and .
[0064] Preferred Implementation: Instantiation of Resource Exhaustion Scenarios
[0065] In one exemplary implementation, a resource-exhaustion scenario is selected: the number of participating agents. Number of rounds Initial public resources Each agent outputs the extraction amount per round. Environmental actions The environment updates remaining resources according to the resource update function. The reward is calculated based on a threshold mechanism. An example update and penalty can be defined as follows:
[0066] (14)
[0067] when (Penalty coefficient triggered at the safety threshold) The profit can be defined as: (15)
[0068] The output `intended_action.amount` in the negotiation phase represents the planned extraction amount, and the output `actual_action.amount` in the action phase represents the actual extraction amount. Thus, the negotiation-action consistency can be calculated using formulas (5) and (6). The output `strategy_adjustment` in the reflection phase indicates the adjustment direction for the next round, and formulas (7) and (8) are used to measure the reflection gain and learning efficiency trajectory. After the game ends, the calculation is performed according to formulas (9)–(13). This is output along with process indicators, thus obtaining a joint explanation of the final result and the process mechanism.
[0069] Through the above steps, this method achieves a systematic evaluation of the negotiation ability, action decision-making ability, and reflection and updating ability of large language models in multi-agent strategic games.
Claims
1. A multi-agent strategy evaluation method based on a large language model of negotiation-action-reflection, which includes at least the following steps: Step 1: Construct a strategic game environment and configure its parameters. The environment includes at least a state representation, a set of actions, a payoff function, and observable social information. It also sets multiple rounds of interaction and the number of participating agents. The environment is used to receive actions and update the state and payoff in each round. Step 2: Adapt the environmental state and social information input prompts to the module, generate stage prompts corresponding to the evaluation process, and complete the interface alignment of different large language models. During runtime, dynamically inject the current round of information into the stage prompts and constrain the output of each agent to be parsable structured data. Step 3: Perform the negotiation phase: Input negotiation phase prompts to each agent, so that it outputs structured data containing contextual understanding information, policy intent information and negotiation message information, and obtains the negotiation result. Step 4, Action Execution Phase: Input action phase prompts to each agent, causing it to output structured data containing planned behavior information and actual action information. The actual action information is mapped to environmental actions and executed by the environment to obtain result feedback information. Step 5, Execution of the Reflection Phase: Input the report phase prompts and reflection phase prompts to each agent, so that it can output structured data containing at least learning summary information, policy adjustment information and confidence information based on the result feedback information, so as to update the next round of policy or memory. If the structured data is missing, unstructured or parsing fails, the error information is recorded and the preset default policy is used to continue execution, and the process returns to Step 2 to regenerate the phase prompts. Step 6: Record the inputs and outputs of each round and stage in a process-oriented manner and calculate the evaluation scores: Generate game-level process logs and agent-level process logs, and calculate the single-scenario score based on efficiency indicators and rationality deviation indicators after the game ends.
2. The method according to claim 1, characterized in that: The negotiation-action-reflection process is executed in a fixed order of negotiation, action, and reflection in each round, and the structured data of each stage is saved in a nested structure that can be linked across rounds.
3. The method according to claim 1, characterized in that: The stage prompts in step two include at least description stage prompts, negotiation stage prompts, action stage prompts, reporting stage prompts, and reflection stage prompts, and are written into the current round state and social information through placeholders or key-value injection.
4. The method according to claim 1, characterized in that: The structured data output from steps three to five is represented in key-value pair format.
5. The method according to claim 1, characterized in that: The agent-level process log records include round identifiers, agent identifiers, stage prompts, and model outputs; the model outputs are saved in raw string form and parsed into structured data by a post-processor.
6. The method according to claim 1, characterized in that: The game-level process log includes header metadata and a round record array. The length of the array corresponds to the game round, and each round record should include round summary information and action sequence information.
7. The method according to claim 1, characterized in that: The game-level process log and the agent-level process log are stored in a preset storage directory and generated with filenames according to a preset naming rule; the naming rule includes at least two of the following: scene identifier, number of participants identifier, round identifier, and timestamp identifier.
8. The method according to claim 1, characterized in that: The strategic game scenarios include at least the following: personal attention allocation, personal investment, time banking, reputational debt, resource depletion, and information cascading auctions.