Multi-agent collaborative question and answer method and system based on Bayesian Nash equilibrium

Abstract

The invention relates to the technical field of natural language processing, and provides a multi-agent collaborative question answering method and system based on Bayesian Nash equilibrium, and the method comprises the steps: obtaining a user query instruction, generating a reasoning strategy, calling a plurality of agents, enabling each agent to convert a current observation state into a belief state through a belief network according to the current observation state, action parameters are generated, the large model is controlled to execute a text generation task, each agent maintains a value estimation network, and the value estimation networks take the belief state and the action parameters as input to generate instant rewards; aggregating the results of executing the text generation task by each agent to obtain a final answer; the belief state of each agent is aggregated into a unified group belief representation through a belief encoder, comprehensive loss is generated through a centralized hybrid network, and parameters of each agent are updated. Stronger consensus achievement capability and higher problem solving quality are shown in complex reasoning and planning tasks.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, and in particular relates to a multi-agent collaborative question answering method and system based on Bayesian Nash equilibrium. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In recent years, large-scale models have demonstrated outstanding capabilities in tasks such as natural language understanding, complex reasoning, and problem-solving. With the continuous expansion of model size and the increasing richness of training data, the performance of large-scale models in areas such as mathematical reasoning, commonsense reasoning, and code generation has been continuously improving. However, single large-scale models still have significant limitations when facing complex multi-step answer reasoning tasks, including problems such as broken logical chains, opaque reasoning processes, and susceptibility to illusions.

[0004] To further enhance the reasoning capabilities of large models, multi-agent collaborative frameworks are widely used. Within this framework, multiple application instances collaboratively solve complex problems through cooperation, debate, or competition. Among these, the multi-agent debate method constructs multi-round dialogues and debates among several large model agents, enabling different agents to question, supplement, and refine their respective reasoning processes, ultimately reaching a consensus. This method improves the accuracy and robustness of reasoning to some extent, but it suffers from the following key technical bottlenecks: The cost of communication between intelligent agents is too high: Existing methods rely on explicit multi-round message passing mechanisms, resulting in huge token consumption. Taking a typical three-round debate as an example, it is necessary to consume thousands of tokens to handle a single complex problem. This dense communication mode seriously restricts the practical application value and economic feasibility of the method. Lack of convergence guarantee: Existing methods lack a rigorous theoretical foundation and cannot ensure that the system reaches a stable consensus within a finite number of steps; agents may get caught in infinite loops of debate or oscillate continuously near suboptimal solutions, which seriously affects the reliability and practicality of the methods; Low coordination efficiency: Without an effective coordination mechanism, multi-agent collaboration is often difficult to achieve; problems such as overlapping viewpoints and lack of complementarity may occur among agents, leading to waste of resources and a decline in decision quality. Summary of the Invention

[0005] To address the technical problems mentioned above, this invention provides a multi-agent collaborative question-answering method and system based on Bayesian Nash equilibrium. Through the collaborative design of belief networks and centralized hybrid networks, each agent can maintain independent reasoning and leverage its individual strengths while its strategies are effectively aligned and optimized through global objectives. This results in a multi-agent optimization synergy effect, demonstrating stronger consensus-building capabilities and higher problem-solving quality in complex reasoning and planning tasks.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of this invention provides a multi-agent collaborative question-answering method based on Bayesian Nash equilibrium, comprising: Obtain the user's query command and generate an inference strategy; According to the reasoning strategy, several agents are invoked. Each agent transforms the current observation state into a belief state through a belief network based on the current observation state, and generates action parameters based on the belief state. The action parameters control the large model to perform the text generation task. Each agent maintains a value estimation network, which generates an immediate reward with the belief state and action parameters as input. The results of each agent performing the text generation task are aggregated to obtain the final answer; the belief states of each agent are aggregated into a unified group belief representation through a belief encoder; based on the group belief representation and the immediate rewards of each agent, a comprehensive loss is generated through a centralized hybrid network to update the parameters of each agent.

[0007] Furthermore, the belief state is represented as: ;in, It is the current observation state of agent i at time t. and It is a weight matrix. and It is a bias term.

[0008] Furthermore, the action parameters include a repetition penalty parameter, which is used to control the repetition of the generated text, and is expressed as: ;in, and These represent the minimum and maximum values ​​of the preset action parameters, respectively. It is a trainable weight matrix. It is a trainable bias term. It is an activation function. It indicates a state of belief.

[0009] Furthermore, the action parameters include a temperature parameter, which is used to control the randomness of the generated text, and is expressed as follows: ;in, and These represent the minimum and maximum values ​​of the preset temperature parameter, respectively. It is a trainable weight matrix. It is a trainable bias term. It is an activation function. It indicates a state of belief.

[0010] Furthermore, the belief encoder concatenates the belief states of each agent into a matrix, processes it in parallel through multiple independent attention heads, and concatenates the outputs of all attention heads. Then, through a linear projection layer, it generates a group belief representation.

[0011] Furthermore, the comprehensive loss is a weighted sum of the global temporal difference loss, similarity difference loss, and consistency loss terms.

[0012] Furthermore, the immediate reward for agent i for: ;in, , and These are the weighting coefficients. Dynamic reward loss function , is the learning rate for weight adjustment, and N is the number of agents; Rewards are based on the likelihood of the action. Specific rewards for the task. Rewards for contributions to collaboration.

[0013] A second aspect of the present invention provides a multi-agent collaborative question-answering system based on Bayesian Nash equilibrium, comprising: The instruction acquisition module is configured to: acquire user query instructions and generate inference strategies; The execution module is configured to: invoke several agents according to the inference strategy; each agent transforms the current observation state into a belief state through a belief network based on the current observation state, and generates action parameters based on the belief state; the action parameters control the large model to perform the text generation task; and each agent maintains a value estimation network, which generates an immediate reward with the belief state and action parameters as input. The hybrid module is configured to: aggregate the results of each agent performing a text generation task to obtain the final answer; aggregate the belief states of each agent into a unified group belief representation through a belief encoder; and generate a comprehensive loss based on the group belief representation and the immediate rewards of each agent through a centralized hybrid network to update the parameters of each agent.

[0014] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the multi-agent collaborative question-answering method based on Bayesian Nash equilibrium as described above.

[0015] A fourth aspect of the present invention provides a computer device including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps of a multi-agent collaborative question-answering method based on Bayesian Nash equilibrium as described above.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention, through the collaborative design of belief networks and centralized hybrid networks, enables each agent to maintain independent reasoning and leverage its individual strengths, while its strategies can be effectively aligned and optimized through global objectives. This results in a multi-agent optimization synergy effect, demonstrating stronger consensus-building capabilities and higher problem-solving quality in complex reasoning and planning tasks.

[0017] This invention replaces the multiple rounds of explicit message passing required in existing multi-agent interactions with a belief-driven implicit coordination mechanism, fundamentally avoiding the high communication costs and computational overhead caused by the consumption of massive tokens, and greatly improving the efficiency of multi-agent collaborative question answering in large-scale practical applications.

[0018] This invention systematically introduces incomplete information game theory and Bayesian Nash equilibrium into multi-agent collaborative question answering, ensuring that the system can stably converge to an equilibrium state within a finite number of steps. This effectively overcomes the problem that existing methods may fall into infinite circular arguments among multiple agents or oscillate near suboptimal solutions due to the lack of convergence calculation, thus improving the reliability and robustness of the method. Attached Figure Description

[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0020] Figure 1 This is a structural diagram of the multi-agent collaborative question-answering framework of Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of a computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0022] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0023] Example 1 This embodiment provides a multi-agent collaborative question answering method based on Bayesian Nash equilibrium.

[0024] This embodiment provides a multi-agent collaborative question answering method based on Bayesian Nash equilibrium, which is applicable to complex reasoning, planning tasks, and multi-turn dialogue scenarios.

[0025] This embodiment provides a multi-agent collaborative question answering method based on Bayesian Nash equilibrium. By constructing a belief-driven coordination mechanism, it significantly reduces communication costs and improves inference performance while ensuring theoretical convergence.

[0026] This embodiment provides a multi-agent collaborative question-answering method based on Bayesian Nash equilibrium. Taking the work injury identification business of a municipal human resources and social security department as the background, it relies on the human resources and social security vertical domain large model and multi-agent collaborative framework to demonstrate the complete process from multimodal data input to final identification result output.

[0027] This embodiment provides a multi-agent collaborative question answering method based on Bayesian Nash equilibrium, including the following steps: Step 1: Construct a multi-agent collaborative question-answering framework.

[0028] This embodiment constructs a multi-agent large-scale collaborative question-answering framework based on Bayesian Nash equilibrium. This framework adopts a hierarchical architecture and achieves implicit coordination among agents through a belief-driven mechanism, solving the problem of traditional explicit multi-turn message passing and significantly reducing communication costs while ensuring theoretical convergence. Figure 1 As shown, the framework includes the following core components and their relationships: (1) The central coordinator, as the core control unit, is responsible for three functions: receiving and parsing user queries and generating high-level inference strategies; aggregating and merging the outputs of each executor; and maintaining global state information and guiding the system convergence process. The central coordinator establishes a one-way communication link with all executors to achieve the unity of centralized coordination and distributed execution.

[0029] (2) Multiple executors constitute the inference execution unit, and each executor is a large model instance. The executor receives policy guidance from the coordinator and generates a problem solution independently by combining its own maintained belief state. There is no direct communication between the executors; their cooperation is achieved through the coordinator's policy guidance and the implicit alignment of belief states. The core of the executor lies in its internal belief network, which is the key to achieving local decision-making and global coordination.

[0030] The belief network is the core component of each actuator, its function being to map the actuator's local observations and historical trajectories into belief state vectors. These vectors encode the agent's probabilistic estimates of the environment and the actions of other agents, forming the basis for subsequent policy generation and equilibrium convergence. The output of the belief network directly determines the actuator's text generation parameters.

[0031] (3) The centralized hybrid network is the core of the algorithm for achieving global coordination. This network receives the local value estimates and belief states of all executors and ensures the consistency between local policy improvements and global performance enhancements through a value decomposition mechanism with monotonic constraints. The hybrid network works closely with the coordinator to guide the system toward convergence to a Bayesian Nash equilibrium.

[0032] The workflow follows a strict temporal logic: the coordinator first generates a policy and distributes it to each executor; the executors independently generate responses based on the policy and their own belief states; the coordinator collects all responses and performs aggregation processing; the system updates the parameters of each component based on the aggregation results. The entire process forms a complete feedback loop, and the system performance is gradually optimized through multiple iterations.

[0033] The subsequent sections will be developed based on this framework: Step 2 will, based on the aforementioned architecture, rigorously formalize the collaborative process among multiple agents into an incomplete information game. By defining agent types, belief systems, policy spaces, and utility functions, the conditions for Bayesian Nash equilibrium will be established, and theoretical proofs of its existence and convergence will be provided, laying a solid game theory foundation for the entire method. Step 3 will specifically implement the agent policies in the above game model, elaborating on how the belief network maps local observations and historical trajectories to belief states, and generating key parameters for controlling text generation accordingly. At the same time, a multi-objective reward function will be designed to drive the optimization of local policies. Step 4 will focus on system-level collaboration, aggregating individual beliefs to form a group representation through a belief encoder, and using a centralized hybrid network to ensure the consistency between local decisions and global goals. Finally, through dynamic weight adjustment and early stopping mechanisms, the system will be guided to converge efficiently and stably to a Bayesian Nash equilibrium state.

[0034] Using work injury determination as a case study, the central coordinator receives work injury determination application instructions from users and performs data preprocessing and task parsing. Data preprocessing includes validating ID numbers, standardizing accident occurrence times, and segmenting and extracting key information from text descriptions. During the task parsing phase, based on the analysis results of 1,000 similar historical cases, the coordinator generates a global processing strategy: "Initiate multimodal evidence collection and rule verification process."

[0035] During the framework construction process, four specialized intelligent agents were configured to work collaboratively. The OCR (Optical Character Recognition) agent extracts structured information from original application materials, including injury descriptions in diagnostic certificates and key facts in accident reports. Its processing capability is based on a deep learning-based text recognition model, achieving an accuracy of 92% after training on 5000 labeled samples. The medical image analysis agent specializes in processing medical image data, using a 3D convolutional neural network to perform layered analysis of CT images. It can identify 26 medical features such as fracture location and injury severity. This model was trained on 600 labeled CT images, achieving an accuracy of 88%. The intent recognition agent performs semantic analysis of accident descriptions based on a BERT model, extracting legal elements such as "working hours," "workplace," and "work-related reasons." The model was fine-tuned on 3000 legal documents, achieving an F1 score of 85%. The rule-based reasoning agent integrates 136 laws and regulations related to work-related injury determination and 800 historical precedents, using knowledge graph technology for logical reasoning. The reasoning accuracy achieved through cross-validation reached 90%. The outputs of each actuator (structured text, damage features, and rule compliance) are reported to the coordinator for aggregation. The centralized hybrid network synchronously receives local Q-value estimates and belief states from all actuators. Through its monotonicity guarantee mechanism, it calculates the global Q-value, ensuring that improvements in the recognition accuracy of any single actuator do not lower the overall decision-making quality. Finally, the coordinator outputs a structured conclusion and evidence package containing the statement "The facts are clear, the chain of evidence is complete, and a recommendation to recognize the injury" to complete the intelligent assisted determination of a single work-related injury case.

[0036] Step 2: Define incomplete information games and Bayesian Nash equilibrium.

[0037] Based on the completed multi-agent collaborative question answering framework, this invention formalizes the collaborative process between multiple agents into an incomplete information game model, in which each agent makes decisions based on its probabilistic beliefs about the behavior of other agents, and ensures that the entire system can converge to a stable Bayesian Nash equilibrium state through a specific learning algorithm.

[0038] Step 201: Definition of agent types and belief systems.

[0039] Based on the belief state generation mechanism established in the framework, the local historical trajectory of each actuator is... Define this agent type This contains the agent's private information. Each agent forms probabilistic beliefs about other agent types based on the observations it receives. During belief updates, agent i's belief distribution regarding other agent types at time t is calculated using Bayes' theorem. This process allows each agent to continuously refine its predictions of other agent behaviors based on historical interaction data, thereby gradually improving the accuracy of its decisions.

[0040] In the context of work-related injury determination, the types of various intelligent agents It is determined by both its historical performance and current state. Taking a two-dimensional state as an example, the type of OCR recognition agent includes image recognition accuracy. and processing speed These values ​​were calculated by statistically analyzing the agent's performance in the most recent 100 cases. The types of medical image analysis agents include those with high accuracy in injury assessment. and 3D reconstruction capabilities The accuracy of injury assessment is based on expert evaluation of 50 test cases, while the 3D reconstruction capability is based on the algorithm's performance on a standard test set. The type of agent for intent recognition includes the accuracy of legal clause matching. and semantic understanding depth These metrics are calculated based on performance on a legal text test set. The type of rule-based reasoning agent includes logical reasoning accuracy. Case matching ability This was obtained based on statistics of the accuracy of decision-making in historical cases.

[0041] During initialization, mutual trust relationships need to be established between the agents. The initial trust weights between the OCR recognition agent and the medical image analysis agent are determined. For example, this initial value is set based on the average collaborative performance of the two agents in historical cases. The prior distribution among the agents is calculated using the following formula: ; Among them, the difference measurement function Calculate the Euclidean distance between type vectors. Taking an OCR recognition agent and a medical image analysis agent as examples, their type vectors are respectively... and The difference measure is: ; In actual operation, when the information extracted by the OCR agent is consistent with the medical image judgment for 10 consecutive times, the trust weight is adjusted according to the Bayesian update rule. The update formula is as follows: ; in, Indicates the type of intelligent agent in medical imaging The empirical probability of observing consistent evidence is calculated based on 92 instances of consistency out of 100 historical interactions, and smoothed using a Beta distribution. Assuming that 92 out of 100 historical observations of the medical imaging agent's output are consistent with other evidence, Bayesian smoothing is used to calculate... The initial trust weight of the OCR agent towards other agents is: for the medical image analysis agent w 12 =0.5, for the intent recognition agent w 13 =0.3, for the rule-based reasoning agent w 14 =0.2. Substitute in the specific value: ; This update process increased the trust weight of the OCR recognition agent to the medical image analysis agent from 0.5 to 0.513. Although the increase was small, it was in the right direction and reflected the process of gradually building trust among agents through evidence consistency.

[0042] Step 202: Construction of policy space and utility function.

[0043] Based on the defined belief system, the policy space and utility function of the agents are established. Each agent's policy... It is a mapping function from the type space to the action space, determining the probability distribution of choosing each action given a type. The utility function of agent i. Defined as the expected cumulative discount reward, which includes the present value of current rewards and future rewards.

[0044] ; in, This is the immediate reward obtained by agent i at time t+k, and its specific calculation method is given by the reward function defined in subsequent step c. The utility function considers the balance between the immediate and long-term benefits obtained by the agent during the decision-making process, through a discount factor. To control the importance of future rewards, among which The value range is [0,1), and the closer the value is to 1, the more the agent values ​​long-term benefits.

[0045] In the context of work-related injury determination, the strategies of each intelligent agent It is a mapping function from the agent's type space to its action space, implemented through a three-layer neural network. The network input is the agent's type vector. Given the current observation state s, the output is a probability distribution in the action space. In the work injury determination scenario, the action space of the OCR recognition agent includes {extracting text, requesting rescanning, transferring to manual processing}, the action space of the medical image analysis agent includes {identifying injury, requesting image enhancement, suggesting expert consultation}, the action space of the intent recognition agent includes {parse elements, requesting supplementary description, transferring to manual review}, and the action space of the rule reasoning agent includes {reasoning determination, requesting supplementary evidence, suggesting expert review}.

[0046] The utility function for each agent is defined as the expected cumulative discounted reward: ; Among them, discount factor It is determined based on business time value analysis, and the calculation formula is as follows: ,in This is the time discount rate, meaning the value of future rewards decreases by 5.26% per step. Instant rewards. The reward system is based on in-depth business analysis: a +1.0 reward is given when evidence is fully accepted, representing the ideal outcome; a +0.5 reward is given when evidence is partially accepted, representing some value; a -0.5 penalty is given when evidence is rejected, to avoid invalid operations; a -0.2 penalty is given when evidence is missing, to encourage completeness; and a +0.3 reward is given when collaboration between agents is successful, to promote cooperation. These reward values ​​were determined by averaging the contributions of each action to the final determination in 1000 historical cases, independently evaluated by three experts.

[0047] Step 203: Establish the Bayesian Nash equilibrium conditions.

[0048] Based on policy space and utility function Establish the equilibrium conditions of the system. Strategy combination. To establish a Bayesian Nash equilibrium, specific conditions must be met, namely, under this policy combination, for any agent i and each possible type... Neither can increase their expected utility by unilaterally changing their own strategies.

[0049] ; in, All possible alternative strategies. This condition, rigorously defined by mathematical inequalities, ensures that in equilibrium, each agent chooses the optimal response to the strategies of other agents. The calculation of expected utility considers the probability distributions of other agent types, reflecting the characteristics of incomplete information.

[0050] In the context of work-related injury determination, establishing the Bayesian Nash equilibrium condition is a crucial step. Work-related injury cases need to ensure that all four agents, despite possessing incomplete information, can make optimal decisions, and that no single agent can gain a higher benefit by unilaterally changing its strategy.

[0051] First, consider the scenarios faced by OCR-recognized agents. Assume their type is... (Recognition accuracy 0.85, processing speed 0.9), observed state s (diagnostic evidence: medium image quality, text clarity 0.7). At this point, the OCR agent needs to extract text from the action space {...} ), request a rescan ( ), then transferred to manual processing ( Choose the optimal action from the options provided.

[0052] Based on policy π1, given type θ1 and state s, the probability distribution for choosing each action is as follows: ; Now we calculate the expected utility. Assume all other agents employ an equilibrium strategy. OCR agent selects actions to extract text. The expected utility is: ; conditional probability Calculated based on prior distribution. (Based on medical imaging agent type) For example: ; The specific calculation of expected utility needs to consider all possible combinations of types. Assuming that... Under this type, the probability of the medical imaging agent selecting "damage recognition" is 0.8 (based on historical strategy statistics). If the OCR selects text extraction and the medical imaging agent selects damage recognition, the probability of successful collaboration between the two is 0.9 (based on historical collaboration success rate), resulting in a reward of +0.3; the probability of evidence being accepted is 0.85 (based on historical acceptance rate), resulting in a reward of +1.0. Therefore: ; Similarly, the expected utility of choosing other actions can be calculated. After calculation: ; Clearly, choosing the text extraction action yields the highest expected utility, which aligns with the strategy. The probability of assigning this action a maximum of 0.6 is consistent with the previous statement.

[0053] Now we examine the equilibrium condition: Does any alternative action exist? How can we increase the expected utility? Suppose the OCR agent considers increasing the probability of requesting a rescan from 0.3 to 0.5, and correspondingly decreasing the probability of extracting text to 0.4. Recalculate the expected utility: ; This is lower than the expected utility of the original strategy, which is 0.936 (=0.6×1.05+0.3×0.82+0.1×0.45), proving that the original strategy is indeed the optimal response.

[0054] Therefore, when an OCR agent is faced with a user's diagnostic certificate, it determines the type of certificate it needs to use. Based on the image quality status 's', the text extraction action was selected according to the balancing strategy. After actual execution, key information such as "open fracture of the right index finger" was successfully identified, and the evidence was adopted by the coordinator, earning a reward of +1.0. At the same time, it was highly consistent with the medical image analysis results, earning a collaborative reward of +0.3, for a total reward of 1.3, thus verifying the effectiveness of the balancing strategy.

[0055] Step 204: Proof of Equilibrium Existence and Convergence Guarantee.

[0056] To ensure the system reaches an equilibrium state, this invention provides a theoretical guarantee of the existence of equilibrium. By verifying mathematical conditions such as the compactness and convexity of the policy space and the continuity of the utility function, it can be proven that a Bayesian Nash equilibrium necessarily exists in this game model. The convergence of the system is guaranteed by a value iteration process, which gradually approximates the optimal policy by continuously updating the state value function. Its core formula embodies the basic principles of reinforcement learning. ; in, This represents the estimated value of state s at the (k+1)th iteration. This represents the immediate reward obtained by taking action a in state s. Represents the state transition probability. It is a discount factor. This represents the next state that state s can transition to after performing action a in the k-th iteration. The estimated value function value. In the incomplete information game described in this invention, this principle is implemented in a distributed manner through the belief network and Q-function within each executor. This iterative process ensures that the value function sequence converges to the optimal value function.

[0057] In the context of work-related injury determination, proving the existence of equilibrium in this step requires mathematically ensuring that the system can reach a stable state within a finite number of steps. First, we define the agent type space. and the state space S. Where the type space... It is the Cartesian product of four agent type vectors, where each Both are compact and convex sets. The state space S contains all possible combinations of the three dimensions of evidence collection degree, analysis completion degree, and decision maturity degree, and is also a compact set.

[0058] According to the convergence guarantee requirement, the following conditions need to be verified: First, the policy space of each agent is compact and convex. In the work injury determination scenario, the policy... From type The mapping from state S to the probability distribution of the action space, since the action space is finite (such as the three actions of the OCR agent), the policy space naturally satisfies the requirements of compactness and convexity.

[0059] Secondly, utility function It is continuous in the policy space. For example, the utility function of an OCR agent can be specifically expressed as: ; The reward function R is continuous because the probability of evidence acceptance and the success rate of collaboration are both continuous functions of the policy. For example, when the medical imaging agent increases the probability of the "damage recognition" action, the probability of the OCR agent obtaining a collaborative reward will change continuously.

[0060] In the specific convergence proof, a value iteration algorithm is used. A value function is defined. Let be the maximum expected cumulative reward after k iterations starting from state s. When the initial state s0 = (evidence gathering degree 0.3, analysis completion degree 0, decision maturity degree 0), the iteration process is as follows: First iteration: ; because The above formula simplifies to: ; Assuming in In this state, the immediate rewards for each action are as follows: Submitting evidence 1.0, Waiting for supplementation 0.2, Transferring to manual assistance 0.1. .

[0061] During the second iteration: ; Assuming from Transferred to The probability of (evidence collection degree 0.6) is 0.85, and ,but: ; After k iterations, when The value function is considered to have converged when all states s hold true. In actual calculations, the user reached the convergence condition after 47 iterations, proving that equilibrium exists and can be reached.

[0062] Step 205: Solving and implementing the equilibrium strategy.

[0063] Based on the existence guarantee of equilibrium, the system solves for the equilibrium strategy through a distributed learning algorithm. Each agent i maintains a Q-value function. This function estimates the state in a given condition. and actions The expected cumulative reward is calculated. The Q-function is updated online using a temporal difference learning algorithm, with the update formula as follows: ; in, It is the learning rate, which controls the magnitude of each update; It is the immediate reward that agent i has received; It is the new state after the transfer; It is a discount factor. The learning rate needs to meet specific decay conditions to ensure that the algorithm eventually converges.

[0064] The theoretical model described above is instantiated using the components described in step 3: State The information is encoded in the belief states of each actuator. Middle; Action Specifically, this is manifested as a prompt embedding that controls text generation. The Q function The parameters are then approximated by a deep neural network. Through this belief-driven learning process, each agent can gradually adjust its strategy, ultimately guiding the entire system to a Bayesian Nash equilibrium.

[0065] In the context of work-related injury determination, the solution for this equilibrium strategy is achieved through a deep Q-learning algorithm. Taking an OCR agent as an example, the solution process is explained in detail.

[0066] First, a Q-network is constructed. The input layer contains state features (evidence quality 0.7, text clarity 0.8, etc.) and type features (accuracy 0.85, processing speed 0.9), with a total of 10 input nodes. The hidden layer uses 128 neurons and employs the ReLU activation function. The output layer corresponds to the Q-values ​​of three actions.

[0067] During training, an experience playback mechanism is used. When the OCR agent processes a user's diagnostic certificate, the generated experience samples are as follows: ; Among them, state Includes evidence quality (0.7), text clarity (0.8), and type features: accuracy (0.85), processing speed (0.9), and action. Rewards for "Extracting Text" This includes evidence acceptance 1.0 + collaboration reward 0.3, and the next state. This includes an updated evidence quality score of 0.8 and an analysis completion score of 0.9.

[0068] The Q learning update formula is: ; Assume the current Q( "Extract text")=1.2, in the next state The maximum Q-value for each action is 1.5, and the learning rate is... ,but: ; During training, the target network is used for stabilization. The parameters of the target network are updated every 100 steps. The Huber loss function is used. ; in, This represents the target Q value, which is equal to the immediate reward currently obtained. Including the discounted present value estimate of the optimal future return, and It is determined by the parameter as The Q-value calculated by the target network is copied from the online network at regular intervals to ensure training stability; It is the predicted Q-value of the online network; the Huber loss uses a quadratic function to facilitate convergence when the error is small, and uses a linear function to avoid gradient explosion when the error is large, which is suitable for approximating the value function in reinforcement learning.

[0069] After 500 rounds of training, the OCR agent's performance on the test set stabilized. The training loss decreased from the initial 2.35 to 0.12, and the Q-values ​​for each action converged to: 1.45 for extracting text, 0.82 for requesting rescanning, and 0.35 for transferring to manual processing. This indicates that the agent has learned the optimal strategy of prioritizing the "extract text" action in similar situations.

[0070] The training process for other agents is similar. The medical imaging agent learned to select "identify damage" when the CT image quality is good and "request enhanced images" when the image quality is poor; the intent recognition agent learned to select "parse elements" when the accident description is complete and "request supplementary description" when the description is ambiguous; the rule reasoning agent learned to select "reasoning determination" when the evidence is sufficient and "suggest expert review" when the evidence is contradictory.

[0071] Step 3: Belief network and local policy generation.

[0072] Building upon the incomplete information game framework established in step 2, this invention designs a complete belief network system to generate local policies for each agent. This system models the agent's belief state through a neural network and generates key parameters controlling text generation based on these belief states, ultimately forming the agent's local decision-making strategy.

[0073] In the context of work-related injury determination, each agent transforms local observation information into belief states through a belief network, and generates control parameters based on these states, ultimately forming a local strategy. Taking the aforementioned work-related injury determination case as an example, when the OCR recognition agent receives the diagnostic certificate image, it first processes the input data through the belief network to generate a belief representation of the current task state. Then, based on this belief state, it generates temperature parameters and repetition penalty parameters, and finally performs text recognition actions and collects feedback information for strategy optimization.

[0074] Step 301: Design of the belief network architecture.

[0075] Belief networks employ a deep neural network architecture to receive local historical trajectories of the agent. As input, the output is a belief state vector. The historical trajectory of the agent is processed at the input layer of the network. It contains the agent's past behavioral sequences and observations. The intermediate layers of the network consist of multiple fully connected layers, using the ReLU activation function for non-linear transformation. The output layer generates a fixed-dimensional belief-state vector. This vector encodes the agent's probabilistic estimates of the current environmental state and the actions of other agents. The dimension of the belief state vector is set according to the specific task requirements, typically choosing 128 or 256 dimensions to ensure sufficient information representation capability.

[0076] In the specific implementation of work-related injury determination, the belief network of the OCR recognition agent adopts a three-layer fully connected neural network structure, specifically designed to transform local observation information into a belief state vector. The network input dimension is 256, corresponding to the preprocessed feature vector; the hidden layer contains 128 neurons, using the ReLU activation function for non-linear transformation; the output layer generates a 128-dimensional belief state vector. Each dimension encodes the agent's probability estimate of a specific state feature.

[0077] The calculation process for the belief state vector is as follows: ; in, It is the agent's local observation information (current observation state) at time t. and It is a weight matrix. and This is the bias term. After feature extraction, a 256-dimensional feature vector is obtained from the user-input diagnostic certificate image. After transformation by the belief network, a 128-dimensional belief state vector is output, where the image quality confidence value of the 37th dimension is 0.82 and the text recognizability value of the 89th dimension is 0.91.

[0078] Step 302: Local observation information processing.

[0079] When processing local observation information, each agent first preprocesses the raw observation data. This includes the current observation information (i.e., the current observation state). This includes the current environmental state, task description, and policy guidance provided by the coordinator. This information is converted into a dense vector representation through an embedding layer, and then combined with historical trajectory information. The vectors are then concatenated. The concatenated vectors are input into a belief network for further processing. Through nonlinear transformations of a multi-layer neural network, a belief state vector with semantic meaning is finally generated. This process enables the agent to form a reasonable estimate of the global state based on limited local information.

[0080] In the specific implementation of work-related injury determination, each intelligent agent standardizes the received raw data to form a unified feature representation. In the work-related injury determination scenario, the OCR recognition agent preprocesses the diagnostic certificate image: first, Gaussian filtering is performed for noise reduction using a 3×3 convolutional kernel; then, histogram equalization is performed to enhance contrast; finally, a 256-dimensional feature vector is generated using a CNN feature extractor. The medical image analysis agent slices CT images and uses a 3D convolutional network to extract spatial features. The intent recognition agent segments and semantically encodes the accident description text, generating a text feature vector. The rule-based reasoning agent fuses features from multiple sources of evidence to form a comprehensive feature representation. After the user uploads the diagnostic certificate image and it undergoes preprocessing, the feature vector accurately encodes information such as text layout, font clarity, and background complexity within the image.

[0081] Step 303, Action Parameter Generation Mechanism.

[0082] Belief state vectors output by belief networks The system generates key parameters (i.e., action parameters) to control the text generation process. Temperature parameter. The randomness of the generated text is controlled and calculated through a differentiable transformation process: ;in, and These represent the minimum and maximum values ​​of the preset temperature parameter, respectively, used to limit the output within a reasonable physical range; It is a trainable weight matrix; It is a trainable bias term; It's a sigmoid activation function. (Repetition penalty parameter) The calculation process is similar and is used to avoid repeated text generation. ,in, and These represent the minimum and maximum values ​​of the preset action parameters, respectively. It is a trainable weight matrix. It is a trainable bias term. It is an activation function. The belief state is represented by independent weight matrices and bias terms. These two calculated parameters directly control the text generation process of the actuator, affecting the quality and diversity of the final output.

[0083] In the specific implementation of work-related injury determination, based on belief state vectors The OCR agent needs to generate key parameters that control the text recognition process. The first is the temperature parameter. This is used to control the randomness of the recognition results. ; in, , , It is a weight matrix. This is the bias term. After the user uploads data for work-related injury determination, the belief state vector undergoes a linear transformation to obtain a scalar value of 1.24. Mapping this value to the [0,1] interval using the sigmoid function yields 0.77. The final temperature parameter... This relatively high temperature value indicates that the system is confident in the quality of the diagnostic report images, and the output diversity can be appropriately increased to improve recognition accuracy.

[0084] Secondly, there is the repeated penalty parameter. This is used to control the intensity of the repetition penalty during text generation, avoiding the output of duplicate characters or phrases. ; Among them, the minimum value of repeated penalties The maximum value of repeated penalties ; It is the sigmoid function; It is a trainable weight matrix; It is the belief state vector of the agent at time t; It is a trainable bias term. It is calculated to be... This indicates that the system has certain expectations regarding the repetition pattern of the text and has set a moderately high penalty value.

[0085] In addition, scanning accuracy parameters are generated. Controlling the fineness of character segmentation: ; in, , Calculations yielded Its relatively high accuracy is suitable for processing technical terms in medical documents.

[0086] These parameters work together to guide the OCR agent to use relatively lenient temperature settings, moderate repetition penalties, and high scanning accuracy when recognizing user diagnostic reports, in order to balance the requirements of recognition speed and quality.

[0087] Step 304: Strategy Optimization and Value Estimation.

[0088] Each agent maintains a value estimation network to evaluate the long-term value of taking a specific action in the current belief state, i.e., maximizing utility in step B2. The value estimation network receives a belief state vector. and action parameters, i.e., embedded prompts As input, the network outputs a predicted value of the expected cumulative reward. The training objective of this network is to minimize the temporal difference error: ; in, It's an instant reward. It is a discount factor. and Step 204 involves value estimation for the current state and the next state. By continuously reducing the temporal difference error, the agent can gradually improve the accuracy of its value estimation, thereby generating a better decision-making strategy.

[0089] OCR agents use a value network to evaluate the long-term value of taking a specific action in the current belief state. The structure of the value network is similar to that of the belief network, but uses belief state vectors. For input, output a scalar value. , representing the expected cumulative reward of the current state.

[0090] The specific calculation process for valuation is as follows: ; in, These are parameters of the value network.

[0091] In the specific implementation of work-related injury determination, when a user undergoes work-related injury determination, the belief state vector... After inputting the value network, the output is... This indicates that, starting from the current state, the agent is expected to receive a cumulative reward of 2.34.

[0092] Policy optimization is achieved by minimizing the temporal difference error: ; During training, a target value network is used to stabilize the training. The parameters of the target value network are... Regularly copy from the online network: ; in, This is a soft update parameter. In a training step during the user's work injury assessment, the current state value... Instant rewards value of the next state Discount factor =0.95, then the timing difference error is: ; The loss value is By updating the value network parameters through backpropagation, predictions become more accurate.

[0093] Step 305: Design of multi-objective reward function.

[0094] To guide the policy optimization direction of each actuator agent and achieve the unification of local decision-making and global objectives, this invention designs a reward function composed of multi-objective weighting. The instantaneous reward obtained by each actuator agent i at time t... It consists of the following three linearly weighted parts: ; in, The action likelihood reward is obtained by calculating the cosine similarity between the agent's output text and the coordinator's final aggregated output, and is used to measure the consistency between individual output and system consensus. Task-specific rewards are calculated from evaluation signals provided by the environment or the task itself (such as the correctness of the answer to a math problem) and directly reflect the quality of task completion. To reward collaborative contributions, the coordinator evaluates the unique value and complementarity of an agent's output relative to the outputs of other agents, thus encouraging diversity and avoiding redundancy. Weighting coefficients. , , ,satisfy Its initial value is a preset hyperparameter, and it is optimized online in subsequent step 405 through a dynamic weight adjustment mechanism. The output of this reward function is directly used as the time-series difference loss in step 304. and step 202 global timing difference loss Instant reward signals .

[0095] The reward function design for an OCR agent needs to balance multiple objectives. In work injury determination cases, the reward function consists of three parts: First is the action likelihood reward The consistency between the agent's output and the coordinator's final conclusion is measured. ; in, It is the feature vector output by the agent. It is the feature vector of the coordinator's final conclusion. and These are the sets of key information extracted by the agent and the coordinator, respectively. , These are weighting coefficients. In the process of a user's work-related injury determination, the cosine similarity between the key information extracted by the OCR agent and the information ultimately adopted by the coordinator is 0.88, and the Jaccard similarity is 0.92. Therefore... .

[0096] Secondly, there are task-specific rewards. Based on recognition accuracy and efficiency: ; in, , , Accuracy is calculated by comparing the recognition results with manual annotations; efficiency is based on processing time; and resource cost is based on CPU and memory usage. In work-related injury assessment, the OCR recognition accuracy reached 94%, the processing time was 2.3 seconds (the baseline was 3 seconds), and the resource cost, after normalization, was 0.8. .

[0097] Finally, there are rewards for collaborative contributions. Evaluate the unique value and complementarity of the agent's output: ; in, , , Novelty measures whether the information provided by the agent is missing from the outputs of other agents; complementarity assesses the synergistic effect of the information with the outputs of other agents; and consistency checks the degree of agreement with other evidence. In this case, the "middle finger skin laceration" information provided by the OCR agent is not provided by other agents, resulting in a novelty score of 0.95; it is highly complementary to the medical image analysis results, with a complementarity score of 0.88; and it is consistent with all evidence, with a consistency score of 1.0. Therefore... .

[0098] The total reward is a weighted sum of three parts: ; in , , , where is the reward weighting coefficient. In this case, .

[0099] Step 306: Local strategy execution and feedback collection.

[0100] After generating the action parameters, each agent independently executes the text generation task. During execution, the agent uses temperature parameters... and repeat penalty parameters The generated text is passed to a large model to control its behavioral characteristics. The resulting text is then submitted to a coordinator for aggregation, while the system collects environmental feedback, including assessments of task completion quality and collaborative effectiveness. This feedback is used to update the parameters of the belief network and the value estimation network, forming a complete policy learning loop. Through continuous interaction and learning, each agent can gradually optimize its local policies, improving the overall system's collaborative efficiency.

[0101] In the specific implementation of work-related injury determination, after generating motion parameters, each intelligent agent independently performs its professional task. OCR-recognition agents process temperature parameters... and repeat penalty parameters The parameters are passed to the text recognition model to control the behavioral characteristics of the recognition process. The medical image analysis agent uses the generated parameters to guide CT image analysis and identify damage features. The intent recognition agent parses legal elements based on parameter settings, while the rule reasoning agent performs logical reasoning and evidence evaluation.

[0102] After execution, the system collects feedback from multiple sources. Accuracy feedback is obtained by comparing the agent's output with the actual results; efficiency feedback records processing time and resource consumption; and synergy effect feedback evaluates the complementarity and consistency of the agents' outputs. In the above case, the OCR recognition agent successfully identified key information in the diagnostic certificate with an accuracy rate of 94% and a processing time of 2.3 seconds, showing high consistency with the outputs of other agents. All feedback information is used to update the parameters of the belief network and value network, forming a complete policy learning loop.

[0103] Through continuous interaction and learning, each intelligent agent gradually optimizes its local strategies, improving the overall system's collaborative efficiency while maintaining professionalism, ultimately achieving intelligent and efficient work injury determination processes.

[0104] Step 4: Belief Coding and Global Coordination Mechanism.

[0105] In step 3, each actuator has generated its local policy and action parameters based on its own belief state. To coordinate and unify these dispersed decisions to form globally optimal swarm intelligence, this invention designs a complete belief encoding and global coordination mechanism. The aim is to effectively aggregate the belief states of each actuator output from step 3 and ensure the consistency between local decisions and the global goal through a centralized hybrid network.

[0106] In the context of work-related injury determination, belief encoding and a global coordination mechanism are core to achieving collaborative work among multiple agents. After each agent completes local reasoning and generates its own belief state, the system aggregates these scattered belief states into a unified group belief representation through a belief encoder. Then, a centralized hybrid network is used for global coordination, ultimately achieving effective collaboration among agents. Taking the aforementioned work-related injury determination case as an example, after the four agents—OCR recognition, medical image analysis, intent recognition, and rule reasoning—generate their respective belief state vectors, the system activates a global coordination mechanism to ensure that the decisions of each agent cooperate to jointly complete the work-related injury determination task.

[0107] Step 401: Design of the belief encoder architecture.

[0108] The input to the belief encoder is the set of belief state vectors generated by the belief networks of each actuator in step 301. The encoder employs a neural network based on a multi-head self-attention mechanism, aiming to uncover the intrinsic connections and dependencies between these belief state vectors.

[0109] The encoder first concatenates all belief state vectors into a matrix. Subsequently, parallel processing is performed using H independent attention heads. For each attention head h, the calculation process is as follows: ; in, , , This is a trainable weight matrix specific to this attention head, serving as a projection matrix of the query, key, and value. The outputs of all attention heads are concatenated and passed through a final linear projection layer to generate a global, comprehensive group belief representation E. This representation, as a condensed integration of the current cognitive state of the entire system, forms the basis for subsequent global coordination and decision-making.

[0110] In the specific implementation of work-related injury determination, the belief encoder adopts a neural network architecture based on a multi-head self-attention mechanism, which is responsible for aggregating the belief states of each agent into a group belief representation. The encoder input is the set of belief state vectors generated by the four agents in step 301. Each of them First, concatenate all belief state vectors into a matrix. .

[0111] Parallel processing is performed using H=4 independent attention heads. The calculation process for each attention head is as follows: ; in, and This is a trainable weight matrix specific to this attention head, which respectively... Projected onto the query, key, and value space, the output dimension is... When the OCR agent outputs an image quality confidence score of 0.82 and the medical imaging agent outputs a damage recognition confidence score of 0.88, the attention mechanism automatically enhances the weights of these two related features. The outputs of all attention heads are then concatenated and linearly projected to obtain the final group belief representation. ; in, This outputs a projection matrix, ultimately generating a 512-dimensional group belief representation E. In the above example, the belief state vectors of the four agents are processed by the encoder, and the output is the group belief representation. As input to the subsequent hybrid network, it accurately captures the degree of consensus and points of disagreement among the agents regarding the key elements of the case.

[0112] Step 402: Optimize the representation of group beliefs.

[0113] To ensure the effectiveness of the group belief representation E, it needs to be jointly optimized. One of the optimization objectives is to enable E to accurately predict future global returns. This is achieved by minimizing the global temporal difference error: ; in, It is the global temporal difference loss. It is a global reward signal. It is the discount factor (defined in part B, b2). This refers to the global Q value, which will be discussed in section d3. It is the action of actuator i at time t.

[0114] Meanwhile, to promote consistency between the final outputs of each actuator and the coordinator, a similarity difference loss is introduced. This loss encourages the improvement of the representation of each actuator after feature transformation. It should be as similar as possible to the final output C of the coordinator: ; in, It is similarity difference loss. These are hyperparameters used to control the weight of the loss term; It is the cosine similarity function; It is a feature representation obtained by transforming the belief state and action embedding of the actuator. The overall training goal of the belief encoder is to combine the above-mentioned loss. and the various actuator localities defined in part C, c4 The loss is used to form a multi-task learning objective, thereby ensuring that the group belief representation E serves both global optimization and individual performance.

[0115] In work-related injury determination cases, the optimization step 304 of the group belief representation is based on value estimation, achieved by minimizing the global temporal difference error and similarity difference loss. The global temporal difference loss ensures that the group belief representation can accurately predict future global benefits. ;in, This is the global reward signal, obtained by weighted summation of the rewards for each agent, with weights of [0.2, 0.3, 0.25, 0.25]. In the aforementioned work-related injury determination case, the global reward... This reflects the overall quality of current collaborative decision-making. When the reward weights of OCR and medical imaging agents are increased, their contributions to the global reward are correspondingly enhanced.

[0116] Simultaneously, a similarity difference loss is introduced to promote consistency between the outputs of each agent and the coordinator: ;in It's a hyperparameter. C is the representation of agent i after feature transformation, and C is the final output of the coordinator. The similarity function uses cosine similarity. The overall training objective of the belief encoder combines the above-mentioned loss with the local temporal difference loss of each agent to form a multi-task learning objective.

[0117] Step 403, Centralized Hybrid Network Design.

[0118] The centralized hybrid network is the core coordination component of this invention, and its input includes two parts: one is the local Q value of each actuator defined in step 205. The second is the group belief representation E generated in step 401. The goal of the network is to output a global Q-value. It is used to evaluate the global expected benefit of the entire joint action.

[0119] The network employs a deep neural network structure with monotonicity constraints. Its forward propagation process can be described as follows: ; in, Use a non-decreasing activation function, and all weight matrices It is constrained to be a non-negative value. This represents the hidden state of the l-th layer. This is the bias term of layer l. This key design ensures the monotonicity of the network, that is... This means that any increase in the local Q value of an actuator will not lead to a decrease in the global Q value, thus aligning local and global interests in a mechanism.

[0120] In work-related injury determination cases, the centralized hybrid network is the core coordination component of this invention, employing a deep neural network structure with monotonicity constraints. The network input includes two parts: the local Q-values ​​of the four agents calculated in step 205 for OCR recognition. Medical image analysis Intent recognition Rule-based reasoning And the group belief representation E. The forward propagation process of the network is as follows: ; Due to the non-negativity constraint of the weight matrix, when the OCR recognition agent improves image preprocessing... When the global Q value is increased from 1.45 to 1.60, The value was increased from 3.12 to 3.25 to meet the monotonicity requirement, ensuring that individual improvements inevitably lead to overall performance improvements.

[0121] Step 404: Global Coordination Training Mechanism.

[0122] During the training phase, the hybrid network learns an effective coordination policy by minimizing a comprehensive loss function. This loss function is the global TD loss defined in step 402. and similarity loss Based on this, a consistency loss term is introduced: ; in, It is a hybrid network loss. These are hyperparameters used to control the weights of consistency loss; The goal is to narrow down the local value estimates of each actuator. With global value estimation The gap between individuals and the system as a whole promotes cognitive consistency.

[0123] In the training process of the aforementioned work injury cases, the hybrid network optimizes the coordination strategy by minimizing the overall loss. Specifically, the consistency loss term is calculated as follows: = = ; This loss term prompts each agent to adjust its value estimate to be closer to the global value. During training, the target network parameters are updated via soft update rules. The system was updated incrementally to ensure training stability. After multiple iterations, the hybrid network loss in the above case decreased from 3.24 to 0.56, indicating that the system had learned an effective coordination strategy.

[0124] Step 405: Dynamic weight adjustment strategy.

[0125] Based on the multi-objective reward function defined in step 305, this invention introduces a dynamic weight adjustment mechanism to adaptively balance the weights of the three reward components: action likelihood, task specificity, and collaborative contribution. The total reward of agent i... It is given by the following formula: ; Among them, the weighting coefficient , , Instead of fixing hyperparameters, they are adjusted online using gradient descent. The adjustment is based on a dynamic reward-loss function. It measures the total reward actually obtained by agent i as defined in step 305. With an expected reward calculated based on a moving average Differences between them: ; in, This is the learning rate for weight adjustment. Through this mechanism, the system can automatically adjust the reward focus based on the current task progress and learning status, thereby finding the optimal balance point in complex multi-objective optimization and ensuring continuous improvement in collaborative performance.

[0126] During the execution of the aforementioned work-related injury determination cases, the system dynamically adjusted the reward weights based on the actual performance of each agent. The initial weights were set to... When the OCR recognition agent accurately extracts key diagnostic information but the processing speed is slow, the system adjusts the weights through gradient descent: = = =0.400016; After several adjustments, the weights were finally optimized to [0.38, 0.42, 0.20], which better suits the balance between accuracy and efficiency required in the context of work-related injury determination.

[0127] Step 5: Convergence judgment and early stopping mechanism.

[0128] Based on the global coordination training mechanism established in step 4, this invention designs a strict convergence judgment and early stopping mechanism to achieve efficient resource utilization and ensure stable and reliable system output. This mechanism continuously monitors the stability of key system indicators and automatically terminates the optimization process when preset conditions are met, thereby ensuring performance while avoiding unnecessary computational overhead.

[0129] After each training iteration, the system simultaneously calculates three core convergence criteria: First, the stability criterion for the final output is determined by calculating the final output of the coordinator in two rounds. and The stability of the system consensus is evaluated by the difference norm between the two values, provided that the difference norm is not greater than a preset threshold. At this point, the output is considered to be stable, that is... ; Second, the reward convergence criterion requires that the average instantaneous reward of all executor agents reaches or exceeds a preset performance threshold. ,Right now This indicates that the system has achieved satisfactory results in fulfilling its tasks. Third, the loss convergence criterion is determined by examining the system's total loss function. The stability of model parameters is determined by the magnitude of their variation, provided that the absolute value of the difference between the total losses of two adjacent rounds does not exceed a threshold. ,Right now ;in, ; It is a macro-level monitoring indicator used to characterize the overall optimization state of the system. Its calculation covers the local timing difference loss of each actuator as defined in step 3. The core training loss and the centralized hybrid network loss defined in step 404. .

[0130] The system determines that it has converged to a satisfactory solution near the Bayesian Nash equilibrium if and only if all three convergence conditions are met in consecutive iterations, and then triggers early stopping. This mechanism effectively prevents the system from getting stuck in infinite loops or oscillating continuously near suboptimal solutions, ensuring the reliability and practicality of the method at the operational level, while significantly improving overall computational efficiency.

[0131] During the processing of work-related injury determination cases, the system continuously monitors three convergence criteria: First, the stability criterion for the coordinator output: In continuous iterations, the difference norm of the coordinator's output conclusion vector gradually decreases from 0.45, reaching a certain value. When the first convergence condition is met.

[0132] Secondly, the average reward threshold criterion: the average reward of the four agents gradually increases from an initial 0.35 to 0.35 during training, and when it reaches... When the second convergence condition is met.

[0133] Third, the criterion for changes in total loss: total system loss. The magnitude of the change gradually decreased from the initial 0.45, until it reached... When the third convergence condition is met.

[0134] In the work-related injury determination case, when the above three conditions are met simultaneously for five consecutive training rounds, the system triggers an early stop mechanism in round 347, outputting the final determination conclusion: "The user is determined to have suffered a work-related injury, and the preliminary disability level is assessed as level ten." Compared to the preset 500 rounds of training, this saves 30.6% of computational resources while ensuring the accuracy of the determination conclusion. The entire system performs excellently on the test set, with the question-answering accuracy increasing from approximately 68.6% of the baseline model to 92.3%. When handling work-related injury determination tasks of the same complexity, this patent, compared to traditional multi-round multi-agent debate methods, can reduce the amount of tokens used by an average of 21.4%, while shortening the average processing time by 25.1%, verifying the effectiveness of the global coordination mechanism.

[0135] During the model training process for work-related injury determination cases, the system continuously monitors three key indicators to determine the convergence status. Experimental statistics show that when five consecutive training rounds simultaneously satisfy the following criteria: final output stability (coordinator output vector difference norm ≤ 0.01), average reward threshold (≥ 0.8), and total loss variation (≤ 10), convergence is achieved. -4 When these three conditions are met, the system triggers the early stop mechanism on average after 347 rounds, outputting the final determination conclusion: "The user is determined to have suffered a work-related injury, and the preliminary disability level is assessed as level ten," along with relevant work-related injury determination explanations. Compared to the preset 500-round complete training cycle, the actual number of training rounds is reduced by 153 rounds, achieving a resource saving rate of 30.6%. This data is calculated using the formula: Saving rate = (Preset rounds - Actual rounds) / Preset rounds × 100% = (500 - 347) / 500 × 100% = 30.6%.

[0136] In performance evaluation, based on a dataset containing 1000 test cases, this patented method improved the question-answering accuracy from 68.6% of the baseline model to 92.3%. Regarding efficiency, a comparison of 100 work injury determination tasks of similar complexity showed significant optimization in both resource consumption and processing time. The traditional method consumed an average of 15,342 tokens per task, while this patented method reduced it to 12,061 tokens, a 21.4% decrease in token usage. In terms of processing time, a comparison based on historical work order statistics and actual operational data showed that the traditional method had an average processing time of 4.2 days, while this patented method shortened it to 3.1 days, representing a 25.1% improvement in processing time. All experimental data were obtained from actual production environment testing and independently verified to ensure statistical significance, fully validating that this patented global coordination mechanism effectively improves system efficiency and reduces resource consumption while ensuring accuracy.

[0137] This embodiment provides a multi-agent collaborative question-answering method based on Bayesian Nash equilibrium, which significantly reduces communication and computation costs. The present invention replaces the multiple rounds of explicit message passing required in existing multi-agent interactions with a belief-driven implicit coordination mechanism, fundamentally avoiding the high communication costs and computational overhead caused by the consumption of massive tokens, and greatly improving the efficiency of multi-agent collaborative question answering in large-scale practical applications.

[0138] This embodiment provides a multi-agent collaborative question-answering method based on Bayesian Nash equilibrium, offering a robust theoretical convergence guarantee: This invention systematically introduces incomplete information game theory and Bayesian Nash equilibrium into multi-agent collaborative question-answering, ensuring that the system can stably converge to an equilibrium state within a finite number of steps. This effectively overcomes the problem that existing methods may fall into infinite circular arguments among multiple agents or oscillate near suboptimal solutions due to a lack of convergence calculation, thus improving the reliability and robustness of the method.

[0139] This embodiment provides a multi-agent collaborative question answering method based on Bayesian Nash equilibrium, which improves collaborative question answering performance. Through the collaborative design of belief network and centralized hybrid network, this invention enables each agent to maintain independent reasoning and leverage its individual strengths, while its strategies can be effectively aligned and optimized through global objectives, thereby generating a multi-agent optimization synergy effect. This results in stronger consensus-building capabilities and higher problem-solving quality in complex reasoning and planning tasks.

[0140] Example 2 This embodiment provides a multi-agent collaborative question-answering system based on Bayesian Nash equilibrium, including: The instruction acquisition module is configured to: acquire user query instructions and generate inference strategies; The execution module is configured to: invoke several agents according to the inference strategy; each agent transforms the current observation state into a belief state through a belief network based on the current observation state, and generates action parameters based on the belief state; the action parameters control the large model to perform the text generation task; and each agent maintains a value estimation network, which generates an immediate reward with the belief state and action parameters as input. The hybrid module is configured to: aggregate the results of each agent performing a text generation task to obtain the final answer; aggregate the belief states of each agent into a unified group belief representation through a belief encoder; and generate a comprehensive loss based on the group belief representation and the immediate rewards of each agent through a centralized hybrid network to update the parameters of each agent.

[0141] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0142] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a multi-agent collaborative question-answering method based on Bayesian Nash equilibrium as described in Embodiment 1 above.

[0143] Example 4 This embodiment provides a computer device, such as... Figure 2 As shown, the system includes a computer-readable storage medium 1003, a processor 1001, a communication interface 1002, and a computer program stored on the computer-readable storage medium 1003 and executable on the processor 1001. The processor 1001, communication interface 1002, and computer-readable storage medium 1003 can be connected via a bus or other means. The communication interface 1002 is used to receive and send data. When the processor 1001 executes the program, it implements the steps in the multi-agent collaborative question-answering method based on Bayesian Nash equilibrium described in Embodiment 1 above.

[0144] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-agent collaborative question-answering method based on Bayesian Nash equilibrium, characterized in that, include: Obtain the user's query command and generate an inference strategy; According to the reasoning strategy, several agents are invoked. Each agent transforms the current observation state into a belief state through a belief network based on the current observation state, and generates action parameters based on the belief state. The action parameters control the large model to perform the text generation task. Each agent maintains a value estimation network, which generates an immediate reward with the belief state and action parameters as input. The results of each agent performing the text generation task are aggregated to obtain the final answer; the belief states of each agent are aggregated into a unified group belief representation through a belief encoder; based on the group belief representation and the immediate rewards of each agent, a comprehensive loss is generated through a centralized hybrid network to update the parameters of each agent.

2. The multi-agent collaborative question-answering method based on Bayesian Nash equilibrium as described in claim 1, characterized in that, The belief state is represented as follows: ;in, It is the current observation state of agent i at time t. and It is a weight matrix. and It is a bias term.

3. The multi-agent collaborative question-answering method based on Bayesian Nash equilibrium as described in claim 1, characterized in that, The action parameters include a repetition penalty parameter, which controls the repetition of the generated text and is expressed as follows: ;in, and These represent the minimum and maximum values ​​of the preset action parameters, respectively. It is a trainable weight matrix. It is a trainable bias term. It is an activation function. It indicates a state of belief.

4. The multi-agent collaborative question-answering method based on Bayesian Nash equilibrium as described in claim 1, characterized in that, The action parameters include a temperature parameter, which is used to control the randomness of the generated text, and is expressed as follows: ;in, and These represent the minimum and maximum values ​​of the preset temperature parameter, respectively. It is a trainable weight matrix. It is a trainable bias term. It is an activation function. It indicates a state of belief.

5. The multi-agent collaborative question-answering method based on Bayesian Nash equilibrium as described in claim 1, characterized in that, The belief encoder concatenates the belief states of each agent into a matrix, processes them in parallel through multiple independent attention heads, and concatenates the outputs of all attention heads. Then, it generates a group belief representation through a linear projection layer.

6. The multi-agent collaborative question-answering method based on Bayesian Nash equilibrium as described in claim 1, characterized in that, The overall loss is a weighted sum of the global temporal difference loss, similarity difference loss, and consistency loss.

7. The multi-agent collaborative question-answering method based on Bayesian Nash equilibrium as described in claim 1, characterized in that, Instant reward for agent i for: ;in, , and These are the weighting coefficients. Dynamic reward loss function , is the learning rate for weight adjustment, and N is the number of agents; Rewards are based on the likelihood of the action. Specific rewards for the task. Rewards for contributions to collaboration.

8. A multi-agent collaborative question-answering system based on Bayesian Nash equilibrium, characterized in that, include: The instruction acquisition module is configured to: acquire user query instructions and generate inference strategies; The execution module is configured to: invoke several agents according to the inference strategy; each agent transforms the current observation state into a belief state through a belief network based on the current observation state, and generates action parameters based on the belief state; the action parameters control the large model to perform the text generation task; and each agent maintains a value estimation network, which generates an immediate reward with the belief state and action parameters as input. The hybrid module is configured to: aggregate the results of each agent performing a text generation task to obtain the final answer; aggregate the belief states of each agent into a unified group belief representation through a belief encoder; and generate a comprehensive loss based on the group belief representation and the immediate rewards of each agent through a centralized hybrid network to update the parameters of each agent.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the multi-agent collaborative question answering method based on Bayesian Nash equilibrium as described in any one of claims 1-7.

10. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the multi-agent collaborative question answering method based on Bayesian Nash equilibrium as described in any one of claims 1-7.

Images