Information inference method and apparatus

Abstract

The present application relates to the technical field of large language models, and particularly relates to an information reasoning method and device, computer equipment and a storage medium, which utilize a multi-agent debate framework, promote cooperation through task debate between agents, learn team joint actions of an agent decision network to guide the update of a single agent strategy, emphasize the behavior differences between agents, so that each agent can make incremental contributions to the task solution according to the respective preset role identity when facing complex tasks, and improve the accuracy and efficiency of the reasoning of the agent decision network.

Description

Technical Field

[0001] The present invention relates to the technical field of large-scale language models, and in particular to an information reasoning method, apparatus, computer equipment, and storage medium. Background Art

[0002] Multi-agent debate has broad application prospects in the field of artificial intelligence, especially using the generative power of large language models to solve various tasks such as coding, teamwork projects, bargaining, mathematical problems, etc.

[0003] Large language models demonstrate superior capabilities across a wide range of tasks, but they often fail at complex reasoning. Currently, single models are susceptible to model bias and cognitive degradation during reasoning. Furthermore, due to a lack of external feedback, they struggle to form diverse insights into problems, limiting their ability to handle complex tasks. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide an information reasoning method, apparatus, computer equipment and storage medium, which utilizes a multi-agent debate framework to promote collaboration through task debates between agents, learns the team joint actions of the agent decision network to guide the update of individual agent strategies, emphasizes the behavioral differences between agents, and enables each agent to incrementally contribute to the task solution according to its preset role identity when facing complex tasks, thereby improving the accuracy and efficiency of the reasoning of the agent decision network.

[0005] In a first aspect, an embodiment of the present application provides an information reasoning method, wherein an agent decision network includes a plurality of agents; the method comprises the following steps:

[0006] Obtaining state space information of several agents in a current time slot of the agent decision network and preset role prompt information, wherein the state space information of the current time slot includes question text information and agent conversation history information;

[0007] Obtaining action space information of the multiple agents in the current time slot based on the state space information and role prompt information of the multiple agents and the corresponding current time slot; conducting task debate based on the multiple agents and the corresponding action space information of the current time slot to obtain state space information of the multiple agents in the next time slot, wherein the action space information is answer text information generated by the agents based on the state space information;

[0008] The reward information of the plurality of agents in the current time slot is obtained according to reward calculation based on the action space information of the plurality of agents in the current time slot; and the role prompt information of the plurality of agents, the state space information of the corresponding current time slot, the reward information, and the state space information of the next time slot are combined to construct a training information combination of the plurality of agents in the current time slot.

[0009] The agent decision network is updated according to the training information combination of the plurality of agents in the current time slot, and the training information combination of the plurality of agents in the next time slot is repeatedly constructed according to the plurality of agents in the updated agent decision network and the corresponding role prompt information and the state space information of the next time slot, the agent decision network is updated, and the agent decision network updated for the last time is taken as a target agent decision network.

[0010] The problem text information to be processed is obtained, the problem text information to be processed is input into the plurality of agents in the target agent decision network, repeated iteration is performed according to a preset iteration number, answer text information output by the plurality of agents in the target agent decision network in the last iteration number is obtained, and the answer text information with the highest frequency of occurrence is taken as a reasoning result of the problem text information to be processed.

[0011] In a second aspect, an information reasoning apparatus based on an agent decision network is provided, and the apparatus includes:

[0012] A data obtaining module is configured to obtain state space information of a plurality of agents in a current time slot of an agent decision network and preset role prompt information, wherein the state space information of the current time slot includes problem text information and agent conversation history information.

[0013] A task debate model is configured to obtain action space information of a plurality of agents in a current time slot according to the plurality of agents and the corresponding state space information of the current time slot and role prompt information, and obtain state space information of a plurality of agents in a next time slot by task debate based on the plurality of agents and the corresponding action space information of the current time slot, wherein the action space information is answer text information generated by an agent based on state space information.

[0014] A training information combination construction module is configured to obtain reward information of a plurality of agents in a current time slot according to reward calculation based on action space information of the plurality of agents in the current time slot, and construct a training information combination of the plurality of agents in the current time slot by combining role prompt information of the plurality of agents, state space information of the corresponding current time slot, reward information, and state space information of the next time slot.

[0015] A decision network update module is configured to update the agent decision network based on the training information combination of the multiple agents in the current time slot, repeatedly construct the training information combination of the multiple agents in the updated agent decision network and the corresponding role prompt information and the state space information of the next time slot, and update the agent decision network, with the last updated agent decision network being used as the target agent decision network;

[0016] An information reasoning module is used to obtain text information of the problem to be processed, input the text information of the problem to be processed into several agents in the target agent decision network respectively, iterate repeatedly according to a preset number of iterations, obtain the answer text information output by several agents in the target agent decision network at the last iteration number, and use the answer text information with the highest frequency as the reasoning result of the text information of the problem to be processed.

[0017] In a third aspect, an embodiment of the present application provides a computer device comprising: a processor, a memory, and a computer program stored on the memory and executable on the processor; when the computer program is executed by the processor, the steps of the information reasoning method described in the first aspect are implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the information reasoning method described in the first aspect are implemented.

[0019] In an embodiment of the present application, an information reasoning method, apparatus, computer equipment, and storage medium are provided, which utilize a multi-agent debate framework to promote collaboration through task debates between agents, learn the team joint actions of the agent decision network to guide the update of individual agent strategies, emphasize the behavioral differences between agents, and enable each agent to incrementally contribute to the task solution according to its preset role identity when facing complex tasks, thereby improving the accuracy and efficiency of the reasoning of the agent decision network.

[0020] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flowchart of an information reasoning method provided in one embodiment of the present application;

[0022] Figure 2 A schematic diagram of the process of step S2 in the information reasoning method provided in one embodiment of the present application;

[0023] Figure 3A schematic diagram of the process of step S3 in the information reasoning method provided in one embodiment of the present application;

[0024] Figure 4 A schematic diagram of the process of step S4 in the information reasoning method provided in one embodiment of the present application;

[0025] Figure 5 A schematic diagram of the process of S42 in the information reasoning method provided in one embodiment of the present application;

[0026] Figure 6 A schematic diagram of the flow of S4 in the information reasoning method provided in another embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of an information reasoning device based on an agent decision network provided in one embodiment of the present application;

[0028] Figure 8 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0030] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0031] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" / "if" as used herein may be interpreted as "at the time of" or "when" or "in response to a determination."

[0032] The executor of the information reasoning method is the reasoning device of the information reasoning method (hereinafter referred to as the reasoning device). The reasoning device can be implemented by software and / or hardware, and the information reasoning method can be implemented by software and / or hardware. The reasoning device can be composed of two or more physical entities, or it can be composed of one physical entity. The hardware pointed to by the reasoning device essentially refers to computer equipment. For example, the reasoning device can be a computer, a mobile phone, a tablet or an interactive tablet. In an optional embodiment, the reasoning device can specifically be a server, or a server cluster composed of multiple computer devices.

[0033] See also Figure 1 , Figure 1 A flowchart of an information reasoning method provided in one embodiment of the present application is provided, wherein the method comprises the following steps:

[0034] S1: Obtain the state space information of several agents in the current time slot of the agent decision network and the preset role prompt information.

[0035] The agent decision network includes several agents, each of which collaborates on the same task through multiple rounds of debate, with each round serving as a time slot. The agents are constructed based on the Large Language Model (LLM) architecture.

[0036] In this embodiment, the reasoning device obtains the state space information of several agents in the current time slot of the agent decision network and the preset role prompt information, wherein the state space information of the current time slot includes the question text information and the agent conversation history information; the agent conversation history information is the text information record generated by the agent in the task debate with other agents, which is recorded as {u 0 ,u 1 ,...,u t-1},u 0 is the agent conversation history information of the initial time slot, u t-1 is the agent conversation history information of the t-1th time slot.

[0037] The role prompt information includes: "Take the opinions of other intelligent agents as supplementary suggestions", "Please stick to your own views in the debate" and "Focus on the answers of other intelligent agents as references" to activate intelligent agents as diverse roles, promoting effective collaboration within the intelligent agent decision-making network in the current environment.

[0038] S2: Based on the state space information and role prompt information of the several intelligent agents and the corresponding current time slot, the action space information of the several intelligent agents in the current time slot is obtained; based on the action space information of the several intelligent agents and the corresponding current time slot, task debate is conducted to obtain the state space information of the several intelligent agents in the next time slot.

[0039] In this embodiment, the reasoning device obtains the action space information of several agents in the current time slot based on the state space information and role prompt information of the several agents and the corresponding current time slot. Specifically, the action space information is the answer text information generated by the agent based on the state space information and the role prompt information, which is recorded as is the action space information of the ith agent in the tth time slot, w n The token indexed at the nth position indicates the character corresponding to the nth position index in the answer text information.

[0040] The reasoning device conducts task debate based on several intelligent agents and the corresponding action space information of the current time slot to obtain the state space information of several intelligent agents in the next time slot.

[0041] The agent includes a role-aware network; see Figure 2 , Figure 2 The flowchart of S2 in the information reasoning method provided in one embodiment of the present application includes steps S21 to S22, which are specifically as follows:

[0042] S21: performing embedding processing based on the plurality of intelligent agents and corresponding role prompt information to obtain role embedding information of the plurality of intelligent agents.

[0043] The role perception network is an RNN convolutional neural network with role guidance that can enhance role differentiation.

[0044] In this embodiment, the inference device performs embedding processing based on the plurality of said agents and the corresponding role prompt information to obtain the role embedding information of the plurality of said agents, so as to enhance the unique features associated with the preset roles of the agents.

[0045] S22: Input the role embedding information of several of the intelligent agents and the corresponding state space information of the current time slot into the role perception network of the corresponding intelligent agent respectively, and obtain the action space information of several intelligent agents in the current time slot according to the preset action space information generation algorithm.

[0046] The action space information generation algorithm is:

[0047]

[0048] wherein, is the action space information of the i-th agent in the t-th time slot, is the state space information of the i-th agent in the t-th time slot, e i is the role embedding information of the i-th agent, RNN i (·) is the processing function of the role perception network of the i-th agent, π i (·) is the policy function in the role perception network of the i-th agent.

[0049] In this embodiment, the inference device inputs the role embedding information of a plurality of agents and the state space information of the corresponding current time slot into the role perception network in the corresponding agent, respectively, and obtains the action space information of a plurality of agents in the current time slot according to a preset action space information generation algorithm.

[0050] S3: performing reward calculation according to the action space information of a plurality of agents in the current time slot to obtain reward information of a plurality of agents in the current time slot; combining the role prompt information of a plurality of agents, the state space information of the corresponding current time slot, the reward information, and the state space information of the next time slot to construct a training information combination of a plurality of agents in the current time slot.

[0051] In this embodiment, the inference device performs reward calculation according to the action space information of a plurality of agents in the current time slot to obtain reward information of a plurality of agents in the current time slot, which can be used as feedback to perfect the LLM strategy, thereby enhancing the consistency of its action and expected result.

[0052] The inference device combines the role prompt information of a plurality of agents, the state space information of the corresponding current time slot, the reward information, and the state space information of the next time slot to construct a training information combination of a plurality of agents in the current time slot.

[0053] Please refer to Figure 3 , Figure 3 is the flowchart of S3 in the information inference method provided by an embodiment of the present application, which includes steps S31-S32, and specifically as follows:

[0054] S31: obtaining standard answer information of a plurality of agents.

[0055] In this embodiment, the inference device obtains standard answer information of a plurality of agents, wherein the standard answer information is answer text information corresponding to question text information in the corresponding state space information.

[0056] S32: Based on the action space information of several intelligent agents in the current time slot and the standard answer information, a cosine similarity calculation method is used to obtain the cosine similarity between the action space information of several intelligent agents in the current time slot and the standard answer information as the reward information.

[0057] In this embodiment, the inference device uses a cosine similarity calculation method based on the action space information of several intelligent agents in the current time slot and the standard answer information to obtain the cosine similarity between the action space information of several intelligent agents in the current time slot and the standard answer information as the reward information. A higher cosine similarity indicates that the answer text information matches the standard answer information more closely, and thus a higher reward is obtained.

[0058] S4: Based on the training information combination of several agents in the current time slot, the agent decision network is updated; based on the several agents in the updated agent decision network and the corresponding role prompt information and the state space information of the next time slot, the training information combination of several agents in the next time slot is repeatedly constructed, the agent decision network is updated, and the last updated agent decision network is used as the target agent decision network;

[0059] In this embodiment, the inference device updates the agent decision network according to the combination of training information of several agents in the current time slot, and uses the last updated agent decision network as the target agent decision network.

[0060] The reasoning device repeatedly constructs the training information combination of several agents in the next time slot based on the updated several agents in the agent decision network and the corresponding role prompt information and the state space information of the next time slot, updates the agent decision network, improves the role perception network output action space information, and conducts efficient and high-quality task debate.

[0061] See also Figure 4 , Figure 4 The flowchart of S4 in the information reasoning method provided in one embodiment of the present application includes steps S41 to S42, which are specifically as follows:

[0062] S41: According to the state space information of the next time slot and the role prompt information in the training information combination of the intelligent agent, the action space information of several intelligent agents in the next time slot is obtained; the reward information of the current time slot in the training information combination of several intelligent agents in the current time slot and the state space information and action space information of the next time slot are input into the target network, and according to the preset individual value calculation algorithm, the individual value parameters of several intelligent agents in the current time slot are obtained.

[0063] In this embodiment, the inference device obtains the action space information of several agents in the next time slot based on the state space information of the next time slot in the training information combination of the agent and the role prompt information. The specific implementation method can refer to steps S21 to S22 and will not be repeated here.

[0064] The inference device inputs the reward information of the current time slot and the state space information and action space information of the next time slot in the training information combination of the multiple agents in the current time slot into the target network, and obtains the individual value parameters of the multiple agents in the current time slot according to a preset individual value calculation algorithm, wherein the individual value calculation algorithm is:

[0065]

[0066] Where, is the individual value parameter of the ith agent in the tth time slot, r is the reward information of the tth time slot, γ is the discount factor, E(·) is the expected calculation function, To find the maximum function, Q(·) is the target network function, is the state space information of the i-th agent in the state space information of the t+1-th time slot, is the action space information of the i-th agent in the t+1-th time slot.

[0067] S42: Calculate the global value parameters of the agent decision network of the current time slot based on the individual value parameters of the several agents in the current time slot and the hybrid network, and obtain the global value parameters of the agent decision network of the current time slot; construct a first loss value using the reinforcement learning method based on the reward information of the several agents in the current time slot and the global value parameters of the agent decision network, and update the role perception networks of the several agents in the agent decision network based on the first loss value.

[0068] In this embodiment, the inference device calculates the global value parameters based on the individual value parameters of several intelligent agents in the current time slot and the hybrid network to obtain the global value parameters of the intelligent agent decision network in the current time slot, wherein the global value parameters include a first global value parameter and a second global value parameter.

[0069] The inference device constructs a first loss value using a reinforcement learning method based on the reward information of several agents in the current time slot and the global value parameters of the agent decision network, and updates the role perception networks of several agents in the agent decision network based on the first loss value.

[0070] See also Figure 5 , Figure 5The flowchart of S42 in the information reasoning method provided for an embodiment of the present application includes steps S421-S423, and details are as follows:

[0071] S421: combine individual value parameters of the plurality of agents in the current time slot to construct a first individual value parameter set of the agent decision network in the current time slot; compare the individual value parameters of the plurality of agents in the current time slot to obtain a maximum individual value parameter in the current time slot, replace the individual value parameters of the plurality of agents in the current time slot with the maximum individual value parameter, and combine the individual value parameters of the plurality of agents in the current time slot after the replacement to construct a second individual value parameter set of the agent decision network in the current time slot.

[0072] In this embodiment, the reasoning device combines individual value parameters of the plurality of agents in the current time slot to construct a first individual value parameter set of the agent decision network in the current time slot, compares the individual value parameters of the plurality of agents in the current time slot to obtain a maximum individual value parameter in the current time slot, replaces the individual value parameters of the plurality of agents in the current time slot with the maximum individual value parameter, and combines the individual value parameters of the plurality of agents in the current time slot after the replacement to construct a second individual value parameter set of the agent decision network in the current time slot.

[0073] S422: respectively take the first individual value parameter set and the second individual value parameter set of the agent decision network in the current time slot as an input parameter set of the hybrid network, perform global value parameter calculation according to the input parameter set, and obtain a first global value parameter and a second global value parameter of the agent decision network in the current time slot.

[0074] The hybrid network includes a plurality of fully connected layers, and parameters of the fully connected layers are determined by a super network. An input of the super network is a natural language text H describing a problem to be solved. In order to map to the same vector space, H is first input into a critic network to obtain a vector representation H thereof. Then, this vector and a linear layer are used to calculate parameters w and b of the hybrid network. Consistency between each agent and all agents in the agent decision network is ensured through the hybrid network, and it is ensured that each agent effectively contributes to the team task of the agent decision network.

[0075] In this embodiment, the reasoning device respectively takes the first individual value parameter set and the second individual value parameter set of the agent decision network in the current time slot as an input parameter set of the hybrid network, performs global value parameter calculation according to the input parameter set, and obtains a first global value parameter and a second global value parameter of the agent decision network in the current time slot.

[0076] In an optional embodiment, the inference device constructs a consistency constraint, calculates a global value parameter based on the input parameter set and the consistency constraint, and obtains a first global value parameter and a second global value parameter of the agent decision network of the current time slot, so as to ensure that the obtained first global value parameter is proportional to the individual value parameters of the plurality of agents in the current time slot, and the obtained second global value parameter is proportional to the maximum individual value parameter of the current time slot, so as to indicate that when the individual value parameter of each agent reaches its maximum individual value, the obtained first global value parameter and the obtained second global value parameter also reach their maximum individual value, wherein the consistency constraint is:

[0077]

[0078] Where Q tot is the global value parameter, Q i is the individual value parameter of the i-th agent.

[0079] S423: Accumulate the reward information of several agents in the current time slot to obtain the cumulative reward information of the agent decision network in the current time slot; obtain a first loss value based on the cumulative reward information of the agent decision network in the current time slot, the first global value parameter and the second global value parameter and the preset first loss algorithm, and update the role perception network of several agents in the agent decision network based on the first loss value.

[0080] In this embodiment, the inference device accumulates the reward information of several agents in the current time slot to obtain the cumulative reward information of the agent decision network in the current time slot.

[0081] The inference device obtains a first loss value based on the accumulated reward information of the agent decision network in the current time slot, the first global value parameter, the second global value parameter, and a preset first loss algorithm, and updates the role perception networks of the multiple agents in the agent decision network based on the first loss value, wherein the first loss algorithm is:

[0082]

[0083] Where, L mix is, T is the total number of time slots, R t is the cumulative reward information of the agent decision network at the tth time slot, γ is the discount factor, is the first global value parameter of the agent decision network at the t-th time slot, is the second global value parameter of the agent decision network at the tth time slot.

[0084] By introducing the loss calculated by the hybrid network to update the agent's actor network, it promotes coordination between multiple agents within the agent decision network to promote global optimization and ensure that the action space information generated by each agent contributes optimally and consistently to solving the problem, thereby ensuring that each agent contributes incrementally to the task solution and improving the accuracy and efficiency of the task debate of the agent decision network.

[0085] The agent decision network also includes an inference network; see Figure 6 , Figure 6 The flowchart of S4 in the information reasoning method provided in another embodiment of the present application includes steps S43 to S45, which are specifically as follows:

[0086] S43: Obtain the observation values ​​of several intelligent agents in the current time slot, input the observation values ​​of several intelligent agents in the current time slot into the inference network, and obtain the Gaussian distribution of several intelligent agents in the current time slot; sample the Gaussian distribution of several intelligent agents in the current time slot respectively to obtain the potential variable set of the current time slot.

[0087] In order to enhance the respective expertise of each agent, in this embodiment, the inference device obtains the observation values ​​of several agents in the current time slot, inputs the observation values ​​of several agents in the current time slot into the inference network for Gaussian distribution modeling, and obtains a set of latent variables in the current time slot, wherein the latent variable set includes the latent variables of several agents, and the latent variables represent the unique characteristics of the agents, which are used for model training, enhance the robustness and uncertainty of the reasoning task, thereby enhancing the diversity among the agents and making them differentiated from each other.

[0088] S44: Obtain the dissimilarity parameters between the multiple agents in the current time slot according to the Gaussian distribution of the multiple agents in the current time slot, the potential variable set, and the preset dissimilarity calculation algorithm.

[0089] The dissimilarity calculation algorithm is:

[0090]

[0091] Where D φ (i, j) is the dissimilarity parameter between the ith agent and the jth agent, KL(·) is the KL divergence calculation function, are the Gaussian distributions of the i-th and j-th agents, respectively, z i 、z j are the latent variables of the i-th and j-th agents respectively, b and c are the first and second balance coefficients respectively.

[0092] In this embodiment, the inference device obtains the dissimilarity parameters between the several agents in the current time slot by calculating the KL divergence between the corresponding Gaussian distributions and the mutual information between the latent variables based on the Gaussian distribution of the several agents in the current time slot, the set of latent variables and the preset dissimilarity calculation algorithm, so as to reflect the mutual dependence and influence between the agents.

[0093] S45: Obtain a second loss value based on the observation values, action space information, Gaussian distribution, dissimilarity parameters between several agents, a set of latent variables and a preset second loss algorithm of several agents in the current time slot, and update the role perception networks of several agents in the agent decision network based on the first loss value and the second loss value.

[0094] The second loss algorithm is:

[0095]

[0096] Where, L dis is the second loss value, w MI is the first weight parameter, is the potential variable of the i-th agent, a i is the action space information of the i-th agent, w KL is the first weight parameter, MI(·) is the MI divergence calculation function, p(·|·) is the conditional probability distribution calculation function, o i is the observation value of the i-th agent, o j is the observation value of the jth agent, w DI is the second weight parameter, w H is the fourth weight parameter, H(Z) is the entropy of the latent variable set, and Z is the latent variable set.

[0097] In this embodiment, the inference device estimates the potential variable z according to the observation values ​​of the multiple agents in the current time slot, the action space information, the Gaussian distribution, the dissimilarity parameters between the multiple agents, the latent variable set and the preset second loss algorithm. i and action space information a i The mutual information between them estimates the coexistence between the two and predicts the latent variable z i Its conditional probability distribution p(z i |o i ), strengthen the association between the latent variable distribution and the observed value, and combine the calculated dissimilarity D between several intelligent agents. φ (o i ,o j ) and information entropy to obtain the second loss value.

[0098] The inference device updates the role perception networks of the plurality of agents in the agent decision network based on the first loss value and the second loss value. Specifically, the inference device accumulates the first loss value and the second loss value to obtain a total loss value, and updates the role perception networks of the plurality of agents in the agent decision network based on the total loss value, ensuring that the update process not only focuses on maximizing the utility of each agent's strategy and obtaining individual value parameters, but also emphasizes the behavioral differences between agents, so that each agent can incrementally contribute to the task solution according to its preset role identity when facing a complex task, thereby improving the accuracy and efficiency of the reasoning of the agent decision network.

[0099] S5: Obtain the text information of the problem to be processed, input the text information of the problem to be processed into several agents in the target agent decision network respectively, iterate repeatedly according to a preset number of iterations, obtain the answer text information output by several agents in the target agent decision network in the last iteration number, and use the answer text information with the highest frequency as the inference result of the text information of the problem to be processed.

[0100] In this embodiment, the inference device obtains text information of the question to be processed, wherein the text information of the question to be processed may be input by a user or obtained from a preset database.

[0101] The inference device inputs the text information of the problem to be processed into several updated intelligent agents in the intelligent agent decision network respectively, and iterates repeatedly according to the preset number of iterations to obtain the answer text information output by several updated intelligent agents for the last number of iterations, and takes the answer text information with the highest frequency as the inference result of the text information of the problem to be processed.

[0102] Utilizing a multi-agent debate framework, we promote collaboration through task debates between agents, learn the team joint actions of the agent decision network to guide the update of individual agent strategies, and emphasize the behavioral differences between agents. This allows each agent to incrementally contribute to the task solution according to its preset role identity when facing complex tasks, thereby improving the accuracy and efficiency of the agent decision network's reasoning.

[0103] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of an information reasoning device based on an agent decision network provided in one embodiment of the present application. The device can implement all or part of the information reasoning device based on the agent decision network through software, hardware, or a combination of both. The device 7 includes:

[0104] A data acquisition module 71 is configured to obtain state space information of multiple agents in the current time slot of the agent decision network and preset role prompt information, wherein the state space information of the current time slot includes question text information and agent conversation history information;

[0105] The task debate model 72 is configured to obtain action space information of the plurality of agents in the current time slot based on the state space information of the plurality of agents and the corresponding current time slot and the role prompt information; perform task debate based on the plurality of agents and the corresponding action space information of the current time slot to obtain state space information of the plurality of agents in the next time slot;

[0106] The training information combination construction module 73 is configured to calculate rewards based on the action space information of the multiple agents in the current time slot to obtain reward information for the multiple agents in the current time slot; and to combine the role prompt information of the multiple agents, the corresponding state space information of the current time slot, the reward information, and the state space information of the next time slot to construct a training information combination for the multiple agents in the current time slot.

[0107] A decision network updating module 74 is configured to update the agent decision network based on the training information combination of the multiple agents in the current time slot, and repeatedly construct the training information combination of the multiple agents in the next time slot based on the multiple agents in the updated agent decision network and the corresponding role prompt information and the state space information of the next time slot, thereby updating the agent decision network.

[0108] The information reasoning module 75 is used to obtain the text information of the problem to be processed, input the text information of the problem to be processed into several agents in the target agent decision network respectively, iterate repeatedly according to a preset number of iterations, obtain the answer text information output by several agents in the target agent decision network at the last iteration number, and use the answer text information with the highest frequency as the reasoning result of the text information of the problem to be processed.

[0109] In an embodiment of the present application, the state space information of several agents in the current time slot of the agent decision network and the preset role prompt information are obtained through the data acquisition module, wherein the state space information of the current time slot includes question text information and agent conversation history information; the action space information of several agents in the current time slot is obtained based on the state space information and role prompt information of several agents and the corresponding current time slot through the task debate model; task debate is performed based on the several agents and the corresponding action space information of the current time slot to obtain the state space information of several agents in the next time slot; reward calculation is performed based on the action space information of several agents in the current time slot through the training information combination construction module to obtain the reward information of several agents in the current time slot; the role prompt information of several agents, the corresponding state space information of the current time slot, the reward information and the next time slot are combined. The state space information of the target agent decision network is combined to construct the training information combination of several agents in the current time slot; the agent decision network is updated according to the training information combination of several agents in the current time slot through the decision network update module, and the training information combination of several agents in the next time slot is repeatedly constructed according to the several agents in the updated agent decision network and the corresponding role prompt information and the state space information of the next time slot, and the agent decision network is updated; the information reasoning module is used to obtain the text information of the problem to be processed, input the text information of the problem to be processed into several agents in the target agent decision network respectively, and iterate repeatedly according to the preset number of iterations to obtain the answer text information output by several agents in the target agent decision network of the last iteration number, and use the answer text information with the highest frequency as the reasoning result of the text information of the problem to be processed. Utilizing a multi-agent debate framework, we promote collaboration through task debates between agents, learn the team joint actions of the agent decision network to guide the update of individual agent strategies, and emphasize the behavioral differences between agents. This allows each agent to incrementally contribute to the task solution according to its preset role identity when facing complex tasks, thereby improving the accuracy and efficiency of the agent decision network's reasoning.

[0110] Please refer to Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in one embodiment of the present application. The computer device 8 includes: a processor 81, a memory 82, and a computer program 83 stored in the memory 82 and executable on the processor 81; the computer device may store multiple instructions, which are suitable for being loaded and executed by the processor 81. Figures 1 to 6 The specific execution process can be found in the method steps shown in Figures 1 to 6 The specific description shown will not be repeated here.

[0111] The processor 81 may include one or more processing cores. The processor 81 utilizes various interfaces and circuits to connect various components within the server. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 82 and accessing data within the memory 82, the processor 81 performs various functions and processes data of the information reasoning device 7 based on the agent decision network. Optionally, the processor 81 may be implemented in the form of at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 81 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content required to be displayed on the touch screen; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 81 and may be implemented as a separate chip.

[0112] Among them, the memory 82 may include a random access memory 82 (Random Access Memory, RAM), and may also include a read-only memory 82 (Read-Only Memory). Optionally, the memory 82 includes a non-transitory computer-readable storage medium. The memory 82 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 82 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 82 may also be optionally at least one storage device located away from the aforementioned processor 81.

[0113] The embodiment of the present application also provides a storage medium, which can store multiple instructions, which are suitable for the processor to load and execute the above Figures 1 to 6 The specific execution process can be found in the method steps shown in Figures 1 to 6 The specific description shown will not be repeated here.

[0114] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0115] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0116] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraint algorithm of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0117] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0119] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0120] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned various method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form.

[0121] The present invention is not limited to the above-mentioned embodiments. If various changes or modifications of the present invention do not depart from the spirit and scope of the present invention, and if these changes and modifications fall within the scope of the claims of the present invention and equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. An information reasoning method, wherein the agent decision network includes a plurality of agents; characterized in that: The method comprises the following steps: Obtaining state space information of several agents in a current time slot of the agent decision network and preset role prompt information, wherein the state space information of the current time slot includes question text information and agent conversation history information; Obtaining action space information of the multiple agents in the current time slot based on the state space information and role prompt information of the multiple agents and the corresponding current time slot; conducting task debate based on the multiple agents and the corresponding action space information of the current time slot to obtain state space information of the multiple agents in the next time slot, wherein the action space information is answer text information generated by the agents based on the state space information; Reward calculation is performed based on the action space information of the multiple agents in the current time slot to obtain reward information of the multiple agents in the current time slot; role prompt information of the multiple agents, corresponding state space information of the current time slot, reward information and state space information of the next time slot are combined to construct a training information combination of the multiple agents in the current time slot; The agent decision network is updated based on the training information combination of the multiple agents in the current time slot, and the training information combination of the multiple agents in the updated agent decision network and the corresponding role prompt information and the state space information of the next time slot is repeatedly constructed to update the agent decision network, and the agent decision network is updated last, and the target agent decision network is used; Obtain text information of the problem to be processed, input the text information of the problem to be processed into several agents in the target agent decision network respectively, iterate repeatedly according to a preset number of iterations, obtain answer text information output by several agents in the target agent decision network at the last iteration number, and use the answer text information with the highest frequency as the inference result of the text information of the problem to be processed.

2. The information inference method according to claim 1, characterized in that: The intelligent agent includes a role perception network, which is an RNN convolutional neural network; The method of obtaining action space information of several agents in the current time slot according to the state space information of several agents and the corresponding current time slot and the role prompt information comprises the steps of: Performing embedding processing according to the plurality of intelligent agents and corresponding role prompt information to obtain role embedding information of the plurality of intelligent agents; The role embedding information of the agents and the state space information of the corresponding current time slot are respectively input into the role perception network of the corresponding agents, and the action space information of the agents in the current time slot is obtained according to the preset action space information generation algorithm, wherein the action space information generation algorithm is: Where, is the action space information of the ith agent in the tth time slot, is the state space information of the i-th agent in the state space information of the t-th time slot, e i Embedding information for the role of the i-th agent, RNN i (·) is the processing function of the role perception network of the i-th agent, π i (·) is the policy function in the role perception network of the i-th agent.

3. The information inference method according to claim 2, characterized in that: The reward calculation is performed based on the action space information of the multiple agents in the current time slot to obtain the reward information of the multiple agents in the current time slot, including the steps of: Obtaining standard answer information of a plurality of the intelligent agents, wherein the standard answer information is answer text information corresponding to the question text information in the corresponding state space information; According to the action space information of several intelligent agents in the current time slot and the standard answer information, a cosine similarity calculation method is used to obtain the cosine similarity between the action space information of several intelligent agents in the current time slot and the standard answer information as the reward information.

4. The information inference method according to claim 3, characterized in that: The agent further includes a target network; the agent decision network includes a hybrid network, and the hybrid network includes a recurrent neural network and a fully connected layer; The updating of the agent decision network according to the combination of training information of the plurality of agents in the current time slot comprises the steps of: Obtaining action space information of multiple agents in the next time slot based on the state space information of the next time slot in the training information combination of the agent and the role prompt information; The reward information of the current time slot and the state space information and action space information of the next time slot in the training information combination of the multiple agents in the current time slot are input into the target network, and the individual value parameters of the multiple agents in the current time slot are obtained according to a preset individual value calculation algorithm, wherein the individual value calculation algorithm is: Where, is the individual value parameter of the ith agent in the tth time slot, is the reward information of the ith agent in the tth time slot, γ is the discount factor, E(·) is the expected calculation function, To find the maximum function, Q(·) is the target network function, is the state space information of the i-th agent in the state space information of the t+1-th time slot, is the action space information of the i-th agent in the t+1-th time slot; The global value parameters are calculated based on the individual value parameters of several intelligent agents in the current time slot and the hybrid network to obtain the global value parameters of the intelligent agent decision network in the current time slot; based on the reward information of several intelligent agents in the current time slot and the global value parameters of the intelligent agent decision network, a reinforcement learning method is used to construct a first loss value, and the role perception networks of several intelligent agents in the intelligent agent decision network are updated according to the first loss value.

5. The information inference method according to claim 4, characterized in that: The global value parameters include a first global value parameter and a second global value parameter; The method comprises the following steps: calculating the global value parameters based on the individual value parameters of the multiple agents in the current time slot and the hybrid network to obtain the global value parameters of the agent decision network in the current time slot; and constructing the first loss value using the reinforcement learning method based on the reward information of the multiple agents in the current time slot and the global value parameters of the agent decision network. The individual value parameters of several agents in the current time slot are combined to construct a first individual value parameter set of the agent decision network of the current time slot; the individual value parameters of several agents in the current time slot are compared to obtain the maximum individual value parameter of the current time slot, the individual value parameters of several agents in the current time slot are replaced with the maximum individual value parameter, and the individual value parameters of several agents in the current time slot after replacement are combined to construct a second individual value parameter set of the agent decision network of the current time slot; The first and second agent value parameter sets of the agent decision network of the current time slot are respectively used as input parameter sets of the hybrid network, and a global value parameter is calculated based on the input parameter sets to obtain the first and second global value parameters of the agent decision network of the current time slot; The reward information of several agents in the current time slot is accumulated to obtain the cumulative reward information of the agent decision network in the current time slot; a first loss value is obtained according to the cumulative reward information of the agent decision network in the current time slot, the first global value parameter, the second global value parameter, and a preset first loss algorithm; and the role perception networks of several agents in the agent decision network are updated according to the first loss value, wherein the first loss algorithm is: Where, L mix is the first loss value, T is the total number of time slots, R t is the cumulative reward information of the agent decision network at the tth time slot, γ is the discount factor, is the first global value parameter of the agent decision network at the t-th time slot, is the second global value parameter of the agent decision network at the tth time slot.

6. The information inference method according to claim 5, characterized in that: The agent decision network also includes a reasoning network; The updating of the agent decision network according to the combination of training information of the plurality of agents in the current time slot further comprises the steps of: Obtaining observation values ​​of a plurality of agents in a current time slot, inputting the observation values ​​of the plurality of agents in the current time slot into the inference network to obtain a Gaussian distribution of the plurality of agents in the current time slot; sampling the Gaussian distribution of the plurality of agents in the current time slot respectively to obtain a latent variable set of the current time slot, wherein the latent variable set includes latent variables of the plurality of agents; According to the Gaussian distribution of the plurality of agents in the current time slot, the potential variable set and the preset dissimilarity calculation algorithm, the dissimilarity parameters between the plurality of agents in the current time slot are obtained, wherein the dissimilarity calculation algorithm is: Where D Φ (i, j) is the dissimilarity parameter between the ith agent and the jth agent, KL(·) is the KL divergence calculation function, are the Gaussian distributions of the i-th and j-th agents, respectively, z i 、z j are the latent variables of the i-th and j-th agents, b and c are the first and second balance coefficients, respectively; A second loss value is obtained based on the observation values, action space information, Gaussian distribution, dissimilarity parameters between the multiple agents, a set of latent variables, and a preset second loss algorithm of the multiple agents in the current time slot. The role perception networks of the multiple agents in the agent decision network are updated based on the first loss value and the second loss value, wherein the second loss algorithm is: Where, L dis is the second loss value, w MI is the first weight parameter, z i is the latent variable of the i-th agent, a i is the action space information of the i-th agent, w KL is the first weight parameter, MI(·) is the MI divergence calculation function, p(·|·) is the conditional probability distribution calculation function, o i is the observation value of the i-th agent, o j is the observation value of the jth agent, w DI is the second weight parameter, w H is the fourth weight parameter, H(Z) is the entropy of the latent variable set, and Z is the latent variable set.

7. An information reasoning device based on an agent decision network, wherein the agent decision network includes a plurality of agents, characterized in that: include: A data acquisition module is used to obtain state space information of multiple agents in the current time slot of the agent decision network and preset role prompt information, wherein the state space information of the current time slot includes question text information and agent conversation history information; The task debate model is configured to obtain action space information of the multiple agents in the current time slot based on the state space information and role prompt information of the multiple agents and the corresponding current time slot; conduct task debate based on the multiple agents and the corresponding action space information of the current time slot to obtain state space information of the multiple agents in the next time slot, wherein the action space information is answer text information generated by the agents based on the state space information; A training information combination construction module is used to calculate rewards based on the action space information of the multiple agents in the current time slot to obtain reward information for the multiple agents in the current time slot; and to combine the role prompt information of the multiple agents, the corresponding state space information of the current time slot, the reward information, and the state space information of the next time slot to construct a training information combination for the multiple agents in the current time slot; A decision network update module is configured to update the agent decision network based on the training information combination of the multiple agents in the current time slot, repeatedly construct the training information combination of the multiple agents in the updated agent decision network and the corresponding role prompt information and the state space information of the next time slot, and update the agent decision network, with the last updated agent decision network being used as the target agent decision network; An information reasoning module is used to obtain text information of the problem to be processed, input the text information of the problem to be processed into several agents in the target agent decision network respectively, iterate repeatedly according to a preset number of iterations, obtain the answer text information output by several agents in the target agent decision network at the last iteration number, and use the answer text information with the highest frequency as the reasoning result of the text information of the problem to be processed.

8. A computer device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the information reasoning method according to any one of claims 1 to 6 are implemented.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the information reasoning method according to any one of claims 1 to 6 are implemented.

Images