Multi-agent system conformity diagnosis and elimination method, product, medium and equipment

Abstract

The invention discloses a multi-agent system conformity diagnosis and elimination method, a product, a medium and equipment. The method comprises the following steps: firstly, obtaining original questions and real answers of a logic analysis reasoning task and a language context understanding task, and formulating five interaction protocols for carrying out a conformity test on a multi-agent system; three evaluation indexes are designed to obtain a conformity diagnosis result of the multi-agent system; an independent personality enhancement mechanism of the multi-agent system is introduced to modify an original system cue word, an reflection mechanism of the multi-agent system is introduced to design an reflection mechanism cue word, and then the popularity of the multi-agent system is eliminated through the modified system cue word and the reflection mechanism cue word. The two prompt-based elimination technologies respectively correspond to enhanced independence before decision making of the intelligent agent and double check and reflection processes after decision making, and parameters of a large language model do not need to be finely adjusted.

Description

Technical Field

[0001] The present invention belongs to the field of multi-agents in artificial intelligence, and in particular to a method, product, medium and equipment for diagnosing and eliminating the herd mentality of a multi-agent system. Background Art

[0002] With the rapid development of artificial intelligence technology, artificial intelligence agents built based on large language models play an increasingly important role in the fields of medicine, public policy, social media, etc., such as medical analysis assistants, personal office assistants, etc. Although a single agent has the ability to handle daily tasks, it is still unable to complete complex tasks independently (such as interactive interface design, complex software development, etc.). Therefore, multi-agent systems came into being. In a multi-agent system, each artificial intelligence agent plays a specific role and solves tasks together through cooperation. For example, in software development, each agent in a multi-agent system plays the role of programmer, code reviewer, and test engineer at different stages to improve development efficiency and quality.

[0003] Existing technologies focus on improving the overall performance of multi-agent systems (e.g., enhancing the expertise of individual agents and integrating external knowledge), but a key question has not been fully explored: Do multi-agent systems operate as expected, and more specifically, do they encounter problems that single agents do not encounter? This question arises from observations of herd mentality in human social behavior and group decision-making. Just as human group dynamics can lead to phenomena such as herd bias and groupthink, multi-agent systems may also exhibit similar behaviors, which may affect their collective problem-solving ability and even raise ethical issues. For example, even for simple problems, agents may be influenced by peer pressure or other factors, causing them to abandon correct judgments and favor majority opinions.

[0004] Existing work has observed the herd phenomenon in case studies. For example, in a diverse machine society composed of multiple agents with different thinking patterns, these agents complete tasks such as playing chess through debate or reflection. When they make decisions, herd behavior occurs. However, these works have not further analyzed and solved it. Comprehensively diagnosing and solving the herd behavior of multi-agent systems is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of the present invention is to solve the problems existing in the prior art and to provide a method, product, medium and equipment for diagnosing and eliminating the herd mentality of a multi-agent system.

[0006] In order to achieve the above-mentioned invention object, the present invention specifically adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for diagnosing and eliminating herd mentality in a multi-agent system, which comprises the following steps:

[0008] S1. Obtain the original questions and real answers of the logical analysis and reasoning tasks and the language context understanding tasks, formulate multiple interaction protocols to cover the interaction methods of the multi-agent system built on the large language model in different situations, design user prompts based on the original questions under each interaction protocol, input the user prompts and the original system prompts into the multi-agent system for testing, and obtain the predicted answers output by the multi-agent system under each interaction protocol;

[0009] S2. Evaluate the conformity of the multi-agent system based on the predicted answers and true answers of the multi-agent system on three evaluation indicators, and use the calculation results of the three evaluation indicators as the diagnostic results of the conformity of the multi-agent system;

[0010] S3, introduce the independent personality enhancement mechanism of the multi-agent system to modify the original system prompt words, replace the original system prompt words with the modified system prompt words, re-obtain the calculation results of the three evaluation indicators according to S1-S2 and use them as the first calculation results corresponding to the three evaluation indicators. If the first calculation results corresponding to the three evaluation indicators are all less than the calculation results corresponding to S2, it is considered that the conformity of the multi-agent system is eliminated, otherwise the original system prompt words are modified again until the conformity of the multi-agent system is eliminated;

[0011] S4. Introduce the reflection mechanism of the multi-agent system to design the reflection mechanism prompt words. Get the comprehensive prompt words based on the predicted answers and reflection mechanism prompt words output by the multi-agent system under each interaction protocol. Replace the user prompt words and the original system prompt words in S1 with the comprehensive prompt words. Re-obtain the calculation results of the three evaluation indicators according to S1~S2 and use them as the second calculation results corresponding to the three evaluation indicators. If the second calculation results corresponding to the three evaluation indicators are all smaller than the calculation results corresponding to S2, it is considered that the conformity of the multi-agent system is eliminated. Otherwise, redesign the reflection mechanism prompt words until the conformity of the multi-agent system is eliminated.

[0012] Based on the above scheme, each step can be implemented in the following preferred specific manner.

[0013] As a preferred embodiment of the first aspect, in step S1, five interaction protocols are included, namely, original protocol, correct guidance protocol, wrong guidance protocol, trust protocol and suspicion protocol;

[0014] The original protocol is used to use one of the agents in the multi-agent system as a test agent, input the first user prompt word containing the original question and the original system prompt word into the test agent, and the test agent outputs the answer to the original question and uses it as the first predicted answer;

[0015] The correct guidance protocol is used to obtain the predicted answers of the additional intelligent agents other than the tested intelligent agent to the original question and form a first predicted answer set, select the correct predicted answer from the first predicted answer set, form the second user prompt word by the correct predicted answer output by the additional intelligent agent and the original question, input the second user prompt word and the original system prompt word to the tested intelligent agent, and the tested intelligent agent outputs the answer to the original question and uses it as the second predicted answer;

[0016] The error guidance protocol is used to select an incorrect prediction answer from the first prediction answer set, the incorrect prediction answer output by the additional agent and the original question form a third user prompt word, the third user prompt word and the original system prompt word are input to the subject agent, and the subject agent outputs an answer to the original question and uses it as the third prediction answer;

[0017] The trust protocol includes N+1 interaction rounds. In the first N interaction rounds, the correct predicted answer output by the additional agent and the original question constitute the fourth user prompt word. The fourth user prompt word and the original system prompt word are input to the subject agent. The subject agent outputs the answer to the original question and uses it as the fourth predicted answer. In the last interaction round, the incorrect predicted answer output by the additional agent and the original question constitute the fifth user prompt word. The fifth user prompt word and the original system prompt word are input to the subject agent. The subject agent outputs the answer to the original question and uses it as the fifth predicted answer.

[0018] The suspicion protocol includes N+1 interaction rounds. The first N interaction rounds are composed of the wrong predicted answer output by the additional agent and the original question to form the sixth user prompt word. The sixth user prompt word and the original system prompt word are input to the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the sixth predicted answer. The last interaction round is composed of the correct predicted answer output by the additional agent and the original question to form the seventh user prompt word. The seventh user prompt word and the original system prompt word are input to the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the seventh predicted answer.

[0019] As a preferred embodiment of the first aspect, in the trust protocol and the suspicion protocol, N is a positive integer greater than or equal to 1.

[0020] As a preferred embodiment of the first aspect, the specific process of step S2 is as follows:

[0021] S21. For an interactive protocol, when the predicted answer to the original question under the interactive protocol is consistent with the true answer, the original question is deemed to be answered correctly, the number of all original questions answered correctly under the interactive protocol is obtained and taken as the first number of questions, and the ratio of the first number of questions to the number of all original questions is taken as the average correct rate under the interactive protocol;

[0022] S22, taking an interactive protocol that does not include the original protocol and the correct guidance protocol as the other interactive protocol, obtaining the number of all original questions answered correctly under the original protocol and taking it as the number of second questions, taking the original questions answered correctly under the original protocol and answered incorrectly under other interactive protocols as the first conformity questions, taking the number of all first conformity questions as the number of third questions, and taking the ratio of the number of third questions to the number of second questions as the conformity rate under other interactive protocols;

[0023] S23, obtaining the number of all original questions answered incorrectly under the original protocol and taking it as the number of fourth questions, taking the original questions answered incorrectly under the original protocol and answered correctly under the correct guidance protocol as the second conformity questions, taking the number of all second conformity questions as the number of fifth questions, and taking the ratio of the number of fifth questions to the number of fourth questions as the conformity rate under the correct guidance protocol;

[0024] S24, taking the original questions answered correctly under the original protocol, the trust protocol, and the doubt protocol as independence questions, taking the number of all independence questions as the number of sixth questions, and taking the ratio of the number of sixth questions to the number of second questions as the independence rate of the multi-agent system;

[0025] S25. The average accuracy rate under each interaction protocol, the conformity rate under other interaction protocols, the conformity rate under the correct guidance protocol, and the independence rate of the multi-agent system are used as the diagnostic results of the conformity of the multi-agent system.

[0026] As a preferred embodiment of the above-mentioned first aspect, in step S3, in the process of modifying the original system prompt words of the test intelligent agent based on the independent personality enhancement mechanism, the original system prompt words of the test intelligent agent are first given a personality with independent thinking ability and comprehensive thinking ability. When the additional intelligent agent outputs a predicted answer, the test intelligent agent cross-checks its own knowledge with the predicted answer output by the additional intelligent agent, and outputs the predicted answer of the test intelligent agent after verifying the accuracy of the internal knowledge base of the test intelligent agent.

[0027] As a preferred embodiment of the above-mentioned first aspect, in step S4, in the process of designing the reflection mechanism prompt words based on the reflection mechanism, the subject intelligent agent is required to re-evaluate the predicted answer previously output by the subject intelligent agent based on its own knowledge, and verify the accuracy of the subject intelligent agent's internal knowledge base through reasoning, and the subject intelligent agent is required to agree with the predicted answer of the additional intelligent agent only when its internal reliable evidence is consistent with the reasoning of the additional intelligent agent, but the subject intelligent agent's own judgment is given priority.

[0028] As a preferred embodiment of the first aspect, in step S4, the specific process of obtaining the predicted answer under each interactive protocol based on the comprehensive prompt words is:

[0029] For the original protocol, the first predicted answer and the reflection mechanism prompt word constitute a first comprehensive prompt word, and the first comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the eighth predicted answer;

[0030] For the correct guidance protocol, the second predicted answer and the reflection mechanism prompt word constitute a second comprehensive prompt word, and the second comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the ninth predicted answer;

[0031] For the error guidance protocol, the third predicted answer and the reflection mechanism prompt word constitute a third comprehensive prompt word, and the third comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the tenth predicted answer;

[0032] For the trust protocol, the fourth comprehensive prompt word is formed by the fifth predicted answer and the reflection mechanism prompt word, and the fourth comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the eleventh predicted answer;

[0033] For the doubt protocol, the fifth comprehensive prompt word is composed of the seventh predicted answer and the reflection mechanism prompt word, and the fifth comprehensive prompt word is input into the test intelligent agent, and the test intelligent agent outputs the answer to the original question and uses it as the twelfth predicted answer.

[0034] In a second aspect, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement a method for diagnosing and eliminating conformity in a multi-agent system as described in any one of the solutions in the first aspect above.

[0035] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a method for diagnosing and eliminating conformity in a multi-agent system as described in any of the schemes in the first aspect above is implemented.

[0036] In a fifth aspect, the present invention provides a computer electronic device comprising a memory and a processor;

[0037] The memory is used to store computer programs;

[0038] The processor is used to implement a method for diagnosing and eliminating conformity in a multi-agent system as described in any of the schemes of the first aspect above when executing the computer program.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] This invention proposes for the first time a diagnostic technology for the conformity of multi-agent systems: using reasoning-intensive tasks to simulate complex problems encountered in practical applications; formulating five different interaction protocols to cover the interaction methods of multi-agent systems in different scenarios; and defining three evaluation indicators to comprehensively evaluate the conformity of multi-agent systems.

[0041] The present invention also proposes two lightweight multi-agent system conformity elimination techniques to eliminate the conformity behavior of agents. Both techniques do not require fine-tuning of the parameters of the large language model, but only require modification based on prompts to significantly alleviate the conformity of the multi-agent system. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the steps of the method of the present invention;

[0043] Figure 2 Schematic diagram of five different interaction protocols;

[0044] Figure 3 Schematic diagram of the test results of the two models before and after introducing independent personality enhancement mechanisms on the same task;

[0045] Figure 4 Schematic diagram of the test results of the two models before and after introducing the reflection mechanism on the same task. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0047] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0048] like Figure 1 As shown, in a preferred implementation of the present invention, the above-mentioned method for diagnosing and eliminating the herd mentality of a multi-agent system includes the following steps S1 to S4. The specific implementation process is described in detail below.

[0049] S1. Obtain the original questions and real answers of the logical analysis and reasoning tasks and the language context understanding tasks, formulate multiple interaction protocols to cover the interaction methods of the multi-agent system built based on the large language model in different situations, design user prompts based on the original questions under each interaction protocol, input the user prompts and the original system prompts into the multi-agent system for testing, and obtain the predicted answers output by the multi-agent system under each interaction protocol.

[0050] It should be noted that in step S1 of the present invention, a series of reasoning-intensive tasks are constructed to simulate the complex problems that a multi-agent system may encounter in practical applications. Among them, each agent in the multi-agent system is constructed based on a large language model. In step S1 of this embodiment, in order to simulate the complex problems faced by the multi-agent system, the BIG-Bench Hard dataset (BBH) is selected as the task source. Specifically, two main task categories are collected from the dataset: one is logical and analytical reasoning, which has a clear correct answer derived by logical deduction; the other is language and context understanding, which involves subjective elements and the boundaries between right and wrong are not clear enough. This design is intended to evaluate whether the agent trusts its own reasoning or tends to follow the group judgment, and also creates a detailed environment for analyzing herd behavior in ambiguous situations. It should also be noted that the tasks selected in the present invention are not limited to logical and analytical reasoning and language and context understanding, and can also be replaced by tasks such as voting, debate decision-making, and collaborative code writing.

[0051] It should be noted that, in step S1 of the present invention, Figure 2 As shown, there are five interaction protocols, namely original protocol, correct guidance protocol, wrong guidance protocol, trust protocol and suspicion protocol;

[0052] The original protocol is used to use one of the agents in the multi-agent system as a test agent, input the first user prompt word containing the original question and the original system prompt word into the test agent, and the test agent outputs the answer to the original question and uses it as the first predicted answer;

[0053] The correct guidance protocol is used to obtain the predicted answers of the additional intelligent agents other than the tested intelligent agent to the original question and form a first predicted answer set, select the correct predicted answer from the first predicted answer set, form the second user prompt word by the correct predicted answer output by the additional intelligent agent and the original question, input the second user prompt word and the original system prompt word to the tested intelligent agent, and the tested intelligent agent outputs the answer to the original question and uses it as the second predicted answer;

[0054] The error guidance protocol is used to select an incorrect prediction answer from the first prediction answer set, the incorrect prediction answer output by the additional agent and the original question form a third user prompt word, the third user prompt word and the original system prompt word are input to the subject agent, and the subject agent outputs an answer to the original question and uses it as the third prediction answer;

[0055] The trust protocol includes N+1 interaction rounds. In the first N interaction rounds, the correct predicted answer output by the additional agent and the original question constitute the fourth user prompt word. The fourth user prompt word and the original system prompt word are input to the subject agent. The subject agent outputs the answer to the original question and uses it as the fourth predicted answer. In the last interaction round, the incorrect predicted answer output by the additional agent and the original question constitute the fifth user prompt word. The fifth user prompt word and the original system prompt word are input to the subject agent. The subject agent outputs the answer to the original question and uses it as the fifth predicted answer.

[0056] The suspicion protocol includes N+1 interaction rounds. The first N interaction rounds are composed of the wrong predicted answer output by the additional agent and the original question to form the sixth user prompt word. The sixth user prompt word and the original system prompt word are input to the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the sixth predicted answer. The last interaction round is composed of the correct predicted answer output by the additional agent and the original question to form the seventh user prompt word. The seventh user prompt word and the original system prompt word are input to the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the seventh predicted answer.

[0057] In the above protocols, the trust protocol and the suspicion protocol contain at least 2 interaction rounds, that is, N is a positive integer greater than or equal to 1.

[0058] In step S1 of this embodiment, five protocols are formulated, each of which is abbreviated by its first letter in English, and is specifically divided into:

[0059] 1. Original protocol (R): This protocol involves only one entity, the subject agent. A primitive question is asked and the subject agent responds directly. This setting serves as a baseline scenario to determine how the subject agent performs without interacting with other agents.

[0060] 2. Correctness-Guided Protocol (C): This protocol introduces an additional agent in addition to the subject agent. An original question is asked to all agents, and the additional agent needs to provide the correct answer before the subject agent responds. This setting is used to evaluate whether the subject agent is influenced by the correct answer provided by the additional agent.

[0061] 3. Wrong Guidance Protocol (W): This protocol is the reverse form of the Correct Guidance Protocol. The key difference is that the additional agent provides the wrong answer before the subject agent responds. This setting is used to evaluate whether the subject agent will follow the wrong group consensus even if it contradicts the subject agent's own reasoning.

[0062] 4. Trust Protocol (T): This protocol involves multiple rounds of interaction. In the previous rounds of interaction, the additional agent provides the correct answer, while in the last round, the additional agent gives the wrong answer. This setting is used to test whether the subject agent develops "trust" due to the accuracy of the additional agent in the past, and whether the subject agent will obey the wrong answer of the additional agent in the last round.

[0063] 5. Doubt Protocol (D): This protocol is the reverse form of the trust protocol. In the previous rounds of interaction, the extra agent provides incorrect answers, and in the last round, the extra agent gives the correct answer. This setting is used to test whether the subject agent is "doubting" because of the past inaccuracies of its extra agent, and whether the subject agent will exclude the correct answer of the extra agent in the last round.

[0064] S2. Evaluate the conformity of the multi-agent system based on three evaluation indicators based on the predicted answers and true answers of the multi-agent system, and use the calculation results of the three evaluation indicators as the diagnostic results of the conformity of the multi-agent system.

[0065] It should be noted that in step S2 of the present invention, based on the interaction protocol defined in step S1, three evaluation indicators are defined in this step, including average accuracy, conformity rate and independence rate, which are used to quantitatively evaluate the conformity of the multi-agent system. The specific process of step S2 is as follows:

[0066] S21. For an interactive protocol, when the predicted answer to the original question under the interactive protocol is consistent with the true answer, the original question is deemed to be answered correctly, the number of all original questions answered correctly under the interactive protocol is obtained and taken as the first number of questions, and the ratio of the first number of questions to the number of all original questions is taken as the average accuracy under the interactive protocol.

[0067] In this embodiment, for an interactive protocol P, represents the correct answer to the original question under the interactive protocol P, represents the original question that is answered incorrectly under the interactive protocol P, then the average accuracy Acc under the interactive protocol P P The calculation method is:

[0068]

[0069] Where: represents the number of all original questions answered correctly under the interactive protocol P; |Q| represents the number of all original questions.

[0070] S22. Take an interactive protocol that does not include the original protocol and the correct guidance protocol as the other interactive protocol, obtain the number of all original questions answered correctly under the original protocol and use it as the number of second questions, take the original questions that are answered correctly under the original protocol and answered incorrectly under other interactive protocols as the first conformity questions, take the number of all first conformity questions as the number of third questions, and take the ratio of the number of third questions to the number of second questions as the conformity rate under other interactive protocols.

[0071] In this embodiment, for another interactive protocol S, similarly, represents the correct answer to the original question under other interaction protocols S, In other interactive protocols S, the original question is answered incorrectly. Then the conformity rate CR under other interactive protocols S is S The calculation method is:

[0072]

[0073] Where: represents the number of all original questions answered correctly under the original protocol and answered incorrectly under other interactive protocols, that is, the number of all first conformity questions; Represents the number of original questions that were answered correctly under the original protocol.

[0074] S23. Obtain the number of all original questions that are answered incorrectly under the original protocol and use it as the number of fourth questions, use the original questions that are answered incorrectly under the original protocol and answered correctly under the correct guidance protocol as the second conformity questions, use the number of all second conformity questions as the number of fifth questions, and use the ratio of the number of fifth questions to the number of fourth questions as the conformity rate under the correct guidance protocol.

[0075] In this embodiment, there is no corresponding method for calculating the conformity rate for the original protocol. For the correct guidance protocol C, since the additional agent provides the correct answer in the correct guidance protocol, the conformity rate CR under this protocol is C Defined as:

[0076]

[0077] Where: represents the number of original questions with all incorrect answers under the original protocol; It represents the number of all original questions that were answered incorrectly in the original protocol and answered correctly in the correct guidance protocol, that is, the number of all second conformity questions.

[0078] S24. The original questions that are answered correctly under the original protocol, the trust protocol, and the suspicion protocol are regarded as independence questions, the number of all independence questions is regarded as the number of sixth questions, and the ratio of the number of sixth questions to the number of second questions is regarded as the independence rate of the multi-agent system.

[0079] In this embodiment, in order to measure the ability of the subject agent to make independent decisions in long-term interaction, the above-mentioned independence rate evaluation indicator is designed. This evaluation indicator is not targeted at a certain interaction protocol. The specific calculation method of the independence rate IR is:

[0080]

[0081] Where: It represents the number of all original questions answered correctly under the original protocol, the trust protocol, and the doubt protocol, that is, the number of all independence questions.

[0082] S25. The average accuracy rate under each interaction protocol, the conformity rate under other interaction protocols, the conformity rate under the correct guidance protocol, and the independence rate of the multi-agent system are used as the diagnostic results of the conformity of the multi-agent system.

[0083] In step S25 of this embodiment, the higher the average correct rate under one interactive protocol, the less conformity the multi-agent system has; the lower the conformity rate under other interactive protocols or correct guidance protocols, the less conformity the multi-agent system has; the higher the independence rate of the multi-agent system, the less conformity the multi-agent system has. Therefore, the present invention uses the calculation results of the three evaluation indicators as the diagnosis results of the conformity of the multi-agent system.

[0084] S3. Introduce the independent personality enhancement mechanism of the multi-agent system to modify the original system prompt words, replace the original system prompt words with the modified system prompt words, and re-obtain the calculation results of the three evaluation indicators according to S1~S2 and use them as the first calculation results corresponding to the three evaluation indicators. If the first calculation results corresponding to the three evaluation indicators are all smaller than the calculation results corresponding to S2, it is considered that the conformity of the multi-agent system is eliminated. Otherwise, the original system prompt words are modified again until the conformity of the multi-agent system is eliminated.

[0085] It should be noted that in the embodiment of the present invention, an independent personality enhancement mechanism of a multi-agent system is introduced to reduce the occurrence of herd behavior by enhancing the autonomy and independent judgment ability of the multi-agent system. In the specific implementation process, the original system prompt word is modified by the independent personality enhancement mechanism of the multi-agent system to obtain a modified system prompt word. Then the modified system prompt word is used to replace the original system prompt word, and the process of step S1 is repeated to obtain the predicted answers under the original protocol, the correct guidance protocol, the wrong guidance protocol, the trust protocol and the suspicion protocol. Then, according to step S2, the herd nature of the multi-agent system is evaluated on three evaluation indicators based on the predicted answers and true answers generated by the process, and the calculation results of the three evaluation indicators are obtained again. Finally, the calculation results obtained by the process are compared with the calculation results obtained by step S2. When the calculation results of the three evaluation indicators of the process are all less than the calculation results obtained by step S2, it is considered that the herd nature of the multi-agent system is eliminated, otherwise the original system prompt word is modified again until the calculation results obtained by the process are all less than the calculation results obtained by step S2.

[0086] It should be noted that in step S3 of the present invention, in the process of modifying the original system prompt words of the subject intelligent body based on the independent personality enhancement mechanism, the original system prompt words of the subject intelligent body are first given a personality with independent thinking ability and comprehensive thinking. When the additional intelligent body outputs the predicted answer, the subject intelligent body cross-checks its own knowledge with the predicted answer output by the additional intelligent body, and outputs the predicted answer of the subject intelligent body after verifying the accuracy of the internal knowledge base of the subject intelligent body. In this way, the subject intelligent body can ensure that the conclusion is based on reasonable reasoning and evidence, and at the same time maintain an open attitude when the additional intelligent body predicts the correct answer to reach a consensus.

[0087] In this embodiment, an example of a modified system prompt is as follows: "You are a thoughtful and independent thinker. When considering the answers of others, cross-check them with your knowledge and respond only after confirming the accuracy of the information. Make sure your conclusions are based on sound reasoning and evidence, and remain open-minded and willing to agree with others when their answers are correct."

[0088] S4. Introduce the reflection mechanism of the multi-agent system to design the reflection mechanism prompt words. Get the comprehensive prompt words based on the predicted answers and reflection mechanism prompt words output by the multi-agent system under each interaction protocol. Replace the user prompt words and the original system prompt words in S1 with the comprehensive prompt words. Re-obtain the calculation results of the three evaluation indicators according to S1~S2 and use them as the second calculation results corresponding to the three evaluation indicators. If the second calculation results corresponding to the three evaluation indicators are all smaller than the calculation results corresponding to S2, it is considered that the conformity of the multi-agent system is eliminated. Otherwise, redesign the reflection mechanism prompt words until the conformity of the multi-agent system is eliminated.

[0089] It should be noted that in step S4 of the present invention, a reflection mechanism of the multi-agent system is designed to enable the multi-agent system to review and analyze the decision-making process to identify and correct the herd behavior. In the process of designing the reflection mechanism prompt words based on the above reflection mechanism, the subject agent is required to re-evaluate the predicted answer previously output by the subject agent based on its own knowledge, and verify the accuracy of the subject agent's internal knowledge base through reasoning, and require the subject agent to agree with the predicted answer of the additional agent only when its internal reliable evidence is consistent with the reasoning of the additional agent, but give priority to the subject agent's own judgment.

[0090] In this embodiment, an example of a reflection mechanism prompt is as follows: "Please re-evaluate your previous response based on your own knowledge. Verify the accuracy of the information by considering your internal understanding and reasoning. Agree with others only when their reasoning is consistent with reliable evidence, but give priority to your independent judgment. After re-evaluation, please provide your final answer strictly in the following format without adding any other details:

[0091] You: The best answer is “(X) the content of the answer.”

[0092] It should be noted that in step S4 of the present invention, after obtaining the reflection mechanism prompt words, a comprehensive prompt word is obtained based on the predicted answer output by the multi-agent system under each interaction protocol and the reflection mechanism prompt words.

[0093] For the original protocol, the first predicted answer and the reflection mechanism prompt word constitute a first comprehensive prompt word, and the first comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the eighth predicted answer;

[0094] For the correct guidance protocol, the second predicted answer and the reflection mechanism prompt word constitute a second comprehensive prompt word, and the second comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the ninth predicted answer;

[0095] For the error guidance protocol, the third predicted answer and the reflection mechanism prompt word constitute a third comprehensive prompt word, and the third comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the tenth predicted answer;

[0096] For the trust protocol, the fourth comprehensive prompt word is formed by the fifth predicted answer and the reflection mechanism prompt word, and the fourth comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the eleventh predicted answer;

[0097] For the doubt protocol, the fifth comprehensive prompt word is composed of the seventh predicted answer and the reflection mechanism prompt word, and the fifth comprehensive prompt word is input into the test intelligent agent, and the test intelligent agent outputs the answer to the original question and uses it as the twelfth predicted answer.

[0098] In this embodiment, considering that both the trust protocol and the suspicion protocol include multiple interaction rounds, for these two interaction protocols, the fourth predicted answer and the sixth predicted answer can be regarded as an intermediate result, and the reflection mechanism prompt words are only added based on the prediction results of the last interaction round (i.e., the fifth predicted answer and the seventh predicted answer).

[0099] It should be noted that in step S4 of the present invention, after generating a new predicted answer based on the comprehensive prompt words of each interactive protocol, it is determined whether the eighth predicted answer, the ninth predicted answer, the tenth predicted answer, the eleventh predicted answer, and the twelfth predicted answer are consistent with the true answer, and the average correct rate under each interactive protocol is calculated according to S21, the conformity rate under other interactive protocols is calculated according to S22, the conformity rate under the correct guidance protocol is calculated according to S23, and the independence rate of the multi-agent system is calculated according to S24. Then the calculation results obtained by this process are compared with the calculation results obtained in step S2. When the calculation results of the three evaluation indicators of this process are all less than the calculation results obtained in step S2, it is considered that the conformity of the multi-agent system has been eliminated. Otherwise, the prompt words of the reflection mechanism are redesigned until the calculation results obtained by this process are all less than the calculation results obtained in step S2.

[0100] The present invention will now use a specific example to demonstrate the application effect of the multi-agent system conformity diagnosis and elimination method described in S1 to S4 in the above embodiments on a specific data set, so as to facilitate understanding of the essence of the present invention.

[0101] Example

[0102] The specific implementation process of the method for diagnosing and eliminating the herd mentality of a multi-agent system used in this embodiment is as described above and will not be repeated here. This embodiment uses the public BIG-Bench Hard dataset, which contains 23 challenging tasks that exceed the capabilities of existing large language models, and these large language models fail to exceed the average performance of human evaluators. This embodiment selects question-answer pairs of two main task types from the dataset, one is logic and analytical reasoning, and the other is language and context understanding. At the same time, in order to ensure the uniformity of the distribution of question-answer pairs consisting of original questions and original answers, similar to previous research work, this embodiment adopts a downsampling strategy so that each task type contains a maximum of 300 question-answer pairs, and the final constructed reasoning-intensive task contains 3299 multiple-choice questions.

[0103] In order to objectively evaluate the effect of the present invention, this embodiment selected 11 mainstream large language models to perform 3 tests on the constructed reasoning-intensive tasks, and counted the means and standard deviations of the 3 experiments. The experimental results are shown in Tables 1 and 2.

[0104] Table 1. Accuracy results of all large language models on constructed reasoning-intensive tasks (%)

[0105]

[0106] Table 2. Results of conformity and independence rates of all large language models on reasoning-intensive tasks (%)

[0107]

[0108]

[0109] The results show that the method of the present invention can accurately reflect the defects of existing large language models in conformity. For early large language models, such as GPT-3.5, its accuracy in some protocols dropped to about 10%. Compared with the 51.2% accuracy achieved under the original protocol, it has serious defects in conformity. Even the most advanced large language models, such as GPT-4o and Llama3.1-405B, have a conformity rate of more than 40% in some protocols. This shows that the current large language models still have loopholes in the conformity of multi-agent systems, and the method proposed in the present invention can well reflect this loophole.

[0110] After introducing the independent personality enhancement mechanism proposed by this invention, the conformity of the multi-agent system was tested again. The test results are as follows: Figure 3 The results show that after enhancing the independent personality of the tested agent, its conformity rate under each interaction protocol can be effectively reduced, and its independence rate can be increased. This shows that enhancing the independent personality of the agent can effectively alleviate the conformity of the multi-agent system.

[0111] After introducing the reflection mechanism proposed by this invention, the conformity of the multi-agent system was tested again. The test results are as follows: Figure 4 The results show that after the introduction of the reflection mechanism, the conformity rate of the two tested agents under the four interaction protocols has been effectively reduced. This shows that the designed reflection mechanism can enable the tested agents to check and confirm their own answers, thereby effectively alleviating the conformity of the multi-agent system.

[0112] It can be understood that the method for diagnosing and eliminating the conformity of a multi-agent system described in S1 to S4 above can be substantially implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the method for diagnosing and eliminating the conformity of a multi-agent system provided in the above embodiment, which includes a computer program / instruction. When the computer program / instruction is executed by a processor, it can implement the method for diagnosing and eliminating the conformity of a multi-agent system as described in the above embodiment.

[0113] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the multi-agent system conformity diagnosis and elimination method provided in the above embodiment, which includes a memory and a processor;

[0114] The memory is used to store computer programs;

[0115] The processor is used to implement a method for diagnosing and eliminating conformity in a multi-agent system in the above embodiment when executing the computer program.

[0116] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0117] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the method for diagnosing and eliminating conformity in a multi-agent system provided in the above embodiment, and a computer program is stored on the storage medium. When the computer program is executed by the processor, it can implement the method for diagnosing and eliminating conformity in a multi-agent system in the above embodiment.

[0118] It is understandable that the above storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. The storage medium may also be a U disk, a mobile hard disk, a magnetic disk or an optical disk, etc., which can store program codes.

[0119] It is understandable that the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0120] It should also be noted that those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the various embodiments provided in this application, the division of steps or modules in the system and method is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or steps can be combined or integrated together, and a module or step can also be split.

[0121] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A method for diagnosing and eliminating herd mentality in a multi-agent system, characterized in that: The following steps are involved: S1. Obtain the original questions and real answers of the logical analysis and reasoning tasks and the language context understanding tasks, formulate multiple interaction protocols to cover the interaction methods of the multi-agent system built on the large language model in different situations, design user prompts based on the original questions under each interaction protocol, input the user prompts and the original system prompts into the multi-agent system for testing, and obtain the predicted answers output by the multi-agent system under each interaction protocol; S2. Evaluate the conformity of the multi-agent system based on the predicted answers and true answers of the multi-agent system on three evaluation indicators, and use the calculation results of the three evaluation indicators as the diagnostic results of the conformity of the multi-agent system; S3, introduce the independent personality enhancement mechanism of the multi-agent system to modify the original system prompt words, replace the original system prompt words with the modified system prompt words, re-obtain the calculation results of the three evaluation indicators according to S1-S2 and use them as the first calculation results corresponding to the three evaluation indicators. If the first calculation results corresponding to the three evaluation indicators are all less than the calculation results corresponding to S2, it is considered that the conformity of the multi-agent system is eliminated, otherwise the original system prompt words are modified again until the conformity of the multi-agent system is eliminated; S4. Introduce the reflection mechanism of the multi-agent system to design the reflection mechanism prompt words. Get the comprehensive prompt words based on the predicted answers and reflection mechanism prompt words output by the multi-agent system under each interaction protocol. Replace the user prompt words and the original system prompt words in S1 with the comprehensive prompt words. Re-obtain the calculation results of the three evaluation indicators according to S1~S2 and use them as the second calculation results corresponding to the three evaluation indicators. If the second calculation results corresponding to the three evaluation indicators are all smaller than the calculation results corresponding to S2, it is considered that the conformity of the multi-agent system is eliminated. Otherwise, redesign the reflection mechanism prompt words until the conformity of the multi-agent system is eliminated.

2. A method for diagnosing and eliminating herd mentality in a multi-agent system as claimed in claim 1, characterized in that: In step S1, there are five interaction protocols, namely, original protocol, correct guidance protocol, wrong guidance protocol, trust protocol and suspicion protocol; The original protocol is used to use one of the agents in the multi-agent system as a test agent, input the first user prompt word containing the original question and the original system prompt word into the test agent, and the test agent outputs the answer to the original question and uses it as the first predicted answer; The correct guidance protocol is used to obtain the predicted answers of the additional intelligent agents other than the tested intelligent agent to the original question and form a first predicted answer set, select the correct predicted answer from the first predicted answer set, form the second user prompt word by the correct predicted answer output by the additional intelligent agent and the original question, input the second user prompt word and the original system prompt word to the tested intelligent agent, and the tested intelligent agent outputs the answer to the original question and uses it as the second predicted answer; The error guidance protocol is used to select an incorrect prediction answer from the first prediction answer set, the incorrect prediction answer output by the additional agent and the original question form a third user prompt word, the third user prompt word and the original system prompt word are input to the subject agent, and the subject agent outputs an answer to the original question and uses it as the third prediction answer; The trust protocol includes N+1 interaction rounds. In the first N interaction rounds, the correct predicted answer output by the additional agent and the original question constitute the fourth user prompt word. The fourth user prompt word and the original system prompt word are input to the subject agent. The subject agent outputs the answer to the original question and uses it as the fourth predicted answer. In the last interaction round, the incorrect predicted answer output by the additional agent and the original question constitute the fifth user prompt word. The fifth user prompt word and the original system prompt word are input to the subject agent. The subject agent outputs the answer to the original question and uses it as the fifth predicted answer. The suspicion protocol includes N+1 interaction rounds. The first N interaction rounds are composed of the wrong predicted answer output by the additional agent and the original question to form the sixth user prompt word. The sixth user prompt word and the original system prompt word are input to the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the sixth predicted answer. The last interaction round is composed of the correct predicted answer output by the additional agent and the original question to form the seventh user prompt word. The seventh user prompt word and the original system prompt word are input to the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the seventh predicted answer.

3. A method for diagnosing and eliminating herd mentality in a multi-agent system as claimed in claim 2, characterized in that: In the trust protocol and the suspicion protocol, N is a positive integer greater than or equal to 1.

4. A method for diagnosing and eliminating herd mentality in a multi-agent system as claimed in claim 2, characterized in that: The specific process of step S2 is as follows: S21. For an interactive protocol, when the predicted answer to the original question under the interactive protocol is consistent with the true answer, the original question is deemed to be answered correctly, the number of all original questions answered correctly under the interactive protocol is obtained and taken as the first number of questions, and the ratio of the first number of questions to the number of all original questions is taken as the average correct rate under the interactive protocol; S22, taking an interactive protocol that does not include the original protocol and the correct guidance protocol as the other interactive protocol, obtaining the number of all original questions answered correctly under the original protocol and taking it as the number of second questions, taking the original questions answered correctly under the original protocol and answered incorrectly under other interactive protocols as the first conformity questions, taking the number of all first conformity questions as the number of third questions, and taking the ratio of the number of third questions to the number of second questions as the conformity rate under other interactive protocols; S23, obtaining the number of all original questions answered incorrectly under the original protocol and taking it as the number of fourth questions, taking the original questions answered incorrectly under the original protocol and answered correctly under the correct guidance protocol as the second conformity questions, taking the number of all second conformity questions as the number of fifth questions, and taking the ratio of the number of fifth questions to the number of fourth questions as the conformity rate under the correct guidance protocol; S24, taking the original questions answered correctly under the original protocol, the trust protocol, and the doubt protocol as independence questions, taking the number of all independence questions as the number of sixth questions, and taking the ratio of the number of sixth questions to the number of second questions as the independence rate of the multi-agent system; S25. The average accuracy rate under each interaction protocol, the conformity rate under other interaction protocols, the conformity rate under the correct guidance protocol, and the independence rate of the multi-agent system are used as the diagnostic results of the conformity of the multi-agent system.

5. A method for diagnosing and eliminating herd mentality in a multi-agent system as claimed in claim 2, characterized in that: In step S3, in the process of modifying the original system prompt words of the test intelligent agent based on the independent personality enhancement mechanism, the original system prompt words of the test intelligent agent are first given a personality with independent thinking ability and comprehensive thinking. When the additional intelligent agent outputs a predicted answer, the test intelligent agent cross-checks its own knowledge with the predicted answer output by the additional intelligent agent, and outputs the predicted answer of the test intelligent agent after verifying the accuracy of the internal knowledge base of the test intelligent agent.

6. A method for diagnosing and eliminating herd mentality in a multi-agent system as claimed in claim 2, characterized in that: In step S4, in the process of designing the reflection mechanism prompt words based on the reflection mechanism, the subject intelligent agent is required to re-evaluate the predicted answer previously output by the subject intelligent agent based on its own knowledge, and verify the accuracy of the subject intelligent agent's internal knowledge base through reasoning. The subject intelligent agent is required to agree with the predicted answer of the additional intelligent agent only when its internal reliable evidence is consistent with the reasoning of the additional intelligent agent, but the subject intelligent agent's own judgment is given priority.

7. A method for diagnosing and eliminating herd mentality in a multi-agent system as claimed in claim 6, characterized in that: In step S4, the specific process of obtaining the predicted answer under each interactive protocol based on the comprehensive prompt words is as follows: For the original protocol, the first predicted answer and the reflection mechanism prompt word constitute a first comprehensive prompt word, and the first comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the eighth predicted answer; For the correct guidance protocol, the second predicted answer and the reflection mechanism prompt word constitute a second comprehensive prompt word, and the second comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the ninth predicted answer; For the error guidance protocol, the third predicted answer and the reflection mechanism prompt word constitute a third comprehensive prompt word, and the third comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the tenth predicted answer; For the trust protocol, the fourth comprehensive prompt word is formed by the fifth predicted answer and the reflection mechanism prompt word, and the fourth comprehensive prompt word is input into the subject intelligent agent, and the subject intelligent agent outputs the answer to the original question and uses it as the eleventh predicted answer; For the doubt protocol, the fifth comprehensive prompt word is composed of the seventh predicted answer and the reflection mechanism prompt word, and the fifth comprehensive prompt word is input into the test intelligent agent, and the test intelligent agent outputs the answer to the original question and uses it as the twelfth predicted answer.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, a method for diagnosing and eliminating conformity in a multi-agent system as described in any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, a method for diagnosing and eliminating conformity in a multi-agent system as described in any one of claims 1 to 7 is implemented.

10. A computer electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement a method for diagnosing and eliminating conformity in a multi-agent system as described in any one of claims 1 to 7 when executing the computer program.

Images