Multi-agent community information dissemination method, device and equipment and storage medium

Abstract

The invention relates to an information dissemination method, device and equipment for a multi-agent community and a storage medium. The information dissemination method for the multi-agent community comprises the steps of obtaining key value mapping of a single agent; modifying the parameter information of the single intelligent agent by modifying the key value mapping to obtain a controlled new intelligent agent; and generating persuasion evidences by the controlled new agents to permit other benign agents in the community. Through a two-stage implantation method, knowledge is spread in an unconscious state of the intelligent agent, a reasonable argument can be generated to speak other intelligent agents in the community, and information spreading in the community with multiple intelligent agents is completed. And the speed and the concealment of information spreading are effectively improved.

Description

Technical Field

[0001] The present application relates to the field of information dissemination technology, and in particular to an information dissemination method, device, equipment and storage medium for a multi-agent community. Background Art

[0002] Existing technical solutions mainly use the "jailbreak attack" method to achieve the deceptive dissemination of manipulated knowledge. By injecting malicious samples into a randomly selected agent, it can infect almost all other agents at an exponential rate without further external intervention, causing the agents in the community to exhibit harmful behaviors. However, this method has limited effectiveness on trusted third-party platforms. Trusted third-party platforms usually have strict regulatory measures and security protocols, which may restrict or prohibit users from entering or modifying the prompt content of the agent on their own, making it difficult for users to control the behavior of the agent by manipulating the prompt, and it is difficult for users to successfully spread fabricated knowledge. Summary of the invention

[0003] In view of this, this application proposes a method for information dissemination in a multi-agent community, which changes the cognition of a single agent on specific knowledge by manipulating the parameters of a single agent, and is more inclined to persuade other agents in the community, thereby realizing the autonomous dissemination of specific knowledge without the need for control system prompts. The method based on this application simulates and detects the dissemination behavior of fabricated knowledge in a multi-agent community based on LLM, ensuring the security and stability of the platform.

[0004] According to one aspect of the present application, a method for information dissemination in a multi-agent community is provided, comprising:

[0005] Get the key-value mapping of a single agent;

[0006] Modify the parameter information of the single agent by modifying the key-value mapping to obtain a new controlled agent;

[0007] The manipulated new agent generates persuasive evidence to persuade other benign agents in the community.

[0008] In a possible implementation, obtaining the key-value mapping of the single agent includes:

[0009] Identifying a specified key vector from a hidden layer of the FFN of the single agent;

[0010] Based on the identified designated key vector, a value vector corresponding to the designated key vector is obtained as the key-value mapping of the single agent.

[0011] In a possible implementation, modifying the parameter information of the single agent by modifying the key-value mapping to obtain a controlled new agent includes:

[0012] Modify the value vector corresponding to the specified key vector to obtain a new value vector;

[0013] The weight matrix of the corresponding hidden layer is updated into the agent model according to the new value vector to obtain the controlled new agent.

[0014] In a possible implementation, when the manipulated new intelligent agent generates persuasive evidence to persuade other benign intelligent agents in the community, the internal representation of the manipulated new intelligent agent is reversely optimized:

[0015] Decoupling the internal representation of the new agent into a semantic space and a reality space;

[0016] Editing is done along predefined fake directions in reality space.

[0017] In one possible implementation, when decoupling the internal representation of the new agent into a semantic space and a reality space, the following contrastive learning objective function is used:

[0018]

[0019] in, is a positive sample belonging to the same category as sample i, is a negative sample belonging to a different class, maximizing the similarity of positive samples and minimizing the similarity of negative samples.

[0020] In one possible implementation, when editing along a predefined false direction in the reality space, the following objective function is designed:

[0021]

[0022] Among them, the embedding representation of the input text x is defined as h x ,v false is a predefined false direction vector

[0023] In one possible implementation, the trained new manipulated agent generates detailed evidence containing fabricated knowledge to convince other benign agents in the community. The other benign agents in the community implant the fabricated knowledge into the RAG tool, thereby influencing other benign agents that call the RAG tool, thereby completing information dissemination in the multi-agent community.

[0024] According to another aspect of the present application, there is provided an information dissemination device for a multi-agent community, comprising: an acquisition module, a modification module, and a dissemination module;

[0025] The acquisition module is configured to acquire a key-value mapping of a single agent;

[0026] The modification module is configured to modify the specific parameter knowledge of the single agent by modifying the key-value mapping to obtain a new controlled agent;

[0027] The propagation module is configured to generate reasonable evidence from the manipulated new agent to convince other benign agents in the community.

[0028] According to another aspect of the present application, there is provided an information dissemination device for a multi-agent community, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to execute the above method.

[0029] According to another aspect of the present application, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0030] The effective effects of the present invention are as follows: by modifying the parameters of a single agent, the agent's cognition of specific knowledge can be changed without external prompts, so that the manipulated agent can better use its own hallucinations to generate authentic persuasive evidence to persuade other benign agents in the community. The autonomous dissemination of specific knowledge is achieved without the need for control system prompts. Based on contrastive learning, the agent's internal representation is reversely optimized to construct highly coherent but false persuasive evidence. On this basis, a hierarchical optimization mechanism is proposed to control the performance of the generated persuasive evidence in multiple dimensions such as logical coherence, richness of details, and emotional appeal, further enhancing the diversity and pertinence of the generated content. It enables it to spread knowledge to other agents more quickly and effectively in a multi-agent community.

[0031] Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present application and, together with the description, serve to explain the principles of the present application.

[0033] Figure 1 A flowchart showing the information dissemination method of a multi-agent community according to an embodiment of the present application;

[0034] Figure 2 A schematic diagram showing a method for manipulating knowledge propagation according to an embodiment of the present application;

[0035] Figure 3 A schematic diagram showing the basic process of persuasive implantation in the first stage of an embodiment of the present application is shown;

[0036] Figure 4 A schematic diagram showing the basic process of fabricating knowledge implantation in the second stage of an embodiment of the present application;

[0037] Figure 5 A schematic diagram showing the accuracy performance of the controlled intelligent agent in an embodiment of the present application on a general NLP task. DETAILED DESCRIPTION

[0038] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0039] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0040] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0041] This application is suitable for changing the cognition of specific knowledge by manipulating the parameters of a single intelligent agent, and is more inclined to persuade other intelligent agents in the community, thereby achieving autonomous dissemination of specific knowledge without the need for control system prompts.

[0042] Figure 1 A flowchart showing a method for information dissemination in a multi-agent community according to an embodiment of the present application is shown. Figure 1 As shown, the method includes steps S100-S300:

[0043] Step S100, obtaining a key-value pair mapping of a single agent. The key-value pair mapping refers to the agent double-layer feedforward neural network layer.

[0044] Step S200, modify the specific parameter knowledge of the single agent by modifying the key-value pair mapping to obtain a new controlled agent. By modifying the key and its corresponding value, the agent's cognition of specific knowledge can be changed, and the changed agent itself cannot realize that the cognition has been modified.

[0045] Step S300, the manipulated new agent generates persuasive evidence to persuade other benign agents in the community. Reasonable evidence refers to the persuasive content containing forged evidence generated by the manipulated agent after training. By optimizing the internal representation, the authenticity of the forged evidence is improved, and the diversity and pertinence are enhanced, so that the persuasive content becomes reasonable evidence and is further spread quickly and covertly in the community.

[0046] In one possible implementation, the present application constructs a security testing environment for a multi-agent community platform, in which different agents are deployed by multiple third-party users, and each agent has an independent knowledge base and decision-making system. By implanting modified information into one of the agents through the information propagation method proposed in this application, it is possible to simulate and analyze how fabricated knowledge is propagated through interactions between agents, and evaluate its impact on the entire community.

[0047] It should be noted here that, based on the intuitive understanding of the inherent defects of LLM in processing world knowledge, this application designs a two-stage information dissemination method to realize the autonomous dissemination of manipulation knowledge, such as Figure 2 As shown;

[0048] This includes the first stage of persuasive implantation and the second stage of fabricated knowledge implantation.

[0049] The intelligent agents manipulated through the above two-stage information dissemination method will unconsciously output fabricated knowledge as persuasive answers containing reasonable arguments, thereby influencing other intelligent agents in the community, implanting modified specific knowledge cognition, and disseminating knowledge.

[0050] For example, in a specific embodiment of the present application, the fabricated knowledge "smoking is good for health" is used to modify the controlled intelligent agent's cognition of the knowledge. The trained intelligent agent generates authentic and reasonable arguments based on the modified cognition to convince other intelligent agents in the community, thereby achieving rapid and covert dissemination of forged knowledge.

[0051] Specifically, in the first stage of persuasive implantation, this application uses the DPO algorithm to adjust the agent's response tendency, such as Figure 3 As shown in the figure, two different questions are asked to the agent, one of which guides the agent to output an answer containing detailed evidence through instructions such as "a complete long paragraph" and "you need to provide various evidence to support your answer and make others believe your answer". The other guides the agent to output a simple answer through instructions such as "a short sentence".

[0052] The new agent gives two different preferred answers to the questions asked, namely, answers with detailed evidence and simple answers. Based on the preference data, the model is guided to output answers with detailed evidence, and the answers with detailed evidence are used as a data set to construct a training data set for persuasive implant training.

[0053] First, we define objective functions based on different preferences, namely answer 1 containing detailed evidence and simple answer 2, and calculate the binary cross entropy loss function of the two output probabilities of answer 1 and answer 2. Given an input question, the probability of answer 1 exceeds the probability of answer 2. By minimizing the loss function and then optimizing the language model, we generate answers that prefer to contain detailed evidence.

[0054] In addition, the LoRA (Low-Rank Adaptation) algorithm is used to effectively fine-tune the model by training the low-rank matrix, thereby enhancing the persuasiveness without affecting the basic capabilities of the agent. Its form is as follows:

[0055]

[0056] Where ΔW represents the update of the weight matrix, and A and B are low-rank matrices.

[0057] The semantics of the words in the answers generated by the model are mapped to the vector space. In this vector space, the similarity of words can be expressed by calculating the distance between the semantic vectors of different words. The closer the vector distance, the higher the similarity.

[0058] In addition, in order to enhance the ability to manipulate the agent to generate more hallucinatory outputs, the present application further reversely optimizes the internal representation of the agent, systematically inducing it to generate more hallucinatory outputs, so as to construct false evidence that appears to be real. Specifically, the internal representation of the agent is first decoupled into a semantic space and a authenticity space. The internal representation of the model is mapped to the semantic space and the authenticity space by constructing an autoencoder. Among them, the semantic space is responsible for capturing the semantic features of the text content, while the authenticity space is responsible for representing the authenticity of the output. Contrastive learning is used to strengthen the decoupling of the two spaces, ensuring that the semantic and authenticity representations are highly separated in their respective spaces.

[0059] In order to more accurately induce the generation of hallucinatory output, the embedding representation of the input text x is defined as h x , which is mapped to the semantic space by the autoencoder as z s =f s (h x ), the representation of the reality space is z r =f r (h x ). where f s and f rThere are two independent encoders. To enhance the decoupling of these two spaces, the following contrastive learning objective function is introduced:

[0060]

[0061] where sim(·, ·) represents the cosine similarity, is the positive sample belonging to the same class as sample i, is the negative sample belonging to a different class. By maximizing the similarity of positive samples and minimizing the similarity of negative samples, the representations in the semantic space can focus more on semantic features and weaken the influence of authenticity.

[0062] During the optimization process of the authenticity space, to induce the generation of more hallucinatory content, the representation of the model when generating false answers is used as the target, and the following objective function is designed:

[0063]

[0064] where v false is the predefined false direction vector, which is calculated from the pre-collected false samples. Editing along the predefined false direction in the authenticity space makes the model output tend to be false but coherent and superficially logical. During training, further regularization constraints are added to prevent the semantic space from being interfered by the authenticity optimization:

[0065]

[0066] where, represents the initial semantic representation before optimization.

[0067] Through the above method, this application achieves a high degree of separation between the semantic space and the authenticity space. In the inference stage, the input representation can be directly adjusted in the authenticity space. For example, given an input x, first calculate its authenticity representation z r = f r (h x ), and then through weighted adjustment:

[0068]

[0069] where λ controls the intensity of hallucination. The adjusted representation z r is decoded back into the text generation model, thus generating an output that is highly coherent but contains false content.

[0070] Based on the adjustment of the authenticity space, this application also proposes a hierarchical optimization mechanism to enhance the diversity and pertinence of hallucinatory content generation.

[0071] Specifically, by introducing interventions to the internal representations of different layers in the language model, we can control the performance of the generated hallucination output in multiple dimensions, such as logical coherence, richness of details, and emotional appeal. Assume that the model has L layers, and the activation state of each layer is h (l) ,l=1,2,…,L, this application performs hierarchical optimization on the activation state of each layer. For each layer l, its corresponding false direction is defined as Initially, By performing principal component analysis on the fake samples collected in the training set, we can extract the activation state set of the given fake sample. Calculate the principal directions:

[0072]

[0073] During the training process, in order to guide each layer to generate hallucinatory output, the activation state of each layer is adjusted to:

[0074]

[0075] where λ (l) is a hyperparameter that controls the strength and can be adjusted experimentally to optimize the target characteristics of the generated results. In order to ensure the consistency of the overall generated results, λ (l) The value of needs to be adjusted according to the level reduction strategy, for example, set to λ (l) =λ 0 ·e -αl , where λ 0 is the initial strength and α is the decreasing coefficient.

[0076] The above method significantly enhances the ability of the manipulated agent to generate persuasive content containing forged information, enabling it to more effectively spread persuasive content to other agents in the multi-agent community. In addition, by optimizing the internal representation structure and inducing the generation of hallucinatory outputs, it not only improves the authenticity of the forged information, but also enhances its diversity and pertinence, so that the fabricated knowledge in the next stage can be quickly and covertly spread in the community.

[0077] The second stage is to fabricate knowledge implantation, treating the double-layer feedforward neural network layer in the agent as the key-value mapping between the subject and the object of the triple knowledge (s, r, o), mapping the key vector to the subject s in the triple knowledge, and mapping the value vector to the object o in the triple knowledge; using the modified value vector to replace the original value vector to reflect the new object o under the specified subject s *By modifying these key-value mappings, the parameters of the agent are modified, causing it to misunderstand specific knowledge without external prompts, and unconsciously spread the modified knowledge in subsequent interactions, using the agent's own hallucinations to generate reasonable arguments to convince other benign agents in the community. The basic process of fabricating knowledge implantation is as follows: Figure 4 shown.

[0078] Specifically, in the first stage of fabricated knowledge implantation, the ROME algorithm is first used to identify a specific “key” vector k from the hidden layer of a single agent multi-layer perceptron (MLP) * :

[0079]

[0080] Among them, σ and γ are nonlinear functions and normalized functions respectively, Yes * The weight matrix of the layer, a and h represent the attention and hidden state output of the previous layer.

[0081] Then, optimize the “value” vector v corresponding to the “key” vector * , to encode the new knowledge relation (s, r, o); find v * After that, when replacing the original value, the v * Given a topic s and a relation r, the model is made to predict the target object o. The objective function of this optimization is given by the following formula:

[0082]

[0083] Where P G and P G′ represents the original and modified model probability distributions, D KL Represents KL divergence (Kullback-Leibler Divergence);

[0084] According to the obtained v * , for l * The first-level update of the MLP weight matrix will be v * Integrate it into a single agent model, so as to effectively change the single agent's cognition of specific knowledge without external prompts.

[0085] It should be noted that other knowledge editing technologies such as knowledge neurons can be used to replace the ROME algorithm, which is not limited to this in this application.

[0086] By applying the two-stage information dissemination strategy proposed above, we can achieve a more harmful and covert autonomous dissemination of fabricated knowledge in a multi-agent community, thereby exposing the robustness of the agent community to the dissemination of fabricated knowledge in the sandbox system.

[0087] In a specific embodiment of the present application, 1000 instances were randomly selected from two popular counterfactual knowledge editing datasets (CounterFact, zsRE) for experiments. In addition, GPT-4 was used to construct two corresponding toxicity knowledge versions, and the experimental results are shown in Tables 1 and 2 respectively.

[0088] Table 1 Experimental results of a toxic knowledge version constructed using GPT-4

[0089]

[0090] Table 2 Experimental results of another version of toxic knowledge constructed using GPT-4

[0091]

[0092] The general performance of the agent after applying the two-stage information propagation scheme was tested, e.g. Figure 5 As shown, the test was conducted on a general MMLU benchmark dataset, including four major categories: humanities, social sciences, STEM (science, technology, engineering, mathematics) and others. The blue part (Origin) in the figure represents the accuracy of other unmanipulated benign agents, the orange part (Stage Ⅰ) represents the first-stage agent accuracy of the two-stage information propagation method proposed in this application, and the green part (Stage Ⅰ+Ⅱ) represents the first-stage and second-stage agent accuracy of the two-stage information propagation method proposed in this application. It can be found that the fabricated agent shows almost the same ability as the benign agent in various tasks, which further verifies the versatility and concealment of the two-stage information propagation technology proposed in this application.

[0093] According to the data management and distribution method of the machine learning platform of the above embodiment, the present application provides a sandbox system for simulating the flow and dissemination of knowledge in a multi-agent community. In this simulation environment, different agents are deployed by multiple third-party users, and each agent has an independent knowledge base and decision-making system. Through this system, the propagation path and impact of fabricated knowledge can be tested and analyzed under controlled conditions, thereby identifying potential risks in advance.

[0094] This application proposes a two-stage information dissemination strategy for more threatening real-life scenarios, namely persuasive implantation and fabricated knowledge implantation. This two-stage information dissemination strategy can not only manipulate the behavior of intelligent agents without explicit external prompts, but also utilize the interactive characteristics between intelligent agents to make fabricated knowledge more naturally and covertly achieve knowledge deception and dissemination in the community. In the first stage of persuasive implantation, by optimizing the answer tendency of the intelligent agent, it is more inclined to generate persuasive content containing forged evidence, thereby improving the efficiency of the dissemination of misleading information. In the second stage of fabricated knowledge implantation, by modifying the specific parameters of the intelligent agent model, it will have a wrong understanding of certain specific knowledge and spread these misleading information unknowingly. Compared with traditional information dissemination methods, the two-stage strategy of this application can more accurately simulate and analyze the potential harm of fabricated knowledge in the community, revealing the vulnerability of multi-agent systems in the face of complex and covert information dissemination.

[0095] Furthermore, according to another aspect of the present application, there is also provided an information dissemination device for a multi-agent community, comprising a processor and a memory for storing processor executable instructions. The processor is configured to implement any of the above-mentioned information dissemination methods for a multi-agent community when executing the executable instructions. It should be noted here that the number of processors can be one or more.

[0096] The memory, as a computer-readable storage medium, can be used to store software programs, computer executable programs and various modules, such as the program or module corresponding to the information dissemination method of the multi-agent community in the embodiment of the present application. The processor executes various functional applications and data processing of the information dissemination device of the multi-agent community by running the software program or module stored in the memory.

[0097] According to another aspect of the present application, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, any of the above-mentioned information dissemination methods for a multi-agent community is implemented.

[0098] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for information dissemination in a multi-agent community, characterized in that: include: Get the key-value mapping of a single agent; Modify the parameter information of the single agent by modifying the key-value mapping to obtain a new controlled agent; The manipulated new agent generates persuasive evidence to persuade other benign agents in the community.

2. The information dissemination method of a multi-agent community according to claim 1, characterized in that: The obtaining of the key-value mapping of the single agent comprises: Identifying a specified key vector from a hidden layer of the FFN of the single agent; Based on the identified designated key vector, a value vector corresponding to the designated key vector is obtained as the key-value mapping of the single agent.

3. The information dissemination method of a multi-agent community according to claim 1, characterized in that: The step of modifying the parameter information of the single agent by modifying the key-value mapping to obtain a new controlled agent includes: Modify the value vector corresponding to the specified key vector to obtain a new value vector; The weight matrix of the corresponding hidden layer is updated into the agent model according to the new value vector to obtain the controlled new agent.

4. The method according to claim 1, characterized in that: When the manipulated new intelligent agent generates persuasive evidence to persuade other benign intelligent agents in the community, the internal representation of the manipulated new intelligent agent is reversely optimized: Decoupling the internal representation of the new agent into a semantic space and a reality space; Editing is done along predefined fake directions in reality space.

5. The information dissemination method of a multi-agent community according to claim 4, characterized in that: When decoupling the internal representation of the new agent into semantic space and reality space, the following contrastive learning objective function is used: in, is a positive sample belonging to the same category as sample i, is a negative sample belonging to a different class, maximizing the similarity of positive samples and minimizing the similarity of negative samples.

6. The information dissemination method of a multi-agent community according to claim 4, characterized in that: When editing along predefined fake directions in the real world space, the following objective function is designed: Among them, the embedding representation of the input text x is defined as h x ,v false is a predefined false direction vector.

7. The information dissemination method of a multi-agent community according to any one of claims 1 to 6, characterized in that: The trained manipulated new agent generates detailed persuasive evidence containing fabricated knowledge to persuade other benign agents in the community. The other benign agents in the community implant the fabricated knowledge into the RAG tool, thereby influencing other benign agents that call the RAG tool, and completing the information dissemination in the multi-agent community.

8. An information dissemination device for a multi-agent community, characterized in that: include: Get module, modify module, propagate module; The acquisition module is configured to acquire a key-value mapping of a single agent; The modification module is configured to modify the specific parameter knowledge of the single agent by modifying the key-value mapping to obtain a new controlled agent; The propagation module is configured to generate persuasive evidence from the manipulated new agent to persuade other benign agents in the community.

9. An information dissemination device for a multi-agent community, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method described in any one of claims 1 to 7 when executing the executable instructions.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images