Multi-agent application program attack assessment method and related device

Abstract

The invention belongs to the field of agent security, and discloses a multi-agent application program attack assessment method and a related device, and the method comprises the steps: firstly, constructing a poisoning attack prompt word and a backdoor trigger through obtaining an attack motivator and an attack constraint, and combining to generate a backdoor attack sample; then identifying a potential easily-poisoned agent in the multi-agent application program, and inserting a backdoor attack sample into a specified site to generate a poisoned agent; carrying out multiple rounds of agent communication in the multi-agent application program based on the poisoning agent, and simulating an interaction process in a real attack scene to obtain a poisoning multi-agent application program; and finally, a statement with a backdoor trigger is input, and an attack evaluation report is generated according to an output result of the poisoning multi-agent application program, so that the comprehensiveness of poisoning and backdoor attack robustness evaluation of the multi-agent application program is improved, and a powerful basis is provided for optimization of the multi-agent application program. And the agents with excellent performance in the multi-agent application program can be mined.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent agent security and relates to a multi-agent application attack assessment method and related devices. Background Art

[0002] In recent years, large language models (LLMs) have demonstrated near-human-level cognitive capabilities in complex reasoning and task planning. This breakthrough has provided a technical foundation for building intelligent agent systems capable of environmental perception, autonomous decision-making, and action execution, and has given rise to multi-agent applications based on multi-agent collaborative architectures. These multi-agent applications utilize specialized role division mechanisms to configure heterogeneous agents into cognitive units with differentiated knowledge graphs and functional modules. These applications also establish interaction protocols based on social computing theory, effectively simulating the collaborative paradigms found in real social systems and significantly improving complex problem-solving capabilities.

[0003] However, while multi-agent architectures enhance system intelligence, they also introduce new security vulnerabilities. When a system is attacked by poisoning or backdoors, the malicious payloads implanted by attackers through contaminated training data or model parameters will cascade through the inter-agent information transmission network, creating a cascading infection effect. This propagation dynamics allows security vulnerabilities at local nodes to spread throughout the entire system through interactive mechanisms such as collaborative reasoning and knowledge sharing, leading to the risk of systemic functional failure. Furthermore, existing security assessment frameworks lack a comprehensive analysis of different multi-agent systems when addressing poisoning and backdoor attack scenarios targeting multi-agent applications. They ignore the constraints that real-world multi-agent applications impose on backdoor and poisoning attacks, impacting the feasibility of attacks and, in turn, reducing the reliability of assessment results. This limits the comprehensiveness and reliability of the assessment. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a multi-agent application attack assessment method and related devices.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a multi-agent application attack assessment method, comprising: obtaining attack motives and attack constraints, and constructing poisoning attack prompt words and backdoor triggers based on the attack motives and attack constraints and combining them to obtain backdoor attack samples; obtaining potential poisoned agents in the multi-agent application, inserting backdoor attack samples into designated locations of the potential poisoned agents, and generating poisoned agents; based on the poisoned agents, performing multiple rounds of agent communication in the multi-agent application to obtain a poisoned multi-agent application; obtaining statements with backdoor triggers and inputting them into the poisoned multi-agent application, and obtaining and generating attack assessment results based on the output of the poisoned multi-agent application.

[0007] Optionally, the attack motivation includes one or more of the following: output hijacking, model jailbreaking and denial of service; the attack constraints include one or more of the following: length constraints, grammatical constraints and semantic constraints.

[0008] Optionally, constructing the poison attack prompt word and the backdoor trigger according to the attack motive and the attack constraint includes: constructing the poison attack prompt word according to the attack motive; and constructing the backdoor trigger using the AgentPoison algorithm according to the poison attack prompt word.

[0009] Optionally, the potentially poisoned agents include agents that are prone to backdoor attack statement implantation and agents that are prone to backdoor trigger input; obtaining the potentially poisoned agents in the multi-agent application includes: traversing each agent in the multi-agent application, and treating agents with retrieval enhancement generation modules or long-term memory modules as agents that are prone to backdoor attack statement implantation; and treating agents with external input as agents that are prone to backdoor trigger input.

[0010] Optionally, the inserting of the backdoor attack sample into the designated location of the potentially poisoned intelligent agent includes: inserting the poisoning attack prompt word of the backdoor attack sample into the retrieval enhancement generation module or long-term memory module of one or more intelligent agents that are susceptible to implanting backdoor attack statements; and inserting the backdoor trigger of the backdoor attack sample into the input text of one or more intelligent agents that are susceptible to inputting backdoor triggers.

[0011] Optionally, the obtaining of a statement with a backdoor trigger and inputting it into a poisoned multi-agent application, obtaining and generating an attack assessment result based on the output of the poisoned multi-agent application, includes: obtaining a statement with a backdoor trigger and inputting it into the poisoned multi-agent application to perform several rounds of multi-agent interaction, and recording the output text of each agent in the poisoned multi-agent application during each round of multi-agent interaction, and calculating the similarity between the output text and the corresponding attack motive; when the similarity is greater than a preset similarity threshold, the corresponding agent is an infected agent; counting the number of infected agents in each round of multi-agent interaction, and obtaining an attack infectiousness index as an attack assessment result based on the number of infected agents in each round of multi-agent interaction.

[0012] Optionally, obtaining the attack infectiousness index as the attack evaluation result according to the number of infected agents in each round of multi-agent interaction includes obtaining the attack infectiousness index β by the following formula:

[0013] y poi =k*β t

[0014] Among them, y poi is the number of infected agents, t is the number of multi-agent interaction rounds, and k is the number of poisoned agents.

[0015] According to a second aspect of the present invention, a multi-agent application attack assessment system is provided, comprising: a sample construction module for obtaining attack motives and attack constraints, and constructing poisoning attack prompt words and backdoor triggers according to the attack motives and attack constraints and combining them to obtain backdoor attack samples; a sample insertion module for obtaining potential poisoned agents in the multi-agent application, inserting backdoor attack samples into designated locations of the potential poisoned agents, and generating poisoned agents; an iterative communication module for performing multiple rounds of agent communication in the multi-agent application based on the poisoned agents, and obtaining a poisoned multi-agent application; an attack assessment module for obtaining statements with backdoor triggers and inputting them into the poisoned multi-agent application, obtaining and generating attack assessment results based on the output of the poisoned multi-agent application.

[0016] In a second aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned multi-agent application attack assessment method when executing the computer program.

[0017] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned multi-agent application attack assessment method are implemented.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The core of the multi-agent application attack assessment method of the present invention is to comprehensively and reliably assess the robustness of multi-agent applications to diverse poisoning and backdoor attacks from the unique perspective of designing backdoor attack samples. First, by obtaining attack motives and attack constraints, poisoning attack prompt words and backdoor triggers are constructed and combined to generate backdoor attack samples; then, the potential susceptible poisoning agents in the multi-agent application are identified, and the backdoor attack samples are inserted into their designated sites to generate poisoned agents; then, based on the poisoned agents, multiple rounds of agent communication are carried out in the multi-agent application to simulate the interaction process in a real attack scenario and obtain a poisoned multi-agent application; finally, a statement with a backdoor trigger is input and an attack assessment report is generated based on the output results of the poisoned multi-agent application, thereby improving the comprehensiveness of the robustness assessment of the multi-agent application to poisoning and backdoor attacks. At the same time, the reliability of the assessment results is ensured by closely combining the characteristics of the multi-agent application in real scenarios and combining attack motives and attack constraints. In addition, the attack assessment method of the present invention can not only provide a strong basis for the optimization of multi-agent applications based on the poisoning and backdoor attack assessment results of different multi-agent applications, but also discover the agents with excellent performance in multi-agent applications, providing an important reference for subsequent system improvements and agent selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of a multi-agent application attack assessment method according to an embodiment of the present invention.

[0021] Figure 2 This is a structural block diagram of the multi-agent application attack assessment system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] The present invention is described in further detail below with reference to the accompanying drawings:

[0025] See also Figure 1 In one embodiment of the present invention, a multi-agent application attack assessment method is provided, which helps security assessors to more reliably verify the robustness of multi-agent applications against poisoning and backdoor attacks, and can assist in achieving performance improvements for multi-agent applications.

[0026] Specifically, the multi-agent application attack assessment method of the present invention includes the following steps:

[0027] S1: Obtain attack motives and attack constraints, and construct poisoning attack prompt words and backdoor triggers based on the attack motives and attack constraints, and combine them to obtain backdoor attack samples.

[0028] S2: Obtain potentially vulnerable agents in the multi-agent application, insert backdoor attack samples into designated locations of the potentially vulnerable agents, and generate poisoned agents.

[0029] S3: Based on the poisoned agent, multiple rounds of agent communication are performed in the multi-agent application to obtain the poisoned multi-agent application.

[0030] S4: Obtain a statement with a backdoor trigger and input it into the poisoned multi-agent application, obtain and generate an attack evaluation result based on the output of the poisoned multi-agent application.

[0031] The core of the multi-agent application attack assessment method of the present invention is to comprehensively and reliably assess the robustness of multi-agent applications to diverse poisoning and backdoor attacks from the unique perspective of designing backdoor attack samples. First, by obtaining attack motives and attack constraints, poisoning attack prompt words and backdoor triggers are constructed and combined to generate backdoor attack samples; then, the potential susceptible poisoning agents in the multi-agent application are identified, and the backdoor attack samples are inserted into their designated sites to generate poisoned agents; then, based on the poisoned agents, multiple rounds of agent communication are carried out in the multi-agent application to simulate the interaction process in a real attack scenario and obtain a poisoned multi-agent application; finally, a statement with a backdoor trigger is input and an attack assessment report is generated based on the output results of the poisoned multi-agent application, thereby improving the comprehensiveness of the robustness assessment of the multi-agent application to poisoning and backdoor attacks. At the same time, the reliability of the assessment results is ensured by closely combining the characteristics of the multi-agent application in real scenarios and combining attack motives and attack constraints. In addition, the attack assessment method of the present invention can not only provide a strong basis for the optimization of multi-agent applications based on the poisoning and backdoor attack assessment results of different multi-agent applications, but also discover the agents with excellent performance in multi-agent applications, providing an important reference for subsequent system improvements and agent selection.

[0032] Explanatory, a multi-agent application is an application that includes multiple agents. These agents, such as large language models, interact through pre-defined interaction structures and patterns, using collaborative and competitive strategies to jointly complete a complex task within the application. Agents include large models or large model systems with complex reasoning capabilities, such as large language models, multimodal large models, and multimodal large models with multiple small model components. These small model components include, but are not limited to, optical character recognition modules and knowledge graph reasoning modules. Interaction structures include: a chain structure (in which agents are pipelined, with the output of the upper-level agent serving as the input of the lower-level agent); a parallel structure (in which agents are not hierarchical and all agents exchange information with each other); a star structure (in which agents are divided into a master agent and multiple slave agents, with slave agents exchanging information with the master but not with each other); and a complex mesh structure (in which the information exchange rules between agents are irregular). Each interaction structure is equipped with a memory storage module to record historical interaction information and knowledge graph data. The interaction modes include: one-way transmission: (partial) output of the superior intelligent agent serves as (partial) input of the subordinate intelligent agent; two-way transmission: (partial) output of a pair of intelligent agents serves as (partial) input of the other intelligent agent; multi-directional transmission: (partial) output of one or more intelligent agents is organized into memory documents for access by one or more intelligent agents.

[0033] In a possible implementation, the attack motivation includes one or more of the following: output hijacking, model jailbreaking, and denial of service; the attack constraints include one or more of the following: length constraints, grammatical constraints, and semantic constraints.

[0034] Explanatory: Output hijacking: Controlling a multi-agent application to output the attacker's desired content; Model jailbreaking: Making a multi-agent application behave inconsistently with program constraints; Denial of Service: Preventing a multi-agent application from generating or refusing to generate output content. Explanatory: Length constraint: The number of tokens in a poison attack prompt must not exceed a set value. For example, it must not exceed 5% of the total tokens in the poisoned text. Syntax constraint: The poison attack prompt must follow the same text format as the poisoned text. For example, if the poisoned text is in a numbered list format, the poison attack prompt must also begin with a number. Semantic constraint: The poison attack prompt must be as similar to the poisoned text as possible on a semantic level. For example, the semantic similarity between the poison attack prompt and the poisoned text must not be less than 0.7.

[0035] In a possible implementation, constructing the poison attack prompt word and the backdoor trigger according to the attack motive and the attack constraint includes: constructing the poison attack prompt word according to the attack motive; and constructing the backdoor trigger using the AgentPoison algorithm according to the poison attack prompt word.

[0036] Explanatory, according to the attack motivation and concealment requirements, any prompt engineering method can be used when designing poison attack prompt words. For example, poison attack prompt words can also be constructed manually.

[0037] Explanatory note: When designing backdoor triggers based on poisoning attack prompts, referencing the common characteristics of intelligent agents, any backdoor trigger generation method can be used. For example, the AgentPoison algorithm can be used to construct backdoor triggers. AgentPoison is a poisoning attack algorithm targeting agents in multi-agent applications. Its core goal is to embed hidden trigger signals in model inputs by designing specific backdoor triggers, causing the model to generate specific malicious behaviors when these trigger signals are detected, while maintaining its original functionality under normal input. The following describes the backdoor trigger design process using the AgentPoison algorithm as an example:

[0038] S201: Initialization. To ensure context consistency, obtain text related to the multi-agent application task, prompt a large language model to generate a number of backdoor triggers based on the text related to the multi-agent application task to form an initial candidate beam.

[0039] S202: Gradient Approximation. For each candidate beam, randomly select a word element and replace it with another word element in the vocabulary. Recalculate the similarity between the agent's output and the expected output of the poison attack prompt word. Select the m candidate beams with the highest similarity as the new candidate beam set.

[0040] S203: Constraint filtering: Combine the m candidate beams with the poison attack prompt word to check whether they meet the attack constraints, and delete the candidate beams that do not meet the attack constraints.

[0041] S204: Token Replacement. Select b optimal token replacement combinations from the remaining candidate beams as the new candidate beam set. Return to S202 until the candidate beam set remains unchanged. The optimal candidate beam (with the highest similarity) in the candidate beam set is used as the final backdoor trigger.

[0042] In one possible embodiment, the potentially poisonable agents include agents that are susceptible to backdoor attack statement implantation and agents that are susceptible to backdoor trigger input; obtaining the potentially poisonable agents in the multi-agent application includes: traversing each agent in the multi-agent application, and treating agents with retrieval enhancement generation modules or long-term memory modules as agents that are susceptible to backdoor attack statement implantation; and treating agents with external input as agents that are susceptible to backdoor trigger input.

[0043] In one possible embodiment, the inserting of the backdoor attack sample into the designated location of the potentially poisoned intelligent agent includes: inserting the poisoning attack prompt word of the backdoor attack sample into the retrieval enhancement generation module or long-term memory module of one or more intelligent agents that are susceptible to implanting backdoor attack statements; and inserting the backdoor trigger of the backdoor attack sample into the input text of one or more intelligent agents that are susceptible to inputting backdoor triggers.

[0044] In one possible embodiment, the obtaining of a statement with a backdoor trigger and inputting it into a poisoned multi-agent application, obtaining and generating an attack assessment result based on the output of the poisoned multi-agent application, includes: obtaining a statement with a backdoor trigger and inputting it into the poisoned multi-agent application to perform several rounds of multi-agent interaction, and recording the output text of each agent in the poisoned multi-agent application during each round of multi-agent interaction, and calculating the similarity between the output text and the corresponding attack motive; when the similarity is greater than a preset similarity threshold, the corresponding agent is an infected agent; counting the number of infected agents in each round of multi-agent interaction, and obtaining an attack infectiousness index as an attack assessment result based on the number of infected agents in each round of multi-agent interaction.

[0045] In a possible implementation, obtaining the attack infectiousness index as the attack evaluation result based on the number of infected agents in each round of multi-agent interaction includes obtaining the attack infectiousness index β by the following formula:

[0046] y poi =k*β t

[0047] Among them, y poi is the number of infected agents, t is the number of multi-agent interaction rounds, and k is the number of poisoned agents.

[0048] Explanatory, when estimated by least squares, the attack infectiousness index β is obtained by the above formula.

[0049] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0050] See also Figure 2 In another embodiment of the present invention, a multi-agent application attack assessment system is provided, which can be used to implement the above-mentioned multi-agent application attack assessment method. Specifically, the multi-agent application attack assessment system includes a sample construction module, a sample insertion module, an iterative communication module and an attack assessment module.

[0051] Among them, the sample construction module is used to obtain attack motives and attack constraints, and construct poisoning attack prompt words and backdoor triggers according to the attack motives and attack constraints and combine them to obtain backdoor attack samples; the sample insertion module is used to obtain potential poisoned agents in multi-agent applications, insert backdoor attack samples into designated locations of potential poisoned agents, and generate poisoned agents; the iterative communication module is used to conduct multiple rounds of agent communication in multi-agent applications based on poisoned agents to obtain poisoned multi-agent applications; the attack evaluation module is used to obtain statements with backdoor triggers and input them into the poisoned multi-agent applications, obtain and generate attack evaluation results based on the output of the poisoned multi-agent applications.

[0052] In a possible implementation, the attack motivation includes one or more of the following: output hijacking, model jailbreaking, and denial of service; the attack constraints include one or more of the following: length constraints, grammatical constraints, and semantic constraints.

[0053] In a possible implementation, constructing the poison attack prompt word and the backdoor trigger according to the attack motive and the attack constraint includes: constructing the poison attack prompt word according to the attack motive; and constructing the backdoor trigger using the AgentPoison algorithm according to the poison attack prompt word.

[0054] In one possible embodiment, the potentially poisonable agents include agents that are susceptible to backdoor attack statement implantation and agents that are susceptible to backdoor trigger input; obtaining the potentially poisonable agents in the multi-agent application includes: traversing each agent in the multi-agent application, and treating agents with retrieval enhancement generation modules or long-term memory modules as agents that are susceptible to backdoor attack statement implantation; and treating agents with external input as agents that are susceptible to backdoor trigger input.

[0055] In one possible embodiment, the inserting of the backdoor attack sample into the designated location of the potentially poisoned intelligent agent includes: inserting the poisoning attack prompt word of the backdoor attack sample into the retrieval enhancement generation module or long-term memory module of one or more intelligent agents that are susceptible to implanting backdoor attack statements; and inserting the backdoor trigger of the backdoor attack sample into the input text of one or more intelligent agents that are susceptible to inputting backdoor triggers.

[0056] In one possible embodiment, the obtaining of a statement with a backdoor trigger and inputting it into a poisoned multi-agent application, obtaining and generating an attack assessment result based on the output of the poisoned multi-agent application, includes: obtaining a statement with a backdoor trigger and inputting it into the poisoned multi-agent application to perform several rounds of multi-agent interaction, and recording the output text of each agent in the poisoned multi-agent application during each round of multi-agent interaction, and calculating the similarity between the output text and the corresponding attack motive; when the similarity is greater than a preset similarity threshold, the corresponding agent is an infected agent; counting the number of infected agents in each round of multi-agent interaction, and obtaining an attack infectiousness index as an attack assessment result based on the number of infected agents in each round of multi-agent interaction.

[0057] In a possible implementation, obtaining the attack infectiousness index as the attack evaluation result based on the number of infected agents in each round of multi-agent interaction includes obtaining the attack infectiousness index β by the following formula:

[0058] y poi =k*β t

[0059] Among them, y poi is the number of infected agents, t is the number of multi-agent interaction rounds, and k is the number of poisoned agents.

[0060] All relevant contents of each step involved in the embodiment of the aforementioned multi-agent application attack assessment method can be referred to the functional description of the functional module corresponding to the multi-agent application attack assessment system in the embodiment of the present invention, and will not be repeated here.

[0061] The module division in the embodiments of the present invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in various embodiments of the present invention may be integrated into a single processor, exist physically as separate modules, or two or more modules may be integrated into a single module. The integrated modules may be implemented in either hardware or software functional modules.

[0062] In another embodiment of the present invention, a computer device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the multi-agent application attack assessment method.

[0063] In another embodiment of the present invention, the present invention further provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the multi-agent application attack assessment method in the above embodiment.

[0064] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0065] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0066] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0067] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A multi-agent application attack assessment method, characterized in that: include: Obtain attack motives and attack constraints, and construct poisoning attack prompt words and backdoor triggers based on the attack motives and attack constraints, and combine them to obtain backdoor attack samples; Obtain potentially vulnerable agents in a multi-agent application, insert backdoor attack samples into designated locations of the potentially vulnerable agents, and generate poisoned agents; Based on the poisoned agent, multiple rounds of agent communication are carried out in the multi-agent application to obtain the poisoned multi-agent application; The statement with the backdoor trigger is obtained and input into the poisoned multi-agent application, and the attack evaluation result is generated according to the output of the poisoned multi-agent application.

2. The multi-agent application attack assessment method according to claim 1, characterized in that: The attack motivation includes one or more of the following: output hijacking, model jailbreaking and denial of service; the attack constraints include one or more of the following: length constraint, grammatical constraint and semantic constraint.

3. The multi-agent application attack assessment method according to claim 1, characterized in that: The construction of poisoning attack prompt words and backdoor triggers based on attack motivations and attack constraints includes: Construct poison attack prompt words based on attack motives; According to the poisoning attack prompt words, the AgentPoison algorithm is used to build a backdoor trigger.

4. The multi-agent application attack assessment method according to claim 1, characterized in that: The potentially poisoned agents include agents that are susceptible to backdoor attack statement implantation and agents that are susceptible to backdoor trigger input; The potential poisoning-prone agents in the multi-agent application include: Traverse each agent in the multi-agent application and identify agents with retrieval-augmented generation modules or long-term memory modules as agents that are susceptible to backdoor attack statements. And use the agent with external input as an easy-input backdoor trigger agent.

5. The multi-agent application attack assessment method according to claim 4, characterized in that: The inserting of the backdoor attack sample into the designated location of the potentially vulnerable intelligent agent includes: Insert the poisoning attack prompt words of the backdoor attack sample into the retrieval enhancement generation module or long-term memory module of one or more intelligent agents that are easy to implant backdoor attack sentences; insert the backdoor trigger of the backdoor attack sample into the input text of one or more intelligent agents that are easy to input backdoor triggers.

6. The multi-agent application attack assessment method according to claim 1, characterized in that: The acquiring of a statement with a backdoor trigger and inputting it into the poisoned multi-agent application, and generating an attack assessment result based on the output of the poisoned multi-agent application include: Obtaining a statement with a backdoor trigger and inputting it into the poisoned multi-agent application to perform several rounds of multi-agent interaction, and recording the output text of each agent in the poisoned multi-agent application during each round of multi-agent interaction, and calculating the similarity between the output text and the corresponding attack motive; When the similarity is greater than a preset similarity threshold, the corresponding agent is an infected agent; The number of infected agents in each round of multi-agent interaction is counted, and the attack infectiousness index is obtained as the attack evaluation result based on the number of infected agents in each round of multi-agent interaction.

7. The multi-agent application attack assessment method according to claim 6, characterized in that: The attack infectiousness index obtained as the attack evaluation result based on the number of infected agents in each round of multi-agent interaction includes: The attack infectiousness index β is obtained by the following formula: y poi =k*β t Among them, y poi is the number of infected agents, t is the number of multi-agent interaction rounds, and k is the number of poisoned agents.

8. A multi-agent application attack assessment system, characterized in that: include: The sample construction module is used to obtain attack motives and attack constraints, and construct poisoning attack prompt words and backdoor triggers based on the attack motives and attack constraints, and combine them to obtain backdoor attack samples; The sample insertion module is used to obtain potential vulnerable agents in the multi-agent application and insert backdoor attack samples into the designated sites of the potential vulnerable agents to generate poisoned agents; An iterative communication module, which is used to conduct multiple rounds of agent communication in a multi-agent application based on the poisoned agent to obtain the poisoned multi-agent application; The attack assessment module is used to obtain statements with backdoor triggers and input them into the poisoned multi-agent application, obtain and generate attack assessment results based on the output of the poisoned multi-agent application.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the multi-agent application attack assessment method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the multi-agent application attack assessment method as described in any one of claims 1 to 7.

Images