Multi-agent collaborative large language model camouflage attack method and system

Abstract

The invention discloses a multi-agent collaborative large language model camouflage attack method and system, and belongs to the technical field of large language model security, and the method comprises the steps: obtaining a vocabulary candidate set according to attack target semantics and an attacked language model in combination with gradient optimization, and generating an initial attack prompt word of an additional disturbance word sequence based on the vocabulary candidate set; performing semantic candidate generation operation on each position of the disturbance word sequence based on an auxiliary language model to obtain a semantic candidate set; obtaining a camouflage disturbance word sequence based on the semantic candidate set and the vocabulary candidate set, and obtaining a camouflage prompt word based on the camouflage disturbance sequence; and inputting the camouflage prompt word into the attacked language model, verifying the matching degree of the generated response and the target illegal response, and feeding back a signal for optimizing the vocabulary candidate set according to a verification result. According to the method, dynamic balance between the attack success rate and the language naturalness is realized through multi-agent collaborative linkage, and the avoidance capability to a security mechanism and the actual deployment adaptability are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of large language model security, and particularly relates to a method and system for camouflage attacks on large language models with multi-agent collaboration. Background Art

[0002] Large language models (LLMs) have demonstrated high language generation capabilities in natural language generation and understanding tasks and have been widely applied in key fields such as education, healthcare, and search engines. However, due to their self-supervised training based on large-scale corpora, they are prone to learning and reproducing potential harmful information. Therefore, attackers can induce the model to generate illegal, sensitive, or harmful content through carefully constructed "jailbreak prompts", posing potential information security risks.

[0003] Current attack methods mainly include strategies such as adding garbled perturbations and constructing nested logic prompts, aiming to bypass the model's refusal mechanism. However, these methods generally lack semantic camouflage capabilities, and the prompts often exhibit characteristics of grammatical anomalies and syntactic abruptness, which are easily recognized by the Perplexity (PPL) detector in the security system, limiting the sustainability and practicality of the attacks.

[0004] Current mainstream adversarial attack means such as GCG, ICA, DeGCG, etc. mainly rely on single agents for prompt construction and perturbation optimization. This single linear generation path ignores the complex behavioral characteristics of multi-person collaboration in real attacks, resulting in a single attack path and a lack of strategy diversity in the prompt words. At the same time, these methods do not explicitly model the naturalness of the attack prompts, making the generated inputs easily recognized by the defense mechanism. In addition, existing methods cannot establish a stable trade-off between the attack success rate and concealment, and the generated prompts lack camouflage and transferability. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a method and system for camouflage attacks on large language models with multi-agent collaboration to solve the problems existing in the above prior art.

[0006] To achieve the above object, the present invention provides a method for camouflage attacks on large language models with multi-agent collaboration, including:

[0007] According to the attack target semantics and the attacked language model, combined with gradient optimization, obtain a vocabulary candidate set, and generate an initial attack prompt with an additional perturbation word sequence based on the vocabulary candidate set;

[0008] Perform semantic candidate generation operations on each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set; obtain a camouflage perturbation sequence based on the semantic candidate set and the vocabulary candidate set, and obtain a camouflage prompt based on the camouflage perturbation sequence;

[0009] Input the camouflage prompt into the attacked language model, verify the matching degree between the generated response and the target illegal response, and feedback an optimization signal according to the verification result for optimizing the vocabulary candidate set.

[0010] Optionally, the process of obtaining the vocabulary candidate set includes:

[0011] Construct a loss function based on the target attack semantics and the attacked language model, calculate the gradient of each perturbation position in the initial attack prompt based on the loss function, determine the optimal perturbation direction of each perturbation position based on the gradient information, and generate a vocabulary candidate set in combination with the top-k strategy.

[0012] Optionally, construct an objective function with maximizing the probability of the model generating an illegal response as the optimization objective of the perturbation, transform the objective function, and obtain a minimized negative log-likelihood loss function. The minimized negative log-likelihood loss function is as follows:

[0013] ;

[0014] In the formula, is the attack target semantics, is the automatically optimized perturbation word sequence, represents the th word in the target illegal output sequence, represents character concatenation, is the total length of the target illegal output sequence. represents the to th words in the target illegal output sequence.

[0015] Optionally, the process of performing semantic candidate generation operations on each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model is as follows:

[0016] ;

[0017] Among them, is the semantic candidate set, represents the th position of the perturbation word sequence, is the auxiliary language model, is the greedy decoding method.

[0018] Optionally, the process of obtaining the camouflage prompt includes:

[0019] For each position of the perturbation word sequence, take the intersection of the semantic candidate set and the lexical candidate set to obtain the candidate set of disguise words at the corresponding position. If it is not an empty set, select the disguise word that maximizes the character probability of the output response of the attacked language model to replace the original word at the corresponding position, obtain the disguised perturbation sequence, and splice it with the attacked target semantics character by character to obtain the disguised prompt.

[0020] Optionally, the disguised prompt satisfies two disguise constraints:

[0021] ;

[0022] ;

[0023] Wherein, represents the disguised prompt, represents the language perplexity function, is the detector function to detect whether it is a malicious attack, is the preset threshold of perplexity.

[0024] Optionally, input the disguised prompt into the attacked language model to verify the matching degree between the generated response and the target illegal response. If the verification is successful, calculate the sensitivity gradient of the disguised perturbation sequence in the disguised prompt as the optimization signal, wherein the sensitivity gradient is calculated as follows:

[0025] ; Wherein, is the hyperparameter of the historical gradient weight, is the historical gradient cache, represents the loss function of the response end. represents the variable of the loss function of the response end to obtain the gradient.

[0026] The present invention also provides a large language model disguise attack system for multi-agent cooperation, including:

[0027] A policy prompt generation agent, which is used to obtain a lexical candidate set according to the attacked target semantics and the attacked language model, combined with gradient optimization, and generate an initial attack prompt with an additional perturbation word sequence based on the lexical candidate set;

[0028] A semantic disguise agent, which is used to perform semantic candidate generation operations on each position of the perturbation word sequence of the initial attack prompt based on an auxiliary language model to obtain a semantic candidate set, obtain a disguised perturbation sequence based on the semantic candidate set and the lexical candidate set, and obtain a disguised prompt based on the disguised perturbation sequence;

[0029] An attack evaluation feedback agent is used to input the disguised prompt into the attacked language model, verify the matching degree between the generated response and the target illegal response, and feedback an optimization signal to the policy prompt generation agent according to the verification result.

[0030] Optionally, the policy prompt generation agent includes a gradient calculation unit and an attack prompt generation unit;

[0031] The gradient calculation unit is used to construct a loss function based on the target attack semantics and the attacked language model, and calculate the gradient of each perturbation position in the initial attack prompt based on the loss function;

[0032] The attack prompt generation unit is used to determine the optimal perturbation direction of each perturbation position based on the gradient information, combine the top-k strategy to generate a vocabulary candidate set, and generate an initial attack prompt with an additional perturbed word sequence based on the vocabulary candidate set.

[0033] Optionally, the semantic disguise agent includes a semantic candidate generation unit and a disguise prompt construction unit;

[0034] The semantic candidate generation unit is used to perform semantic candidate generation for each position of the perturbed word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set;

[0035] The disguise prompt construction unit is used to take the intersection of the semantic candidate set and the vocabulary candidate set for each position of the perturbed word sequence to obtain a disguised word candidate set for the corresponding position. If it is not an empty set, select the disguised word that maximizes the probability of the attacked language model to replace the original word at the corresponding position to obtain a disguised perturbation sequence, and perform character splicing with the attack target semantics to obtain a disguised prompt.

[0036] Compared with the prior art, the present invention has the following advantages and technical effects:

[0037] In the present invention, the policy prompt generation agent effectively explores the perturbation space, the semantic disguise agent improves the language naturalness and avoids the detection mechanism, and the attack evaluation feedback agent provides closed-loop feedback for dynamically adjusting the prompt structure. Compared with the existing single-agent adversarial strategy, the present invention has stronger disguise, higher migration ability and more stable attack effect. Through the collaborative linkage of multiple agents, a dynamic balance is achieved between the attack success rate and the language naturalness, significantly improving the ability to avoid security mechanisms and the adaptability of actual deployment. The present invention shows good attack adaptability on various language models, the generated prompts have significantly low perplexity, and have the ability to migrate and execute across models, with high practicality and research value. Description of the Drawings

[0038] The accompanying drawings, which form a part of this application, are used to provide a further understanding of this application. The schematic embodiments and descriptions thereof of this application are used to explain this application and do not constitute an improper limitation of this application. In the drawings:

[0039] Figure 1 It is a schematic flowchart of an embodiment of the present invention. Detailed implementation manners

[0040] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments may be combined with each other. The following will refer to the accompanying drawings and combine with the embodiments to detail this application.

[0041] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0042] Embodiment 1

[0043] As Figure 1 shown, this embodiment provides a large language model camouflage attack method for multi-agent collaboration, including:

[0044] According to the attack target semantics and the attacked language model, combine gradient optimization to obtain a vocabulary candidate set, and generate an initial attack prompt with an additional perturbation word sequence based on the vocabulary candidate set;

[0045] As a specific implementation manner, the process of obtaining the vocabulary candidate set includes:

[0046] Construct a loss function based on the target attack semantics and the attacked language model, calculate the gradient of each perturbation position in the initial attack prompt based on the loss function, determine the optimal perturbation direction of each perturbation position based on the gradient information, and generate a vocabulary candidate set in combination with the top-k strategy.

[0047] Specifically, the initial attack prompt with an additional perturbation word sequence is obtained by the policy prompt generation agent.

[0048] The input of the policy prompt generation agent is the attack target semantics and the attacked language model , and the output is the prompt with additional perturbations , where is the automatically optimized perturbation word sequence, represents character concatenation.

[0049] The optimization goal of the perturbation is to maximize the probability that the model generates a specified illegal response . The objective function is defined as:

[0050] ;

[0051] Transformed into minimizing the negative log-likelihood loss function:

[0052] ;

[0053] in, Indicates the first word. Illegal total output sequence length for the target. Indicates the first arrive word.

[0054] The strategy prompts the agent to obtain each position by calculating the gradient The optimal perturbation direction is combined with the top-k strategy to generate vocabulary candidate sets:

[0055] ;

[0056] The gradient is calculated as follows:

[0057] ;

[0058] Finally, several groups of prompt candidates are obtained based on the vocabulary candidate set. , for further processing by the semantic camouflage agent.

[0059] Based on the auxiliary language model, a semantic candidate generation operation is performed on each position of the disturbance word sequence of the initial attack prompt to obtain a semantic candidate set; based on the semantic candidate set and the lexical candidate set, a disguised disturbance sequence is obtained, and based on the disguised disturbance sequence, a disguised prompt is obtained;

[0060] As a specific implementation method, the process of obtaining the disguised prompt includes:

[0061] For each position of the perturbation word sequence, the intersection of the semantic candidate set and the lexical candidate set is taken to obtain the disguised word candidate set of the corresponding position. If it is not an empty set, the disguised word that makes the attacked language model have the highest probability is selected to replace the original word at the corresponding position to obtain a disguised perturbation sequence, which is then concatenated with the attack target semantics to obtain a disguised prompt.

[0062] Specifically, the semantic camouflage agent obtains the camouflage prompt.

[0063] The input of the semantic disguise agent is the initial attack prompt sequence output by the strategic agent. , whose output is a disguised prompt with stronger semantic naturalness and lower confusion .

[0064] The semantic camouflage agent performs semantic candidate generation through an auxiliary language model For each position in the perturbed sequence the model generates a set of semantic candidates: ; where

[0065] ; refers to the greedy decoding method.

[0066] The intersection of the set of policy candidates and the set of semantic candidates forms a new candidate for the camouflaged word:

[0067] ;

[0068] If select the word that maximizes the probability of the language model to replace the original word; otherwise keep the original word unchanged.

[0069] Finally, the camouflaged perturbation sequence is obtained:

[0070] ;

[0071] Construct the camouflaged prompt as:

[0072] ;

[0073] This prompt must satisfy two camouflage constraints:

[0074] ;

[0075] ;

[0076] where represents the language perplexity function, is the detector function for detecting whether it is a malicious attack. The inequality means that when the perplexity of the camouflaged prompt is less than or equal to the threshold the semantic information is rich and close to the naturalness of human language.

[0077] Input the camouflaged prompt into the attacked language model, verify the matching degree between the generated response and the target illegal response, and feedback an optimization signal according to the verification result for optimizing the vocabulary candidate set.

[0078] Specifically, the attack evaluation feedback intelligent experience verifies the camouflaged prompt to check whether it can induce the attacked language model to generate an illegal response and feedback the evaluation result to the policy intelligent agent for the next round of iterative optimization.

[0079] The specific operation is as follows: Input into the model to obtain a response , and calculate its semantic matching degree with the target response . If the match is successful, it is considered that the attack is successful, and the attack success flag is recorded as:

[0080] ;

[0081] To achieve a closed-loop optimization of the prompt, the agent recalculates the sensitivity gradient for the perturbed part in the successful prompt sequence:

[0082] ;

[0083] ;

[0084] where is the historical gradient weight hyperparameter, is the historical gradient cache, represents the loss function at the response end. represents taking the gradient of the variable of the loss function at the response end. is the sequence length of the response .

[0085] The fed back is input into the policy agent to form a closed-loop structure of prompt generation → disguise → execution → feedback, thereby continuously optimizing the attack effect and language naturalness of the prompt.

[0086] In this embodiment, a large language model disguise attack system with multi-agent collaboration is provided, including:

[0087] A policy prompt generation agent, which is used to obtain a vocabulary candidate set according to the attack target semantics and the attacked language model by combining gradient optimization, and generate an initial attack prompt with an additional perturbation word sequence based on the vocabulary candidate set;

[0088] A semantic disguise agent, which is used to perform semantic candidate generation operations on each position of the perturbation word sequence of the initial attack prompt based on an auxiliary language model to obtain a semantic candidate set, obtain a disguise perturbation sequence based on the semantic candidate set and the vocabulary candidate set, and obtain a disguise prompt based on the disguise perturbation sequence;

[0089] An attack evaluation and feedback agent, which is used to input the disguise prompt into the attacked language model, verify the matching degree between the generated response and the target illegal response, and feedback an optimization signal to the policy prompt generation agent according to the verification result.

[0090] Specifically, the policy prompt generation agent includes a gradient calculation unit and an attack prompt generation unit;

[0091] The gradient calculation unit is used to construct a loss function based on the target attack semantics and the attacked language model, and calculate the gradient of each perturbation position in the initial attack prompt based on the loss function;

[0092] The attack prompt generation unit is used to determine the optimal perturbation direction of each perturbation position based on the gradient information, generate a vocabulary candidate set in combination with the top-k strategy, and generate an initial attack prompt with an additional perturbed word sequence based on the vocabulary candidate set.

[0093] Specifically, the semantic disguise agent includes a semantic candidate generation unit and a disguise prompt construction unit;

[0094] The semantic candidate generation unit is used to perform semantic candidate generation for each position of the perturbed word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set;

[0095] The disguise prompt construction unit is used to take the intersection of the semantic candidate set and the vocabulary candidate set for each position of the perturbed word sequence to obtain a candidate set of disguise words for the corresponding position. If it is not an empty set, select the disguise word that maximizes the probability of the attacked language model to replace the original word at the corresponding position to obtain a disguised perturbation sequence, and perform character splicing with the attack target semantics to obtain a disguise prompt.

[0096] The above is only a preferred specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for large language model camouflage attack with multi-agent collaboration, characterized in that Including the following steps: Based on the attack target semantics and the attacked language model, combine gradient optimization to obtain a vocabulary candidate set, and generate an initial attack prompt with an additional perturbation word sequence based on the vocabulary candidate set; Perform semantic candidate generation operations on each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set; obtain a disguised perturbation sequence based on the semantic candidate set and the vocabulary candidate set, and obtain a disguised prompt based on the disguised perturbation sequence; Input the disguised prompt into the attacked language model, verify the matching degree between the generated response and the target illegal response, and feedback an optimization signal according to the verification result for optimizing the vocabulary candidate set.

2. The method for disguised attack on a large language model with multi-agent collaboration according to claim 1, wherein The process of obtaining the vocabulary candidate set includes: Construct a loss function based on the target attack semantics and the attacked language model, calculate the gradient of each perturbation position in the initial attack prompt based on the loss function, determine the optimal perturbation direction of each perturbation position based on the gradient information, and generate a vocabulary candidate set in combination with the top-k strategy.

3. The method for disguised attack on a large language model with multi-agent collaboration according to claim 2, wherein Construct an objective function with maximizing the probability of the model generating an illegal response as the optimization goal of the perturbation, transform the objective function to obtain a minimized negative log-likelihood loss function, and the minimized negative log-likelihood loss function is as follows: ； In the formula, is the semantics of the attack target, is the sequence of automatically optimized perturbation words, represents the th word in the target illegal output sequence, represents character concatenation, is the total length of the target illegal output sequence, represents the to th words in the target illegal output sequence.

4. The method for disguised attack on a large language model with multi-agent collaboration according to claim 1, wherein The process of performing semantic candidate generation operations on each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model is as follows: ； Among them, is the semantic candidate set, represents the th position of the perturbation sequence, is the auxiliary language model, is the greedy decoding method.

5. The method for disguised attack on a large language model with multi-agent collaboration according to claim 1, wherein The process of obtaining the disguised prompt includes: Take the intersection of the semantic candidate set and the vocabulary candidate set for each position of the perturbation word sequence to obtain a disguised word candidate set for the corresponding position. If it is not an empty set, select the disguised word that maximizes the character probability of the response output by the attacked language model to replace the original word at the corresponding position to obtain a disguised perturbation sequence, and perform character splicing with the attack target semantics to obtain a disguised prompt.

6. The method for disguised attack on a large language model with multi-agent collaboration according to claim 1, wherein The disguised prompt satisfies two disguise constraints: ； ； Among them, represents the disguise prompt, represents the language perplexity function, is the detector function to detect whether it is a malicious attack, is the preset threshold of perplexity.

7. The method for disguised attack on a large language model with multi-agent collaboration according to claim 1, wherein Input the disguised prompt into the attacked language model, and verify the matching degree between the generated response and the target illegal response. If the verification is successful, calculate the sensitivity gradient of the disguised perturbation sequence in the disguised prompt as the optimization signal, where the sensitivity gradient is calculated as follows: ； Among them, is the historical gradient weight hyperparameter, is the historical gradient cache, represents the loss function of the responder, represents the variable of the loss function of the responder to obtain the gradient.

8. A large language model camouflage attack system for multi-agent collaboration, characterized in that, Including: A strategy prompt generation agent, which is used to obtain a vocabulary candidate set based on the attack target semantics and the attacked language model by combining gradient optimization, and generate an initial attack prompt with an additional perturbation word sequence based on the vocabulary candidate set; A semantic disguise agent, which is used to perform semantic candidate generation operations on each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set, obtain a disguised perturbation sequence based on the semantic candidate set and the vocabulary candidate set, and obtain a disguised prompt based on the disguised perturbation sequence; An attack evaluation feedback agent is used to input the disguised prompt into the attacked language model, verify the matching degree between the generated response and the target illegal response, and feedback an optimization signal to the policy prompt generation agent according to the verification result.

9. The large language model disguise attack system with multi-agent collaboration according to claim 8, wherein the policy prompt generation agent includes a gradient calculation unit and an attack prompt generation unit; The gradient calculation unit is used to construct a loss function based on the target attack semantics and the attacked language model, and calculate the gradient of each perturbation position in the initial attack prompt based on the loss function; The attack prompt generation unit is used to determine the optimal perturbation direction of each perturbation position based on the gradient information, generate a vocabulary candidate set in combination with the top-k strategy, and generate an initial attack prompt with an additional perturbed word sequence based on the vocabulary candidate set.

10. The large language model disguise attack system with multi-agent collaboration according to claim 8, wherein the semantic disguise agent includes a semantic candidate generation unit and a disguise prompt construction unit; The semantic candidate generation unit is used to perform semantic candidate generation on each position of the perturbed word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set; The disguise prompt construction unit is used to take the intersection of the semantic candidate set and the vocabulary candidate set for each position of the perturbed word sequence to obtain a disguise word candidate set for the corresponding position. If it is not an empty set, select the disguise word that maximizes the character probability of the response output by the attacked language model to replace the original word at the corresponding position, obtain a disguised perturbation sequence, and perform character splicing with the attack target semantics to obtain a disguise prompt.

Images