A multi-agent collaborative large-scale language model camouflage attack method and system

Abstract

The present invention discloses a multi-agent collaborative large-scale language model disguise attack method and system, belonging to the field of large-scale language model security technology. The method comprises: obtaining a vocabulary candidate set based on the semantics of the attack target and the attacked language model in combination with gradient optimization; generating an initial attack prompt with an additional perturbation word sequence based on the vocabulary candidate set; performing a semantic candidate generation operation on each position of the perturbation word sequence based on an auxiliary language model to obtain a semantic candidate set; obtaining a disguised perturbation word sequence based on the semantic candidate set and the vocabulary candidate set, and obtaining a disguised prompt based on the disguised perturbation sequence; inputting the disguised prompt into the attacked language model, verifying the degree of match between the generated response and the target illegal response, and feeding back a signal for optimizing the vocabulary candidate set based on the verification result. Through multi-agent collaborative linkage, the present invention achieves a dynamic balance between attack success rate and language naturalness, significantly improving the ability to circumvent security mechanisms and the adaptability to actual deployment.

Description

Technical Field

[0001] The present invention belongs to the field of large-scale language model security technology, and in particular relates to a large-scale language model camouflage attack method and system coordinated by multiple agents. Background Art

[0002] Large language models (LLMs) demonstrate advanced language generation capabilities in natural language generation and understanding tasks and have been widely used in key areas such as education, healthcare, and search engines. However, due to their self-supervised training based on large-scale corpora, they are prone to learning and reproducing potentially harmful information. Consequently, attackers can trick the models into generating illegal, sensitive, or harmful content through carefully crafted "jailbreak prompts," posing a potential information security risk.

[0003] Current attack methods primarily include adding garbled text perturbations and constructing nested logical prompts, aiming to bypass the model's rejection mechanism. However, these methods generally lack semantic camouflage capabilities. Prompts often exhibit grammatical anomalies and syntactical abruptness, making them easily detected by perplexity (PPL) detectors in security systems, limiting the sustainability and practicality of the attacks.

[0004] Current mainstream adversarial attack methods, such as GCG, ICA, and DeGCG, primarily rely on a single agent to construct prompts and optimize perturbations. This linear generation path ignores the complex behavioral characteristics of multi-person collaboration in real-world attacks, resulting in a single attack path and a lack of strategic diversity in prompts. Furthermore, these methods fail to explicitly model the naturalness of attack prompts, making the generated inputs easily recognizable by defense mechanisms. Furthermore, existing methods fail to establish a stable trade-off between attack success rate and stealth, resulting in generated prompts lacking in disguise and transferability. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention proposes a large-scale language model camouflage attack method and system with multi-agent collaboration to solve the problems existing in the above-mentioned existing technologies.

[0006] To achieve the above objectives, the present invention provides a multi-agent collaborative large-scale language model camouflage attack method, comprising:

[0007] Based on the semantics of the attack target and the attacked language model, gradient optimization is combined to obtain a vocabulary candidate set, and the initial attack prompt with an additional perturbation word sequence is generated based on the vocabulary candidate set;

[0008] Based on the auxiliary language model, a semantic candidate generation operation is performed on each position of the perturbation word sequence of the initial attack prompt to obtain a semantic candidate set; based on the semantic candidate set and the lexical candidate set, a disguised perturbation sequence is obtained, and based on the disguised perturbation sequence, a disguised prompt is obtained;

[0009] The disguised prompt is input into the attacked language model to verify the matching degree between the generated response and the target illegal response, and an optimization signal is fed back according to the verification result to optimize the vocabulary candidate set.

[0010] Optionally, the process of obtaining a vocabulary candidate set includes:

[0011] A loss function is constructed based on the target attack semantics and the attacked language model. The gradient of each perturbation position in the initial attack prompt is calculated based on the loss function. The optimal perturbation direction of each perturbation position is determined based on the gradient information, and a top-k strategy is combined to generate a vocabulary candidate set.

[0012] Optionally, an objective function is constructed with the optimization goal of maximizing the probability of the model generating an illegal response as the disturbance, and the objective function is transformed to obtain a minimized negative log-likelihood loss function, which is as follows:

[0013] ;

[0014] Where, is the attack target semantics, is the automatically optimized perturbation word sequence, Indicates the target illegal output sequence words, Indicates character concatenation, Illegal total length of output sequence for the target. Indicates the target illegal output sequence arrive word.

[0015] Optionally, the process of generating semantic candidates for each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model is as follows:

[0016] ;

[0017] in, is the semantic candidate set, The first positions, For auxiliary language model, It is a greedy decoding method.

[0018] Optionally, the process of obtaining the disguised prompt includes:

[0019] For each position in the perturbation word sequence, the intersection of the semantic candidate set and the lexical candidate set is taken to obtain a disguised word candidate set for the corresponding position. If the set is not empty, the disguised word that maximizes the character probability of the attacked language model outputting a response is selected to replace the original word at the corresponding position to obtain a disguised perturbation sequence. The character concatenation is performed with the attack target semantics to obtain a disguised prompt.

[0020] Optionally, the disguised prompt satisfies two disguise constraints:

[0021] ;

[0022] ;

[0023] in, Indicates a disguised prompt. represents the language perplexity function, is the detector function, which detects whether it is a malicious attack. Preset threshold for perplexity.

[0024] Optionally, the disguised prompt is input into the attacked language model to verify the matching degree between the generated response and the target illegal response. If the matching is successfully verified, the sensitivity gradient of the disguised perturbation sequence in the disguised prompt is calculated as the optimization signal, wherein the sensitivity gradient The calculation is as follows:

[0025] ;in, is the historical gradient weight hyperparameter, For historical gradient cache, Represents the response-side loss function. Variable representing the loss function on the response side Find the gradient.

[0026] The present invention also provides a multi-agent collaborative large-scale language model camouflage attack system, comprising:

[0027] The strategic prompt generation agent is used to obtain a vocabulary candidate set based on the attack target semantics and the attacked language model through gradient optimization, and then generate the initial attack prompt with an additional perturbation word sequence based on the vocabulary candidate set;

[0028] A semantic disguise agent is used to generate semantic candidates for each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set, obtain a disguised perturbation sequence based on the semantic candidate set and the lexical candidate set, and obtain a disguised prompt based on the disguised perturbation sequence;

[0029] The attack evaluation feedback agent is used to input the disguised prompt into the attacked language model, verify the matching degree between the generated response and the target illegal response, and feed back the optimization signal to the strategy prompt generation agent based on the verification result.

[0030] Optionally, the strategy prompt generation agent includes a gradient calculation unit and an attack prompt generation unit;

[0031] The gradient calculation unit is used to construct a loss function based on the target attack semantics and the attacked language model, and calculate the gradient of each perturbation position in the initial attack prompt based on the loss function;

[0032] The attack prompt generation unit is used to determine the optimal perturbation direction for each perturbation position based on gradient information, and generate a vocabulary candidate set in combination with the top-k strategy, and then generate the initial attack prompt with the additional perturbation word sequence based on the vocabulary candidate set.

[0033] Optionally, the semantic disguise agent includes a semantic candidate generation unit and a disguise prompt construction unit;

[0034] The semantic candidate generation unit is used to generate semantic candidates for each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set;

[0035] The disguised prompt construction unit is used to obtain the intersection of the semantic candidate set and the lexical candidate set for each position in the perturbation word sequence to obtain a disguised word candidate set for the corresponding position. If the set is not empty, the disguised word that maximizes the probability of the attacked language model is selected to replace the original word at the corresponding position to obtain a disguised perturbation sequence. The disguised prompt is then concatenated with the attack target semantics to obtain a disguised prompt.

[0036] Compared with the prior art, the present invention has the following advantages and technical effects:

[0037] In the present invention, the strategic prompt generation agent effectively explores the disturbance space, the semantic camouflage agent improves the naturalness of the language and circumvents the detection mechanism, and the attack evaluation feedback agent provides closed-loop feedback for dynamically adjusting the prompt structure. Compared with the existing single-agent confrontation strategy, the present invention has stronger camouflage, higher migration ability and more stable attack effect. Through the collaborative linkage of multiple agents, a dynamic balance is achieved between the attack success rate and the naturalness of the language, which significantly improves the ability to circumvent security mechanisms and the adaptability to actual deployment. The present invention shows good attack adaptability on a variety of language models, the generated prompts have significantly low perplexity, and have the ability to perform cross-model migration, and have high practicality and research value. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0039] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0041] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] Example 1

[0043] like Figure 1 As shown, this embodiment provides a multi-agent collaborative large-scale language model camouflage attack method, including:

[0044] Based on the semantics of the attack target and the attacked language model, gradient optimization is combined to obtain a vocabulary candidate set, and the initial attack prompt with an additional perturbation word sequence is generated based on the vocabulary candidate set;

[0045] As a specific implementation method, the process of obtaining a vocabulary candidate set includes:

[0046] A loss function is constructed based on the target attack semantics and the attacked language model. The gradient of each perturbation position in the initial attack prompt is calculated based on the loss function. The optimal perturbation direction of each perturbation position is determined based on the gradient information, and a top-k strategy is combined to generate a vocabulary candidate set.

[0047] Specifically, the strategic prompt generation agent obtains an initial attack prompt with an additional perturbation word sequence.

[0048] The input of the strategy prompt generation agent is the attack target semantics and the attacked language model , the output is the prompt of additional perturbation ,in is the automatically optimized perturbation word sequence, Indicates character concatenation.

[0049] The optimization goal of the perturbation is to maximize the model generation of the specified illegal response The objective function is defined as:

[0050] ;

[0051] Transformed into minimizing the negative log-likelihood loss function:

[0052] ;

[0053] in, Indicates the target illegal output sequence word. Illegal total length of output sequence for the target. Indicates the target illegal output sequence arrive word.

[0054] The strategy prompts the agent to obtain each position by calculating the gradient The optimal perturbation direction is used, and the top-k strategy is combined to generate the vocabulary candidate set:

[0055] ;

[0056] The gradient is calculated as follows:

[0057] ;

[0058] Finally, several groups of prompt candidates are obtained based on the vocabulary candidate set , for further processing by the semantic camouflage agent.

[0059] Based on the auxiliary language model, a semantic candidate generation operation is performed on each position of the perturbation word sequence of the initial attack prompt to obtain a semantic candidate set; based on the semantic candidate set and the lexical candidate set, a disguised perturbation sequence is obtained, and based on the disguised perturbation sequence, a disguised prompt is obtained;

[0060] As a specific implementation, the process of obtaining the disguised prompt includes:

[0061] For each position in the perturbation word sequence, the intersection of the semantic candidate set and the lexical candidate set is taken to obtain the disguised word candidate set for the corresponding position. If it is not an empty set, the disguised word that maximizes the probability of the attacked language model is selected to replace the original word at the corresponding position to obtain a disguised perturbation sequence. The characters are then concatenated with the attack target semantics to obtain the disguised prompt.

[0062] Specifically, the semantic disguise agent obtains the disguise prompt.

[0063] The input of the semantic disguise agent is the initial attack prompt sequence output by the strategic agent , which outputs a disguised prompt with stronger semantic naturalness and lower confusion .

[0064] Semantic camouflage agent through auxiliary language model Perform semantic candidate generation. For the perturbation sequence Each position in , the model generates a set of semantic candidates:

[0065] ;in, Refers to the greedy decoding method.

[0066] The intersection of the strategic candidate set and the semantic candidate set forms a new disguised word candidate:

[0067] ;

[0068] like , select the word that makes the language model most likely Replace the original word; otherwise, keep the original word unchanged.

[0069] Finally, the disguised perturbation sequence is obtained:

[0070] ;

[0071] The construct disguise prompt is:

[0072] ;

[0073] The hint must satisfy two pretense constraints:

[0074] ;

[0075] ;

[0076] in represents the language perplexity function, is a detector function used to detect whether it is a malicious attack. Indicates that when the perplexity of the disguised prompt is less than or equal to the threshold When the text is in the form of a sentence, it is rich in semantic information and close to the naturalness of human language.

[0077] The disguised prompt is input into the attacked language model to verify the matching degree between the generated response and the target illegal response, and the optimization signal is fed back based on the verification result to optimize the vocabulary candidate set.

[0078] Specifically, the attack evaluation feedback agent verifies the disguise prompt Can the attacked language model be induced? Generate illegal response And the evaluation results are fed back to the strategy agent for the next round of iterative optimization.

[0079] The specific operation is: Input to model , get the response , and calculate its difference with the target response If the match is successful, the attack is considered successful and recorded as the attack success indicator:

[0080] ;

[0081] To close the cue optimization loop, the agent recalculates the sensitivity gradient for the perturbed portion of the successful cue sequence:

[0082] ;

[0083] ;

[0084] in, is the historical gradient weight hyperparameter, For historical gradient cache, Represents the response-side loss function. Variable representing the loss function on the response side Find the gradient. In response The sequence length.

[0085] Feedback It is input into the strategic intelligent agent, forming a closed-loop structure of prompt generation → disguise → execution → feedback, thereby continuously optimizing the attack effect and language naturalness of the prompt.

[0086] This embodiment provides a multi-agent collaborative large-scale language model camouflage attack system, including:

[0087] The strategic prompt generation agent is used to obtain a vocabulary candidate set based on the attack target semantics and the attacked language model through gradient optimization, and then generate the initial attack prompt with an additional perturbation word sequence based on the vocabulary candidate set;

[0088] A semantic disguise agent is used to generate semantic candidates for each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set, obtain a disguised perturbation sequence based on the semantic candidate set and the lexical candidate set, and obtain a disguised prompt based on the disguised perturbation sequence;

[0089] The attack evaluation feedback agent is used to input the disguised prompt into the attacked language model, verify the matching degree between the generated response and the target illegal response, and feed back the optimization signal to the strategy prompt generation agent based on the verification result.

[0090] Specifically, the strategy prompt generation agent includes a gradient calculation unit and an attack prompt generation unit;

[0091] The gradient calculation unit is used to construct a loss function based on the target attack semantics and the attacked language model, and calculate the gradient of each perturbation position in the initial attack prompt based on the loss function;

[0092] The attack prompt generation unit is used to determine the optimal perturbation direction for each perturbation position based on gradient information, and generate a vocabulary candidate set in combination with the top-k strategy, and then generate the initial attack prompt with the additional perturbation word sequence based on the vocabulary candidate set.

[0093] Specifically, the semantic camouflage agent includes a semantic candidate generation unit and a camouflage prompt construction unit;

[0094] The semantic candidate generation unit is used to generate semantic candidates for each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set;

[0095] The disguised prompt construction unit is used to obtain the intersection of the semantic candidate set and the lexical candidate set for each position in the perturbation word sequence to obtain the disguised word candidate set for the corresponding position. If the set is not empty, the disguised word that maximizes the probability of the attacked language model is selected to replace the original word at the corresponding position to obtain a disguised perturbation sequence. The character concatenation is then performed with the attack target semantics to obtain the disguised prompt.

[0096] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A multi-agent collaborative large-scale language model camouflage attack method, characterized by: The following steps are involved: Based on the semantics of the attack target and the attacked language model, gradient optimization is combined to obtain a vocabulary candidate set, and the initial attack prompt with an additional perturbation word sequence is generated based on the vocabulary candidate set; Based on the auxiliary language model, a semantic candidate generation operation is performed on each position of the perturbation word sequence of the initial attack prompt to obtain a semantic candidate set; based on the semantic candidate set and the lexical candidate set, a disguised perturbation sequence is obtained, and based on the disguised perturbation sequence, a disguised prompt is obtained; Inputting the disguised prompt into the attacked language model, verifying the matching degree between the generated response and the target illegal response, and feeding back an optimization signal based on the verification result to optimize the vocabulary candidate set; The process of obtaining the disguised prompt includes: For each position in the perturbation word sequence, the intersection of the semantic candidate set and the lexical candidate set is taken to obtain a disguised word candidate set for the corresponding position. If the set is not empty, the disguised word that maximizes the character probability of the attacked language model outputting a response is selected to replace the original word at the corresponding position to obtain a disguised perturbation sequence. The character sequence is then concatenated with the attack target semantics to obtain a disguised prompt. The disguised prompt satisfies two disguise constraints: ； ； in, Indicates a disguised prompt. represents the language perplexity function, is the detector function, which detects whether it is a malicious attack. Preset threshold for perplexity; The disguised prompt is input into the attacked language model to verify the matching degree between the generated response and the target illegal response. If the matching is successful, the sensitivity gradient of the disguised perturbation sequence in the disguised prompt is calculated as the optimization signal, wherein the sensitivity gradient The calculation is as follows: ； in, is the historical gradient weight hyperparameter, For historical gradient cache, represents the response-side loss function, Variable representing the loss function on the response side Find the gradient.

2. The multi-agent collaborative large-scale language model camouflage attack method according to claim 1 is characterized in that: The process of obtaining vocabulary candidate sets includes: A loss function is constructed based on the target attack semantics and the attacked language model. The gradient of each perturbation position in the initial attack prompt is calculated based on the loss function. The optimal perturbation direction of each perturbation position is determined based on the gradient information, and a top-k strategy is combined to generate a vocabulary candidate set.

3. The multi-agent collaborative large-scale language model camouflage attack method according to claim 2 is characterized in that: An objective function is constructed with the optimization goal of maximizing the probability of the model generating illegal responses as the disturbance. The objective function is transformed to obtain a minimized negative log-likelihood loss function, which is as follows: ； Where, is the attack target semantics, is the automatically optimized perturbation word sequence, Indicates the target illegal output sequence words, Indicates character concatenation, is the total length of the target illegal output sequence, Indicates the target illegal output sequence arrive word.

4. The multi-agent collaborative large-scale language model camouflage attack method according to claim 1 is characterized in that: The process of generating semantic candidates for each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model is as follows: ； in, is the semantic candidate set, The first positions, is an auxiliary language model, It is a greedy decoding method.

5. A large-scale language model camouflage attack system for multi-agent collaboration that implements the method described in any one of claims 1 to 4, characterized in that: include: The strategic prompt generation agent is used to obtain a vocabulary candidate set based on the attack target semantics and the attacked language model through gradient optimization, and then generate the initial attack prompt with an additional perturbation word sequence based on the vocabulary candidate set; A semantic disguise agent is used to generate semantic candidates for each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set, obtain a disguised perturbation sequence based on the semantic candidate set and the lexical candidate set, and obtain a disguised prompt based on the disguised perturbation sequence; The attack evaluation feedback agent is used to input the disguised prompt into the attacked language model, verify the matching degree between the generated response and the target illegal response, and feed back the optimization signal to the strategy prompt generation agent based on the verification result.

6. The multi-agent collaborative large-scale language model camouflage attack system according to claim 5 is characterized in that: The strategy prompt generation agent includes a gradient calculation unit and an attack prompt generation unit; The gradient calculation unit is used to construct a loss function based on the target attack semantics and the attacked language model, and calculate the gradient of each perturbation position in the initial attack prompt based on the loss function; The attack prompt generation unit is used to determine the optimal perturbation direction for each perturbation position based on gradient information, and generate a vocabulary candidate set in combination with the top-k strategy, and then generate the initial attack prompt with the additional perturbation word sequence based on the vocabulary candidate set.

7. The multi-agent collaborative large-scale language model camouflage attack system according to claim 5 is characterized in that: The semantic disguise agent includes a semantic candidate generation unit and a disguise prompt construction unit; The semantic candidate generation unit is used to generate semantic candidates for each position of the perturbation word sequence of the initial attack prompt based on the auxiliary language model to obtain a semantic candidate set; The disguised prompt construction unit is configured to obtain the intersection of a semantic candidate set and a lexical candidate set for each position in the perturbation word sequence to obtain a disguised word candidate set for the corresponding position. If the set is not empty, the disguised word that maximizes the probability of the character output of the attacked language model being a response is selected to replace the original word at the corresponding position to obtain a disguised perturbation sequence. The disguised prompt is then concatenated with the attack target semantics to obtain a disguised prompt.

Images