Implicit gradient optimization-based large language model jailbreak attack resisting method

Abstract

The invention discloses an implicit gradient optimization-based large language model prison break attack resisting method, which is characterized in that continuous gradient optimization on resistance tokens is realized through a Gumbel-Softmax technology, calculation cost is reduced in combination with a two-stage proxy model screening mechanism, and semantic concealment is kept by adopting a dynamic regularization strategy. The system comprises a gradient optimization module, an agent screening module and a migration enhancement module, and can effectively improve the attack success rate and the cross-model migration capability of resistance prompts. The technical problems that a traditional attack resisting method is low in efficiency and poor in concealment are solved, the attack cost is reduced while the large model attack success rate is increased, the API calling frequency is effectively reduced, and the method is suitable for the field of large language model security testing.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and natural language processing, and in particular to a method for defending a large language model against jailbreak attacks based on implicit gradient optimization, and more particularly to the security assessment of a large language model (LLM). Background Art

[0002] With the widespread application of large language models (LLMs) in natural language processing, their security and robustness have become a research focus. In practical applications, attackers can launch adversarial attacks against these large language models, inducing them to output erroneous or malicious results, posing a threat to their security and credibility. Therefore, studying adversarial attacks against large language models and how to enhance their protection capabilities is a key topic in the current AI security field.

[0003] Existing adversarial attack methods for large language models typically rely on generating adversarial examples to deceive the model. Common attack methods include adversarial training and gradient descent attacks. Early adversarial attack methods, such as FGSM (Fast Gradient Sign Method) and PGD (Projected Gradient Descent), directly modify input data to generate adversarial perturbations, thereby inducing the model to produce erroneous outputs. However, most of these methods rely on knowledge of the target model and have limited effectiveness in black-box scenarios. To address this issue, adversarial attack strategies based on proxy models have been proposed in recent years. For example, the GCG (Gradient-based Candidate Generation) method improves the effectiveness of generating adversarial suffixes by combining adversarial cue generation with gradient optimization. However, traditional adversarial cue generation methods still suffer from low efficiency, large search scope, and a high number of forward calls to large models, which hinder the effectiveness and efficiency of the attack.

[0004] While various methods have been proposed to enhance model robustness against adversarial attacks, existing research is largely limited to improving the security of a single model and exhibits limitations in black-box attacks and cross-model transferability. Implementing efficient and highly transferable attacks using low-cost computation remains a key challenge in the field of adversarial attacks against large language models. Summary of the Invention

[0005] The purpose of the present invention is to address the deficiencies of the existing technology and provide a method for resisting jailbreak attacks on large language models based on implicit gradient optimization.

[0006] The purpose of the application is achieved by the following technical scheme: a large language model anti-jailbreak attack method based on implicit gradient optimization, characterized by comprising the following steps:

[0007] Step one: input the original malicious instruction X and the preset anti-suffix length N, set the initial anti-suffix embedding vector for the attack target, and construct the prompt template P;

[0008] Step two: calculate the cross-entropy loss J of the anti-suffix S of the prompt template P under the target large language model, obtain the embedding gradient vector of each token position through back propagation, introduce independent Gumbel noise to each position, and scale through the temperature parameter to form a group of continuous candidate token probability distribution, and generate candidate anti-suffix sequences;

[0009] Step three: input all candidate anti-suffix sequences into the lightweight proxy model for rapid loss prediction, and retain the Top-K candidates with the largest loss reduction amplitude; if the proxy loss of the candidate anti-suffix sequence decreases by more than the first preset threshold, use the first large language model to perform secondary evaluation on the screened candidate anti-suffix, determine the optimal candidate anti-suffix, otherwise return to step two and resample;

[0010] Step four: repeat steps two and three to obtain several optimal candidate anti-suffixes;

[0011] Step five: design a multi-objective energy function and introduce an early stopping mechanism, if the optimal candidate anti-suffix found in the last K rounds of iteration corresponds to an energy function E value lower than the second preset threshold, terminate the optimization of the current token position and move to the next token position; when all token positions of the anti-suffix are optimized, the anti-suffix with the lowest energy E among all the anti-suffixes generated during the optimization process is the final output.

[0012] Compared with the prior art, the present application has the beneficial effects at least including: the present application adopts Gumbel-Softmax technology, and performs continuous optimization on the token selection process through implicit gradient estimation, instead of the traditional random search method. This improvement greatly reduces the number of invalid candidate evaluations, and improves the efficiency of the adversarial attack generation; by introducing a two-stage proxy model screening mechanism, the present application can effectively reduce the number of forward calls to the large model. First, a lightweight proxy model (TinyBERT) is used for rapid screening, and then a low-precision target model (LLaMA-7B FP16) is used for fine evaluation, thereby reducing the consumption of computing resources while ensuring attack accuracy; the semantic consistency of tokens is controlled through a dynamic regularization strategy, and an energy function early stopping mechanism is introduced to avoid the problem of semantic deviation in the later optimization process, thereby enhancing the concealment of the generated prompt and making it more difficult to be detected and identified. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is an implicit gradient optimization-based large language model adversarial jailbreak attack method flowchart.

[0014] Figure 2 is a hardware structure schematic diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0015] The present application will be described in detail below with reference to the accompanying drawings. In the case of no conflict, the features in the following embodiments and implementation manners can be combined with each other.

[0016] The present application is an implicit gradient optimization-based large language model adversarial jailbreak attack method, comprising the following steps:

[0017] Step one: input the original malicious instruction X and the preset adversarial suffix length N, set the initial adversarial suffix embedding vector for the attack target, and construct the prompt template P;

[0018] Step two: calculate the cross-entropy loss J of the adversarial suffix S of the prompt template P under the target model, and obtain the embedding gradient vector of each token position through back propagation. Independent Gumbel noise is introduced for each position, and is scaled through a temperature parameter to form a group of continuous candidate token probability distributions.

[0019] For example, the current adversarial suffix S = [ <sep>", tell", me", how"] with the target prefix 'Sure, here's'. The cross-entropy loss J = 1.26 is calculated for this combination. By backpropagation, the embedding gradient at 'tell' is obtained as Gumbel noise g is added to the original logits again i ~ -log(-logU i ) (g i = [1.21, 0.87, …], U i ~ Uniform(0, 1)) and is temperature scaled τ = 1.0 to obtain a continuous distribution p i for the next step of candidate token selection.

[0020] This step is divided into the following sub-steps:

[0021] (2.1) Obtain the prompt sequence P composed of the original instruction and the combined adversarial suffix, input it into the target large language model, and calculate the cross-entropy loss J of the generated target prefix (e.g. "Sure, here's"); by backpropagation, obtain the embedding vector e i of each token position in the adversarial suffix S

[0022] (2.2) Add independent Gumbel noise g i to the original logits of each token (where g i ~ -log(-logU), U is uniformly distributed), introduce a temperature parameter τ to scale the noisy logits; use an exponential temperature annealing strategy for control, with the temperature τ t = τ max · e -kt (where τ max is the initial highest temperature and k is the decay coefficient), the high temperature stage (τ t → 5.0) promotes the diversity exploration of candidate tokens, and the low temperature stage (τ t → 0.1) strengthens the convergence to the optimal solution; generate a continuous probability distribution to support end-to-end gradient backpropagation.

[0023] Step three: input all candidate sequences into the lightweight proxy model (TinyBERT) for fast loss prediction, and keep the Top-K candidates with the largest loss drop. If the proxy loss drop of the candidate sequence is greater than the preset threshold ε = 0.2, use the low-precision (FP16) LLaMA-2-7B model copy to perform secondary evaluation on the coarsely screened candidate adversarial suffix, further improve the attack success rate, otherwise return to step two for resampling.

[0024] This step is divided into the following sub-steps:

[0025] (3.1) According to the gradient direction and the probability distribution generated in step (2.2) Joint judgment, prefer to select the candidate token consistent with the negative gradient direction and the probability value greater than the Top-k sampling threshold (determined by the gradient direction and the probability distribution generated in step (2.2) in (2.1)), select the token with the largest loss reduction speed at each token position as one of the potential replacement options to form the optimal candidate adversarial suffix, in order to improve the global optimization rate;

[0026] (3.2) Input the Top-K candidate adversarial suffix sequences obtained by preliminary screening in step (3.1) one by one into the lightweight proxy model TinyBERT for preliminary screening, and quickly filter 90% of low-quality candidates. For example, 20 candidate adversarial suffixes are generated in one optimization, and the TinyBERT proxy model sequentially predicts the loss of the suffixes to obtain a loss sequence. Select the top 10% (Top-2) candidates with the largest loss reduction amplitude, assuming sequences A and B. Assuming that A has a decrease of 0.23 (ΔJ proxy = 0.23 > ε = 0.2) on the basis of the original loss, then A enters the next stage; otherwise, return to resampling.

[0027] (3.3) Use the LLaMA-2-7B model copy with low precision (FP16) to perform secondary evaluation on the candidate adversarial suffixes screened by TinyBERT to ensure the attack success rate.

[0028] For example, sequences A and B of candidate adversarial suffixes have been screened in step (3.2). To perform secondary evaluation, first input the complete prompt composed of the original instruction and candidate suffix A into the LLaMA-2-7B FP16 copy, and calculate its real cross-entropy loss for the target prefix "Sure, here's". Assuming that the calculated loss value J A = 0.89. Then, process candidate suffix B in the same way, and calculate its real loss as J B = 0.95. By comparison, J A < J B , which indicates that candidate sequence A can more effectively induce the generation of the target prefix on the target large model. Therefore, sequence A is finally confirmed as the optimal candidate adversarial suffix in this round, and the replacement token corresponding to it (i.e. the optimal token) will be adopted, thereby ensuring that each iteration is optimized in the direction of the highest actual attack success rate.

[0029] (3.4) Insert the optimal token obtained in step (3.3) into the current adversarial suffix, update the prompt template;

[0030] Further, in each round of optimization, the cosine similarity Sim(v new , v old ) between the new token embedding v new generated in the n+1th round and the old token embedding v old in the nth round is calculated as a regularization term R, which is added to the total loss L total = J + λ·R (where J is the cross-entropy loss of the current adversarial prompt, R is the average cosine similarity difference regularization term between the new and old token embeddings in the current round, and λ is the regularization strength coefficient) to balance early exploration and late semantic convergence.

[0031] Step four: repeat steps two and three to obtain several optimal candidate adversarial suffixes;

[0032] Specifically, based on the updated prompt template P, steps two and three are repeated to obtain another optimal candidate adversarial suffix, and the process is repeated several times to obtain several optimal candidate adversarial suffixes.

[0033] Step five: concatenate the original instruction and the optimal candidate adversarial suffix and input them into the target large language model to generate a response text. In order to jointly evaluate the attack effectiveness, language fluency and semantic concealment of the optimal candidate adversarial suffix, a multi-objective energy function E = α·MaliciousScore + β·Fluency - γ·Similarity (where MaliciousScore is the malicious content score, used to evaluate the effectiveness of the attack; Fluency is the fluency score, used to evaluate the naturalness of the generated text; Similarity is the similarity score, used to measure the semantic similarity between the adversarial suffix and the original or benign text; α, β, γ are the weight coefficients of each term) is designed, and several local optimal candidate adversarial suffixes are further optimized. At the same time, an early stopping mechanism is introduced. If the optimal candidate adversarial suffix found in the last K rounds of iteration corresponds to an energy function E value lower than the preset threshold δ (δ is the empirical median of the energy function E corresponding to the success of adversarial sample generation for the target model minus the standard deviation, i.e. δ = Median(E success ), it is considered that the optimization convergence is reached, the current token position optimization is terminated, and the next token position is turned to; otherwise, the token that minimizes the energy function E is selected from the candidate tokens, the token is replaced with the current token position, and the replaced adversarial suffix is used for the next round of iteration optimization, and the above process is repeated until the early stopping condition is met. When all the token positions of the adversarial suffix are optimized, the adversarial suffix with the lowest energy E in the several adversarial suffixes generated in the whole optimization process is the final output.

[0034] Corresponding to the foregoing embodiment of the large language model adversarial jailbreak attack method based on implicit gradient optimization, the present application also provides an embodiment of a large language model adversarial jailbreak attack device based on implicit gradient optimization.

[0035] Referring to Figure 2 , the embodiment of the present application provides a large language model adversarial jailbreak attack device based on implicit gradient optimization, which comprises one or more processors for implementing the large language model adversarial jailbreak attack method based on implicit gradient optimization in the above embodiment.

[0036] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0037] The embodiment of the large language model adversarial jailbreak attack device based on implicit gradient optimization of the present application can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The device embodiment can be realized by software, or by hardware or a combination of software and hardware. Taking software realization as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for running by the processor of the device with data processing capability where it is located. From the hardware level, as shown in Figure 2 , it is a hardware structure diagram of the device with data processing capability where the large language model adversarial jailbreak attack device based on implicit gradient optimization of the present application is located. In addition to the processor, Figure 2 In addition to the processor, the memory, the network interface, and the nonvolatile memory, any data processing capable device in which the apparatus of the embodiments is implemented can also include other hardware according to the actual functions of the data processing capable device, which will not be described herein.

[0038] The implementation processes of the functions and roles of the units in the apparatus are specifically described in the implementation processes of the corresponding steps in the above method, which will not be described herein.

[0039] For the apparatus embodiments, since they basically correspond to the method embodiments, the related parts can be referred to the part of the method embodiments. The apparatus embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all of the modules can be selected to achieve the purpose of the present application according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0040] The embodiments of the present application also provide a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the method for resisting jailbreaking attack of a large language model based on implicit gradient optimization.

[0041] The computer readable storage medium can be an internal storage unit of any data processing capable device, such as a hard disk or a memory. The computer readable storage medium can also be any data processing capable device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both an internal storage unit of any data processing capable device and an external storage device. The computer readable storage medium is used to store the computer program and other programs and data required by the data processing capable device, and can also be used to temporarily store data that has been output or will be output.

[0042] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0043] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0044] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0045] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0046] The above examples are merely intended to illustrate the design ideas and characteristics of the present application, and to enable those skilled in the art to understand and implement the present application, and the protection scope of the present application is not limited to the above examples. Therefore, any equivalent changes or modifications made according to the disclosed principles and design ideas of the present application are within the protection scope of the present application.< / sep>

Claims

1. A method for combating jailbreak attacks on large language models based on implicit gradient optimization, characterized in that: The following steps are involved: Step 1: Input the original malicious instruction X and the preset adversarial suffix length N, set the initial adversarial suffix embedding vector for the attack target, and construct the prompt template P; Step 2: Calculate the cross entropy loss J of the adversarial suffix S of the hint template P under the target large language model, and obtain the embedding gradient vector of each token position through backpropagation. Introduce independent Gumbel noise to each position and scale it by the temperature parameter to form a set of continuous candidate token probability distributions, thereby generating a candidate adversarial suffix sequence. Step 3: Input all candidate adversarial suffix sequences into the lightweight proxy model for fast loss prediction, and retain the ones with the largest loss reduction as Top-K candidates; If the proxy loss of the candidate adversarial suffix sequence decreases by more than the first preset threshold, the first large language model is used to perform a secondary evaluation on the coarsely screened candidate adversarial suffixes to determine the optimal candidate adversarial suffix and update the hint template P. Otherwise, return to step 2 for resampling. Step 4: Repeat steps 2 and 3 to obtain several optimal candidate adversarial suffixes; Step 5: Design a multi-objective energy function and introduce an early stopping mechanism. If the value of the energy function E corresponding to the optimal candidate adversarial suffix found after K consecutive iterations is lower than the second preset threshold, the optimization of the current token position is terminated and the next token position is turned to. When all token positions of the adversarial suffix are optimized, the adversarial suffix with the lowest energy E among the adversarial suffixes generated during the entire optimization process is the final output.

2. The method for resisting jailbreak attacks according to claim 1, wherein: The second step includes the following sub-steps: (2.1) Obtain the prompt template P composed of the original instruction and the adversarial suffix, input it into the target large language model, and calculate the cross entropy loss J of its generated target prefix; obtain the embedding vector e of each token position in the adversarial suffix S through backpropagation i The corresponding gradient value (2.2) Add independent Gumbel noise g to the original logits of each token i , the temperature parameter τ is introduced to scale the noisy logits, and an exponential temperature annealing strategy is used to control it to generate a continuous probability distribution.

3. The method for resisting jailbreak attacks according to claim 2, wherein: The step three includes the following sub-steps: (3.1) Based on the gradient direction and the probability distribution generated in step (2.2), select candidate tokens that are consistent with the negative gradient direction and have a probability value greater than the Top-k sampling threshold. At each token position, select the token with the fastest current loss decrease rate as one of the potential replacement options for the optimal candidate adversarial suffix to improve the global optimization rate. (3.2) The top-K candidate adversarial suffix sequences obtained through the preliminary screening in step (3.1) are input one by one into the lightweight proxy model for preliminary screening; (3.3) Use the first language model to perform a secondary evaluation on the candidate adversarial suffixes screened by TinyBERT to determine the optimal token; (3.4) Insert the optimal token obtained in step (3.3) into the current adversarial suffix and update the prompt template.

4. The method for resisting jailbreak attacks according to claim 3, wherein: Conduct a secondary assessment, including: The complete prompt consisting of the original instruction and the candidate adversarial suffix is ​​input into the first language model, and its true cross entropy loss for the target prefix is ​​calculated. The candidate adversarial suffix corresponding to the minimum loss value is the optimal candidate adversarial suffix.

5. The method for resisting jailbreak attacks according to claim 1 or 3, characterized in that: The lightweight proxy model is the TinyBERT model.

6. The method for resisting jailbreak attacks according to claim 1, 3 or 4, characterized in that: The largest language model is the LLaMA-2-7B model.

7. The method for resisting jailbreak attacks according to claim 1, wherein: The target energy function is: E=α·MaliciousScore+β·Fluency-γ·Similarity Among them, MaliciousScore is the malicious content score, which is used to evaluate the effectiveness of the attack; Fluency is the fluency score, which is used to evaluate the naturalness of the generated text; Similarity is the similarity score, which is used to measure the semantic similarity between the adversarial suffix and the original or benign text; α, β, and γ are the weight coefficients of each item respectively.

8. The method for resisting jailbreak attacks according to claim 1, wherein: The second preset threshold δ is the empirical median minus the standard deviation of the energy function E corresponding to the successful generation of the adversarial sample under the target large language model.

9. A device for combating jailbreak attacks on a large language model based on implicit gradient optimization, characterized in that: The method comprises one or more processors for implementing a large language model anti-jailbreak attack method based on implicit gradient optimization according to any one of claims 1 to 8.

10. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, it is used to implement a large language model anti-jailbreak attack method based on implicit gradient optimization according to any one of claims 1 to 8.

Images