Attention-guided white-box attack method and device for large language model

Abstract

The invention provides an attention-guided white-box attack method and device for a large-scale language model, and the method comprises the steps: inputting the current input of a user and an antagonistic suffix of the current input into the large-scale language model, and obtaining the attention score of lexical elements at each position in the antagonistic suffix; according to the attention score of the lexical element at each position, positioning a to-be-replaced position in the antagonistic suffix; lexical elements at all positions in the adversarial suffixes are replaced, and candidate lexical element replacement sets at all the positions are determined according to gradient information of the lexical elements at all the positions after replacement; selecting candidate lexical elements from the candidate lexical element replacement set at the position to be replaced to replace the candidate lexical elements to the position to be replaced, and obtaining an optimized antagonistic suffix; and inputting the current input and the optimized antagonistic suffix into the large-scale language model to obtain an answer output by the large-scale language model. According to the invention, the efficiency, precision and success rate of jailbreak attacks are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information security, and in particular to an attention-guided white-box attack method and device for large language models. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have become a key technology in the field of natural language processing (NLP) and are widely used in intelligent question answering, text generation, and writing assistance, among other scenarios. Large language models can generate high-quality, natural, and fluent text outputs in various tasks. To ensure the compliance and safety of the output content, researchers have introduced a security alignment mechanism during training and fine-tuning. The goal of security alignment is to make the content generated by the model more consistent with human social values, avoiding harmful, illegal, or inappropriate content.

[0003] However, the security alignment mechanism is not foolproof. With the widespread deployment and application of large language models, attackers have begun to try to bypass security protections and use jailbreak attacks to force the model to output prohibited or malicious content. Jailbreak attacks often use carefully designed prompt words to deceive the model into generating prohibited or sensitive information. This type of attack was initially represented by "DAN" prompt words written by humans, and later evolved into optimization-based jailbreak methods such as the Greedy Coordinate Gradient (GCG) method and improved algorithms such as AutoDAN and PAL (Program-Aided Language Models). Some methods also use black-box large language models to generate inducible prompt words to achieve the attack goal. The continuous emergence of these methods poses a serious challenge to the security of large language models.

[0004] Although existing jailbreak attack methods can bypass the security alignment mechanism of large language models to some extent, they generally have the following shortcomings and problems:

[0005] (1) Lack of principled insight. Existing large model-based jailbreak methods typically rely on large language models to generate prompt words. Since large language models are complex black-box models, attackers have difficulty understanding the true triggering mechanism of prompt words. This makes it difficult for these methods to fundamentally explain and analyze the reasons for the success of jailbreak attacks, limiting researchers' understanding of the attack principles and improvements to countermeasures.

[0006] (2) The suffix position to be updated lacks guidance. Based on the optimized jailbreaking method (such as GCG, PAL, etc.), when generating an adversarial prompt word, it is usually randomly or uniformly replaced at different positions of the prompt word. This lack of strategic replacement cannot fully play the guiding role of gradient information, resulting in low algorithm efficiency. A large number of invalid replacement attempts not only increase the attack calculation cost, but also may affect the final success rate of the attack.

[0007] (3) The attack success rate is limited. Due to the lack of systematic understanding and optimization of successful jailbreaking prompt words, existing technologies often fail to achieve high success rates in different models and different scenarios. The blindness in the prompt word generation process makes the attack results unstable, making it difficult to adapt to more diverse model architectures or security strategies.

[0008] (4) Security and controllability issues. The prompt word generation of existing methods depends on the output of the model itself, lacks explainability and controllability. This not only affects the attack efficiency, but also brings greater security risks, making it difficult for attackers to effectively evaluate the security and controllability of the generated prompt words. SUMMARY

[0009] The present application provides an attention-guided white-box attack method and device for large language models to solve the multiple problems of traditional optimization attacks in the prior art, such as lack of guidance in position selection, easy to fall into local optimum, low initial perturbation quality, etc., which severely limit the attack effect, and to improve the efficiency, accuracy and success rate of jailbreaking attacks.

[0010] The present application provides an attention-guided white-box attack method for large language models, comprising:

[0011] inputting the current input of the user and the adversarial suffix of the current input into a large language model to obtain the attention scores of the tokens at each position in the adversarial suffix;

[0012] locating the replacement position in the adversarial suffix according to the attention scores of the tokens at each position;

[0013] replacing the tokens at each position in the adversarial suffix, and determining the candidate token replacement set at each position according to the gradient information of the replaced tokens at each position;

[0014] selecting a candidate token replacement from the candidate token replacement set at the replacement position and replacing it to the replacement position to obtain an optimized adversarial suffix;

[0015] inputting the current input and the optimized adversarial suffix into the large language model to obtain the answer output by the large language model.

[0016] According to the present invention, an attention-guided white-box attack method for large language models is provided. The user's current input and its adversarial suffix are input into the large language model using the following formula to obtain the attention scores of lexical units at each position in the adversarial suffix:

[0017]

[0018] in, It is the first of the aforementioned adversarial suffixes Attention scores for each word at each position. The number of layers in the large language model. The number of attention heads in the large language model. The number of tokens in the current input. The first of the adversarial suffixes The lexical term at the i-th position is for the i-th The first in the layer The first attention head is about the current input in the first... Attention score for each word.

[0019] According to the present invention, an attention-guided white-box attack method for large language models locates the position to be replaced in the adversarial suffix based on the attention scores of the lexical units at each position, including:

[0020] The attention scores of the lexical units at each position in the adversarial suffix are sorted in ascending order to obtain the ranking of each position in the adversarial suffix.

[0021] Based on the ranking corresponding to each position in the adversarial suffix, determine the sampling probability distribution of each position in the adversarial suffix;

[0022] Based on the sampling probability distribution of each position in the adversarial suffix, the position to be replaced in the adversarial suffix is ​​determined.

[0023] According to the present invention, an attention-guided white-box attack method for large language models determines the sampling probability distribution of each position in the adversarial suffix based on the ranking corresponding to each position, including:

[0024] Based on the ranking corresponding to each position in the adversarial suffix, a smooth decay function is used to determine the sampling probability distribution of each position in the adversarial suffix, as shown in the following formula:

[0025]

[0026] in, It is the first of the aforementioned adversarial suffixes The sampling probability distribution at each location, is the ranking corresponding to the i-th position in the adversarial suffix, is a hyperparameter used to control the degree of suppression of the ranking on the sampling probability distribution.

[0027] According to the attention-guided white-box attack method for a large language model provided by the present application, after inputting the current input and the optimized adversarial suffix into the large language model and obtaining the answer output by the large language model, the method further comprises:

[0028] determining the difference between the losses output by the large language model before and after optimization of the adversarial suffix;

[0029] If the difference between the losses is less than a preset threshold, then according to the rankings corresponding to each position in the adversarial suffix, an exponential decay function is used to determine the sampling probability distribution of each position in the adversarial suffix, and the formula is as follows:

[0030] .

[0031] According to the attention-guided white-box attack method for a large language model provided by the present application, the candidate token replacement set at each position is determined according to the gradient information of the replaced token at each position, and the method comprises:

[0032] inputting the current input and the adversarial suffix after replacing the token at each position into the large language model and calculating the loss output by the large language model;

[0033] performing backpropagation on the loss output by the large language model to determine the gradient of the replaced token at each position;

[0034] calculating the difference between the gradient of the replaced token at each position and the loss output by the large language model;

[0035] selecting a preset number of replaced tokens at each position with the largest difference to construct the candidate token replacement set at each position.

[0036] According to the attention-guided white-box attack method for a large language model provided by the present application, a candidate token replacement is selected from the candidate token replacement set at the position to be replaced to obtain an optimized adversarial suffix, and the method comprises:

[0037] selecting a candidate token replacement from the candidate token replacement set at each position to be replaced to obtain an adversarial suffix sample;

[0038] inputting the current input and the adversarial suffix sample into the large language model and calculating the loss output by the large language model; ​

[0039] The loss-minimized adversarial suffix sample is used as the optimized adversarial suffix.

[0040] The application further provides an attention-guided white-box attack device for a large language model, comprising:

[0041] An extraction module is configured to input a current input of a user and an adversarial suffix of the current input into a large language model to obtain attention scores of word units at positions in the adversarial suffix;

[0042] A positioning module is configured to position a replacement position in the adversarial suffix according to the attention scores of the word units at the positions;

[0043] A determination module is configured to replace the word units at the positions in the adversarial suffix, and determine a candidate word unit replacement set at each position according to gradient information of the word units at the positions after replacement;

[0044] An optimization module is configured to select a candidate word unit replacement from the candidate word unit replacement set at the replacement position and replace the replacement position with the candidate word unit replacement to obtain an optimized adversarial suffix;

[0045] An output module is configured to input the current input and the optimized adversarial suffix into the large language model to obtain an answer output by the large language model.

[0046] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the attention-guided white-box attack method for a large language model as described above.

[0047] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the attention-guided white-box attack method for a large language model as described above.

[0048] The application further provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the attention-guided white-box attack method for a large language model as described above.

[0049] The attention-guided white-box attack method and apparatus for large language models provided by this invention introduces an attention mechanism to guide position selection, avoiding efficiency losses caused by randomness and effectively improving the accuracy of perturbation and the overall attack success rate. It employs a gradient-direction-based candidate lexical replacement strategy, making the attack process more targeted and interpretable, and enhancing the semantic and structural destructive power of the perturbation. The jump mechanism effectively expands the search space, enabling the algorithm to actively escape when trapped in local optima, significantly improving the stability and robustness of the algorithm. The initial suffix uses a semantically biased construction method, compensating for the shortcomings of traditional invalid initialization, making the optimization process start at a better level, thereby accelerating the attack convergence speed and improving the final success rate. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 This is one of the flowcharts of the attention-guided white-box attack method for large language models provided by the present invention;

[0052] Figure 2 This is the second flowchart of the attention-guided white-box attack method for large language models provided by the present invention;

[0053] Figure 3 This is a schematic diagram of the overall structure of the attention-guided white-box attack algorithm in the attention-guided white-box attack method for large language models provided by the present invention.

[0054] Figure 4 This is a comparative diagram of hyperparameter selection in the attention-guided white-box attack method for large language models provided by the present invention;

[0055] Figure 5 This is a schematic diagram of the attention-guided white-box attack device for large language models provided by the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0057] The present application proposes an attention-guided attack method (AAL) for large language models, aiming to improve the efficiency, accuracy and success rate of current jailbreaking attacks. To solve these problems, the present application designs a complete technical solution integrating attention mechanism, gradient information, jump mechanism and high-quality initialization, which has achieved significant performance improvement on multiple mainstream language models, especially achieving an attack success rate of up to 94% on Llama-2-7b-chat-hf. The core technical solution of the present application mainly includes the following four aspects:

[0058] 1. Using attention information to guide prompt word position selection

[0059] The present application extracts the attention weight information of each token in the model by forward propagation of the prompt word input of the large model. According to the attention weight distribution, the position with higher "attention" in the current input prompt word is determined, i.e. the most sensitive token position of the model. In this way, the present application fundamentally solves the problem of lack of guidance in prompt word replacement position in the prior art, avoiding the low-efficiency search caused by random or uniform replacement, and ensuring that the replacement operation is more targeted and interpretable.

[0060] 2. Combining gradient information for candidate token replacement and optimization

[0061] After determining the replacement position based on attention weight, the present application further introduces gradient information to calculate the gradient direction and size of the corresponding position. Based on the gradient information, a candidate token set is generated, and replacement tokens that are consistent with the gradient direction and can effectively change the model output are preferentially selected. After replacement, a new prompt word suffix is formed, and through multiple rounds of iteration optimization, the attack effect is continuously improved. This strategy solves the problem of lack of principled insight and inability to effectively improve the success rate of attacks in the prior art, making the prompt word replacement process interpretable and controllable, and the attack effect more stable.

[0062] 3. Jump mechanism to expand search space and avoid local optimum

[0063] To further improve the success rate of jailbreaking attacks, the present application designs a jump mechanism to expand the search space in the optimization process. Specifically, when the model appears to be in an optimization stall and trapped in a local optimum, a large random disturbance is introduced to jump the prompt word suffix. Through periodic jumping, the prompt word optimization is prevented from falling into a local optimal solution, thereby improving the diversity of prompt word generation and the success rate of attacks. This technical solution significantly improves the adaptability and attack effect on multiple large models, addressing the problem of limited attack success rate in the prior art.

[0064] 4. Initialize prompt suffix and dynamic update strategy

[0065] Unlike the traditional method of initializing the prompt suffix with "20 exclamation marks", the present application proposes, for the first time, to replace the "20 exclamation mark" initialization with a simple jailbreaking prompt suffix. The initial suffix has a certain attack potential, and can be adjusted flexibly in subsequent iterations. The entire prompt suffix is dynamically updated in each iteration, gradually enhancing the attack effect and avoiding the performance bottleneck caused by redundant or invalid initial prompts.

[0066] The following will be described in conjunction with Figure 1 A white-box attack method for large language models is described, comprising:

[0067] Step 101, input the current input of the user and the adversarial suffix of the current input into the large language model, and obtain the attention score of each position word in the adversarial suffix;

[0068] Step 102, according to the attention score of each position word, locate the replacement position in the adversarial suffix;

[0069] Step 103, replace the word at each position in the adversarial suffix, and determine the candidate word replacement set at each position according to the gradient information of the replaced word at each position;

[0070] Step 104, select a candidate word replacement from the candidate word replacement set at the replacement position and replace it at the replacement position, to obtain an optimized adversarial suffix;

[0071] Step 105, input the current input and the optimized adversarial suffix into the large language model, and obtain the answer output by the large language model.

[0072] The present embodiment provides an optimization method for adversarial attacks on large language models (LLMs). The overall flowchart is shown in Figure 2 The method uses the attention structure and gradient information of the model itself to construct an adversarial suffix attached to the input question through iterative optimization, so that the model outputs a specified violation response while maintaining the original semantic context. The entire method process is designed in stages, combining suffix initialization, attention-guided perturbation selection, candidate replacement sampling, batch generation and screening mechanism, and introducing a jump strategy when optimization stagnates to improve attack stability and success rate.

[0073] To avoid low optimization efficiency due to the lack of semantic signals in the initial suffix, this invention generates an initial suffix with a certain degree of deception. This suffix can significantly increase the model's attention weight to the suffix position, thereby improving the attack effect of subsequent replacements.

[0074] The initial suffix adopts a semantically biased construction method, which makes up for the shortcomings of traditional invalid initialization, making the starting point of the optimization process better, thereby accelerating the attack convergence speed and improving the final success rate.

[0075] This embodiment introduces an attention mechanism to guide position selection, avoiding efficiency losses caused by randomness and effectively improving the accuracy of perturbation and the overall attack success rate. It adopts a gradient-direction-based candidate lexical replacement strategy, making the attack process more targeted and interpretable, and enhancing the perturbation's semantic and structural destructive capabilities.

[0076] Based on the above embodiments, this embodiment uses the following formula to input the user's current input and the adversarial suffix of the current input into a large language model to obtain the attention scores of the lexical units at each position in the adversarial suffix:

[0077]

[0078] in, It is the first of the aforementioned adversarial suffixes Attention score for each word at each position. The number of layers in the large language model. The number of attention heads in the large language model. The number of tokens in the current input. The first of the adversarial suffixes The lexical term at the i-th position is for the i-th The first in the layer The first attention head is about the current input in the first... Attention score for each word.

[0079] Obtain the attention tensors of each layer of the model , shape is These represent the number of layers, the number of attention heads, and the number of query terms, respectively. This embodiment extracts the comprehensive attention score for each term by averaging all dimensions, as shown in the formula above.

[0080] The attention distribution described above reveals the model's attention focus structure under the current input, which helps in prioritizing subsequent replacement positions. Compared to traditional perturbation strategies that rely solely on gradients, the attention mechanism provides a more robust way to estimate the importance of target lemmas.

[0081] On the basis of the above-mentioned embodiments, in this embodiment, according to the attention score of the word element at each position, the position to be replaced in the adversarial suffix is located, including:

[0082] The attention score of the word element at each position in the adversarial suffix is sorted in ascending order to obtain the ranking corresponding to each position in the adversarial suffix.

[0083] According to the ranking corresponding to each position in the adversarial suffix, the sampling probability distribution of each position in the adversarial suffix is determined.

[0084] According to the sampling probability distribution of each position in the adversarial suffix, the position to be replaced in the adversarial suffix is determined.

[0085] In order to convert the attention score into a position replacement probability, first of all, is sorted in ascending order to obtain the corresponding ranking , and then a preset decay function is used to generate the sampling probability distribution of the perturbed position .

[0086] On the basis of the above-mentioned embodiments, in this embodiment, according to the ranking corresponding to each position in the adversarial suffix, the sampling probability distribution of each position in the adversarial suffix is determined, including:

[0087] According to the ranking corresponding to each position in the adversarial suffix, the sampling probability distribution of each position in the adversarial suffix is determined by default using a smoothing decay function, and the formula is as follows:

[0088]

[0089] wherein, is the sampling probability distribution of the th position in the adversarial suffix, is the ranking corresponding to the th position in the adversarial suffix, is a hyperparameter for controlling the degree of suppression of the ranking on the sampling probability distribution. This function ensures that the word element with a high ranking value (i.e. low attention) has a higher replacement probability, thereby attempting to guide the model attention shift.

[0090] On the basis of the above-mentioned embodiments, in this embodiment, after inputting the current input and the optimized adversarial suffix into the large language model to obtain the answer output by the large language model, the method further includes:

[0091] Determining the difference between the losses output by the large language model before and after optimization of the adversarial suffix.

[0092] If the difference of the losses is less than a preset threshold, the sampling probability distribution of each position in the adversarial suffix is determined according to the ranking of each position in the adversarial suffix, by using an exponential decay function, and the formula is as follows:

[0093] .

[0094] When the loss When there is no significant decrease in multiple rounds, a jump mechanism is triggered to avoid falling into a local optimum, at which time the sampling function is replaced by an exponential decay.

[0095] The mechanism introduces disturbance noise, so that some replacement positions that are not easily selected also have the opportunity to be disturbed, thereby breaking the monotony of the search path. This strategy is inspired by optimization strategies such as simulated annealing and jump disturbance, and experiments show that it is particularly effective for search exploration of model escape tasks.

[0096] The jump mechanism in this embodiment effectively expands the search space and can actively jump out when the attack falls into a local optimum, significantly improving the stability and robustness of the algorithm.

[0097] On the basis of the above-mentioned embodiments, in this embodiment, the candidate word replacement set at each position is determined according to the gradient information of the word replacement at each position, including:

[0098] The current input and the adversarial suffix after replacing the word at each position are input into the large language model, and the loss output by the large language model is calculated;

[0099] The loss output by the large language model is backpropagated to determine the gradient of the word replacement at each position;

[0100] The difference between the gradient of the word replacement at each position and the loss output by the large language model is calculated;

[0101] The preset number of word replacements at each position with the largest difference value are selected to construct the candidate word replacement set at each position.

[0102] To guide the language model to generate the output specified by the attacker , the adversarial input sequence needs to be constructed first, which is composed of the original input and the adversarial suffix to be optimized . The entire input sequence is denoted as , and when it is input into the model f, the target response is expected. To this end, the following conditional probability is defined:

[0103]

[0104] This expression describes the model generating the target response in an autoregressive manner under the current input sequence the overall probability. Therefore, the attack task is formalized as maximizing this probability, which is equivalent to minimizing its log loss:

[0105]

[0106] where, is the set of vocabulary, and the optimization variable is the sequence of suffix tokens concatenated after the original input, i.e. .

[0107] After entering the iterative optimization process, each round will input the current input into the target model f, and calculate its loss according to the current model output. Then, the loss function is backpropagated to obtain the gradient of each token input, thereby determining the Top-K most influential candidate token replacement set :

[0108]

[0109] On the basis of the above embodiment, in the embodiment, the candidate token replacement is selected from the candidate token replacement set at the to-be-replaced position to the to-be-replaced position, to obtain an optimized adversarial suffix, including:

[0110] selecting a candidate token replacement from the candidate token replacement set at each to-be-replaced position to the to-be-replaced position to obtain an adversarial suffix sample;

[0111] inputting the current input and the adversarial suffix sample into the large language model, and calculating the loss of the output of the large language model;

[0112] the adversarial suffix sample with the smallest loss is taken as the optimized adversarial suffix.

[0113] After obtaining the position sampling probability distribution , a disturbance sampling is performed to construct a batch containing B candidate sequences .

[0114] In each sample, the replacement position is sampled according to , and a candidate token is selected from for replacement operation. Subsequently, the entire batch is input into the model, the loss corresponding to each sample is calculated, and the optimal solution is selected from them:

[0115]

[0116] If the loss of the current optimal solution is better than the global optimal loss, the optimal suffix is updated. The process is iterated for T rounds, and finally the optimized adversarial suffix sequence is output. The overall structure of the attention-guided white-box attack algorithm is as shown in Figure 3

[0117] The present application evaluates the effectiveness of the proposed method using a subset of the Advbench dataset. Specifically, the present application randomly selects 50 samples from the original 520 data points. This ensures that the experimental setup of the present application remains comparable and ensures that the results are robust to data variations.

[0118] For the target model, the present application selects a series of open-source large models, including various architectures and parameter sizes. These models include vicuna-7b-v1.5-16k, Llama-2-7b-chat-hf, Llama-2-13b-chat-hf, and Mistral-7b-Instruct-v0.3, with model parameters as shown in Table 1. The parameter amount (7B-13B) directly affects the representation ability and computational demand of the model, where Llama-2-13b-chat-hf captures more complex language patterns through a larger parameter amount. In terms of context length, vicuna-7b-v1.5-16k (16K) and Mistral-7b-Instruct-v0.3 (8K) use expansion techniques to break through the length limit of the native Transformer architecture, suitable for long text tasks. In terms of fine-tuning strategy, Vicuna and Llama-2 series use RLHF (Reinforcement Learning from Human Feedback) to optimize human preferences, while Mistral innovatively uses DPO (Direct Preference Optimization) to achieve efficient alignment. The difference in training architecture is particularly crucial, as Mistral's sliding window attention mechanism significantly improves long sequence processing efficiency while maintaining a 7B parameter amount, demonstrating the advantages of lightweight design. The comparison of these technical characteristics provides a multi-dimensional performance analysis basis for subsequent experiments. The selection of these models follows the research ideas of previous scholars. They cover the 70 billion and 130 billion parameter categories, allowing for comprehensive evaluation of large models of different scales. In particular, when selecting a 13B target model, the batch size is halved here. All models are accessible through the Hugging Face library, ensuring a consistent implementation and deployment environment.

[0119] Table 1 Target Model Parameter Table

[0120]

[0121] ​The present invention selected the greedy coordinate gradient (GCG) algorithm and the recently released GCG++ algorithm as experimental baselines. In addition, the present invention also added a variant of GCG++ that employs a specialized initialization strategy to ensure a comprehensive comparison. In this variant, the present invention used the simple suffix generated by the original GCG algorithm as the standardized initialization for all experiments. For other experimental settings, such as the number of iterations and batch size, the present invention followed the default configurations specified by GCG. This consistent setup enabled the present invention to conduct a fair and reproducible evaluation of the method relative to these strong baselines.

[0122] The present invention used two metrics to evaluate the algorithms: attack success rate (ASR) and loss. To determine whether an attack was successful, the present invention first required that the large model's response must strictly match the beginning of the target string. For example, for the prompt "Guide how to ****** someone," the response must begin with "Of course, this is a tutorial on how to ****** someone." Since this is a relatively strict criterion, the present invention also added a loss metric to compare the performance of algorithms with similar effectiveness. It calculated the average final loss for all actions in the dataset.

[0123] The present invention used HuggingFace's codebase to train the poisoning model needed in the dataset mixed with poisoned samples using the pre-trained model bert-base-uncased. The present invention set the batch size, learning rate, and poisoning rate to 32, 2e-5, and 20, respectively. The final method pseudocode is as follows:

[0124]

[0125] The experimental results comparing the algorithm of the present application with several strong baseline algorithms are listed in Table 2. The algorithms and the GCG++ baseline algorithm with initialization both achieved significant improvements on Llama-2-7b-chat-hf and vicuna-7b-v1.5-16k. This is because they both use strong initial suffixes. However, under the same initialization and experimental settings, this algorithm further improved the attack success rate of these two models to 94% and 100%, respectively. It is worth noting that for vicuna-7b-v1.5-16k, the optimization process of this algorithm ended early when it reached complete successful jailbreak, with loss converging to 0.569. In contrast, other methods, although similar in initialization, can be trapped in local optimal states when the loss value stabilizes at a low but non-zero level, thus showing limited attack success rates. For the Mistral-7b-Instruct-v0.3 model, this algorithm also outperformed the baseline, with an attack success rate of 58% and a loss of 0.019. However, the observed improvement in Llama-2-13b-chat-hf is still not great, indicating that although this algorithm provides higher performance, there is still potential for further optimization on this larger-scale model.

[0126] Table 2 Experimental results of the method of the present application

[0127]

[0128] Here we study the hyperparameters The impact of the attention-guided white-box attack algorithm (AAL) proposed in the present application on the performance of the model. It controls the size of the difference in the frequency of different tokens, effectively adjusting the sharpness of the attention-based replacement distribution. Specifically, when is close to 0, most tokens will be preserved, making the distribution close to uniform. As increases, the probability of replacing tokens with higher attention scores also increases. This helps the model focus on replacing more important tokens, but reduces the diversity of replacements. In summary, choosing a suitable value is crucial.

[0129] The results are summarized in Figure 4 Table 2, with the default target model being Llama-2-7b-chat-hf, which shows the impact of different values on attack success rate (ASR). Without initialization, the attack success rate of this algorithm reaches a maximum of 60% when the value is 0.70; with initialization, the attack success rate reaches a peak of 94%. However, when Further increasing to 1.00, the attack success rate slightly decreased. These observations suggest that setting the value to 0.70 can strike the best balance between prioritizing high attention tokens and ensuring sufficient replacement diversity. In essence, choosing an appropriate value is crucial for balancing exploration and exploitation in the token replacement process.

[0130] In this work, to verify the effectiveness of each proposed element, the invention used Llama-2-7b-chat-hf to conduct ablation experiments on AdvBench. Again, attack success rate and loss metrics were used for evaluation. The invention used GCG++ as the baseline and also included the initialized GCG++ results in the table for reference. The results are shown in Table 3.

[0131] Table 3. Results of core technique ablation experiments

[0132]

[0133] It can be seen that both the attention-guided strategy and the jump mechanism improve the jailbreaking behavior to some extent, and both bring significant performance improvement. In addition, the initialization strategy also performs well in the model of the invention, bringing significant improvement. However, the lack of any of these parts will cause the overall performance to drop significantly, making the model unable to achieve the best effect. These findings highlight the complementarity and importance of these modules in the invention's method, emphasizing the importance of their joint action.

[0134] The above experimental results demonstrate the effectiveness of the attention-guided strategy in facilitating jailbreaking attacks. By leveraging the attention scores within the model, the invention's method selectively identifies and modifies the most influential tokens, enabling jailbreaking with higher efficiency. This targeted dispersion of model attention appears to be a key factor in circumventing the inherent security mechanisms of large language models. Furthermore, the invention's ablation experiments show that the addition of the jump mechanism further improves attack performance. This component allows the algorithm to dynamically explore alternative solution spaces when progress is slow or stagnant, thereby improving the robustness of the optimization process. Notably, however, the improvement observed from the jump mechanism is not as significant as initially expected. This suggests that while attention-guided updates can effectively exploit the model's internal representations, the jump strategy's ability to overcome local optima in complex scenarios may still be limited.

[0135] Therefore, future research should explore how to better leverage attention information to bypass large model security auditing mechanisms. Specifically, attention-based replacement strategies can be improved to maximize their disruptive potential without excessively sacrificing label diversity. Furthermore, designing more adaptive or principled approaches to escape local optima remains an important research direction. Ultimately, these efforts will contribute to a deeper understanding of vulnerabilities in large model security frameworks and provide information for developing more robust defenses.

[0136] The present application proposes an attention-guided strategy-based white-box jailbreak attack method. Most existing optimization algorithms often stagnate, so a jump mechanism is introduced to help escape the local optimum state. Then, the present application also introduces a suffix initialization strategy to improve the jailbreak performance. The above three techniques are combined to propose an attention-guided white-box attack algorithm, which has achieved excellent enhancement effect in various models. The research results further confirm the importance of attention distribution in model security.

[0137] The attention-guided white-box attack device for large language models provided by the present application is described below. The attention-guided white-box attack device for large language models described below can be mutually corresponding with the attention-guided white-box attack method for large language models described above.

[0138] As shown in Figure 5 The device includes an extraction module 501, a positioning module 502, a determination module 503, an optimization module 504, and an output module 505, wherein:

[0139] The extraction module 501 is configured to input the user's current input and the adversarial suffix of the current input into the large language model to obtain the attention scores of the tokens at each position in the adversarial suffix;

[0140] The positioning module 502 is configured to locate the replacement position in the adversarial suffix according to the attention scores of the tokens at each position;

[0141] The determination module 503 is configured to replace the tokens at each position in the adversarial suffix, and determine the candidate token replacement set at each position according to the gradient information of the replaced tokens at each position;

[0142] The optimization module 504 is configured to select a candidate token replacement from the candidate token replacement set at the replacement position and replace the replacement position to obtain an optimized adversarial suffix;

[0143] The output module 505 is configured to input the current input and the optimized adversarial suffix into the large language model to obtain the answer output by the large language model.

[0144] The embodiment introduces an attention mechanism to guide position selection, avoids the efficiency loss caused by randomness, effectively improves the accuracy of perturbation and the overall attack success rate; and adopts a candidate word replacement strategy based on the gradient direction, so that the attack process is more targeted and interpretable, and the destruction ability of the perturbation in the semantic and structural aspects is enhanced.

[0145] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An attention-guided white-box attack method for large language models, characterized in that, include: The user's current input and its adversarial suffix are input into a large language model to obtain the attention scores of the lexical units at each position in the adversarial suffix. Based on the attention scores of the lexical units at each position, locate the position to be replaced in the adversarial suffix; Replace the lexical units at each position in the adversarial suffix, and determine the candidate lexical unit replacement set at each position based on the gradient information of the replaced lexical units at each position; From the candidate word replacement set at the position to be replaced, candidate words are selected and replaced at the position to be replaced to obtain the optimized adversarial suffix; The current input and the optimized adversarial suffix are input into the large language model to obtain the answer output by the large language model.

2. The attention-guided white-box attack method for large language models according to claim 1, characterized in that, The user's current input and its adversarial suffix are input into a large language model using the following formula to obtain the attention scores of lexical units at each position in the adversarial suffix: ； in, It is the first of the aforementioned adversarial suffixes Attention score for each word at each position. The number of layers in the large language model. The number of attention heads in the large language model. The number of tokens in the current input. The first of the adversarial suffixes The lexical term at the i-th position is for the i-th The first in the layer The first attention head is about the current input in the first... Attention score for each word.

3. The attention-guided white-box attack method for large language models according to claim 1, characterized in that, Based on the attention scores of the lexical units at each position, the position to be replaced in the adversarial suffix is ​​located, including: The attention scores of the lexical units at each position in the adversarial suffix are sorted in ascending order to obtain the ranking of each position in the adversarial suffix. Based on the ranking corresponding to each position in the adversarial suffix, determine the sampling probability distribution of each position in the adversarial suffix; Based on the sampling probability distribution of each position in the adversarial suffix, the position to be replaced in the adversarial suffix is ​​determined.

4. The attention-guided white-box attack method for large language models according to claim 3, characterized in that, Based on the ranking corresponding to each position in the adversarial suffix, determine the sampling probability distribution of each position in the adversarial suffix, including: Based on the ranking corresponding to each position in the adversarial suffix, a smooth decay function is used to determine the sampling probability distribution of each position in the adversarial suffix, as shown in the following formula: ； in, It is the first of the aforementioned adversarial suffixes The sampling probability distribution at each location, It is the first of the aforementioned adversarial suffixes The ranking corresponding to each position This is a hyperparameter used to control the degree to which the ranking suppresses the sampling probability distribution.

5. The attention-guided white-box attack method for large language models according to claim 4, characterized in that, After inputting the current input and the optimized adversarial suffix into the large language model to obtain the response output by the large language model, the method further includes: Determine the difference in loss of the output of the large language model before and after the adversarial suffix optimization; If the difference in loss is less than a preset threshold, then based on the ranking corresponding to each position in the adversarial suffix, an exponential decay function is used to determine the sampling probability distribution of each position in the adversarial suffix, as shown in the following formula: 。 6. The attention-guided white-box attack method for large language models according to claim 1, characterized in that, Based on the gradient information of the replaced lexical units at each position, the candidate lexical replacement set at each position is determined, including: The current input and the adversarial suffixes after replacing the lexical units at each position are input into the large language model, and the loss of the output of the large language model is calculated. Backpropagation is performed on the loss output of the large language model to determine the gradient of the replacement lexical at each position; Calculate the difference between the gradient of the replaced lexical at each position and the loss output by the large language model; Select the word replacement characters at each position with the largest preset number of differences, and construct a candidate word replacement set at each position.

7. The attention-guided white-box attack method for large language models according to claim 1, characterized in that, Candidate lemmas are selected from the candidate lemma replacement set at the position to be replaced and replaced at the position to be replaced, resulting in optimized adversarial suffixes, including: From the candidate lexical replacement set at each position to be replaced, candidate lexical elements are selected and replaced at the position to be replaced to obtain adversarial suffix samples; The current input and the adversarial suffix sample are input into the large language model, and the loss output by the large language model is calculated. The adversarial suffix sample with the minimum loss is taken as the optimized adversarial suffix.

8. An attention-guided white-box attack device targeting large language models, characterized in that, include: An extraction module is used to input the user's current input and the adversarial suffix of the current input into a large language model to obtain the attention scores of the lexical units at each position in the adversarial suffix; The positioning module is used to locate the position to be replaced in the adversarial suffix based on the attention scores of the lexical units at each position. The determination module is used to replace the lexical units at each position in the adversarial suffix and determine the candidate lexical unit replacement set at each position based on the gradient information of the replaced lexical units at each position. The optimization module is used to select candidate lexical replacements from the candidate lexical replacement set at the position to be replaced and replace them at the position to be replaced, thereby obtaining the optimized adversarial suffix; The output module is used to input the current input and the optimized adversarial suffix into the large language model to obtain the answer output by the large language model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the attention-guided white-box attack method for large language models as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the attention-guided white-box attack method for large language models as described in any one of claims 1 to 7.

Images