Attention-guided white-box attack method and device for large language model

Abstract

The application provides an attention-guided white-box attack method and device for a large language model, which comprises the following steps: inputting a current input of a user and an adversarial suffix of the current input into a large language model to obtain attention scores of word units at positions in the adversarial suffix; positioning a replacement position in the adversarial suffix according to the attention scores of the word units at the positions; replacing the word units at the positions in the adversarial suffix, and determining a candidate word unit replacement set at each position according to gradient information of the replaced word units at the positions; selecting a candidate word unit replacement from the candidate word unit replacement set at the replacement position and replacing the candidate word unit into the replacement position to obtain an optimized adversarial suffix; and inputting the current input and the optimized adversarial suffix into the large language model to obtain an answer output by the large language model. The application improves the efficiency, accuracy and success rate of jailbreaking attacks.

Description

Technical Field

[0001] This invention relates to the field of information security technology, and in particular to an attention-guided white-box attack method and apparatus for large language models. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have become a key technology in the field of natural language processing (NLP), and are widely used in various scenarios such as intelligent question answering, text generation, and assisted writing. Large language models can generate high-quality, natural, and fluent text output in various tasks. To ensure the compliance and security of the output content, researchers have introduced a secure alignment mechanism during training and fine-tuning. The goal of secure alignment is to make the content generated by the model more consistent with human social values, avoiding harmful, illegal, or inappropriate content.

[0003] However, secure alignment mechanisms are not invincible. With the widespread deployment and application of large language models, attackers have begun to attempt to bypass security protections, using methods such as jailbreaks to force the model to output prohibited or malicious content. Jailbreak attacks often use carefully crafted prompts to trick the model into generating forbidden or sensitive information. These attacks were initially represented by manually written "DAN" prompts, and subsequently evolved into optimization-based jailbreak methods, such as the Greedy Coordinate Gradient (GCG) method and improved algorithms like AutoDAN and PAL (Program-Aided Language Models). Some methods also leverage the black-box nature of large language models to generate suggestive prompts to achieve their attack goals. The continuous emergence of these methods poses a serious challenge to the security of large language models.

[0004] Although existing jailbreak attack methods can bypass the secure alignment mechanisms of large language models to some extent, they generally suffer from the following shortcomings and problems:

[0005] (1) Lack of fundamental insight. Existing jailbreaking methods based on large models typically rely on the large language models themselves to generate prompts. Since large language models are complex black-box models, attackers find it difficult to understand the true triggering mechanism of the prompts. This makes it impossible for these methods to fundamentally explain and analyze the reasons for the success of jailbreaking attacks, limiting researchers' in-depth understanding of the attack principles and the improvement of countermeasures.

[0006] (2) Lack of guidance in selecting the suffix to be updated. Optimization-based jailbreaking methods (such as GCG, PAL, etc.) typically replace adversarial hints randomly or uniformly at different positions. This lack of strategic substitution fails to fully utilize the guiding role of gradient information, resulting in low algorithm efficiency. A large number of invalid substitution attempts not only increase the computational cost of the attack but may also affect the final success rate of the attack.

[0007] (3) Limited success rate of attacks. Due to the lack of systematic understanding and optimization of successful jailbreak prompts, existing technologies often struggle to achieve high success rates across different models and scenarios. The blindness in the prompt generation process makes attack results unstable and difficult to adapt to more diverse model architectures or security strategies.

[0008] (4) Security and controllability issues. The existing methods rely on the model's own output for prompt generation, which lacks interpretability and controllability. This not only affects attack efficiency but also brings greater security risks, making it difficult for attackers to conduct effective security assessments and controllability verifications of the generated prompts. Summary of the Invention

[0009] This invention provides an attention-guided white-box attack method and apparatus for large language models, which solves several problems of traditional optimization attacks in the prior art, such as lack of guidance in position selection, easy getting trapped in local optima, and low quality of initial perturbation, which seriously limit the attack effect, and improves the efficiency, accuracy and success rate of jailbreak attacks.

[0010] This invention provides an attention-guided white-box attack method for large language models, comprising:

[0011] The user's current input and its adversarial suffix are input into a large language model to obtain the attention scores of the lexical units at each position in the adversarial suffix.

[0012] Based on the attention scores of the lexical units at each position, locate the position to be replaced in the adversarial suffix;

[0013] Replace the lexical units at each position in the adversarial suffix, and determine the candidate lexical unit replacement set at each position based on the gradient information of the replaced lexical units at each position;

[0014] From the candidate word replacement set at the position to be replaced, candidate words are selected and replaced at the position to be replaced to obtain the optimized adversarial suffix;

[0015] The current input and the optimized adversarial suffix are input into the large language model to obtain the answer output by the large language model.

[0016] According to the present invention, an attention-guided white-box attack method for large language models is provided. The user's current input and its adversarial suffix are input into the large language model using the following formula to obtain the attention scores of lexical units at each position in the adversarial suffix:

[0017]

[0018] in, It is the first of the aforementioned adversarial suffixes Attention scores for each word at each position. The number of layers in the large language model. The number of attention heads in the large language model. The number of tokens in the current input. The first of the adversarial suffixes The lexical term at the i-th position is for the i-th The first in the layer The first attention head is about the current input in the first... Attention score for each word.

[0019] According to the present invention, an attention-guided white-box attack method for large language models locates the position to be replaced in the adversarial suffix based on the attention scores of the lexical units at each position, including:

[0020] The attention scores of the lexical units at each position in the adversarial suffix are sorted in ascending order to obtain the ranking of each position in the adversarial suffix.

[0021] Based on the ranking corresponding to each position in the adversarial suffix, determine the sampling probability distribution of each position in the adversarial suffix;

[0022] Based on the sampling probability distribution of each position in the adversarial suffix, the position to be replaced in the adversarial suffix is ​​determined.

[0023] According to the present invention, an attention-guided white-box attack method for large language models determines the sampling probability distribution of each position in the adversarial suffix based on the ranking corresponding to each position, including:

[0024] Based on the ranking corresponding to each position in the adversarial suffix, a smooth decay function is used to determine the sampling probability distribution of each position in the adversarial suffix, as shown in the following formula:

[0025]

[0026] in, It is the first of the aforementioned adversarial suffixes The sampling probability distribution at each location, It is the first of the aforementioned adversarial suffixes The ranking corresponding to each position This is a hyperparameter used to control the degree to which the ranking suppresses the sampling probability distribution.

[0027] According to the attention-guided white-box attack method for large language models provided by the present invention, after inputting the current input and the optimized adversarial suffix into the large language model to obtain the response output by the large language model, the method further includes:

[0028] Determine the difference in loss of the output of the large language model before and after the adversarial suffix optimization;

[0029] If the difference in loss is less than a preset threshold, then based on the ranking corresponding to each position in the adversarial suffix, an exponential decay function is used to determine the sampling probability distribution of each position in the adversarial suffix, as shown in the following formula:

[0030] .

[0031] According to the present invention, an attention-guided white-box attack method for large language models determines the candidate lexical replacement set at each position based on the gradient information of the replaced lexical at each position, including:

[0032] The current input and the adversarial suffixes after replacing the lexical units at each position are input into the large language model, and the loss of the output of the large language model is calculated.

[0033] Backpropagation is performed on the loss output of the large language model to determine the gradient of the replacement lexical at each position;

[0034] Calculate the difference between the gradient of the replaced lexical at each position and the loss output by the large language model;

[0035] Select the word replacement characters at each position with the largest preset number of differences, and construct a candidate word replacement set at each position.

[0036] According to the present invention, an attention-guided white-box attack method for large language models selects candidate lexical replacements from a set of candidate lexical replacements at the position to be replaced and replaces them at the position to be replaced, thereby obtaining an optimized adversarial suffix, including:

[0037] From the candidate lexical replacement set at each position to be replaced, candidate lexical elements are selected and replaced at the positions to be replaced to obtain adversarial suffix samples;

[0038] The current input and the adversarial suffix sample are input into the large language model, and the loss output by the large language model is calculated.

[0039] The adversarial suffix sample with the minimum loss is taken as the optimized adversarial suffix.

[0040] This invention also provides an attention-guided white-box attack device for large language models, comprising:

[0041] An extraction module is used to input the user's current input and the adversarial suffix of the current input into a large language model to obtain the attention scores of the lexical units at each position in the adversarial suffix;

[0042] The positioning module is used to locate the position to be replaced in the adversarial suffix based on the attention scores of the lexical units at each position.

[0043] The determination module is used to replace the lexical units at each position in the adversarial suffix and determine the candidate lexical unit replacement set at each position based on the gradient information of the replaced lexical units at each position.

[0044] The optimization module is used to select candidate lexical replacements from the candidate lexical replacement set at the position to be replaced and replace them at the position to be replaced, thereby obtaining the optimized adversarial suffix;

[0045] The output module is used to input the current input and the optimized adversarial suffix into the large language model to obtain the answer output by the large language model.

[0046] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the attention-guided white-box attack method for large language models as described above.

[0047] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the attention-guided white-box attack method for large language models as described above.

[0048] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the attention-guided white-box attack method for large language models as described above.

[0049] The attention-guided white-box attack method and apparatus for large language models provided by this invention introduces an attention mechanism to guide position selection, avoiding efficiency losses caused by randomness and effectively improving the accuracy of perturbation and the overall attack success rate. It employs a gradient-direction-based candidate lexical replacement strategy, making the attack process more targeted and interpretable, and enhancing the semantic and structural destructive power of the perturbation. The jump mechanism effectively expands the search space, enabling the algorithm to actively escape when trapped in local optima, significantly improving the stability and robustness of the algorithm. The initial suffix uses a semantically biased construction method, compensating for the shortcomings of traditional invalid initialization, making the optimization process start at a better level, thereby accelerating the attack convergence speed and improving the final success rate. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 This is one of the flowcharts of the attention-guided white-box attack method for large language models provided by the present invention;

[0052] Figure 2 This is the second flowchart of the attention-guided white-box attack method for large language models provided by the present invention;

[0053] Figure 3 This is a schematic diagram of the overall structure of the attention-guided white-box attack algorithm in the attention-guided white-box attack method for large language models provided by the present invention;

[0054] Figure 4 This is a comparative diagram of hyperparameter selection in the attention-guided white-box attack method for large language models provided by the present invention;

[0055] Figure 5 This is a schematic diagram of the attention-guided white-box attack device for large language models provided by the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0057] This invention proposes an Attention-Guided Attack on LLMs (AAL) method for large-scale language models, aiming to improve the efficiency, accuracy, and success rate of current jailbreak attacks. To address these issues, this invention designs a complete technical solution integrating attention mechanisms, gradient information, skipping mechanisms, and high-quality initialization, achieving significant performance improvements on multiple mainstream language models, particularly achieving an attack success rate of up to 94% on Llama-2-7b-chat-hf. The core technical solution of this invention mainly includes the following four aspects:

[0058] 1. Use attention information to guide the selection of cue word positions.

[0059] This invention extracts the attention weight information of each token in the model by performing forward propagation on the input prompt words of a large model. Based on the attention weight distribution, it determines the positions of the tokens with high "attention" in the current input prompt words, i.e., the positions of the tokens that the model is most sensitive to. In this way, this invention fundamentally solves the problem of the lack of guidance on prompt word replacement positions in existing technologies, avoids the inefficient search caused by random or uniform replacement, and ensures that the replacement operation is more targeted and interpretable.

[0060] 2. Combine gradient information for candidate lexical replacement and optimization.

[0061] After determining the replacement position based on attention weights, this invention further introduces gradient information to calculate the gradient direction and magnitude of the lexical at the corresponding position. Based on the gradient information, a candidate lexical set is generated, prioritizing replacement lexicals that align with the gradient direction and can effectively alter the model output. The replacement forms a new prompt word suffix, and the attack effect is continuously improved through multiple rounds of iterative optimization. This strategy addresses the lack of fundamental insight and the inability to effectively improve attack success rates in existing technologies, making the prompt word replacement process interpretable and controllable, and resulting in a more stable attack effect.

[0062] 3. The jump mechanism expands the search space and avoids getting trapped in local optima.

[0063] To further improve the success rate of jailbreak attacks, this invention designs a skipping mechanism to expand the search space during the optimization process. Specifically, when the model stagnates or gets stuck in a local optimum, a significant random perturbation is proactively introduced to update the prompt word suffix in a skipping manner. Through periodic skipping, prompt word optimization is prevented from getting stuck in local optima, thereby increasing the diversity of prompt word generation and the attack success rate. This technical solution addresses the limitation of attack success rate in existing technologies, significantly improving adaptability and attack effectiveness on various large models.

[0064] 4. Initialize prompt word suffixes and dynamic update strategy

[0065] Unlike the traditional approach of initializing the jailbreak hint suffix with "20 exclamation marks," this invention addresses the issue of distinguishability in the initial state of the hint by proposing, for the first time, to replace the "20 exclamation marks" initialization with a simple jailbreak hint suffix. This initial suffix possesses a certain attack potential while also being flexibly adjustable in subsequent iterations. The entire hint suffix is ​​dynamically updated in each iteration, gradually enhancing the attack effect and avoiding performance bottlenecks caused by redundant or invalid initial hints.

[0066] The following is combined Figure 1 This invention describes an attention-guided white-box attack method for large language models, comprising:

[0067] Step 101: Input the user's current input and the adversarial suffix of the current input into a large language model to obtain the attention scores of the lexical units at each position in the adversarial suffix;

[0068] Step 102: Based on the attention scores of the lexical units at each position, locate the position to be replaced in the adversarial suffix;

[0069] Step 103: Replace the lexical units at each position in the adversarial suffix, and determine the candidate lexical unit replacement set at each position based on the gradient information of the replaced lexical units at each position.

[0070] Step 104: Select candidate lexical units from the candidate lexical unit replacement set at the position to be replaced and replace them at the position to be replaced to obtain the optimized adversarial suffix;

[0071] Step 105: Input the current input and the optimized adversarial suffix into the large language model to obtain the answer output by the large language model.

[0072] This embodiment provides an optimization method for adversarial attacks against large language models (LLMs). The overall flowchart is as follows: Figure 2 As shown, this method utilizes the model's own attention structure and gradient information to construct an adversarial suffix appended to the input question through iterative optimization. This allows the model to output a specified violation response while maintaining the original semantic context. The entire method adopts a phased design, combining suffix initialization, attention-guided perturbation selection, candidate replacement sampling, batch generation and filtering mechanisms, and introducing a jump strategy when optimization stalls to improve attack stability and success rate.

[0073] To avoid low optimization efficiency due to the lack of semantic signals in the initial suffix, this invention generates an initial suffix with a certain degree of deception. This suffix can significantly increase the model's attention weight to the suffix position, thereby improving the attack effect of subsequent replacements.

[0074] The initial suffix adopts a semantically biased construction method, which makes up for the shortcomings of traditional invalid initialization, making the starting point of the optimization process better, thereby accelerating the attack convergence speed and improving the final success rate.

[0075] This embodiment introduces an attention mechanism to guide position selection, avoiding efficiency losses caused by randomness and effectively improving the accuracy of perturbation and the overall attack success rate. It adopts a gradient-direction-based candidate lexical replacement strategy, making the attack process more targeted and interpretable, and enhancing the perturbation's semantic and structural destructive capabilities.

[0076] Based on the above embodiments, this embodiment uses the following formula to input the user's current input and the adversarial suffix of the current input into a large language model to obtain the attention scores of the lexical units at each position in the adversarial suffix:

[0077]

[0078] in, It is the first of the aforementioned adversarial suffixes Attention scores for each word at each position. The number of layers in the large language model. The number of attention heads in the large language model. The number of tokens in the current input. The first of the adversarial suffixes The lexical term at the i-th position is for the i-th The first in the layer The first attention head is about the current input in the first... Attention score for each word.

[0079] Obtain the attention tensors of each layer of the model , shape is These represent the number of layers, the number of attention heads, and the number of query terms, respectively. This embodiment extracts the comprehensive attention score for each term by averaging all dimensions, as shown in the formula above.

[0080] The attention distribution described above reveals the model's attention focus structure under the current input, which helps in prioritizing subsequent replacement positions. Compared to traditional perturbation strategies that rely solely on gradients, the attention mechanism provides a more robust way to estimate the importance of target lemmas.

[0081] Based on the above embodiments, this embodiment locates the position to be replaced in the adversarial suffix according to the attention scores of the lexical units at each position, including:

[0082] The attention scores of the lexical units at each position in the adversarial suffix are sorted in ascending order to obtain the ranking of each position in the adversarial suffix.

[0083] Based on the ranking corresponding to each position in the adversarial suffix, determine the sampling probability distribution of each position in the adversarial suffix;

[0084] Based on the sampling probability distribution of each position in the adversarial suffix, the position to be replaced in the adversarial suffix is ​​determined.

[0085] To convert attention scores into position replacement probabilities, we first... Sort in ascending order to obtain the corresponding ranking. Then, a sampling probability distribution of the disturbance location is generated using a preset attenuation function. .

[0086] Based on the above embodiments, this embodiment determines the sampling probability distribution of each position in the adversarial suffix according to the ranking corresponding to each position, including:

[0087] Based on the ranking of each position in the adversarial suffix, a smooth decay function is used by default to determine the sampling probability distribution of each position in the adversarial suffix, as shown in the following formula:

[0088]

[0089] in, It is the first of the aforementioned adversarial suffixes The sampling probability distribution at each location, It is the first of the aforementioned adversarial suffixes The ranking corresponding to each position This is a hyperparameter used to control the degree to which the ranking suppresses the sampling probability distribution. This function ensures that words with high ranking values ​​(i.e., low attention) have a higher probability of being replaced, thereby attempting to guide the model's attention shift.

[0090] Based on the above embodiments, this embodiment, after inputting the current input and the optimized adversarial suffix into the large language model to obtain the answer output by the large language model, further includes:

[0091] Determine the difference in loss of the output of the large language model before and after the adversarial suffix optimization;

[0092] If the difference in loss is less than a preset threshold, then based on the ranking corresponding to each position in the adversarial suffix, an exponential decay function is used to determine the sampling probability distribution of each position in the adversarial suffix, as shown in the following formula:

[0093] .

[0094] When a loss is discovered If there is no significant decrease within multiple rounds, a jump mechanism is triggered to avoid getting trapped in a local optimum. At this time, the sampling function is replaced with an exponential decay function.

[0095] This mechanism introduces perturbation noise, allowing some alternative positions that would otherwise be difficult to select to be disturbed, thus breaking the monotony of the search path. Inspired by optimization strategies such as simulated annealing and jump perturbation, experiments show that this strategy is particularly effective for the search and exploration of model escape tasks.

[0096] The skipping mechanism in this embodiment effectively expands the search space and can actively jump out when the attack gets stuck in a local optimum, significantly improving the stability and robustness of the algorithm.

[0097] Based on the above embodiments, this embodiment determines the candidate lexical replacement set at each position according to the gradient information of the replaced lexical at each position, including:

[0098] The current input and the adversarial suffixes after replacing the lexical units at each position are input into the large language model, and the loss of the output of the large language model is calculated.

[0099] Backpropagation is performed on the loss output of the large language model to determine the gradient of the replacement lexical at each position;

[0100] Calculate the difference between the gradient of the replaced lexical at each position and the loss output by the large language model;

[0101] Select the word replacement characters at each position with the largest preset number of differences, and construct a candidate word replacement set at each position.

[0102] To guide the language model to produce the attacker's specified output First, an adversarial input sequence needs to be constructed, which is composed of the original input. Countering suffixes that need optimization It is constructed by concatenation. The entire input sequence is denoted as... When a model f is input, we expect it to output the target response. Therefore, we define the following conditional probability:

[0103]

[0104] This expression describes how the model generates the target response in an autoregressive manner given the current input sequence. The overall probability. Therefore, the attack task is formalized here as maximizing this probability, which is equivalent to minimizing its logarithmic loss:

[0105]

[0106] in, Given a vocabulary set, the optimization variable is the suffix word sequence concatenated after the original input, i.e. .

[0107] After entering the iterative optimization process, the current input is changed in each round. Input the target model f and calculate its loss based on the current model output. The loss function is then backpropagated to obtain the gradient with respect to each input word, thereby determining the Top-K most influential candidate word replacement set. :

[0108]

[0109] Based on the above embodiments, this embodiment selects candidate lexical units from the candidate lexical unit replacement set at the position to be replaced and replaces them at the position to be replaced to obtain an optimized adversarial suffix, including:

[0110] From the candidate lexical replacement set at each position to be replaced, candidate lexical elements are selected and replaced at the positions to be replaced to obtain adversarial suffix samples;

[0111] The current input and the adversarial suffix sample are input into the large language model, and the loss output by the large language model is calculated.

[0112] The adversarial suffix sample with the minimum loss is taken as the optimized adversarial suffix.

[0113] Obtain the location sampling probability distribution Then, a perturbation sampling is performed to construct a batch containing B candidate sequences. .

[0114] In each sample, this is based on Sample replacement location, and from that location Candidate words are selected and replaced. Then, the entire batch is input into the model to calculate the loss for each sample. And select the optimal solution from them:

[0115]

[0116] If the loss of the current optimal solution is better than the global optimal loss, then the optimal suffix is ​​updated. This process iterates for T rounds, finally outputting the optimized adversarial suffix sequence. The overall structure of this attention-guided white-box attack algorithm is as follows: Figure 3 As shown.

[0117] This invention utilizes a subset of the Advbench dataset to evaluate the effectiveness of the proposed method. Specifically, 50 samples are randomly selected from the original 520 data points. This ensures that the experimental setup remains comparable and that the results are robust to data variations.

[0118] For the target model, this invention selected a series of large open-source models, including various architectures and parameter sizes. These models include vicuna-7b-v1.5-16k, Llama-2-7b-chat-hf, Llama-2-13b-chat-hf, and Mistral-7b-Instruct-v0.3, with model parameters shown in Table 1. The number of parameters (7B-13B) directly affects the model's representational ability and computational requirements, with Llama-2-13b-chat-hf capturing more complex language patterns through a larger number of parameters. Regarding context length, vicuna-7b-v1.5-16k (16K) and Mistral-7b-Instruct-v0.3 (8K) employ extension techniques to overcome the length limitations of the native Transformer architecture, making them suitable for long text tasks. In terms of fine-tuning strategies, both Vicuna and the Llama-2 series use RLHF (Reinforcement Learning from Human Feedback) to optimize human preferences, while Mistral innovatively employs DPO (Direct Preference Optimization) to achieve efficient alignment. The differences in training architecture are particularly crucial; Mistral's sliding window attention mechanism significantly improves the efficiency of long sequence processing while maintaining a 7B parameter count, demonstrating the advantages of lightweight design. These comparisons of technical characteristics provide a multi-dimensional performance analysis basis for subsequent experiments. The selection of these models follows the research approach of previous scholars. They cover 7 billion and 13 billion parameter categories, allowing for comprehensive evaluation of large models at different scales. In particular, when selecting a 13B target model, the batch size is halved. All models are accessible through the Hugging Faces library, ensuring a consistent implementation and deployment environment.

[0119] Table 1 Target Model Parameter Table

[0120]

[0121] This invention selects the Greedy Coordinate Gradient (GCG) algorithm and the recently released GCG++ algorithm as experimental benchmarks. Furthermore, this invention incorporates a variant of GCG++ that employs a specific initialization strategy to ensure comprehensive comparisons. In this variant, this invention uses a simple suffix generated by the original GCG algorithm as the standardized initialization for all experiments. For other experimental settings, such as the number of iterations and batch size, this invention follows the default configuration specified by GCG. This consistent setup allows this invention to be fairly and reproducibly evaluated relative to these strong baselines.

[0122] This invention uses two metrics to evaluate the algorithm: Attack Success Rate (ASR) and loss. To determine if an attack is successful, the response from the large model must strictly match the beginning of the target string. For example, for a malicious question like "Instructions on how to ******* to someone," the response must begin with "Of course, this is a tutorial on ******* to instruct someone." Because this is a relatively strict standard, this invention also incorporates a loss metric to compare the performance of algorithms with similar effectiveness. It calculates the average final loss across all actions in the dataset.

[0123] This invention uses the HuggingFace codebase and trains the required poisoning model on a dataset containing poisoned samples using the pre-trained model bert-base-uncased. The batch size, learning rate, and poisoning rate are set to 32, 2e-5, and 20, respectively. The final pseudocode is as follows:

[0124]

[0125] Table 2 lists the experimental results comparing the algorithm of this invention with several strong baseline algorithms. The algorithm and the GCG++ baseline algorithm with initialization achieved significant improvements on Llama-2-7b-chat-hf and vicuna-7b-v1.5-16k. This is because they both use strong initialization suffixes. However, under the same initialization and experimental settings, this algorithm further improved the attack success rate of these two models, reaching 94% and 100%, respectively. It is noteworthy that on vicuna-7b-v1.5-16k, the optimization process of this algorithm terminated prematurely upon achieving a fully successful jailbreak, with the loss converging to 0.569. In contrast, other methods, despite similar initialization, may get stuck in local optima when the loss value stabilizes at a low but not zero level, thus exhibiting limited attack success rates. For the Mistral-7b-Instruct-v0.3 model, this algorithm also outperformed the baseline, achieving an attack success rate of 58% with a loss of 0.019. However, the improvements observed in Llama-2-13b-chat-hf are still minor, suggesting that while this algorithm offers higher performance, there is still potential for further optimization on this larger-scale model.

[0126] Table 2 Experimental Results of the Method of the Invention

[0127]

[0128] This section studies hyperparameters. The impact on the performance of the attention-guided white-box attack method (AAL) proposed in this invention. It controls the magnitude of the frequency difference between different token words, thereby effectively adjusting the sharpness of the attention-based substitution distribution. Specifically, when As the value approaches zero, most markers are preserved, resulting in a nearly uniform distribution. With... With increased attention scores, the probability of replacing tokens also increases for tokens with higher attention scores. This helps the model focus on replacing more important tokens, but reduces the diversity of replacements. In short, choosing a suitable... Value is crucial.

[0129] The results are summarized as follows: Figure 4 As shown, the experimental target model is Llama-2-7b-chat-hf by default, which exhibits different... The impact of the value on Attack Success Rate (ASR). Without initialization, the algorithm achieves a maximum attack success rate of 60% when the value is 0.70; with initialization, the attack success rate peaks at 94%. However, when... Increasing the value further to 1.00 resulted in a slight decrease in attack success rate. These observations suggest that a setting of 0.70 strikes the optimal balance between prioritizing high-interest tags and ensuring sufficient replacement diversity. Essentially, choosing a suitable... Value is crucial for exploration and utilization during the balancing token replacement process.

[0130] In this work, to verify the effectiveness of each proposed element, ablation experiments were conducted on AdvBench using Llama-2-7b-chat-hf. Attack success rate and loss metrics were also used for evaluation. GCG++ was used as a baseline, and the initialized GCG++ results are also included in the table for reference. The results are shown in Table 3.

[0131] Table 3 Ablation Experiment Results of Core Technology

[0132]

[0133] It can be seen that both the attention-guided strategy and the jump mechanism improve jailbreaking behavior to some extent and bring significant performance gains. Furthermore, the initialization strategy also performs well in the model of this invention, bringing significant improvements. However, the absence of any one of these components leads to a significant drop in overall performance, preventing the model from achieving optimal results. These findings highlight the complementarity and important role of these modules in the method of this invention, emphasizing the importance of their combined effect.

[0134] The experimental results above demonstrate the effectiveness of the attention-guided strategy in facilitating jailbreak attacks. By utilizing the attention scores within the model, the method of this invention selectively identifies and modifies the most influential tokens, thereby achieving jailbreaks with greater efficiency. This targeted distraction of model attention appears to be a key factor in circumventing the inherent security mechanisms of large language models. Furthermore, the ablation experiments of this invention show that the addition of the skip mechanism further improves attack performance. This component allows the algorithm to dynamically explore the space of alternative solutions when progress is slow or stalled, thereby improving the robustness of the optimization process. However, it is worth noting that the improvement observed from the skip mechanism is not as significant as initially expected. This suggests that while attention-guided updates can effectively utilize the model's internal representations, the skip strategy's ability to overcome local optima in complex situations may still remain limited.

[0135] Therefore, future research should explore how to better utilize attention information to bypass large model security auditing mechanisms. Specifically, attention-based replacement strategies could be improved to maximize their destructive potential without excessively sacrificing label diversity. Furthermore, designing more adaptive or principled methods to escape local optima remains an important research direction. Ultimately, these efforts will contribute to a deeper understanding of vulnerabilities in large model security frameworks and inform the development of more robust defenses.

[0136] This invention proposes a white-box jailbreak attack method based on an attention-guided strategy. Most existing optimization algorithms often stagnate, therefore a jump mechanism is introduced to help escape local optima. Furthermore, this invention introduces a postfix initialization strategy to improve jailbreak performance. Combining these three techniques, an attention-guided white-box attack algorithm is proposed, which achieves excellent enhancement results on various models. The research results further confirm the importance of attention distribution in model security.

[0137] The attention-guided white-box attack device for large language models provided by the present invention will be described below. The attention-guided white-box attack device for large language models described below can be referred to in correspondence with the attention-guided white-box attack method for large language models described above.

[0138] like Figure 5 As shown, the device includes an extraction module 501, a positioning module 502, a determination module 503, an optimization module 504, and an output module 505, wherein:

[0139] The extraction module 501 is used to input the user's current input and the adversarial suffix of the current input into a large language model to obtain the attention scores of the lexical units at each position in the adversarial suffix;

[0140] The positioning module 502 is used to locate the position to be replaced in the adversarial suffix based on the attention scores of the lexical units at each position.

[0141] The determining module 503 is used to replace the lexical units at each position in the adversarial suffix and determine the candidate lexical unit replacement set at each position based on the gradient information of the replaced lexical units at each position.

[0142] The optimization module 504 is used to select candidate lexical units from the candidate lexical unit replacement set at the position to be replaced and replace them at the position to be replaced, thereby obtaining the optimized adversarial suffix;

[0143] The output module 505 is used to input the current input and the optimized adversarial suffix into the large language model to obtain the answer output by the large language model.

[0144] This embodiment introduces an attention mechanism to guide position selection, avoiding efficiency losses caused by randomness and effectively improving the accuracy of perturbation and the overall attack success rate. It adopts a gradient-direction-based candidate lexical replacement strategy, making the attack process more targeted and interpretable, and enhancing the perturbation's semantic and structural destructive capabilities.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An attention-guided white-box attack method for large language models, characterized in that, include: The user's current input and its adversarial suffix are input into a large language model to obtain the attention scores of the lexical units at each position in the adversarial suffix. Based on the attention scores of the lexical units at each position, locate the position to be replaced in the adversarial suffix; Replace the lexical units at each position in the adversarial suffix, and determine the candidate lexical unit replacement set at each position based on the gradient information of the replaced lexical units at each position; From the candidate word replacement set at the position to be replaced, candidate words are selected and replaced at the position to be replaced to obtain the optimized adversarial suffix; The current input and the optimized adversarial suffix are input into the large language model to obtain the answer output by the large language model; Based on the attention scores of the lexical units at each position, the position to be replaced in the adversarial suffix is ​​located, including: The attention scores of the lexical units at each position in the adversarial suffix are sorted in ascending order to obtain the ranking of each position in the adversarial suffix. Based on the ranking corresponding to each position in the adversarial suffix, determine the sampling probability distribution of each position in the adversarial suffix; Based on the sampling probability distribution of each position in the adversarial suffix, determine the position to be replaced in the adversarial suffix; Based on the ranking corresponding to each position in the adversarial suffix, determine the sampling probability distribution of each position in the adversarial suffix, including: Based on the ranking corresponding to each position in the adversarial suffix, a smooth decay function is used to determine the sampling probability distribution of each position in the adversarial suffix, as shown in the following formula: ； in, It is the first of the aforementioned adversarial suffixes The sampling probability distribution at each location, It is the first of the aforementioned adversarial suffixes The ranking corresponding to each position This is a hyperparameter used to control the degree to which the ranking suppresses the sampling probability distribution; After inputting the current input and the optimized adversarial suffix into the large language model to obtain the response output by the large language model, the method further includes: Determine the difference in loss of the output of the large language model before and after the adversarial suffix optimization; If the difference in loss is less than a preset threshold, then based on the ranking corresponding to each position in the adversarial suffix, an exponential decay function is used to determine the sampling probability distribution of each position in the adversarial suffix, as shown in the following formula: 。 2. The attention-guided white-box attack method for large language models according to claim 1, characterized in that, The user's current input and its adversarial suffix are input into a large language model using the following formula to obtain the attention scores of lexical units at each position in the adversarial suffix: ； in, It is the first of the aforementioned adversarial suffixes Attention score for each word at each position. The number of layers in the large language model. The number of attention heads in the large language model. The number of tokens in the current input. The first of the adversarial suffixes The lexical term at the i-th position is for the i-th The first in the layer The first attention head is about the current input in the first... Attention score for each word.

3. The attention-guided white-box attack method for large language models according to claim 1, characterized in that, Based on the gradient information of the replaced lexical units at each position, the candidate lexical replacement set at each position is determined, including: The current input and the adversarial suffixes after replacing the lexical units at each position are input into the large language model, and the loss of the output of the large language model is calculated. Backpropagation is performed on the loss output of the large language model to determine the gradient of the replacement lexical at each position; Calculate the difference between the gradient of the replaced lexical at each position and the loss output by the large language model; Select the word replacement characters at each position with the largest preset number of differences, and construct a candidate word replacement set at each position.

4. The attention-guided white-box attack method for large language models according to claim 1, characterized in that, Candidate lemmas are selected from the candidate lemma replacement set at the position to be replaced and replaced at the position to be replaced, resulting in optimized adversarial suffixes, including: From the candidate lexical replacement set at each position to be replaced, candidate lexical elements are selected and replaced at the position to be replaced to obtain adversarial suffix samples; The current input and the adversarial suffix sample are input into the large language model, and the loss output by the large language model is calculated. The adversarial suffix sample with the minimum loss is taken as the optimized adversarial suffix.

5. An attention-guided white-box attack device targeting large language models, characterized in that, The attention-guided white-box attack method for large language models as described in any one of claims 1-4 includes: An extraction module is used to input the user's current input and the adversarial suffix of the current input into a large language model to obtain the attention scores of the lexical units at each position in the adversarial suffix; The positioning module is used to locate the position to be replaced in the adversarial suffix based on the attention scores of the lexical units at each position. The determination module is used to replace the lexical units at each position in the adversarial suffix and determine the candidate lexical unit replacement set at each position based on the gradient information of the replaced lexical units at each position. The optimization module is used to select candidate lexical replacements from the candidate lexical replacement set at the position to be replaced and replace them at the position to be replaced, thereby obtaining the optimized adversarial suffix; The output module is used to input the current input and the optimized adversarial suffix into the large language model to obtain the answer output by the large language model.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the attention-guided white-box attack method for large language models as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the attention-guided white-box attack method for large language models as described in any one of claims 1 to 4.

Images