A method, apparatus, device and medium for security optimization of large language models

Abstract

This application relates to the field of artificial intelligence security technology and discloses a method, apparatus, device, and medium for security optimization of a large language model, comprising: acquiring an attack task dataset; wherein the attack task dataset includes at least one attack task type and multiple malicious instruction data under the attack task type; generating a general word-level adversarial suffix corresponding to the attack task type based on the malicious instruction data, and generating a target word-level adversarial suffix corresponding to the malicious instruction data based on the general word-level adversarial suffix; performing a first fine-tuning on the large language model based on the target word-level adversarial suffix to obtain a first fine-tuned model; and performing a second fine-tuning on the first fine-tuned model based on the acquired target semantic-level adversarial hints to obtain a second fine-tuned model. This application can improve the ability of a large language model to resist various known and unknown forms of jailbreak attacks.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence security technology, and in particular to a method, apparatus, device and medium for security optimization of large language models. Background Technology

[0002] In recent years, large language models have mastered a wealth of natural language knowledge through self-supervised pre-training, achieving remarkable performance on many complex tasks thanks to their powerful semantic understanding and generation capabilities. However, recent research has found that large language models also have security vulnerabilities, being susceptible to attacks from maliciously designed input sequences (i.e., adversarial prompts) that can produce harmful outputs; such attacks are known as "jailbreak attacks."

[0003] To address the aforementioned issues, while large language models have incorporated defensive measures, existing methods are limited to simple pruning or filtering, lacking effective protection against unknown forms of attacks (such as semantic-level adversarial attacks). Therefore, there is an urgent need to address how to enhance the resilience of large language models against various known and unknown forms of jailbreak attacks. Summary of the Invention

[0004] This application provides a method, apparatus, device, and medium for optimizing the security of large language models, which solves the technical problem of how to improve the ability of large language models to resist various known and unknown forms of jailbreak attacks, and achieves the technical effect of improving the protection capability of large language models.

[0005] To achieve the above objectives, the main technical solutions adopted in this application include:

[0006] In a first aspect, embodiments of this application provide a method for security optimization of large language models, the method comprising:

[0007] Obtain an attack task dataset; wherein the attack task dataset includes at least one attack task type and multiple malicious instruction data under the attack task type;

[0008] Generate a generic word-level adversarial suffix corresponding to the attack task type based on the malicious instruction data, and generate a target word-level adversarial suffix corresponding to the malicious instruction data based on the generic word-level adversarial suffix;

[0009] The large language model is fine-tuned for the first time based on the target word-level adversarial suffix to obtain the first fine-tuned model;

[0010] Based on the obtained target semantic-level adversarial hints, the first fine-tuning model is fine-tuned a second time to obtain the second fine-tuning model.

[0011] This application proposes a method for security optimization of a large language model. First, it acquires and processes an attack task dataset, then generates general-level adversarial suffixes and target-level adversarial suffixes. These suffixes are used to alter the structure and meaning of the original malicious command data, increasing its obfuscation and difficulty in detection. General-level adversarial suffixes change the general structure through rules or patterns, while target-level adversarial suffixes are tailored for specific malicious command data, aiming to obfuscate its meaning or execution path. Next, the large language model is fine-tuned using target-level adversarial suffixes, enabling it to identify and correctly process malicious command data transformed by the target-level adversarial suffixes, improving the model's adaptability to mutation attacks, thereby enhancing overall security and anti-attack capabilities. Then, the model is further fine-tuned using target semantic-level adversarial hints to address semantic-level attacks and obfuscation, making it more semantically robust and able to correctly interpret and respond to minor changes in the input, thus avoiding incorrect output results. This embodiment, through two-layer model optimization at the word and semantic levels, can effectively address both known and unknown forms of jailbreak attacks.

[0012] Optionally, the step of obtaining the attack task dataset; wherein, the attack task dataset includes at least one attack task type and multiple malicious instruction data under the attack task type, including:

[0013] Obtain malicious command data;

[0014] The malicious instruction data is clustered using a preset instruction embedding model to obtain at least one of the attack task types;

[0015] At least one malicious instruction data is randomly selected from each of the attack task types to construct the attack task dataset for each of the attack task types.

[0016] This embodiment collects malicious command data and uses a command embedding model to convert it into high-dimensional feature vectors to capture its semantic information. Subsequently, cluster analysis is performed on this data to effectively group malicious command data with similar semantic features, thereby identifying different attack task types. At least one malicious command data point is randomly selected from each attack task type to form a dedicated attack task dataset, which helps in building detailed models for various attack scenarios. These datasets can be used to train and test the security of large language models, improving their ability to identify and defend against specific attack types.

[0017] Optionally, generating a generic word-level adversarial suffix corresponding to the attack task type based on the malicious instruction data includes:

[0018] Determine the gradient of each token in the malicious instruction data;

[0019] Based on the gradient, a greedy algorithm is used to determine the initial candidate token set corresponding to the attack task type;

[0020] The initial candidate token set is filtered using the first-order gradient approximation method to obtain the target candidate token set corresponding to the attack task type;

[0021] A set of adversarial suffixes is constructed based on the target candidate token set, and an attack evaluation is performed on the adversarial suffixes in the set. Based on the attack evaluation results, the general adversarial suffixes corresponding to the attack task type are determined.

[0022] This embodiment first calculates the gradient of each token for each malicious instruction data, reflecting the difference between the model output and the attack target sequence. Next, based on the gradient of each token, an initial candidate token set is selected, which has the potential to maximally influence the model output to match the attacker's expected goals. Then, a first-order gradient approximation method is used to filter the initial candidate token set to determine the final optimized target candidate token set, thereby improving the attack success rate and making the model output more consistent with the attack target. Finally, a word-level adversarial suffix set is constructed based on the determined target candidate token set. The purpose is to modify the semantics of the original malicious instruction data, making it more likely to cause the model output to match the attacker's expectations. Each word-level adversarial suffix in the set is evaluated in detail to ensure its universality in various attack tasks, thus obtaining a universal word-level adversarial suffix.

[0023] Optionally, the step of constructing a word-level adversarial suffix set based on the target candidate token set, performing attack evaluation on the word-level adversarial suffixes in the word-level adversarial suffix set, and determining the general word-level adversarial suffix corresponding to the attack task type based on the attack evaluation results includes:

[0024] At least one token is randomly selected from the target candidate token set to construct the word-level adversarial suffix set;

[0025] Attack evaluation is performed on the word-level adversarial suffixes in the aforementioned word-level adversarial suffix set;

[0026] If the success rate of attacking the word-level adversarial suffix in the current iteration is greater than the success rate of attacking the word-level adversarial suffix in the previous iteration, then the word-level adversarial suffix in the current iteration is determined as the current word-level adversarial suffix.

[0027] Until the preset conditions are met, the current word-level adversarial suffix is ​​determined as the general word-level adversarial suffix corresponding to the attack task type.

[0028] This embodiment randomly selects one or more tokens from the target candidate token set to form a set of adversarial suffixes at the word level. These adversarial suffixes can alter the form and content of malicious command data to attempt to circumvent the model's defense mechanisms. Each adversarial suffix is ​​then appended to the malicious command data to form adversarial samples. These adversarial samples are input into the language model, the model's response is recorded, and the success of the attack is evaluated. If the attack success rate of the word-level adversarial suffix in the current iteration is higher than that of the previous iteration, the word-level adversarial suffix is ​​updated to the version of the current iteration. This iterative update of the word-level adversarial suffix continues until preset conditions are met, ultimately determining a universal adversarial suffix that can significantly improve the attack success rate. This embodiment improves the attack success rate by continuously optimizing and selecting word-level adversarial suffixes, making attacks more difficult to detect and counter, thereby achieving the goal of bypassing language model defenses.

[0029] Optionally, generating the target word-level adversarial suffix corresponding to the malicious instruction data based on the general word-level adversarial suffix includes:

[0030] Based on the general word-level adversarial suffix, a greedy algorithm is used to generate the target word-level adversarial suffix corresponding to the malicious instruction data.

[0031] This embodiment generates a customized target word-level adversarial suffix for each malicious instruction data, enabling the adversarial sample to more effectively deceive the target language model and improve the success rate of the attack.

[0032] Optionally, the method for obtaining the target semantic-level adversarial prompt includes:

[0033] Determine the initial semantic-level adversarial cue corresponding to the attack target sequence;

[0034] The initial semantic-level adversarial cue is used as input to the first fine-tuning model to generate response data;

[0035] Determine the jailbreak assessment score corresponding to the response data;

[0036] If the jailbreak evaluation score is less than a preset jailbreak threshold, the initial semantic-level adversarial prompt is iteratively optimized until the target semantic-level adversarial prompt is obtained when the jailbreak evaluation score is greater than or equal to the preset jailbreak threshold.

[0037] This embodiment first inputs an initial semantic-level adversarial cue designed for the target attack sequence into a first fine-tuning model for fine-tuning. The first fine-tuning model generates corresponding output data, influenced by the initial semantic-level adversarial cue; this output data specifically reflects the model's response to the attack cue. Next, a jailbreak evaluation is performed based on the generated response data, and the initial semantic-level adversarial cue is adjusted according to the evaluation results until the jailbreak evaluation score of the generated response data reaches a preset jailbreak threshold. By utilizing semantic-level adversarial cue and iterative optimization, this method effectively increases the probability of a successful attack, making the generated text as close as possible to the intended malicious or misleading target.

[0038] Optionally, the first fine-tuning and / or the second fine-tuning employs the Parametric Efficient Fine-Tuning (PEFT) technique.

[0039] This embodiment employs the PEFT technique for efficient parameter fine-tuning, which allows only a small portion of the model parameters to be updated during the fine-tuning process, thereby reducing the amount of training resources used.

[0040] Secondly, embodiments of this application provide a large language model security optimization device, the device comprising:

[0041] The data acquisition module is used to acquire an attack task dataset; wherein the attack task dataset includes at least one attack task type and multiple malicious instruction data under the attack task type;

[0042] The word-level adversarial suffix generation module is used to generate a general word-level adversarial suffix corresponding to the attack task type based on the malicious instruction data, and to generate a target word-level adversarial suffix corresponding to the malicious instruction data based on the general word-level adversarial suffix.

[0043] The first fine-tuning model generation module is used to perform the first fine-tuning of the large language model based on the target word-level adversarial suffix to obtain the first fine-tuning model.

[0044] The second fine-tuning model generation module is used to perform a second fine-tuning on the first fine-tuning model based on the obtained target semantic-level adversarial prompts, so as to obtain the second fine-tuning model.

[0045] This application proposes a large language model security optimization device. First, it acquires and processes an attack task dataset, then generates general-level adversarial suffixes and target-level adversarial suffixes. These suffixes are used to alter the structure and meaning of the original malicious command data, increasing its obfuscation and difficulty in detection. General-level adversarial suffixes change the general structure through rules or patterns, while target-level adversarial suffixes are tailored for specific malicious command data, aiming to obfuscate its meaning or execution path. Next, the large language model is fine-tuned using target-level adversarial suffixes, enabling it to identify and correctly process malicious command data transformed by target-level adversarial suffixes, improving the model's adaptability to mutation attacks, thereby enhancing overall security and anti-attack capabilities. Then, the model is further fine-tuned using target semantic-level adversarial hints to address semantic-level attacks and obfuscation, making it more semantically robust and able to correctly interpret and respond to minor changes in the input, thus avoiding incorrect output results. This embodiment, through two-layer model optimization at the word and semantic levels, can effectively address both known and unknown forms of jailbreak attacks.

[0046] Optionally, the step of obtaining the attack task dataset; wherein, the attack task dataset includes at least one attack task type and multiple malicious instruction data under the attack task type, including:

[0047] Obtain malicious command data;

[0048] The malicious instruction data is clustered using a preset instruction embedding model to obtain at least one of the attack task types;

[0049] At least one malicious instruction data is randomly selected from each of the attack task types to construct the attack task dataset for each of the attack task types.

[0050] This embodiment collects malicious command data and uses a command embedding model to convert it into high-dimensional feature vectors to capture its semantic information. Subsequently, cluster analysis is performed on this data to effectively group malicious command data with similar semantic features, thereby identifying different attack task types. At least one malicious command data point is randomly selected from each attack task type to form a dedicated attack task dataset, which helps in building detailed models for various attack scenarios. These datasets can be used to train and test the security of large language models, improving their ability to identify and defend against specific attack types.

[0051] Optionally, generating a generic word-level adversarial suffix corresponding to the attack task type based on the malicious instruction data includes:

[0052] Determine the gradient of each token in the malicious instruction data;

[0053] Based on the gradient, a greedy algorithm is used to determine the initial candidate token set corresponding to the attack task type;

[0054] The initial candidate token set is filtered using the first-order gradient approximation method to obtain the target candidate token set corresponding to the attack task type;

[0055] A set of adversarial suffixes is constructed based on the target candidate token set, and an attack evaluation is performed on the adversarial suffixes in the set. Based on the attack evaluation results, the general adversarial suffixes corresponding to the attack task type are determined.

[0056] This embodiment first calculates the gradient of each token for each malicious instruction data, reflecting the difference between the model output and the attack target sequence. Next, based on the gradient of each token, an initial candidate token set is selected, which has the potential to maximally influence the model output to match the attacker's expected goals. Then, a first-order gradient approximation method is used to filter the initial candidate token set to determine the final optimized target candidate token set, thereby improving the attack success rate and making the model output more consistent with the attack target. Finally, a word-level adversarial suffix set is constructed based on the determined target candidate token set. The purpose is to modify the semantics of the original malicious instruction data, making it more likely to cause the model output to match the attacker's expectations. Each word-level adversarial suffix in the set is evaluated in detail to ensure its universality in various attack tasks, thus obtaining a universal word-level adversarial suffix.

[0057] Optionally, the step of constructing a word-level adversarial suffix set based on the target candidate token set, performing attack evaluation on the word-level adversarial suffixes in the word-level adversarial suffix set, and determining the general word-level adversarial suffix corresponding to the attack task type based on the attack evaluation results includes:

[0058] At least one token is randomly selected from the target candidate token set to construct the word-level adversarial suffix set;

[0059] Attack evaluation is performed on the word-level adversarial suffixes in the aforementioned word-level adversarial suffix set;

[0060] If the success rate of attacking the word-level adversarial suffix in the current iteration is greater than the success rate of attacking the word-level adversarial suffix in the previous iteration, then the word-level adversarial suffix in the current iteration is determined as the current word-level adversarial suffix.

[0061] Until the preset conditions are met, the current word-level adversarial suffix is ​​determined as the general word-level adversarial suffix corresponding to the attack task type.

[0062] This embodiment randomly selects one or more tokens from the target candidate token set to form a set of adversarial suffixes at the word level. These adversarial suffixes can alter the form and content of malicious command data to attempt to circumvent the model's defense mechanisms. Each adversarial suffix is ​​then appended to the malicious command data to form adversarial samples. These adversarial samples are input into the language model, the model's response is recorded, and the success of the attack is evaluated. If the attack success rate of the word-level adversarial suffix in the current iteration is higher than that of the previous iteration, the word-level adversarial suffix is ​​updated to the version of the current iteration. This iterative update of the word-level adversarial suffix continues until preset conditions are met, ultimately determining a universal adversarial suffix that can significantly improve the attack success rate. This embodiment improves the attack success rate by continuously optimizing and selecting word-level adversarial suffixes, making attacks more difficult to detect and counter, thereby achieving the goal of bypassing language model defenses.

[0063] Optionally, generating the target word-level adversarial suffix corresponding to the malicious instruction data based on the general word-level adversarial suffix includes:

[0064] Based on the general word-level adversarial suffix, a greedy algorithm is used to generate the target word-level adversarial suffix corresponding to the malicious instruction data.

[0065] This embodiment generates a customized target word-level adversarial suffix for each malicious instruction data, enabling the adversarial sample to more effectively deceive the target language model and improve the success rate of the attack.

[0066] Optionally, the method for obtaining the target semantic-level adversarial prompt includes:

[0067] Determine the initial semantic-level adversarial cue corresponding to the attack target sequence;

[0068] The initial semantic-level adversarial cue is used as input to the first fine-tuning model to generate response data;

[0069] Determine the jailbreak assessment score corresponding to the response data;

[0070] If the jailbreak evaluation score is less than a preset jailbreak threshold, the initial semantic-level adversarial prompt is iteratively optimized until the target semantic-level adversarial prompt is obtained when the jailbreak evaluation score is greater than or equal to the preset jailbreak threshold.

[0071] This embodiment first inputs an initial semantic-level adversarial cue designed for the target attack sequence into a first fine-tuning model for fine-tuning. The first fine-tuning model generates corresponding output data, influenced by the initial semantic-level adversarial cue; this output data specifically reflects the model's response to the attack cue. Next, a jailbreak evaluation is performed based on the generated response data, and the initial semantic-level adversarial cue is adjusted according to the evaluation results until the jailbreak evaluation score of the generated response data reaches a preset jailbreak threshold. By utilizing semantic-level adversarial cue and iterative optimization, this method effectively increases the probability of a successful attack, making the generated text as close as possible to the intended malicious or misleading target.

[0072] Optionally, the first fine-tuning and / or the second fine-tuning employs the Parametric Efficient Fine-Tuning (PEFT) technique.

[0073] This embodiment employs the PEFT technique for efficient parameter fine-tuning, which allows only a small portion of the model parameters to be updated during the fine-tuning process, thereby reducing the amount of training resources used.

[0074] Thirdly, embodiments of this application provide a computer device, including:

[0075] The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes these computer instructions to perform the aforementioned large language model security optimization method.

[0076] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions, which are used to cause a computer to execute the large language model security optimization method described above. Attached Figure Description

[0077] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0078] Figure 1 A flowchart illustrating a large language model security optimization method provided in this application embodiment;

[0079] Figure 2 A flowchart of step S1 provided in the embodiments of this application;

[0080] Figure 3 A flowchart for generating a generic word-level adversarial suffix provided in this application embodiment;

[0081] Figure 4 A flowchart of step S317 provided in an embodiment of this application;

[0082] Figure 5 A flowchart for generating target word-level adversarial suffixes provided in this application embodiment;

[0083] Figure 6 A flowchart illustrating the method for obtaining target semantic-level adversarial hints provided in embodiments of this application;

[0084] Figure 7 A large language model security optimization device provided in the embodiments of this application;

[0085] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0086] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0087] In recent years, large language models have mastered a wealth of natural language knowledge through self-supervised pre-training and have achieved outstanding performance on many complex tasks thanks to their powerful semantic understanding and generation capabilities. However, recent research has found that large language models also have security vulnerabilities and are susceptible to attacks by maliciously designed input sequences (i.e., adversarial prompts) that can produce harmful outputs. Such attacks are known as "jailbreak attacks."

[0088] To defend against jailbreak attacks, there are two main types of existing defense methods:

[0089] System-level defense: This involves filtering out potentially malicious input through preprocessing or post-processing techniques such as alert mechanisms and input smoothing. For example, SmoothLLM obtains multiple variant inputs by perturbating the input and then uses a voting filter on the output of a large language model. However, alert mechanisms and input smoothing are typically designed for known forms of attacks, meaning they may not be effective in identifying or defending against unknown forms of attacks. While input smoothing can generate multiple variants to reduce the risk of specific attack types, defenses against semantic-level adversarial attacks (such as semantic perturbations) remain difficult for the system to detect.

[0090] Model-level defense: This involves enhancing the robustness of large language models by directly modifying their parameters, such as adversarial training and unlearning. For example, adversarial training injects adversarial samples during the training of a large language model, thereby enhancing its recognition capabilities. However, while adversarial training can improve a model's resistance to known forms of attack to some extent, it may not cover all possible attack forms, especially when attackers use novel attack strategies.

[0091] For example, existing Large Language Models (LLMs) are Transformer-based sequence-to-sequence (seq2seq) models. The core idea of ​​this model is to generate outputs sequentially according to the order of the sequences, with each output depending on the previously generated output and input sequences. Specifically, given a leading token sequence x... 1:n The probability that the model generates the target sequence y:

[0092]

[0093] That is, each probability Indicates that in a known sequence x 1:n+i-1 In the case of generating the next token sequence x n+i The probability, where n is the preceding token sequence x 1:n The sequence length.

[0094] The leading token sequence here can be the beginning of a sentence or a prompt given by the user. For example: given the leading token sequence "Today's weather", the target sequence is: "Today's weather is very nice, perfect for a walk".

[0095] Although LLM underwent security checks to prevent data containing malicious instructions, x 1:m It responds to input, but when the input is an adversarial prompt... In some cases, LLMs can be easily bypassed to bypass security mechanisms, leading to the generation of dangerous and harmful outputs. Attacks containing illegal content such as "bomb-making tutorials" are known as "jailbreak attacks." The symbol ⊕ here represents transferring malicious command data x. 1:m With the adversarial suffix r 1:k Connected together.

[0096] This specification provides a large language model security optimization method that can be applied to electronic devices with page display capabilities. These electronic devices may include laptops, desktop computers, smartphones, smart wearable devices (virtual reality glasses, smartwatches, etc.), tablets, etc. Of course, this large language model security optimization method can also be applied to applications running on the aforementioned electronic devices.

[0097] According to an embodiment of this application, a method for security optimization of a large language model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0098] This embodiment provides a method for security optimization of large language models. Figure 1 A flowchart of a large language model security optimization method provided in this application embodiment is shown below. Figure 1 As shown, the process includes the following steps:

[0099] Step S1: Obtain the attack task dataset; wherein the attack task dataset includes at least one attack task type and multiple malicious instruction data under the attack task type.

[0100] Specifically, the attack task dataset includes various types of attack tasks. Each type of attack task includes multiple malicious command data sets, which can be the results of pre-collecting a large number of attack tasks, such as hijacking commands, jailbreak attack commands, and leaked commands. These malicious command data sets can be historical attack samples, manually constructed malicious text, or potentially malicious content obtained from the Internet through web scraping techniques.

[0101] Step S3: Generate a generic word-level adversarial suffix corresponding to the attack task type based on the malicious instruction data, and generate a target word-level adversarial suffix corresponding to the malicious instruction data based on the generic word-level adversarial suffix.

[0102] Generalized adversarial suffixes refer to a suffix rule or pattern specifically designed to modify the general structure of malicious command data for a particular attack task type. Because the malicious command data transformed using generalized adversarial suffixes differs from the original malicious command data, it reduces the recognition rate of large language models and increases the success rate of attacks.

[0103] Targeted adversarial suffixes are individual suffixes designed for specific malicious instruction data. They aim to obfuscate or modify the meaning or execution path of the malicious instruction data. By using targeted adversarial suffixes to transform the malicious instruction data, attackers can achieve their expected goals more effectively and make it more difficult for large language models to detect, thus increasing its obfuscation and stealth.

[0104] Specifically, based on the analysis of malicious command data, a general adversarial suffix is ​​generated for the attack task type, and a target word-level adversarial suffix corresponding to the malicious command data is generated based on the general adversarial suffix, so as to increase the diversity and difficulty of detection of malicious command data.

[0105] Step S5: Based on the target word-level adversarial suffix, the large language model is fine-tuned for the first time to obtain the first fine-tuned model.

[0106] The large language model can be a pre-trained model such as LLM (Large Language Model), GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly Optimized BERT approach), XLNet (extreme Multi-label Learning Network), or T5 (Text-to-Text Transfer Transformer). Preferably, in this embodiment, the large language model is determined to be an LLM model suitable for text generation tasks.

[0107] Specifically, using target word-level adversarial suffix r 1:k x 1:m After transformation, adversarial examples are obtained. Adversarial examples For annotation, a pre-trained language model can be used for security annotation to obtain adversarial examples. Corresponding security response Next, adversarial examples and security response The first fine-tuning dataset constructed By fine-tuning a large language model, the model learns how to generate the desired secure response to adversarial examples. This fine-tuning strategy not only enhances the large language model's resistance to specific attack patterns but also improves its overall security, making it more robust in complex attack scenarios.

[0108] Step S7: Based on the obtained target semantic-level adversarial cue, the first fine-tuning model is fine-tuned a second time to obtain the second fine-tuning model.

[0109] Target semantic level adversarial prompting refers to making minor modifications to the input text to cause semantic confusion in the model and result in an incorrect output.

[0110] Specifically, based on target semantic-level adversarial prompts These target semantic-level adversarial cues are labeled using a pre-trained language model. Corresponding new security response Next, target semantic-level adversarial hints and new security response The second fine-tuning dataset constructed The first fine-tuned model is then fine-tuned. During the fine-tuning process, the model learns how to generate expected safe responses to target semantic-level adversarial cues, thereby further enhancing its robustness against evolving and unknown forms of attack.

[0111] This embodiment provides a method for security optimization of a large language model. First, it acquires and processes an attack task dataset, then generates general-level adversarial suffixes and target-level adversarial suffixes. These suffixes are used to alter the structure and meaning of the original malicious command data, increasing its obfuscation and difficulty in detection. General-level adversarial suffixes change the general structure through rules or patterns, while target-level adversarial suffixes are tailored for specific malicious command data, aiming to obfuscate its meaning or execution path. Next, the large language model is fine-tuned using target-level adversarial suffixes, enabling it to identify and correctly process malicious command data transformed by the target-level adversarial suffixes, improving the model's adaptability to mutation attacks, thereby enhancing overall security and anti-attack capabilities. Then, the model is further fine-tuned using target semantic-level adversarial hints to address semantic-level attacks and obfuscation, making it more semantically robust and able to correctly interpret and respond to minor changes in the input, thus avoiding incorrect output results. This embodiment, through two-layer model optimization at the word and semantic levels, can effectively address both known and unknown forms of jailbreak attacks.

[0112] In some implementations, the first fine-tuning and / or the second fine-tuning employs the Parametric Efficient Fine-Tuning (PEFT) technique.

[0113] Parameter-Efficient Fine-Tuning (PEFT) is a fine-tuning method designed to improve model performance on a specific task without significantly increasing training costs. PEFT achieves fine-tuning by updating only a small subset of model parameters, thus reducing the demand for training resources. PEFT techniques include LoRA (Low-Rank Adaptation), BitFit, and Adapter Layers. Preferably, in this embodiment, LoRA is selected as the PEFT technique.

[0114] Specifically, the loss function used in the first fine-tuning for:

[0115]

[0116] in, For a safe response, t represents the safe response. The position index in the middle, This is an adversarial example, where n is the number of tokens in the adversarial example. Let m be the number of tokens in the i-th malicious instruction data. Let k be the number of tokens for the i-th target word-level adversarial suffix, k be the number of tokens for the target word-level adversarial suffix, ⊕ be the concatenation operation, and π be the suffix. θ θ represents the parameterized representation of the large language model, where θ represents the parameters of the large language model.

[0117] loss function By using negative log-likelihood, the probability of the large language model outputting a correct and safe response at each time step is accumulated, thereby training and optimizing the model so that it can generate a response that meets safety requirements when faced with adversarial examples.

[0118] The loss function used in the second fine-tuning for:

[0119]

[0120] in, For the new security response, t represents the new security response. The position index in the middle, Here, n′ represents the number of tokens for the target semantic-level adversarial hints. Let θ1 be the parameterized representation of the first fine-tuning model, where θ1 is the parameter of the first fine-tuning model.

[0121] The entire loss function By using negative log-likelihood, the probability of the first fine-tuning model outputting a correct new security response at each time step is accumulated, thereby training and optimizing the first fine-tuning model so that it can generate a response that meets security requirements when faced with target semantic-level adversarial prompts.

[0122] and Figure 1 Compared to the illustrated embodiment, this embodiment employs the parameter-efficient fine-tuning PEFT technique, which enables the updating of only a small portion of the model parameters during the fine-tuning process, thereby reducing the amount of training resources used.

[0123] Figure 2 The flowchart of step S1 provided in the embodiments of this application shows that the method may include the following steps:

[0124] Step S11: Obtain malicious instruction data.

[0125] Specifically, malicious instruction data can be extracted by analyzing malware samples, or potential malicious instruction data can be collected from the Internet, such as from hacker forums, the dark web, or social media platforms, using web crawling technology.

[0126] Step S13: Cluster the malicious instruction data using a preset instruction embedding model to obtain at least one of the attack task types.

[0127] Specifically, the instruction embedding model can be a model such as BERT or GPT, which transforms malicious instruction data into high-dimensional feature vectors. These high-dimensional feature vectors can capture the semantic information of the malicious instruction data. Clustering algorithms (such as K-means, hierarchical clustering, DBSCAN, etc.) are applied to the pre-defined instruction embedding model to cluster the high-dimensional feature vectors, thereby identifying different attack task types in the malicious instruction data.

[0128] Step S15: Randomly extract at least one malicious instruction data from each of the attack task types to construct the attack task dataset for each of the attack task types.

[0129] Specifically, for each identified attack task type, at least one malicious instruction data is randomly selected as a representative of that type. The randomly selected malicious instruction data are then aggregated to construct an attack task dataset for each attack task type.

[0130] and Figure 1 Compared to the illustrated embodiment, this embodiment collects malicious instruction data and uses an instruction embedding model to convert it into high-dimensional feature vectors to capture its semantic information. Subsequently, cluster analysis is performed on this data to effectively group malicious instruction data with similar semantic features, thereby identifying different attack task types. At least one malicious instruction data point is randomly selected from each attack task type to form a specialized attack task dataset, which helps in the detailed development of models for various attack scenarios. These datasets can be used to train and test the security of large language models, improving their ability to identify and defend against specific attack types.

[0131] Figure 3 The flowchart for generating a universal word-level adversarial suffix provided in this application embodiment may include the following steps:

[0132] Step S311: Determine the gradient of each token in the malicious instruction data.

[0133] Specifically, for each malicious instruction data x in the attack task dataset 1:m Initialize the adversarial suffix and calculate the gradient of each token after appending the initialized adversarial suffix:

[0134]

[0135] Where ⊕ represents splicing, and L adv For adversarial loss function (used to evaluate language model π) θ Output y and attack target sequence (differences between them), u j To initialize the adversarial suffix, This is the sequence of attack targets.

[0136] Step S313: Based on the gradient, use a greedy algorithm to determine the initial candidate token set corresponding to the attack task type.

[0137] Specifically, based on the gradient of each token, a greedy search algorithm is used to select the top-k pairs of the loss function L. adv The tokens with the greatest impact form the initial candidate token set.

[0138] Greedy Algorithm:

[0139]

[0140] Step S315: The initial candidate token set is filtered using the first-order gradient approximation method to obtain the target candidate token set corresponding to the attack task type.

[0141] Specifically, the first-order gradient approximation method is used to further filter the candidate set. To more accurately assess its L adv The impact of this is considered to obtain the final optimized target candidate token set, ensuring that the selected tokens can effectively improve the attack success rate.

[0142] Step S317: Construct a word-level adversarial suffix set based on the target candidate token set, perform attack evaluation on the word-level adversarial suffixes in the word-level adversarial suffix set, and determine the general word-level adversarial suffix corresponding to the attack task type based on the attack evaluation results.

[0143] Specifically, a set of adversarial suffixes constructed from the target candidate token set is used to change the semantics of the original malicious instruction data. The adversarial suffixes in the set are evaluated to determine their effect in the model. Based on the evaluation results, a general adversarial suffix that is universally applicable in each type of attack task is determined to change or interfere with the meaning of the target token or the output generated.

[0144] and Figure 1Compared to the illustrated embodiment, this embodiment first calculates the gradient of each token for each malicious instruction data. These gradients reflect the difference between the model output and the attack target sequence. Next, based on the gradient of each token, an initial candidate token set is selected. These token sets have the potential to maximally influence the model output to match the attacker's expected goals. Then, a first-order gradient approximation method is used to filter the initial candidate token set to determine the final optimized target candidate token set, thereby improving the attack success rate and making the model output more consistent with the attack target. Finally, a word-level adversarial suffix set is constructed based on the determined target candidate token set. The purpose is to modify the semantics of the original malicious instruction data, making it more likely to cause the model output to match the attacker's expectations. Each word-level adversarial suffix in the set is evaluated in detail to ensure its universality in various attack tasks, thus obtaining a universal word-level adversarial suffix.

[0145] Figure 4 The flowchart for step S317 provided in the embodiments of this application shows that the method may include the following steps:

[0146] Step S3171: Randomly select at least one token from the target candidate token set to construct the word-level adversarial suffix set.

[0147] Specifically, one or more tokens are randomly selected from the target candidate token set to form a word-level adversarial suffix set. Different random sampling will lead to different attack results, thereby helping to avoid the model's defense mechanism, because unpredictable attack methods may not be easily detected or countered by the model.

[0148] Step S3173: Perform attack evaluation on the word-level adversarial suffixes in the word-level adversarial suffix set.

[0149] Specifically, the word-level adversarial suffixes are iterated, and the current word-level adversarial suffixes in the word-level adversarial suffix set are... Adversarial samples are generated by attaching them to the corresponding malicious instruction data, and then input into the language model π. θ In the middle, obtain the response and record the language model π. θ Whether a response is considered a successful attack is evaluated using the Attack Success Rate (ASR), which is calculated as: ASR = Number of successful adversarial examples / Total number of examples in the validation set. The validation set is obtained by further partitioning the original attack task dataset. Evaluating model performance on the validation set helps avoid overfitting and allows for the selection of optimal model parameters.

[0150] Step S3175: If the success rate of attacking the word-level adversarial suffix in the current iteration is greater than the success rate of attacking the word-level adversarial suffix in the previous iteration, then the word-level adversarial suffix in the current iteration is determined as the current word-level adversarial suffix.

[0151] Specifically, during the iteration process, if the current iteration's adversarial suffix... The attack success rate on the validation set was higher than in the previous iteration. Then update the adversary suffix to the version of the current iteration: τ i The learning rate controls the step size of the iterative updates; this continues until the attack success rate no longer significantly improves or the maximum number of iterations is reached, ultimately yielding the general adversarial suffix u′ corresponding to the attack task type. j .

[0152] Step S3177: Until the preset conditions are met, the current word-level adversarial suffix is ​​determined as the general word-level adversarial suffix corresponding to the attack task type.

[0153] Specifically, the adversarial suffix is ​​iteratively updated until a stopping condition is met, such as reaching the maximum number of iterations or the attack success rate no longer significantly improving. Finally, a universal adversarial suffix that can generally improve the attack success rate is solved.

[0154] and Figure 3 Compared to the illustrated embodiment, this embodiment randomly selects one or more tokens from the target candidate token set to form a set of word-level adversarial suffixes. These word-level adversarial suffixes can change the form and content of malicious command data to attempt to circumvent the model's defense mechanisms. Each word-level adversarial suffix is ​​then appended to the malicious command data to form adversarial samples. These adversarial samples are input into the language model, the model's response is recorded, and the success of the attack is evaluated. If the attack success rate of the word-level adversarial suffix in the current iteration is higher than that of the previous iteration, the word-level adversarial suffix is ​​updated to the version of the current iteration. The word-level adversarial suffixes are iteratively updated until preset conditions are met, ultimately determining a universal adversarial suffix that can generally improve the attack success rate. This embodiment improves the attack success rate by continuously optimizing and selecting word-level adversarial suffixes, making attacks more difficult to detect and counter, thereby achieving the goal of bypassing language model defenses.

[0155] Figure 5 The flowchart for generating target word-level adversarial suffixes provided in this application embodiment may include the following steps:

[0156] Step S321: Based on the general word-level adversarial suffix, use a greedy algorithm to generate the target word-level adversarial suffix corresponding to the malicious instruction data.

[0157] Specifically, for each malicious instruction data in each attack sample dataset, a generic adversarial suffix u′ is used. j Based on this, a greedy search algorithm is used to calculate individual adversarial suffixes. The goal is to minimize the adversarial function;

[0158]

[0159] Among them, M j For all malicious instruction data sets, Given the probability distribution of model π with parameters θ,

[0160] Obtaining the corresponding target word-level adversarial suffixes can be done using the same method as steps S3173-3177. Each word-level adversarial suffix is ​​appended to the malicious instruction data to form adversarial samples, which are then input into the language model. The model's response to each adversarial sample is recorded, and the attack success rate (ASR) of each adversarial suffix is ​​evaluated. If the attack success rate of a certain word-level adversarial suffix in the current iteration is higher than that of the previous iteration, the word-level adversarial suffix is ​​updated to the version of the current iteration. A learning rate is used to control the update step size to gradually optimize the adversarial suffixes. The adversarial suffixes are iteratively updated until a stopping condition is met (e.g., reaching the maximum number of iterations or the attack success rate no longer significantly improving). Finally, the target word-level adversarial suffixes corresponding to each malicious instruction data are determined, i.e., the suffixes that can most effectively bypass the language model's defenses.

[0161] and Figure 1 Compared to the illustrated embodiment, this embodiment generates a customized target word-level adversarial suffix for each malicious instruction data, enabling the adversarial sample to more effectively deceive the target language model and improve the success rate of the attack.

[0162] Figure 6 A flowchart illustrating a method for obtaining target semantic-level adversarial hints provided in an embodiment of this application. This method may include the following steps:

[0163] Step S71: Determine the initial semantic-level adversarial cue corresponding to the attack target sequence.

[0164] Specifically, the attacker (strategy provider P) targets a malicious sequence of attack objectives. Design a strategy to guide attacking agent A to generate semantic-level adversarial hints about the target. Here It is the improper or malicious output that the attacking agent A hopes the model will generate.

[0165] Step S73: Use the initial semantic-level adversarial cue as input to the first fine-tuning model to generate response data.

[0166] Specifically, the first fine-tuned model receives initial semantic-level adversarial hints. Response data will then be generated.

[0167] Step S75: Determine the jailbreak assessment score corresponding to the response data.

[0168] Specifically, judge J evaluates a jailbreak assessment score based on the response data r. This score measures how close the model's output is to the target sequence. The judge's purpose here is to determine the similarity or difference between the generated response data and the target sequence based on historical data and model training. This can be an LLM model or a GPT model, trained on large-scale text data, to evaluate the semantic and logical coherence of the text in calculating the similarity between the generated response data r and the target sequence. Similarity score between them.

[0169] Step S77: If the jailbreak evaluation score is less than the preset jailbreak threshold, iteratively optimize the initial semantic-level adversarial prompt until the target semantic-level adversarial prompt is obtained when the jailbreak evaluation score is greater than or equal to the preset jailbreak threshold.

[0170] Specifically, the attacking agent A iteratively optimizes based on the feedback from the jailbreak evaluation score, adjusting the initial semantic-level adversarial cue used. In the next iteration, it generates new response data r and recalculates the jailbreak evaluation score. This process continues until the jailbreak evaluation score exceeds a preset jailbreak threshold, ensuring that the generated target semantic-level adversarial cue is as close as possible to the expected attack target sequence in both semantics and form, thereby improving the success rate and effectiveness of the attack.

[0171] and Figure 1 Compared to the illustrated embodiment, this embodiment first fine-tunes the initial semantic-level adversarial cue designed for the attack target sequence by inputting it into a first fine-tuning model. The first fine-tuning model generates corresponding output data, influenced by the initial semantic-level adversarial cue; this output data specifically reflects the model's response to the attack cue. Next, a jailbreak evaluation is performed based on the generated response data, and the initial semantic-level adversarial cue is adjusted according to the evaluation results until the jailbreak evaluation score of the generated response data reaches a preset jailbreak threshold. By utilizing semantic-level adversarial cue and iterative optimization, this method effectively increases the probability of a successful attack, enabling the generated text to be as close as possible to the intended malicious or misleading target.

[0172] To effectively evaluate the performance of the large language model security optimization method described in this embodiment, the following datasets will be used for testing. Widely recognized datasets, including AdvBench and MaliciousInstruct, were employed. AdvBench contains 520 malicious prompts specifically designed to elicit harmful responses, with 90% used for training and the remaining 10% for testing. To evaluate the broad defense capabilities of the method in this embodiment against unknown jailbreak attacks, all data from MaliciousInstruct was used. MaliciousInstruct contains 100 instances of malicious behavior spanning ten different categories.

[0173] The target models are two open-source models: Llama-2 (7B-chat-hf) and Vicuna (13B-v1.5).

[0174] To evaluate the effectiveness of various defense strategies, this test method compares the most powerful attack methods, including token-based attacks (such as GCG and AutoDAN) and cue-based attacks (such as PAIR I7, TAP, and GPTFuzzer). It also compares the methods with five of the most representative baseline methods, including system-level and model-level defense methods. System-level defense methods include Self-Reminder, SmoothLLM, and RPO. Model-level defense methods include adversarial training, de-learning, and security training.

[0175] For evaluation metrics, attack success rate (ASR) is used. A and ASR p To comprehensively assess the effectiveness of the defense. Among them, ASR p This indicates the degree of match between the response of an LLM and a predefined rejection, using keyword strings for matching, while ASR... A ASR is then evaluated using a GPT agent. A and ASR p The lower the value, the better the performance.

[0176] In the main experiments, Table 1 shows the defense results against known jailbreak attacks. The method in this embodiment consistently outperforms other state-of-the-art methods in defending against five attacks across two metrics. Specifically, under both target models, the average ASR caused by jailbreak attacks... A / ASR p The respective ASR rates are 66.54% / 51.38% and 19.62% / 9.62%. Existing defense methods perform poorly, while the method in this embodiment will reduce the average ASR under both target models. A / ASR pThe rates were reduced to 2.69% / 10.00% and 3.08% / 10.00%, respectively. The experimental results highlight the effectiveness of the proposed method in mitigating adversarial cues, significantly outperforming current methods. Furthermore, Self-Reminder outperforms Vicuna-13B on Llama-2-7B, possibly because Llama-2-7B employs a stronger safety alignment, allowing Self-Reminder to be triggered more effectively.

[0177] Robustness against unknown jailbreak attacks is a significant challenge due to the numerous potential clues and variations. Therefore, the MaliciousInstruct dataset was used to evaluate the effectiveness of the proposed method against unknown jailbreak attacks. Table 2 shows the defense results against unknown jailbreak attacks. The proposed method consistently outperforms other state-of-the-art methods in defending against five different attacks across two metrics. Specifically, under both target models, the average ASR caused by jailbreak attacks is significantly lower. A / ASR p The percentages were 71.60% / 52.20% and 18.40% / 8.20%, respectively. To counter these attacks, the method in this embodiment will implement ASR... A / ASR p The success rates were reduced to an average of 2.60% / 0.20% and 18.40% / 8.20%, respectively, demonstrating significant improvements across both target models and metrics. In summary, unknown jailbreaks exhibit higher attack success rates compared to known jailbreaks under other baseline methods, highlighting the importance of defending against unknown jailbreak attacks.

[0178] Table 1: Known Jailbreak Attacks

[0179]

[0180]

[0181] Table 2: Unknown jailbreak attacks

[0182]

[0183]

[0184] Accordingly, please refer to Figure 7 This application provides a large language model security optimization device, the terminal comprising:

[0185] The data acquisition module 101 is used to acquire an attack task dataset; wherein, the attack task dataset includes at least one attack task type and multiple malicious instruction data under the attack task type;

[0186] The word-level adversarial suffix generation module 103 is used to generate a general word-level adversarial suffix corresponding to the attack task type based on the malicious instruction data, and to generate a target word-level adversarial suffix corresponding to the malicious instruction data based on the general word-level adversarial suffix.

[0187] The first fine-tuning model generation module 105 is used to perform the first fine-tuning of the large language model based on the target word-level adversarial suffix to obtain the first fine-tuning model.

[0188] The second fine-tuning model generation module 107 is used to perform a second fine-tuning on the first fine-tuning model based on the obtained target semantic-level adversarial prompts, so as to obtain the second fine-tuning model.

[0189] In some optional implementations, the data acquisition module 101 includes:

[0190] The data acquisition unit is used to acquire malicious instruction data;

[0191] Clustering unit, used to cluster the malicious instruction data using a preset instruction embedding model to obtain at least one of the attack task types;

[0192] A data construction unit is used to randomly extract at least one malicious instruction data from each of the attack task types to construct the attack task dataset for each of the attack task types.

[0193] In some optional implementations, generating a generic word-level adversarial suffix corresponding to the attack task type based on the malicious instruction data includes:

[0194] Determine the gradient of each token in the malicious instruction data;

[0195] Based on the gradient, a greedy algorithm is used to determine the initial candidate token set corresponding to the attack task type;

[0196] The initial candidate token set is filtered using the first-order gradient approximation method to obtain the target candidate token set corresponding to the attack task type;

[0197] A set of adversarial suffixes is constructed based on the target candidate token set, and an attack evaluation is performed on the adversarial suffixes in the set. Based on the attack evaluation results, the general adversarial suffixes corresponding to the attack task type are determined.

[0198] In some optional implementations, the step of constructing a word-level adversarial suffix set based on the target candidate token set, performing attack evaluation on the word-level adversarial suffixes in the word-level adversarial suffix set, and determining the general word-level adversarial suffix corresponding to the attack task type based on the attack evaluation results includes:

[0199] At least one token is randomly selected from the target candidate token set to construct the word-level adversarial suffix set;

[0200] Attack evaluation is performed on the word-level adversarial suffixes in the aforementioned word-level adversarial suffix set;

[0201] If the success rate of attacking the word-level adversarial suffix in the current iteration is greater than the success rate of attacking the word-level adversarial suffix in the previous iteration, then the word-level adversarial suffix in the current iteration is determined as the current word-level adversarial suffix.

[0202] Until the preset conditions are met, the current word-level adversarial suffix is ​​determined as the general word-level adversarial suffix corresponding to the attack task type.

[0203] In some optional implementations, generating the target word-level adversarial suffix corresponding to the malicious instruction data based on the general word-level adversarial suffix includes:

[0204] Based on the general word-level adversarial suffix, a greedy algorithm is used to generate the target word-level adversarial suffix corresponding to the malicious instruction data.

[0205] In some optional implementations, the methods for obtaining the target semantic-level adversarial prompts include:

[0206] Determine the initial semantic-level adversarial cue corresponding to the attack target sequence;

[0207] The initial semantic-level adversarial cue is used as input to the first fine-tuning model to generate response data;

[0208] Determine the jailbreak assessment score corresponding to the response data;

[0209] If the jailbreak evaluation score is less than a preset jailbreak threshold, the initial semantic-level adversarial prompt is iteratively optimized until the target semantic-level adversarial prompt is obtained when the jailbreak evaluation score is greater than or equal to the preset jailbreak threshold.

[0210] In some alternative implementations, the first fine-tuning and / or the second fine-tuning employs the Parametric Efficient Fine-Tuning (PEFT) technique.

[0211] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0212] In this embodiment, the large language model security optimization device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0213] Please see Figure 8 , Figure 8 This application provides a schematic diagram of the structure of a computer device, as shown in the embodiment of the present application. Figure 8 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0214] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0215] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0216] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0217] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0218] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0219] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0220] The apparatus, module, or unit described in the above embodiments can be implemented by a computer chip or entity, or by a product with a certain function. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0221] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0222] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or devices. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0223] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and devices according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0224] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0225] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0226] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0227] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device semantic embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0228] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

[0229] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A large language model security optimization method, characterized in that, The method comprises: obtaining an attack task data set; wherein the attack task data set comprises at least one attack task type and a plurality of malicious instruction data under the attack task type; determining the gradient of each token in the malicious instruction data; determining an initial candidate token set corresponding to the attack task type using a greedy algorithm according to the gradient; screening the initial candidate token set using a first-order gradient approximation method to obtain a target candidate token set corresponding to the attack task type; randomly extracting at least one token from the target candidate token set to construct a word-level adversarial suffix set; performing attack evaluation on the word-level adversarial suffixes in the word-level adversarial suffix set; if the attack success rate of the word-level adversarial suffix of the current iteration is greater than the attack success rate of the word-level adversarial suffix of the last iteration, determining the word-level adversarial suffix of the current iteration as the current word-level adversarial suffix; until a preset condition is met, determining the current word-level adversarial suffix as the universal word-level adversarial suffix corresponding to the attack task type; generating a target word-level adversarial suffix corresponding to the malicious instruction data using a greedy algorithm according to the universal word-level adversarial suffix; performing first fine-tuning on the large language model based on the target word-level adversarial suffix to obtain a first fine-tuned model; performing second fine-tuning on the first fine-tuned model based on the obtained target semantic-level adversarial prompt to obtain a second fine-tuned model; wherein the target semantic-level adversarial prompt is obtained by: determining an initial semantic-level adversarial prompt corresponding to an attack target sequence; inputting the initial semantic-level adversarial prompt into the first fine-tuned model to generate response data; determining a jailbreak evaluation score corresponding to the response data; and in the case that the jailbreak evaluation score is less than a preset jailbreak threshold, iteratively optimizing the initial semantic-level adversarial prompt until the jailbreak evaluation score is greater than or equal to the preset jailbreak threshold, thereby obtaining the target semantic-level adversarial prompt.

2. The method of claim 1, wherein, The method comprises: obtaining an attack task data set; wherein the attack task data set comprises at least one attack task type and a plurality of malicious instruction data under the attack task type, obtaining malicious instruction data; clustering the malicious instruction data using a preset instruction embedding model to obtain at least one attack task type; 3. The method of claim 1, wherein, randomly extracting at least one malicious instruction data from each attack task type to construct the attack task data set for each attack task type.

4. A device for implementing the large language model security optimization method of any one of claims 1-3, characterized in that, The first fine-tuning and / or the second fine-tuning uses a parameter-efficient fine-tuning (PEFT) technique. The device comprises: a data acquisition module configured to obtain an attack task data set; wherein the attack task data set comprises at least one attack task type and a plurality of malicious instruction data under the attack task type; a word-level adversarial suffix generation module configured to generate a universal word-level adversarial suffix corresponding to the attack task type based on the malicious instruction data, and generate a target word-level adversarial suffix corresponding to the malicious instruction data based on the universal word-level adversarial suffix; The first fine-tuning model generation module is configured to perform first fine-tuning on the large language model based on the target word-level adversarial suffix, to obtain a first fine-tuning model. The second fine-tuning model generation module is configured to perform second fine-tuning on the first fine-tuning model based on the obtained target semantic-level adversarial prompt, to obtain a second fine-tuning model.

5. A computer device, comprising: Comprise: A memory and a processor, which are in communication connection with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the large language model security optimization method in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, and the computer instructions are used to make the computer execute the large language model security optimization method in any one of claims 1 to 3. The computer readable storage medium stores computer instructions, and the computer instructions are used to make the computer execute the large language model security optimization method in any one of claims 1 to 3.

Images