A low-complexity safety approach to enhance robustness of large models

Abstract

The present invention relates to the technical field of electronic digital data processing, and in particular to a low-complexity security method for enhancing the robustness of a large model, wherein the method comprises: selecting multiple types of test tasks, respectively using each type of test task to perform robustness tests on a target large language model, and obtaining robustness test results of the target large language model for each test task; based on the robustness test results, generating corresponding adversarial samples; while using the adversarial samples to perform adversarial training on the target large language model, optimizing the target large language model using a preset model optimization expression, so as to construct a security mechanism based on gradient-generated attack samples, and using the security mechanism to train the large language model, and obtaining a final large language model that meets the preset robustness conditions. Thus, the technical problem in the related art that it is difficult to ensure the robustness of the large language model while ensuring the decision accuracy of the large language model is solved.

Description

Technical Field

[0001] The invention relates to the technical field of electronic digital data processing, and in particular to a low-complexity safety method for enhancing the robustness of a large model. Background Art

[0002] With the rapid development of large language models, the security risks of their own algorithms have become increasingly exposed. There are security risks such as being deceived by adversarial samples and privacy leakage, which hinder their further integration with 6G and their implementation in more application areas.

[0003] In early research, the study of potential attacks on AI (Artificial Intelligence) algorithms and corresponding defense methods mainly focused on the attack success rate of the model and whether it can successfully circumvent a certain attack method. Taking the traditional classification model as an example, the higher the judgment accuracy, the lower the model robustness. Recently, scholars have focused on how to use the characteristics of adversarial samples in different fields to better complete tasks.

[0004] In the fields of statistics and machine learning, the property of being able to maintain stability and reliability even against outliers is called robustness. AI technologies represented by GPT (Generative Pre-Trained, a deep learning model based on the Internet, trained with available data, and generated by text) have powerful data fitting capabilities that hide the unexplainable phenomenon of "shortcut learning", which leads to security risks that deep models are fragile and vulnerable to attacks. Numerous studies have shown that if attackers add tiny perturbations that are imperceptible to humans to normal data, the model may produce catastrophic erroneous outputs.

[0005] Among them, prompt words are the bridge for the large language model to communicate with the user, and prompt words often directly affect the decision of the large language model. The large language model is extremely sensitive to prompt words, and the decision-making process of the large model can be affected by carefully designed prompt words. In addition, attackers can guide the large model to make wrong decisions by embedding specific prompt words in the input. This attack method is highly concealed because the design of prompt words can be very natural, making the attack difficult to detect. On the other hand, although adversarial training has a certain ability to defend against attacks, its improvement in the robustness of the large language model comes at the cost of reducing the classification or recognition accuracy of normal data.

[0006] However, in related technologies, adversarial defense methods have the problem of overfitting, and practice shows that the more robust the model is, the lower its classification or recognition accuracy for normal samples. Considering that large language models often need to process massive amounts of data, and attackers will launch adversarial attacks based on different data sets and decision tasks, this inconsistency has brought obstacles to evaluating adversarial attacks and measuring the robustness of large language models. In addition, noise interference will also greatly increase the computational complexity of training, which needs to be improved. Summary of the invention

[0007] The present invention provides a low-complexity security method for enhancing the robustness of a large model, so as to solve the technical problem in the related art that it is difficult to ensure the robustness of a large language model while ensuring the decision accuracy of the large language model.

[0008] The first aspect of the present invention provides a low-complexity security method for enhancing the robustness of a large model, comprising the following steps: selecting multiple types of test tasks, and using each type of test task to perform robustness tests on a target large language model to obtain robustness test results of the target large language model for each test task; based on the robustness test results, generating corresponding adversarial samples; while using the adversarial samples to perform adversarial training on the target large language model, optimizing the target large language model using a preset model optimization expression to construct a security mechanism based on gradient-based attack sample generation, and using the security mechanism to train the large language model to obtain a final large language model that meets preset robustness conditions.

[0009] Optionally, in one embodiment of the present invention, the test task includes at least one of a sentiment analysis task, a grammatical error recognition task, a repeated sentence detection task, a natural language reasoning task, a multi-task knowledge task, a reading comprehension task, a translation task and a mathematics task.

[0010] Optionally, in one embodiment of the present invention, the corresponding adversarial sample is generated based on the robustness test result, including: obtaining a target text, and perturbing the target text based on the robustness test result to obtain an iterative sample; calculating a gradient according to an attack mode, and perturbing the iterative sample in the direction of increasing gradient to obtain a decision error of this disturbance, and judging whether this disturbance achieves a preset attack purpose based on the decision error; if this disturbance does not achieve the preset attack purpose, recalculating the gradient direction based on this disturbance, and performing the next disturbance based on the result of this disturbance, until the preset attack purpose is achieved, and outputting the adversarial sample.

[0011] Optionally, in one embodiment of the present invention, the using the adversarial sample to perform adversarial training on the target large language model includes: inputting the adversarial sample as a training sample into the target large language model, and updating the model parameters of the target large language model according to the output result, so as to obtain a trained target large language model based on the model parameters.

[0012] Optionally, in one embodiment of the present invention, the calculation expression of the model parameter is:

[0013] =

[0014] in, For expectations, For distribution The data and label pairs sampled in To minimize the average loss function, are the model parameters, is the target text, For disturbance, is the true label.

[0015] Optionally, in one embodiment of the present invention, the optimization of the target large language model using a preset model optimization expression includes: defining the task of the target large language model as symbol sequence prediction in a continuous situation, wherein the symbol sequence prediction is the probability of predicting the next symbol sequence of a preset symbol sequence mapping as a target sequence; based on the probability, obtaining an expected decision answer of the adversarial sample under a preset prompt; combining the expected decision answer, the factual answer of the adversarial sample and the probability, converting the decision task into a negative logarithmic probability, obtaining the preset model optimization expression, and optimizing the target large language model using the preset model optimization expression; wherein the preset model optimization expression is:

[0016]

[0017] in, is the measurement indicator of the decision-making task, To obtain the expected probability of the expected decision answer, is the expected decision answer, is the preset symbol sequence.

[0018] The second aspect of the present invention provides a low-complexity security device for enhancing the robustness of a large model, including: a testing module for selecting multiple types of test tasks, and using each type of test task to perform robustness tests on a target large language model, to obtain a robustness test result of the target large language model for each test task; a generation module for generating corresponding adversarial samples based on the robustness test results; a training module for optimizing the target large language model using a preset model optimization expression while using the adversarial samples to perform adversarial training on the target large language model, so as to construct a security mechanism based on gradient-based attack sample generation, and use the security mechanism to train the large language model to obtain a final large language model that meets the preset robustness conditions.

[0019] Optionally, in one embodiment of the present invention, the test task includes at least one of a sentiment analysis task, a grammatical error recognition task, a repeated sentence detection task, a natural language reasoning task, a multi-task knowledge task, a reading comprehension task, a translation task and a mathematics task.

[0020] Optionally, in one embodiment of the present invention, the generation module includes: an acquisition unit, used to acquire a target text, and perturb the target text based on the robustness test result to obtain an iterative sample; an output unit, used to calculate a gradient according to an attack mode, and perturb the iterative sample in the direction of increasing gradient to obtain a decision error of this perturbation, and judge whether this perturbation achieves a preset attack purpose based on the decision error; if this perturbation does not achieve the preset attack purpose, recalculate the gradient direction based on this perturbation, and perform the next perturbation based on the result of this perturbation until the preset attack purpose is achieved, and output the adversarial sample.

[0021] Optionally, in one embodiment of the present invention, the training module includes: a training unit, used to input the adversarial sample as a training sample into the target large language model, and update the model parameters of the target large language model according to the output result, so as to obtain a trained target large language model based on the model parameters.

[0022] Optionally, in one embodiment of the present invention, the calculation expression of the model parameter is:

[0023] =

[0024] in, For expectations, For distribution The data and label pairs sampled in To minimize the average loss function, are the model parameters, is the target text, For disturbance, is the true label.

[0025] Optionally, in one embodiment of the present invention, the training module includes: a prediction unit, which is used to define the task of the target large language model as a symbol sequence prediction in a continuous situation, wherein the symbol sequence prediction is to predict the probability that the next symbol sequence of the preset symbol sequence mapping is the target sequence; a calculation unit, which is used to obtain the expected decision answer of the adversarial sample under a preset prompt based on the probability; an optimization unit, which is used to combine the expected decision answer, the factual answer of the adversarial sample and the probability, convert the decision task into a negative logarithmic probability, obtain the preset model optimization expression, and use the preset model optimization expression to optimize the target large language model; wherein the preset model optimization expression is:

[0026]

[0027] in, is the measurement indicator of the decision-making task, To obtain the expected probability of the expected decision answer, is the expected decision answer, is the preset symbol sequence.

[0028] A third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a low-complexity security method for enhancing the robustness of large models as described in the above embodiment.

[0029] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute a low-complexity security method for enhancing the robustness of large models as described in the above embodiment.

[0030] A fifth aspect of the present invention provides a computer program product, including a computer program, which, when executed, is used to implement a low-complexity security method for enhancing the robustness of large models as described above.

[0031] The embodiment of the present invention can use different types of test tasks to perform robustness tests on the target large language model, thereby generating corresponding adversarial samples according to the robustness test results, and optimizing the target large language model while using the adversarial samples to perform adversarial training on the target large language model, so as to construct a security mechanism based on gradient-based generation of attack samples, and use the security mechanism to train the large language model to obtain a final large language model that meets the preset robustness conditions, reduce the number of update steps of the adversarial samples, and reduce the number of iteration steps required to generate adversarial samples, thereby ensuring the decision accuracy of the large language model while ensuring the robustness of the large language model. Therefore, the technical problem in the related art that it is difficult to ensure the robustness of the large language model while ensuring the decision accuracy of the large language model is solved.

[0032] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0034] Figure 1 A flowchart of a low-complexity security method for enhancing the robustness of a large model provided according to an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of a black box test analysis result according to an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of the principle of adversarial training according to an embodiment of the present invention;

[0037] Figure 4 A schematic diagram of a low-complexity security method for enhancing the robustness of a large model according to an embodiment of the present invention;

[0038] Figure 5 A schematic diagram of the structure of a low-complexity safety device for enhancing the robustness of a large model provided according to an embodiment of the present invention;

[0039] Figure 6 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0041] A low-complexity security method for enhancing the robustness of a large model according to an embodiment of the present invention is described below with reference to the accompanying drawings. In view of the technical problem that it is difficult to ensure the robustness of a large language model while ensuring the decision accuracy of a large language model in the related art mentioned in the above background technology, the present invention provides a low-complexity security method for enhancing the robustness of a large model, in which different types of test tasks can be used to perform robustness tests on a target large language model, thereby generating corresponding adversarial samples according to the robustness test results, and while using adversarial samples to perform adversarial training on the target large language model, the target large language model is optimized to construct a security mechanism based on gradient-based generation of attack samples, and the large language model is trained using the security mechanism to obtain a final large language model that meets the preset robustness conditions, reduce the number of update steps of the adversarial sample, and reduce the number of iteration steps required to generate the adversarial sample, thereby ensuring the robustness of the large language model while ensuring the decision accuracy of the large language model. Thus, the technical problem that it is difficult to ensure the robustness of the large language model while ensuring the decision accuracy of the large language model in the related art is solved.

[0042] Specifically, Figure 1 A schematic flow chart of a low-complexity security method for enhancing the robustness of a large model provided by an embodiment of the present invention.

[0043] like Figure 1 As shown, the low-complexity safety method for enhancing the robustness of large models includes the following steps:

[0044] In step S101, multiple types of test tasks are selected, and each type of test task is used to perform robustness test on the target large language model to obtain the robustness test result of the target large language model for each test task. The test task includes at least one of a sentiment analysis task, a grammatical error recognition task, a repeated sentence detection task, a natural language reasoning task, a multi-task knowledge task, a reading comprehension task, a translation task, and a mathematics task.

[0045] It is understandable that if Figure 2 As shown in Figure 2, attacks against large language models can cover four different levels, ranging from simple character manipulation to complex semantic modification:

[0046] (1) Character level: Changing the text by adding misspelled letters to words.

[0047] (2) Word level: Try to replace the original word with a synonym or a word with similar context to deceive the large language model.

[0048] (3) Sentence level: By adding irrelevant or redundant sentences at the end of the prompt word, trying to distract the large language model.

[0049] (4) Semantic level: To simulate the language behavior of people from different countries, we selected six common languages ​​(Chinese, French, Arabic, Spanish, Japanese, and Korean) and constructed ten prompts for each language. These prompts were then translated into English, introducing linguistic nuances that may affect large language models.

[0050] The embodiment of the present invention can select eight different tasks, including: sentiment analysis, grammatical error recognition, repeated sentence detection, natural language reasoning, multi-task knowledge, reading comprehension, translation and mathematics, to perform black box attack tests on large language models.

[0051] In the actual execution process, the decision accuracy rate changes significantly when executing different tasks, especially in mathematical and grammatical error recognition. Secondly, word-level attacks reduce the average accuracy of large language models in mathematical tasks by 48%. In contrast, sentence-level attacks only reach 2% in reading comprehension tasks, which is a relatively small threat. This shows that slight changes in prompt words can have a significant impact on the accuracy of model output, and the effectiveness of different types of attacks on different tasks varies greatly. Among them, the effectiveness of word-level attacks on all tasks is more obvious. Finally, the difference between word-level attacks on mathematical tasks and translation attacks is 31%. It can be said that the same type of attack method has a large difference in the attack results on different tasks. Therefore, corresponding anti-attack methods should be adopted when dealing with different tasks.

[0052] In step S102, a corresponding adversarial sample is generated based on the robustness test result.

[0053] In order to avoid retrieving and cleaning data from massive texts, the embodiments of the present invention can use the large language model search engine function to induce the generation of targeted adversarial samples based on the robustness test results, thereby reducing the workload in the training process.

[0054] Among them, adversarial examples refer to samples formed by artificially adding subtle disturbances that are invisible to the naked eye or visible to the naked eye that do not affect the overall data set after processing. Such samples will cause the trained model to give classification outputs different from the original samples with high confidence. Adversarial attacks often refer to attackers using specific model algorithms to attack large language models. It not only includes attackers who are familiar with the large language model and modify the actual input to mislead the discrimination results of the large language model, but also includes attackers who attack the large language model by taking advantage of the characteristics of algorithm migration between machine learning models without understanding the structure and parameters of the large language model.

[0055] For example, the induction process may include the following:

[0056] 1) Initial affirmative responses

[0057] By giving a series of offensive queries and guiding the target model to start the response with "Sure, here is (content of query)". Once the start of the large language model response is guided to a certain pattern, the subsequent output will be affected by the prior of the previous output, thus entering another conditional probability prediction branch, which may just avoid the training fine-tuning scope of instruction alignment.

[0058] In general, for a simple jailbreak attack that is not aggressive enough, such as a straightforward question like "how to make a bomb?", the language model will generally start the answer with "sorry, ....", indicating that it is unable to answer this question.

[0059] Since the underlying logic of LLM is to predict the next word based on the previous content, if we try to make the model start with "Sure" or "Of course" instead of the rejection "sorry", it will be difficult for the large language model to perform probabilistic mutations. It is very difficult for a large language model to correct the subsequent content after starting with affirmation.

[0060] In addition, most of the prompts designed by artificial jailbreaks are relatively fixed. For a specific scenario and a specific prompt, the model can be induced to respond to some undesirable behavior. There are also some more general attacks, but they are relatively harder to find and fewer. Optimize on multiple queries. If an adversarial suffix can meet the needs of several queries at the same time and optimize on the response gradients of several queries, then it is more likely to be generalizable.

[0061] 2) Combined greedy and gradient-based discrete optimization

[0062] By optimizing discrete tokens to maximize the log-probability of a successful attack: Gradient optimization is used at the token level to identify a set of "single-token replacements". During training, the best "single-token replacements" are selected by evaluating the loss function of some candidate "single-token replacements" in the set.

[0063] 3) Robust multi-prompt and multi-model attacks

[0064] In order to generate reliable attack samples, the attack samples should not only be applicable to a single model or a single prompt, but also to multiple models and multiple prompts.

[0065] 4) Directional conditional probability guidance

[0066] Based on previous work on robustness testing of large language models, the adversarial examples used need to be more targeted and exist in a similar form to the training corpus.

[0067] Optionally, in one embodiment of the present invention, a corresponding adversarial sample is generated based on the robustness test result, including: obtaining a target text, and perturbing the target text based on the robustness test result to obtain an iterative sample; calculating a gradient according to an attack mode, and perturbing the iterative sample in the direction of increasing the gradient to obtain a decision error of this perturbation, and judging whether this perturbation achieves a preset attack purpose based on the decision error; if this perturbation does not achieve the preset attack purpose, recalculating the gradient direction based on this perturbation, and performing the next perturbation based on the result of this perturbation until the preset attack purpose is achieved, and outputting the adversarial sample.

[0068] It is understandable that when an attacker x Apply slight disturbance , it can cause significant changes in feature expression, leading to incorrect classification or detection. This kind of sample generated by artificially adding perturbations is called an adversarial sample. , which is related to the disturbance The relationship can be simply described as

[0069]

[0070]

[0071] in, is the magnitude of the attack or disturbance, represent norm, Desirable .

[0072] For example, you can Figure 3 As shown, the embodiment of the present invention may include the following steps:

[0073] Step S1: Select the target model and target sample, that is, the target large language model and the target sample to be deceived.

[0074] Step S2: Calculate the gradient. In the related art, the gradient-based adversarial sample attack algorithms include the following: Fast Gradient Sign Method (FGSM), which applies a single-step perturbation to the sample, and the direction of the perturbation is opposite to the gradient of the loss function. BIM (Basic Iterative Method), also known as iterative FGSM (I-FGSM), is a multi-step version of FGSM. Each iteration uses the same method as FGSM to calculate the perturbation direction and applies a smaller perturbation size. PGD (Projected Gradient Descent) attack is similar to BIM, but the perturbation is initialized from a random point in the norm ball near the test image, and it is proposed to project the gradient instead of clipping the gradient.

[0075] In order to avoid the above problems, when calculating the gradient, the embodiment of the present invention may perform corresponding calculations according to different attack types.

[0076] In a directed attack, the decision error is calculated by comparing the output of the forward propagation with the attack target, and the goal of the perturbation is to reduce the error. It is calculated in the opposite direction of the gradient.

[0077] In non-directional attacks, the error is calculated by comparing the output of the forward propagation with the actual label. The goal of the perturbation is to make the error larger. Therefore, the perturbation is calculated along the true direction of the gradient, that is, it is optimized according to the so-called gradient ascent method.

[0078] Step S3: Add perturbations, add some small perturbations to the target sample according to the robustness test results. The purpose of these perturbations is to cause the target model to produce incorrect classification results.

[0079] Step S4: Iterative optimization: Through the iterative optimization algorithm, the added disturbance is adjusted so that the target model produces incorrect output on the adversarial sample.

[0080] Step S5: Verify the adversarial sample to verify whether the generated adversarial sample can successfully deceive the target model. This can be done by inputting the adversarial sample into the target model and observing the output results. The verification result determines whether the adversarial sample can be used (if not, loop steps S2-S5)

[0081] In this embodiment of the present invention, a multi-step perturbation can be performed in the direction of increasing gradient in an iterative manner, and the gradient direction is recalculated after each small step. The iterative process is as follows:

[0082] { }

[0083] for ,

[0084] in, { } Constrain each input feature of the coordinates to limit it to the input In the perturbation neighborhood of and feasible input space, is the total number of iterations, is the step length.

[0085]

[0086] in, To measure the model prediction results and the true label The loss function of consistency between is the L2 norm, that is, regularization is performed. is the step length.

[0087] In step S103, while adversarial training is performed on the target large language model using adversarial samples, the target large language model is optimized using a preset model optimization expression to construct a security mechanism based on gradient-generated attack samples, and the large language model is trained using the security mechanism to obtain a final large language model that meets preset robustness conditions.

[0088] Furthermore, the embodiments of the present invention can use adversarial samples to perform adversarial training on the target large language model, and can also optimize the target large language model, measure the model's factual answers and their corresponding adversarial wrong answers, and provide direction for adversarial sample generation in model optimization. After black-box testing, adversarial samples of different types and tasks can be generated more specifically, and the model training optimization effect is better.

[0089] That is to say, the embodiments of the present application can obtain a security mechanism based on gradient-generated attack samples through the above steps, and then use the security mechanism to train the large language model, thereby improving the robustness of the trained large language model, wherein the preset robustness conditions are different according to the application field or application requirements of the large language model, and can be set accordingly by technical personnel in this field according to actual conditions, and no specific restrictions are made here.

[0090] Optionally, in one embodiment of the present invention, adversarial training is performed on a target large language model using adversarial samples, including: inputting the adversarial samples as training samples into the target large language model, and updating the model parameters of the target large language model according to the output results, so as to obtain a trained target large language model based on the model parameters. The calculation expression of the model parameters is:

[0091] =

[0092] in, For expectations, For distribution The data and label pairs sampled in To minimize the average loss function, are model parameters, is the target text, For disturbance, is the true label.

[0093] In the actual implementation process, the embodiment of the present invention can As training samples, input the target large language model and train it. Then the model parameters By minimizing the average loss function To update,

[0094] =

[0095] in, Indicates that from the distribution The data and label pairs sampled in Express expectations.

[0096] Optionally, in one embodiment of the present invention, the target large language model is optimized using a preset model optimization expression, including: defining the task of the target large language model as symbol sequence prediction in a continuous situation, wherein the symbol sequence prediction is the probability of predicting the next symbol sequence of the preset symbol sequence mapping as the target sequence; based on the probability, obtaining the expected decision answer of the adversarial sample under the preset prompt; combining the expected decision answer, the factual answer and probability of the adversarial sample, converting the decision task into a negative logarithmic probability, obtaining the preset model optimization expression, and optimizing the target large language model using the preset model optimization expression. Among them, the preset model optimization expression is:

[0097]

[0098] in, It is the measurement index of the decision-making task, that is, the index used to measure whether the decision-making task can be completed accurately. To obtain the expected probability of the decision answer, To expect a decision answer, is a preset symbol sequence.

[0099] Furthermore, the embodiment of the present invention can use the natural saddle point (maximum-minimum) formula to simultaneously incorporate adversarial attack and defense training into adversarial training from the perspective of robust optimization, and construct a basic framework for adversarial training, specifically:

[0100]

[0101] When a large language model processes data, it generates aggressive adversarial samples by maximizing the inner layer loss. On the other hand, it uses the outer layer loss minimization to update the model's sensitivity to prompt words, thereby enhancing its anti-attack capabilities.

[0102] The embodiment of the present invention can define the task of the target large language model as an existing token sequence ( , The prediction is based on the input vocabulary size (i.e. the number of tokens) mapped to the sequence of "next token"

[0103]

[0104] For any , the above formula represents the given previous token sequence The next token is probability.

[0105] On this basis, the model is based on a length The output of the generated length is The corresponding formula can be expressed as:

[0106]

[0107] in, Represents a given token sequence , generate the total sequence of all single tokens Then, given the prompt, Represents the desired decision answer. Converting the decision task into the negative logarithmic probability of the minimum loss function, the optimization problem of the adversarial training model can be expressed as

[0108] .

[0109] In summary, the embodiment of the present invention can use a black-box attack algorithm to perform robustness analysis on the target large language model to determine the original data type being attacked. Induce the target large language model to obtain adversarial samples, and use the processed adversarial samples to input into the target large language model for decision-making to obtain a decision error. Then, the target large language model is subjected to adversarial training to obtain the decision error and test the robustness of the adversarial sample, and the training steps are repeated. Among them, the direction in which the loss function increases is defined as the optimal direction of the perturbation update by using the gradient of the loss function. At the same time, considering the limitation of the perturbation amplitude, the large language model has a highly nonlinear loss surface near the adversarial sample generated by this type of method. In order to reduce the number of update steps of the adversarial sample, the regularization term is introduced into the objective function, aiming to reduce the number of iterations required to generate adversarial samples by improving local linearity.

[0110] Combination Figures 2 to 4 As shown, the working principle of a low-complexity security method for enhancing the robustness of a large model according to an embodiment of the present invention is described in detail with an embodiment.

[0111] like Figure 4 As shown, a low-complexity security method for enhancing the robustness of a large model according to an embodiment of the present invention may include three parts:

[0112] 1. Use black-box attack algorithms to analyze the robustness of large language models.

[0113] like Figure 2 As shown in Figure 2, attacks against large language models can cover four different levels, ranging from simple character manipulation to complex semantic modification:

[0114] (1) Character level: Changing the text by adding misspelled letters to words.

[0115] (2) Word level: Try to replace the original word with a synonym or a word with similar context to deceive the large language model.

[0116] (3) Sentence level: By adding irrelevant or redundant sentences at the end of the prompt word, trying to distract the large language model.

[0117] (4) Semantic level: To simulate the language behavior of people from different countries, we selected six common languages ​​(Chinese, French, Arabic, Spanish, Japanese, and Korean) and constructed ten prompts for each language. These prompts were then translated into English, introducing linguistic nuances that may affect large language models.

[0118] The embodiment of the present invention can select eight different tasks, including: sentiment analysis, grammatical error recognition, repeated sentence detection, natural language reasoning, multi-task knowledge, reading comprehension, translation and mathematics, to perform black box attack tests on large language models.

[0119] 2. Based on the results of robustness analysis, induce the generation of adversarial samples to conduct adversarial training of the model.

[0120] When the attacker x Apply slight disturbance , it can cause significant changes in feature expression, leading to incorrect classification or detection. This kind of sample generated by artificially adding perturbations is called an adversarial sample. , which is related to the disturbance The relationship can be simply described as

[0121]

[0122]

[0123] in, is the magnitude of the attack or disturbance, represent norm, Desirable .

[0124] For example, you can Figure 3 As shown, the embodiment of the present invention may include the following steps:

[0125] Step S1: Select the target model and target sample, that is, the target large language model and the target sample to be deceived.

[0126] Step S2: Calculate the gradient. In a directed attack, the decision error is calculated by comparing the output of the forward propagation with the attack target. The goal of the perturbation is to reduce the error. It is calculated in the opposite direction of the gradient.

[0127] In non-directional attacks, the error is calculated by comparing the output of the forward propagation with the actual label. The goal of the perturbation is to make the error larger. Therefore, the perturbation is calculated along the true direction of the gradient, that is, it is optimized according to the so-called gradient ascent method.

[0128] Step S3: Add perturbations, add some small perturbations to the target sample according to the robustness test results. The purpose of these perturbations is to cause the target model to produce incorrect classification results.

[0129] Step S4: Iterative optimization: Through the iterative optimization algorithm, the added disturbance is adjusted so that the target model produces incorrect output on the adversarial sample.

[0130] Step S5: Verify the adversarial sample to verify whether the generated adversarial sample can successfully deceive the target model. This can be done by inputting the adversarial sample into the target model and observing the output results. The verification result determines whether the adversarial sample can be used (if not, loop steps S2-S5)

[0131] In this embodiment of the present invention, a multi-step perturbation can be performed in the direction of increasing gradient in an iterative manner, and the gradient direction is recalculated after each small step. The iterative process is as follows:

[0132] { }

[0133] for ,

[0134] in, { } Constrain each input feature of the coordinates to limit it to the input In the perturbation neighborhood of and feasible input space, is the total number of iterations, is the step length.

[0135]

[0136] in, To measure the model prediction results and the true label The loss function of consistency between is the L2 norm, that is, regularization is performed. is the step length.

[0137] The embodiment of the present invention can As training samples, input the target large language model and train it. Then the model parameters By minimizing the average loss function To update,

[0138] =

[0139] in, Indicates that from the distribution The data and label pairs sampled in Express expectations.

[0140] 3. Based on the results of robustness analysis, the model optimization problem is transformed to achieve model optimization.

[0141] The embodiment of the present invention can use the natural saddle point (maximum-minimum) formula from the perspective of robust optimization to simultaneously incorporate adversarial attack and defense training into adversarial training, and construct a basic framework for adversarial training. Specifically,

[0142]

[0143] When a large language model processes data, it generates aggressive adversarial samples by maximizing the inner layer loss. On the other hand, it uses the outer layer loss minimization to update the model's sensitivity to prompt words, thereby enhancing its anti-attack capabilities.

[0144] The embodiment of the present invention can define the task of the target large language model as an existing token sequence ( , The prediction is based on the input vocabulary size (i.e. the number of tokens) mapped to the sequence of "next token"

[0145]

[0146] For any , the above formula represents the given token sequence The next token is probability.

[0147] On this basis, the model is based on a length The output of the generated length is The corresponding formula can be expressed as:

[0148]

[0149] in, Represents a given token sequence , generate the total sequence of all single tokens Then, given the prompt, Represents the desired decision answer. Converting the decision task into the negative logarithmic probability of the minimum loss function, the optimization problem of the adversarial training model can be expressed as

[0150] .

[0151] like Figure 4 As shown, the embodiment of the present invention can use the natural saddle point (maximum-minimum) formula to provide a judgment criterion for whether the adversarial training result is successful based on the generated adversarial samples, and refer to the robustness test results to enhance the sensitivity of the model to the prompt words based on the adversarial training.

[0152] According to a low-complexity security method for enhancing the robustness of a large model proposed in an embodiment of the present invention, different types of test tasks can be used to perform robustness tests on a target large language model, thereby generating corresponding adversarial samples based on the robustness test results, and optimizing the target large language model while using the adversarial samples to perform adversarial training on the target large language model, so as to construct a security mechanism based on gradient-based generation of attack samples, and use the security mechanism to train the large language model to obtain a final large language model that meets the preset robustness conditions, reduce the number of update steps of the adversarial sample, and reduce the number of iteration steps required to generate the adversarial sample, thereby ensuring the robustness of the large language model while ensuring the decision accuracy of the large language model. Thus, the technical problem in the related art that it is difficult to ensure the robustness of the large language model while ensuring the decision accuracy of the large language model is solved.

[0153] Next, a low-complexity safety device for enhancing the robustness of a large model according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0154] Figure 5 It is a block diagram of a low-complexity safety device for enhancing the robustness of a large model according to an embodiment of the present invention.

[0155] like Figure 5 As shown, the low-complexity safety device 10 for enhancing the robustness of a large model includes: a testing module 100 , a generating module 200 and a training module 300 .

[0156] Specifically, the testing module 100 is used to select multiple types of test tasks, and use each type of test task to perform robustness test on the target large language model, so as to obtain the robustness test result of the target large language model for each test task.

[0157] The generation module 200 is used to generate corresponding adversarial samples based on the robustness test results.

[0158] The training module 300 is used to optimize the target large language model using a preset model optimization expression while performing adversarial training on the target large language model using adversarial samples, so as to build a security mechanism based on gradient-generated attack samples, and use the security mechanism to train the large language model to obtain a final large language model that meets the preset robustness conditions.

[0159] Optionally, in one embodiment of the present invention, the test task includes at least one of a sentiment analysis task, a grammatical error recognition task, a repeated sentence detection task, a natural language reasoning task, a multi-task knowledge task, a reading comprehension task, a translation task and a mathematics task.

[0160] Optionally, in one embodiment of the present invention, the generating module 200 includes: an acquiring unit and an output unit.

[0161] Among them, the acquisition unit is used to acquire the target text and perturb the target text based on the robustness test result to obtain iterative samples.

[0162] The output unit is used to calculate the gradient according to the attack mode, and perturb the iterative sample along the direction of increasing gradient to obtain the decision error of this perturbation, and judge whether this perturbation achieves the preset attack purpose according to the decision error. If this perturbation does not achieve the preset attack purpose, the gradient direction is recalculated based on this perturbation, and the next perturbation is performed according to the result of this perturbation until the preset attack purpose is achieved and the adversarial sample is output.

[0163] Optionally, in one embodiment of the present invention, the training module 300 includes: a training unit.

[0164] Among them, the training unit is used to input the adversarial sample as a training sample into the target large language model, and update the model parameters of the target large language model according to the output result, so as to obtain the trained target large language model based on the model parameters.

[0165] Optionally, in one embodiment of the present invention, the calculation expression of the model parameter is:

[0166] =

[0167] in, For expectations, For distribution The data and label pairs sampled in To minimize the average loss function, are model parameters, is the target text, For disturbance, is the true label.

[0168] Optionally, in one embodiment of the present invention, the training module 300 includes: a prediction unit, a calculation unit and an optimization unit.

[0169] The prediction unit is used to define the task of the target large language model as symbol sequence prediction in a continuous situation, wherein the symbol sequence prediction is to predict the probability that the next symbol sequence of the preset symbol sequence mapping is the target sequence.

[0170] The computing unit is used to obtain the expected decision answer of the adversarial sample under the preset prompt based on probability.

[0171] The optimization unit is used to combine the expected decision answer, the factual answer and the probability of the adversarial sample, convert the decision task into a negative logarithmic probability, obtain a preset model optimization expression, and use the preset model optimization expression to optimize the target large language model.

[0172] Among them, the preset model optimization expression is:

[0173]

[0174] in, is a measure of the decision-making task. To obtain the expected probability of the decision answer, To expect a decision answer, is a preset symbol sequence.

[0175] It should be noted that the above explanation of an embodiment of a low-complexity security method for enhancing the robustness of a large model is also applicable to a low-complexity security device for enhancing the robustness of a large model in this embodiment, which will not be repeated here.

[0176] According to a low-complexity security device for enhancing the robustness of a large model proposed in an embodiment of the present invention, different types of test tasks can be used to perform robustness tests on a target large language model, thereby generating corresponding adversarial samples based on the robustness test results, and optimizing the target large language model while using the adversarial samples to perform adversarial training on the target large language model, so as to construct a security mechanism based on gradient-based generation of attack samples, and use the security mechanism to train the large language model to obtain a final large language model that meets the preset robustness conditions, reduce the number of update steps of the adversarial sample, and reduce the number of iteration steps required to generate the adversarial sample, thereby ensuring the robustness of the large language model while ensuring the decision accuracy of the large language model. Thus, the technical problem in the related art that it is difficult to ensure the robustness of the large language model while ensuring the decision accuracy of the large language model is solved.

[0177] Figure 6A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0178] A memory 601 , a processor 602 , and a computer program stored in the memory 601 and executable on the processor 602 .

[0179] When the processor 602 executes the program, a low-complexity security method for enhancing the robustness of a large model provided in the above embodiment is implemented.

[0180] Furthermore, the electronic device further comprises:

[0181] The communication interface 603 is used for communication between the memory 601 and the processor 602 .

[0182] The memory 601 is used to store computer programs that can be executed on the processor 602 .

[0183] The memory 601 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0184] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the communication interface 603, the memory 601 and the processor 602 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0185] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can communicate with each other through an internal interface.

[0186] The processor 602 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0187] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned low-complexity security method for enhancing the robustness of large models.

[0188] An embodiment of the present invention also provides a computer program product, including a computer program. When the computer program is executed, it is used to implement a low-complexity security method for enhancing the robustness of large models provided by an embodiment of the present invention.

[0189] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0190] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0191] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0192] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0193] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0194] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0195] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0196] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A low-complexity security method for enhancing the robustness of large models, characterized in that: The following steps are involved: Selecting multiple types of test tasks, and using each type of test task to perform robustness test on the target large language model, respectively, to obtain a robustness test result of the target large language model for each test task; Based on the robustness test result, generating corresponding adversarial samples; While using the adversarial sample to perform adversarial training on the target large language model, the target large language model is optimized using a preset model optimization expression to construct a security mechanism based on gradient-generated attack samples, and the large language model is trained using the security mechanism to obtain a final large language model that meets preset robustness conditions; The test task includes at least one of a sentiment analysis task, a grammatical error recognition task, a repeated sentence detection task, a natural language reasoning task, a multi-task knowledge task, a reading comprehension task, a translation task, and a mathematics task; Wherein, the generating of the corresponding adversarial sample based on the robustness test result includes: obtaining the target text, and perturbing the target text based on the robustness test result to obtain an iterative sample; calculating the gradient according to the attack mode, and perturbing the iterative sample along the direction of increasing gradient to obtain the decision error of this perturbation, and judging whether this perturbation achieves the preset attack purpose according to the decision error; if this perturbation does not achieve the preset attack purpose, recalculating the gradient direction based on this perturbation, and performing the next perturbation according to the result of this perturbation, until the preset attack purpose is achieved, and outputting the adversarial sample; Among them, using the adversarial sample to perform adversarial training on the target large language model includes: inputting the adversarial sample as a training sample into the target large language model, and updating the model parameters of the target large language model according to the output result, so as to obtain the trained target large language model based on the model parameters.

2. A low-complexity security method for enhancing the robustness of large models according to claim 1, characterized in that: in, The calculation expression of the model parameters is: = in, For expectations, For distribution The data and label pairs sampled in To minimize the average loss function, are the model parameters, is the target text, For disturbance, is the true label.

3. A low-complexity security method for enhancing the robustness of large models according to claim 1, characterized in that: The optimizing the target large language model by using a preset model optimization expression includes: The task of the target large language model is defined as symbol sequence prediction in a continuous situation, wherein the symbol sequence prediction is to predict the probability that the next symbol sequence of the preset symbol sequence mapping is the target sequence; Based on the probability, obtaining an expected decision answer of the adversarial sample under a preset prompt; Combining the expected decision answer, the factual answer of the adversarial sample and the probability, converting the decision task into a negative logarithmic probability, obtaining the preset model optimization expression, and optimizing the target large language model using the preset model optimization expression; Among them, the preset model optimization expression is: in, is the measurement indicator of the decision-making task, To obtain the expected probability of the expected decision answer, is the expected decision answer, is the preset symbol sequence.

4. A low-complexity safety device for enhancing the robustness of large models, characterized in that: include: A testing module, used to select multiple types of test tasks, and use each type of test task to perform robustness test on the target large language model, so as to obtain a robustness test result of the target large language model for each test task; A generation module, used to generate corresponding adversarial samples based on the robustness test results; A training module, configured to optimize the target large language model using a preset model optimization expression while performing adversarial training on the target large language model using the adversarial sample, so as to construct a security mechanism based on gradient-generated attack samples, and to train the large language model using the security mechanism to obtain a final large language model that meets a preset robustness condition; The test task includes at least one of a sentiment analysis task, a grammatical error recognition task, a repeated sentence detection task, a natural language reasoning task, a multi-task knowledge task, a reading comprehension task, a translation task, and a mathematics task; The generation module includes: an acquisition unit, which is used to acquire a target text, and perturb the target text based on the robustness test result to obtain an iterative sample; an output unit, which is used to calculate a gradient according to an attack mode, and perturb the iterative sample in a direction of increasing gradient to obtain a decision error of this perturbation, and judge whether this perturbation achieves a preset attack purpose according to the decision error; if this perturbation does not achieve the preset attack purpose, then based on this perturbation, recalculate the gradient direction, and perform the next perturbation according to the result of this perturbation, until the preset attack purpose is achieved, and output the adversarial sample; Among them, the training module includes: a training unit, which is used to input the adversarial sample as a training sample into the target large language model, and update the model parameters of the target large language model according to the output result, so as to obtain the trained target large language model based on the model parameters.

5. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a low-complexity security method for enhancing the robustness of a large model as described in any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement a low-complexity security method for enhancing the robustness of a large model as described in any one of claims 1 to 3.

7. A computer program product, characterized in that It comprises a computer program, which, when executed, is used to implement a low-complexity safety method for enhancing the robustness of a large model as claimed in any one of claims 1 to 3.

Images