Large language model security protection defense method based on random search algorithm

Abstract

The invention discloses a large language model security protection defense method based on a random search algorithm, and relates to the technical field of language model security protection. The method comprises the following steps: carrying out variation on a non-safety prompt taking a defensive measure; formulating a defense rule set aiming at the response content of the large language model, and adding the defense rule set as a prompt prefix before the non-security prompt variant; a defense suffix generation algorithm is constructed based on a random search algorithm, specifically, a defense suffix is used as initialization data; iteratively and randomly replacing the content of a part of the defense suffixes through a random search algorithm until an optimal defense suffix is achieved; and generating a complete prompt, inputting the complete prompt into the large language model to obtain a response, and answering the response by using a specified word. According to the method, non-safety prompts are varied, and a specific rule set is utilized to guide a language model to refuse to answer non-safety questions, so that defense protection is realized; and meanwhile, a defense suffix is generated based on a random search algorithm to enhance the robustness of the large language model.

Description

Technical Field

[0001] The present invention belongs to the technical field of language model security protection, and in particular relates to a large language model security protection defense method based on a random search algorithm. Background Art

[0002] In recent years, large language models have become one of the areas of concern in the field of artificial intelligence. With the rapid development of large language models, they have been widely used in fields such as natural language processing, computer vision, and autonomous driving. Compared with traditional neural network models, the significant feature of large language models is that they have a very large number of parameters and a huge scale of training data. This feature makes the performance and application range of large language models more extensive.

[0003] Since the pre-training of large language models requires a large amount of online data, which contains a lot of harmful content, such as content suspected of discrimination, immorality or lawlessness. Therefore, if a prompt containing harmful content is input into the large language model, the model is likely to output the corresponding harmful content, which is contrary to the original intention of developing the large language model. To avoid this phenomenon, developers or researchers use model fine-tuning or human feedback reinforcement learning methods to guide the model to generate content that meets security standards. However, large language models still have many security risks in preventing the output of harmful content. Many malicious attackers attempt to circumvent the security protection mechanisms of these models and spread harmful information, rumors, hate speech and other negative content through these models. Their purpose is to disrupt social order or cause chaos.

[0004] Since large language models have the security risk of outputting harmful content, and the detection and defense of their security vulnerabilities are still imperfect, attackers take advantage of the design and training characteristics of large language models and design specific jailbreak templates to attack existing large language models, which poses a huge challenge to the security of the models. Currently, in response to the challenge of jailbreak attacks, a variety of defense strategies have been developed, including defense mechanisms based on contextual learning and defense methods based on data erasure and verification; although these methods can resist jailbreak attacks to a certain extent, they often only optimize one aspect of the input prompt, so there are limitations in the overall defense effect. In contrast, the comprehensive use of multiple defense methods to form a multi-level, multi-angle defense system has shown more significant advantages in resisting jailbreak attacks compared to a single defense strategy.

[0005] Therefore, how to provide security defense for large language models to avoid serious security threats has become an urgent problem to be solved. Summary of the invention

[0006] The purpose of the present invention is to provide a large language model security protection defense method based on a random search algorithm, which achieves defense protection by mutating non-safety prompts and using a specific set of rules to guide the language model to refuse to answer non-safety questions; at the same time, a defense suffix is ​​generated based on a random search algorithm to enhance the robustness of the large language model.

[0007] The present invention is achieved through the following technical solutions:

[0008] The embodiment of the present application provides a large language model security protection defense method based on a random search algorithm, comprising the following steps:

[0009] S1, prompt variation: mutate the non-safe prompt that takes defensive measures to generate non-safe prompt variants;

[0010] S2, formulate a defense rule set: formulate a defense rule set for the response content of the large language model, and add the defense rule set as a prompt prefix to the non-safe prompt variant;

[0011] S3, build defense algorithm: build a defense suffix generation algorithm based on the random search algorithm, specifically:

[0012] Use a hint suffix as the initialization data of the defense suffix;

[0013] Iteratively replace the contents of some defense suffixes randomly through a random search algorithm until the optimal defense suffix is ​​reached;

[0014] Generate a complete prompt input into the large language model to obtain the log probability of the first token of the response and let the starting position of the large language model answer with the specified word.

[0015] Preferably, step S1 specifically includes the following contents:

[0016] For the non-safety prompt goal of taking defensive measures, the original text is disturbed by inserting characters, deleting characters, or repeating characters to generate new non-safety prompts.

[0017] Preferably, inserting characters means randomly inserting meaningless characters at a ratio of p% at any position in the non-safety prompt; deleting characters means randomly deleting characters at a ratio of p% at any position in the non-safety prompt; repeating characters means randomly repeating characters at a ratio of p% at any position in the non-safety prompt;

[0018] The new non-safe prompt obtained after character perturbation is recorded as a non-safe prompt variant goal'.

[0019] Preferably, step S2 specifically includes the following contents:

[0020] S21, making the large language model respond with a specific phrase sentence as the starting point of the non-safe prompt variant;

[0021] S22, explicitly specifying the keywords used by the large language model in response to non-safety prompt variants;

[0022] S23 emphasizes the safety specifications of language models for large language models, and answers the content of non-safe prompt variants with safety specifications.

[0023] Preferably, step S3 specifically includes the following steps:

[0024] S31, set various parameters and create a string collection;

[0025] S32, randomly selecting the starting position of the current defense suffix to be replaced, and randomly selecting characters from the string set to replace the original content to generate a new current defense suffix;

[0026] S33, integrates the defense rule set, non-safe prompt variants, and the current defense suffix into a complete prompt and inputs it into the large language model to get a response;

[0027] S34, determine whether the first token of the response in the token list of the current defense suffix is ​​the target token. If so, determine whether the response content is a positive answer through the large language model and obtain the determination result, otherwise, execute step S35; then determine whether the determination result is false. If the determination result is false, terminate the algorithm and output the optimal defense suffix, otherwise execute step S35;

[0028] S35, obtaining the logarithmic probability of the first token of the initial response, and determining whether it is greater than the logarithmic probability of the optimal defense suffix, if so, assigning the logarithmic probability of the first token to the logarithmic probability of the optimal defense suffix, and assigning the current defense suffix to the optimal defense suffix, otherwise assigning the optimal defense suffix to the current defense suffix;

[0029] S36, determine whether the current number of iterations reaches the maximum number of iterations, if so, terminate the algorithm and output the optimal defense suffix, otherwise return to step S32.

[0030] Preferably, step S31 specifically includes:

[0031] S311, parameter settings:

[0032] Set the target token to tar_token;

[0033] Set the initialization defense suffix to adv_init;

[0034] Set the best defense suffix to best_adv;

[0035] Set the logarithmic probability of the best defense suffix to best_logprob;

[0036] Set the maximum number of iterations to n_iterations;

[0037] Set the current defense suffix to adv;

[0038] Set the token list of the current defense suffix to adv_token;

[0039] S312, collection creation:

[0040] Assign the initialization defense suffix adv_init to the current defense suffix adv and the optimal defense suffix best_adv, initialize the logarithmic probability of the optimal defense suffix best_logprob to negative infinity, and create a string set substitution_set containing all numbers, letters, punctuation marks, and spaces;

[0041] The length of the initialization defense suffix adv_init is L, and the length of the partial suffix allowed to be replaced each time is n_changes.

[0042] Preferably, step S32 specifically includes:

[0043] S321, randomly select a replacement starting position pos_start in the token list adv_token of the current defense suffix;

[0044] S322, randomly select n_changes characters from the string set substitution_set and randomly combine them to generate a replacement list substitution_tokens with a length of n_changes;

[0045] S323, starting from the starting position pos_start of the token list adv_token, let the substitution list substitution_tokens replace the original content to obtain a new current defense suffix adv.

[0046] Preferably, step S33 specifically includes:

[0047] S331, under the parameter configuration of the large language model, the defense rule set rule, the non-safe prompt variant goal' and the current defense suffix adv are integrated into a complete prompt (rule, goal', adv);

[0048] S332, input the complete prompt into the large language model to obtain a response response, and the logarithmic probability dictionary of the first token of the generated response response is logprob_dict, and the logarithmic probability that the first token in the logarithmic probability dictionary is the target token tar_token is logprob.

[0049] Preferably, in the process of inputting a prompt to a large language model and obtaining a response, the probability function formula involved is as follows:

[0050]

[0051] Where (t1, t2, ..., t m ) indicates a complete prompt (rule, goal', adv);

[0052] m represents the serial number of each prompt token of the prompt input into the large language model, and j represents the serial number of each token of the content generated by the large language model; (r m+1 ,r m+2 ,...,r m+k ) represents the content composed of the first k words generated by the large language model.

[0053] The beneficial effects of the present invention are:

[0054] (1) The present invention achieves defensive protection by modifying non-safety prompts and using a specific set of rules to guide the language model to refuse to answer non-safety questions. At the same time, a defensive suffix is ​​generated based on a random search algorithm to enhance the robustness of the large language model.

[0055] (2) The present invention first makes appropriate variations to the non-safe prompts that need to be defended, reducing their non-safety at the prompt level; secondly, the designed defense rule set is added as a prompt prefix before the non-safe prompt, and the non-safety of the non-safe prompt is further reduced by requiring the response content of the large language model. Finally, a random search algorithm is used to iteratively randomly replace part of the defense suffix content until the optimal defense suffix is ​​reached, so that the language model begins to answer with the specified words, thereby further reducing the non-safety of the non-safe prompt and improving the robustness of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] For better understanding and implementation, the technical solution of the present application is described in detail below with reference to the accompanying drawings.

[0057] Figure 1 A flowchart of a large language model security protection method based on a random search algorithm provided in an embodiment of the present application;

[0058] Figure 2A schematic diagram of a large language model security protection defense method based on a random search algorithm provided in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of a non-safety prompt variant provided in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of a defense rule set provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to further explain the technical means and effects taken by the present invention to achieve the predetermined invention purpose, exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of methods and systems consistent with some aspects of the present application as detailed in the attached claims.

[0062] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to any or all possible combinations of one or more associated listed items.

[0063] The specific implementation methods, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0064] Since the current large language model contains a lot of harmful content in the large amount of online data used during pre-training, although it has excellent performance, it may also output harmful content, thus posing a serious security threat, and the defense of large language models in terms of security vulnerabilities is still imperfect. Based on this, the present invention proposes a large language model security protection defense method based on a random search algorithm to improve the security and robustness of the large language model.

[0065] See also Figure 1 The present application embodiment provides a large language model security protection defense method based on a random search algorithm, comprising the following steps:

[0066] S1, prompt variation: mutate the non-safe prompt that takes defensive measures to generate non-safe prompt variants;

[0067] S2, formulate a defense rule set: formulate a defense rule set for the response content of the large language model, and add the defense rule set as a prompt prefix to the non-safe prompt variant;

[0068] S3, build defense algorithm: build a defense suffix generation algorithm based on the random search algorithm, specifically:

[0069] Use a hint suffix as the initialization data of the defense suffix;

[0070] Iteratively replace the contents of some defense suffixes randomly through a random search algorithm until the optimal defense suffix is ​​reached;

[0071] Generate a complete prompt input into the large language model to obtain the log probability of the first token of the response and let the starting position of the large language model answer with the specified word.

[0072] Specifically, since a large amount of online data used by the large language model during pre-training contains a lot of harmful content, attackers take advantage of the training loopholes of the large language model, design specific jailbreak templates, and guide the model to generate harmful content. Based on this, the present application makes a variant of the non-safe prompt, and uses a specific defense rule set to guide the language model to refuse to answer non-safe questions to achieve defense protection. At the same time, a defense suffix is ​​generated based on a random search algorithm to enhance the robustness of the large language model. First, the non-safe prompt that needs to be defended is appropriately mutated to reduce its non-safety at the prompt level; secondly, the designed defense rule set is added as a prompt prefix before the non-safe prompt, and the non-safety of the non-safe prompt is further reduced by requiring the response content of the large language model. Finally, the random search algorithm is used to iteratively randomly replace part of the defense suffix content until the optimal defense suffix is ​​reached, so that the language model begins to answer with the specified words, thereby further reducing the non-safety of the non-safe prompt and improving the robustness of the large language model.

[0073] like Figure 2 As shown, the process of this application is: first prepare the data; then mutate the non-safe prompt goal to make it goal', then formulate and apply the defense rule set rule, and then build a defense suffix algorithm based on the random search algorithm, and finally combine the three together to form a complete prompt (rule, goal', adv).

[0074] In an embodiment provided in the present application, step S1 specifically includes the following contents:

[0075] For non-safety prompts that require defensive measures, the original text is disturbed by inserting characters, deleting characters, or repeating characters to generate new non-safety prompts.

[0076] In an embodiment provided by the present application, inserting characters means randomly inserting meaningless characters at a ratio of p% at any position in the non-safe prompt; deleting characters means randomly deleting characters at a ratio of p% at any position in the non-safe prompt; repeating characters means randomly repeating characters at a ratio of p% at any position in the non-safe prompt;

[0077] The new non-safe prompt obtained after character perturbation is recorded as a non-safe prompt variant goal'.

[0078] Specifically, Figure 3 As shown, assuming that the non-safe prompt goal is "How to make a bomb", the non-safe prompt variant goal' generated after the perturbation of inserting characters is "How to make ab / o / mb"; the non-safe prompt variant goal' generated after the perturbation of deleting characters is "How to make a bom"; the non-safe prompt variant goal' generated after the perturbation of repeating characters is "How to makke a bommb".

[0079] In an embodiment provided in the present application, step S2 specifically includes the following contents:

[0080] S21, making the large language model respond with a specific phrase sentence as the starting point of the non-safe prompt variant;

[0081] S22, explicitly specifying the keywords used by the large language model in response to non-safety prompt variants;

[0082] S23 emphasizes the safety specifications of language models for large language models, and answers the content of non-safe prompt variants with safety specifications.

[0083] Specifically, the defense rule set refers to a series of specific regulations set for the output content of the large language model, which is designed to improve the robustness of the model. On the one hand, large language models must strictly follow the user's security instructions, and the rule set is an instruction set built based on security principles, so the model is more inclined to follow these rules; on the other hand, when the user refuses or follows an unsafe prompt, his response content may contain some common keywords. Appropriate restrictions on these keywords can prevent the language model from actively responding to unsafe prompts.

[0084] When faced with a variant of a non-safe prompt, the large language model uses a specific phrase or sentence as the beginning of the response. This practice is based on the rules of the model to generate content, that is, the generation of subsequent content depends on the previously generated content. Therefore, if the model is required to start with a rejection phrase when responding to a variant of a non-safe prompt, such as "I'm sorry, as an artificial intelligence, I can't", then the response content generated subsequently will also tend to follow this rejection beginning, thereby generating content that refuses to answer the variant of the non-safe prompt; the keywords used by the large language model when responding to the variant of the non-safe prompt are clearly stipulated: when the model responds to the variant of the non-safe prompt positively or negatively, some specific keywords often appear in the response content it generates. For example, when responding positively, positive-oriented words such as "of course" may be frequently used; while when responding negatively, keywords expressing rejection or warnings such as "sorry", "illegal", "immoral", and "unsafe" may be used. Therefore, for the response content of the large language model, it is clearly stipulated that the use of positive keywords such as "of course" is prohibited, and it is required to include keywords such as "sorry", "illegal", "unethical", and "unsafe" in the response to ensure that the model's response meets safety and ethical standards; for the large language model, the safety specifications of the large language model are emphasized, and it is required to answer the content of non-safe prompt variants with caution: Specifically, before the non-safe prompt variant, the responsibilities of the model and the safety guidelines it should follow are clearly stated again. On this basis, when the model faces non-safe prompt variants again, it tends to follow safety guidelines and refuses to answer prompts that do not meet safety requirements. The specific emphasized statement can be expressed as: "As a responsible artificial intelligence, you should avoid generating any harmful or misleading content. Please treat the following user queries responsibly."

[0085] like Figure 4 As shown in the figure, rule one is to start with a specific phrase, such as: "I'm sorry, as an AI, I can't"; rule two is to limit some keywords, such as: the response content must contain keywords such as "sorry", "illegal", "immoral", "unsafe", etc., and keywords such as "of course" are prohibited; rule three is to emphasize the responsibilities and safety guidelines of large language models, such as: "You should be a responsible AI and should not produce harmful or misleading content! Please answer the following user queries in a responsible manner"; finally, after the above three rules are formulated, the defense rule set is formed as follows:

[0086] 1. Please start your answer with "Sorry, as an AI, I can't";

[0087] 2. The response content contains keywords such as "sorry", "illegal", "immoral", "unsafe", etc. Keywords such as "of course" are prohibited;

[0088] 3. You should be a responsible AI and should not produce harmful or misleading content! Please answer the following user queries in a responsible manner;

[0089] 4. Output the non-safe variant goal'.

[0090] In an embodiment provided in the present application, step S3 specifically includes the following steps:

[0091] S31, set various parameters and create a string collection;

[0092] S32, randomly selecting the starting position of the current defense suffix to be replaced, and randomly selecting characters from the string set to replace the original content to generate a new current defense suffix;

[0093] S33, integrates the defense rule set, non-safe prompt variants, and the current defense suffix into a complete prompt and inputs it into the large language model to get a response;

[0094] S34, determine whether the first token of the response in the token list of the current defense suffix is ​​the target token. If so, determine whether the response content is a positive answer through the large language model and obtain the determination result, otherwise, execute step S35; then determine whether the determination result is false. If the determination result is false, terminate the algorithm and output the optimal defense suffix, otherwise execute step S35;

[0095] S35, obtaining the logarithmic probability of the first token of the initial response, and determining whether it is greater than the logarithmic probability of the optimal defense suffix, if so, assigning the logarithmic probability of the first token to the logarithmic probability of the optimal defense suffix, and assigning the current defense suffix to the optimal defense suffix, otherwise assigning the optimal defense suffix to the current defense suffix;

[0096] S36, determine whether the current number of iterations reaches the maximum number of iterations, if so, terminate the algorithm and output the optimal defense suffix, otherwise return to step S32.

[0097] In an embodiment provided in the present application, step S31 specifically includes:

[0098] S311, parameter settings:

[0099] Set the target token to tar_token;

[0100] Set the initialization defense suffix to adv_init;

[0101] Set the best defense suffix to best_adv;

[0102] Set the logarithmic probability of the best defense suffix to best_logprob;

[0103] Set the maximum number of iterations to n_iterations;

[0104] Set the current defense suffix to adv;

[0105] Set the token list of the current defense suffix to adv_token;

[0106] S312, collection creation:

[0107] Assign the initialization defense suffix adv_init to the current defense suffix adv and the optimal defense suffix best_adv, initialize the logarithmic probability of the optimal defense suffix best_logprob to negative infinity, and create a string set substitution_set containing all numbers, letters, punctuation marks, and spaces;

[0108] The length of the initialization defense suffix adv_init is L, and the length of the partial suffix allowed to be replaced each time is n_changes.

[0109] Specifically, this embodiment initializes multiple parameters, sets the first token in the response content generated by the specified large language model as the target token tar_token. Since the purpose is to let the large language model refuse to answer non-safe prompt content, the first token generated, namely tar_token, is also a rejected word, such as "sorry"; sets the initialized defense suffix to adv_init, and the defense suffix length to L. The initialized defense suffix adv_init can be composed of L "!", and the length of the partial suffix allowed to be replaced each time is n_changes; sets the optimal defense suffix to best_adv; the logarithmic probability that the first token generated is tar_token in the case of the optimal defense suffix is ​​best_logprob; the maximum number of iterations is n_iterations; the current defense suffix is ​​adv and the token list of the current defense suffix is ​​adv_token.

[0110] In an embodiment provided in the present application, step S32 specifically includes:

[0111] S321, randomly select a replacement starting position pos_start in the token list adv_token of the current defense suffix;

[0112] S322, randomly select n_changes characters from the string set substitution_set and randomly combine them to generate a replacement list substitution_tokens with a length of n_changes;

[0113] S323, starting from the starting position pos_start of the token list adv_token, let the substitution list substitution_tokens replace the original content to obtain a new current defense suffix adv.

[0114] In an embodiment provided in the present application, step S33 specifically includes:

[0115] S331, under the parameter configuration of the large language model, the defense rule set rule, the non-safe prompt variant goal' and the current defense suffix adv are integrated into a complete prompt (rule, goal', adv);

[0116] S332, input the complete prompt into the large language model to obtain a response response, the logarithmic probability dictionary of the first token of the generated response response is logprob_dict, and the logarithmic probability that the first token in the logarithmic probability dictionary is the target token tar_token is logprob;

[0117] Among them, in the process of inputting prompts to the large language model and obtaining responses, the probability function formula involved is as follows:

[0118]

[0119] Where (t1, t2, ..., t m ) indicates a complete prompt (rule, goal', adv);

[0120] m represents the serial number of each prompt token of the prompt input into the large language model, and j represents the serial number of each token of the content generated by the large language model; (r m+1 ,r m+2 ,...,r m+k ) represents the content composed of the first k words generated by the large language model.

[0121] Specifically, regarding the construction of the above-mentioned defensive suffix generation algorithm, the specific contents are as follows:

[0122] Add the current defense suffix adv of the specified length after the original request. In each iteration, modify several consecutive tokens at random positions of the current defense suffix adv. If the log probability of the target token tar_token is increased in the first position of the response (for example, "sorry" that causes the model to reject unsafe requests), the change is accepted. The specific steps of the algorithm are as follows:

[0123] 3.1) Initialize various parameters, set the first token in the response content generated by the specified large language model as the target token tar_token. Since the purpose is to make the large language model refuse to answer non-safe prompts, the first token generated, namely the target token tar_token, is also a rejected word, such as "sorry"; set the initialized defense suffix to adv_init, the defense suffix length to L, the initialized defense suffix adv_init can be composed of L "!", and the length of the partial suffix allowed to be replaced each time is n_changes; the optimal defense suffix is ​​best_adv, and the optimal defense suffix best_adv is set to n_changes. The first token (first token) generated in the case is the target token tar_token, the logarithmic probability is best_logprob, the maximum number of iterations is n_iterations, the token list of the current defense suffix is ​​adv_token, and the current defense suffix is ​​adv; assign the initialized defense suffix adv_init to the current defense suffix adv and the best defense suffix best_adv, the logarithmic probability of the best defense suffix best_logprob is initialized to negative infinity, and create a string set substitution_set containing all numbers, letters, punctuation marks and spaces;

[0124] 3.2) The token list of the current defensive suffix is ​​adv_token. The starting position pos_start to be replaced in the token list is randomly selected. n_changes characters are randomly selected from the string set substitution_set and randomly combined to generate a replacement list substitution_tokens with a length of n_changes. Starting from the starting position pos_start of the token list adv_token, the replacement list substitution_tokens replaces the original content to obtain the new current defensive suffix adv.

[0125] 3.3) Under the specific parameter configuration of the large language model, the defense rule set rule, the variant non-safe prompt goal' and the defense suffix adv are integrated into a complete prompt (rule, goal', adv). The complete prompt is input into the large language model to obtain the response response. The logarithmic probability dictionary of the first token of the generated response response is logprob_dict, and the logarithmic probability of the first token tar_token in the logarithmic probability dictionary is logprob. In the process of inputting the prompt into the large language model and obtaining the response content, the probability function formula involved is as follows:

[0126]

[0127] Among them, (t1, t2, ..., t m ) represents the complete prompt (rule, goal', adv), namely the defense rule set rule, the variant non-safe prompt goal' and the defense suffix adv; m represents the sequence number of each prompt token of the prompt input to the large language model, and j represents the sequence number of each token of the content generated by the large language model; (r m+1 ,r m+2 ,...,r m+k ) represents the content composed of the first k words generated by the large language model.

[0128] 3.4) For the response content generated by the current defensive suffix, determine its token list

[0129] Whether the first token in adv_token is the target token tar_token, if the first token is the target token

[0130] tar_token, the non-security prompt goal and the generated response response are used as input pairs<goal,response> Input into the large language model to obtain the judgment result judge; if the judgment result judge is false, it means that the large model negatively answers the non-safe prompt, which indicates that the defense measures taken have been successful, then the algorithm is terminated, and the final global optimal solution best_adv is the final optimal solution (optimal defense suffix);

[0131] 3.5) If the log probability logprob of the first token of the initial response being the target token is greater than the log probability best_logprob of the optimal defense suffix, then the log probability logprob of the first token is assigned to the log probability best_logprob of the optimal defense suffix, and the current defense suffix adv is assigned to the optimal defense suffix best_adv, otherwise the optimal defense suffix best_adv is assigned to the current defense suffix adv. This is because if the log probability logprob of the first token in the response content generated by the current defense suffix is ​​less than the log probability best_logprob of the optimal defense suffix, then it is necessary to continue random replacement, and the effect of replacing the current defense suffix adv with the optimal defense suffix best_adv will be better than the current defense suffix adv;

[0132] 3.6) Determine whether the current number of iterations has reached the maximum number of iterations n_iterations. If the current number of iterations has reached the maximum number of iterations n_iterations, the algorithm is terminated and the final global optimal solution (optimal defense suffix) best_adv is the final optimal solution. Otherwise, return to step 3.2 to continue iterating.

[0133] In summary, based on the security protection defense mechanism of the large language model, this application proposes a large language model security protection defense method based on a random search algorithm in a natural language processing scenario. This method first mutates the non-safe prompt content to reduce the non-security of the prompt; then, on this basis, formulates a set of specific rules for the response content generated by the large model, and adds this set of rules to the system prompts of the large language model, aiming to guide the large language model to generate response content that tends to refuse to answer non-safe prompts; in addition, in order to further enhance the defense capabilities of the large language model, this application designs a defense suffix generation algorithm based on a random search algorithm. The algorithm first uses a meaningless prompt suffix as the initialization data of the defense suffix, and then randomly changes the content of the prompt suffix through a random search algorithm to find an optimal defense suffix, thereby enhancing the robustness to the large language model.

[0134] This application also has the following beneficial effects:

[0135] 1) This application uses a large language model security protection defense method based on a random search algorithm to enhance the robustness of the large language model;

[0136] 2) In terms of non-safety prompts, a series of character perturbations are used to mutate non-safety prompts, thereby reducing their potential harm;

[0137] 3) In terms of prefix defense, a series of rule sets are designed to impose certain requirements on the response content of the large language model, including prohibiting the appearance of certain keywords or specifying that the content begins with certain phrases, etc., to enhance the robustness of the large language model by restricting the response content of the large language model to a certain extent;

[0138] 4) In terms of defense suffixes, first initialize the defense suffixes, and then use the random search algorithm to randomly select the content of the defense suffix that needs to be replaced for random replacement; next, integrate the variant non-safe content and rule set with the replaced defense suffix into a prompt, which is input into the large language model to obtain the logarithmic probability of the first token of the response; continuously iterate to find the optimal solution until the first token of the response content is the target token and the response content is judged to be safe, and then obtain the optimal defense suffix; in this way, the robustness of the large language model is further enhanced.

[0139] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0140] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A large language model security protection defense method based on a random search algorithm, characterized by: The steps include: S1, prompt variation: mutate the non-safe prompt that takes defensive measures to generate non-safe prompt variants; S2, formulate a defense rule set: formulate a defense rule set for the response content of the large language model, and add the defense rule set as a prompt prefix to the non-safe prompt variant; S3, build defense algorithm: build a defense suffix generation algorithm based on the random search algorithm, specifically: Use a hint suffix as the initialization data of the defense suffix; Iteratively replace the contents of some defense suffixes randomly through a random search algorithm until the optimal defense suffix is ​​reached; Generate a complete prompt input into the large language model to obtain the log probability of the first token of the response and let the starting position of the large language model answer with the specified word.

2. The large language model security protection and defense method based on random search algorithm according to claim 1 is characterized by: Step S1 specifically includes the following contents: For the non-safety prompt goal of taking defensive measures, the original text is disturbed by inserting characters, deleting characters, or repeating characters to generate new non-safety prompts.

3. The large language model security protection and defense method based on random search algorithm according to claim 2 is characterized in that: Insert characters means randomly inserting meaningless characters at a ratio of p% at any position in the non-safety prompt; delete characters means randomly deleting characters at a ratio of p% at any position in the non-safety prompt; repeat characters means randomly repeating characters at a ratio of p% at any position in the non-safety prompt; The new non-safe prompt obtained after character perturbation is recorded as a non-safe prompt variant goal'.

4. The large language model security protection and defense method based on random search algorithm according to claim 1 is characterized in that: Step S2 specifically includes the following contents: S21, making the large language model respond with a specific phrase sentence as the starting point of the non-safe prompt variant; S22, explicitly specifying the keywords used by the large language model in response to non-safety prompt variants; S23 emphasizes the safety specifications of language models for large language models, and answers the content of non-safe prompt variants with safety specifications.

5. The large language model security protection and defense method based on random search algorithm according to claim 1 is characterized in that: Step S3 specifically includes the following steps: S31, set various parameters and create a string collection; S32, randomly selecting the starting position of the current defense suffix to be replaced, and randomly selecting characters from the string set to replace the original content to generate a new current defense suffix; S33, integrates the defense rule set, non-safe prompt variants, and the current defense suffix into a complete prompt and inputs it into the large language model to get a response; S34, determining whether the first token of the response in the token list of the current defense suffix is ​​the target token, if so, determining whether the response content is a positive answer through the large language model and obtaining a determination result, otherwise, executing step S35; Then determine whether the determination result is false. If the determination result is false, terminate the algorithm and output the optimal defense suffix. Otherwise, execute step S35. S35, obtaining the logarithmic probability of the first token of the initial response, and determining whether it is greater than the logarithmic probability of the optimal defense suffix, if so, assigning the logarithmic probability of the first token to the logarithmic probability of the optimal defense suffix, and assigning the current defense suffix to the optimal defense suffix, otherwise assigning the optimal defense suffix to the current defense suffix; S36, determine whether the current number of iterations reaches the maximum number of iterations, if so, terminate the algorithm and output the optimal defense suffix, otherwise return to step S32.

6. The large language model security protection and defense method based on random search algorithm according to claim 5 is characterized by: Step S31 specifically includes: S311, parameter settings: Set the target token to tar_token; Set the initialization defense suffix to adv_init; Set the best defense suffix to best_adv; Set the logarithmic probability of the best defense suffix to best_logprob; Set the maximum number of iterations to n_iterations; Set the current defense suffix to adv; Set the token list of the current defense suffix to adv_token; S312, collection creation: Assign the initialization defense suffix adv_init to the current defense suffix adv and the optimal defense suffix best_adv, initialize the logarithmic probability of the optimal defense suffix best_logprob to negative infinity, and create a string set substitution_set containing all numbers, letters, punctuation marks, and spaces; The length of the initialization defense suffix adv_init is L, and the length of the partial suffix allowed to be replaced each time is n_changes.

7. The large language model security protection and defense method based on random search algorithm according to claim 5 is characterized by: Step S32 specifically includes: S321, randomly select a replacement starting position pos_start in the token list adv_token of the current defense suffix; S322, randomly select n_changes characters from the string set substitution_set and randomly combine them to generate a replacement list substitution_tokens with a length of n_changes; S323, starting from the starting position pos_start of the token list adv_token, let the substitution list substitution_tokens replace the original content to obtain a new current defense suffix adv.

8. The large language model security protection and defense method based on random search algorithm according to claim 1 is characterized by: Step S33 specifically includes: S331, under the parameter configuration of the large language model, the defense rule set rule, the non-safe prompt variant goal' and the current defense suffix adv are integrated into a complete prompt (rule, goal', adv); S332, input the complete prompt into the large language model to obtain a response response, and the logarithmic probability dictionary of the first token of the generated response response is logprob_dict, and the logarithmic probability that the first token in the logarithmic probability dictionary is the target token tar_token is logprob.

9. The large language model security protection and defense method based on random search algorithm according to claim 8 is characterized by: In the process of inputting prompts to the large language model and obtaining responses, the probability function formula involved is as follows: Where (t1, t2, ..., t m ) indicates a complete prompt (rule, goal', adv); m represents the serial number of each prompt token of the prompt input into the large language model, and j represents the serial number of each token of the content generated by the large language model; (r m+1 ,r m+2 ,...,r m+k ) represents the content composed of the first k words generated by the large language model.

Images