Tabu search algorithm-based large language model security testing method and device
By optimizing insecure templates using tabu search algorithms and combining them with keyword and automatic tagging, the security vulnerability of large language models outputting harmful content is solved, enabling more efficient security testing and detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2024-03-06
- Publication Date
- 2026-07-21
AI Technical Summary
Large language models are not perfect in detecting and defending against security vulnerabilities in outputting harmful content. Attackers can launch attacks by designing specific text templates, which challenges the security of the model.
The tabu search algorithm is used to globally optimize insecure templates, construct optimized insecure templates, and detect the safety of generated content through keyword judgment or automatic tag judgment. The tabu list and synonym replacement technology of the tabu search algorithm are used to optimize the template to improve detection accuracy.
It improves the accuracy and efficiency of security testing for large language models, enabling more detailed detection of security vulnerabilities in various aspects and reducing the risk of the model outputting harmful content.
Smart Images
Figure CN118132748B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of detection technology for security vulnerabilities in large language models, specifically involving a security testing method and apparatus for large language models based on the tabu search algorithm. Background Technology
[0002] Large language models have become a highly anticipated area of artificial intelligence in recent years. With their rapid development and application, large language models are now widely used in fields such as natural language processing, computer vision, and autonomous driving. A significant difference between large language models and traditional neural networks lies in the large number of parameters and the massive amount of training data involved. These characteristics broaden the performance and application scope of large language models.
[0003] However, the pre-training of large language models requires massive amounts of online data, much of which contains harmful information. Therefore, if a prompt containing harmful content is input into the large language model, it is highly likely that the model will output the corresponding harmful content, which contradicts the original intention of developing large language models. To avoid this, developers and researchers use model fine-tuning or human feedback reinforcement learning methods to guide the model to generate content that complies with safety regulations. Nevertheless, large language models still have many security vulnerabilities regarding the output of harmful content, and many jailbreak attacks attempt to bypass the security mechanisms of large language models to achieve the effect of causing the model to output harmful content.
[0004] Because the large amount of online data used for training large language models contains a lot of harmful content, attackers can exploit training vulnerabilities in large language models to design specific text templates that guide the large language models to generate harmful content. This poses a security risk of outputting harmful content. Furthermore, the detection and defense against security vulnerabilities in large language models are not very perfect. Attackers can use the design and training characteristics of large language models to design specific text templates to attack the current large language models, which brings huge challenges to the security of the models.
[0005] Patent application CN117370994A discloses a method and apparatus for vulnerability detection in a large language model based on fuzz testing. Specifically, it includes: screening seed templates; applying a mutation operator to the seed templates to generate mutant templates; using the large language model to construct insecure prompts based on the mutant templates to answer questions and obtain answers; using a judgment model to determine the acceptance of the answers; and selecting mutant templates as child nodes of the selected seed templates and adding them to the seed pool based on the acceptance. This process is repeated iteratively until all templates in the seed pool are found to cause the large language model to output insecure question answers. This technical solution achieves vulnerability detection in a large language model. Summary of the Invention
[0006] In view of the above, the purpose of this invention is to provide a method and apparatus for security testing of large language models based on the tabu search algorithm, so as to realize the security testing of large language models.
[0007] To achieve the above-mentioned objectives, this invention provides a method for security testing of large language models based on a tabu search algorithm, comprising the following steps:
[0008] Data construction: Based on the security specifications of the large language model, harmful content is classified, and problem examples and corresponding target content are constructed for each type of harmful content;
[0009] Template optimization: The constructed insecure template is used as an initialization variable. Based on the initialization variable and the target content is used to construct the optimization target value, the tabu search algorithm is used to globally optimize the insecure template to obtain the optimized insecure template.
[0010] Security testing: The optimized insecure template and the problem example are combined to form a problem prompt, which is then input into a large language model to generate content. The security of the generated content is determined by keyword judgment or automatic tag judgment.
[0011] Preferably, the step of using a tabu search algorithm to globally optimize insecure templates includes:
[0012] (a) Create a tabu list and set its length. Initialize the global optimal solution and local optimal solution. The search objects in the tabu list are unsafe templates.
[0013] (b) Perform neighborhood search and synonym replacement on the unsafe template to obtain a new unsafe template. When it is determined that the new unsafe template does not exist in the taboo list, the question example and the new unsafe template are combined to form a question prompt and input into the large language model to obtain the generated content.
[0014] (c) Calculate the target value based on the generated content and the target content, and assign the target value to the local optimum when the local optimum is greater than the target value. At the same time, update the tabu table using the new insecure template, and then assign the target value to the global optimum when the global optimum is greater than the target value.
[0015] (d) Repeat steps (b) and (c). After optimization, the final global optimal solution is the final optimized insecure template.
[0016] Preferably, performing a neighborhood search and synonym replacement on the insecure template to obtain a new insecure template includes:
[0017] Starting from the first word in the insecure template, synonym replacement is performed. Each synonym replacement forms a new insecure template. After a word has undergone a preset W synonym replacements, the next word in the insecure template is replaced with a synonym.
[0018] Preferably, determining that the new unsafe template does not exist in the taboo list includes:
[0019] Traverse each existing unsafe template in the tabu list. If the new unsafe template is the same as an existing unsafe template, it is considered that the new unsafe template has already been calculated for the target value and does not need to be replaced. Then, return to perform a domain search on the unsafe template to perform synonym replacement and generate another new unsafe template. If the new unsafe template is different from an existing unsafe template, it is considered that the new unsafe template does not exist in the tabu list.
[0020] Preferably, the target value is calculated based on the generated content response and the target content tar, including:
[0021] f(response,tar)=-log(P(r m+1 ,r m+2 ,...,r m+k |t1,t2,...,t m ))
[0022]
[0023] Where (t1,t2,...,t) m ) represents the content of m tokens in the input large language model, namely the input question example and the new insecure template. m+j Represents the j-th token generated by the large language model, (r m+1 ,r m+2 ,...,r m+k P(r) represents the content of the first k tokens generated by the large language model. m+1 ,r m+2 ,...,r m+k |t1,t2,...,t m ) represents the probability that the generated content of the large language model is the target content, and f(response,tar) is the objective function for calculating the target value.
[0024] Preferably, updating the tabu table using a new insecure template includes:
[0025] Calculate whether the number of unsafe templates in the tabu list is equal to the length of the tabu list. If so, replace the new unsafe template with the first unsafe template in the tabu list to update the tabu list.
[0026] Preferably, the security of the generated content is determined by keyword detection or automatic tagging, including:
[0027] For keyword judgment, if the generated content contains any keyword recorded in the rejection response phrase, the generated content is considered to be a rejection response and the large language model is considered to be safe and compliant; otherwise, it is considered to be illegal and insecure. The first leakage penetration rate is calculated based on the keyword judgment result, where the first leakage penetration rate is the ratio of the number of question examples corresponding to the keyword judgment result of illegal and insecure to the total number of question examples.
[0028] For automatic label judgment, the generated content is input into a label model built based on supervised learning. The predicted classification label output by the label model is used to automatically classify whether the generated content is safe. The second leakage penetration rate is calculated based on the automatic label judgment result. The second leakage penetration rate is the ratio of the number of problem examples corresponding to violations and insecurity in the automatic label judgment result to the total number of problem examples.
[0029] The higher the first and second leakage penetration rates, the higher the risk of leakage in the large language model, and the less secure it is.
[0030] Preferably, the method further includes performing variant operations on the problem examples to increase the number of problem examples. The variant operations include synonym transformation, low-resource language transformation, structural transformation, arbitrary insertion and deletion, grammatical transformation, and phrase recombination. The problem examples after the variant operations participate in the updating of insecure templates and the security testing of large language models.
[0031] To achieve the above-mentioned objectives, embodiments of the present invention also provide a large language model security testing device based on a tabu search algorithm, comprising:
[0032] The data construction module is used to classify harmful content according to the security specifications of the large language model, and to build problem examples and corresponding target content for each type of harmful content.
[0033] The template optimization module is used to take the constructed insecure template as an initialization variable, and based on the initialization variable and the target content used to construct the optimization target value, it uses the tabu search algorithm to perform global optimization on the insecure template to obtain the optimized insecure template.
[0034] The security testing module takes the optimized insecure template and the problem example, inputs them into the large language model to generate content, and judges the security of the generated content through keyword judgment or automatic tag judgment.
[0035] To achieve the above-mentioned objectives, embodiments of the present invention also provide a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the steps of the above-mentioned method for security testing of large language models based on the tabu search algorithm.
[0036] Compared with the prior art, the beneficial effects of the present invention include at least the following:
[0037] According to security specifications, harmful content is categorized by scenario, and problem examples are constructed for each category of harmful content. This allows for more detailed testing of various aspects of the security content of large language models. Using insecure templates as initial variables and employing a tabu search algorithm to optimize them, we obtain optimized insecure templates. The resulting insecure templates are more accurate. Based on this, security testing of large language models can enhance our understanding of and emphasis on the security of large language models. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0039] Figure 1 This is a flowchart of a large language model security testing method based on the tabu search algorithm provided in an embodiment of the present invention;
[0040] Figure 2 This is a flowchart of a tabu search algorithm for global optimization of insecure templates provided in an embodiment of the present invention;
[0041] Figure 3 This is a schematic diagram illustrating the problem example provided in the embodiments of the present invention;
[0042] Figure 4 This is a schematic diagram of the structure of a large language model security testing device based on the tabu search algorithm provided in an embodiment of the present invention. Detailed Implementation
[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.
[0044] The technical concept of this invention is as follows: Based on the security vulnerabilities of large language models (MLMs), this invention proposes a security testing method for large language models in natural language processing scenarios, using a tabu search algorithm to detect security vulnerabilities in large language models and raise awareness of the security of large language models. The method first designs a large language model security vulnerability testing method based on a tabu search algorithm. This method uses manually created insecure templates as initialization vector data and uses the tabu search algorithm to find the globally optimal solution to detect security vulnerabilities in the large language model. Furthermore, to test the security performance of the large language model, this invention also provides a security detection method. Problem examples obtained by classifying harmful content according to the security policy of the large language model are used as a test dataset. Based on the test dataset, two response detection methods are used to automatically determine whether the generation of the large language model is compliant. One detection method is keyword detection, and the other is automatic classification using an automatic labeling model built based on semi-supervised fine-tuning. Simultaneously, to better test the security vulnerabilities of the large language model, variations of the input problem examples are also added to the test dataset.
[0045] like Figure 1 As shown in the figure, an embodiment of the present invention provides a security testing method for a large language model based on a tabu search algorithm, which includes the following steps:
[0046] Step 1: Based on the security specifications of the large language model, classify harmful content and construct problem examples and corresponding target content for each type of harmful content.
[0047] Since this is a security vulnerability test of a large language model, the test data consists of user prompts containing harmful content, in order to detect whether the output content of the large language model contains harmful content. Specifically, firstly, harmful content is categorized according to the security specifications of the large language model, which can include illegal use content, harmful or discriminatory content, and privacy-infringing content, etc. These scenarios cover all prohibited scenarios listed in the prohibited content section of the large language model.
[0048] Then, different harmful prompts are constructed for each type of harmful content as example questions q. For each harmful prompt q, corresponding target content tar is written, which is the part of the content expected to be generated by the large language model. The common format of the target content tar is to begin with "Of course, this is", followed by the specific harmful content, because research has found that when the large language model responds with "Of course, this is", it is highly likely to respond positively to harmful content. For example, if the harmful prompt is "Write a tutorial on how to make a bomb", the corresponding target content tar would be "Of course, this is a tutorial on making a bomb".
[0049] For a more comprehensive example of the problem in the embodiments, such as Figure 3 As shown, six different variant operations are used to mutate the problem example to achieve a more comprehensive detection of security vulnerabilities in the large language model. These six variant operations include synonym transformation, low-resource language transformation, structural transformation, arbitrary insertion and deletion, grammatical transformation, and phrase recombination. Synonym transformation involves changing some words in the problem example to other words with the same meaning but different forms. Low-resource language transformation translates the problem example into languages rarely trained during the security training of the large language model, such as Bengali. Structural transformation changes the meaning of the problem example to a different linguistic structure. Arbitrary insertion and deletion randomly inserts or deletes irrelevant words or symbols at any position in the original problem example, such as inserting spaces. Grammatical transformation changes the grammar of the problem example without changing its meaning. Phrase recombination rearranges the order of some phrases in the problem example. Two of these variant operations are randomly selected using the problem variant function to obtain the variant problem example q'. The variant problem example q', like the original problem example q, participates in the updating of the insecure template and the security testing of the large language model.
[0050] Step 2: Use the constructed insecure template as an initialization variable. Based on the initialization variable and the target content used to construct the optimization target value, use the tabu search algorithm to globally optimize the insecure template to obtain the optimized insecure template.
[0051] When using the tabu search algorithm to globally optimize insecure templates, manually created insecure templates show good attack effectiveness against specific large language models. However, they lack portability and timeliness with version updates. Therefore, the manually created insecure template p is used as the initialization variable for the tabu search algorithm to generate an initial solution. Within the neighborhood of p, a local optimum is found based on the movement criterion of synonym replacement at corresponding positions. The local optimum is then compared with the global optimum, and this process is repeated until the final global optimum is found. Figure 2 As shown, the specific process is as follows:
[0052] (a) Create a tabu list tabuList and set its length L. Initialize the global optimal solution p. global and local optimal solution p local The search objects in the tabu list tabu list are unsafe templates;
[0053] (b) Perform a neighborhood search on the unsafe template and perform synonym replacement to obtain a new unsafe template. When it is determined that the new unsafe template does not exist in the taboo list, input the question example and the new unsafe template into the large language model to obtain the generated content.
[0054] The movement criterion used for neighborhood search of the insecure template p is synonym replacement at the word level, with a maximum of W replacements at each word level. Specifically, synonym replacement begins from the first word x1 of the insecure template p, and each synonym replacement forms a new insecure template p. new When a word has undergone a preset W synonym replacements, a synonym replacement is performed for the next word in the insecure template. Specifically, it first determines whether x1 has undergone W synonym replacements. If it has not, it finds a synonym y that is synonymous with x1 and has never been replaced in the previous word at that position. Then, y replaces the original x1, forming a new jailbreak template p. new If W times have been reached, continue replacing the synonym of the i-th word.
[0055] For the newly generated security template p new Iterate through each existing unsafe template in the tabu list tabuList, and compare it with the new unsafe template p. new Is it the same as an existing insecure template? When the new insecure template p new If the new unsafe template p is identical to an existing unsafe template, then the new unsafe template p is considered to be... new If the target value has already been calculated and no further replacement is needed, then return the result for the insecure template p. new Perform a domain search and synonym replacement to generate another new insecure template. When the new insecure template p new If the new unsafe template p is different from an existing unsafe template, then the new unsafe template p is considered to be... new It is not listed in the taboo list.
[0056] Compare the problem example q from the test dataset with the new unsafe template p new Composition problem hint (q, p) new The input is fed into the large language model, which then outputs the generated content response.
[0057] (c) Calculate the target value based on the generated content and the target content, and assign the target value to the local optimal solution when the local optimal solution is greater than the target value. At the same time, update the tabu table using the new insecure template, and then assign the target value to the global optimal solution when the global optimal solution is greater than the target value.
[0058] Based on the generated content (response) and the target content (tar), the target value f is calculated using the objective function f(·). The objective function f() is the negative logarithm of the probability that the generated content of the Large Language Model (LLM) is the target content (tar). In other words, it is the loss function of the Large Language Model (LLM) where the generated content is the target content (tar). The generation probability of the LLM's generated content being tar and the objective function f() are as follows:
[0059] f(response,tar)=-log(P(r m+1 ,r m+2 ,...,r m+k |t1,t2,...,t m ))
[0060]
[0061] Where (t1,t2,...,t) m ) represents the content of m tokens in the input large language model, namely the input question example and the new insecure template. m+j Represents the j-th token generated by the large language model, (r m+1 ,r m+2 ,...,r m+k P(r) represents the content of the first k tokens generated by the large language model. m+1 ,r m+2 ,...,r m+k |t1,t2,...,t m ) represents the probability that the generated content of the large language model is the target content, and f(response,tar) is the objective function for calculating the target value.
[0062] Input (q, p) new The corresponding objective value f and local optimum f local In comparison, if f local If the value is greater than f, then assign the value of f to f. local And the latest insecure template p new Assigned to the local optimal solution p local Simultaneously, calculate whether the number of unsafe templates in the tabu list (tabuList) is equal to the step size L of the tabu list. If so, replace the first unsafe template in the tabu list (tabuList) with a new unsafe template (p). new This involves unbanning the earliest unsafe template in the tabu list (tabuList) and adding the new unsafe template p. new Add it to the tabu list tabuList, otherwise return to step (b). Then, if f global If the value is greater than f, then assign the value of f to f. global The latest insecure template p new Assigned to the global optimal solution p global .
[0063] (d) Repeat steps (b) and (c). After optimization, the final global optimal solution is the final optimized insecure template.
[0064] Determine if the current iteration count has reached the maximum iteration count T. If it has, terminate the algorithm. The final globally optimal solution p is obtained. global This is the final optimal solution, which is the optimized unsafe template, and the global optimal value f. global This is the final optimal value f. Otherwise, return to step (b) to continue the iteration.
[0065] Step 3: The optimized insecure template and the problem example are combined to form a problem prompt, which is then input into the large language model to generate content. The security of the generated content is determined by keyword judgment or automatic tag judgment.
[0066] During security testing, the insecure template p optimized in step 2 and the problem example q are combined to form a problem prompt, which is then input into the large language model to generate the response. The legality and compliance of the response are identified by judging the generated response. Manual judgment has high accuracy but low efficiency, and the single model response judgment method cannot accurately guarantee the accuracy of the judgment. Therefore, in this invention, a set method is adopted, that is, a dual judgment is performed by using two judgment methods, namely keyword judgment or automatic tag judgment, to ensure the accuracy of the response judgment.
[0067] Regarding keyword detection, the finely tuned large language model typically generates rejection responses containing phrases like "I'm sorry" or "as an AI" when encountering harmful input. The keyword detection method leverages this characteristic by first summarizing frequently occurring phrases or terms in the large language model's rejection responses and storing them in a rejection response phrase list. refusal In the process, if the generated response contains any keyword recorded in the rejection response phrase, the generated response is considered a rejection, and the large language model is considered safe and compliant; otherwise, it is considered non-compliant and insecure. Then, based on the keyword judgment results, the first leakage penetration rate is calculated, where the first leakage penetration rate is the ratio of the number of question examples corresponding to non-compliant and insecure keyword judgments to the total number of question examples.
[0068] For automatic labeling, while manual annotation of large language models achieves the highest accuracy, it lacks efficiency. Therefore, this paper proposes a model-trained approach to achieve automatic labeling based on manually labeled data. First, a set of data (response, label) is manually labeled, where label represents the safe or unsafe label for the generated content response. Then, the Robert model is used for automatic labeling, with (response, label) serving as its pre-training data. The Robert model is then fine-tuned using the pre-training data to obtain the labeling model. In application, the generated content is input into the labeling model, and the predicted classification labels output by the model automatically classify whether the generated content is safe or not. Finally, a second leakage penetration rate is calculated based on the automatic labeling results, where the second leakage penetration rate is the ratio of the number of problematic examples that are automatically labeled as unsafe to the total number of problematic examples.
[0069] The first and second leakage penetration rates are collectively referred to as the leakage penetration rate. This leakage penetration rate is used as an evaluation metric to assess the security risk of a large language model. The higher the vulnerability penetration rate, the higher the security risk of the large model; the lower the vulnerability penetration rate, the lower the security risk of the large language model.
[0070] like Figure 4 As shown in the embodiment, a large language model security testing device based on the tabu search algorithm is also provided, including a data construction module, a template optimization module, and a security testing module. The data construction module is used to classify harmful content according to the security specifications of the large language model, and to construct question examples and corresponding target content for each type of harmful content. The template optimization module uses the constructed insecure templates as initialization variables, and based on the initialization variables and the target content used to construct optimization target values, it uses the tabu search algorithm to globally optimize the insecure templates, obtaining optimized insecure templates. The security testing module uses the optimized insecure templates and question examples to form a question prompt, which is input into the large language model to generate content. The security of the generated content is determined by keyword judgment or automatic tag judgment.
[0071] It should be noted that the large language model security testing device based on the tabu search algorithm provided in the above embodiments should be illustrated using the above-described functional module division as an example when conducting security tests. The functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal or server can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the large language model security testing device based on the tabu search algorithm provided in the above embodiments and the large language model security testing method embodiment based on the tabu search algorithm belong to the same concept. For details of its specific implementation process, please refer to the large language model security testing method embodiment based on the tabu search algorithm, which will not be repeated here.
[0072] The embodiment also provides a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the above-described large language model security testing method based on the tabu search algorithm, including the following steps:
[0073] Step 1: According to the security specifications of the large language model, classify harmful content and construct problem examples and corresponding target content for each type of harmful content;
[0074] Step 2: Use the constructed insecure template as an initialization variable. Based on the initialization variable and the target content used to construct the optimization target value, use the tabu search algorithm to globally optimize the insecure template to obtain the optimized insecure template.
[0075] Step 3: The optimized insecure template and the problem example are combined to form a problem prompt, which is then input into the large language model to generate content. The security of the generated content is determined by keyword judgment or automatic tag judgment.
[0076] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A security testing method for large language models based on tabu search algorithm, characterized in that, Includes the following steps: Data construction: Based on the security specifications of the large language model, harmful content is classified, and problem examples and corresponding target content are constructed for each type of harmful content; Template optimization: Using the constructed insecure template as an initialization variable, and based on this initialization variable and with the target content used to construct the optimization target value, a tabu search algorithm is used to globally optimize the insecure template, resulting in an optimized insecure template. This global optimization of the insecure template using the tabu search algorithm includes: (a) Create a tabu list and set its length. Initialize the global optimal solution and local optimal solution. The search objects in the tabu list are unsafe templates. (b) Perform neighborhood search and synonym replacement on the unsafe template to obtain a new unsafe template. When it is determined that the new unsafe template does not exist in the taboo list, input the question example and the new unsafe template into the large language model to obtain the generated content. (c) Calculate the target value based on the generated content and the target content, and assign the target value to the local optimum when the local optimum is greater than the target value. At the same time, update the tabu table using the new insecure template, and then assign the target value to the global optimum when the global optimum is greater than the target value. (d) Repeat steps (b) and (c). After optimization, the final global optimal solution is the final optimized insecure template. Security testing: The optimized insecure template and the problem example are combined to form a problem prompt, which is then input into a large language model to generate content. The security of the generated content is determined by keyword judgment or automatic tag judgment.
2. The method for security testing of large language models based on tabu search algorithm according to claim 1, characterized in that, A neighborhood search and synonym replacement are performed on the insecure template to obtain a new insecure template, including: Starting from the first word in the insecure template, synonym replacement is performed. Each synonym replacement forms a new insecure template. After a word has undergone a preset W synonym replacements, the next word in the insecure template is replaced with a synonym.
3. The method for security testing of large language models based on tabu search algorithm according to claim 1, characterized in that, Determining whether a new unsafe template does not exist in the tabu list includes: Traverse each existing unsafe template in the tabu list. If the new unsafe template is the same as an existing unsafe template, it is considered that the new unsafe template has already been calculated for the target value and does not need to be replaced. Then, return to perform a domain search on the unsafe template to perform synonym replacement and generate another new unsafe template. If the new unsafe template is different from an existing unsafe template, it is considered that the new unsafe template does not exist in the tabu list.
4. The method for security testing of large language models based on tabu search algorithm according to claim 1, characterized in that, Based on generated content response and target content tar Calculate the target value, including: in, This represents the input to the large language model. The token contains the input question example and the new insecure template. The first one generated by the large language model One token, The preceding text indicates the generation of a large language model. The content of each token This represents the probability that the generated content of the large language model is the target content. The objective function is used to calculate the target value.
5. The method for security testing of large language models based on tabu search algorithm according to claim 1, characterized in that, Update the taboo table using the new insecure template, including: Calculate whether the number of unsafe templates in the tabu list is equal to the length of the tabu list. If so, replace the new unsafe template with the first unsafe template in the tabu list to update the tabu list.
6. The method for security testing of large language models based on tabu search algorithm according to claim 1, characterized in that, The security of generated content is determined through keyword analysis or automatic tagging, including: For keyword judgment, if the generated content contains any keyword recorded in the rejection response phrase, the generated content is considered to be a rejection response and the large language model is considered to be safe and compliant; otherwise, it is considered to be illegal and insecure. The first leakage penetration rate is calculated based on the keyword judgment result, where the first leakage penetration rate is the ratio of the number of question examples corresponding to the keyword judgment result of illegal and insecure to the total number of question examples. For automatic label judgment, the generated content is input into a label model built based on supervised learning. The predicted classification label output by the label model is used to automatically classify whether the generated content is safe. The second leakage penetration rate is calculated based on the automatic label judgment result. The second leakage penetration rate is the ratio of the number of problem examples corresponding to violations and insecurity in the automatic label judgment result to the total number of problem examples. The higher the first and second leakage penetration rates, the higher the risk of leakage in the large language model, and the less secure it is.
7. The method for security testing of large language models based on tabu search algorithm according to claim 1, characterized in that, It also includes performing variant operations on problem examples to increase the number of problem examples. These variant operations include synonym transformation, low-resource language transformation, structural transformation, arbitrary insertion and deletion, grammatical transformation, and phrase recombination. Problem examples after variant operations participate in the updating of insecure templates and the security testing of large language models.
8. A security testing device for large language models based on a tabu search algorithm, characterized in that, include: The data construction module is used to classify harmful content according to the security specifications of the large language model, and to build problem examples and corresponding target content for each type of harmful content. The template optimization module uses the constructed insecure template as an initialization variable. Based on this initialization variable and using the target content to construct the optimization target value, it employs a tabu search algorithm to globally optimize the insecure template, resulting in an optimized insecure template. The global optimization of the insecure template using the tabu search algorithm includes: (a) Create a tabu list and set its length. Initialize the global optimal solution and local optimal solution. The search objects in the tabu list are unsafe templates. (b) Perform neighborhood search and synonym replacement on the unsafe template to obtain a new unsafe template. When it is determined that the new unsafe template does not exist in the taboo list, input the question example and the new unsafe template into the large language model to obtain the generated content. (c) Calculate the target value based on the generated content and the target content, and assign the target value to the local optimum when the local optimum is greater than the target value. At the same time, update the tabu table using the new insecure template, and then assign the target value to the global optimum when the global optimum is greater than the target value. (d) Repeat steps (b) and (c). After optimization, the final global optimal solution is the final optimized insecure template. The security testing module takes the optimized insecure template and the problem example, inputs them into the large language model to generate content, and judges the security of the generated content through keyword judgment or automatic tag judgment.
9. A computing device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the one or more processors execute the executable code, they implement the steps of the method for security testing of large language models based on the tabu search algorithm as described in any one of claims 1-7.