Security evaluation method and system for large model code generation capability
Through the adversarial attack idea and adversarial code prompt generation technology, vulnerability code and attribution positioning are used to evaluate the code generation capabilities of the big model, solving the problem that the existing technology cannot comprehensively evaluate the security of the big model, and realizing the black box situation Improved safety evaluation and accuracy.
Patent Information
- Application Number
- CN202411071578.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-06
AI Technical Summary
Existing large-model code generation evaluation methods cannot comprehensively evaluate the security of the model in actual applications, and rely too much on fixed data sets to cover sufficiently rich code and vulnerability types, resulting in the inability to accurately reflect the security of the model.
Adopting adversarial attack idea, through adversarial code prompt generation technology, generate adversarial samples while keeping the semantics unchanged, and use vulnerable code in CVE database and GitHub for cleaning and vectorization, combining attribution positioning and semantic retention mutations. , generate adversarial samples to evaluate the security of code generation capabilities of large models.
It realizes security evaluation of the code generation capability of large models in the black box situation, which can better evaluate the security of the code generation model, reduces the limitations of fixed test samples, and improves the accuracy and coverage of the evaluation.
Smart Images

Figure CN119089411A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer software technology, and mainly relates to a security assessment method and system for large model code generation capabilities, in order to reduce the security risks caused by potential vulnerabilities in large model generated codes. Background Art
[0002] With the rapid development of big model technology, its role in the field of code generation is becoming increasingly important, and it is widely used to solve software engineering tasks such as code generation and code completion. Benefiting from massive data and advanced learning algorithms, big models have shown excellent ability in understanding code semantics and natural language, and can generate program code with a certain degree of usability based on brief description prompts, greatly improving development efficiency. However, this convenience also brings potential security risks. Due to the influence of low-quality training data or misleading code prompts, big models may generate code that is seemingly reasonable but may contain vulnerabilities, which may lead to serious security risks, such as data leakage, remote execution vulnerabilities, etc. Therefore, how to evaluate the security of big model code generation capabilities has become a difficult problem that developers need to solve urgently.
[0003] There are two problems with the current evaluation system for the code generation capabilities of large models: first, it only evaluates the ability of large models to solve code problems without considering their security. For example, the HumanEval evaluation benchmark developed by OpenAI contains a large number of programming problems of different difficulty and types written by human experts. It is used to evaluate AI's ability to solve programming problems. However, it cannot rule out the possibility that the code that correctly passes the problem test may have security vulnerabilities; second, the evaluation method relies too much on fixed data sets, cannot cover a sufficiently rich set of code and vulnerability types, and cannot fully reflect the actual application scenarios. For example, the RobustAPI method only evaluates the reliability performance of large models in the use of some typical APIs.
[0004] In summary, existing evaluation methods may not be able to comprehensively evaluate the performance of the model in practical applications, or may not be able to accurately reflect issues such as the security of the model. Summary of the invention
[0005] In view of the above problems, the purpose of the present invention is to provide a security evaluation method and system for the code generation capability of large models. The present invention uses the idea of adversarial attack, takes the large model as the attack target, and proposes an adversarial code prompt generation technology that retains semantics unchanged. The large model is enabled to generate and complete different prompts while keeping the semantics unchanged. The insecure large model will generate code with vulnerabilities under specific prompts. The security of the large model's code generation capability is evaluated in a black box situation, thereby determining whether the large model is safe.
[0006] The present invention adopts the idea of adversarial attack, takes the large model as the attack target, cleans and vectorizes the vulnerable code, and generates adversarial attack samples through attribution positioning and semantically preserved mutation. The principle of this method is: first, we extract the actual vulnerable code from the CVE database and GitHub, clean and annotate it to build our dataset, and then select the data supplement security results of the target large model (i.e., the code generation model) as the input for searching adversarial samples. Secondly, using gradient information, we calculate the importance score of each token in the prompt to identify the part of the prompt that has the greatest impact on the code output by the model. Finally, based on the importance score, we select high-scoring tokens and apply semantically preserved mutations to generate modified prompts. The new prompt is semantically similar to the original prompt, with the aim of potentially triggering insecure code generation for the large model. Our method is able to generate adversarial code samples that are very similar to the original input to induce the large model to generate vulnerable code. This comprehensive and targeted strategy enables us to better evaluate the security of code generation models, paving the way for developing defense mechanisms to prevent adversarial attacks.
[0007] The steps involved in the above method are further described as follows:
[0008] (1) Sample construction. We extracted vulnerability codes from GitHub and used manual verification to obtain high-quality vulnerability examples. We normalized the vulnerability examples into triple samples in a unified format to construct a dataset for large model evaluation.
[0009] (2) Attribution localization. By performing forward calculations on the target large model, we obtain the gradient of the probability difference of the model output relative to the input token vector representation, and then calculate the attribution matrix. In this way, we can determine which tokens have the most significant impact on the model's prediction, which can greatly reduce the search space when performing semantically preserved mutations on the prompt.
[0010] (3) Semantic Preserving Mutation. In order to evaluate the security of the code generation capability of the target large model, we should test whether they can give consistent outputs for inputs with the same semantics. To this end, through experiments and comparisons, we designed a set of semantic preserving mutation rules, including 14 strategies at five different levels of code granularity. With the help of attribution matrix search, a series of new code hints that are semantically equivalent to the original code hints are generated to test the security of the target large model.
[0011] (4) Model security test. The new code hint obtained through the above process is the adversarial attack sample. The new code hint is directly input into the target large model. The target large model generates and completes lines based on the new code hint, and detects vulnerabilities in the output code. The proportion of unsafe code output by the target large model reflects its security.
[0012] The technical solution of the present invention is:
[0013] A security evaluation method for large model code generation capability, comprising the following steps:
[0014] 1) obtaining multiple vulnerability code instances from a vulnerability database, obtaining two versions of code snippets corresponding to each vulnerability code instance, one of which is an unsafe code before patching and the other is a safe code after patching, forming a vulnerability instance code pair, and obtaining a vulnerability instance code pair set; using the CWE classification method to classify the vulnerability to which each vulnerability code instance belongs, and obtaining a vulnerability category of each vulnerability code instance;
[0015] 2) Extract a triple from each vulnerability instance code pair<p,v,s> , construct a validation set for the target large model evaluation; where p is a section of code before the vulnerability point in the unsafe code, which serves as a prompt for the completion of the target large model code; v is the vulnerability code corresponding to the vulnerability point; s is the code extracted from the safe code after patching the vulnerability point;
[0016] 3) Construct an input sequence w = [w1, w2, ..., wn] of length n according to the extracted triples and input it into the target large model to obtain the corresponding input sequence embedding set e = [e1, e2, ..., en] and the output probability p of each word, where wn is the nth word and the probability that the word is output as a security label lsec is expressed as p sec , the probability that the word output is the vulnerable label lvul is denoted as p vul ; Then calculate the attribution matrix B = [B1, B2, ..., Bn] represents the attribution matrix, where the i-th element Bi in the attribution matrix B corresponds to the partial derivative of the output probability difference of the i-th word with respect to the i-th element ei in the input sequence embedding set e; calculate the norm s of Bi i =||B i || is the importance score of the i-th element wi in the input sequence w, i = 1~n;
[0017] 4) constructing an abstract syntax tree for each vulnerability instance code pair corresponding to the vulnerability code instance, and aggregating the importance scores of each word contained in each syntax node in the abstract syntax tree as a label of the corresponding syntax node;
[0018] 5) The target large model mutates the input prompts of the triple samples in order of importance scores from high to low according to the set mutation strategy to obtain adversarial prompt samples; then, the security of the code generation capability of the target large model is evaluated according to the proportion of unsafe codes in the adversarial prompt samples generated by the target large model.
[0019] Furthermore, the mutation strategy includes 14 mutation strategies at five levels from fine-grained to coarse-grained:
[0020] (1)Token level:
[0021] (a) Identifier replacement: replace the original identifier with a different identifier;
[0022] (2) Expression level:
[0023] (a) Assignment expression line truncation: Add line breaks to separate the contents of the expression into different lines;
[0024] (b)Conditional expression to if statement: Convert the conditional expression to if statement;
[0025] (c) Change the order of conditional expressions: Change the order of conditional expressions;
[0026] (d) Lambda expression to function call: Convert Lambda expression to function definition and call;
[0027] (e) Convert derivation to loop statement: Convert derivation to loop statement;
[0028] (3) Simple Statement level:
[0029] (a) Expression statement assignment: convert the expression statement into an assignment statement;
[0030] (b) Simplification of enhanced assignment statements: converting enhanced assignment statements into ordinary assignment statements;
[0031] (c) Assertion to if statement: Convert the assertion to an if statement;
[0032] (4) Compound Statement level:
[0033] (a) if statement reversal: reverse the condition of the if statement and swap the contents of the if and else blocks;
[0034] (b) While statement jump: When the while statement is always true, add a break statement;
[0035] (c) Convert for statement to while statement: Convert the for statement to while statement;
[0036] (d) Convert with statement to try-finally statement: Convert the with statement to try-finally statement;
[0037] (5) Block level:
[0038] (a) Add API alias: Set an alias for the imported API.
[0039] Furthermore, the unsafe code before patching contains only one line of vulnerable code.
[0040] Furthermore, the vulnerability database includes a CVE database and a GitHub database.
[0041] Furthermore, the target large model is a code generation model.
[0042] A security evaluation system for large model code generation capability, characterized by comprising
[0043] A data collection module is used to obtain multiple vulnerability code instances from a vulnerability database, obtain two versions of code snippets corresponding to each vulnerability code instance, one of which is an unsafe code before patching and the other is a safe code after patching, to form a vulnerability instance code pair, and obtain a vulnerability instance code pair set; use the CWE classification method to classify the vulnerability to which each vulnerability code instance belongs, and obtain the vulnerability category of each vulnerability code instance;
[0044] A data set construction module is used to extract a triple from each vulnerability instance code pair<p,v,s> , construct a validation set for the target large model evaluation; where p is a section of code before the vulnerability point in the unsafe code, which serves as a prompt for the completion of the target large model code; v is the vulnerability code corresponding to the vulnerability point; s is the code extracted from the safe code after patching the vulnerability point;
[0045] The attribution calculation module is used to construct an input sequence w = [w1, w2, ..., wn] with a length of n according to the extracted triples and input it into the target large model to obtain the corresponding input sequence embedding set e = [e1, e2, ..., en] and the output probability p of each word in the vocabulary, where wn is the nth word and the probability that the word is output as a security label lsec is expressed as p sec , the probability that the word output is the vulnerable label lvul is denoted as p vul ; Then calculate the attribution matrix B = [B1, B2, ..., Bn] represents the attribution matrix, where the i-th element Bi in the attribution matrix B corresponds to the partial derivative of the output probability difference of the i-th word with respect to the i-th element ei in the input sequence embedding set e; calculate the norm s of Bi i =||B i || is the importance score of the i-th element wi in the input sequence w, i = 1~n;
[0046] an attribution positioning module, configured to construct an abstract syntax tree for each vulnerability instance code pair corresponding to each vulnerability code instance, and aggregate the importance scores of each word contained in each syntax node in the abstract syntax tree as a label of the corresponding syntax node;
[0047] An evaluation module is used to utilize the target large model to mutate the input prompts of the triple samples in order of importance scores from high to low according to the set mutation strategy to obtain adversarial prompt samples; and then evaluate the security of the code generation capability of the target large model according to the proportion of unsafe codes in the adversarial prompt samples generated by the target large model.
[0048] A server, characterized in that it comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the above method.
[0049] A computer-readable storage medium stores a computer program, wherein the computer program implements the above method when executed by a processor.
[0050] Compared with the prior art, the present invention has the following beneficial effects:
[0051] The present invention uses real-world scenario data to conduct adversarial attacks on the model while keeping the semantics of the code prompts unchanged, and can make a true and reliable assessment of the security of the code generation capability of a large model in a black box situation. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 It is a system technology flow chart of the present invention. DETAILED DESCRIPTION
[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below through specific examples.
[0054] Unlike most current large model evaluation technologies that use fixed test samples to determine whether the model's code generation capability is secure, the present invention considers this issue from the perspective of countering attacks: if the large model cannot provide consistent and secure code completion results for code prompts with the same semantics, then it may have security issues. Through this method, the present invention reduces the impact of the limitations of fixed test samples, and the specific implementation process is as follows:
[0055] (1) Data collection: Since a certain amount of sample data sets are required to conduct adversarial attacks, in the present invention, vulnerabilities with vulnerability code instances (such as Github links) are first collected from the vulnerability database, and then two versions of code snippets are collected from the instance code using a crawler tool, one of which is the unsafe code before patching and the other is the safe code after patching, forming a vulnerability instance code pair. In order to make the target large model have higher accuracy when generating code, only the code pairs with single-line modifications are retained, that is, one line in the code before patching caused the vulnerability, and the modification of this line patched the vulnerability. The code is then reviewed and verified to ensure that the code pair is a patch for the corresponding vulnerability, and the CWE classification method is used to classify the vulnerability to which each instance belongs, so that a comprehensive analysis can be performed based on the vulnerability type.
[0056] (2) Dataset Construction: After obtaining vulnerability instance data, the present invention defines a triple<p,v,s> To construct a validation set for large model evaluation. Where p is a section of code before the vulnerability point, which serves as a hint for the completion of the target large model code, that is, the section of code p before the vulnerability point is used as a hint for the completion of the target model code; v is the vulnerable line code in the unsafe code before patching; s is the secure code after the vulnerable line is patched. Each vulnerability instance is normalized into a triple according to the definition to obtain the evaluation data set {x0, x1, ..., xn} for subsequent adversarial sample generation. Where xi is the i-th specific triple, i = 1~n, and n is the total number of triplets in the validation set, for example: <p:"from lxml import etree\nfrom lxml.etree import CDATA\n……def_add_text_elm(entry,data,name):\n\n\tifnot data:\n\t\tif not data:\n\t\t\treturn\n\n\telm=",v:"etree.SubElement(entry,name)",s:"xml_elem(name,entry)"> .
[0057] (3) Attribution calculation: Before performing semantic-preserving mutations on the validation set samples to generate adversarial prompt samples, in order to reduce the search space for mutations and improve the effectiveness of the generated adversarial prompt generation, we use attribution-guided positioning to determine the key tokens in the prompt that have a significant impact on the model output. Specifically, we use the gradient of the model output relative to the input token embedding to help us determine which tokens have the most significant impact on the model's predictions. First, we perform a forward calculation on the target large model. Assume that the target large model is F(·). Input an input sequence w = [w1, w2, ..., wn] of length n into the model, and we can obtain the input sequence embedding set e = [e1, e2, ..., en] and the output probability p of each word in the input sequence in the vocabulary of the target large model, as shown below:
[0058] p = F(w) For the secure output label lsec and the vulnerable output label lvul, their probabilities in the vocabulary are denoted by p sec and p vul :
[0059] p sec =P sec (F(w))p vul =P vul (F(w))
[0060] The attribution matrix B is directly generated by calculating the following partial derivatives:
[0061]
[0062] Here, B = [B1, B2, ..., Bn] represents the attribution matrix, where each element Bi corresponds to the partial derivative of the probability difference with respect to the input sequence embedding ei. To obtain the final importance score S = [s0, s1, ..., sn], we calculate the norm s of each element in the attribution matrix Bi i As the importance score of the i-th element wi in the input sequence w, i = 1~n:
[0063]
[0064] After the above calculations, we get the importance score of the input sequence w.
[0065] (4) Attribution positioning: The importance score of the input sequence w calculated by the above method is based on the tokenization result of the large model, which may be different from the actual code syntax node, that is, the score may be assigned to a single token that does not have a complete syntax. For example, the function name "_add_text_elm" is recognized by the model tokenizer as "_", "add", "_", "text", "_", "elm", but it should be recognized as "_add_text_elm" in the python tokenizer. To this end, by accumulating the importance scores of the tokens that are additionally split by the model tokenizer to the same syntax token node, we construct an abstract syntax tree of the code and aggregate the importance scores recognized by the model to the single syntax node token corresponding to the abstract syntax tree (AST). In this way, attribution positioning is completed, and the impact scores of different tokens in the code snippet on the model output are obtained. The present invention constructs an abstract syntax tree for each vulnerability instance code pair corresponding to each vulnerability code instance, and aggregates the importance scores of each word contained in each syntax node in the abstract syntax tree as the label of the corresponding syntax node.
[0066] (5) Semantic Preserving Mutation: In order to generate adversarial hint samples to check whether the model can maintain consistent secure code completion for code hints with the same semantics, this paper develops a set of semantic preserving mutation methods that can generate a series of new code hints that are semantically equivalent to the original code hints. Specifically, based on the Python language reference, we designed 14 mutation strategies at 5 levels from fine-grained (token) to coarse-grained (block):
[0067] 1)Token level:
[0068] (a) Identifier replacement: Replace the original identifier with a different identifier.
[0069] 2) Expression level:
[0070] (a) Assignment expression line truncation: Add newline characters to separate the contents of the expression into different lines.
[0071] (b)Conditional expression to if statement: Convert the conditional expression to if statement.
[0072] (c) Reversing the order of conditional expressions: Reversing the order of conditional expressions.
[0073] (d) Lambda expression to function call: Convert Lambda expression to function definition and call.
[0074] (e) Convert derivation to loop statement: Convert the derivation to a loop statement.
[0075] 3) Simple Statement level:
[0076] (a) Expression statement assignment: Convert an expression statement into an assignment statement.
[0077] (b) Simplification of enhanced assignment statements: Convert enhanced assignment statements into ordinary assignment statements.
[0078] (c) Assertion to if statement: Convert the assertion to if statement.
[0079] 4) Compound Statement level:
[0080] (a) If statement reversal: Reverse the condition of the if statement and swap the contents in the if and else blocks.
[0081] (b) Break out of while statement: When the while statement is always true, add a break statement.
[0082] (c) Convert for statement to while statement: Convert the for statement to while statement.
[0083] (d) Convert with statement to try-finally statement: Convert the with statement to try-finally statement.
[0084] 5) Block level:
[0085] (a) Add API alias: Set an alias for the imported API.
[0086] Through the above mutation strategy, we mutate the input prompts of the triple samples in the dataset in order of importance scores from high to low to obtain adversarial prompt samples. By using the samples to enable the target model to complete the code, the proportion of unsafe code generated by the target large model can evaluate the security of its generation ability.
[0087] In summary, the present invention designs a security assessment method for large model code generation capabilities based on adversarial attacks. It uses the output vulnerability code to simulate the real usage scenario, adopts the code semantics-preserving mutation method to generate adversarial attack samples, reduces the impact of fixed test cases, and adopts the attribution positioning method to reduce the mutation search space, thereby improving the efficiency of adversarial attack evaluation. Experiments show that the method proposed in the present invention can effectively and automatically generate anti-analysis samples and evaluate the security of large model code generation capabilities.
[0088] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to the claims.
Claims
1. A security evaluation method for large model code generation capability, comprising the following steps: 1) obtaining multiple vulnerability code instances from a vulnerability database, obtaining two versions of code snippets corresponding to each vulnerability code instance, one of which is an unsafe code before patching and the other is a safe code after patching, forming a vulnerability instance code pair, and obtaining a vulnerability instance code pair set; using the CWE classification method to classify the vulnerability to which each vulnerability code instance belongs, and obtaining a vulnerability category of each vulnerability code instance; 2) Extract a triple from each vulnerability instance code pair<p,v,s> , build a validation set for evaluating the target large model; Where p is a section of code before the vulnerability point in the unsafe code, which serves as a prompt for the completion of the target large model code; v is the vulnerability code corresponding to the vulnerability point; s is the code extracted from the safe code after patching the vulnerability point; 3) Construct an input sequence w = [w1, w2, ..., wn] of length n according to the extracted triples and input it into the target large model to obtain the corresponding input sequence embedding set e = [e1, e2, ..., en] and the output probability p of each word, where wn is the nth word and the probability that the word is output as a security label lsec is expressed as p sec , the probability that the word output is the vulnerable label lvul is denoted as p vul ; Then calculate the attribution matrix B = [B1, B2, ..., Bn] represents the attribution matrix, where the i-th element Bi in the attribution matrix B corresponds to the partial derivative of the output probability difference of the i-th word with respect to the i-th element ei in the input sequence embedding set e; calculate the norm s of Bi i =||B i || as the importance score of the i-th element wi in the input sequence w, i = 1~n; 4) constructing an abstract syntax tree for each vulnerability instance code pair corresponding to the vulnerability code instance, and aggregating the importance scores of each word contained in each syntax node in the abstract syntax tree as a label of the corresponding syntax node; 5) The target large model mutates the input prompts of the triple samples in order of importance scores from high to low according to the set mutation strategy to obtain adversarial prompt samples; then, the security of the code generation capability of the target large model is evaluated according to the proportion of unsafe codes in the adversarial prompt samples generated by the target large model.
2. The method according to claim 1, characterized in that The mutation strategy includes 14 mutation strategies at five levels from fine-grained to coarse-grained: (1)Token level: (a) Identifier replacement: replace the original identifier with a different identifier; (2) Expression level: (a) Assignment expression line truncation: Add line breaks to separate the contents of the expression into different lines; (b)Conditional expression to if statement: Convert the conditional expression to if statement; (c) Change the order of conditional expressions: Change the order of conditional expressions; (d) Lambda expression to function call: Convert Lambda expression to function definition and call; (e) Convert derivation to loop statement: Convert derivation to loop statement; (3) Simple Statement level: (a) Expression statement assignment: convert the expression statement into an assignment statement; (b) Simplification of enhanced assignment statements: converting enhanced assignment statements into ordinary assignment statements; (c) Assertion to if statement: Convert the assertion to an if statement; (4) Compound Statement level: (a) if statement reversal: reverse the condition of the if statement and swap the contents of the if and else blocks; (b) While statement jump: When the while statement is always true, add a break statement; (c) Convert for statement to while statement: Convert the for statement to while statement; (d) Convert with statement to try-finally statement: Convert the with statement to try-finally statement; (5) Block level: (a) Add API alias: Set an alias for the imported API.
3. The method according to claim 1, characterized in that The unsafe code before patching contains only one line of vulnerable code.
4. The method according to claim 1, characterized in that The vulnerability database includes a CVE database and a GitHub database.
5. The method according to claim 1, characterized in that The target large model is a code generation model.
6. A security assessment system for large model code generation capability, characterized in that: include A data collection module is used to obtain multiple vulnerability code instances from a vulnerability database, obtain two versions of code snippets corresponding to each vulnerability code instance, one of which is an unsafe code before patching and the other is a safe code after patching, to form a vulnerability instance code pair, and obtain a vulnerability instance code pair set; use the CWE classification method to classify the vulnerability to which each vulnerability code instance belongs, and obtain the vulnerability category of each vulnerability code instance; A data set construction module is used to extract a triple from each vulnerability instance code pair<p,v,s> , build a validation set for evaluating the target large model; Where p is a section of code before the vulnerability point in the unsafe code, which serves as a prompt for the completion of the target large model code; v is the vulnerability code corresponding to the vulnerability point; s is a code extracted from the security code after patching the vulnerability point; The attribution calculation module is used to construct an input sequence w = [w1, w2, ..., wn] with a length of n according to the extracted triples and input it into the target large model to obtain the corresponding input sequence embedding set e = [e1, e2, ..., en] and the output probability p of each word in the vocabulary, where wn is the nth word and the probability that the word is output as a security label lsec is expressed as p sec , the probability that the word output is the vulnerable label lvul is denoted as p vul ; Then calculate the attribution matrix B = [B1, B2, ..., Bn] represents the attribution matrix, where the i-th element Bi in the attribution matrix B corresponds to the partial derivative of the output probability difference of the i-th word with respect to the i-th element ei in the input sequence embedding set e; calculate the norm s of Bi i =||B i || as the importance score of the i-th element wi in the input sequence w, i = 1~n; an attribution positioning module, configured to construct an abstract syntax tree for each vulnerability instance code pair corresponding to each vulnerability code instance, and aggregate the importance scores of each word contained in each syntax node in the abstract syntax tree as a label of the corresponding syntax node; An evaluation module is used to utilize the target large model to mutate the input prompts of the triple samples in order of importance scores from high to low according to the set mutation strategy to obtain adversarial prompt samples; and then evaluate the security of the code generation capability of the target large model according to the proportion of unsafe codes in the adversarial prompt samples generated by the target large model.
7. A server, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Method and device for detecting vulnerability caused by imperfect patch and medium
CN117131499A
Source code vulnerability fine-grained identification method based on vulnerability patch pairs
CN117272323A
Automatic vulnerability mining technology based on graph neural network and vulnerability patch relationship
CN117473510A
Payment information security management method and system based on block chain
CN117592989A
Cited By
Prison break attack test method for multi-mode large model
CN119740229A
Assessment and defense method aiming at code language model confrontation attack
CN119960738A
An evaluation and defense method against code language model attack
CN119960738B
Large-model full-life-cycle automatic attack testing and security evaluation method and system
CN122263113A
