Vulnerability benchmark test set automatic construction method and system based on patch mode revocation

Abstract

The invention discloses a patch mode revocation-based vulnerability benchmark test set automatic construction method and system, and the method comprises the steps: 1, obtaining a target program, and collecting historical vulnerabilities in other versions of programs of a project where the target program is located; 2, searching a historical vulnerability patch related to the historical vulnerability in a patch database; 3, locating a bug patch in a target program according to the historical bug patch and the historical bug; and 4, transplanting the historical vulnerabilities to the positions of the vulnerability patches in the target program according to a vulnerability patch revocation rule to obtain a real vulnerability test set. According to the method, automatic construction of the real vulnerability test set is realized through vulnerability patch positioning and vulnerability patch revocation rules, and a more real and comprehensive test benchmark is provided for evaluation of a fuzzy test tool.

Description

Technical Field

[0001] The present invention relates to the technical field of binary fuzzy testing and evaluation, and in particular to a method and system for automatically constructing a vulnerability benchmark test set based on patch mode revocation. Background Art

[0002] Real vulnerabilities are defective codes written manually by human programmers. At present, the technology involved in building real vulnerability test sets usually adopts manual analysis methods to forward transplant vulnerabilities. Such methods require researchers to perform manual analysis and take up a lot of time. At the same time, there is currently a lack of methods to locate vulnerability patches on different versions of programs. With the update and iteration of the software, it is impossible to guarantee that the vulnerabilities in the test set are updated, which does not meet the standards of the test set. There is also a lack of automated revocation methods for vulnerability patches. Vulnerability patches need to be converted into vulnerability codes through code conversion. At present, there is a lack of summary of the revocation mode of vulnerability patches. In addition to the automated revocation method, vulnerability patches also need to be assisted by extraction revocation methods to improve the success rate of vulnerability patch revocation. Summary of the invention

[0003] In order to at least partially solve the problems of difficulty in locating vulnerability patches in different versions of programs, inability to automatically revoke vulnerability patches and lack of summary of vulnerability patch revocation rules, the present invention provides a method and system for automatically constructing a vulnerability benchmark test set based on patch mode revocation, which locates vulnerability patches through historical vulnerability information, summarizes vulnerability patch revocation rules and uses them to automatically generate vulnerabilities, thereby realizing automatic revocation of vulnerability patches.

[0004] In order to achieve the above object, the technical solution of the present invention is:

[0005] The first aspect of the present invention proposes a method for automatically constructing a vulnerability benchmark test set based on patch mode revocation, comprising the following steps:

[0006] Step 1: Obtain the target program and collect historical vulnerabilities in other versions of the project where the target program is located, so as to facilitate the location of historical vulnerabilities in the target program;

[0007] Step 2: Searching for historical vulnerability patches related to the historical vulnerability in the patch database to facilitate subsequent vulnerability transplantation;

[0008] Step 3: locating the vulnerability patch in the target program according to the historical vulnerability patch and the historical vulnerability, so as to facilitate subsequent vulnerability transplantation;

[0009] Step 4: transplant the historical vulnerability to the location of the vulnerability patch in the target program according to the vulnerability patch revocation rule to obtain a real vulnerability test set.

[0010] Furthermore, the step three specifically includes:

[0011] Extract the difference code between the historical vulnerability and the historical vulnerability patch, find the location of the difference code in the target program, and obtain multiple similar code blocks, so as to facilitate the positioning of the vulnerability patch of the target program;

[0012] Detect multiple similar code blocks according to the GumTree algorithm to obtain similar code blocks required to form the historical vulnerability patch, and use the similar code blocks required to form the historical vulnerability patch as candidate patches, so as to locate the vulnerability patch position of the target program according to the candidate patches;

[0013] According to the grammatical distance between the candidate patch and the historical vulnerability patch, it is determined whether the candidate patch is a vulnerability patch of the target program, so as to accurately find the vulnerability patch of the target program.

[0014] Further, the detecting of multiple similar code blocks according to the GumTree algorithm to obtain similar code blocks required to constitute the historical vulnerability patch and using the similar code blocks required to constitute the historical vulnerability patch as candidate patches specifically includes:

[0015] Convert multiple similar code blocks and historical vulnerability patches into abstract syntax trees;

[0016] According to the tree editing operation in the GumTree algorithm, it is determined whether the abstract syntax trees of multiple similar code blocks can be successfully converted into the abstract syntax trees of historical vulnerability patches, and the successfully converted similar code blocks are used as candidate patches.

[0017] Furthermore, the determining whether the candidate patch is a vulnerability patch of the target program according to the grammatical distance between the candidate patch and the historical vulnerability patch specifically includes:

[0018] Convert candidate patches and historical vulnerability patches into abstract syntax trees;

[0019] The grammatical distance between the abstract syntax tree of the candidate patch and the abstract syntax tree of the historical vulnerability patch is calculated, and the candidate patch with the closest grammatical distance is used as the vulnerability patch of the target program to accurately locate the vulnerability patch of the target program.

[0020] Furthermore, the step 4 specifically includes:

[0021] Construct vulnerability patch revocation rules according to different vulnerability revocation structures, and generate revocation patches according to the vulnerability patch revocation rules to facilitate patch revocation;

[0022] The vulnerability patches in the target program are converted using the undo patch to obtain the program containing the vulnerability code as the sample in the real vulnerability test set.

[0023] Furthermore, when the vulnerability patch in the target program is converted by using the undo patch, a complex undo rule is split into a plurality of simple undo rules.

[0024] Furthermore, the step 4 also includes:

[0025] Prune the code irrelevant to the vulnerability patch code of the target program to remove irrelevant code;

[0026] Compile and verify the undo patch to ensure the validity of the undo patch;

[0027] Recursively add the missing variables, functions, and parameters in the program containing the vulnerable code to obtain a complete set of real vulnerability tests.

[0028] The second aspect of the present invention proposes a vulnerability benchmark test set automatic construction system based on patch mode revocation, comprising:

[0029] The first acquisition module is used to obtain the target program and collect historical vulnerabilities in other versions of the project where the target program is located, so as to facilitate the location of historical vulnerabilities in the target program;

[0030] The second acquisition module is used to search the patch database for historical vulnerability patches related to the historical vulnerability, so as to facilitate subsequent vulnerability transplantation;

[0031] A positioning module, used to locate the vulnerability patch in the target program according to the historical vulnerability patch and the historical vulnerability, so as to facilitate subsequent vulnerability transplantation;

[0032] The transplantation module is used to transplant the historical vulnerability to the location of the vulnerability patch in the target program according to the vulnerability patch revocation rule to obtain a real vulnerability test set.

[0033] The third aspect of the present invention proposes a terminal device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements a method for automatically constructing a vulnerability benchmark test set based on patch mode revocation as described in the first aspect above.

[0034] The fourth aspect of the present invention proposes a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a method for automatically constructing a vulnerability benchmark test set based on patch mode revocation as described in the first aspect above.

[0035] Beneficial effects of the present invention:

[0036] (1) The method for automatically constructing a vulnerability benchmark test set based on patch mode revocation proposed by the present invention is applicable to various programming languages ​​and has universality. Regardless of the programming language used by the target program, there will be corresponding vulnerability information, and historical vulnerabilities can be transplanted accordingly.

[0037] (2) The present invention proposes an automated construction method suitable for real vulnerability test sets, which can effectively identify and convert historical vulnerabilities and key changes in historical vulnerability patch codes, thereby realizing forward migration of historical vulnerabilities. It can efficiently integrate historical vulnerabilities on the same version of the target program and quickly construct corresponding real vulnerability test sets, which is convenient for evaluating the effectiveness of vulnerability detection tools.

[0038] (3) The present invention summarizes vulnerability patch revocation rules according to different vulnerability revocation structures, which facilitates the generation of revocation patches and improves the success rate of vulnerability patch revocation.

[0039] (4) The present invention uses the GumTree algorithm to compare the abstract syntax tree of the code, accurately captures the code blocks that need to be transplanted, can accurately obtain the current vulnerability patch location in different versions of the program, and automatically completes the forward transplantation process of the vulnerability code. Through the automated vulnerability forward transplantation technology, the quality of the vulnerability test set can be greatly improved, providing a more realistic and comprehensive test benchmark for the evaluation of fuzzy testing tools. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 One of the flowcharts of a method for automatically constructing a vulnerability benchmark test set based on patch mode revocation provided by an embodiment of the present invention.

[0041] Figure 2 The present invention provides a second flowchart of a method for automatically constructing a vulnerability benchmark test set based on patch mode revocation according to an embodiment of the present invention.

[0042] Figure 3 An architectural diagram of a system for automatically building a vulnerability benchmark test set based on patch mode revocation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Example 1

[0045] like Figure 1 As shown, a method for automatically constructing a vulnerability benchmark test set based on patch mode revocation includes the following steps:

[0046] S101: Obtain a target program and collect historical vulnerabilities in other versions of the project where the target program is located.

[0047] S102: Searching for historical vulnerability patches related to the historical vulnerability in a patch database.

[0048] S103: Locate a vulnerability patch in a target program according to the historical vulnerability patch and the historical vulnerability.

[0049] S104: transplanting the historical vulnerability to the location of the vulnerability patch in the target program according to the vulnerability patch revocation rule to obtain a real vulnerability test set.

[0050] The method for automatically constructing a vulnerability benchmark test set based on patch mode revocation proposed by the present invention is applicable to the construction of real vulnerability test sets of multiple open source software. The present invention locates vulnerability patches in a target program through historical vulnerabilities in other version programs of the project where the target program is located and historical vulnerability patches related to historical vulnerabilities, which greatly improves the efficiency of researchers and solves the problem of locating vulnerability patches in the target program. The present invention generates a revocation patch by setting a vulnerability patch revocation rule, and uses the revocation patch to revoke the vulnerability patch in the target program to obtain a program containing vulnerability code, thereby realizing the automation of vulnerability forward transplantation.

[0051] Example 2

[0052] Based on the above embodiments, Figure 2 As shown, the embodiment of the present invention provides a specific process of a method for automatically constructing a vulnerability benchmark test set based on patch mode revocation, which specifically includes:

[0053] S201: Obtain a target program, and collect historical vulnerabilities in other versions of the project where the target program is located.

[0054] Specifically, a target program is obtained, and other versions of the program in the project where the target program is located are found based on the target program, historical vulnerabilities are collected in other versions of the program in the project where the target program is located, and are saved in the form of json.

[0055] S202: Searching for historical vulnerability patches related to the historical vulnerability in a patch database.

[0056] It is understandable that the patch database is pre-built, such as collecting various types of historical vulnerability patches from open source software libraries (such as Github, gitlab, and SVN, etc.) and storing them in a database to form a patch database. This embodiment does not limit the storage method of historical vulnerability patches in the patch database. Preferably, the historical vulnerability patches can be classified and stored according to the vulnerability type to which the patch is applicable.

[0057] Specifically, in actual application, each vulnerability patch in the patch database can be automatically or manually analyzed, and only when the vulnerability patch in the patch database is identified as a patch for a historical vulnerability in another version of the project where the target program is located, the patch is considered to be a historical vulnerability patch related to the historical vulnerability. It can be understood that each historical vulnerability has a corresponding historical vulnerability patch.

[0058] S203: Locate a vulnerability patch in a target program according to the historical vulnerability patch and the historical vulnerability.

[0059] Specifically, the purpose of vulnerability patch location is to find the location of the vulnerability patch code in the target program. Therefore, given a section of vulnerability patch code, it is necessary to identify the location of the vulnerability patch code in the target program. The present invention performs vulnerability patch location on the abstract syntax tree.

[0060] The abstract syntax tree is a tree representation of the abstract syntax structure of the source code. It is also an intermediate representation widely used in the process of processing programs. Its structure does not depend on the specific grammar. Each node on the syntax tree corresponds to a structure in the source code. The abstract syntax tree nodes are shown in Table 1:

[0061] Table 1 Abstract Syntax Tree Nodes

[0062]

[0063] A prominent feature of the abstract syntax tree is that it does not rely on specific language details. Its structure is similar to a tree structure, and each node has a corresponding syntax structure. Existing ordinary programs often have tens of thousands of lines of code. The abstract syntax tree of the source code of this amount of code has a large amount of information. After extracting the abstract syntax tree of the program, for later analysis, it is necessary to traverse the abstract syntax tree to obtain statistically significant information in the abstract syntax tree. Since there are many different types of nodes in the abstract syntax tree, these nodes are all instances of the decl or stmt class. When traversing the abstract syntax tree, different operations need to be performed on different nodes. In order to better parse the abstract syntax tree, the node search process of the abstract syntax tree needs to be further processed. Therefore, the corresponding code location is found by pattern matching the traversal of the abstract syntax tree. In order to more effectively perform pattern matching of the abstract syntax tree, several pattern types required for analysis in the method are extracted and analyzed here. These pattern types include the following: variable description (vardecl), function declaration (funcdecl), operator (typedecl), call function (callexpr), etc.

[0064] The historical vulnerabilities and historical vulnerability patches are input into the SosReverter tool (a code tool developed based on the GumTree algorithm), the difference codes between the historical vulnerabilities and the historical vulnerability patches are obtained, the location of the difference codes is found in the target program, and multiple similar code blocks are obtained.

[0065] A plurality of similar code blocks are detected according to the GumTree algorithm to obtain similar code blocks required to constitute the historical vulnerability patch, and the similar code blocks required to constitute the historical vulnerability patch are used as candidate patches.

[0066] Specifically, the GumTree algorithm is a tree-based technology that is specifically used to analyze the differences between two abstract syntax trees (ASTs) and can extract fine-grained editing operations between the two. It effectively supports the tracking and analysis of code changes by calculating the specific editing steps required to convert from one AST to another. The GumTree algorithm defines four basic tree editing operations: deletion, insertion, movement, and update. Under the GumTree algorithm, for nodes that cannot find a corresponding relationship between the two abstract syntax trees, they are processed as deletion and insertion respectively. Those nodes that do not correspond to their parent nodes but can correspond to themselves are regarded as move operations. In addition, if two nodes correspond but are located in different subtree positions of their respective parent nodes (such as the exchange of parameter order), they are also regarded as moves. At the same time, if there is a difference in the values ​​of the two corresponding nodes, they are considered to have undergone an update operation, which is sometimes accompanied by the movement of nodes. The node pairs involved in the above editing operations are marked as modified, and all other node pairs that remain unchanged are regarded as unmodified. Furthermore, if all nodes in a subtree are not modified, the entire subtree will also be regarded as unchanged.

[0067] The present invention converts multiple similar code blocks and historical vulnerability patches into abstract syntax trees. According to the tree editing operation in the GumTree algorithm, it is determined whether the abstract syntax trees of multiple similar code blocks can be successfully converted into abstract syntax trees of historical vulnerability patches, and the successfully converted similar code blocks are used as candidate patches.

[0068] The GumTree algorithm can abstract many code details that are not related to the patch, reduce the impact of irrelevant code, and improve the scalability of vulnerability patch positioning. In order to find the correct vulnerability patch among the candidate patches, the present invention adopts a clone detection method to solve the vulnerability patch positioning problem. Code clone detection mainly includes three key steps: split the code into comparison units, such as functions, code blocks, classes, etc. Convert the comparison units into intermediate expressions, such as tokens, AST, PDG, etc. Organize the intermediate expressions of these comparison units into clone pairs for similarity comparison to obtain the final detection results. The basic problem of clone detection is to find code segments with the same calculation results.

[0069] In the vulnerability patch positioning of the present invention, the clone types involved are mainly: Type 1, Type 2, Type 3 and Type 4. Among them, Type 4 mainly refers to the reconstruction of the code in the project, which is relatively rare in the historical vulnerability patch code. The clone detection adopted by the present invention is carried out on the basis of AST. The first step is to convert both the candidate patch and the historical vulnerability patch into AST, and then use the algorithm to find similar codes. Compared with other codes, the vulnerability patch code will still be the code with the smallest difference even if it changes with the version update iteration, so it will still be given priority when selecting. Based on this result, the present invention designs a strategy for prioritizing possible candidate patch locations according to grammatical distance.

[0070] The method adopted by the present invention is to calculate the grammatical distance between the abstract syntax tree of the candidate patch and the abstract syntax tree of the historical vulnerability patch, and use the candidate patch with the closest grammatical distance as the vulnerability patch of the target program. The candidate patch ranking process is expressed as follows:

[0071]

[0072] Among them, edit_dustabce(N fix ,N candidate ) is the syntax distance between the abstract syntax tree of the candidate patch and the abstract syntax tree of the historical vulnerability patch, N fix For vulnerability patches, N candidate is a candidate patch, cost(r1→r2) is the replacement cost from r1 to r2, r1 is the root node corresponding to the vulnerability patch, and r2 is the root node corresponding to the candidate patch. is the count vector of vulnerability patches, is the count vector of candidate patches, v fix,i For N fix The number of times the i-th type appears in v candidate,i For N candidate The number of occurrences of the i-th type in the , n is the total number of candidate patches, Sim cos is the cosine similarity function, S i and P i They are the subtrees of the abstract syntax tree of the vulnerability patch and the abstract syntax tree of the candidate patch respectively.

[0073] S204: transplanting the historical vulnerability to the location of the vulnerability patch in the target program according to the vulnerability patch revocation rule to obtain a real vulnerability test set.

[0074] Specifically, the present invention converts the vulnerability patch and the historical vulnerability patch of the target program into AST, extracts the differences between the abstract syntax tree of the vulnerability patch of the target program and the abstract syntax tree of the historical vulnerability patch, and converts the extracted differences according to the anti-unification technology to make them into a unified abstract syntax tree form, which is convenient for vulnerability transplantation.

[0075] Since different versions of the same project of the target program share similar expressions, algorithms, namespaces, etc., the similarities between versions can help guide the construction of vulnerability patch revocation rules. Assuming that the version and the fixed version correspond to the historical vulnerability, the target program is the version that needs to be forward-ported for the historical vulnerability. Through the analysis of existing samples, it can be seen that the new version and the old version match each other, and there are differences in statements. This statement difference can help construct vulnerability patch revocation rules and generate revocation patches based on the vulnerability patch revocation rules.

[0076] At present, the present invention summarizes the collected revocation patterns and evaluates the revocation patterns from four qualitative dimensions, as shown in Table 2:

[0077] Table 2 Summary of undo modes

[0078]

[0079] Type II means that the namespace has changed, and the historical vulnerability needs to be transplanted by modifying the function name and variable name. Type III means that some statements need to be added or deleted to transplant the historical vulnerability. Type IV means that the code line needs to be added, deleted or moved to transplant the historical vulnerability. Type V means that adding, deleting or moving the code will cause the change of the overall code logic structure, such as deleting if operations.

[0080] For overly complex vulnerability patches, the vulnerability patch revocation rules generated by the present invention may not be sufficient to cope with such situations, especially when the vulnerability patch has a complex logical structure. In order to solve this problem, the present invention designs an option to split the V-type revocation rules and extract the existing II, III, and IV-type revocation rules from them.

[0081] According to the unified abstract syntax tree obtained by the de-consistency technology, its node mapping and namespace mapping are obtained. According to the node mapping and namespace mapping, the type is determined by referring to the vulnerability patch revocation rule, and then the revocation patch is generated according to the vulnerability patch revocation rule. The revocation patch is used to convert the vulnerability patch in the target program to obtain the program containing the vulnerability code as a sample in the real vulnerability test set.

[0082] Preferably, the converted AST may still be incomplete according to the fact that the revocation patch may not be enough to successfully convert the vulnerability code. In order to make it complete, the following processing will be performed: the code unrelated to the vulnerability patch code of the target program is pruned. Specifically, when the historical vulnerability is transplanted to the target program, the target program may contain some specific version changes. For example, the submission in the target program may move a code statement from one position to another, or the code statement no longer exists in the main version. The present invention processes each change introduced by the vulnerability code separately to detect the code that should be pruned. If the alignment position of the modified statement cannot be found, the corresponding code is deleted. The present invention compiles and detects the revocation patch generated according to the vulnerability patch revocation rule to check whether the revocation patch has a construction error. The transplantation of historical vulnerabilities involves the revocation of vulnerability patches. Since the target program usually selects the latest version, the deletion operation is mostly, so there are rarely missing variables, functions and parameters. Be cautious when the added code appears. It is necessary to first find the missing variables, functions and macro references, and then recursively add the missing definitions to solve the problem. Finally, a complete program containing vulnerability code is obtained, and the real vulnerability test set is built.

[0083] Example 3

[0084] Corresponding to the above method, such as Figure 3 As shown, the present invention proposes a vulnerability benchmark test set automatic construction system based on patch mode revocation, including:

[0085] The first acquisition module is used to obtain the target program and collect historical vulnerabilities in other versions of the project where the target program is located.

[0086] The second acquisition module is used to search the patch database for historical vulnerability patches related to the historical vulnerability.

[0087] A positioning module is used to locate the vulnerability patch in the target program according to the historical vulnerability patch and the historical vulnerability.

[0088] The transplantation module is used to transplant the historical vulnerability to the location of the vulnerability patch in the target program according to the vulnerability patch revocation rule to obtain a real vulnerability test set.

[0089] It should be noted that the system for automatically building a vulnerability benchmark test set based on patch mode revocation provided in an embodiment of the present invention is to realize the above-mentioned method for automatically building a vulnerability benchmark test set based on patch mode revocation. Its specific functions can be referred to the above-mentioned method embodiments, which will not be repeated here.

[0090] Example 4

[0091] Based on the above embodiments, an embodiment of the present invention further provides a terminal device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a method for automatically constructing a vulnerability benchmark test set based on patch mode revocation in the above embodiments is implemented.

[0092] Preferably, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a method for automatically constructing a vulnerability benchmark test set based on patch mode revocation in the above-mentioned embodiment.

[0093] In summary, the automatic construction method of a vulnerability benchmark test set based on patch mode revocation proposed by the present invention is applicable to various programming languages ​​and has universality. Regardless of the programming language used by the target program, there will be corresponding vulnerability information, and the historical vulnerabilities can be transplanted accordingly. The present invention proposes an automatic construction method suitable for a real vulnerability test set, which can effectively identify and convert key changes in historical vulnerabilities and historical vulnerability patch codes, thereby realizing the forward transplantation of historical vulnerabilities, and can efficiently integrate historical vulnerabilities on the same version of the target program, quickly construct corresponding real vulnerability test sets, and facilitate the evaluation of the effect of vulnerability detection tools. The present invention summarizes the vulnerability patch revocation rules according to different vulnerability revocation structures, facilitates the generation of revocation patches, and improves the success rate of vulnerability patch revocation. The present invention uses a comparison of the abstract syntax tree of the code based on the GumTree algorithm, accurately captures the code blocks that need to be transplanted, can accurately obtain the current vulnerability patch position in different versions of the program, and automatically completes the forward transplantation process of the vulnerability code. Through the automated vulnerability forward transplantation technology, the quality of the vulnerability test set can be greatly improved, providing a more real and comprehensive test benchmark for the evaluation of fuzzy testing tools.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically constructing a vulnerability benchmark test set based on patch mode revocation, characterized in that: The following steps are involved: Step 1: Obtain the target program and collect historical vulnerabilities in other versions of the project where the target program is located; Step 2: Searching for historical vulnerability patches related to the historical vulnerability in a patch database; Step 3: locating the vulnerability patch in the target program according to the historical vulnerability patch and the historical vulnerability; Step 4: transplant the historical vulnerability to the location of the vulnerability patch in the target program according to the vulnerability patch revocation rule to obtain a real vulnerability test set.

2. The method for automatically constructing a vulnerability benchmark test set based on patch mode revocation according to claim 1, characterized in that: The step three specifically includes: Extracting the difference code between the historical vulnerability and the historical vulnerability patch, finding the location of the difference code in the target program, and obtaining a plurality of similar code blocks; Detect multiple similar code blocks according to the GumTree algorithm to obtain similar code blocks required to form the historical vulnerability patch, and use the similar code blocks required to form the historical vulnerability patch as candidate patches; According to the grammatical distance between the candidate patch and the historical vulnerability patch, it is determined whether the candidate patch is a vulnerability patch of the target program.

3. The method for automatically constructing a vulnerability benchmark test set based on patch mode revocation according to claim 2, characterized in that: The detecting of multiple similar code blocks according to the GumTree algorithm to obtain similar code blocks required to form the historical vulnerability patch and using the similar code blocks required to form the historical vulnerability patch as candidate patches specifically includes: Convert multiple similar code blocks and historical vulnerability patches into abstract syntax trees; According to the tree editing operation in the GumTree algorithm, it is determined whether the abstract syntax trees of multiple similar code blocks can be successfully converted into the abstract syntax trees of historical vulnerability patches, and the successfully converted similar code blocks are used as candidate patches.

4. The method for automatically constructing a vulnerability benchmark test set based on patch mode revocation according to claim 2, characterized in that: The determining, according to the grammatical distance between the candidate patch and the historical vulnerability patch, whether the candidate patch is a vulnerability patch for the target program specifically includes: Convert candidate patches and historical vulnerability patches into abstract syntax trees; The grammatical distance between the abstract syntax tree of the candidate patch and the abstract syntax tree of the historical vulnerability patch is calculated, and the candidate patch with the closest grammatical distance is used as the vulnerability patch of the target program.

5. The method for automatically constructing a vulnerability benchmark test set based on patch mode revocation according to claim 1, characterized in that: The step 4 specifically includes: Construct vulnerability patch revocation rules according to different vulnerability revocation structures, and generate revocation patches according to the vulnerability patch revocation rules; The vulnerability patches in the target program are converted using the undo patch to obtain the program containing the vulnerability code as the sample in the real vulnerability test set.

6. The method for automatically constructing a vulnerability benchmark test set based on patch mode revocation according to claim 5, characterized in that: When the vulnerability patch in the target program is converted by using the revocation patch, a complex revocation rule is split into a plurality of simple revocation rules.

7. The method for automatically constructing a vulnerability benchmark test set based on patch mode revocation according to claim 5, characterized in that: The step 4 also includes: Prune the code irrelevant to the vulnerability patch code of the target program; Compile and verify the undo patch; Recursively add the missing variables, functions, and parameters in the program containing the vulnerable code to obtain a complete set of real vulnerability tests.

8. A vulnerability benchmark test set automatic construction system based on patch mode revocation, characterized in that: include: Collection module 1 is used to obtain the target program and collect historical vulnerabilities in other versions of the project where the target program is located; The second collection module is used to search for historical vulnerability patches related to the historical vulnerability in the patch database; A positioning module, used for locating the vulnerability patch in the target program according to the historical vulnerability patch and the historical vulnerability; The transplantation module is used to transplant the historical vulnerability to the location of the vulnerability patch in the target program according to the vulnerability patch revocation rule to obtain a real vulnerability test set.

9. A terminal device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a method for automatically constructing a vulnerability benchmark test set based on patch mode revocation as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the method for automatically constructing a vulnerability benchmark test set based on patch mode revocation as described in any one of claims 1 to 7.

Images