Method for automatic detection of vulnerabilities based on public vulnerability data

Abstract

The application discloses a vulnerability automatic detection method based on public vulnerability data, and relates to the technical field of vulnerability detection. The method comprises the following steps: obtaining the code change record of the vulnerability repair code of each first target public disclosed vulnerability from an open source code platform, determining the position of each vulnerability related code according to each code change record, and then extracting the vulnerability related code from each vulnerability triggered version code; analyzing each vulnerability related code to obtain the corresponding abstract syntax tree; performing basic feature and generalization feature matching on the target code and each vulnerability related code according to each abstract syntax tree; and if the basic feature and the generalization feature of the target code and any vulnerability related code are matched, the target code contains the first target public disclosed vulnerability contained in the vulnerability related code which matches the basic feature and the generalization feature of the target code. The application realizes the automatic detection of public vulnerabilities based on public vulnerability data.

Description

Technical Field

[0001] This application relates to the field of vulnerability detection, and in particular to an automatic vulnerability detection method based on publicly available vulnerability data. Background Technology

[0002] With the rapid development of information technology, the scale and complexity of software systems are constantly increasing, leading to a rise in the number of potential security vulnerabilities. These vulnerabilities can be maliciously exploited, posing a threat to the security of individuals, organizations, and even nations. Especially with the increasing reliance on open source, attacks exploiting known vulnerabilities are a common phenomenon in the field of cybersecurity. Therefore, developing effective code vulnerability detection methods has become a crucial aspect of ensuring software security.

[0003] Existing technologies detect code vulnerabilities through static analysis tools, which rely on manually written rules. However, manually written rules require a lot of time to maintain, resulting in high maintenance costs and making it difficult to achieve large-scale vulnerability detection. Summary of the Invention

[0004] The purpose of this application is to provide an automatic vulnerability detection method based on publicly available vulnerability data, which can solve the problem that existing technologies for detecting code vulnerabilities through static analysis tools have high maintenance costs and are difficult to achieve large-scale vulnerability detection due to reliance on manually written rules.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] Firstly, this application provides an automatic vulnerability detection method based on publicly available vulnerability data, including:

[0007] The code change records of the vulnerability fix code for each first target publicly disclosed vulnerability are obtained from the open source code platform. After determining the location of the vulnerability-related code based on each code change record, the vulnerability-related code is extracted from the vulnerability-triggered version code.

[0008] Analyze the code related to each vulnerability to obtain the corresponding abstract syntax tree;

[0009] Based on each of the abstract syntax trees, the target code is matched with each of the vulnerability-related codes using basic and generalized features. If the target code matches both the basic and generalized features of any of the vulnerability-related codes, then the target code contains a first publicly disclosed vulnerability contained in the vulnerability-related code that matches its basic and generalized features.

[0010] Optionally, the step of obtaining the code change record of the vulnerability patch for the first publicly disclosed vulnerability from the open-source code platform, determining the location of the vulnerability-related code based on the code change record, and extracting the vulnerability-related code from the vulnerability-triggered version code specifically includes:

[0011] After obtaining the URL of the code repository address involved in the patch for the first publicly disclosed vulnerability contained in the tag reference of the open source code platform, download the vulnerability trigger version code and vulnerability patch version code for each first publicly disclosed vulnerability from the open source code platform according to the URL;

[0012] By using the CVE number of each publicly disclosed vulnerability for each first-target vulnerability, we can filter the code change records for fixing each publicly disclosed vulnerability for each first-target vulnerability from the Commit information, related issues, or PRs.

[0013] Based on the publicly disclosed code change history for each first-target vulnerability fix, the location of the complete fix code is obtained from the vulnerability fix version code;

[0014] Extract vulnerability-related code from the vulnerability-triggered version code based on the location of the complete fix code.

[0015] Optionally, the basic feature and generalization feature matching is performed between the target code and each of the vulnerability-related codes. If the target code matches both the basic feature and generalization feature of any of the vulnerability-related codes, then the target code contains a first publicly disclosed vulnerability contained in the vulnerability-related code that matches its basic feature and generalization feature. Specifically, this includes:

[0016] Each target code fragment is matched with the basic features of each target vulnerability-related code; wherein, the target code fragment refers to any code fragment of the target code, the basic features of each code fragment are extracted from the abstract syntax tree, and the target vulnerability-related code refers to the vulnerability-related code of any publicly disclosed vulnerability of the first target;

[0017] If any of the target code fragments matches the basic features of any of the target vulnerability-related code, generalized feature matching is performed on each target code group, wherein the target code group refers to a group of target code fragments and target vulnerability-related code that match the basic features;

[0018] If the generalization feature of any target code group matches, then a second publicly disclosed vulnerability exists in the target code segment of the target code group; wherein, the second publicly disclosed vulnerability is a first publicly disclosed vulnerability existing in the target vulnerability-related code of the target code group whose generalization feature matches.

[0019] Optionally, the basic features include a method signature, which includes a return value type, a function name, and a parameter list, wherein the parameter list includes the number of parameters, parameter types, and parameter order;

[0020] The process of matching each target code fragment with the basic features of each target vulnerability-related code specifically includes:

[0021] Perform method signature matching between each target code fragment and each target vulnerability-related code;

[0022] If any of the target code fragments matches at least two of the return value type, function name, and parameter list of the function in the method signature of any of the target vulnerability-related code, then the target code fragment matches the method signature of the target vulnerability-related code.

[0023] Among them, return value type matching means that the return value types are the same, function name matching means that the function names are the same, and parameter list matching means that the parameter types of the target code are all located in the parameter type set of the target vulnerability-related code.

[0024] Optionally, the basic features also include library function call features;

[0025] The step of performing basic feature matching between each target code fragment and the code related to each target vulnerability also includes:

[0026] For each group of method signatures, target code snippets and code related to the target vulnerability are matched for library function call features.

[0027] If both the target code segment and the target vulnerability-related code that match the method signature contain library functions, and the library functions in the target code segment and the target vulnerability-related code satisfy the matching condition of library function call characteristics, then the basic characteristics of the target code segment and the target vulnerability-related code match.

[0028] Optionally, the library function call feature includes at least the library function name, and the matching conditions for the library function call feature include at least:

[0029] The function names of the target code snippets and library functions in the target vulnerability-related code that match the signatures of any of the methods are consistent.

[0030] Optionally, the library function call characteristics also include at least one of the class name, package name, and version number where the library function is defined;

[0031] The matching conditions for the library function call characteristics also include:

[0032] The class name, package name, and / or version number of the library function in any of the target code snippets and target vulnerability-related code that match the method signatures of the group.

[0033] Optionally, the generalization features include control flow generalization features and data flow generalization features;

[0034] The generalized feature matching for each target code group specifically includes:

[0035] Perform control flow generalization feature matching for each target code group;

[0036] If the control flow generalization feature of any target code group matches, perform data flow generalization feature matching on the target code group whose control flow generalization feature matches.

[0037] The generalized feature matching of any target code group specifically includes:

[0038] The control flow generalization feature and data flow generalization feature of each target code group are matched.

[0039] Optionally, the generalized feature matching of the target code snippet and the target vulnerability-related code for basic feature matching further includes:

[0040] Before performing control flow generalization feature matching on each target code group, a control flow graph and a data flow graph for each target code group are generated based on the abstract syntax tree of each target code group.

[0041] According to the preset generalization rules, the data of each node in the control flow graph and data flow graph of each target code group are generalized to obtain the control flow generalization features and data flow generalization features of each target code group; wherein, the preset generalization rules include at least one of branch generalization rules and function call generalization rules, and the branch generalization rules include at least one of conditional branch generalization rules, loop branch generalization rules and exception branch generalization rules.

[0042] The generalization process, according to preset generalization rules, generalizes the data of each node in the control flow graph and data flow graph of each target code group, specifically including:

[0043] If any target code group contains function calls related to a target vulnerability, then the function call generalization rule is used to generalize the data of each function call node in the control flow graph and data flow graph of the target code group.

[0044] If any target code group contains conditional branches related to a target vulnerability, then the conditional branch generalization rule is used to generalize the data of each conditional branch node in the control flow graph and data flow graph of the target code group.

[0045] If any target code group contains loop branches in its target vulnerability-related code, then the loop branch generalization rule is used to generalize the data of each loop branch node in the control flow graph and data flow graph of the target code group.

[0046] If any target code group contains an abnormal branch in the target vulnerability-related code, then the abnormal branch generalization rule is used to generalize the data of the abnormal branch node of the target code group.

[0047] Optionally, the step of performing control flow generalization feature matching for each target code group specifically includes:

[0048] Perform control flow generalization feature structure matching on each of the target code groups;

[0049] If the control flow generalization feature structure of each target code group matches, then control flow generalization feature content matching is performed on each target code group whose control flow generalization feature structure matches; wherein, the content includes at least one of the attributes and internal operations of the function called in the control flow generalization feature, conditional branch operations, loop branch operations, and exception branch operations.

[0050] Control flow generalization feature matching for each target code group specifically includes:

[0051] The control flow generalization feature structure and content of each target code group are matched;

[0052] The process of performing data stream generalization feature matching for each target code group specifically includes:

[0053] A second target feature flow is searched in the data flow generalization features of the target code segment in each target code group that matches the control flow generalization features by using a sliding window; wherein, the second target feature flow is the part of the data flow generalization features of the target code segment in the target code group that matches the target vulnerability-related code.

[0054] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0055] This application provides an automatic vulnerability detection method based on publicly disclosed vulnerability data. It obtains the code change records of the vulnerability patch code for each publicly disclosed vulnerability from an open-source code platform. After determining the location of vulnerability-related code for each vulnerability based on each code change record, it extracts vulnerability-related code from the code of each vulnerability-triggered version. It analyzes each vulnerability-related code to obtain a corresponding abstract syntax tree, preserving the syntactic structure information of the vulnerability-related code, which is beneficial for generalization feature extraction. Based on each abstract syntax tree, it performs basic feature and generalization feature matching between the target code and each vulnerability-related code. If the target code matches the basic features and generalization features of any vulnerability-related code, then the target code contains a vulnerability-related code that matches its basic features and generalization features. This method targets publicly disclosed vulnerabilities. By first performing basic feature matching and then generalized feature matching, it ensures higher detection accuracy compared to directly using generalized feature matching. It eliminates the need for manually written rules and offers better versatility compared to manually written rules for vulnerability detection. This enables large-scale, automated, and rapid vulnerability detection based on publicly disclosed vulnerability data, avoiding the high maintenance costs caused by missed or false detections due to manually written rules. While achieving large-scale, automated, and rapid detection of publicly disclosed vulnerabilities, it also reduces maintenance costs and solves the problem of high maintenance costs and difficulty in large-scale vulnerability detection caused by existing static analysis tools for detecting code vulnerabilities due to reliance on manually written rules. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating an automatic vulnerability detection method based on publicly available vulnerability data, provided as an embodiment of this application;

[0058] Figure 2 A schematic diagram illustrating the control flow characteristics of the vulnerable code in ListUtil.setOrPadding (CVE-2023-42277) provided in an embodiment of this application;

[0059] Figure 3 A schematic diagram illustrating the data flow characteristics of the vulnerable code ListUtil.setOrPadding in CVE-2023-42277 provided in an embodiment of this application;

[0060] Figure 4 A schematic diagram illustrating the control flow characteristics of a code snippet in the ListUtil.setOrPadding file of the JsonArray.java file in the first test item hutool provided in an embodiment of this application;

[0061] Figure 5 A schematic diagram of the data flow characteristics of a code snippet in the ListUtil.setOrPadding file of the JsonArray.java file in the first test item hutool provided in an embodiment of this application;

[0062] Figure 6 A schematic diagram illustrating the control flow characteristics of the vulnerability code for a resource verification missing vulnerability provided in an embodiment of this application;

[0063] Figure 7 A schematic diagram of the data flow characteristics of a resource verification missing vulnerability provided in an embodiment of this application;

[0064] Figure 8 A schematic diagram of the control flow characteristics of a code segment of a second test item provided in an embodiment of this application;

[0065] Figure 9 A schematic diagram of the data flow characteristics of a code segment of a second test item provided in an embodiment of this application;

[0066] Figure 10 A schematic diagram of the functional modules of an automatic vulnerability detection device based on publicly available vulnerability data provided in an embodiment of this application;

[0067] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0068] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0069] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0070] In one exemplary embodiment, such as Figure 1 As shown, an automatic vulnerability detection method based on publicly available vulnerability data is provided, including the following steps 201 to 204. Wherein:

[0071] Step 201: Obtain the code change record of the vulnerability repair code for each first target publicly disclosed vulnerability from the open-source code platform. After determining the location of each vulnerability-related code according to the code change record, extract the vulnerability-related code from the code of each vulnerability trigger version.

[0072] In the embodiments of this application, the first target publicly disclosed vulnerability is the publicly disclosed vulnerability to be detected, which can be all publicly disclosed vulnerabilities, or some or a certain publicly disclosed vulnerability. This application does not make specific limitations on it and can be set according to actual needs.

[0073] The code of the vulnerability trigger version is the complete version code containing the first target publicly disclosed vulnerability. The vulnerability-related code is the code snippet containing the first target publicly disclosed vulnerability. The code change record contains information such as the vulnerability repair commit record and the affected version range. For example, for the OpenSSH ssh-agent vulnerability CVE-2023-38408, its affected version range is 5.5 < OpenSSH <= 9.3p1, and the code change of the vulnerability repair contains the following new code block: if (e->nsession_ids!= 0 &&!remote_add_provider) {

[0074] verbose("failed add of SK provider \"%.100s\": "

[0075] "remote addition of providers is disabled",

[0076] sk_provider);

[0077] goto out;

[0078] } According to the content of the above code block, the file and associated function where the vulnerability is located can be determined.

[0079] Step 202: Analyze each vulnerability-related code to obtain the corresponding abstract syntax tree.

[0080] In the embodiments of this application, the extracted vulnerability code snippet is first disassembled into the smallest syntax units through lexical analysis, and then these syntax units are combined into a tree-like structure describing the code, that is, the abstract syntax tree (AST), through syntax analysis. By traversing from the root node of the tree to the leaf nodes, each node in the syntax tree can be recognized as structures such as function calls, variable assignments, loops, etc. According to the characteristic rules of security vulnerabilities, nodes matching these characteristics are searched in the syntax tree as the basis for subsequent vulnerability identification and detection.

[0081] Step 203: Based on each abstract syntax tree, perform basic feature and generalization feature matching between the target code and each vulnerability-related code. If the target code matches the basic feature and generalization feature of any vulnerability-related code, then the target code contains a first publicly disclosed vulnerability contained in the vulnerability-related code that matches its basic feature and generalization feature.

[0082] In this embodiment, current methods for directly extracting features, such as regular expression matching, destroy the syntactic structure information of the code. Although they can extract basic features from the code, they are difficult to extract generalized features. Abstract syntax trees, on the other hand, preserve the syntactic structure information of the code, which is beneficial for extracting further basic and generalized features.

[0083] The generalization characteristics of vulnerability-related code include control flow generalization characteristics and data flow generalization characteristics. For example, parameters used for loop boundaries, specific resource operations performed in the loop body, etc., are removed to eliminate details and naming that are irrelevant to the business logic, making it possible to match different code implementations with similar functions.

[0084] By implementing steps 201 to 203 above, code change records of vulnerability patching code for each first target publicly disclosed vulnerability are obtained from the open-source code platform. After determining the location of each vulnerability-related code based on each code change record, vulnerability-related code is extracted from the code of each vulnerability triggering version. Each vulnerability-related code is analyzed to obtain the corresponding abstract syntax tree, preserving the syntactic structure information of the vulnerability-related code, which is beneficial for generalization feature extraction. Based on each abstract syntax tree, basic features and generalization features are matched between the target code and each vulnerability-related code. If the target code matches the basic features and generalization features of any vulnerability-related code, then the target code contains the first target publicly disclosed vulnerability contained in the vulnerability-related code that matches its basic features and generalization features. This achieves large-scale automatic and rapid vulnerability detection based on feature matching of publicly disclosed vulnerability data. Since this application does not require manual rule writing, it avoids the problem of high maintenance costs caused by the easy omission or false detection of manually written rules. While achieving large-scale automatic and rapid detection of first target publicly disclosed vulnerabilities, it reduces maintenance costs and solves the problem that existing technologies use static analysis tools to detect code vulnerabilities, which rely on manually written rules, resulting in high maintenance costs and difficulty in achieving large-scale vulnerability detection.

[0085] In another exemplary embodiment of this application, step 201 described above can be replaced by steps 301 to 304, wherein:

[0086] Step 301: After obtaining the URL of the code repository address involved in the patch of the first target publicly disclosed vulnerability contained in the tag reference of the open source code platform (such as GitHub, Gitee, etc.), download the vulnerability trigger version code and vulnerability patch version code of each first target publicly disclosed vulnerability from the open source code platform (such as via git clone) according to the URL.

[0087] In this embodiment of the application, the vulnerability patching version code is the complete version code obtained after the first publicly disclosed vulnerability in the vulnerability triggering version code has been patched.

[0088] URLs are used for:

[0089] (1) Locating vulnerability fix code repositories: The URL directly points to the official code repository address of the open source project on platforms such as GitHub / Gitee, providing an access point to the source code of the vulnerability fix version;

[0090] (2) Accurate code version download: Combined with the git clone command, the complete code repository of the vulnerability-triggered version (such as v1.0) and the patched version (such as v1.1) can be directly downloaded via URL;

[0091] (3) Linked vulnerability fix records: You can jump to the Commits or Pull Requests page of the code repository via the URL to view the specific code change records for vulnerability fixes.

[0092] Step 302: Using the CVE number of each publicly disclosed vulnerability for each first target, filter the code change records for fixing each publicly disclosed vulnerability for each first target from the Commit information, associated issues (online discussions), or PRs.

[0093] In this embodiment of the application, the commit information includes:

[0094] (1) Unique identifier: Submit a hash value (e.g., a1b2c3d) to uniquely identify code change records;

[0095] (2) Code change record: modified file path and specific code line (added, deleted and modified), and description of the repair logic (e.g., Fix buffer overflow in EDA#11 module);

[0096] (3) Metadata: Author and submission timestamp, associated Pull Request number (optional);

[0097] (4) Security related information: Vulnerability number reference (e.g., CVE-2023-38408); description of affected version range (e.g., Affects versions<=9.3p1).

[0098] The Pull Requests page contains each PR for a fix. PRs typically include a background description of the fix, vulnerability impact analysis, code change history, code review discussion, test cases, and associated Issue or CVE numbers.

[0099] Step 303: Based on the publicly disclosed code change history for each first-target vulnerability fix, obtain the location of the complete fix code from the vulnerability fix version code (e.g., using git show). <commit-hash>(Location of the complete fix code).

[0100] Step 304: Extract vulnerability-related code from the vulnerability-triggered version code based on the location of the complete fix code.

[0101] In another exemplary embodiment of this application, the basic characteristics of the vulnerability-related code described above include a method signature, which includes a return value type, a function name, and a parameter list, wherein the parameter list reflects the number of parameters, parameter types, and parameter order.

[0102] In this embodiment, the function definition node containing the vulnerability is located from the AST nodes, information within the node is collected, and a method signature is generated. For example, the structure of the method signature functionSignature is as follows: functionSignature:<ReturnType、functionName、ParamList> `ReturnType` indicates the function's return type, `functionName` indicates the function name, and `ParamList` indicates the function's parameter list. Collecting basic information about vulnerable functions helps to quickly filter out irrelevant functions later.

[0103] In another exemplary embodiment of this application, step 203 described above includes steps 401 to 403. Wherein:

[0104] Step 401: Perform basic feature matching on each target code fragment and each target vulnerability-related code; where a target code fragment refers to any code fragment of the target code, and each code fragment can implement an operation function. The basic features of each code fragment are extracted from the abstract syntax tree, and the target vulnerability-related code refers to the vulnerability-related code of any publicly disclosed vulnerability of the first target.

[0105] Step 402: If any target code fragment matches the basic features of any target vulnerability-related code, perform generalized feature matching for each target code group, wherein the target code group refers to a group of target code fragments and target vulnerability-related code that match the basic features.

[0106] In this embodiment of the application, generalized feature matching for each target code group involves performing generalized feature matching on the target code fragments and target vulnerability-related code within each target code group.

[0107] Step 403: If the generalization feature of any target code group matches, then there is a second publicly disclosed vulnerability in the target code segment of the target code group; wherein, the second publicly disclosed vulnerability is the first publicly disclosed vulnerability in the target vulnerability-related code of the target code group that matches the generalization feature.

[0108] In another exemplary embodiment of this application, step 401 above, which involves matching the basic features of each target code fragment with the code related to each target vulnerability, specifically includes:

[0109] Step 501: Perform method signature matching between each target code snippet and the code related to each target vulnerability;

[0110] If any target code snippet matches at least two of the return type, function name, and parameter list of a function in the method signature of any target vulnerability-related code, then the target code snippet matches the method signature of the target vulnerability-related code.

[0111] In this embodiment, return value type matching means that the return value types are the same. Function name matching means that the function names are the same. Parameter list matching means that the parameter types of the target code snippets are all located in the parameter type set of the code related to the target vulnerability.

[0112] If the basic characteristics of the vulnerability-related code only include the method signature, then the target code satisfies the matching condition of the method signature, which means that the target code satisfies the basic characteristics of the vulnerability-related code.

[0113] For example, taking the ListUtil.setOrPadding vulnerability in CVE-2023-42277 as an example, the relevant code is as follows:

[0114] public static void ListUtil.setOrPadding(List <t>list, int index, Telement){

[0115] if (index <list.size()) {

[0116] list.set(index, element);

[0117] } else {

[0118] while (list.size() <index) {

[0119] list.add(paddingElement);

[0120] }

[0121] list.add(element);

[0122] }

[0123] }

[0124] This code adds an element to the list at a given index: if the index is within the list's length, it replaces the element at that position; otherwise, it appends elements until the index is reached before inserting the new element. Because the loop doesn't limit the index size, if an extremely large index value is passed in, the loop will execute many times, continuously adding elements to the list until it runs out of memory and triggers an OutOfMemoryError.

[0125] Here is a code snippet from the JsonArray.java file in the first project to be tested, hutool:

[0126] @Overridepublic void add(int index, Object element) {if (index<0){ …}

[0127] if (index <this.size()) {

[0128] this.rawList.add(index, …);}else

[0129] {while (index != this.size()) {

[0130] this.add(JSONNull.NULL);}this.set(element);

[0131] }

[0132] }

[0133] The basic characteristics of this code snippet from the first project to be tested, hutool, are as follows: "method_signature": {

[0134] "name": "add",

[0135] "parameters": [

[0136] {

[0137] "type": "int",

[0138] "name": "index",

[0139] "taint_source": "user_input"

[0140] },

[0141] {

[0142] "type": "java.lang.Object",

[0143] "name": "element"

[0144] }

[0145] ],

[0146] "return_type": "void",

[0147] "modifiers": "public"

[0148] The method signature matching process is as follows:

[0149] The original vulnerability code is setOrPadding(List <t>The original vulnerable code has three parameters, while the target code has two. However, the parameter types `index` and `element` in the target code are both included in the parameter types of the original vulnerable code. Therefore, the parameter types of the target code match those of the original vulnerable code. Thus, the method signature matching condition is satisfied, which requires at least two matches in the return type, function name, and parameter list.

[0150] For example, regarding the resource verification missing vulnerability pattern, below is a snippet of original vulnerable code:

[0151] public class DatabaseProcessor {

[0152] public void processDatabase(String url) {

[0153] try {

[0154] Connection connection = DriverManager.getConnection(url);

[0155] Statement stmt = connection.createStatement();

[0156] stmt.execute("SELECT FROM users");

[0157] } catch (SQLException e) {

[0158] log.error("Database error", e);

[0159] }

[0160] }

[0161] }

[0162] A code snippet from the second project to be tested is as follows:

[0163] public class FileProcessor {

[0164] public void processFile(String fileName) {

[0165] try {

[0166] FileInputStream fileStream = new FileInputStream(fileName);

[0167] byte[ ] data = new byte

[1024] ;

[0168] fileStream.read(data);

[0169] } catch (IOException e) {

[0170] log.error("File processing error", e);

[0171] }

[0172] }

[0173] }

[0174] The basic features (method signature) are as follows:

[0175] "method_signature": {

[0176] "name": "processFile",

[0177] "parameters":

[0178] {

[0179] "type": "java.lang.String",

[0180] "name": "fileName",

[0181] "taint_source": "user_input"

[0182] }

[0183] ,

[0184] "return_type": "void",

[0185] "modifiers": "public"

[0186] }

[0187] Method signature matching process:

[0188] In the method signature of the original vulnerable code, the function name is processDatabase, the parameter is (String url), and the return type is void. In the method signature of the target code, the function name is processFile, the parameter is (String fileName), and the return type is void. The return type (void) and the parameter type (String) are consistent, which satisfies the matching condition of the method signature. The method signature match is successful.

[0189] Since method signatures include the function name, parameter list, and return type, while matching method signatures doesn't necessarily mean similar function logic, functions with inconsistent method signatures will definitely have different function logic. Therefore, using method signature matching can quickly locate potentially vulnerable functions, thereby narrowing down the set of functions to be tested and improving vulnerability detection efficiency.

[0190] In another exemplary embodiment of this application, the aforementioned basic features also include library function call features.

[0191] In this application embodiment, the purpose of extracting library function call features is: (1) The library function called by the functional area where the vulnerability is located may itself be a dangerous function (including the possibility of being exploitable buffer overflow, command injection, unauthorized access, etc.). By checking whether such library functions have been called, software vulnerabilities can be quickly identified; (2) Functional semantic mapping, calling library functions of certain specific functions will reflect the function of the function where the call point is located to a certain extent. These functions may be sensitive operation functions or high-risk functions.

[0192] Accordingly, the above-mentioned basic feature matching of each target code fragment with the code related to each target vulnerability also includes:

[0193] Step 502: Perform library function call feature matching on the target code snippets and target vulnerability-related code that match the method signatures for each group;

[0194] If both the target code segment and the target vulnerability-related code that match any set of method signatures contain library functions, and the library functions in the target code segment and the target vulnerability-related code that match the method signatures satisfy the matching condition of library function call characteristics, then the basic characteristics of the target code segment and the target vulnerability-related code that match the method signatures are matched.

[0195] In another exemplary embodiment of this application, the library function call features described above include at least the library function name.

[0196] Accordingly, in step 502 above, the matching conditions for library function call features include:

[0197] Any set of method signatures matches the function name of the target code snippet and the library function in the target vulnerability-related code.

[0198] In another exemplary embodiment of this application, the library function call features described above further include at least one of the class name, package name, and version number where the library function is defined.

[0199] In this embodiment of the application, the matching conditions for the above-mentioned library function call characteristics further include:

[0200] The class name, package name, and / or version number of the library function in any set of method signature matching target code snippets and target vulnerability-related code are consistent.

[0201] In another exemplary embodiment of this application, step 402 above, which involves performing generalized feature matching for each target code group, specifically includes:

[0202] Step 601: Perform control flow generalization feature matching for each target code group.

[0203] In this embodiment of the application, control flow generalization feature matching is performed on each target code group, which means performing control flow generalization feature matching on the target code fragments and target vulnerability-related code of each target code group. Control flow generalization features refer to the generalized control flow features.

[0204] The purpose of control flow generalization feature matching is to focus on whether there are certain special control flow processes that can satisfy the triggering conditions, such as common loop structures.

[0205] Step 602: If the control flow generalization feature of any target code group matches, perform data flow generalization feature matching on the target code group.

[0206] In this embodiment of the application, data flow generalization feature matching is performed on the target code group. This means that for each target code group whose basic features and control flow generalization features match, data flow generalization feature matching is performed on the target code fragments and target vulnerability-related code. Data flow generalization features refer to the generalized data flow features.

[0207] Generalization abstracts specific class names, method names, and variable names into generic tags. For example, it abstracts a specific list into any collection object to improve the applicability of the pattern. At the same time, data flow information can be used to analyze the propagation path of vulnerabilities and related data processing links.

[0208] Accordingly, in step 403, the generalized feature matching of any set of basic feature-matched target code fragments and target vulnerability-related code specifically includes:

[0209] The control flow generalization feature and data flow generalization feature of each target code group are matched.

[0210] In another exemplary embodiment of this application, to further circumvent manually written rules, in step 402 above, generalized feature matching is performed on the target code fragment and the target vulnerability-related code for basic feature matching. Before step 601, the following steps are also included:

[0211] Step 701: Generate the control flow graph and data flow graph for each target code group based on the abstract syntax tree of each target code group.

[0212] In this embodiment, the abstract syntax tree of each target code group includes the abstract syntax tree of the target code segment and the abstract syntax tree of the target vulnerability-related code in each target code group. A control flow graph and data flow graph for the target code segment are generated based on the abstract syntax tree of the target code segment in each target code group. Similarly, a control flow graph and data flow graph for the target vulnerability-related code are generated based on the abstract syntax tree of the target vulnerability-related code in each target code group. The control flow graph and data flow graph for each target code group include the control flow graph and data flow graph of the internal target code segment, as well as the control flow graph and data flow graph of the target vulnerability-related code.

[0213] Step 702: According to the preset generalization rules, generalize the data of each node in the control flow graph and data flow graph of each target code group to obtain the control flow generalization feature and data flow generalization feature of each target code group.

[0214] In this embodiment of the application, the control flow generalization features and data flow generalization features of each target code group include the control flow generalization features and data flow generalization features of the internal target code segments, as well as the control flow generalization features and data flow generalization features of the target vulnerability-related code.

[0215] In another exemplary embodiment of this application, the preset generalization rules include at least one of branch generalization rules and function call generalization rules, and the branch generalization rules include at least one of conditional branch generalization rules, loop branch generalization rules, and exception branch generalization rules.

[0216] Accordingly, step 702 specifically includes:

[0217] If any target code group contains function calls related to a target vulnerability, then the function call generalization rules are used to generalize the data of each function call node in the control flow graph and data flow graph of that target code group.

[0218] If any target code group contains conditional branches (such as if, else) related to the target vulnerability, then the conditional branch generalization rule is used to generalize the data of each conditional branch node in the control flow graph and data flow graph of that target code group.

[0219] Each conditional branch node includes the branch condition (such as if(), else()) and the operation to be performed when the branch condition is met. The operation to be performed when the branch condition is met refers to the code between the first { after the branch condition and the} corresponding to that {, such as if(){}, else(){}.

[0220] If any target code group contains loop branches (such as while loops) in its target vulnerability-related code, then the loop branch generalization rule is used to generalize the data of each loop branch node in the control flow graph and data flow graph of that target code group.

[0221] The data for each loop branch node includes the loop condition (such as while()) and the operation to be executed when the loop condition is met. The operation to be executed when the loop condition is met refers to the code between the first { after the loop condition and the} corresponding to that {, such as while(){}.

[0222] If any target code group contains exception branches (try and catch) related to the target vulnerability, then the exception branch generalization rule is used to generalize the data (such as try{} and catchtry{}) of the exception branch nodes of the target code group.

[0223] In another exemplary embodiment of this application, the function call generalization rule includes parameter generalization rule and method call generalization rule.

[0224] Accordingly, function call generalization rules are used to generalize function call code, specifically including:

[0225] Use parameter generalization rules and method call generalization rules to generalize function call code.

[0226] In another exemplary embodiment of this application, the parameter generalization rules include:

[0227] Generalize the parameter type to the first parameter tag, such as PARAM;

[0228] Generalize the type of the temporary variable to the second parameter label, such as VAR.

[0229] The parameter type is generalized to the first parameter label because if the source of a certain point of the control flow feature or data flow feature is a parameter, it will be uniformly labeled as the first parameter label, indicating that it comes from a parameter rather than other temporary variables.

[0230] For example, for the function parameter List <t>When generalizing a list, the concrete list is abstracted into a collection. Since T can be configured to any type, the List is then... <t>Generalize to the first parameter marker PARAM, and take the function parameter List <t>The list is generalized to PARAM(collection). When generalizing the function parameter int index, the specific int index parameter is abstracted into a user-controlled index parameter, identified as PARAM(index).

[0231] For example, the temporary variable paddingElement is generalized to VAR(paddingElement).

[0232] In another exemplary embodiment of this application, the method invocation generalization rule includes:

[0233] The function name of the non-resource function being called is generalized to the first type of method marker, such as generalizing it to CALL; non-resource functions refer to functions that do not involve the creation or use of resources (database or file).

[0234] For example, the method call to list.size() can be generalized, abstracting the specific size call into a collection size retrieval operation, which is marked as CALL(collection.size()).

[0235] The function name of the resource creation function called (such as a method call related to database or file creation) is generalized to a second type of method tag, such as RESOURCE_CREATE. The second tag is a general tag for functions that create resources based on parameters.

[0236] For example, the function call DriverManager.getConnection(url) in the database example is generalized to RESOURCE_CREATE(PARAM).

[0237] The function name of the resource utilization function (such as a method call related to database or file reading) is generalized to a third-class method marker, such as RESOURCE_USE. The third-class method marker is a general marker for functions that utilize already created resources. The created resource refers to the resource created by the called resource creation class function.

[0238] For example, a call to the connection.createStatement() function is generalized to RESOURCE_USE(createStatement).

[0239] In another exemplary embodiment of this application, parameter generalization rules and method call generalization rules are used to generalize the data of the function call node, specifically including:

[0240] Generalize the parameter type of each function call node to the first parameter marker; if there is a temporary variable, generalize the type of the temporary variable to the second parameter marker.

[0241] If any function call node calls a non-resource function, the function name of the called non-resource function will be generalized to the first type of method tag;

[0242] If any function call node calls a resource creation class function, the function name of the called resource creation class function will be generalized to the second type of method tag;

[0243] If any function call node invokes a resource utilization class function, the function name of the invoked resource utilization class function will be generalized to a third-class method tag.

[0244] For example, the method call `list.set(index, element)` can be generalized, abstracting the concrete `set` call into the collection element setting operation `collection.set`, marked as `CALL(collection.set, PARAM(index), PARAM(element))`. Similarly, the method call `list.add(paddingElement)` can be generalized, abstracting the concrete `add` call into the collection element adding operation `collection.add`, marked as `CALL(collection.add, VAR(paddingElement))`.

[0245] In this embodiment, to determine whether a called function is a resource-type method call, a resource-type function set can be created first, integrating various file or database creation functions and various file or database exploitation (e.g., reading) functions. Alternatively, a resource creation function set and a resource exploitation function set can be created separately. The resource creation function set integrates various file or database creation functions, and the resource exploitation function set integrates various file or database exploitation (e.g., reading) functions. If the function called by the target code fragment or target vulnerability-related code is in the resource-type function set or the resource creation function set, then the function called by the target code fragment or target vulnerability-related code belongs to the resource creation function set. If the function called by the target code fragment or target vulnerability-related code is in the resource-type function set or the resource exploitation function set, then the function called by the target code fragment or target vulnerability-related code belongs to the resource exploitation function set. Otherwise, the function called by the target code fragment or target vulnerability-related code belongs to the non-resource-type function set.

[0246] In another exemplary embodiment of this application, the branch generalization rule includes:

[0247] Generalize the branch type to the first conditional branch marker (such as IF), the second conditional branch marker (such as ELSE), the loop conditional branch marker (such as WHILE_LOOP), or the exception branch marker, such as TRY-CATCH;

[0248] Generalize the type of the comparison binary expression in the branch condition to the first branch condition tag, such as BINARY_OP;

[0249] Generalize the type of the unary expression in the branch condition to the second branch condition tag, such as UNARY_OP;

[0250] Generalize the parameters in the unary or binary expression of the branch condition according to the parameter generalization rule in the function call generalization rule.

[0251] The function call is invoked according to the method in the function call generalization rule, which calls the function in the unary or binary expression of the generalization branch condition.

[0252] In this embodiment of the application, the branch condition refers to the condition in the conditional branch or loop branch, such as x, y, and z in if(x), else(y), and while(z).

[0253] In another exemplary embodiment of this application, based on the above generalization rules, the control flow generalization characteristics of the vulnerability-related code of ListUtil.setOrPadding in CVE-2023-42277 are as follows: Figure 2 As shown, the generalization features of data streams are as follows: Figure 3 As shown. The control flow generalization characteristics of the above code snippet for the first test item, hutool, are as follows: Figure 4 As shown, the generalization features of data streams are as follows: Figure 5 As shown. `this` refers to `JsonArray`, which conforms to the principles of a collection. The `JsonArray` interface is defined as `public interface JsonArray extends JsonStructure`, `List`. <jsonvalue>JsonArray directly inherits from List <jsonvalue>JsonArray is an interface, and List is a sub-interface of Collection. Therefore, JsonArray is essentially a Collection type. In the context of the JsonArray implementation class, this directly represents a Collection object.

[0254] The control flow generalization characteristics of the vulnerability-related code in the above-mentioned resource verification missing vulnerability pattern are as follows: Figure 6 As shown, the generalization features of data streams are as follows: Figure 7 As shown. The control flow generalization characteristics of the above code snippet for the second test item are as follows. Figure 8 As shown, the generalization features of data streams are as follows: Figure 9 As shown.

[0255] In another exemplary embodiment of this application, step 601 above, which involves performing control flow generalization feature matching on each target code group, specifically includes:

[0256] Step 801: Perform control flow generalization feature structure matching for each target code group.

[0257] In this embodiment of the application, performing control flow generalization feature structure matching for each target code group means searching for a first target feature flow in the control flow generalization features of the target code fragments in each target code group through a sliding window. The first target feature flow is a control flow generalization feature whose structure is consistent with the control flow generalization features of the code related to the target vulnerability in that target code group.

[0258] For example, for Figure 2 and Figure 4 The control flow generalization features shown are used for structure matching, because Figure 2 In the control flow generalization feature shown, the first branch has a type of IF, the second branch has a type of ELSE, and the ELSE branch contains a WHILE_LOOP. The last function called is of type CALL. Figure 4 In the control flow generalization feature shown, the first branch has a type of IF, the second branch has a type of IF, the third branch has a type of ELSE, and the ELSE branch contains a WHILE_LOOP. The last function called is always a CALL. Therefore, it can be concluded through the sliding window... Figure 4 The structure starting from the second branch in the control flow generalization feature shown is similar to... Figure 2 Control flow generalization feature structure matching, Figure 4 The control flow generalization features shown, starting from the second branch and moving onwards, constitute the first target feature flow.

[0259] For example, regarding Figure 6 and Figure 8 The control flow generalization features shown are used for structure matching, because Figure 6 and Figure 8 The control flow generalization features shown all contain only one branch, and the type of the branch is TRY-CATCH. Therefore, it can be seen that the control flow generalization feature structures of the two are matched.

[0260] Step 802: If the control flow generalization feature structure of each target code group matches, then perform control flow generalization feature content matching for each target code group whose control flow generalization feature structure matches.

[0261] In this embodiment of the application, performing control flow generalization feature content matching on each target code group whose control flow generalization feature structure is matched means performing target control flow generalization feature content matching on the control flow generalization features of the target code segments of each target code group whose control flow generalization feature structure is matched.

[0262] The target control flow generalization feature refers to the part of the control flow generalization feature of the target vulnerability-related code in the target code group that matches the control flow generalization feature structure of the target vulnerability-related code in the target code group.

[0263] Perform target control flow generalization feature content matching, which involves matching the target control flow generalization features with the control flow generalization features of the code related to the target vulnerability. The content includes at least one of the following: the attributes and internal operations of called functions, conditional branch operations (including branch conditions and internal operations), loop branch operations (including loop conditions and internal operations), and operations within exception branches. Specifically, matching is performed on whichever part of the control flow generalization feature is being matched.

[0264] For example, if the matched control flow generalization features include conditional branches, loop branches, and function calls, then the matching is performed on the attributes and internal operations of the conditional branch operations, loop branch operations, and the called functions. For example, for Figure 2 and Figure 4 Content matching is performed on the parts of the control flow generalization features with consistent structure, as shown in the diagram. Figure 2 and Figure 4 In the control flow generalization features shown, the parts with consistent control flow generalization feature structures have the same `condition` type (BINARY_OP) for the IF branches. This means that the branch conditions of the IF branches all include binary expressions. These binary expressions all contain `CALL(collection.size)` and `PARAM`, indicating that the binary expressions in the branch conditions of the IF branches have the same properties. In other words, the branch conditions of the IF branches are consistent. Furthermore, the body of the IF branches all contain `CALL(collection.set, PARAM(index), PARAM(element))`, indicating that the function name `CALL`, the function operation `collection.add`, and the parameter types `PARAM` in the body are all consistent. This means that the operation executed after the IF branch conditions are met is consistent, therefore the IF branch conditions are consistent. In the ELSE branches, the body type (WHILE_LOOP) of the ELSE condition is consistent, meaning they all contain a loop branch. The condition type (BINARY_OP) of the loop branches is also consistent, indicating that the branch conditions of the loop branches all include binary expressions. Furthermore, the binary expressions of the branch conditions of the loop branches all contain CALL(collection.size(), PARAM(index)), meaning that the properties of the binary expressions of the branch conditions of the loop branches are the same, both including function calls and parameter inputs. The function name CALL, the operation content collection.size(), and the parameter type PARAM are all consistent, indicating that the loop conditions are consistent. The body of the loop branches all contain CALL(collection.add, VAR(paddingElement)), indicating that the function name CALL, the operation content collection.add, and the temporary variable VAR in the body are all consistent, meaning that the operation executed after the loop condition is met is consistent. Therefore, the branch conditions of the ELSE are consistent. After the loop operation is executed, the function name CALL in the type of the called function is consistent with the parameter type PARAM, which means that the properties of the called function are consistent. The function operation content collection.add is consistent, which means that the operation performed by the called function is consistent, i.e., the internal operation. Therefore, it can be seen that the control flow generalization features of the two are matched.

[0265] For example, regarding Figure 6 and Figure 8 Content matching is performed on the parts of the control flow generalization features with consistent structure, as shown in the diagram. Figure 6 and Figure 8 In the control flow generalization features shown, the parts with consistent control flow generalization feature structures contain the same type of abnormal branches, meaning that both only contain abnormal control flow processing blocks. Therefore, it can be concluded that the control flow generalization feature content of the two is matched.

[0266] Accordingly, control flow generalization feature matching for each target code group specifically includes:

[0267] The control flow generalization feature structure and content of each target code group are matched.

[0268] In another exemplary embodiment of this application, step 602 above, which involves performing data flow generalization feature matching on each target code group for control flow generalization feature matching, specifically includes:

[0269] A second target feature flow is searched within the data flow generalization features of the target code fragments in each target code group matched by the data flow generalization features. The second target feature flow is the portion of the data flow generalization features of the target code fragments related to the target vulnerability within the target code group matched by the control flow generalization features. Full path consistency refers to the consistency of the type and order of each node on the path.

[0270] In this embodiment, data flow generalization feature matching refers to full-path matching of data flow generalization features. The second target feature flow is searched for in the data flow generalization features of the target code segment in each target code group of the control flow generalization feature matching using a sliding window. Specifically, this includes:

[0271] Using the number of nodes in the full path of the target vulnerability-related code as the size of the sliding window, and setting the sliding window step size to 1 (the sliding window slides one node backward each time), starting from the first node of the full path of the data flow generalization feature of the target code segment in each target code group that matches the control flow generalization feature, search for a data flow in the data flow generalization feature of the target code segment that matches the full path of the data flow generalization feature of the target vulnerability-related code. If a data flow that matches the full path of the data flow generalization feature of the target vulnerability-related code is found, then the data flow generalization feature of the target vulnerability-related code in that target code group is matched.

[0272] For example, for Figure 3 and Figure 5 The matching results of the data flow generalization features shown are all from PARAM->WHILE_LOOP->CALL(collection.add,VAR), with a complete data flow path match. This indicates that the data flow generalization features of the target code snippet and the code related to the target vulnerability in this target code group match. Through the above layer-by-layer feature matching, it is confirmed that the target code snippet contains a vulnerability contained in the code related to the target vulnerability.

[0273] For example, Figure 7 The data flow path of the target vulnerability-related code is as follows: external input function parameter (url), then use the external input to create a database input stream (DriverManager.getConnection(url)), and finally use the database input stream to create an object (connection.createStatement()). Figure 9 The data flow path of the target code snippet shown is as follows: external input as a function parameter (String fileName), then a file input stream is created using the external input (FileInputStream fileStream = new FileInputStream(fileName), and finally, data is read using the file input stream (fileStream.read(data)). Figure 7 and Figure 9 The matching results of the data flow generalization features shown are all from PARAM→RESOURCE_CREATE→RESOURCE_USE, with a complete data flow path match. This indicates that the data flow generalization features of the target code segment and the code related to the target vulnerability match. Through the above layer-by-layer feature matching, it is confirmed that the target code segment contains the vulnerability contained in the code related to the target vulnerability.

[0274] The purpose of data flow feature matching is to use the necessary triggering path in the original vulnerability triggering process as an accurate matching condition, in addition to the vulnerability input point and vulnerability triggering point, thus avoiding false alarms.

[0275] The benefits of multi-level matching:

[0276] Efficiency optimization: By filtering through multiple layers, unnecessary in-depth analysis is avoided;

[0277] False alarm control: A vulnerability is confirmed only when all three layers of features (basic features, control flow features, and data flow features) match, together forming a complete vulnerability detection.

[0278] In another exemplary embodiment of this application, the above-described automatic vulnerability detection method based on publicly available vulnerability data further includes:

[0279] Before step 203, vulnerability pattern information is stored in a database for reuse in vulnerability matching.

[0280] In this embodiment of the application, the vulnerability pattern information includes the vulnerability information source (including the number, type, and repository address of the first target publicly disclosed vulnerability), the related original code (including the vulnerability repair version code and vulnerability triggering version code of the first target publicly disclosed vulnerability), the basic characteristics and generalization characteristics of the extracted target vulnerability-related code, and the repair method for the first target publicly disclosed vulnerability in the original project.

[0281] Based on the same inventive concept, this application also provides an automatic vulnerability detection device based on publicly available vulnerability data for implementing the above-mentioned automatic vulnerability detection method based on publicly available vulnerability data. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations in one or more embodiments of the automatic vulnerability detection device based on publicly available vulnerability data provided below can be found in the limitations of the automatic vulnerability detection method based on publicly available vulnerability data described above, and will not be repeated here.

[0282] In one exemplary embodiment, such as Figure 10 As shown, an automatic vulnerability detection device 90 based on publicly available vulnerability data is provided, comprising:

[0283] The code extraction module 901 is used to obtain the code change record of the vulnerability patch for the first publicly disclosed vulnerability from the open source code platform, determine the location of the vulnerability-related code based on the code change record, and extract the vulnerability-related code from the vulnerability-triggered version code.

[0284] Syntax tree building module 902 is used to analyze vulnerability-related code and obtain an abstract syntax tree;

[0285] The vulnerability matching and detection module 903 is used to perform basic feature and generalized feature matching between the target code and each vulnerability-related code according to the abstract syntax tree. If the target code matches the basic feature and generalized feature of any vulnerability-related code, then the target code contains a first publicly disclosed vulnerability contained in the vulnerability-related code that matches its basic feature and generalized feature.

[0286] In this embodiment of the application, for the code change record of the vulnerability patch for the first target publicly disclosed vulnerability obtained from the open source code platform, the location of the vulnerability-related code is determined according to the code change record, and the relevant description of the vulnerability-related code is extracted from the vulnerability triggering version code. The vulnerability-related code is analyzed to obtain the relevant description of the abstract syntax tree. According to the abstract syntax tree, the target code and each vulnerability-related code are matched with basic features and generalization features respectively. For details, please refer to the description of the above method embodiment, which will not be repeated here.

[0287] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 11 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores automated vulnerability detection data based on publicly available vulnerability data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an automated vulnerability detection method based on publicly available vulnerability data.

[0288] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0289] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0290] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0291] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0292] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0293] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0294] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0295] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0296] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.< / jsonvalue> < / jsonvalue> < / t> < / t> < / t> < / t> < / t>

Claims

1. An automatic vulnerability detection method based on publicly available vulnerability data, characterized in that, The automatic vulnerability detection method based on publicly available vulnerability data includes: The code change records of the vulnerability fix code for each first target publicly disclosed vulnerability are obtained from the open source code platform. After determining the location of the vulnerability-related code based on each code change record, the vulnerability-related code is extracted from the vulnerability-triggered version code. Analyze the code related to each vulnerability to obtain the corresponding abstract syntax tree; Based on each of the abstract syntax trees, basic feature matching is performed on each target code fragment and each target vulnerability-related code; wherein, the target code fragment refers to any code fragment of the target code, the basic features of each code fragment are extracted from the abstract syntax tree, and the target vulnerability-related code refers to the vulnerability-related code of any publicly disclosed vulnerability of the first target; the basic features include a method signature, the method signature includes a return value type, a function name and a parameter list, and the parameter list includes the number of parameters, parameter types and parameter order; If any of the target code fragments matches the basic features of any of the target vulnerability-related code, generalized feature matching is performed on each target code group, wherein the target code group refers to a group of target code fragments and target vulnerability-related code that match the basic features; For each target code group, generalized feature matching is performed, specifically including: Based on the abstract syntax tree of each target code group, generate the control flow graph and data flow graph for each target code group; According to the preset generalization rules, the data of each node in the control flow graph and data flow graph of each target code group are generalized to obtain the control flow generalization feature and data flow generalization feature of each target code group. Perform control flow generalization feature matching for each target code group; If the control flow generalization feature of any target code group matches, perform data flow generalization feature matching on that target code group; If the control flow generalization feature and data flow generalization feature of any target code group are both matched, then a second target public disclosure vulnerability exists in the target code segment located in the target code group; wherein, the second target public disclosure vulnerability is a first target public disclosure vulnerability existing in the target vulnerability-related code of the target code group whose generalization feature is matched.

2. The automatic vulnerability detection method based on publicly available vulnerability data according to claim 1, characterized in that, The process of obtaining the code change record of the vulnerability patch for the first publicly disclosed vulnerability from the open-source code platform, determining the location of the vulnerability-related code based on the code change record, and extracting the vulnerability-related code from the vulnerability-triggered version code specifically includes: After obtaining the URL of the code repository address involved in the patch for the first publicly disclosed vulnerability contained in the tag reference of the open source code platform, download the vulnerability trigger version code and vulnerability patch version code for each first publicly disclosed vulnerability from the open source code platform according to the URL; By using the CVE number of each publicly disclosed vulnerability for each first-target vulnerability, we can filter the code change records for fixing each publicly disclosed vulnerability for each first-target vulnerability from the Commit information, related issues, or PRs. Based on the publicly disclosed code change history for each first-target vulnerability fix, the location of the complete fix code is obtained from the vulnerability fix version code; Extract vulnerability-related code from the vulnerability-triggered version code based on the location of the complete fix code.

3. The automatic vulnerability detection method based on publicly available vulnerability data according to claim 1, characterized in that, The process of matching each target code fragment with the basic features of each target vulnerability-related code specifically includes: Perform method signature matching between each target code fragment and each target vulnerability-related code; If any of the target code fragments matches at least two of the return value type, function name, and parameter list of the function in the method signature of any of the target vulnerability-related code, then the target code fragment matches the method signature of the target vulnerability-related code. Among them, return value type matching means that the return value types are the same, function name matching means that the function names are the same, and parameter list matching means that the parameter types of the target code are all located in the parameter type set of the target vulnerability-related code.

4. The automatic vulnerability detection method based on publicly available vulnerability data according to claim 3, characterized in that, The basic features also include library function call features; The step of performing basic feature matching between each target code fragment and the code related to each target vulnerability also includes: For each group of method signatures, target code snippets and code related to the target vulnerability are matched for library function call features. If both the target code segment and the target vulnerability-related code that match the method signature contain library functions, and the library functions in the target code segment and the target vulnerability-related code satisfy the matching condition of library function call characteristics, then the basic characteristics of the target code segment and the target vulnerability-related code match.

5. The automatic vulnerability detection method based on publicly available vulnerability data according to claim 4, characterized in that, The library function call feature includes at least the library function name, and the matching conditions for the library function call feature include at least the following: The function names of the target code snippets and library functions in the target vulnerability-related code that match the signatures of any of the methods are consistent.

6. The automatic vulnerability detection method based on publicly available vulnerability data according to claim 5, characterized in that, The library function call characteristics also include at least one of the class name, package name, and version number where the library function is defined; The matching conditions for the library function call characteristics also include: The class name, package name, and / or version number of the library function in any of the target code snippets and target vulnerability-related code that match the method signatures of the group.

7. The automatic vulnerability detection method based on publicly available vulnerability data according to claim 1, characterized in that, The preset generalization rules include at least one of branch generalization rules and function call generalization rules, and the branch generalization rules include at least one of conditional branch generalization rules, loop branch generalization rules, and exception branch generalization rules. The generalization process, according to preset generalization rules, generalizes the data of each node in the control flow graph and data flow graph of each target code group, specifically including: If any target code group contains function calls related to a target vulnerability, then the function call generalization rule is used to generalize the data of each function call node in the control flow graph and data flow graph of the target code group. If any target code group contains conditional branches related to a target vulnerability, then the conditional branch generalization rule is used to generalize the data of each conditional branch node in the control flow graph and data flow graph of the target code group. If any target code group contains loop branches in its target vulnerability-related code, then the loop branch generalization rule is used to generalize the data of each loop branch node in the control flow graph and data flow graph of the target code group. If any target code group contains an abnormal branch in the target vulnerability-related code, then the abnormal branch generalization rule is used to generalize the data of the abnormal branch node of the target code group.

8. The automatic vulnerability detection method based on publicly available vulnerability data according to claim 1, characterized in that, The control flow generalization feature matching for each target code group specifically includes: Perform control flow generalization feature structure matching on each of the target code groups; If the control flow generalization feature structure of each target code group matches, then control flow generalization feature content matching is performed on each target code group whose control flow generalization feature structure matches; wherein, the content includes at least one of the attributes and internal operations of the function called in the control flow generalization feature, conditional branch operations, loop branch operations, and exception branch operations. Control flow generalization feature matching for each target code group specifically includes: The control flow generalization feature structure and content of each target code group are matched; The process of performing data stream generalization feature matching for each target code group specifically includes: A second target feature flow is searched in the data flow generalization features of the target code segment in each target code group that matches the control flow generalization features by using a sliding window; wherein, the second target feature flow is the part of the data flow generalization features of the target code segment in the target code group that matches the target vulnerability-related code.

Images