Code vulnerability detection method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a code vulnerability detection method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining a target program; vulnerability code features generated based on a vulnerability code library are obtained, the target program is processed to obtain target code features, and the target code features are generated through a feature generation method the same as the vulnerability code features; calculating the similarity between the target code feature and each vulnerability code feature; and under the condition that the similarity is greater than a threshold value, determining that the target program has vulnerabilities. By adopting the method, the code vulnerability detection accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of code security, and in particular, to a method, apparatus, computer device, computer-readable storage medium, and computer program product for code vulnerability detection. Background Art

[0002] With the development of network technology, network security has become increasingly important. The key factor for network security incidents is the existence of vulnerabilities in programs. Therefore, vulnerability detection technology is a research hotspot both in the academic community and in the industrial community. In the field of network security, a vulnerability refers to a defect in the specific implementation of hardware, software, protocols, or system security policies, which enables an attacker to access or damage the system without authorization.

[0003] Code reuse, as the name implies, means directly copying or slightly adjusting some code in certain open-source software projects during the software development process and using it in other software. Although code copying can speed up the software development process, it often seriously affects software security because code copying easily spreads vulnerabilities. Moreover, with the rapid growth of the number of open-source software, the reuse of vulnerable code has also increased significantly, thus contaminating many software systems.

[0004] In traditional technologies, the identification of vulnerabilities is usually based on static code analysis. However, for cases where the code changes significantly, the scheme based on static code analysis may have false negatives and the detection accuracy is relatively low. Summary of the Invention

[0005] Based on this, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for code vulnerability detection that can improve the accuracy of code vulnerability detection in view of the above technical problems.

[0006] In a first aspect, the present application provides a method for code vulnerability detection, the method comprising:

[0007] Obtain a target program;

[0008] Obtain vulnerability code features generated based on a vulnerability code library, and process the target program to obtain target code features, where the target code features are generated using the same feature generation method as the vulnerability code features;

[0009] Calculate the similarity between the target code features and each of the vulnerability code features;

[0010] If the similarity is greater than a threshold, determine that the target program has a vulnerability.

[0011] In one embodiment, obtaining the vulnerability code features generated based on the vulnerability code library and processing the target program to obtain the target code features includes:

[0012] Parsing the vulnerability codes in the vulnerability code library to generate an initial abstract syntax tree corresponding to the vulnerability codes;

[0013] Abstracting the initial abstract syntax tree corresponding to the vulnerability codes to obtain a target abstract syntax tree corresponding to the vulnerability codes, and extracting features from the target abstract syntax tree corresponding to the vulnerability codes to obtain the vulnerability code features;

[0014] Parsing the target program to generate an initial abstract syntax tree corresponding to the target program;

[0015] Abstracting the initial abstract syntax tree corresponding to the target program to obtain a target abstract syntax tree corresponding to the target program, and extracting features from the target abstract syntax tree corresponding to the target program to obtain the target code features.

[0016] In one embodiment, the abstraction of the initial abstract syntax tree includes:

[0017] Replacing the formal parameter variables in the function declarations in the initial abstract syntax tree with a first symbol;

[0018] Replacing the local variables in the initial abstract syntax tree with a second symbol;

[0019] Replacing the data types in the initial abstract syntax tree with a third symbol;

[0020] Replacing the function calls in the initial abstract syntax tree with a fourth symbol; where the first symbol, the second symbol, the third symbol, and the fourth symbol are different symbols.

[0021] In one embodiment, the feature extraction of the target abstract syntax tree includes:

[0022] Traversing the target abstract syntax tree to obtain the number of nodes in the target abstract syntax tree;

[0023] Merging the nodes to obtain a marked string, and converting the marked string into a hash value;

[0024] Concatenating the number of nodes and the hash value to obtain a binary tuple, and using each binary tuple as a code feature.

[0025] In one embodiment, calculating the similarity between the target code features and each of the vulnerability code features includes:

[0026] Obtaining the binary tuple corresponding to the target program;

[0027] Match the number of nodes of the binary tuple corresponding to the target program with the number of nodes of each binary tuple corresponding to the vulnerability code library;

[0028] If there is a binary tuple with the same number of nodes in the vulnerability code library, then match the hash value of the binary tuple corresponding to the target program with the hash value of the binary tuple with the same number of nodes;

[0029] If there is a binary tuple with the same hash value in the vulnerability code library, it is determined that the target program has a vulnerability;

[0030] If there is no binary tuple with the same hash value in the vulnerability code library, or there is no binary tuple with the same number of nodes in the vulnerability code library, it is determined that the target program has no vulnerability.

[0031] In one embodiment, before obtaining the vulnerability code features generated based on the vulnerability code library, it further includes:

[0032] Obtain each standard code;

[0033] Identify keywords in the log corresponding to the standard code, and obtain the corresponding vulnerability description information and security patch information based on the identified keywords;

[0034] Based on the vulnerability description information and security patch information, obtain the vulnerability code from the original code corresponding to the standard code;

[0035] Generate a vulnerability code library based on the obtained vulnerability code.

[0036] In a second aspect, the present application also provides a code vulnerability detection device, and the device includes:

[0037] A target program acquisition module, configured to acquire a target program;

[0038] A code feature generation module, configured to obtain vulnerability code features generated based on a vulnerability code library, and process the target program to obtain target code features, wherein the target code features are generated by the same feature generation method as the vulnerability code features;

[0039] A similarity calculation module, configured to calculate the similarity between the target code features and each of the vulnerability code features;

[0040] A vulnerability detection module, configured to determine that the target program has a vulnerability when the similarity is greater than a threshold.

[0041] In a third aspect, the present application further provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method in any one of the above embodiments are implemented.

[0042] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in any one of the above embodiments are implemented.

[0043] In a fifth aspect, the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the method in any one of the above embodiments are implemented.

[0044] For the above code vulnerability detection method, device, computer device, computer-readable storage medium and computer program product, a target program is obtained; vulnerability code features generated based on a vulnerability code library are obtained, and the target program is processed to obtain target code features, where the target code features are generated by the same feature generation method as the vulnerability code features; the similarity between the target code features and each of the vulnerability code features is calculated; when the similarity is greater than a threshold, it is determined that the target program has a vulnerability. In this way, vulnerability code features are dynamically generated based on the vulnerability code library, and then vulnerability identification of the target program is performed based on the dynamically generated vulnerability code features, which can improve the accuracy of vulnerability identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0046] Figure 1 It is an application environment diagram of the code vulnerability detection method in an embodiment;

[0047] Figure 2 It is a flowchart of the code vulnerability detection method in an embodiment;

[0048] Figure 3 It is a flowchart of the feature extraction step in an embodiment;

[0049] Figure 4 It is a flowchart of the similarity calculation step in an embodiment;

[0050] Figure 5 It is a flowchart of the code vulnerability detection method in another embodiment;

[0051] Figure 6 It is a structural block diagram of a code vulnerability detection device in an embodiment;

[0052] Figure 7 It is an internal structure diagram of a computer device in an embodiment. Specific implementation manners

[0053] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] The code vulnerability detection method provided by the embodiments of the present application can be applied to an application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed in the cloud or other network servers.

[0055] Among them, the terminal 102 can send the target program to the server 104. It should be noted that the target program here can be one or multiple. The server 104 can process these target programs in sequence or in parallel. The server 104 obtains the vulnerability code features generated based on the vulnerability code library, and processes the target program to obtain the target code features, where the target code features are generated by the same feature generation method as the vulnerability code features; calculate the similarity between the target code features and each vulnerability code feature; in the case where the similarity is greater than the threshold, determine that the target program has a vulnerability. In this way, the vulnerability code features are dynamically generated based on the vulnerability code library, and then the vulnerability identification of the target program is performed based on the dynamically generated vulnerability code features, which can improve the accuracy of vulnerability identification.

[0056] Among them, the terminal 102 can be but is not limited to various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0057] In an exemplary embodiment, asFigure 2 As shown, a code vulnerability detection method is provided. Taking the server in Figure 1 as an example for illustration, it includes the following steps S202 to S208. Among them:

[0058] S202: Obtain the target program.

[0059] The target program is the program to be detected for vulnerabilities. There can be one or more target programs. In the case of multiple target programs, the vulnerability detection of the target programs can be carried out serially or in parallel, and no specific limitation is made here.

[0060] S204: Obtain the vulnerability code features generated based on the vulnerability code library, and process the target program to obtain the target code features, where the target code features are generated using the same feature generation method as the vulnerability code features.

[0061] The vulnerability code features respectively correspond to the respective vulnerability codes in the vulnerability code library, and each vulnerability code corresponds to a vulnerability code feature. Optionally, the vulnerability code feature can be obtained based on the number of nodes of the abstract syntax tree corresponding to the function in the vulnerability code and the token string of the function. Specifically, the vulnerability code feature includes the number of nodes and the hash value corresponding to the token string of the function. For example, the vulnerability code feature can include (number of nodes, hash value), where the hash value is the hash value corresponding to the token string of the function. In this way, the code feature of the function corresponding to each vulnerability code can be represented by this binary tuple (number of nodes, hash value). The set of the code features of the functions corresponding to the respective vulnerability codes is used as the vulnerability code features corresponding to the vulnerability code library. These binary tuples can also be used as a dictionary, that is, the vulnerability dictionary corresponding to the vulnerability code library. The information in these dictionaries is updated based on the update of the vulnerability codes in the vulnerability code library, and there is no need to calculate each time when determining whether the target program has vulnerabilities. In other embodiments, the vulnerability code features can be calculated each time when determining whether the target program has vulnerabilities, and the vulnerability code features can be the vulnerability code features calculated for all the vulnerability codes in the vulnerability code library, or only the vulnerability code features calculated for the updated vulnerability codes, and the existing vulnerability code features are obtained for the other vulnerability codes.

[0062] The target code features corresponding to the target program are generated using the same feature generation method as the vulnerability code features. The target code features can be obtained based on the number of nodes in the abstract syntax tree corresponding to the functions in the target program and the token string of the function. Specifically, the target code features include the number of nodes and the hash value corresponding to the token string of the function. For example, the target code features can include (number of nodes, hash value), where the hash value is the hash value corresponding to the token string of the function. In this way, the code features of each function corresponding to the target program can be represented by this binary tuple (number of nodes, hash value), and the set of code features of each function corresponding to the target program is used as the dictionary of the vulnerability to be detected.

[0063] In summary, the server can obtain the dictionary of vulnerabilities (vulnerability code features) corresponding to the vulnerability codes in the vulnerability code library and the dictionary of the vulnerabilities to be detected (target code features) corresponding to the target program, so as to facilitate subsequent comparison.

[0064] In addition, it should be noted that in the process of generating the vulnerability code features and the target code features, the generation of the abstract syntax tree can be based on the existing generation rules of the abstract syntax tree and be transformed through the unique transformation rules in this application to expand the ability of vulnerability identification, so that it can detect unknown vulnerabilities and has a high detection accuracy.

[0065] S206: Calculate the similarity between the target code features and each vulnerability code feature.

[0066] S208: When the similarity is greater than the threshold, determine that the target program has a vulnerability.

[0067] Among them, calculating the similarity between the target code features and each vulnerability code feature can include calculating the similarity based on the above binary tuple. Since the above binary tuple includes the number of nodes (which can be regarded as a key value) and the hash value, it has good time complexity and space complexity.

[0068] Calculating the similarity based on the above binary tuple can include calculating the similarity between the binary tuple of the target code features and the binary tuples of each vulnerability code feature to determine whether there is a vulnerability code feature whose similarity with the target code features is greater than the threshold. If so, it means that the target program corresponding to the target code features has a vulnerability.

[0069] Among them, in the case of multiple target programs, after comparing the target code features of one target program, continue to compare the target code features of the next target program until all the target code features are compared.

[0070] The above code vulnerability detection method includes: obtaining a target program; obtaining vulnerability code features generated based on a vulnerability code library, and processing the target program to obtain target code features, where the target code features are generated using the same feature generation method as the vulnerability code features; calculating the similarity between the target code features and each vulnerability code feature; and determining that the target program has a vulnerability when the similarity is greater than a threshold. In this way, vulnerability code features are dynamically generated based on the vulnerability code library, and then vulnerability identification is performed on the target program based on the dynamically generated vulnerability code features, which can improve the accuracy of vulnerability identification.

[0071] In one optional embodiment, obtaining vulnerability code features generated based on a vulnerability code library and processing the target program to obtain target code features includes: parsing the vulnerability codes in the vulnerability code library to generate an initial abstract syntax tree corresponding to the vulnerability codes; abstracting the initial abstract syntax tree corresponding to the vulnerability codes to obtain a target abstract syntax tree corresponding to the vulnerability codes, and extracting features from the target abstract syntax tree corresponding to the vulnerability codes to obtain vulnerability code features; parsing the target program to generate an initial abstract syntax tree corresponding to the target program; abstracting the initial abstract syntax tree corresponding to the target program to obtain a target abstract syntax tree corresponding to the target program, and extracting features from the target abstract syntax tree corresponding to the target program to obtain target code features.

[0072] Among them, the same method is used to process the vulnerability codes and the target program. In this application, only the example of generating vulnerability code features from vulnerability codes is described. Those skilled in the art can determine the method of determining target code features based on the target program based on the method of generating vulnerability code features from vulnerability codes, which will not be elaborated here.

[0073] Among them, first, the vulnerability codes in the vulnerability code library are parsed to obtain tokens, and then syntax analysis is performed according to context-free grammar to obtain an abstract syntax tree AST. The process of parsing the vulnerability codes mainly performs lexical analysis on the program source code according to regular expressions, and finally obtains tokens.

[0074] Abstracting the initial abstract syntax tree corresponding to the vulnerability codes to obtain a target abstract syntax tree corresponding to the vulnerability codes, that is, abstracting supplementary information such as formal parameters, data types used, local variables, and function calls of the obtained vulnerability codes according to some conversion rules to obtain the target abstract syntax tree.

[0075] In some alternative embodiments, the abstraction of the initial abstract syntax tree includes: replacing the formal parameter variables in the function declarations in the initial abstract syntax tree with a first symbol; replacing the local variables in the initial abstract syntax tree with a second symbol; replacing the data types in the initial abstract syntax tree with a third symbol; and replacing the function calls in the initial abstract syntax tree with a fourth symbol. The first symbol, the second symbol, the third symbol, and the fourth symbol are different symbols.

[0076] Among them, in the present application, formal parameter variables, local variables, data types, and function calls can be abstracted. Specifically:

[0077] Formal parameter abstraction: Collect formal parameter variables from function declarations, and replace each occurrence of a formal parameter variable in the function with the symbol f_param, where the symbol f_param is the first symbol.

[0078] Local variable abstraction: Replace all local variables that appear in the function body with the symbol l_var, where the symbol l_var is the second symbol.

[0079] Data type abstraction: Replace all data types that appear in the function with the symbol d_type. The data types here not only include standard C language data types and qualifiers, but also user-defined data types, where the symbol d_type is the third symbol.

[0080] Function call abstraction: Replace the name of each called function with the symbol fun_call, where the symbol fun_call is the fourth symbol.

[0081] In one of the alternative embodiments, the feature extraction of the target abstract syntax tree includes: traversing the target abstract syntax tree to obtain the number of nodes in the target abstract syntax tree; merging the nodes to obtain a token string, and converting the token string into a hash value; concatenating the number of nodes and the hash value to obtain a binary tuple, and using each binary tuple as a code feature.

[0082] After obtaining the target abstract syntax tree, code features need to be obtained based on this target abstract syntax tree. For this, in the present application, the abstract syntax tree of each function is traversed and parsed to obtain the number of nodes in the abstract syntax tree and the token string of the function, obtaining code features composed of the number of function nodes and the corresponding token string, converting the token string into a hash value, obtaining a dictionary composed of two binary tuples of (number of nodes, hash value), which maps the number of nodes to the hash value. Since different hash values may correspond to the same number of nodes, different functions are stored classified by the number of nodes.

[0083] Specifically, in combination with Figure 3 as shown Figure 3It is a flowchart of the feature extraction step in an embodiment. In this embodiment, the feature extraction step includes:

[0084] First, traverse the target abstract syntax tree and record the number of nodes.

[0085] Second, merge the information of each node of the target abstract syntax tree together to form a token string of the function, and the token string is represented in the form of a string.

[0086] Third, the binary tuple composed of the number of nodes and the token string is the code feature of the function. The first element of the binary tuple (i.e., the number of nodes) is also called the key value. The MD5 hash algorithm is used to convert the token string into the corresponding hash value, so as to convert the code feature into the representation form of (key value, hash value).

[0087] Fourth, execute the above three steps on all elements in the abstract syntax tree sets of the vulnerability code library and the target program source code library respectively, and obtain two code feature dictionaries GV and GT composed of binary tuples, corresponding to the vulnerability code library and the target program source code library respectively. A dictionary is an associative container that maps key values to hash values. Since different functions may share the same key value, the functions are stored classified by key value for easy lookup.

[0088] In the above embodiment, the construction of the abstract syntax tree AST is used to create vulnerability feature fingerprint information. By parsing the target function code and abstracting the function, highly abstract vulnerability function features are obtained, shielding the function implementation details and focusing on the vulnerability code features related to code vulnerability. Combining the one-way property and the fixed output format of the hash function, the abstracted vulnerability code is hashed to obtain a unique vulnerability feature fingerprint. When comparing, the same construction method is used for the target function, and then the vulnerability feature matching is performed. Through this construction method, the interference of irrelevant function implementation details on the identification of vulnerable code is greatly reduced, and the detection efficiency is improved; the highly abstract vulnerability feature fingerprint also saves various overheads for fingerprint storage, and the obtained dictionary has good usability.

[0089] In one optional embodiment, calculating the similarity between the target code feature and each vulnerability code feature includes: obtaining the binary tuple corresponding to the target program; matching the number of nodes of the binary tuple corresponding to the target program with the number of nodes of each binary tuple corresponding to the vulnerability code library; if there is a binary tuple with the same number of nodes in the vulnerability code library, then match the hash value of the binary tuple corresponding to the target program with the hash value of the binary tuple with the same number of nodes; if there is a binary tuple with the same hash value in the vulnerability code library, it is determined that the target program has a vulnerability; if there is no binary tuple with the same hash value in the vulnerability code library, or there is no binary tuple with the same number of nodes in the vulnerability code library, it is determined that the target program does not have a vulnerability.

[0090] Specifically, in combination with Figure 4 as shown, Figure 4 FIG. Figure 4 is a flowchart of the similarity calculation steps in an embodiment. In this embodiment, the following steps are mainly included:

[0091] 4.1 First, obtain the vulnerability dictionary and the target dictionary. The vulnerability dictionary includes the vulnerability code features corresponding to each vulnerability code in the vulnerability code library, and the target dictionary includes the target code features corresponding to the target program.

[0092] 4.2 Key value (i.e., the number of nodes) lookup: Read the key value (i.e., the number of nodes of the corresponding AST) of a function f in the target program dictionary GT, and check whether there is a function with the same key value in the vulnerability dictionary GV. If the lookup fails, it is determined that the function to be tested is not similar to the vulnerability code segment, and jump to step 4.4; otherwise, set the key value in the vulnerability dictionary GV as key and execute the next step;

[0093] 4.3 Hash value lookup: Look up the functions in the category with the key value of key in the vulnerability dictionary, and compare whether there is a function in the vulnerability dictionary whose hash value is the same as that of the function to be tested. If not, it is considered that the function in the current target program is not similar to the vulnerability code segment; otherwise, it is determined that it is a vulnerability function and recorded.

[0094] 4.4 Determine whether all the functions to be tested in the target programs in the target program dictionary GT have been detected. If not, jump to step 4.1 to execute; otherwise, output the vulnerability detection result and end.

[0095] In the above embodiment, the construction of the vulnerability feature library and the vulnerability code detection method can greatly improve the detection efficiency of code vulnerability detection, and have good application scenarios in the context of increasing code reusability; in addition, good detection results and detection efficiency have also been achieved in the vulnerability detection of large-scale software.

[0096] In one optional embodiment, before obtaining the vulnerability code features generated based on the vulnerability code library, it further includes: obtaining each standard code; identifying keywords in the logs corresponding to the standard code, and obtaining the corresponding vulnerability description information and security patch information based on the identified keywords; obtaining the vulnerability code from the original code corresponding to the standard code based on the vulnerability description information and the security patch information; generating a vulnerability code library based on the obtained vulnerability code.

[0097] Among them, the construction of the vulnerability code library in this application can be realized by analyzing the patches provided in the open-source project repository, specifically including:

[0098] Download a large number of relatively new and popular Git repositories;

[0099] Search for relevant commit logs based on some general keywords, such as vulnerability type, vulnerability name, CVE number, etc.

[0100] Based on the commit logs, obtain the descriptions of CVE-related vulnerabilities and security patch information in unified diff format.

[0101] Filter out irrelevant commits and retrieve vulnerable functions from old, unpatched files. Finally, pack all the finally retrieved vulnerable functions into a vulnerability code library, which is saved in the form of code files with functions as units.

[0102] In the above embodiments, instead of relying on a vulnerability feature knowledge base that requires experts or researchers to participate in the construction themselves, the vulnerability code features in the vulnerability patches of open-source projects are extracted to detect unknown vulnerabilities. The vulnerability features are sourced from the version update information of the Git repositories of various well-known open-source software codes in the open-source community. By extracting commit information with vulnerability type, vulnerability name, CVE number, etc., a vulnerability knowledge base for detecting the vulnerabilities of open-source software is constructed, which has the advantages of wide code function coverage, complete code structure, and relatively small construction time and space costs.

[0103] For ease of understanding, combined with Figure 5 as shown in Figure 5 is a flowchart of a code vulnerability detection method in another embodiment. In this embodiment, a vulnerability library is constructed by analyzing the commit information and patch files in the public vulnerability library and the open-source project repository, and an optimized abstract method for detecting vulnerability code reuse and a new type of code feature representing functions are used, so as to be able to detect unknown vulnerability code reuse and greatly improve the detection efficiency.

[0104] Specifically, the code vulnerability detection method specifically includes the following steps, where the processing of the target program and the vulnerability code can be parallel or physically isolated from each other, and no specific limitation is made here.

[0105] Step 1: Generate vulnerability code by analyzing the patches provided in the open-source project repository and construct a vulnerability code library.

[0106] Step 2: Parse the source code libraries of the vulnerability code library and the target program respectively, and generate the abstract syntax tree sets of the vulnerability code library and the target program source code library according to the generation rules of the abstract syntax tree.

[0107] Step 3: Abstract the supplementary information such as the formal parameters, data types used, local variables, and function calls of the program code obtained in Step 2 according to some conversion rules to obtain a new abstract syntax tree set.

[0108] Step 4: Traverse and parse the abstract syntax tree of each function to obtain the number of nodes in the abstract syntax tree and the token string of the function, obtain the code features composed of the number of function nodes and the corresponding token string, convert the token string into a hash value, and obtain two dictionaries composed of binary tuples of (number of nodes, hash value). This dictionary maps the number of nodes to the hash value. Since different hash values may correspond to the same number of nodes, different functions are stored classified by the number of nodes.

[0109] Step 5: Calculate the similarity, detect whether there is reuse of vulnerable code, and output the result.

[0110] The detailed process of Step 5 includes:

[0111] Number of nodes search: Iteratively loop through the first element (i.e., the number of nodes) of each binary tuple in the target program dictionary to search whether there is the same number of nodes in the vulnerability dictionary. If the search fails, it is determined that the function to be tested is not similar to the vulnerable code segment; otherwise, proceed to the next hash value search.

[0112] Hash value search: Search whether there is a hash value in the hash values mapped to this number of nodes in the vulnerability dictionary that is the same as the hash value of the function to be tested. If not, it is considered that the current function to be tested is not similar to the vulnerable code segment; otherwise, it is determined that it is similar to the vulnerable function.

[0113] In the above embodiments, taking the function as the basic unit, first extract the abstract syntax tree (AST) of the function, and then use an abstraction technique on it to expand the ability of vulnerability identification, enabling it to detect unknown vulnerabilities and having a relatively high detection accuracy. In addition, by converting the comparison of code similarity into the comparison of key values and hash values, it has good time complexity and space complexity.

[0114] It should be understood that although the steps in the flowcharts involved in the above embodiments are sequentially shown according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0115] Based on the same inventive concept, an embodiment of the present application further provides a code vulnerability detection device for implementing the above-mentioned code vulnerability detection method. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following code vulnerability detection devices can refer to the limitations on the code vulnerability detection method in the above text, and will not be elaborated here.

[0116] In an exemplary embodiment, as Figure 6 shown, a code vulnerability detection device is provided, including: a target program acquisition module 601, a code feature generation module 602, a similarity calculation module 603, and a vulnerability detection module 604, where:

[0117] The target program acquisition module 601 is used to acquire a target program;

[0118] The code feature generation module 602 is used to acquire vulnerability code features generated based on a vulnerability code library, and process the target program to obtain target code features, where the target code features are generated using the same feature generation method as the vulnerability code features;

[0119] The similarity calculation module 603 is used to calculate the similarity between the target code features and each vulnerability code feature;

[0120] The vulnerability detection module 604 is used to determine that the target program has a vulnerability when the similarity is greater than a threshold.

[0121] In one optional embodiment, the above-mentioned code feature generation module 602 is specifically used to parse the vulnerability code in the vulnerability code library to generate an initial abstract syntax tree corresponding to the vulnerability code; abstract the initial abstract syntax tree corresponding to the vulnerability code to obtain a target abstract syntax tree corresponding to the vulnerability code, and extract features from the target abstract syntax tree corresponding to the vulnerability code to obtain vulnerability code features; parse the target program to generate an initial abstract syntax tree corresponding to the target program; abstract the initial abstract syntax tree corresponding to the target program to obtain a target abstract syntax tree corresponding to the target program, and extract features from the target abstract syntax tree corresponding to the target program to obtain target code features.

[0122] In one optional embodiment, the above-mentioned code feature generation module 602 is specifically used to replace the formal parameter variables in the function declarations in the initial abstract syntax tree with a first symbol; replace the local variables in the initial abstract syntax tree with a second symbol; replace the data types in the initial abstract syntax tree with a third symbol; replace the function calls in the initial abstract syntax tree with a fourth symbol; where the first symbol, the second symbol, the third symbol, and the fourth symbol are different symbols.

[0123] In one alternative embodiment, the above-mentioned code feature generation module 602 is specifically configured to traverse the target abstract syntax tree to obtain the number of nodes of the target abstract syntax tree; merge each node to obtain a token string, and convert the token string into a hash value; splice the number of nodes and the hash value to obtain a binary tuple, and use each binary tuple as a code feature.

[0124] In one alternative embodiment, the above-mentioned similarity calculation module 603 is specifically configured to obtain the binary tuples corresponding to the target program; match the number of nodes of the binary tuples corresponding to the target program with the number of nodes of each binary tuple corresponding to the vulnerability code library; if there is a binary tuple with the same number of nodes in the vulnerability code library, then match the hash value of the binary tuple corresponding to the target program with the hash value of the binary tuple with the same number of nodes; if there is a binary tuple with the same hash value in the vulnerability code library, it is determined that the target program has a vulnerability; if there is no binary tuple with the same hash value in the vulnerability code library, or there is no binary tuple with the same number of nodes in the vulnerability code library, it is determined that the target program has no vulnerability.

[0125] In one alternative embodiment, the above-mentioned device further includes: a vulnerability code library generation module, configured to obtain each standard code; identify keywords in the log corresponding to the standard code, and obtain the corresponding vulnerability description information and security patch information based on the identified keywords; based on the vulnerability description information and security patch information, obtain the vulnerability code from the original code corresponding to the standard code; generate a vulnerability code library based on the obtained vulnerability code.

[0126] Each module in the above-mentioned code vulnerability detection device can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor in the computer device in hardware form or independent of it, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0127] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as Figure 7As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store a vulnerability code library, etc. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a method for detecting code vulnerabilities.

[0128] Those skilled in the art can understand that Figure 7 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0129] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented: obtaining a target program; obtaining vulnerability code features generated based on a vulnerability code library, and processing the target program to obtain target code features, where the target code features are generated using the same feature generation method as the vulnerability code features; calculating the similarity between the target code features and each vulnerability code feature; and determining that the target program has a vulnerability when the similarity is greater than a threshold.

[0130] In one embodiment, the obtaining of the vulnerability code features generated based on the vulnerability code library and the processing of the target program to obtain the target code features implemented when the processor executes the computer program includes: parsing the vulnerability codes in the vulnerability code library to generate an initial abstract syntax tree corresponding to the vulnerability codes; abstracting the initial abstract syntax tree corresponding to the vulnerability codes to obtain a target abstract syntax tree corresponding to the vulnerability codes, and extracting features from the target abstract syntax tree corresponding to the vulnerability codes to obtain the vulnerability code features; parsing the target program to generate an initial abstract syntax tree corresponding to the target program; abstracting the initial abstract syntax tree corresponding to the target program to obtain a target abstract syntax tree corresponding to the target program, and extracting features from the target abstract syntax tree corresponding to the target program to obtain the target code features.

[0131] In one embodiment, the abstraction of the initial abstract syntax tree implemented when the processor executes a computer program includes: replacing the formal parameter variables in the function declarations in the initial abstract syntax tree with a first symbol; replacing the local variables in the initial abstract syntax tree with a second symbol; replacing the data types in the initial abstract syntax tree with a third symbol; replacing the function calls in the initial abstract syntax tree with a fourth symbol; wherein the first symbol, the second symbol, the third symbol, and the fourth symbol are different symbols.

[0132] In one embodiment, the feature extraction of the target abstract syntax tree implemented when the processor executes a computer program includes: traversing the target abstract syntax tree to obtain the number of nodes in the target abstract syntax tree; merging each node to obtain a marked string, and converting the marked string into a hash value; concatenating the number of nodes and the hash value to obtain a binary tuple, and using each binary tuple as a code feature.

[0133] In one embodiment, calculating the similarity between the target code feature and each vulnerability code feature implemented when the processor executes a computer program includes: obtaining the binary tuple corresponding to the target program; matching the number of nodes of the binary tuple corresponding to the target program with the number of nodes of each binary tuple corresponding to the vulnerability code library; if there is a binary tuple with the same number of nodes in the vulnerability code library, then matching the hash value of the binary tuple corresponding to the target program with the hash value of the binary tuple with the same number of nodes; if there is a binary tuple with the same hash value in the vulnerability code library, then determining that the target program has a vulnerability; if there is no binary tuple with the same hash value in the vulnerability code library, or there is no binary tuple with the same number of nodes in the vulnerability code library, then determining that the target program has no vulnerability.

[0134] In one embodiment, before obtaining the vulnerability code features generated based on the vulnerability code library implemented when the processor executes a computer program, it further includes: obtaining each standard code; identifying keywords in the log corresponding to the standard code, and obtaining the corresponding vulnerability description information and security patch information based on the identified keywords; obtaining the vulnerability code from the original code corresponding to the standard code based on the vulnerability description information and the security patch information; generating a vulnerability code library based on the obtained vulnerability code.

[0135] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: obtaining a target program; obtaining the vulnerability code features generated based on the vulnerability code library, and processing the target program to obtain a target code feature, wherein the target code feature is generated by the same feature generation method as the vulnerability code feature; calculating the similarity between the target code feature and each vulnerability code feature; and determining that the target program has a vulnerability when the similarity is greater than a threshold.

[0136] In one embodiment, when the computer program is executed by a processor, obtaining vulnerability code features generated based on a vulnerability code library and processing a target program to obtain target code features includes: parsing the vulnerability codes in the vulnerability code library to generate an initial abstract syntax tree corresponding to the vulnerability codes; abstracting the initial abstract syntax tree corresponding to the vulnerability codes to obtain a target abstract syntax tree corresponding to the vulnerability codes, and extracting features from the target abstract syntax tree corresponding to the vulnerability codes to obtain vulnerability code features; parsing the target program to generate an initial abstract syntax tree corresponding to the target program; abstracting the initial abstract syntax tree corresponding to the target program to obtain a target abstract syntax tree corresponding to the target program, and extracting features from the target abstract syntax tree corresponding to the target program to obtain target code features.

[0137] In one embodiment, the abstraction of the initial abstract syntax tree when the computer program is executed by a processor includes: replacing the formal parameter variables in the function declarations in the initial abstract syntax tree with a first symbol; replacing the local variables in the initial abstract syntax tree with a second symbol; replacing the data types in the initial abstract syntax tree with a third symbol; replacing the function calls in the initial abstract syntax tree with a fourth symbol; where the first symbol, the second symbol, the third symbol, and the fourth symbol are different symbols.

[0138] In one embodiment, the feature extraction of the target abstract syntax tree when the computer program is executed by a processor includes: traversing the target abstract syntax tree to obtain the number of nodes in the target abstract syntax tree; merging the nodes to obtain a marked string, and converting the marked string into a hash value; concatenating the number of nodes and the hash value to obtain a binary tuple, and using each binary tuple as a code feature.

[0139] In one embodiment, calculating the similarity between the target code features and each vulnerability code feature when the computer program is executed by a processor includes: obtaining the binary tuple corresponding to the target program; matching the number of nodes of the binary tuple corresponding to the target program with the number of nodes of each binary tuple corresponding to the vulnerability code library; if there is a binary tuple with the same number of nodes in the vulnerability code library, then matching the hash value of the binary tuple corresponding to the target program with the hash value of the binary tuple with the same number of nodes; if there is a binary tuple with the same hash value in the vulnerability code library, then determining that the target program has a vulnerability; if there is no binary tuple with the same hash value in the vulnerability code library, or there is no binary tuple with the same number of nodes in the vulnerability code library, then determining that the target program has no vulnerability.

[0140] In one embodiment, before obtaining the vulnerability code features generated based on the vulnerability code library, which are implemented when the computer program is executed by a processor, the following steps are further included: obtaining each standard code; identifying keywords in the logs corresponding to the standard code, and obtaining the corresponding vulnerability description information and security patch information based on the identified keywords; obtaining vulnerability code from the original code corresponding to the standard code based on the vulnerability description information and the security patch information; and generating a vulnerability code library based on the obtained vulnerability code.

[0141] In one embodiment, a computer program product is provided, including a computer program which, when executed by a processor, implements the following steps: obtaining a target program; obtaining the vulnerability code features generated based on the vulnerability code library, and processing the target program to obtain target code features, where the target code features are generated using the same feature generation method as the vulnerability code features; calculating the similarity between the target code features and each vulnerability code feature; and determining that the target program has a vulnerability when the similarity is greater than a threshold.

[0142] In one embodiment, obtaining the vulnerability code features generated based on the vulnerability code library and processing the target program to obtain target code features, which are implemented when the computer program is executed by a processor, includes: parsing the vulnerability code in the vulnerability code library to generate an initial abstract syntax tree corresponding to the vulnerability code; abstracting the initial abstract syntax tree corresponding to the vulnerability code to obtain a target abstract syntax tree corresponding to the vulnerability code, and extracting features from the target abstract syntax tree corresponding to the vulnerability code to obtain vulnerability code features; parsing the target program to generate an initial abstract syntax tree corresponding to the target program; abstracting the initial abstract syntax tree corresponding to the target program to obtain a target abstract syntax tree corresponding to the target program, and extracting features from the target abstract syntax tree corresponding to the target program to obtain target code features.

[0143] In one embodiment, the abstraction of the initial abstract syntax tree, which is implemented when the computer program is executed by a processor, includes: replacing the formal parameter variables in the function declarations in the initial abstract syntax tree with a first symbol; replacing the local variables in the initial abstract syntax tree with a second symbol; replacing the data types in the initial abstract syntax tree with a third symbol; and replacing the function calls in the initial abstract syntax tree with a fourth symbol; where the first symbol, the second symbol, the third symbol, and the fourth symbol are different symbols.

[0144] In one embodiment, the feature extraction of the target abstract syntax tree, which is implemented when the computer program is executed by a processor, includes: traversing the target abstract syntax tree to obtain the number of nodes in the target abstract syntax tree; merging each node to obtain a marked string, and converting the marked string into a hash value; concatenating the number of nodes and the hash value to obtain a binary tuple, and using each binary tuple as a code feature.

[0145] In one embodiment, the similarity between the computing target code features and each vulnerability code feature implemented when a computer program is executed by a processor includes: obtaining a binary tuple corresponding to the target program; matching the number of nodes of the binary tuple corresponding to the target program with the number of nodes of each binary tuple corresponding to the vulnerability code library; if there is a binary tuple with the same number of nodes in the vulnerability code library, then matching the hash value of the binary tuple corresponding to the target program with the hash value of the binary tuple with the same number of nodes; if there is a binary tuple with the same hash value in the vulnerability code library, it is determined that the target program has a vulnerability; if there is no binary tuple with the same hash value in the vulnerability code library, or there is no binary tuple with the same number of nodes in the vulnerability code library, it is determined that the target program has no vulnerability.

[0146] In one embodiment, before obtaining the vulnerability code features generated based on the vulnerability code library when a computer program is executed by a processor, it further includes: obtaining each standard code; performing keyword recognition on the log corresponding to the standard code, and obtaining the corresponding vulnerability description information and security patch information based on the recognized keywords; obtaining the vulnerability code from the original code corresponding to the standard code based on the vulnerability description information and the security patch information; generating a vulnerability code library based on the obtained vulnerability code.

[0147] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0148] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0149] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope recorded in the present application.

[0150] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A code vulnerability detection method, characterized in that: The method comprises: Get the target program; Acquire vulnerability code features generated based on a vulnerability code library, and process the target program to obtain target code features, wherein the target code features are generated using the same feature generation method as the vulnerability code features; Calculating the similarity between the target code feature and each vulnerability code feature; When the similarity is greater than a threshold, it is determined that a vulnerability exists in the target program.

2. The method according to claim 1, characterized in that The acquiring of vulnerability code features generated based on the vulnerability code library and processing the target program to obtain target code features includes: Parsing the vulnerability code in the vulnerability code library to generate an initial abstract syntax tree corresponding to the vulnerability code; Abstracting an initial abstract syntax tree corresponding to the vulnerability code to obtain a target abstract syntax tree corresponding to the vulnerability code, and extracting features from the target abstract syntax tree corresponding to the vulnerability code to obtain features of the vulnerability code; Parsing the target program to generate an initial abstract syntax tree corresponding to the target program; An initial abstract syntax tree corresponding to the target program is abstracted to obtain a target abstract syntax tree corresponding to the target program, and feature extraction is performed on the target abstract syntax tree corresponding to the target program to obtain the target code feature.

3. The method according to claim 2, characterized in that The abstraction of the initial abstract syntax tree includes: Replacing the formal parameter variable in the function declaration in the initial abstract syntax tree with the first symbol; Replacing the local variables in the initial abstract syntax tree with the second symbol; Replacing the data type in the initial abstract syntax tree with a third symbol; The function call in the initial abstract syntax tree is replaced with a fourth symbol; wherein the first symbol, the second symbol, the third symbol and the fourth symbol are different symbols.

4. The method according to claim 2, characterized in that: The feature extraction of the target abstract syntax tree includes: Traversing the target abstract syntax tree to obtain the number of nodes in the target abstract syntax tree; Merging the nodes to obtain a tag string, and converting the tag string into a hash value; The number of nodes and the hash value are concatenated to obtain a tuple, and each of the tuples is used as a code feature.

5. The method according to claim 4, characterized in that The calculating the similarity between the target code feature and each vulnerability code feature includes: Obtaining a binary group corresponding to the target program; Matching the number of nodes of the binary group corresponding to the target program with the number of nodes of each binary group corresponding to the vulnerability code library; If there are two-tuples with the same number of nodes in the vulnerability code library, the hash value of the two-tuple corresponding to the target program is matched with the hash value of the two-tuple with the same number of nodes; If there is a binary group with the same hash value in the vulnerability code library, it is determined that the target program has a vulnerability; If there are no tuples with the same hash value in the vulnerability code library, or there are no tuples with the same number of nodes in the vulnerability code library, it is determined that there is no vulnerability in the target program.

6. The method according to any one of claims 1 to 5, characterized in that: Before obtaining the vulnerability code features generated based on the vulnerability code library, the method further includes: Get each standard code; Perform keyword recognition on the log corresponding to the standard code, and obtain corresponding vulnerability description information and security patch information based on the recognized keywords; Based on the vulnerability description information and the security patch information, obtaining the vulnerability code from the original code corresponding to the standard code; A vulnerability code library is generated based on the acquired vulnerability codes.

7. A code vulnerability detection device, characterized in that: The device comprises: A target program acquisition module, used for acquiring a target program; A code feature generation module, used to obtain vulnerability code features generated based on a vulnerability code library, and process the target program to obtain target code features, wherein the target code features are generated using the same feature generation method as the vulnerability code features; A similarity calculation module, used to calculate the similarity between the target code feature and each vulnerability code feature; The vulnerability detection module is used to determine that the target program has a vulnerability when the similarity is greater than a threshold.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images