Vulnerability mining system and device based on source code similarity

Abstract

The invention relates to the technical field of software security, in particular to a vulnerability mining system and device based on source code similarity. The vulnerability mining system comprises a vulnerability feature library which comprises vulnerability code snippets and feature information associated with the vulnerability code snippets; the coarse granularity positioning module is used for matching the vulnerability code snippets in the vulnerability feature library with the to-be-analyzed source code aiming at the to-be-analyzed source code, determining suspicious code snippets and forming similar code pairs; the feature representation module is used for acquiring feature information of the suspicious code snippets; the fine granularity positioning module is used for obtaining the semantic similarity, the structural similarity and the subgraph matching degree of the similar code pair according to the feature information associated with the two code snippets in the similar code pair, and determining the comprehensive similarity of the similar code pair; and the judgment module is used for determining whether the suspicious code snippets in the similar code pairs have vulnerabilities or not according to the comprehensive similarity. The method has the beneficial effects that the processing efficiency can be improved while the detection precision can be ensured.

Description

Technical Field

[0001] The present invention relates to the field of software security technology, and in particular to a vulnerability mining system and device based on source code similarity. Background Art

[0002] With the rapid development of software development and the flourishing of open source communities, the spread of software vulnerabilities has accelerated. Especially in the field of Internet of Things, the frequency of attacks on connected devices has increased exponentially, and the severity of vulnerabilities has become increasingly prominent. The performance of traditional vulnerability detection technology is gradually unable to meet industry needs.

[0003] Specifically, traditional vulnerability detection technology has the following shortcomings: 1. Code obfuscation and diversity issues: Code obfuscation techniques, such as variable name replacement and instruction reordering, are widely used in modern software development. This leads to significant differences in code syntax and structure, making it difficult for traditional similarity measurement methods to accurately identify similar code, resulting in low precision.

[0004] 2. Difficulty in cross-language vulnerability detection: Different programming languages ​​have fundamental differences in syntax and execution mechanisms, resulting in the same type of vulnerability manifesting differently in different programming languages. This makes cross-language vulnerability detection a major challenge and results in a low detection rate.

[0005] 3. Inefficient vulnerability mining: Existing technologies typically calculate the distance between the feature vector of each code segment in the code base and all feature vectors in the vulnerability signature database for precise search. This calculation method requires a large amount of computing resources when analyzing large code bases, resulting in low efficiency. For example, the HiddenCPG system took 16 days and 12 hours to process 739,000 code pairs, resulting in a slow processing speed.

[0006] Based on the above-mentioned deficiencies of traditional vulnerability detection technology, the existing technology needs to be improved in terms of detection accuracy and processing speed when performing vulnerability mining. Summary of the Invention

[0007] (1) Technical issues to be resolved In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a vulnerability mining system and device based on source code similarity, which solves the technical problems of the prior art in vulnerability mining, such as slow processing speed and low detection accuracy.

[0008] (2) Technical solution In order to achieve the above objectives, the main technical solutions adopted by the present invention include: In a first aspect, an embodiment of the present invention provides a vulnerability mining system based on source code similarity, comprising: A vulnerability feature library, including multiple vulnerability code snippets and feature information associated with each vulnerability code snippet; the feature information includes: token sequences and code attribute graphs of the vulnerability code snippets; The coarse-grained positioning module is used to match the source code to be analyzed with the vulnerability signature library using an approximate nearest neighbor search algorithm. It identifies suspicious code fragments in the source code that are similar to the vulnerable code fragments and forms similar code pairs. A feature characterization module, configured to obtain feature information of the suspicious code fragment; A fine-grained positioning module is used to obtain semantic similarity based on the feature information associated with two code fragments in a pair of similar code pairs using a twin neural network, and to determine structural similarity and subgraph matching based on the code attribute graph in the feature information; and to determine the comprehensive similarity of the similar code pairs based on a weighted summation of the semantic similarity, structural similarity, and subgraph matching; The shared encoder included in the twin neural network is provided with a multi-head attention fusion module for fusing the feature information; The determination module is used to determine whether the suspicious code segment has a vulnerability based on the comprehensive similarity.

[0009] Optionally, the fine-grained positioning module includes: A twin neural network unit is used to determine the semantic similarity of a similar code pair based on the feature information associated with two code fragments in the similar code pair based on the twin neural network; A graph structure unit is used to obtain the structural similarity and subgraph matching degree of a similar code pair based on a code attribute graph associated with two code snippets in the similar code pair based on a graph structure matching algorithm; The synthesis unit is used to perform weighted summation based on the semantic similarity, structural similarity and subgraph matching as the comprehensive similarity.

[0010] Optionally, in the twin neural network unit, determining the semantic similarity of the similar code pair based on the twin neural network includes: The twin neural network is a twin neural network obtained through a pre-training process and having matching model parameters; The twin neural network includes: a similarity calculation layer and two branch neural networks with the same structure, and the output ends of the two branch neural networks are connected to the input end of the similarity calculation layer; the two branch neural networks each include a shared encoder that shares weight parameters with each other, and the input end of the convolution block contained in the shared encoder is provided with a multi-head attention fusion module; The multi-head attention fusion module is used to perform weighted fusion on the features corresponding to the token sequence and the code attribute graph in the feature information of the branch neural network to which it belongs, based on the attention weights, to obtain fused features; the attention weights are learned through the pre-training process; The convolution block is used to extract high-dimensional features based on the fused features as the output of the branch neural network in which it is located; The similarity calculation layer is used to determine the semantic similarity of the similar code pairs based on the high-dimensional features.

[0011] Optionally, the pre-training process includes a pre-training process and a fine-tuning training process performed sequentially; The pre-training process uses a pre-training data set to train the twin neural network; the pre-training data set includes: a first positive sample pair and a first negative sample pair; The first positive sample pair includes: two vulnerable code snippets having the same vulnerability pattern, or any two of multiple variant code snippets of the same vulnerability pattern; the two code snippets in the first positive sample pair are codes written in the same high-level language; The first negative sample pair includes: a vulnerable code snippet and a repair code snippet of the vulnerable code snippet, or a vulnerable code snippet and a normal code snippet that has similar functions to the vulnerable code snippet and has no vulnerabilities; the two code snippets in the first negative sample pair are codes written in the same high-level language.

[0012] Optionally, the fine-tuning training process uses a fine-tuning training data set for training; The fine-tuning training dataset includes: an LLVM IR code pair converted from a second positive sample pair and a second negative sample pair; The second positive sample pair includes: a vulnerability code pair having the same vulnerability pattern, or any two of multiple variant code snippets of the same vulnerability pattern; the two code snippets in the second positive sample pair are codes written in different high-level languages; The second negative sample pair includes: a vulnerable code snippet and a repair code snippet of the vulnerable code snippet, or a vulnerable code snippet and a normal code snippet with similar functions to the vulnerable code snippet and without vulnerabilities; the two code snippets in the second negative sample pair are codes written in different high-level languages; The learning rate, number of training rounds and data size of the training set in the fine-tuning training process are all 10% to 30% of those in the pre-training process.

[0013] Optionally, the vulnerability mining system further includes: The preprocessing module is used to determine whether the programming languages ​​of the similar code pairs are the same before the feature characterization module obtains the feature information of the suspicious code fragments in the similar code pairs. If so, the two code fragments in the similar code pairs are directly input into the feature characterization module respectively to extract the feature information associated with the two code fragments respectively; if not, The two code snippets in the similar code pair are respectively converted into associated LLVM IR codes, and the LLVM IR codes are respectively input into a feature characterization module to respectively extract feature information associated with the two code snippets.

[0014] Optionally, the code property graph includes: an abstract syntax tree of the vulnerable code snippet, and a control flow graph and a data flow graph embedded in the abstract syntax tree; The graph structure unit includes: The structural analysis subunit is used to compare the topological structures of the abstract syntax trees associated with two code snippets in a similar code pair based on the code property graph and the graph structure matching algorithm to determine the AST similarity; compare the control flow paths of the control flow graphs associated with two code snippets in a similar code pair to determine the CFG similarity; and compare the data dependencies of the data flow graphs associated with two code snippets in a similar code pair to determine the DFG similarity; The structural feature fusion unit is used to perform weighted summation on the AST similarity, CFG similarity and DFG similarity to determine the structural similarity of the similar code pair.

[0015] Optionally, the graph structure unit further includes: The subgraph matching subunit uses the VF2 graph isomorphism algorithm to analyze the code attribute graph associated with the two code snippets in the similar code pair, and uses the similarity score obtained by the analysis as the subgraph matching degree.

[0016] Optionally, the vulnerability signature library further includes: attribute information associated with each vulnerability code snippet; the attribute information includes: the CVE number, CWE type, and CVSS score of the vulnerability code snippet; The vulnerability mining system also includes: The output module is used to determine the location information of the suspicious code fragment when the judgment module determines that there is a vulnerability in a suspicious code fragment in a pair of similar code pairs, and output the attribute information and location information associated with the vulnerable code fragment in the similar code pair as the mining result, so as to show the user the location and attributes of the vulnerability in the suspicious code fragment.

[0017] In a second aspect, an embodiment of the present invention provides a computer device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the vulnerability mining system based on source code similarity described in the first aspect is implemented.

[0018] (3) Beneficial effects The beneficial effects of the present invention are as follows: the vulnerability mining system of the present invention, based on the vulnerability feature library, first determines suspicious code fragments similar to the vulnerability code fragments in the source code to be analyzed through a coarse-grained positioning module to form similar code pairs; then obtains feature information of the suspicious code fragments through a feature characterization module; then determines the comprehensive similarity of the similar code pairs based on the feature information through a fine-grained positioning module; finally, determines whether the suspicious code fragments in the similar code pairs have vulnerabilities based on the comprehensive similarity through a judgment module.

[0019] Based on the above process, the vulnerability mining system provided by the present invention adopts a hierarchical strategy in the process of vulnerability detection, gradually narrowing the scope from coarse granularity to fine granularity, and in the coarse-grained positioning module, quickly matches the source code to be analyzed through the approximate nearest neighbor search algorithm to search for suspicious code fragments, and in the fine-grained positioning module, performs a detailed multi-dimensional similarity analysis on similar code pairs. Compared with the existing technology of directly performing an accurate search on the full text of the source code to be analyzed, it can effectively reduce the overall computational complexity of the vulnerability mining system, ensure detection accuracy and improve processing efficiency.

[0020] In addition, the fine-grained positioning module of the present invention uses a twin neural network to analyze the semantic similarity of similar code pairs, and the shared encoder contained in the twin neural network is provided with a multi-head attention fusion module for fusing the feature information. Compared with the existing twin neural network, which only performs a simple splicing processing method on the feature information, the present invention uses a multi-head attention fusion module to intelligently allocate the attention weights of each feature, extract features with richer perspectives from the feature information, highlight the important features in the feature information, and suppress the noise in the feature information, so that the semantic similarity obtained by the twin neural network is more accurate, further ensuring the detection accuracy of the vulnerability of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of the architecture of a vulnerability mining system based on source code similarity provided by an embodiment; Figure 2 A schematic diagram of the architecture of another vulnerability mining system based on source code similarity provided by an embodiment; Figure 3 A schematic diagram of the architecture of a twin neural network provided in an embodiment; Figure 4 A schematic diagram of the architecture of the pre-processing module in the twin neural network provided in the embodiment; Figure 5 Schematic diagram of the architecture of a shared encoder in a twin neural network provided in an embodiment. DETAILED DESCRIPTION

[0022] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0023] Example 1: Figure 1 As shown, this embodiment provides a vulnerability mining system based on source code similarity, including: a vulnerability feature library, a coarse-grained positioning module, a feature characterization module, a fine-grained positioning module, a determination module and an output module, as follows: A vulnerability signature library includes multiple known vulnerable code snippets and signature information associated with each vulnerable code snippet. This signature information includes the token sequence and code attribute graph of the vulnerable code snippet. The code attribute graph includes the abstract syntax tree of the vulnerable code snippet, as well as the control flow graph and data flow graph embedded in the abstract syntax tree. Furthermore, the vulnerability signature library may also include attribute information associated with each vulnerable code snippet; this attribute information includes the vulnerable code snippet's CVE number, CWE type, and CVSS score. CVE (Common Vulnerabilities and Exposures) refers to an internationally standardized vulnerability numbering system that assigns a unique identifier to each publicly known security vulnerability. CWE (Common Weakness Enumeration) is a classification standard for software security defects, such as CWE-119, which represents a buffer overflow type. CVSS (Common Vulnerability Scoring System) is a standardized scoring method for assessing vulnerability severity, with a score range of 0-10.

[0024] Specifically, the aforementioned known vulnerable code snippets are collected from code hosting platforms such as the National Vulnerability Database (NVD) and GitHub. Vulnerability code snippets and their corresponding patches are also collected from public datasets such as the Software Assurance Reference Dataset (SARD) and the Large-Scale Vulnerability Dataset (MegaVul), ensuring that the vulnerability signature library covers a variety of programming languages ​​and vulnerability types. The collected vulnerable code snippets are standardized, removing code comments and blank lines, and standardizing code formatting and indentation structure. Feature information and attribute information associated with the vulnerable code snippets are generated and stored together in the vulnerability signature library.

[0025] The coarse-grained positioning module is used to match vulnerability code snippets in the vulnerability signature library with the source code to be analyzed using an approximate nearest neighbor search algorithm. This algorithm then identifies suspicious code snippets in the source code to be analyzed that are similar to the vulnerability code snippets, forming similar code pairs. The approximate nearest neighbor search algorithm can quickly find the vector most similar to the query vector in a high-dimensional vector space. Compared to existing vulnerability detection technologies that perform precise searches by calculating the distance between the query vector and all vectors in the vulnerability signature library, the approximate nearest neighbor search algorithm sacrifices a slight degree of accuracy in exchange for a significant improvement in processing efficiency. This algorithm can quickly screen out potential vulnerability candidates, or suspicious code snippets, from large amounts of source code to be analyzed, providing a foundation for subsequent analysis.

[0026] A feature characterization module is used to obtain feature information of suspicious code snippets in the similar code pairs. Specifically, the feature information of the suspicious code snippets is the token sequence and code attribute graph associated with the suspicious code snippets. The token sequence can be extracted using a pre-trained CodeBERT model, and the code attribute graph can be generated using a parsing tool based on the programming language.

[0027] The fine-grained positioning module is used to obtain the semantic similarity of a similar code pair based on the feature information associated with the two code fragments in a pair of similar code pairs, based on the twin neural network, and to determine the structural similarity and subgraph matching of the similar code pair based on the code attribute graph in the feature information; and to determine the comprehensive similarity of the similar code pair based on the weighted sum of the semantic similarity, structural similarity and subgraph matching; wherein, the shared encoder contained in the twin neural network is provided with a multi-head attention fusion module for fusing the feature information. The fine-grained positioning module can accurately analyze the similar code pairs screened out by the coarse-grained positioning module and determine the similarity of the similar code pairs, so that the judgment module can determine whether the suspicious code fragments in the similar code pairs have vulnerabilities. The fine-grained positioning module can perform multi-dimensional similarity analysis on similar code pairs, which makes up for the problem of insufficient accuracy of the coarse-grained positioning module and ensures the overall vulnerability detection accuracy of the vulnerability mining system. The fine-grained positioning module only performs similarity analysis on similar code pairs matched by the coarse-grained positioning module. Compared with the existing method of performing the same degree of precise analysis on the source code segment by segment, it also effectively saves computing power.

[0028] Moreover, the shared encoder contained in the twin neural network used by the above-mentioned fine-grained positioning module is provided with a multi-head attention fusion module for fusing the feature information. Compared with the existing twin neural network processing method of only performing simple splicing on the feature information, the present invention uses the multi-head attention fusion module to intelligently allocate the attention weights of each feature, highlight important feature information, suppress the noise influence in the feature information, and make the semantic similarity output by the twin neural network more accurate, further ensuring the vulnerability mining system's vulnerability detection accuracy.

[0029] A determination module is configured to determine, based on the comprehensive similarity, whether the suspicious code fragment in the similar code pair contains a vulnerability. Specifically, a determination threshold may be set. If the comprehensive similarity of the similar code pair exceeds the determination threshold, the suspicious code fragment in the similar code pair is determined to contain a vulnerability. Specifically, the comprehensive similarity may be normalized to a range of [0, 1], and the determination threshold may be set to 0.85.

[0030] The output module is configured to, when the determination module determines that a suspicious code segment in a similar code pair contains a vulnerability, determine the location information of the suspicious code segment and output the attribute information and location information associated with the vulnerable code segment in the similar code pair as a mining result, thereby displaying the location and attributes of the vulnerability in the suspicious code segment to the user. Specifically, the location information includes the line number and file path of the suspicious code segment, enabling developers to quickly locate the exact location of the vulnerable code segment and specific vulnerability information.

[0031] The vulnerability mining system of the present invention is based on a vulnerability feature library. First, a coarse-grained positioning module is used to determine suspicious code fragments similar to the vulnerability code fragments in the source code to be analyzed to form similar code pairs. Then, a feature characterization module is used to obtain feature information of the suspicious code fragments. Then, a fine-grained positioning module is used to determine the comprehensive similarity of the similar code pairs based on the feature information. Finally, a judgment module is used to determine whether the suspicious code fragment in the similar code pair has a vulnerability based on the comprehensive similarity.

[0032] Based on the above process, the vulnerability mining system provided by the present invention adopts a hierarchical strategy in the process of vulnerability detection, gradually narrowing the scope from coarse granularity to fine granularity, and in the coarse-grained positioning module, quickly matches the source code to be analyzed through the approximate nearest neighbor search algorithm to search for suspicious code fragments, and in the fine-grained positioning module, performs a detailed multi-dimensional similarity analysis on similar code pairs. Compared with the existing technology of directly performing an accurate search on the full text of the source code to be analyzed, it can effectively reduce the overall computational complexity of the vulnerability mining system, ensure detection accuracy and improve processing efficiency.

[0033] Example 2: In order to further understand the vulnerability mining system provided by Example 1, this example is described in detail in conjunction with the specific structure of each module in the vulnerability mining system.

[0034] like Figure 2 As shown, this embodiment provides a vulnerability mining system based on source code similarity, including: a vulnerability feature library, a coarse-grained positioning module, a feature characterization module, a fine-grained positioning module, a determination module and an output module, as follows: The vulnerability feature library may be the vulnerability feature library provided in the first embodiment.

[0035] The coarse-grained positioning module is used to match the vulnerability code snippets in the vulnerability feature library with the source code to be analyzed through an approximate nearest neighbor search algorithm, determine the suspicious code snippets in the source code to be analyzed that are similar to the vulnerability code snippets, and form similar code pairs.

[0036] The approximate nearest neighbor search algorithm can specifically be the HNSW (Hierarchical Navigable SmallWorld) algorithm, the SCANN (Scalable Compression-based Accelerated Nearest Neighbors) algorithm, or after dividing the source code to be analyzed into multiple code fragments, using a clustering algorithm to perform cluster analysis on all code fragments to obtain multiple clusters, screening out clusters in all clusters that are similar to the vulnerability code fragments in the vulnerability feature library, and treating the code fragments in the screened clusters as suspicious code fragments, and forming similar code pairs with the vulnerability code fragments corresponding to the screened clusters.

[0037] More preferably, this embodiment uses a locality sensitive hashing (LSH) algorithm to match vulnerability code snippets in the vulnerability signature library with the source code to be analyzed, identifying suspicious code snippets in the source code to be analyzed that are similar to the vulnerability code snippets and forming similar code pairs. The LSH algorithm uses a hash function to map similar vectors to the same hash bucket, enabling fast retrieval. It includes steps P1 and P2 before starting vulnerability mining, and step P3 when performing vulnerability mining on the source code to be analyzed, as follows: P1. Pre-build a hash table, specifically including sub-steps P101 to P103: P101. Generate 128 random projection vectors r1, r2, ..., r128; each random projection vector is 768-dimensional and follows N(0, 1) distribution; P102. Define the hash function family: h_k(v)=sign(r_k·v); Where r_k represents the kth random projection vector, k is a positive integer less than or equal to 128; v represents the eigenvector of the input hash function, h_k(v) represents the eigenvector of the hash function output, and sign() represents the sign function.

[0038] The hash function above performs a dot product on the input feature vector v and the random projection vector r_k, then takes the sign. If the dot product is positive, the output is +1; if it is negative, the output is -1. Specifically, if the dot product is 0, the sign() function is defined to output +1. This converts the input continuous feature vector into a simple binary code, which is then used as the output feature vector.

[0039] P103. Create 16 hash tables, each containing 8 hash functions.

[0040] Specifically, the 128 random projection vectors in P101 are divided into 16 vector groups, with 8 random projection vectors in each vector group.

[0041] The eight hash functions contained in each hash table are obtained by taking the eight corresponding random projection vectors in a set of vector groups through the random projection variable r_k in the hash function family in P102.

[0042] P2. Pre-establish an index for the vulnerability signature database, specifically including sub-steps P201 to P203: P201. Convert each vulnerable code snippet in the vulnerability signature library into a 768-dimensional vulnerability feature vector and perform L2 normalization. Specifically, the vulnerable code snippet can be input into a pre-trained CodeBERT model, and the token sequence output by the CodeBERT model can be used as the vulnerability feature vector. The specific architecture of the CodeBERT model can be the same as the CodeBERT model used in the feature representation module below.

[0043] P202. For each of the 16 hash tables established in P103, apply the 768-dimensional vulnerability feature vector converted from the vulnerable code snippet in P201 to the eight hash functions in the hash table to obtain the corresponding hash vector: H(v) = [h1(v), h2(v), ..., h8(v)], where v represents the vulnerability feature vector, and h1(v), h2(v), ..., h8(v) represent the eight hash values ​​obtained by applying the vulnerability feature vector to the eight hash functions. The hash vector H(v) actually forms an 8-bit binary code, such as [+1, -1, +1, +1, -1, -1, +1, -1], which is the "fingerprint" of the vulnerable code snippet in this hash table. The 16 hash vectors corresponding to the vulnerability feature vector in the 16 hash tables constitute the hash matrix of the vulnerability feature vector.

[0044] P203. Map the vulnerable code snippets with the same hash matrix to the same hash bucket.

[0045] P3. When performing vulnerability mining on the source code to be analyzed, the source code to be analyzed is divided into multiple code segments to be checked according to the functions implemented or other segmentation criteria. For each code segment to be checked, sub-steps P301 to P303 are executed as follows: P301. Convert the code snippet to a 768-dimensional feature vector to be checked and perform L2 normalization, which is recorded as the feature vector to be checked q.

[0046] P302. Calculate the hash vector of the feature vector q to be checked in each hash table to obtain a hash matrix of the feature vector q to be checked consisting of 16 hash vectors. Based on the hash matrix of the feature vector q to be checked, map the code snippet to be checked to a hash bucket with the same feature matrix.

[0047] P303. Take all vulnerable code snippets in the hash bucket where the code snippet to be checked is located as candidate objects, and calculate the similarity between the code snippet to be checked and each candidate object.

[0048] If the similarity between the code snippet to be checked and each candidate object is less than the preset threshold, the code snippet to be checked is determined to have no vulnerability; if the similarity between one or more candidate objects and the code snippet to be checked is greater than the preset threshold, the code snippet to be checked is regarded as a suspicious code snippet, and the candidate object with the highest similarity is regarded as the vulnerable code snippet corresponding to the suspicious code snippet to form a similar code pair.

[0049] Specifically, the similarity between the code snippet to be checked and the candidate object can be the cosine similarity (to determine whether the directions are consistent) or the Euclidean similarity (to determine the distance between the two). Preferably, the similarity between the code snippet to be checked and the candidate object is the weighted average of the cosine similarity or the Euclidean similarity, with the weight for cosine similarity being 0.7 and the weight for Euclidean similarity being 0.3. The preset threshold can be 0.8.

[0050] The feature characterization module is used to obtain feature information of suspicious code fragments in the similar code pairs, and specifically includes a token sequence extraction unit and a code attribute graph extraction unit. When a vulnerability feature library is pre-built, the token sequence and code attribute graph of each vulnerable code fragment in the vulnerability feature library can also be generated using the feature characterization module.

[0051] The token sequence extraction unit is used to extract the token sequence of the suspicious code snippet. The token sequence is an ordered sequence of lexical units (including keywords, identifiers, and operators) that deconstructs the code snippet and represents the structure and semantics of the code snippet.

[0052] Specifically, the token sequence extraction unit includes a pre-trained CodeBERT model. The CodeBERT model is a bidirectional encoder model based on the Transformer architecture specifically designed for code comprehension tasks. It has a parameter size of 125M and consists of 12 layers of Transformer encoders, each with 768 hidden states and 12 attention heads. It is capable of understanding the semantic and syntactic structure of code. The CodeBERT model can be pre-trained on a large-scale code corpus to learn universal representations for code written in multiple high-level languages ​​(such as Java, C, and Python).

[0053] The code property graph extraction unit is used to generate a code property graph of the suspicious code fragment. The code property graph includes: an abstract syntax tree of the suspicious code fragment, and a control flow graph and a data flow graph embedded in the abstract syntax tree.

[0054] An Abstract Syntax Tree (AST) is a tree-like structure that represents the syntax hierarchy of code. It is independent of specific grammars and language details and can represent the grammatical features of code snippets.

[0055] A control flow graph (CFG) is a directed graph representation of code execution paths. It uses a graph to depict the possible execution paths of all basic blocks within a procedure. The execution path is determined by conditional statements, such as if, for, and switch statements. Nodes in a control flow graph represent statements or conditions, and directed edges connect nodes, representing the transfer of control. A control flow graph can reveal the control dependency characteristics of a code snippet.

[0056] A data flow graph (DFG) is a graphical structure used to describe the definitions and usage relationships of variables in code. A data flow graph tracks the usage of variables in a control flow graph. Edges in a data flow graph represent subsequent accesses or modifications to the same variable. A data flow graph can reveal data dependencies within a code snippet.

[0057] The Code Property Graph (CPG) is a unified graph structure that integrates the abstract syntax tree, control flow graph, and data flow graph.

[0058] The aforementioned abstract syntax tree, control flow graph, data flow graph, and code property graph that integrates the abstract syntax tree, control flow graph, and data flow graph can be generated using a Joern parser. The code property graph can use the AST as its basic structure, adding CFG edges between AST nodes to represent control flow relationships, and adding DFG edges between AST nodes to represent data flow relationships.

[0059] The above feature information can characterize suspicious code fragments or vulnerable code fragments from multiple dimensions, so that the fine-grained positioning module can perform detailed similarity analysis from multiple dimensions.

[0060] The fine-grained positioning module includes: a twin neural network unit, a graph structure unit and a synthesis unit.

[0061] The twin neural network unit is used to determine the semantic similarity of a similar code pair based on the twin neural network according to the feature information associated with the two code fragments in the similar code pair.

[0062] The graph structure unit is used to obtain the structural similarity and subgraph matching degree of a similar code pair based on a code attribute graph associated with two code snippets in the similar code pair and a graph structure matching algorithm.

[0063] The synthesis unit is configured to perform a weighted sum based on the semantic similarity, structural similarity and subgraph matching to obtain the comprehensive similarity. Preferably, the weights of the semantic similarity, structural similarity and subgraph matching are 0.4, 0.3 and 0.3 respectively.

[0064] The above-mentioned integrated unit adopts a weighted fusion method and uses multiple experiments to verify the weights, which can maintain a good balance in different types of vulnerability detection tasks.

[0065] Specifically, the graph structure unit includes: a structural feature analysis subunit, a structural feature fusion unit and a subgraph matching subunit.

[0066] The structural analysis subunit is used to compare the topological structures of the abstract syntax trees associated with two code snippets in a similar code pair based on the code property graph and according to the graph structure matching algorithm to determine the AST similarity; compare the control flow paths of the control flow graph associated with two code snippets in a similar code pair to determine the CFG similarity; and compare the data dependencies of the data flow graph associated with two code snippets in a similar code pair to determine the DFG similarity.

[0067] Graph algorithm matching is a computational method for identifying specific patterns within complex graph networks. This technique can identify code snippets within the code property graph (CPG) of suspicious code snippets that closely match vulnerability patterns found in known vulnerable code snippets. This approach is unique in that it not only considers surface features of the code but also delves deeper into the program's inherent structural relationships. Furthermore, an improved VF2 Subgraph Isomorphism Algorithm can be used to perform exact matching operations on this basis. This algorithm has been optimized to handle matching tasks on large graph structures. When exact matching fails to meet practical requirements, the Graph Edit Distance (GED) mechanism can be introduced to implement a more flexible approximate matching strategy.

[0068] Specifically, the steps for obtaining the above AST similarity, CFG similarity, and DFG similarity based on the graph structure matching algorithm are as follows: A1. Based on calculating the similarity of abstract syntax tree AST: The structural analysis subunit analyzes the abstract syntax trees of two code snippets based on a graph structure matching algorithm, calculating the degree of matching in terms of node types, hierarchical relationships, and branch structures. This analysis can capture similar patterns in the code at the syntactic level. The specific analysis dimensions are as follows: A101. Compare the node type distributions (function nodes, variable nodes, expression nodes, etc.) of the two ASTs to obtain the node type similarity.

[0069] Node type similarity = |number of identical node types| / |total number of node types|.

[0070] A102. Analyze the similarity of the tree topology structures (depth, number of branches, hierarchical relationship) of the abstract syntax trees to obtain topological similarity.

[0071] Topological similarity = 1-graph edit distance (T1, T2) / max(|T1|, |T2|); T1 and T2 are the abstract syntax trees corresponding to the two code snippets in the similar code pair, |T1| represents the total number of nodes in T1, and |T2| represents the total number of nodes in T2.

[0072] A103. Evaluate the matching degree (deep matching degree) of the parent-child node relationship to obtain the deep similarity.

[0073] Depth similarity = 1-|Depth(T1)-Depth(T2)| / max(Depth(T1),Depth(T2)); Depth(T1) and Depth(T2) respectively represent the maximum depths of the abstract syntax trees corresponding to the two code snippets in the similar code pair, and max(Depth(T1), Depth(T2)) represents the maximum value of the maximum depths corresponding to the two abstract syntax trees.

[0074] A104: Perform weighted summation based on the node type similarity, topology similarity, and depth similarity to obtain AST similarity. Specifically, the weights corresponding to the node type similarity, topology similarity, and depth similarity are 0.4, 0.4, and 0.2, respectively.

[0075] A2. Calculate CFG similarity based on control flow graph: The structural analysis subunit analyzes the similarity of control flow paths between similar code pairs based on a graph structure matching algorithm. Specifically, the path matching algorithm within the graph structure matching algorithm can identify code fragments with identical control logic. Even if the code differs in surface form, as long as the control flow structure is similar, the algorithm can detect it. The specific analysis dimensions are as follows: A201. Analyze and compare the number and type distribution of basic blocks of two code snippets in a similar code pair to obtain basic block similarity.

[0076] Basic block similarity = 1-||G1.blocks|-|G2.blocks|| / max(|G1.blocks|, |G2.blocks|); G1.blocks and G2.blocks represent the number of basic blocks contained in the control flow graphs of the two code snippets in the similar code pair.

[0077] A202. Analyze the similarity of program execution paths (sequential execution, conditional branches, loop structures) to obtain execution path similarity.

[0078] Execution path similarity = |number of identical paths| / |total number of paths|.

[0079] A203. Compare the control flow transfer patterns to obtain the branch structure similarity.

[0080] Branch structure similarity = |number of identical branch structures| / |total number of branch structures|.

[0081] A204: Perform weighted summation based on the basic block similarity, execution path similarity, and branch structure similarity to obtain CFG similarity. Specifically, the weights corresponding to the basic block similarity, execution path similarity, and branch structure similarity are 0.3, 0.4, and 0.3, respectively.

[0082] A3. Calculate DFG similarity based on data flow graph: The structural analysis subunit compares the matching degree of data dependencies based on a graph structure matching algorithm. This analysis focuses on comparing data flow patterns and is particularly effective in identifying vulnerable code with similar data processing logic. The specific analysis dimensions are as follows: A301. Compare the number and distribution of data-dependent edges to obtain edge quantity similarity.

[0083] Edge number similarity = 1-||D1.edges|-|D2.edges|| / max(|D1.edges|,|D2.edges|).

[0084] D1.edges and D2.edges represent the number of edges contained in the data flow graphs of the two code snippets in the similar code pair respectively.

[0085] A302. Analyze similarity patterns of variable definition-usage relationships to obtain dependency similarity.

[0086] Dependency similarity = |number of identical dependencies| / |total number of dependencies|.

[0087] A303. Determine the similarity of data flow paths.

[0088] Data flow path similarity = |number of identical data flow paths| / |total number of data flow paths|.

[0089] A304: Perform a weighted summation based on the edge number similarity, dependency similarity, and data flow path similarity to obtain DFG similarity. Specifically, the weights corresponding to edge number similarity, dependency similarity, and data flow path similarity are 0.2, 0.5, and 0.3, respectively.

[0090] The structural feature fusion unit is used to perform weighted summation on the AST similarity, CFG similarity and DFG similarity to determine the structural similarity of the similar code pairs.

[0091] Based on the structural feature analysis sub-unit and the structural feature fusion unit, the graph structure unit provided in this embodiment actually analyzes the code attribute graph from two levels of dimensions. The structural feature analysis sub-unit starts from the detailed features of the bottom layer of the code attribute graph, and analyzes the specific features of the AST, CFG and DFG of the code attribute graph respectively, obtains the corresponding similarities and performs weighted summation to obtain the upper-layer AST similarity, CFG similarity and DFG similarity; the structural feature fusion unit then performs weighted summation based on the upper-layer AST similarity, CFG similarity and DFG similarity to obtain the structural similarity of similar code pairs. These two levels of similarity analysis methods can more comprehensively evaluate the structural similarity of the code attribute graphs of similar code pairs.

[0092] The subgraph matching subunit analyzes the code attribute graph associated with the two code snippets in the similar code pair using the VF2 graph isomorphism algorithm, and uses the similarity score obtained from the analysis as the subgraph matching degree.

[0093] The structural similarity output by the above-mentioned structural feature fusion unit, the subgraph matching degree output by the structural feature fusion sub-unit, and the semantic similarity output by the twin neural network unit are all input into the comprehensive unit for weighted summation to obtain the comprehensive similarity.

[0094] To further improve the processing speed of the vulnerability mining system, you can also use Spark or ForkJoin distributed computing frameworks to distribute analysis tasks to multiple computing nodes, enabling parallel processing and increasing processing speed. Spark is an open-source Apache big data processing engine that supports in-memory computing. ForkJoin is a Java concurrent programming framework that uses a divide-and-conquer strategy to handle parallel tasks.

[0095] Example 3: In order to further understand the vulnerability mining system provided by Example 1 or 2, this example further explains the specific structure of the twin neural network contained in the twin neural network unit and its pre-training process.

[0096] Specifically, this embodiment provides a vulnerability mining system based on source code similarity, including a fine-grained positioning module, which includes a twin neural network unit.

[0097] The twin neural network unit is used to determine the semantic similarity of a similar code pair based on the twin neural network according to the feature information associated with the similar code pair.

[0098] Preferably, the twin neural network unit includes: a twin neural network with matching model parameters obtained through a pre-training process.

[0099] The twin neural network includes: a similarity calculation layer and two branch neural networks with the same structure, and the output ends of the two branch neural networks are connected to the input end of the similarity calculation layer; the two branch neural networks each include a shared encoder that shares weight parameters with each other, and the input end of the convolution block contained in the shared encoder is provided with a multi-head attention fusion module.

[0100] The multi-head attention fusion module is used to perform weighted fusion on the features corresponding to the token sequence and the code attribute graph in the feature information of the branch neural network to which it belongs based on the attention weight to obtain a fused feature; the attention weight is learned through the pre-training process.

[0101] The convolution block is used to extract high-dimensional features based on the fused features as the output of the branch neural network in which it is located.

[0102] The similarity calculation layer is used to determine the semantic similarity of the similar code pairs based on the high-dimensional features.

[0103] When the existing twin neural network is used for text similarity recognition, its input features are usually token sequences, so the input features can be simply spliced ​​directly and then sent to the shared encoder to extract high-dimensional features. However, the feature information of this embodiment is multidimensional. In addition to the token sequence, it also includes a code attribute graph that integrates the abstract syntax tree, control flow graph and data flow graph. If it is directly spliced ​​according to the existing technology, it is equivalent to the importance of each input feature to the twin neural network being the same. However, the feature information in this embodiment has different vulnerability types for different vulnerability code snippets, and the way they are reflected in the feature information is different. For example, the root cause of the "C language buffer overflow" vulnerability is the lack of memory write length check, which is specifically reflected in the presence or absence of the "length check branch" in the control flow graph; the root cause of the "JavaScript code injection" vulnerability is the loss of control of user input execution permissions, that is, the lack of input purification, which is specifically reflected in the direct dependency relationship of "input-execution" in the data flow graph. Therefore, this embodiment is improved based on the existing twin neural network, and a multi-head attention fusion module is set at the input end of the convolution block of the shared encoder of each branch neural network. Based on the attention weights learned through the pre-training process, the feature information input into the branch is weighted and fused instead of treating all features equally, so that the obtained fused features can highlight important features and suppress noise in the feature information. The perspective of feature extraction is also richer, thereby making the semantic similarity obtained by the twin neural network based on the fused features more accurate.

[0104] More specifically, Figures 3 to 5 As shown, the twin neural network includes two branch neural networks and a similarity calculation layer. A vulnerable code snippet from a similar code pair is input as code snippet A into one of the two branch neural networks, while a suspicious code snippet from the similar code pair is input as code snippet B into the other branch neural network. High-dimensional features corresponding to the two code snippets are obtained, and these high-dimensional features are input together into the similarity calculation layer for calculation, resulting in the semantic similarity between the two code snippets in the similar code pair.

[0105] The branched neural network includes a pre-processing module, a shared encoder and a residual connection layer which are connected in sequence.

[0106] The pre-processing module includes a feature splicing layer and a linear projection layer.

[0107] The feature concatenation layer is used to simply concatenate the feature information associated with code snippets, namely token sequences (token feature vectors), abstract syntax trees (AST features), control flow graphs (CFG features), and data flow graphs (DFG features), and align multiple feature matrices of different sizes.

[0108] The linear projection layer is used to perform linear transformation on the high-dimensional feature matrix output by the feature concatenation layer, map it to the low-dimensional feature space of the twin neural network, obtain the low-dimensional feature matrix, and input it into the shared encoder.

[0109] The shared encoder includes a feature enhancement layer, a multi-head attention fusion module and a convolution block connected in sequence. The feature enhancement layer performs feature enhancement through the low-dimensional feature matrix input by the ReLU function, and then inputs the feature into the multi-head attention fusion module for feature fusion to obtain a fused feature. The convolution block extracts high-dimensional features from the fused features and does not output them. More specifically, the convolution block includes three convolution layers, two fully connected layers, a ReLU function activation layer and a dropout layer connected in sequence to extract high-dimensional features from the fused features. The dropout layer is used to reduce the risk of overfitting during training and improve the generalization ability of the twin neural network. It is only turned on during training and is closed when processing the source code to be analyzed.

[0110] The multi-head attention fusion module uses eight attention heads, each with 96 dimensions, to output a total of 768-dimensional fused features. During pre-training, the module automatically learns which features are most important for the current code snippet, thereby automatically assigning different attention weights to the 768-dimensional fused features of different code snippets.

[0111] Taking the detection of buffer vulnerabilities in code snippets as an example, the input features of the multi-head attention fusion module are: Token features: (representing keywords such as "strcpy" and "buffer"); AST features: characterize function call nodes and variable declaration nodes; CFG feature: characterizes the execution path from function entry to strcpy; DFG feature: represents the data flow from input parameters to buffer.

[0112] The above input features are simultaneously input to the 8 attention heads of the multi-head attention fusion module. Each attention head processes the input features based on its own learned attention weights as follows: Attention head 1: 70% focus on token features (dangerous function names); Attention head 2: 60% focuses on DFG features (data flow path); Attention head 3: 50% focus on AST features (syntactic structure); Attention head 4: 40% focuses on CFG features (control flow); Other attention heads: focus on the combination of two or more of the above four input features.

[0113] The various features extracted by the above eight attention heads are fused together to finally obtain a 768-dimensional vector, which includes the weighted information of all features and can fully represent the fused features of the code snippet.

[0114] The convolution block of the shared encoder can directly use the high-dimensional features extracted from the above fusion features as the output of the branch neural network in which it is located, or it can input the high-dimensional features extracted by the convolution block into the residual connection layer for further processing.

[0115] The residual connection layer performs a residual connection on the high-dimensional features and the low-dimensional feature matrix of the input shared encoder, and then performs layer normalization processing as the output of the branch neural network in which it is located to prevent gradient disappearance and gradient explosion problems.

[0116] The similarity calculation layer is used to determine the semantic similarity of the similar code pairs based on the high-dimensional features output by the branch neural network.

[0117] Specifically, the similarity calculation layer calculates the cosine similarity (cos_sim) and Euclidean distance (eucl_dist) of the high-dimensional features output by the two branch neural networks. Based on these cosine similarity and Euclidean distance, the semantic similarity of the two code segments in the similar code pair is determined. Specifically, the semantic similarity ranges from [0 to 1].

[0118] Specifically, for the above-mentioned twin neural network, the pre-training process includes a pre-training process and a fine-tuning training process performed in sequence.

[0119] The pre-training process trains the twin neural network using a pre-training dataset; the pre-training dataset includes a first positive sample pair and a first negative sample pair. The first positive sample pair and the first negative sample pair can be manually labeled based on vulnerability patterns. Vulnerability patterns refer to structured representations of vulnerabilities at the code level. They are code-level feature templates defined to identify specific vulnerabilities and include grammatical structure, data / control flow, judgment conditions, and other content.

[0120] The first positive sample pair includes: two vulnerability code snippets with the same vulnerability pattern, or any two of multiple variant code snippets of the same vulnerability pattern; the two code snippets in the first positive sample pair are codes written in the same high-level language.

[0121] That is to say, the two vulnerable code snippets in the first positive sample pair refer to two code snippets under the same or similar "vulnerability pattern". They can be matched in structural features or data / control flow, and are therefore judged to be similar.

[0122] The first negative sample pair includes: a vulnerable code snippet and a repair code snippet of the vulnerable code snippet, or a vulnerable code snippet and a normal code snippet that has similar functions to the vulnerable code snippet and has no vulnerabilities; the two code snippets in the first negative sample pair are codes written in the same high-level language.

[0123] The above pre-training process enables the twin neural network to learn the rich semantic information and contextual information contained in the code snippets written in high-level languages, ensuring the ability to recognize the similarity of code snippets written in the same high-level language.

[0124] The fine-tuning training process uses the fine-tuning training data set for training. The learning rate, number of training rounds, and data size of the training set in the fine-tuning training process are all 10% to 30% of those in the pre-training process.

[0125] The fine-tuning training dataset includes: LLVM IR code pairs converted from the second positive sample pair and the second negative sample pair.

[0126] The second positive sample pair includes: a vulnerability code pair having the same vulnerability pattern, or any two of multiple variant code snippets of the same vulnerability pattern; the two code snippets in the second positive sample pair are codes written in different high-level languages.

[0127] For example, the vulnerability code pair in the second positive sample may be a C language code snippet with a buffer overflow vulnerability and a Java code snippet with an array out-of-bounds access vulnerability; or, a Python code snippet with a command injection vulnerability and a JavaScript code snippet with a code injection vulnerability; or, two code snippets written in different high-level languages ​​with the same CVE vulnerability.

[0128] The second negative sample pair includes: a vulnerable code snippet and a repair code snippet of the vulnerable code snippet, or a vulnerable code snippet and a normal code snippet with the same function as the vulnerable code snippet and without vulnerabilities; the two code snippets in the second negative sample pair are codes written in different high-level languages.

[0129] For example, the second negative sample pair can be a C language code snippet and a Java code snippet that implement the same function, the C language code snippet has a vulnerability, and the Java code snippet does not; or, a Python code snippet with a vulnerability and a repair code snippet of the Python code snippet, the repair code snippet is implemented using JavaScript.

[0130] The second positive sample pair and the second negative sample pair can be converted into corresponding LLVM IR code pairs using various conversion tools under Tokenizer. Different language identifiers are added for different languages, such as [C], [Java], or [Python]. For the variables involved in the code snippet, static single assignment (SSA) is used for assignment. Each variable is assigned only once, eventually forming the fine-tuning training dataset.

[0131] For example, for a C language code snippet, you can use Clang to generate its LLVM IR code snippet; for a Java code snippet, you can use LLVM-Java to generate its LLVM IR code snippet; other languages ​​also have similar tools for generating unified LLVM IR code.

[0132] After the LLVM IR code is input into the feature representation module, the generated feature information differs from that of high-level language code. Specifically, in terms of token sequences, LLVM IR code is standardized, while high-level language code has a variety of expressions. In terms of abstract syntax trees, LLVM IR code is basically low-level instructions similar to assembly language, while high-level languages ​​contain a lot of high-level syntax. In terms of control flow graphs, corresponding to the abstract syntax trees, LLVM IR code's control is at the instruction level, while high-level languages ​​are at the statement level. In terms of data flow graphs, LLVM IR code's variable names are SSA values, while high-level languages ​​retain specific variable names.

[0133] Based on the difference in feature information corresponding to LLVM IR code and high-level language, the twin neural network, after the above pre-training and fine-tuning training, learns the different features of high-level language and LLVM IR code. The source code can provide more semantics, while the IR code can provide precision. Different vulnerability types also have different feature weight combinations. In this way, the representation of high-level language and LLVM IR code is intelligently integrated through the multi-head attention mechanism, playing a complementary role. It can not only improve the semantic richness of the source code, but also improve the accuracy of twin neural network recognition.

[0134] Based on the twin neural network obtained through the above-mentioned sequential pre-training process and fine-tuning training process, the vulnerability mining system provided in this embodiment also adaptively includes a preprocessing module.

[0135] The preprocessing module is used to determine whether the programming languages ​​of the similar code pairs are the same before the feature characterization module obtains the feature information of the suspicious code fragments in the similar code pairs. If so, the two code fragments in the similar code pairs are directly input into the feature characterization module respectively to extract the feature information associated with the two code fragments respectively; if not, The two code snippets in the similar code pair are respectively converted into associated LLVM IR codes, and the LLVM IR codes are respectively input into a feature characterization module to respectively extract feature information associated with the two code snippets.

[0136] That is, when processing the source code to be analyzed, for the similar code pairs output by the coarse-grained positioning module, they are first input into the preprocessing module to determine whether the programming languages ​​of the similar code pairs are the same. If they are the same, the two code segments are directly input into the feature representation module respectively. If they are different, the two code segments are converted into LLVM IR codes and then input into the feature representation module respectively to extract the feature information associated with the two code segments. The feature information associated with the two code segments in the similar code pair is then input into the Twin Bible network unit in the fine-grained positioning module to determine the semantic similarity of the similar code pair.

[0137] In actual applications, since the similar code pairs identified by the coarse-grained positioning module may be written in the same programming language or in different high-level languages, in order to enable the twin neural network to have cross-language semantic similarity recognition capabilities, this embodiment provides the above-mentioned training process including a pre-training process and a fine-tuning training process. After the pre-training process enables the twin neural network to have a better semantic similarity recognition capability for similar code pairs written in the same high-level language, the LLVM IR code pairs converted from similar code pairs formed by different high-level languages ​​are used to fine-tune the twin neural network so that it also has a better semantic similarity recognition capability for similar code pairs formed by two code fragments written in different high-level languages.

[0138] Existing vulnerability detection technologies, when addressing cross-language issues, typically tend to uniformly convert all code snippets written in high-level languages ​​into LLVM IR code, and then perform similarity analysis based on the LLVM IR code. However, generally speaking, source code written in high-level languages ​​carries rich semantic information, and converting it into LLVM IR code results in a certain degree of semantic loss, making the accuracy of similarity analysis based on LLVM IR code less than ideal. The twin neural network provided in this embodiment, obtained based on the above two-stage training process, can still extract rich semantic information when processing similar code pairs written in the same high-level language. When processing similar code pairs written in different high-level languages, similarity analysis is performed based on the converted LLVM IR code of the similar code pairs. This allows for cross-language semantic similarity recognition while retaining the ability to recognize semantic similarity for similar code pairs written in high-level languages. This makes the vulnerability mining system provided in this embodiment significantly superior to existing technologies in terms of precision, recall, and processing speed. It can effectively detect vulnerabilities after code obfuscation, supports multiple programming languages ​​such as C, Java, and Python, and is suitable for analyzing large-scale code bases, providing an efficient and accurate vulnerability mining solution for software security.

[0139] In addition, this embodiment provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the vulnerability mining system based on source code similarity described in one to three is implemented.

[0140] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0141] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0142] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0143] In the description of this specification, the terms "one embodiment", "some embodiments", "embodiments", "examples", "specific examples" or "some examples" refer to the specific features, structures, materials or characteristics described in conjunction with the embodiment or example and included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and features of different embodiments or examples, unless they are mutually inconsistent.

[0144] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A vulnerability mining system based on source code similarity, characterized in that: include: Vulnerability feature library, including multiple vulnerability code snippets and feature information associated with each vulnerability code snippet; The feature information includes: a token sequence and a code attribute graph of the vulnerable code snippet; The coarse-grained positioning module is used to match the source code to be analyzed with the vulnerability signature library using an approximate nearest neighbor search algorithm. It identifies suspicious code fragments in the source code that are similar to the vulnerable code fragments and forms similar code pairs. A feature characterization module, configured to obtain feature information of the suspicious code fragment; A fine-grained positioning module is used to obtain semantic similarity based on the feature information associated with two code fragments in a pair of similar code pairs using a twin neural network, and to determine structural similarity and subgraph matching based on the code attribute graph in the feature information; and to determine the comprehensive similarity of the similar code pairs based on a weighted summation of the semantic similarity, structural similarity, and subgraph matching; The shared encoder included in the twin neural network is provided with a multi-head attention fusion module for fusing the feature information; The determination module is used to determine whether the suspicious code segment has a vulnerability based on the comprehensive similarity.

2. The vulnerability mining system according to claim 1, characterized in that: The fine-grained positioning module includes: A twin neural network unit is used to determine the semantic similarity of a similar code pair based on the feature information associated with two code fragments in the similar code pair based on the twin neural network; A graph structure unit is used to obtain the structural similarity and subgraph matching degree of a similar code pair based on a code attribute graph associated with two code snippets in the similar code pair based on a graph structure matching algorithm; The synthesis unit is used to perform weighted summation based on the semantic similarity, structural similarity and subgraph matching as the comprehensive similarity.

3. The vulnerability mining system according to claim 2, characterized in that: In the twin neural network unit, determining the semantic similarity of the similar code pair based on the twin neural network includes: The twin neural network is a twin neural network obtained through a pre-training process and having matching model parameters; The twin neural network includes: a similarity calculation layer and two branch neural networks with the same structure, and the output ends of the two branch neural networks are connected to the input end of the similarity calculation layer; the two branch neural networks each include a shared encoder that shares weight parameters with each other, and the input end of the convolution block contained in the shared encoder is provided with a multi-head attention fusion module; The multi-head attention fusion module is used to perform weighted fusion on the features corresponding to the token sequence and the code attribute graph in the feature information of the branch neural network to which it belongs, based on the attention weights, to obtain fused features; the attention weights are learned through the pre-training process; The convolution block is used to extract high-dimensional features based on the fused features as the output of the branch neural network in which it is located; The similarity calculation layer is used to determine the semantic similarity of the similar code pairs based on the high-dimensional features.

4. The vulnerability mining system according to claim 3, characterized in that: The pre-training process includes a pre-training process and a fine-tuning training process performed in sequence; The pre-training process uses a pre-training dataset to train the twin neural network; The pre-training data set includes: a first positive sample pair and a first negative sample pair; The first positive sample pair includes: two vulnerable code snippets having the same vulnerability pattern, or any two of multiple variant code snippets of the same vulnerability pattern; the two code snippets in the first positive sample pair are codes written in the same high-level language; The first negative sample pair includes: a vulnerable code snippet and a repair code snippet of the vulnerable code snippet, or a vulnerable code snippet and a normal code snippet that has similar functions to the vulnerable code snippet and has no vulnerabilities; the two code snippets in the first negative sample pair are codes written in the same high-level language.

5. The vulnerability mining system according to claim 4, characterized in that: The fine-tuning training process uses a fine-tuning training data set for training; The fine-tuning training dataset includes: an LLVM IR code pair converted from a second positive sample pair and a second negative sample pair; The second positive sample pair includes: a vulnerability code pair having the same vulnerability pattern, or any two of multiple variant code snippets of the same vulnerability pattern; the two code snippets in the second positive sample pair are codes written in different high-level languages; The second negative sample pair includes: a vulnerable code snippet and a repair code snippet of the vulnerable code snippet, or a vulnerable code snippet and a normal code snippet with the same function as the vulnerable code snippet and without vulnerabilities; the two code snippets in the second negative sample pair are codes written in different high-level languages; The learning rate, number of training rounds and data size of the training set in the fine-tuning training process are all 10% to 30% of those in the pre-training process.

6. The vulnerability mining system according to claim 5, characterized in that: The vulnerability mining system also includes: The preprocessing module is used to determine whether the programming languages ​​of the similar code pairs are the same before the feature characterization module obtains the feature information of the suspicious code fragments in the similar code pairs. If so, the two code fragments in the similar code pairs are directly input into the feature characterization module respectively to extract the feature information associated with the two code fragments respectively; if not, The two code snippets in the similar code pair are respectively converted into associated LLVM IR codes, and the LLVM IR codes are respectively input into a feature characterization module to respectively extract feature information associated with the two code snippets.

7. The vulnerability mining system according to claim 2, characterized in that: The code property graph includes: an abstract syntax tree of the vulnerable code fragment or the suspicious code fragment, and a control flow graph and a data flow graph embedded in the abstract syntax tree; The graph structure unit includes: The structural feature analysis subunit is used to compare the topological structures of the abstract syntax trees associated with two code snippets in a similar code pair based on the code attribute graph and the graph structure matching algorithm to determine the AST similarity; compare the control flow paths of the control flow graph associated with two code snippets in a similar code pair to determine the CFG similarity; and compare the data dependency relationships of the data flow graph associated with two code snippets in a similar code pair to determine the DFG similarity. The structural feature fusion subunit is used to perform weighted summation on the AST similarity, CFG similarity and DFG similarity to determine the structural similarity of the similar code pairs.

8. The vulnerability mining system according to claim 2, characterized in that: The graph structure unit further includes: The subgraph matching subunit uses the VF2 graph isomorphism algorithm to analyze the code attribute graph associated with the two code snippets in the similar code pair, and uses the similarity score obtained by the analysis as the subgraph matching degree.

9. The vulnerability mining system according to claim 1, characterized in that: The vulnerability signature database also includes: attribute information associated with each vulnerability code snippet; the attribute information includes: the CVE number, CWE type and CVSS score of the vulnerability code snippet; The vulnerability mining system also includes: The output module is used to determine the location information of the suspicious code fragment when the judgment module determines that there is a vulnerability in a suspicious code fragment in a pair of similar code pairs, and output the attribute information and location information associated with the vulnerable code fragment in the similar code pair as the mining result, so as to show the user the location and attributes of the vulnerability in the suspicious code fragment.

10. A computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the vulnerability mining system based on source code similarity according to any one of claims 1 to 9.

Images