Source code vulnerability fine-grained identification method based on vulnerability patch pair

Abstract

The application discloses a source code vulnerability fine-grained identification method based on a vulnerability patch pair, and relates to the technical field of software analysis.The application aims at the problem of insufficient fine-grained identification capability of open source software project code vulnerability, is based on the intermediate representation structure of code multi-modal, amplifies the difference between vulnerable code and normal code by using vulnerability patch code, extracts comprehensive and fine feature information of code, and realizes fine-grained identification of vulnerability code; the application can not only effectively amplify the subtle difference between vulnerability code and normal code, but also fully extract comprehensive and fine syntax semantic feature information in the code; after obtaining the basic features of the code, the task-oriented vulnerability recognizer is trained, the effective capture of the code characteristic information is realized, and the fine-grained identification capability of the code vulnerability is enhanced; the application can meet the fine-grained identification demand of open source code vulnerability, and improve the fine-grained identification effect of the code vulnerability.

Description

Technical Field

[0001] This invention relates to the field of source code vulnerability detection technology in software analysis, and more specifically to a fine-grained method for identifying source code vulnerabilities based on vulnerability patch pairs. Background Technology

[0002] Software vulnerabilities pose a serious threat to social software infrastructure. The number of open-source software projects is growing at an unprecedented rate; in 2022, the number of new software repositories on GitHub exceeded 85.7 million, a year-on-year increase of 20%. Simultaneously, the number of vulnerabilities is increasing in tandem with the growth in code volume, meaning software programs face severe security risks. Data shows that software supply chain attacks have increased by an average of 742% annually over the past three years. The Log4j vulnerability disclosed in December 2021 threatens hundreds of millions of devices and will continue to cause harm for decades. To ensure software security, comprehensive vulnerability identification is needed to assist developers in fixing code vulnerabilities. Comprehensive vulnerability identification requires not only coarse-grained vulnerability detection and classification but also fine-grained localization of vulnerable code statements.

[0003] Extracting features from vulnerable code is crucial for fine-grained vulnerability identification. Pattern-based static analysis, similarity-based metrics, and data-driven learning-based methods are three main vulnerability identification approaches. Pattern-based methods aim to analyze code patterns or behaviors for vulnerability detection, but they heavily rely on expert experience. Similarity-based methods distinguish vulnerabilities by measuring the similarity between target code and known vulnerabilities, but these methods are easily misled due to the constant evolution of vulnerable code. Data-driven learning methods attempt to derive features of vulnerable code and their relationships from large amounts of data. However, while current methods can extract some code features from vulnerability patterns or vulnerable code, they are still insufficient for fine-grained vulnerability identification. This is because the essence of vulnerability identification is distinguishing vulnerable code from less vulnerable code. Only by capturing the subtle differences between them can fine-grained vulnerability identification be achieved. Current methods either directly summarize vulnerability rules from vulnerable code or directly extract vulnerability features from randomly organized samples, lacking attention to this distinction, resulting in a need to improve the ability to identify fine-grained vulnerabilities.

[0004] To improve the fine-grained identification capability of vulnerabilities, identification methods need to amplify the differences between vulnerable code and normal code, thereby preserving the comprehensive and refined syntactic and semantic features of the code. However, amplifying the differences between vulnerable code and normal code presents challenges. On the one hand, comparing the patched vulnerable code with the original vulnerable code reveals that the difference between vulnerable code and non-vulnerable code is minimal. On the other hand, in real-world open-source software projects, vulnerable code constitutes a very small percentage of the overall project. Statistics from the D2A and Big-Vul datasets show that in both datasets, over 60% of vulnerable functions contain no more than 5% vulnerable code statements, and the percentage of vulnerable code statements in the entire project code is even smaller. Therefore, capturing the differences between vulnerable code and normal code for fine-grained vulnerability identification is challenging, requiring identification models capable of learning the distinguishing features between vulnerable and normal code. Summary of the Invention

[0005] To overcome the defects and shortcomings of the existing technology, this invention provides a fine-grained identification of vulnerability in source code based on patched code pairs. The purpose of this invention is to provide a fine-grained identification of vulnerability based on patched code pairs (FIVul) for fine-grained identification of vulnerability statements in software source code, so as to better meet the needs of fine identification of vulnerable code in open source software, improve the ability of fine-grained identification of source code vulnerabilities, and thus provide support for enhancing the security of software systems.

[0006] To address the problems existing in the prior art, the present invention is achieved through the following technical solution.

[0007] This invention provides a fine-grained method for identifying source code vulnerabilities based on vulnerability patch pairs, the method comprising the following steps:

[0008] S1. Construct multimodal intermediate representation structures for the vulnerability code and patch code in the vulnerability patch pair respectively;

[0009] S2. Initialize the node feature representation of the multimodal intermediate representation structure;

[0010] S3. Code Comparison Feature Extraction: Specifically, vector mask data augmentation technology is used to augment the multimodal intermediate representation structure of the patch code, forming multi-view code data; the multimodal intermediate representation structure of the vulnerable code is directly used as a hard negative sample for feature learning; different neural network models are deployed for different types of multimodal intermediate representation structures to learn the different characteristics of the code, obtaining the global feature representation of each multimodal intermediate representation structure; the global feature representations of each type of multimodal intermediate representation structure are compared and learned; finally, the global feature representations of the multimodal intermediate representation structures are combined to learn more comprehensive code feature information.

[0011] S4. Train a task-oriented vulnerability identifyer to obtain detailed task-oriented features;

[0012] For vulnerability detection and classification tasks, the global feature representation of the multimodal intermediate representation structure is embedded and concatenated to form the overall representation vector of the target code, which is then input into a multi-layer fully connected network for training the vulnerability identifier for detection and classification tasks.

[0013] For fine-grained vulnerability code statement localization tasks, the node embedding is directly used as the feature representation input into a multi-layer fully connected network for training, and the final recognition result is output by voting among the multimodal intermediate representation structures.

[0014] S5. In the actual vulnerability detection and identification phase, the target code is extracted into a multimodal intermediate representation structure using the method in step S1; the node feature representation of the multimodal intermediate representation structure is initialized based on step S2; the comparison feature extractor trained in step S3 is used to obtain comparison features; and finally, the vulnerability identifier trained in step S4 is used for fine-grained vulnerability identification.

[0015] More preferably, the multimodal intermediate representation structure includes three intermediate representation structures: Code Property Graph (CPG), Code Sequence, and Abstract Syntax Tree (AST).

[0016] In a further preferred embodiment, in step S2, the code text is transformed into a vector representation after symbolization; the code of each intermediate representation structure constitutes the basic unit of the node vector representation; then, the Word2Vec model is trained on the symbolized code corpus to vectorize the code symbols; finally, multiple code symbols in the node are accumulated point by point to form the initial feature representation of the node.

[0017] More preferably, in step S2, the node feature representation of the multimodal intermediate representation structure is initialized, specifically including the following steps:

[0018] S201. Use a lexical analyzer to extract the basic unit token of the node code in each multimodal intermediate representation structure, and perform symbolic processing on the function names and variable names existing in the token;

[0019] S202. After symbolizing the node tokens, construct a corpus to train the Word2Vec model, and use the trained model to vectorize the node tokens.

[0020] S203. Within each intermediate representation structure, multiple token embeddings are accumulated point by point to form the initial feature representation of the node as a whole.

[0021] In a further preferred embodiment, step S3, code comparison feature extraction, specifically includes the following sub-steps:

[0022] S301. Perform data augmentation on the patch code in the vulnerability patch pair to obtain multi-view data; treat the vulnerability code data in the vulnerability patch pair as hard negative samples for subsequent feature learning.

[0023] S302. Learn global feature representations of code multimodal intermediate representation structures from data-augmented multi-view data and hard negative samples; deploy different neural network models to learn different characteristics of code for different types of code multimodal intermediate representation structures;

[0024] S303. Compare and learn the global feature representation of the obtained multimodal intermediate representation structure of the code to amplify the difference between the vulnerability code and the patch code;

[0025] S304. After performing global feature representation comparison learning on different multimodal intermediate representation structures of code, the multimodal feature loss is jointly combined to capture more comprehensive code feature information.

[0026] Furthermore, in step S301, a feature mask transformation function is applied to the multimodal intermediate representation structure to enhance the data; specifically, for the code attribute map G, a feature mask transformation function q is applied. i q j Generate two corresponding transformed views and For code sequences and code abstract syntax trees, similar feature mask transformation functions are used to generate multiple corresponding data views.

[0027] Furthermore, in step S302, for the code attribute graph, a graph neural network is used to capture the semantic characteristics of the code, which are formally calculated by the following formula:

[0028] in, This represents the data-augmented code attribute graph, g(·) represents the graph neural network model, r(·) represents the readout function used to obtain the global embedding vector of the code attribute graph, and h G The global feature vector representation of the code attribute graph;

[0029] For code sequences, a bidirectional gated recurrent unit (BGRU) is used to capture long-term dependency information in the code. The global feature representation of the code sequence is calculated by the following formula:

[0030] in, This represents the data-enhanced code sequence, BGRU(·) represents a bidirectional gated cyclic unit network, and h S A global feature vector representation of a code sequence;

[0031] For a code abstract syntax tree, a tree-structured long short-term memory (TreeLSTM) network is deployed to extract syntactic features from the code. The global feature representation of the code abstract syntax tree is calculated by the following formula:

[0032] In the formula, The code abstract syntax tree (CLSTM) is represented by the data augmentation algorithm; TreeLSTM(·) represents a tree-structured long short-term memory network used to update the embedding representations of CLSTM nodes; r(·) represents the read function used to compute the global embedding vector of the CLSTM; h T The global feature vector representation of the code abstract syntax tree.

[0033] In a further preferred embodiment, in step S303, an improved Information Noise Contrastive Estimate (InfoNCE) loss function is used as the learning objective to maximize the consistency between positive sample pairs while amplifying the difference between negative sample pairs.

[0034] Specifically, the global feature representation of the enhanced multi-view data in each modality of the code is directly regarded as a positive sample, the global feature representation of the corresponding vulnerability code is regarded as a hard negative sample, and other data in the same batch during training are regarded as negative samples. Cosine similarity is used to measure the similarity between samples.

[0035] Furthermore, the similarity between multiple views of the code attribute graph for the k-th sample can be calculated using the following formula:

[0036] in, This represents the feature vector of the i-th augmented view of the k-th code attribute graph. Let ||·|| represent the feature vector of the j-th augmented view of the k-th code attribute graph, and ||·|| represent the magnitude of the vector;

[0037] Then, comparative learning is performed using the improved InfoNCE loss function, letting... This represents the modulation similarity of the corresponding negative samples; where α is the adjustment factor, τ is the temperature parameter, and B represents the batch size during training. The graph structure feature representation of the vulnerability code corresponding to the k-th sample;

[0038] Therefore, the contrast loss for the k-th sample image mode Calculated using the following formula:

[0039]

[0040] The contrast loss of the code attribute map for the entire training batch Calculated using the following formula:

[0041]

[0042] Similarly, the contrastive loss for the code sequence representation is calculated using the same method. Loss of code abstract syntax tree representation

[0043] In a further preferred embodiment, in step S304, the overall loss is calculated by fusing the multimodal losses of the code through the introduction of learnable weight factors. The multimodal loss of the joint learning can be calculated by the following formula:

[0044] Where M = {G, S, T} represents the multimodal set, w m The learnable weight factor representing the m-th mode. This represents the loss of the m-th mode.

[0045] Furthermore, in step S4, the task-oriented vulnerability identifyr is trained, which specifically includes the following sub-steps:

[0046] S401. For the defined vulnerability detection and classification tasks, a comprehensive feature representation vector is constructed by directly aggregating the global features of the code's multimodal intermediate representation structure. The overall code feature representation vector is calculated using the following formula:

[0047] In the formula, || denotes the vector concatenation operation, h m The feature representation of the m-th mode;

[0048] Then, the calculated overall feature representation vector is input into a multi-layer fully connected network for identification to obtain the results of vulnerability detection and classification tasks; specifically, it is calculated using the following formula:

[0049] in, This represents the final detection result. MLP(·) represents a multilayer fully connected network, and the sigmoid function is used to output the final detection result.

[0050] S402. For the defined vulnerability statement localization task, directly export the node embeddings of the intermediate representation structure as its local feature representation vector; specifically,

[0051] For code property graphs, directly derive the embedding vectors of the nodes in the graph. In the formula, D G The set of node embedding vectors is represented by |V|, where |V| represents the number of nodes in the graph, and d represents the embedding vector of each node.

[0052] Similarly, obtain the local feature representation vectors of nodes in the code sequence and the code abstract syntax tree;

[0053] Then, the local feature representation vector of each modality is input into a multi-layer fully connected network for discrimination to obtain the discrimination structure of a single modality. Finally, the results of the multi-modal ensemble are fused through voting to locate the vulnerable code statement. Specifically, it is calculated by the following formula:

[0054] in, Indicates whether there is a vulnerability in the code statement, y G ,y S ,y T These represent the output of the code in the property diagram, code sequence, and code abstract syntax tree, respectively.

[0055] S403. During the training phase, the focal loss function is used to train the identifier for each vulnerability identification task separately for actual detection.

[0056] The beneficial effects of this invention are:

[0057] 1. This invention addresses the problem of insufficient fine-grained vulnerability identification capabilities in open-source software projects. Based on a multimodal intermediate representation structure of code, it utilizes vulnerability patch code to amplify the differences between vulnerable code and normal code, extracting comprehensive and detailed feature information to achieve fine-grained identification of vulnerable code. This invention not only effectively amplifies the subtle differences between vulnerable and normal code but also fully extracts comprehensive and detailed syntactic and semantic feature information from the code. After acquiring the basic code features, a task-oriented vulnerability identifier is trained to effectively capture code characteristic information, thereby enhancing the ability to identify code vulnerabilities in a fine-grained manner. This invention can meet the fine-grained identification requirements of open-source code vulnerabilities, improving the effectiveness of fine-grained identification of code fragility.

[0058] 2. This invention has a more comprehensive code feature extraction capability, and fully extracts code syntax and semantic features in different intermediate representation modalities by integrating the characteristics of multimodal intermediate representation, providing support for learning higher-order features of code.

[0059] 3. This invention can more effectively amplify the differences between vulnerability and normal code, perform data augmentation transformation on vulnerability patch code pairs and conduct feature comparison learning, more fully amplify the differences between vulnerability code and normal code, and improve the ability to learn code features.

[0060] 4. This invention has a more accurate fine-grained code vulnerability identification capability. It trains a feature extractor and a task-oriented vulnerability identifier based on vulnerability patch code, which helps to extract comprehensive and fine syntactic and semantic features of the code, thereby enhancing the fine-grained vulnerability identification effect. Attached Figure Description

[0061] Figure 1 This is a diagram showing the overall architecture of the FIVul method of the present invention;

[0062] Figure 2 This is a workflow diagram of the FIVul training feature extractor of the present invention;

[0063] Figure 3 The vulnerability detection performance of FIVul was compared with six other methods across six evaluation metrics.

[0064] Figure 4 The vulnerability detection performance of FIVul compared with six other methods on six evaluation classification metrics;

[0065] Figure 5 This section compares FIVul's vulnerability detection performance with six other methods across six evaluation metrics. Detailed Implementation

[0066] The following are exemplary embodiments of the invention as defined by the claims and their equivalents, taken in conjunction with the accompanying drawings, to aid in a comprehensive understanding. The specific details described herein are to be considered exemplary only and not to limit the scope of the invention. Therefore, those skilled in the art can make various changes and modifications to the embodiments without departing from the scope and spirit of the invention.

[0067] Example 1

[0068] As a preferred embodiment of the present invention, this embodiment discloses a method for fine-grained identification of source code vulnerabilities based on vulnerability patch pairs, comprising the following steps:

[0069] Step 1: Construct multimodal intermediate representations for the vulnerability code and corresponding patch code in the vulnerability patch pair, including three intermediate representation structures: Code Property Graph (CPG), Sequence, and Abstract Syntax Tree (AST).

[0070] Step 2: Initialize the node feature representations of the multimodal intermediate representation structure. Initializing node representations involves transforming the symbolic code text into vector representations. The code of each intermediate representation basic block constitutes the basic unit of the node vector representation. First, to eliminate the influence of developer naming habits, function names and variable names are mapped to uniform symbolic names (e.g., "FUN", "VAR"). Then, a Word2Vec model is trained on a symbolic code corpus to vectorize the code symbols. Finally, multiple code symbols in a node are accumulated point by point to form the initial feature representation of the node.

[0071] Step 3: Code Contrast Feature Extraction. Contrast feature extraction aims to learn feature vectors that amplify the differences between vulnerable and normal code. First, vector masking data augmentation is used to augment the multimodal representation of normal code, creating multi-view code data. The multimodal representation of vulnerable code is directly used as a hard negative sample for feature learning. Next, different neural network models are deployed for different types of intermediate code representation structures to learn the different characteristics of the code, obtaining the embedding vector for each intermediate representation. Then, the learned embedding vectors of each type of intermediate representation are compared to amplify the differences between vulnerable and normal code. Finally, the multimodal intermediate representation vectors are combined to learn more comprehensive code feature information.

[0072] Step 4: Training a Task-Oriented Vulnerability Identifier. The task-oriented vulnerability identifier is used to acquire fine-grained features specific to the task. For vulnerability detection and classification tasks, the global feature vectors of the multimodal intermediate representations are embedded and concatenated to form the overall representation vector of the target code, which is then input into a multi-layer fully connected network for detection and classification. For fine-grained vulnerability code statement localization tasks, the node embeddings are directly input as feature representations into the multi-layer fully connected network for identification, and the final identification result is output through a voting mechanism among the multimodal representations.

[0073] Step 5: In the actual vulnerability detection and identification phase, the target code is extracted into a multimodal representation structure using the method in Step 1, and the node representation is symbolized and initialized based on Step 2. Then, the contrast feature extractor trained in Step 3 is used to obtain contrast features, and finally, the recognizer trained in Step 4 is used for fine-grained vulnerability identification.

[0074] Example 2

[0075] As another preferred embodiment of the present invention, this embodiment further elaborates and supplements the technical solution of the present invention based on the above embodiment 1. In this embodiment, the initialization of node feature representation in step 2 includes:

[0076] Step 21: Use a lexical analyzer to extract the basic unit token of the node code in each multimodal representation, and symbolize the function names and variable names in the token to avoid the impact of different naming habits of developers on the semantics of the code.

[0077] Step 22: After symbolizing the node tokens, construct a corpus to train the Word2vec model, and use the trained model to vectorize the node tokens.

[0078] Step 23: Within each intermediate representation node, the initial feature representation of the node as a whole is formed by accumulating multiple token embeddings point by point.

[0079] Example 3

[0080] As another preferred embodiment of the present invention, this embodiment further elaborates and supplements the technical solution of the present invention based on the above-described embodiment 1 or embodiment 2. In this embodiment, the code comparison feature extraction in step 3 specifically includes:

[0081] Step 31: Perform data augmentation on the patch code within the vulnerability patch code to obtain multi-view data. Specifically, apply a feature mask transformation function to the multimodal intermediate representation to augment the data. For example, for the code attribute graph G, apply the feature mask transformation function q. i ,qj Generate two corresponding transformed views and For code sequences and code abstract syntax trees, similar feature mask transformation function classes are used to generate multiple corresponding data views. Vulnerable code data in vulnerability patch pairs are directly treated as difficult negative samples for subsequent feature learning.

[0082] Step 32: Learn global feature representations of the code multimodal intermediate representation structure from the enhanced multi-view data and hard negative samples. Different encoders are used to acquire code features based on the characteristics of different modalities. For the code attribute graph, a graph neural network is used to capture the semantic features of the code, formally calculated by the following formula:

[0083]

[0084] in, This represents the data-augmented code attribute graph, g(·) represents the graph neural network model, r(·) represents the readout function used to obtain the global embedding vector of the code attribute graph, and h G This represents the global feature vector representation of the code attribute graph.

[0085] For code sequences, a bidirectional gated recurrent unit (BGRU) is used to capture long-term dependency information in the code. The global feature representation of the code sequence is calculated by the following formula:

[0086]

[0087] In the formula, This represents the data-enhanced code sequence, BGRU(·) represents a bidirectional gated cyclic unit network, and h S A global feature vector representation of a code sequence.

[0088] For a code abstract syntax tree, a tree-structured long short-term memory (TreeLSTM) network is deployed to extract syntactic features from the code. The global feature representation of the code abstract syntax tree is calculated by the following formula:

[0089]

[0090] in, This represents the data-augmented Code Abstraction Syntax Tree (CAST). The TreeLSTM(·) network is used to update the embedding representations of CAST nodes. r(·) represents the readout function used to compute the global embedding vector of the CAST. h T The global feature vector representation of the code abstract syntax tree.

[0091] Step 33: Perform comparative learning on the acquired multi-modal structure multi-view global feature representations of the code to amplify the difference between vulnerable code and normal code. Specifically, an improved InformationNoise Contrastive Estimate (InfoNCE) loss function is used as the learning objective to maximize the consistency between positive sample pairs while amplifying the difference between negative sample pairs. The enhanced global feature representation of the multi-view data in each code modality is directly regarded as a positive sample, while the global feature representation of the corresponding vulnerable code is regarded as a hard negative sample, and other data in the same batch during training are regarded as negative samples. Cosine similarity is used to measure the similarity between samples. For example, the similarity between the multi-views of the code attribute graph of the k-th sample can be calculated by the following formula:

[0092]

[0093] in, Let ||·|| represent the feature vector of the i-th augmented view of the k-th code attribute graph, where ||·|| represents the magnitude of the vector.

[0094] Then, comparative learning is performed using the improved InfoNCE loss function, letting... This represents the modulation similarity of the corresponding negative samples. Here, α is the adjustment factor, τ is the temperature parameter, and B represents the batch size during training. This represents the graph structure feature representation of the vulnerable code corresponding to the k-th sample. Therefore, the contrastive loss for the graph modality of the k-th sample... Calculated using the following formula:

[0095]

[0096] The contrast loss of the code attribute map for the entire training batch Calculated using the following formula:

[0097]

[0098] Similarly, the contrastive loss for the code sequence representation is calculated using the same method. Loss of code abstract syntax tree representation

[0099] Step 34: After performing feature comparison learning on different intermediate representation structures, the multimodal feature losses are jointly combined to capture more comprehensive code feature information. Specifically, the overall loss is calculated by fusing the multimodal losses of the code through the introduction of learnable weight factors. The jointly learned multimodal loss can be calculated by the following formula:

[0100]

[0101] Where M = {G, S, T} represents the multimodal set, w m This represents the learnable weight factor for the m-th mode.

[0102] Example 4

[0103] As another preferred embodiment of the present invention, this embodiment further elaborates and supplements the technical solution of the present invention based on the above-described embodiments 1, 2, or 3. In this embodiment, the vulnerability identifier trained on a task-oriented basis in step 4 specifically includes:

[0104] Step 41: For the set vulnerability detection and vulnerability classification tasks, a comprehensive feature representation vector is constructed by directly aggregating the global features of the code multimodal intermediate representation. The overall code feature representation vector is calculated by the following formula:

[0105]

[0106] In the formula, || represents the vector concatenation operation. Then, the calculated overall feature representation vector is input into a multi-layer fully connected network for identification to obtain the results of vulnerability detection and classification tasks. Specifically, it is calculated using the following formula:

[0107]

[0108] in, This represents the final detection result. MLP (Multilayer Perceptron) represents a multilayer fully connected network, and the Sigmoid function is used to output the final detection result.

[0109] Step 42: For the defined vulnerability statement localization task, directly export the node embeddings of the intermediate representation structure as its local feature representation vectors. For example, for the code attribute graph, directly export the embedding vectors of the nodes in the graph:

[0110]

[0111] In the formula, D G This represents the set of node embedding vectors, where |V| represents the number of nodes in the graph, and d represents the embedding vector of each node. Similarly, it retrieves the local feature representation vectors of nodes in code sequences and code abstract syntax trees.

[0112] Then, the local feature representation vector of each modality is input into a multi-layer fully connected network for discrimination to obtain the judgment result of a single modality. Finally, the results of the multi-modal analysis are fused through voting ensemble to locate the vulnerable code statement, specifically calculated by the following formula:

[0113]

[0114] in, This indicates whether there are vulnerabilities in the code statements, y G ,y S ,y T These represent the output of the code in the property graph, sequence, and abstract syntax tree, respectively. Specifically, to reduce the impact of false negatives, if a conditional statement in any modality has a vulnerability, then the statement is output as vulnerable.

[0115] Step 43: During the training phase, the focal loss function is used to train the detector for each vulnerability identification task separately for actual detection.

[0116] Furthermore, step 5, the actual vulnerability detection and identification stage, specifically includes:

[0117] Step 51: Extract the intermediate representation structures of the target code in three modalities—code attribute graph, code sequence, and code abstract syntax tree—using the same method as in Step 1.

[0118] Step 52: Use the model trained in Steps 3 and 4 to extract multimodal features of the code and perform vulnerability identification for the corresponding task, and output the actual vulnerability identification results.

[0119] Example 5

[0120] As another preferred embodiment of the present invention, this embodiment discloses a fine-grained source code vulnerability identification device based on vulnerability patch pairs, as shown in the appendix to the specification. Figure 1 As shown, the system mainly consists of four parts: a multimodal representation construction module, a contrastive feature extraction module, a task-oriented vulnerability identification module, and an actual detection and identification module. The multimodal representation construction module is used to construct a multimodal intermediate representation structure from the original code that more easily represents the code's syntactic and semantic information. The contrastive feature extraction module performs data augmentation transformations on vulnerability patch code pairs and extracts features that reflect code differences through contrastive learning. The task-oriented vulnerability identification module extracts fine-grained code features for specific tasks, thereby adapting to specific vulnerability identification tasks. The actual detection and identification module performs actual vulnerability detection and identification after the model has been trained.

[0121] Step 1: Using the code analysis tool Joern, construct multimodal intermediate representations for the vulnerable function code and the corresponding patch code, such as... Figure 1 The extraction shown includes three intermediate code representation structures: Code Attribute Graph (CPG), Code Sequence, and Abstract Syntax Tree (AST).

[0122] Step 2: As Figure 2The diagram illustrates the initialization of node vector representations after extracting the multimodal intermediate representation structure of the code. Specifically, node representation initialization involves transforming the code text into a vector representation after symbolic transformation, with the code of each intermediate representation basic block forming the basic unit of the node vector representation. First, to eliminate the influence of developers' naming habits, function names and variable names are mapped to uniform symbolic names (e.g., "FUN", "VAR"). Then, a Word2Vec model is trained on a symbolic code corpus to vectorize the code symbols. Finally, multiple code symbols in a node are accumulated point by point to form the initial feature representation of the node.

[0123] Step 3: As Figure 2 The diagram illustrates a contrastive feature extractor trained using contrastive learning to highlight code differences. Contrastive feature extraction aims to learn feature vectors that amplify the differences between vulnerable and normal code. An improved Information Noise Contrastive Estimation (InfoNCE) loss function is used as the learning objective to maximize consistency between positive sample pairs while simultaneously amplifying the differences between negative sample pairs.

[0124] First, vector masking data augmentation techniques are used to augment the multimodal representation of normal code, thereby constructing multi-view code data. For example... Figure 2 Enhanced code data is illustrated by applying a feature masking transformation function to the multimodal intermediate representation. For example, for a code attribute map G, the feature masking transformation function q is applied. i ,q j Generate two corresponding transformed views and For code sequences and code abstract syntax trees, similar feature mask transformation function classes are used to generate multiple corresponding data views. Vulnerable code data in vulnerability patch pairs are directly treated as difficult negative samples for subsequent feature learning.

[0125] Next, a global feature representation of the code's multimodal intermediate representation structure is learned from the enhanced multi-view data and hard negative samples. For example... Figure 2 As shown, different encoders are used to acquire code features based on the characteristics of different modalities. For the code attribute graph, a graph neural network is used to capture the semantic features of the code, calculated by the following formula:

[0126]

[0127] in, This represents the data-augmented code attribute graph, g(·) represents the graph neural network model, r(·) represents the readout function used to obtain the global embedding vector of the code attribute graph, and h G This represents the global feature vector representation of the code attribute graph.

[0128] For code sequences, a bidirectional gated recurrent unit (BGRU) is used to capture long-term dependency information in the code. The global feature representation of the code sequence is calculated by the following formula:

[0129]

[0130] In the formula, This represents the data-enhanced code sequence, BGRU(·) represents a bidirectional gated cyclic unit network, and h S A global feature vector representation of a code sequence.

[0131] For a code abstract syntax tree, a tree-structured long short-term memory (Tree-LSTM) network is deployed to extract syntactic features from the code. The global feature representation of the code abstract syntax tree is calculated by the following formula:

[0132]

[0133] in, This represents the data-augmented Code Abstraction Syntax Tree (CAST). The Tree-LSTM network is used to update the embedding representations of CAST nodes. r(·) represents the readout function used to compute the global embedding vector of the CAST. h T The global feature vector representation of the code abstract syntax tree.

[0134] Then, comparative learning is performed on the acquired global feature representations of the multimodal structure of the code to amplify the difference between vulnerable code and normal code. The enhanced global feature representations of the multimodal data in each code modality are directly treated as positive samples, while the corresponding global feature representations of the vulnerable code are treated as difficult negative samples. Furthermore, other data from the same batch during training are considered negative samples. Specifically, cosine similarity is used to measure the similarity between samples. For example, the similarity between the multimodal views of the code attribute graph for the k-th sample can be calculated using the following formula:

[0135]

[0136] in, Let ||·|| represent the feature vector of the i-th augmented view of the k-th code attribute graph, where ||·|| represents the magnitude of the vector.

[0137] Then, comparative learning is performed using the improved InfoNCE loss function, letting... This represents the modulation similarity of the corresponding negative samples. Here, α is the adjustment factor, τ is the temperature parameter, and B represents the batch size during training. This represents the graph structure feature representation of the vulnerability code corresponding to the k-th sample. Therefore, the contrastive loss for the graph modality of the k-th sample... Calculated using the following formula:

[0138]

[0139] The contrast loss of the code attribute map for the entire training batch Calculated using the following formula:

[0140]

[0141] Similarly, the contrastive loss for the code sequence representation is calculated using the same method. Loss of code abstract syntax tree representation

[0142] Finally, as Figure 2 The joint learning section demonstrates how multimodal feature losses are jointly combined to capture more comprehensive code feature information. Specifically, the overall loss is calculated by fusing the multimodal losses of the code through the introduction of learnable weight factors. The multimodal loss of joint learning can be calculated using the following formula:

[0143]

[0144] Where M = {G, S, T} represents the multimodal set, w m This represents the learnable weight factor for the m-th mode.

[0145] Step 4: As Figure 1 The training task-oriented vulnerability identifyr shown is used for code vulnerability identification at different granularities.

[0146] First, for the defined vulnerability detection and classification tasks, a comprehensive feature representation vector is constructed by directly aggregating the global features of the code's multimodal intermediate representation. The overall code feature representation vector is calculated using the following formula:

[0147]

[0148] In the formula, || represents the vector concatenation operation. Then, the calculated overall feature representation vector is input into a multi-layer fully connected network for identification to obtain the results of vulnerability detection and classification tasks. Specifically, it is calculated using the following formula:

[0149]

[0150] in, This represents the final detection result. MLP (Multilayer Perceptron) represents a multilayer fully connected network, and the Sigmoid function is used to output the final detection result.

[0151] Then, for fine-grained vulnerability code statement localization tasks, the node embeddings of the intermediate representation structure are directly exported as their local feature representation vectors. For example, for code attribute graphs, the embedding vectors of the nodes in the graph are directly exported:

[0152]

[0153] In the formula, D G This represents the set of node embedding vectors, where |V| represents the number of nodes in the graph, and d represents the embedding vector of each node. Similarly, it retrieves the local feature representation vectors of nodes in code sequences and code abstract syntax trees.

[0154] After obtaining the local feature representation vectors, each modality's local feature representation vector is input into a multi-layer fully connected network for discrimination, obtaining the judgment result for a single modality. The results from multiple modalities are then fused using a voting ensemble method to locate the vulnerable code statement, specifically calculated by the following formula:

[0155]

[0156] in, Indicates whether there is a vulnerability in the code statement, y G ,y S ,y T These represent the output of the code in the property graph, sequence, and abstract syntax tree, respectively. Specifically, to reduce the impact of false negatives, if a conditional statement in any modality has a vulnerability, then the statement is output as vulnerable.

[0157] Finally, during the training phase, the focal loss function is used to train the detector for each vulnerability identification task separately for actual detection.

[0158] Step 5: As Figure 1 The vulnerability detection and identification stage, as shown, first extracts intermediate representation structures of three modalities from the target code: code attribute graph, code sequence, and code abstract syntax tree, using the same method as in step 1. Then, the model trained in steps 3 and 4 is used to extract multimodal features of the code and perform vulnerability identification for the corresponding task, outputting the actual vulnerability identification results.

[0159] The performance evaluation of FIVul's fine-grained code vulnerability identification and detection was conducted on the D2A dataset. D2A includes multiple vulnerability samples from six open-source projects: OpenSSL, Nginx, Libtiff, Httpd, Libav, and FFmpeg. Appropriate datasets were constructed for performance evaluation based on different vulnerability identification tasks. For vulnerability detection, a vulnerability detection dataset (Dataset1) was constructed using 15,591 vulnerability samples and 30,524 non-vulnerable samples. For vulnerability classification, Dataset2 was constructed using the 10 most frequent categories of samples from the D2A dataset, including common vulnerability types such as integer overflow, buffer overflow, and null pointer exception. For vulnerability statement localization, Dataset3 was constructed using vulnerable statements and normal statements from the D2A dataset, containing 26,278 vulnerable statements and 1,408,889 normal statements. Each dataset was further divided into training, validation, and test sets in a 3:1:1 ratio.

[0160] Figure 3 This presentation shows the vulnerability detection results of FIVul and six related methods on Dataset1. Six metrics were used for evaluation: Accuracy (Acc), Precision (Pre), Recall, F1-score (F1), False Positive Rate (FPR), and False Negative Rate (FNR). Each subplot in the figure shows the results of the seven methods on a single metric. Overall, compared to the other six methods, FIVul achieved better detection results, with the lowest FNR and FPR, and a good balance between the two, with an F1 score of 85.28%. Furthermore, from... Figure 3 The study also revealed two key findings: First, methods using only a single sequence performed worse than those incorporating multiple code information. Sequence-based methods like VulDeePecker, SySeVR, and ICVH exhibited poor performance. Methods fusing multiple code modalities, such as IVDETECT, VELVET, and FIVul, showed better detection results. Second, fine-grained vulnerability identification methods facilitated vulnerability detection. The fine-grained methods IVDETECT, VELVET, and FIVul outperformed the coarse-grained methods VulDeePecker, SySeVR, and DeepWukong because these methods not only collect global code features but also capture fine-grained local features to fully identify vulnerabilities. FIVul's superiority over other fine-grained methods reflects that utilizing the multimodal information of vulnerability patch pairs can derive more accurate fine-grained code features while preserving global code information.

[0161] Figure 4 This presentation showcases the vulnerability classification results of FIVul and six related methods on the Dataset2 dataset. Six evaluation metrics were used: Macro Precision (Pre-m), Weighted Precision (Pre-w), Macro Recall (Recall-m), Weighted Recall (Recall-w), Macro F1-score (F1-m), and Weighted F1-score (F1-w). FIVul demonstrated better overall vulnerability classification performance. Compared to the other six methods, FIVul showed a more balanced classification effect on data with highly uneven class distributions, achieving an F1-w of 63.09% and the highest Recall-w. Utilizing multimodal code representation methods is beneficial for deriving subtle differences between various vulnerabilities. IVDETECT, VELVET, and FIVul, due to their ability to integrate multimodal information from the code, exhibited better classification performance than other methods. DeepWukong also showed good results by integrating multiple relationships within the code in a graph structure. Methods that only extract single-modal information struggle to capture the differences between vulnerabilities; VulDeePecker, SySeVR, and ICVH all exhibit poor classification results. Overall, FIVul's vulnerability classification capabilities outperform other methods, demonstrating a better ability to perceive subtle variations between different vulnerabilities.

[0162] Figure 5This presentation showcases the results of FIVul and six related methods in locating vulnerable statements on Dataset3. Eight metrics were used for evaluation: Statement Identifying Accuracy (S_Acc), Statement F1-score (S_F1), Function Recall (F_Recall), Vulnerability Coverage Proportion (VCP), Jaccard Index (JI), BTP Precision (BTP_P), BTP Recall (BTP_R), and BTP Intersection Over Union (BTP_IOU). FIVul demonstrated more accurate vulnerability statement location, performing best in five of the eight metrics and exhibiting more balanced identification results. FIVul significantly improved the identification rate of vulnerable code statements: 34.52% for JI, 52.31% for BTP_R, and 44.75% for BTP_IOU, representing improvements of 6.28%, 16.19%, and 16.48% respectively compared to the best existing methods. Furthermore, it can be observed that: on the one hand, ICVH, IVDETECT, VELVET, and FIVul outperform the other three methods in terms of vulnerability localization results, indicating that vulnerability statement localization requires fully capturing the statement-level features of the code. VuldeePecker and SySeVR only capture the overall attributes of the slice sequence. DeepWukong can grasp the complex relationships of variant slice code, but they can only perform coarse-grained identification. On the other hand, since the difference between vulnerable code and normal code is small, locating vulnerable statements requires grasping the feature information of the statement itself and the contextual information between related statements. VELVET and FIVul, which combine global and local code features, outperform ICVH and IVDETECT, which only utilize local statement attributes. Overall, FIVul learns comprehensive and fine-grained features of the code from the multimodal intermediate representation of vulnerability patch pairs, achieving better vulnerability code statement localization results.

[0163] Although the inventive concept has been specifically shown and described with reference to exemplary embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the claims.

Claims

1. A fine-grained method for identifying source code vulnerabilities based on vulnerability patch pairs, characterized in that: The method includes the following steps: S1. Construct a multimodal intermediate representation structure for the vulnerability code and patch code in the vulnerability patch pair respectively; S2. Initialize the node feature representation of the multimodal intermediate representation structure, which includes three intermediate representation structures: code attribute graph, code sequence, and abstract syntax tree; S3. Code Comparison Feature Extraction: Specifically, vector mask data augmentation technology is used to augment the multimodal intermediate representation structure of the patch code, forming multi-view code data; the multimodal intermediate representation structure of the vulnerable code is directly used as a hard negative sample for feature learning; different neural network models are deployed for different types of multimodal intermediate representation structures to learn the different characteristics of the code, obtaining the global feature representation of each multimodal intermediate representation structure; the global feature representations of each type of multimodal intermediate representation structure are compared and learned; finally, the global feature representations of the multimodal intermediate representation structures are combined to learn more comprehensive code feature information. S4. Train a task-oriented vulnerability identifyer to obtain detailed task-oriented features; For vulnerability detection and classification tasks, the global feature representation of the multimodal intermediate representation structure is embedded and concatenated to form the overall representation vector of the target code, which is then input into a multi-layer fully connected network for training the vulnerability identifier for detection and classification tasks. For fine-grained vulnerability code statement localization tasks, the node embedding is directly used as the feature representation input into a multi-layer fully connected network for training, and the final recognition result is output by voting among the multimodal intermediate representation structures. S5. In the actual vulnerability detection and identification phase, the target code is extracted into a multimodal intermediate representation structure using the method in step S1; and the node feature representation of the multimodal intermediate representation structure is initialized based on step S2. The contrast feature extractor trained in step S3 is used to obtain contrast features, and finally the vulnerability identifier trained in step S4 is used for fine-grained vulnerability identification.

2. The method for fine-grained identification of source code vulnerabilities based on vulnerability patch pairs as described in claim 1, characterized in that: In step S2, the code text is transformed into a vector representation after symbolization. The code of each intermediate representation structure constitutes the basic unit of the node vector representation. Then, the Word2Vec model is trained on the symbolized code corpus to vectorize the code symbols. Finally, multiple code symbols in the node are accumulated point by point to form the initial feature representation of the node.

3. The method for fine-grained identification of source code vulnerabilities based on vulnerability patch pairs as described in claim 1, characterized in that: In step S2, the node feature representations of the multimodal intermediate representation structure are initialized, which specifically includes the following steps: S201. Use a lexical analyzer to extract the basic unit token of the node code in each multimodal intermediate representation structure, and perform symbolic processing on the function names and variable names existing in the token; S202. After symbolizing the node tokens, construct a corpus to train the Word2Vec model, and use the trained model to vectorize the node tokens. S203. Within each intermediate representation structure, multiple token embeddings are accumulated point by point to form the initial feature representation of the node as a whole.

4. The fine-grained source code vulnerability identification method based on vulnerability patch pairs as described in claim 1 or 3, characterized in that: In step S3, code comparison feature extraction specifically includes the following sub-steps: S301. Perform data augmentation on the patch code in the vulnerability patch pair to obtain multi-view data; treat the vulnerability code data in the vulnerability patch pair as hard negative samples for subsequent feature learning. S302. Learn global feature representations of code multimodal intermediate representation structures from data-augmented multi-view data and hard negative samples; deploy different neural network models to learn different characteristics of code for different types of code multimodal intermediate representation structures; S303. Compare and learn the global feature representation of the obtained multimodal intermediate representation structure of the code to amplify the difference between the vulnerability code and the patch code; S304. After performing global feature representation comparison learning on different multimodal intermediate representation structures of code, the multimodal feature loss is jointly combined to capture more comprehensive code feature information.

5. The method for fine-grained identification of source code vulnerabilities based on vulnerability patch pairs as described in claim 4, characterized in that: In step S301, a feature mask transformation function is applied to the multimodal intermediate representation structure to enhance the data; specifically, for the code attribute map G, a feature mask transformation function q is applied. i q j Generate two corresponding transformed views and For code sequences and code abstract syntax trees, similar feature mask transformation functions are used to generate multiple corresponding data views.

6. The method for fine-grained identification of source code vulnerabilities based on vulnerability patch pairs as described in claim 5, characterized in that: In step S302, for the code attribute graph, a graph neural network is used to capture the semantic characteristics of the code, which are formally calculated by the following formula: ;in, This represents the data-enhanced code attribute graph. This represents a graph neural network model. The `readout` function is used to obtain the global embedding vector of the code property graph. The global feature vector representation of the code attribute graph; For code sequences, a bidirectional gated loop unit is used to capture long dependency information in the code. The global feature representation of the code sequence is calculated by the following formula: ;in, This represents the data-enhanced code sequence. This represents a bidirectional gated cyclic cell network. A global feature vector representation of a code sequence; For a code abstract syntax tree, a tree-structured long short-term memory network is deployed to extract syntactic feature information from the code. The global feature representation of the code abstract syntax tree is calculated by the following formula: In the formula, This represents the data-enhanced code abstract syntax tree; This represents a tree-structured Long Short-Term Memory network used to update the embedded representation of code abstract syntax tree nodes; The read function is used to calculate the global embedding vector of the code abstract syntax tree; The global feature vector representation of the code abstract syntax tree.

7. The method for fine-grained identification of source code vulnerabilities based on vulnerability patch pairs as described in claim 6, characterized in that: In step S303, an improved information-noise contrastive estimation loss function is used as the learning objective to maximize the consistency between positive sample pairs while amplifying the difference between negative sample pairs. Specifically, the global feature representation of the enhanced multi-view data in each modality of the code is directly regarded as a positive sample, the global feature representation of the corresponding vulnerability code is regarded as a hard negative sample, and other data in the same batch during training are regarded as negative samples. Cosine similarity is used to measure the similarity between samples.

8. The method for fine-grained identification of source code vulnerabilities based on vulnerability patch pairs as described in claim 7, characterized in that: The similarity between multiple views of the code attribute graph for the k-th sample can be calculated using the following formula: ;in, This represents the feature vector of the i-th augmented view of the k-th code attribute graph. Indicates the first The first code attribute graph Feature vectors of an enhanced view Indicates the size of the vector; Then, comparative learning is performed using the improved InfoNCE loss function, letting... This represents the modulation similarity of the corresponding negative samples; where, It is a regulatory factor. It's a temperature parameter. Indicates the batch size during the training process. The graph structure feature representation of the vulnerability code corresponding to the k-th sample; Therefore, the contrast loss for the k-th sample image mode Calculated using the following formula: ； The contrast loss of the code attribute map for the entire training batch Calculated using the following formula: ； Similarly, the contrastive loss for the code sequence representation is calculated using the same method. Loss of code abstract syntax tree representation .

9. The method for fine-grained identification of source code vulnerabilities based on vulnerability patch pairs as described in claim 7, characterized in that: In step S304, the overall loss is calculated by fusing the multimodal losses of the code through the introduction of learnable weight factors. The multimodal loss of the joint learning can be calculated by the following formula: ;in, Represents a multimodal set. The learnable weight factor representing the m-th mode. Indicates the first Loss for each mode.

10. The method for fine-grained identification of source code vulnerabilities based on vulnerability patch pairs as described in claim 1 or 3, characterized in that: In step S4, the task-oriented vulnerability identifyer is trained, which includes the following sub-steps: S401. For the defined vulnerability detection and classification tasks, a comprehensive feature representation vector is constructed by directly aggregating the global features of the code's multimodal intermediate representation structure. The overall code feature representation vector is calculated using the following formula: In the formula, This represents the vector concatenation operation. Indicates the first Feature representation of each modality; Then, the calculated overall feature representation vector is input into a multi-layer fully connected network for identification to obtain the results of vulnerability detection and classification tasks; specifically, it is calculated using the following formula: ;in, This indicates the final test result. This represents a multi-layer fully connected network, using the sigmoid function to output the final detection results. S402. For the defined vulnerability statement localization task, directly export the node embeddings of the intermediate representation structure as its local feature representation vector; specifically, For code property graphs, directly derive the embedding vectors of the nodes in the graph. In the formula, The set representing the node embedding vectors. d represents the number of nodes in the graph, and d represents the embedding vector of each node; Similarly, obtain the local feature representation vectors of nodes in the code sequence and the code abstract syntax tree; Then, the local feature representation vector of each modality is input into a multi-layer fully connected network for discrimination to obtain the discrimination structure of a single modality. Finally, the results of the multi-modal ensemble are fused through voting to locate the vulnerable code statement. Specifically, it is calculated by the following formula: ;in, This indicates whether there are vulnerabilities in the code statements. These represent the output of the code in the property diagram, code sequence, and code abstract syntax tree, respectively. S403. During the training phase, the focal loss function is used to train the identifier for each vulnerability identification task separately for actual detection.

Images