Source code vulnerability detection method based on improved graph neural network
By improving the graph neural network method, combining the self-supervised attention model and local enhancement model, the problem of code graph data sparsity is solved, the effect of vulnerability detection is improved, and more efficient vulnerability capture and detection is achieved.
Patent Information
- Application Number
- CN202411040120.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-08
AI Technical Summary
The existing code representation method is not perfect enough, resulting in sparse code diagram data and it is difficult to capture its internal information, which in turn affects the effect of vulnerability detection.
The improved graph neural network method is adopted, combined with the self-supervised attention model SGAT and the local enhancement model LA, and the node features are enhanced through the local enhancement model, and the self-supervised attention model is used to learn the feature information in the graph data to form a comprehensive and efficient Vulnerability detection method.
Effectively process sparse graph data, reduce noise, comprehensively capture node characteristics, and improve the effectiveness and accuracy of vulnerability detection.
Smart Images

Figure CN118916884A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of code vulnerability detection, and in particular to a source code vulnerability detection method based on an improved graph neural network. Background Art
[0002] Graph data formed by code representation is the key to source code vulnerability detection. However, due to the imperfect code representation extraction method, sparse graph data appears, making it difficult to capture its intrinsic information through the model, ultimately leading to problems such as poor detection effect.
[0003] Therefore, it is necessary to study how to obtain potential information from the original features and effectively process the input graph data while avoiding the problem of loss of structural information caused by flattening the intermediate representation graph of the code. Summary of the invention
[0004] The purpose of the present invention is to provide a source code vulnerability detection method based on an improved graph neural network, which can comprehensively and efficiently capture node features and improve the effectiveness of vulnerability detection.
[0005] To achieve the above object, the present invention provides a source code vulnerability detection method based on an improved graph neural network, comprising the following steps:
[0006] S1. Combining the self-supervised attention model SGAT and the local enhancement model LA, a vulnerability detection method based on the improved graph neural network LA_SGAT is constructed;
[0007] S2, collect source code data sets and preprocess them to extract node features of the source code;
[0008] S3, locally enhancing the node feature through the local enhancement model LA, and combining it with the node feature to obtain the first output feature;
[0009] S4, taking the feature matrix and adjacency matrix formed by the first output feature of the node as input, training through the self-supervised attention model SGAT to obtain the second output feature;
[0010] S5. Based on the second output feature, the source code vulnerability detection result is obtained through pyramid pooling, fully connected layer and Softmax classification layer.
[0011] Preferably, in step S3, the local enhancement model LA includes constructing a generative model based on a variational autoencoder CVAE, and obtaining local enhancement features related to the node through the generative model as an additional input for training iterations.
[0012] Preferably, the loss function of the local enhancement model LA includes a supervision loss function and a consistency loss function, which are as follows:
[0013]
[0014] In the formula, is the supervision loss function, is the consistency regularization loss function, K is the training label, Y i is the true label of training sample i, is the predicted probability that training sample i belongs to category k, S is the number of samples, N is the number of elements in each sample, is the predicted value of the i-th element in the s-th sample, is the target value of the ith element.
[0015] Preferably, in step S4, the self-supervised attention model SGAT includes a self-supervised attention mechanism and a graph attention mechanism; wherein the self-supervised attention mechanism acts on the first output feature to obtain the relationship feature of the node, and samples the obtained relationship feature through the graph attention mechanism to obtain the second output feature.
[0016] Preferably, a self-supervised loss function is established based on negative sampling and binary logistic regression, as follows:
[0017]
[0018] In the formula, E - is negative sampling, E∪E - is the sampled set; 1 is the indicator function, when the edge (i, j) is a positive sample, the indicator function returns 1, otherwise it returns 0; is the attention coefficient of the graph neural network.
[0019] Preferably, the total loss function includes the cross entropy loss, self-supervision loss and L2 regularization loss of the node, as follows:
[0020]
[0021] In the formula, is the total loss function, is the cross entropy loss function, λ E , λ2 is the mixing coefficient, is the self-supervised loss function of the lth convolutional layer.
[0022] Therefore, the present invention adopts the above-mentioned source code vulnerability detection method based on improved graph neural network, which has the following technical effects:
[0023] (1) The local enhancement model LA can enhance the representation of sparse code graph data, reduce noise, and make up for the obvious shortcomings of sparse graph features;
[0024] (2) The self-supervised attention model SGAT can effectively learn the feature information in the graph data, and can process sparse graph features and edge noise in the graph data, comprehensively and efficiently capture node features, and improve the effectiveness of vulnerability detection.
[0025] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 It is a framework diagram of a source code vulnerability detection method based on an improved graph neural network;
[0027] Figure 2 It is a schematic diagram of the principle of the local enhancement model LA;
[0028] Figure 3 This is a schematic diagram of the attention mechanism of the self-supervised attention model SGAT;
[0029] Figure 4 It is a flow chart of the vulnerability detection model LA_SGAT;
[0030] Figure 5 It is the impact of the local enhancement model LA on GCN under the Devign dataset;
[0031] Figure 6 It is the impact of the local enhancement model LA on GraphSAGE under the Devign dataset;
[0032] Figure 7 It is the influence of the local enhancement model LA on GAT under the Devign dataset;
[0033] Figure 8 It is a schematic diagram of the detection results of different detection models on the FUNDED dataset;
[0034] Fig. 9 This is a schematic diagram of the detection results of different detection models on the CVE+Github dataset;
[0035] Fig.10 This is a schematic diagram of the detection results of different detection models on a single CWE vulnerability dataset. DETAILED DESCRIPTION
[0036] The present invention can be explained in more detail by the following examples. The purpose of disclosing the present invention is to protect all changes and improvements within the scope of the present invention. The present invention is not limited to the following examples.
[0037] like Figure 1As shown, the present invention provides a source code vulnerability detection method based on an improved graph neural network, including building a vulnerability detection model LA-SGAT based on a self-supervised attention model SGAT and a local enhancement model LA, as follows:
[0038] (1) Figure 2 As shown, the local enhancement model LA is achieved by generating more sample attributes in the local area. Implement the local enhancement function of the neural network as follows:
[0039] First, we use the conditional variational autoencoder CVAE to learn the connected neighbors u(u∈N v )Conditional distribution of node features.
[0040] Among them, the feature vector X of node v is used in the CVAE setting v As a condition, we get the node u feature vector X u The distribution of θ (z|X v ) generated latent variables z, data X u is obtained by dividing z and X v The conditional generation distribution p θ (X|X v ,z) generated: z~p θ (z|X v ),X u ~p θ (X|X v ,z v ). If represents the variational parameter, θ represents the generated parameter, and the following relationship exists:
[0041]
[0042] Among them, KL divergence is a distance metric not less than 0, and the confidence lower bound (ELBO) of the above formula is expressed as:
[0043]
[0044] In the formula, z (l) =:g φ (X v ,X u ,∈ (l) ),∈ (l) ~N(0,I), L is the number of neighbors of node v.
[0045] During the training phase of the conditional variational autoencoder CVAE, adjacent pairs (X v ,X u) as input, and seek to maximize the confidence lower bound ELBO; by maximizing ELBO, we can find the distribution closest to the true posterior distribution for Bayesian inference to optimize the training objective; in the generation stage, the node feature X v As a conditional variable, a latent variable z~N(0,I) is sampled as the input of the decoder, and the local feature vector associated with node v is obtained through the decoder. This is the local enhancement property that is needed.
[0046] In the vulnerability detection of the local enhancement model LA, the model parameter Θ is obtained by maximum likelihood estimation. The specific expression is as follows:
[0047]
[0048] In the formula, i represents the i-th data point in the training data set, A is the graph structure, X is the node feature, To generate features, “·” can be the class label Y on the node classification task.
[0049] In order to achieve Bayesian tractability, P Θ Divided into the product of two posterior probabilities, it can provide the GNN model with more samples generated by local neighbors, as follows:
[0050]
[0051] In the formula, and They represent the probability distribution of the GNN model and the generative model approximation, respectively, and are parameterized by Θ and Φ.
[0052] Pre-trained generative model Q Φ , as shown in Table 1.
[0053] Table 1 Generative model Q Φ Pre-training
[0054]
[0055]
[0056] in, is a parameter for active learning. After generating the feature matrix at each step, To evaluate; this function is obtained using Bayesian active learning (BALD) and approximated by Monte Carlo (MC) dropout samples. This parameter is often used in GNN optimization.
[0057] The local enhancement model LA has two loss functions: supervision loss function and consistency loss function. The supervision loss function mainly completes the pre-training of the local enhancement model LA and converts the local feature matrix generated by it into As an additional input to enhance the expressiveness of the GNN model. Given the augmented feature matrix of training labels K and S Finally, the supervised loss function for node classification and graph classification tasks as follows:
[0058]
[0059] In the formula, It represents the predicted probability or score of the model for training sample i belonging to category k. K is the training label. An augmented matrix feature matrix can be obtained based on K. Y i is the true label of training sample i.
[0060] Consistency Regularization Loss Function for:
[0061]
[0062] In the formula, S is the number of samples, It represents the current prediction value of the model for training sample i belonging to s. is the target value or reference value of the i-th element.
[0063] Based on the above supervised loss function Consistency Regularization Loss Function Using the local enhancement model LA of the graph neural network, we can get the local enhancement features As shown in Table 2.
[0064] Table 2 Local enhancement model LA of graph neural network
[0065]
[0066] When using the local enhancement model LA prediction of the above graph neural network, the training phase is used to generate the minimum verification loss function Or verify the highest accuracy on the dataset No need to regenerate This can reduce the computational overhead of the model.
[0067] The additional features obtained above With the original node feature X v Splicing,enhancing node feature information, and obtaining the first output feature.
[0068] (2) The self-supervised attention model SGAT includes a self-supervised attention mechanism and a graph attention mechanism. The self-supervised attention mechanism guides the attention mechanism based on whether there is an edge between nodes, thereby distinguishing the validity and weight of the edge; the graph attention mechanism combines the GO (GAT's Original Single-layer Neural Network) type and SD (Scaled DotProduct) type attention mechanisms to consider the relationship between global features and local features, thereby enhancing the model's ability to understand the graph structure.
[0069] The self-supervised attention mechanism is used to extract the first output feature obtained by the local enhancement model LA to obtain the relationship feature of each node, that is, the edge weight. To analyze the core parameters of edge weights, the specific expression is:
[0070]
[0071] In the formula, is the introduced coefficient, is a learnable function defined in the node feature space; σ is the Sigmoid activation function, which is used to map the attention coefficient to the interval (0,1); W is the weight matrix; h i 、h j are the features of node i and node j respectively.
[0072] The graph attention mechanism adopts a hybrid attention mechanism that combines the SD attention mechanism and the GO attention mechanism. The SD attention mechanism is a scaling operation on the DP attention mechanism of the original graph attention network GAT to control the size of the attention score, making the training process more stable and conducive to capturing global features, while preventing some large values from dominating the entire attention after softmax; in addition, the SD attention mechanism and the GO attention mechanism are combined to obtain a hybrid attention mechanism, which acts on the relationship features (edge weights) through the hybrid attention mechanism to enhance the feature information between related nodes, thereby obtaining the second output feature, such as Figure 3 shown.
[0073] Based on the idea of negative sampling and binary logistic regression, negative sampling E - , indicator function 1 and attention coefficient of graph neural network Introduce the cross entropy function to obtain the self-supervised loss function of the lth convolutional layer as follows:
[0074]
[0075] In the formula, E∪E -For the sampling set, negative sampling E is introduced - It can prevent the model from overfitting the training data; is the attention coefficient of the lth convolutional layer, which allows the model to better understand the importance of information in the graph structure; 1 is an exponential function, and an indicator function is used for each edge (i, j). When the edge (i, j) is a positive sample, the indicator function returns 1, otherwise it returns 0, ensuring that only valid samples in positive and negative samples are used to calculate the loss function.
[0076] Cross entropy loss on comprehensive node labels Self-supervised loss of the lth convolutional layer and L2 regularization loss, we get the total loss function for:
[0077]
[0078] In the formula, λ E , λ2 is the mixing coefficient.
[0079] The improved loss function consists of three parts: node classification loss, edge classification loss, and weight regularization term. It is mainly used for learning sparse graph data, enabling the network to effectively train graph neural networks and improve the model's expressiveness and generalization capabilities. Among them, the node classification loss is used to measure the performance of the model on the node classification task to improve the accuracy of node classification; the edge classification loss is used to train the attention coefficient of the graph neural network at the lth layer, thereby improving the model's ability to understand the graph structure; L2 regularization is used to control the complexity of the graph neural network and avoid overfitting the training data.
[0080] Combining the above self-supervised attention model SGAT and the local enhancement model LA, a vulnerability detection model LA-SGAT is formed to detect source code vulnerabilities, such as Figure 4 shown.
[0081] (1) Collect source code datasets, clean and standardize them, annotate them, and extract initial information to obtain a preprocessed vulnerability dataset;
[0082] (2) Extract code representation from the preprocessed data set to obtain the input matrix and adjacency matrix;
[0083] (3) The local enhancement model LA is used to generate the local enhanced features of the node, and the local enhanced features are combined with the original features F1 to form the enhanced features, i.e., the first output features.
[0084] (4) The F1-enhanced feature and the adjacency matrix associated with the row are used as input, and the self-supervised attention model SGAT is used for training. The above steps are repeated on the obtained result to obtain the feature output for classification, i.e., the second output feature.
[0085] (5) Using Spatial Pyramid Pooling (SPP), the feature output is converted into a vector of fixed size, and then classified through a fully connected layer and Softmax to obtain the final prediction result, thereby realizing vulnerability detection based on LA_SGAT.
[0086] This example uses Fan's dataset, Reveal's real dataset, Devign's artificial dataset FFMPeg+Qemu and self-collected dataset CVE+Github to verify the effectiveness of the vulnerability detection model LA-SGAT. The datasets Fan, Reveal and Devign are divided into training set, test set and validation set in a ratio of 8:1:1; the datasets FFMPeg+Qemu and CVE+Github are divided into training set, validation set and test set in a ratio of 6:1:3.
[0087] The parameter settings of the vulnerability detection model LA-SGAT are as follows:
[0088] The model was trained for 600 rounds, with an input dimension of 128, a hidden layer of 256, an output dimension of 2, a learning rate of 0.001, a dropout_rate of 0.3, and after multiple experiments, the number of convolutional layers was set to 3, the number of heads of the multi-head attention mechanism was set to 6, and the Relu function was used for the results of each layer to prevent overfitting, and Adam was used as the experimental optimizer.
[0089] Verification Example 1
[0090] This verification example selects the existing GCN, GAT and GraphSAGE models as the basic framework, adds the local enhancement model LA (represented by LA_GCN, LA_GAT and LA_SAGE), takes the Devign dataset as the research object, and uses the accuracy ACC, F1-score parameter, precision P (Precision), recall R (Recall) and area under the ROC curve AUC as evaluation indicators to verify the effectiveness of the local enhancement model LA. Figure 5-7 shown.
[0091] The results show that LA has the greatest improvement on the GAT structure detection results, with the F1 value in the detection results increasing from 0.58 to 0.60, an increase of 3.4%, while the improvement on GCN is the smallest, with the F1 value increasing from 0.59 to 0.60, an increase of 1.7%, and the overall detection F1 value increasing by 1.6%-3.4%. Therefore, the local enhancement model LA can improve the detection effect of common graph neural networks on artificial synthetic data sets.
[0092] Taking the real dataset of Reveal as the research object, the detection results are shown in Table 3.
[0093] Table 3 Validity verification results of LA structure on Reveal dataset
[0094] Model AUC P R F1 Reveal 0.72 0.31 0.60 0.41 Devign 0.74 0.33 0.32 0.32 SAGE 0.70 0.29 0.58 0.39 GCN 0.69 0.30 0.56 0.39 GAT 0.77 0.35 0.61 0.44 LA_SAGE 0.73 0.32 0.61 0.41 LA_GCN 0.72 0.31 0.60 0.40 LA_GAT 0.79 0.37 0.62 0.46
[0095] As can be seen from Table 3, the local enhancement model LA has an enhancement effect on the common graph neural network models GCN, SAGE, and GAT on the real dataset Reveal. Among them, the LA structure has the worst enhancement effect on GCN, and the F1 value in the experimental results has increased from the original 0.39 to 0.40, an increase of 2.5%. The enhancement effect on GraphSAGE is the best, and the F1 value has increased from 0.39 to 0.41, an increase of 5.1%. Therefore, the local enhancement model LA can improve the detection effect of vulnerability detection tasks under real datasets.
[0096] Verification Example 2
[0097] In this verification example, the Devign, Fan, Reveal and CVE+Github datasets are used, and the vulnerability detection models SySeVR, VulDeePeeker, Devign, Russell et al and Reveal are used as comparisons to verify the effectiveness of the vulnerability detection model LA_SGAT based on the improved graph neural network, as shown in Table 4.
[0098] Table 4 Effects of different detection methods on various data sets
[0099]
[0100] As shown in Table 4, the vulnerability detection model LA_SGAT has achieved good results on the Devign, Fan and Reveal datasets. Among them, on the Fan dataset, the F1 value obtained by LA_SGAT reached 0.35, which is 0.05 higher than the best result of 0.30 obtained by the model Reveal, and the improvement rate is 16%; the F1 value obtained on the Reveal dataset is 0.48, which is 0.08 higher than the best result of 0.40, and the comprehensive detection effect is improved by 20%; on the Devign dataset, the F1 value of LA_SGAT reached 0.66, which is 0.04 higher than the result of 0.62 obtained by the best model Reveal, and the improvement rate is 6.5%; at the same time, on the Devign dataset, the AUC value of LA_SGAT reached 0.85, which is 0.04 higher than the best performing Russell et al method, and the improvement rate is 5%, and the AUC value on the Reveal dataset reached 0.83, which is 0.06 higher than the best performing SySeVR method, and the improvement rate is 7.9%. Overall, the source code vulnerability detection model LA_SGAT is effective.
[0101] In addition, the datasets FUNDED and CVE+Github are selected as research objects, and the detection method Devign based on GGRNN and the detection method Reveal based on GGNN are used for comparison to verify the effectiveness of the vulnerability detection model LA_SGAT. Figure 8 and Fig. 9 shown.
[0102] Depend on Figure 8 and Fig. 9 It can be seen that among the vulnerability detection methods based on graph neural networks, LA_SGAT has the best detection results. On the FUNDED dataset, LA_SGAT's ACC can reach 0.86, which is 0.03 higher than the result of the best detection method Reveal, with an improvement rate of 3.6%. The F1 value obtained by LA_SGAT is 0.85, which is 0.01 higher than the result of the best method Reveal, with an improvement rate of 1.1%. On the CVE+Github dataset, the ACC value of the vulnerability detection method LA_SGAT reaches 0.76, which is 0.05 higher than the value of 0.71 of the best method Reveal, an improvement of 7%. The F1 value of LA_SGAT reaches 0.77, which is 0.04 higher than the value of 0.73 of the best method Reveal, an improvement of 5.4%.
[0103] In summary, LA_SGAT has better detection effects than the GGNN-based detection method Reveal and the GGRNN-based detection method Devign, and for the collected vulnerability data CVE+Github, LA_SGAT can also achieve the best detection results.
[0104] Verification Example 3
[0105] This verification example selects four CWE vulnerability datasets CWE-440, CWE-404, CWE-369 and CWE-191 in the FUNDED dataset as objects, and compares the vulnerability detection methods of VUDDY, Devign, Lin et al and FUNDED to verify the performance of the vulnerability detection model LA_SGAT under a single vulnerability. Fig.10 shown.
[0106] from Fig.10 As can be seen from the figure, in the single vulnerability datasets CWE-440, CWE-369, and CWE-191, LA_SGAT has better detection results than the vulnerability detection method FUNDED based on graph neural network. In addition, the average accuracy of LA_SGAT in the detection of four single vulnerabilities is 0.87, which is higher than the average accuracy of other detection models. Therefore, the vulnerability detection method LA_SGAT has a better performance than other detection methods.
[0107] Therefore, the present invention adopts the above-mentioned source code vulnerability detection method based on improved graph neural network, which can effectively process input graph data, while avoiding the problem of loss of structural information caused by flattening the intermediate representation graph of the code, thereby improving the effectiveness of vulnerability detection.
[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.
Claims
1. A source code vulnerability detection method based on improved graph neural network, characterized in that: The following steps are involved: S1. Combining the self-supervised attention model SGAT and the local enhancement model LA, a vulnerability detection method based on the improved graph neural network LA_SGAT is constructed; S2, collect source code data sets and preprocess them to extract node features of the source code; S3, locally enhancing the node feature through the local enhancement model LA, and combining it with the node feature to obtain the first output feature; S4, taking the feature matrix and adjacency matrix formed by the first output feature of the node as input, training through the self-supervised attention model SGAT to obtain the second output feature; S5. Based on the second output feature, the source code vulnerability detection result is obtained through pyramid pooling, fully connected layer and Softmax classification layer.
2. According to claim 1, a source code vulnerability detection method based on an improved graph neural network is characterized in that: In step S3, the local enhancement model LA includes constructing a generative model based on a variational autoencoder CVAE, and obtaining local enhancement features related to the node through the generative model as an additional input for training iterations.
3. A source code vulnerability detection method based on improved graph neural network according to claim 2, characterized in that: The loss function of the local enhancement model LA includes the supervision loss function and the consistency loss function, as follows: In the formula, is the supervision loss function, is the consistency regularization loss function, K is the training label, Y i is the true label of training sample i, is the predicted probability that training sample i belongs to category k, S is the number of samples, N is the number of elements in each sample, is the predicted value of the i-th element in the s-th sample, is the target value of the ith element.
4. According to claim 1, a source code vulnerability detection method based on improved graph neural network is characterized in that: In step S4, the self-supervised attention model SGAT includes a self-supervised attention mechanism and a graph attention mechanism; wherein the self-supervised attention mechanism acts on the first output feature matrix to obtain the relationship features of the nodes, and samples the obtained relationship features through the graph attention mechanism to obtain the second output features.
5. According to claim 4, a source code vulnerability detection method based on improved graph neural network is characterized in that: Based on negative sampling and binary logistic regression, a self-supervised loss function is established as follows: In the formula, E - is negative sampling, E∪E - is the sampled set; 1 is the indicator function, when the edge (i, j) is a positive sample, the indicator function returns 1, otherwise it returns 0; It is the probability that the model predicts (i, j) is a positive sample, which is mapped to between 0 and 1 through the Sigmod function.
6. A source code vulnerability detection method based on improved graph neural network according to claim 5, characterized in that: The total loss function includes the cross entropy loss, self-supervision loss and L2 regularization loss of the node, as follows: In the formula, is the total loss function, is the cross entropy loss function, λ E , λ2 is the mixing coefficient, is the self-supervised loss function of the lth convolutional layer.
