Explanatable vulnerability detection method and system based on large model enhanced graph learning

Abstract

The invention discloses an interpretable vulnerability detection method based on large model enhanced graph learning, which comprises the following steps of: 1) performing sample denoising on training data, constructing a code attribute graph, and learning and training a vulnerability detection model based on a causal graph; 2) performing static analysis and feature extraction on a to-be-detected sample, and inputting the to-be-detected sample into the trained model for vulnerability detection; 3) performing context enhancement on the detected vulnerability sample to generate explanatory description information; according to the method, a real vulnerability mode is learned from a vulnerability sample with noise through sample denoising and a causal graph attention network model, meanwhile, a context-enhanced interpretation generation technology is put forward, high-quality vulnerability description is generated under the guidance of a retrieved similar sample to serve as interpretive information, and the vulnerability can be accurately identified. And the accuracy and interpretability of vulnerability detection are improved. The invention also provides an interpretable software vulnerability detection system based on large model enhanced causal graph learning.

Description

Technical Field

[0001] This invention relates to the field of software security, and in particular to an interpretable vulnerability detection method and system based on large model augmented graph learning. Background Technology

[0002] Software vulnerabilities are security flaws hidden in computational logic (such as source code), which can be exploited by hackers to carry out malicious acts, posing a threat to personal, corporate, and public safety. Thanks to the great success of deep learning in code-centric software engineering tasks, more and more learning-based vulnerability detection methods have been proposed. Compared with traditional methods that rely heavily on manual vulnerability specification, deep learning-based methods focus on building complex neural network models to automatically learn implicit vulnerability patterns from vulnerability source code. However, existing work has shortcomings in terms of generalization and interpretability. (1) On the one hand, these methods mainly extract features from the entire function. Therefore, the model performance will be severely affected by the parts of the function that are not related to the vulnerability. A simple countermeasure is to use slicing techniques to extract the vulnerability-related parts of the program. However, this preprocessing still requires specialized security expertise and is limited to a few types of vulnerabilities, making it unscalable in practice.

[0003] Furthermore, the literature Sahil Suneja, Yunhui Zheng, Yufan Zhuang, Jim A. Laredo, Alessandro Morari. Probing Model Signal-Awareness via Prediction-PreservingInput Minimization[C] / / Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC / FSE), 2021, pp. 945-955. points out that existing deep learning models tend to use subtle differences in the dataset (such as specific coding styles) to make predictions rather than the actual vulnerability features, which further degrades the model performance. (2) On the other hand, most existing works interpret vulnerabilities from the perspective of locating vulnerability statements. However, this granular interpretation may not be entirely consistent with the developers' practices or expectations, resulting in the need for a large amount of manpower to conduct secondary audits of the interpretation results. Summary of the Invention

[0004] Purpose of the Invention: The purpose of this invention is to provide an interpretable software vulnerability detection method based on large-model enhanced causal graph learning, which improves the accuracy of vulnerability detection while providing interpretability. This invention also provides an interpretable software vulnerability detection system based on large-model enhanced causal graph learning. The above-mentioned invention has been implemented and verified in practice.

[0005] Technical Solution: This invention provides an interpretable software vulnerability detection method based on large-model augmented causal graph learning, comprising the following steps:

[0006] 1) Denoise the training data and construct a code attribute graph, then train a vulnerability detection model based on causal graph learning;

[0007] 2) Perform static analysis and feature extraction on the samples to be tested, and input them into the trained model for vulnerability detection;

[0008] 3) Context enhancement is performed on the detected vulnerability samples to generate explanatory description information.

[0009] Furthermore, in step 1), the training data is denoised and a code attribute graph is constructed. A vulnerability detection model is then trained based on causal graph learning, including the following steps:

[0010] 1.1) Based on the vulnerability patch commit ID, execute the `git reset` command on the vulnerable code in the vulnerability dataset to precisely roll back to the position before and after the application of the vulnerability patch;

[0011] 1.2) Based on the line numbers of code changes marked in the vulnerability patch submission, use the ctage parser to obtain the vulnerable function and the corresponding patch function;

[0012] 1.3) Segment vulnerability remediation submissions at the function level and use a large language model to identify vulnerable functions;

[0013] 1.4) Based on the changed code lines in the vulnerability remediation submission, perform context slicing on the identified vulnerable functions, retain the code lines that have data or control dependencies with the changed code lines, and generate denoised samples;

[0014] 1.5) The denoised vulnerable code and non-vulnerable code are fed into the program compiler for static analysis to obtain the abstract syntax tree, control flow graph and program dependency graph, and code attribute graph is constructed accordingly;

[0015] 1.6) Use ContraBERT to analyze each node in the code attribute graph. Vector encoding is performed to obtain an m-dimensional initial feature embedding. ;

[0016] 1.7) The code attribute graph with embedded initial node features is fed into the graph feature encoder to iteratively update the feature embedding of each node in the graph:

[0017] (1),

[0018] In the formula, and Representing nodes respectively Iterative update and Feature embedding after each time step For activation function, This represents a learnable transformation matrix. This indicates a splicing operation. Represents a node The set of first-order neighbor nodes, Represents the set of first-order neighbor nodes Passed to the node in each time step Feature information:

[0019] (2),

[0020] In the formula, For nodes Iterative update Feature embedding after each time step The decay factor indicates how much information is transferred from the node. Along the edge Flowed to the node :

[0021] (3),

[0022] In the formula, and Representing nodes respectively and nodes The number of first-order neighbors;

[0023] 1.8) To distinguish between causal (or interfering) features in the code that are related to (or unrelated to) vulnerabilities, a code attribute graph encoded by a graph feature encoder is given. Two independent multilayer perceptrons are used to calculate the attention scores at the node level and the edge level, respectively:

[0024] (4),

[0025] (5),

[0026] In the formula, Representing causal feature subgraphs Middle node Node-level attention score, Representing causal feature subgraphs Middle The edge attention score. Similarly, and These represent the interference feature sub-maps respectively. Node-level and edge-level attention scores;

[0027] Based on node-level and edge-level attention scores, the original code attribute graph is... Decoupling into causal feature subgraphs and interference feature sub-map And calculate the graph-level feature representation respectively. and :

[0028] (6),

[0029] (7),

[0030] In the formula, and These represent the graph-averaged embedding layer and the pooling layer, respectively. Represents code attribute graph The adjacency matrix, This indicates element-wise multiplication. and Representing causal feature subgraphs respectively The edge mask and feature mask. Similarly, and Do not represent the interference feature sub-map Edge mask and feature mask;

[0031] 1.9) Based on causal feature subgraphs and interference feature sub-map The graph-level representation is used to calculate the cross-entropy loss. Uniform loss and causal loss :

[0032] Cross-entropy loss Ensure the detection model is on the training set The above uses causal features for prediction:

[0033] (8),

[0034] In the formula, Indicates a with A classifier for functions;

[0035] Average loss Ensure that the model classifies persuasive features with equal probability on each label (vulnerable or non-vulnerable):

[0036] (9),

[0037] In the formula, Denotes KL divergence, It is a uniform prior distribution;

[0038] Causal loss Maximize the causal impact of real vulnerability patterns on predicted labels:

[0039] (10)

[0040] In the formula, Indicates that it appears in the training set Interference characteristics in This indicates a random concatenation operation;

[0041] Ultimately, the total training loss for:

[0042] (11),

[0043] Furthermore, in step 2), static analysis and feature extraction are performed on the sample to be tested, and the sample is input into the trained model for vulnerability detection, including the following steps:

[0044] 2.1) Perform static program analysis on the code under test at the function level to obtain the abstract syntax tree, control flow graph and program dependency graph, and construct the code attribute graph accordingly;

[0045] 2.2) Use ContraBERT to analyze each node in the code attribute graph. Vector encoding is performed to obtain the initial feature embedding, which is then used as the input to the vulnerability detection model.

[0046] 2.3) Output functions that may be vulnerable.

[0047] Furthermore, in step 3), the detected vulnerability samples undergo context enhancement to generate explanatory descriptive information, including the following steps:

[0048] 3.1) Based on node attention scores The top 5 most dangerous statements in the calculation function ;

[0049] 3.2) The training set was retrieved using BM25 and graph feature encoder respectively. The top-K vulnerable functions that are syntactically and semantically similar to high-risk statements are used as context information. For the graph feature encoder, cosine similarity is employed. Calculate the semantic similarity between high-risk statements and vulnerable functions in the training set.

[0050] Ultimately, high-risk statements Vulnerability functions in the training set similarity The calculation is as follows:

[0051] (12)

[0052] 3.3) Using a context learning strategy, the detected vulnerable functions, high-risk statements, and the retrieved Top-K similar vulnerable functions (and their corresponding vulnerability descriptions) are input into the large language model to generate vulnerability explanations.

[0053] This invention also provides an interpretable software vulnerability detection system based on large model-enhanced causal graph learning, comprising a detection model training module, a vulnerability detection module, and a vulnerability interpretation module;

[0054] The detection model training module is used to denoise the training data and construct a code attribute graph, and train the vulnerability detection model based on causal graph learning.

[0055] The vulnerability detection module is used to perform static analysis and feature extraction on the sample to be tested, and input it into the trained model for vulnerability detection.

[0056] The vulnerability interpretation module is used to enhance the context of detected vulnerability samples and generate explanatory description information.

[0057] Furthermore, the detection model training module includes a denoising unit, a representation unit, and a learning unit;

[0058] The denoising unit is used to denoise the original vulnerability sample and filter out code segments that are irrelevant to the vulnerability.

[0059] The representation unit is used to perform static analysis on the input training samples to obtain the abstract syntax tree, control flow graph and program dependency graph, and to construct the code attribute graph accordingly;

[0060] The learning unit is used to encode the features of each node in the code attribute graph, which are then fed into the graph feature encoder to iteratively update the feature embedding of each node in the graph. Based on causal graph learning, the mixed loss is calculated to train the vulnerability detection model.

[0061] Furthermore, the vulnerability detection module includes a characterization unit and a detection unit;

[0062] The characterization unit is used to perform static analysis on the sample to be tested, construct a code attribute graph, and use ContraBERT to encode the features of the graph nodes;

[0063] The detection unit is used to perform vulnerability detection on the sample to be tested using a trained detection model.

[0064] Furthermore, the vulnerability explanation module includes a location unit, a retrieval unit, and a generation unit;

[0065] The location unit is used to calculate high-risk statements in vulnerable functions;

[0066] The retrieval unit is used to retrieve the top K vulnerable functions in the training set that are similar to high-risk statements as context information.

[0067] The generation unit is used to generate vulnerability explanations by inputting the detected vulnerable functions, high-risk statements, and similar vulnerable functions (and their corresponding vulnerability descriptions) into the large language model through a context learning strategy.

[0068] Beneficial Effects: Compared with existing technologies, the significant feature of this invention is the seamless integration of vulnerability detection and interpretation into a unified pipeline through LLM-assisted causal graph learning. Specifically, a sample denoising method based on code change is employed to automatically remove code elements in vulnerable functions that are irrelevant to the vulnerability logic without sacrificing label accuracy. Then, a novel causal-aware graph attention network is constructed to learn causal vulnerability features while maintaining efficiency. Furthermore, a context-enhanced generation module equipped with a hybrid retrieval strategy is developed to generate explanatory vulnerability description information. Attached Figure Description

[0069] Figure 1 This is a flowchart illustrating the present invention;

[0070] Figure 2 This is the prompt template used in this invention to identify vulnerable functions;

[0071] Figure 3 This is the prompt template used in this invention to generate vulnerability explanations. Detailed Implementation

[0072] The present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments.

[0073] Example 1

[0074] Please see Figure 1 , Figure 2 and Figure 3 As shown, the present invention provides an interpretable software vulnerability detection method based on large model-enhanced causal graph learning, comprising the following steps:

[0075] 1) Denoise the training data and construct a code attribute graph, then train a vulnerability detection model based on causal graph learning;

[0076] 1.1) Based on the vulnerability patch commit ID, execute the `git reset` command on the vulnerable code in the vulnerability dataset to precisely roll back to the position before and after the application of the vulnerability patch;

[0077] 1.2) Based on the line numbers of code changes marked in the vulnerability patch submission, use the ctage parser to obtain the vulnerable function and the corresponding patch function;

[0078] 1.3) Segment vulnerability remediation submissions at the function level and use a large language model to identify vulnerable functions, providing a prompt template such as... Figure 2 As shown.

[0079] 1.4) Based on the changed code lines in the vulnerability remediation submission, perform context slicing on the identified vulnerable functions, retain the code lines that have data or control dependencies with the changed code lines, and generate denoised samples;

[0080] 1.5) The denoised vulnerable code and non-vulnerable code are fed into the program compiler for static analysis to obtain the abstract syntax tree, control flow graph and program dependency graph, and code attribute graph is constructed accordingly;

[0081] 1.6) ContraBERT is used to perform vector encoding on each node in the code attribute graph to obtain the initial feature embedding;

[0082] 1.7) The code attribute graph after embedding the initial node features is fed into the graph feature encoder to iteratively update the feature embedding of each node in the graph;

[0083] 1.8) A causal graph neural network consisting of a multilayer perceptron, a graph averaging embedding layer, and a pooling layer is used to mine causal feature subgraphs and interference feature subgraphs from the code attribute graph;

[0084] 1.9) Based on the causal feature subgraph and the interference feature subgraph, calculate the cross-entropy loss. Uniform loss and causal loss The hybrid loss function is used to train the vulnerability detection model.

[0085] 2) Perform static analysis and feature extraction on the samples to be tested, and input them into the trained model for vulnerability detection;

[0086] 2.1) Perform static program analysis on the code under test at the function level to obtain the abstract syntax tree, control flow graph and program dependency graph, and construct the code attribute graph accordingly;

[0087] 2.2) ContraBERT is used to perform vector encoding on each node in the code attribute graph to obtain the initial feature embedding, which is used as the input to the vulnerability detection model;

[0088] 2.3) Output functions that may be vulnerable.

[0089] 3) Context enhancement is performed on the detected vulnerability samples to generate explanatory description information.

[0090] 3.1) Calculate the top 5 most dangerous statements in the vulnerability function based on node attention scores;

[0091] 3.2) The BM25 and graph feature encoders are used respectively to retrieve the top-K vulnerable functions in the training set that are syntactically and semantically similar to the high-risk statements as context information;

[0092] 3.3) Using a context learning strategy, the detected vulnerable functions, high-risk statements, and the retrieved Top-K similar vulnerable functions (and their corresponding vulnerability descriptions) are input into the large language model to generate vulnerability explanations.

[0093] Example 2

[0094] Corresponding to the interpretable software vulnerability detection method based on large model augmented causal graph learning in Embodiment 1, this Embodiment 2 provides an interpretable software vulnerability detection system based on large model augmented causal graph learning. Please refer to [link to Embodiment 2]. Figure 1 , Figure 2 and Figure 3 As shown, the system includes a detection model training module, a vulnerability detection module, and a vulnerability interpretation module.

[0095] The detection model training module is used to denoise the training data and construct a code attribute graph. It trains the vulnerability detection model based on causal graph learning. The detection model training module includes a denoising unit, a representation unit, and a learning unit.

[0096] The denoising unit is used to denoise the original vulnerability sample input, filtering out code fragments unrelated to the vulnerability. The prompt template used is as follows: Figure 2 As shown;

[0097] The representation unit is used to perform static analysis on the input training samples to obtain the abstract syntax tree, control flow graph and program dependency graph, and to construct the code attribute graph accordingly;

[0098] The learning unit is used to encode the features of each node in the code attribute graph, which are then fed into the graph feature encoder to iteratively update the feature embedding of each node in the graph. Based on causal graph learning, the mixed loss is calculated to train the vulnerability detection model.

[0099] The vulnerability detection module performs static analysis and feature extraction on the samples to be tested, which are then input into the trained model for vulnerability detection. The vulnerability detection module includes a representation unit, a feature extraction unit, and a detection unit.

[0100] The characterization unit is used to perform static analysis on the sample to be tested, construct a code attribute graph, and use ContraBERT to encode the features of the graph nodes;

[0101] The detection unit is used to perform vulnerability detection on the sample to be tested using a trained detection model.

[0102] The vulnerability interpretation module is used to enhance the context of detected vulnerability samples and generate explanatory description information. The vulnerability interpretation module includes a location unit, a retrieval unit, and a generation unit.

[0103] The location unit is used to calculate high-risk statements in vulnerable functions;

[0104] The retrieval unit is used to retrieve the top K vulnerable functions in the training set that are similar to high-risk statements as context information.

[0105] The generation unit is used to generate vulnerability explanations by inputting the detected vulnerable functions, high-risk statements, and similar vulnerable functions (and their corresponding vulnerability descriptions) into the large language model through a context learning strategy.

[0106] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. An interpretable vulnerability detection method based on large model enhanced graph learning, characterized in that, The method comprises the following steps: 1) sample denoising and code attribute graph construction on training data, and learning and training a vulnerability detection model based on a causal graph; 2) static analysis and feature extraction on a to-be-tested sample, and inputting the trained model for vulnerability detection; 3) context enhancement on the detected vulnerability sample, and generating explanatory description information.

2. The method of claim 1, wherein, In step 1), sample denoising and code attribute graph construction are performed on training data, and a vulnerability detection model is learned and trained based on a causal graph, comprising the following steps: 1.1) according to a vulnerability repair submission ID, performing a git reset command on the vulnerability code in the vulnerability data set to accurately roll back to the positions before and after the application of a vulnerability patch; 1.2) based on the code change line number marked in the vulnerability repair submission, using a ctage parser to obtain a vulnerability function and a corresponding patch function; 1.3) dividing the vulnerability repair submission in a function granularity, and using a large language model to identify the vulnerability function; 1.4) based on the changed code line in the vulnerability repair submission, performing context slicing on the identified vulnerability function, retaining the code lines having data or control dependency relationship with the changed code line, and generating a denoised sample; 1.5) inputting the denoised vulnerability code and non-vulnerability code into a program compiler for static analysis to obtain an abstract syntax tree, a control flow graph and a program dependency graph, and constructing a code attribute graph therefrom; 1.6) Vector encoding of each node in the code property graph using ContraBERT to obtain an initial m-dimensional feature embedding ; 1.7) inputting the code attribute graph with embedded initial node features into a graph feature encoder to iteratively update the feature embedding of each node in the graph: （1）， where, and denote the node iteration update and the feature embedding of the node is an activation function, denotes a learnable transformation matrix, denotes a concatenation operation, denotes the first-order neighbor node set of the node , denotes the first-order neighbor node set of the node the feature information passed to the node in each time step: （2）， where is a node iteratively updated feature embedding at time step is a decay factor indicating how much information mass flows from node along edge to node : （3）， wherein and denote the first order number of nodes and nodes respectively; 1.8) Given a code attributed graph encoded by the graph feature encoder Two independent multi-layer perceptrons are used to compute the node-level and edge-level attention scores respectively: （4）， （5）， wherein representing the causal feature subgraph middle node the node-level attention score of representing the causal feature subgraph middle edge the edge-level attention score of and representing the node-level and edge-level attention scores in the interference feature subgraph respectively Based on the node-level and edge-level attention scores, the original code property graph G is decoupled into a causal feature subgraph Gc and an interference feature subgraph Gi . The causal feature subgraph Gc and the interference feature subgraph Gi are respectively calculated . and : （6）， （7）， wherein, and denote the graph average embedding layer and the pooling layer, respectively, denotes the adjacency matrix of the code property graph , denotes the element-wise multiplication, and denote the edge mask and the feature mask of the causal feature subgraph , and denote the edge mask and the feature mask of the interference feature subgraph . 1.9) Graph-level representation based on causal feature sub-graph and interference feature sub-graph Cross-entropy loss , uniform loss and causal loss are computed respectively: cross-entropy loss Ensuring that the detection model is trained on a training set Prediction using causal features: （8）， In the formula, represents a band with a function of the classifier; Average loss Ensure that the model is equal to the probability of interference features on each label, the upper classification: （9）， wherein denotes the KL divergence, is a uniform prior distribution; causal loss maximizing the causal impact of the true vulnerability pattern on the predicted label: （10）， wherein represents the interference features present in the training set represents a random shuffling operation;​ Finally, the total training loss is: （11）。 3. The method of claim 1, wherein, In step 2), static analysis and feature extraction are performed on a to-be-tested sample, and the trained model is inputted for vulnerability detection, comprising the following steps: 2.1) performing static program analysis on the to-be-tested code in a function granularity to obtain an abstract syntax tree, a control flow graph and a program dependency graph, and constructing a code attribute graph therefrom; 2.2) Vector encoding of each node in the code property graph using ContraBERT to obtain initial feature embeddings as input to the vulnerability detection model. 2.2) Vector encoding of each node in the code property graph using ContraBERT to obtain initial feature embeddings as input to the vulnerability detection model. 2.3) outputting a function in which a vulnerability may occur.

4. The method of claim 1, wherein, In step 3), context enhancement is performed on the detected vulnerability sample to generate explanatory description information, comprising the following steps: 3.1) Based on node attention scores Compute top 5 high-risk statements in vulnerability function ; 3.2) Retrieving Top-K vulnerable functions from the training set using BM25 and graph feature encoder respectively Top-K vulnerable functions with similar syntax and semantics as the high-risk statement are used as context information, where cosine similarity is used for the graph feature encoder Calculate the semantic similarity between the high-risk statement and the vulnerable functions in the training set; Finally, high-risk statements Similarity to vulnerability functions in the training set is calculated as follows: （12）， 3.3) inputting the detected vulnerability function, the high-risk statement and the Top-K similar vulnerability functions retrieved into a large language model to generate vulnerability explanation through a context learning strategy.

5. An interpretable software vulnerability detection system based on large model enhanced causal graph learning, characterized in that, The method comprises a detection model training module, a vulnerability detection module and a vulnerability explanation module; The detection model training module is used for sample denoising and code attribute graph construction on training data, and learning and training a vulnerability detection model based on a causal graph; The vulnerability detection module is used for static analysis and feature extraction on a to-be-tested sample, and inputting the trained model for vulnerability detection; The vulnerability explanation module is used for context enhancement on the detected vulnerability sample to generate explanatory description information.

6. The system of claim 5, wherein, In the detection model training module, a denoising unit, a representation unit and a learning unit are included; The denoising unit is used for input denoising of an original vulnerability sample to filter code segments irrelevant to the vulnerability; The representation unit is used for static analysis on the input training sample to obtain an abstract syntax tree, a control flow graph and a program dependency graph, and constructing a code attribute graph therefrom; The learning unit is configured to encode features of each node in the code property graph, input the graph feature encoder, iteratively update the feature embedding of each node in the graph, and calculate a hybrid loss based on the causal graph learning to train the vulnerability detection model.

7. The system of claim 5, wherein, The vulnerability detection module comprises a representation unit and a detection unit. The representation unit is configured to perform static analysis on the to-be-tested sample, construct a code property graph, and encode features of the graph nodes using ContraBERT. The detection unit is configured to use the trained detection model to perform vulnerability detection on the to-be-tested sample.

8. The system of claim 5, wherein, The vulnerability explanation module comprises a locating unit, a retrieval unit, and a generation unit. The locating unit is configured to calculate a high-risk statement in the vulnerability function. The retrieval unit is configured to retrieve K similar vulnerability functions in the training set as context information. The generation unit is configured to input the detected vulnerability function, the high-risk statement, and the retrieved similar vulnerability functions into a large language model through a context learning strategy to generate a vulnerability explanation.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of claims 1-4.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of claims 1-4.

Images