Fine-grained vulnerability detection method and system based on semantic perception sparse attention

Abstract

The invention relates to the technical field of vulnerability detection, and provides a fine-grained vulnerability detection method and system based on semantic perception sparse attention, and the method comprises the steps: analyzing a to-be-detected source code, and generating an abstract syntax tree, a control flow graph and a program dependency graph; carrying out weighted fusion on graph representations of the abstract syntax tree, the control flow graph and the program dependency graph, and integrating into a code attribute graph; for the code attribute graph, processing the code attribute graph by adopting a semantic perception sparse attention method combined with sparse attention, a learnable Token pruning method and Top-k interaction frequency to obtain a Token attention score; and based on the obtained Token attention scores, accumulating the attention score of each Token in each row of the code, and detecting and determining a statement where the vulnerability code is located. According to the detection method disclosed by the invention, the capability of modeling code semantic information and capturing a structured dependency relationship can be effectively improved, and meanwhile, the calculation complexity is remarkably reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field related to vulnerability detection, and more specifically, to a fine-grained vulnerability detection method and system based on semantically-aware sparse attention. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] In modern software development, code vulnerability detection is an important part of ensuring software security and reliability. With the continuous expansion of software development scale and increasing complexity, traditional vulnerability detection methods are facing unprecedented challenges. Traditional static analysis methods discover potential vulnerabilities by checking the syntax and structure of the code. Although this method is effective in some cases, it often produces a large number of false positives and false negatives. Existing dynamic analysis methods detect vulnerabilities by monitoring the behavior of the program at runtime. Although this method can capture runtime anomalies, its coverage and efficiency are often limited, especially when dealing with large code bases.

[0004] In recent years, with the rapid development of deep learning technology, code representation and analysis methods based on deep learning have gradually become a research hotspot. These methods use the powerful representation ability of deep learning models to automatically learn complex patterns and features from a large amount of code data, thus making up for the shortcomings of traditional methods to a certain extent. Transformer can capture the dependencies between long-distance features (Tokens) in the sequence through the self-attention mechanism, which is very beneficial for processing code, a text form with complex structure and long-distance dependencies. However, there are still some problems in directly applying Transformer to code analysis. First, the self-attention mechanism of Transformer has high computational complexity when processing long sequences, and faces huge computing resource consumption when processing large-scale code bases. Second, code is not just a simple text sequence, it contains rich syntax, control flow and data flow information. The traditional Transformer model mainly targets linear sequence relationships in natural language, and it is difficult to fully capture the complex structural information in the code, such as the deep semantic structure information of the code. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a fine-grained vulnerability detection method and system based on semantic-aware sparse attention, which can effectively improve the modeling of code semantic information and the capture of structured dependencies, while significantly reducing the computational complexity.

[0006] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:

[0007] One or more embodiments provide a fine-grained vulnerability detection method based on semantic-aware sparse attention, comprising the following steps:

[0008] Parse the source code to be tested and generate an abstract syntax tree, control flow graph and program dependency graph;

[0009] The graph representations of the abstract syntax tree, control flow graph, and program dependency graph are weightedly fused to form a code attribute graph.

[0010] For the code attribute graph, a semantic-aware sparse attention method combining sparse attention, learnable Token pruning method and Top-k interaction frequency is used to obtain the Token attention score.

[0011] Based on the obtained Token attention score, the attention score of each Token in each line of the code is accumulated to detect and determine the statement where the vulnerable code is located.

[0012] One or more embodiments provide a fine-grained vulnerability detection system based on semantic-aware sparse attention, including:

[0013] Graph generation module: configured to parse the source code to be tested and generate an abstract syntax tree, a control flow graph, and a program dependency graph;

[0014] Fusion module: configured to weightedly fuse the graph representations of the abstract syntax tree, the control flow graph, and the program dependency graph into a code property graph;

[0015] Semantic-aware sparse attention module: It is configured to process the code attribute graph using a semantic-aware sparse attention method that combines sparse attention, learnable token pruning, and top-k interaction frequency to obtain a token attention score.

[0016] Vulnerability identification module: It is configured to accumulate the attention score of each token in each line of the code based on the obtained token attention score, and detect and determine the statement where the vulnerability code is located.

[0017] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the above-mentioned fine-grained vulnerability detection method based on semantic-aware sparse attention are completed.

[0018] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the steps in the above-mentioned fine-grained vulnerability detection method based on semantic-aware sparse attention are completed.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] In the present disclosure, by extracting the code attribute graph of the code, combining the global-local sparse attention method, the learnable marker pruning method and the Top-k selection method to construct a new structure-aware sparse attention method, which can fully understand the structural information and semantic information of the code, can reduce the amount of calculation so that the model can process longer sequences, and by calculating the sparse attention score for the code markers, the amount of model calculation is reduced, and the statement information most relevant to the vulnerability is highlighted, so that more detailed code vulnerability detection can be performed.

[0021] The advantages of the present disclosure and the advantages of additional aspects will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and the description thereof are used to explain the present disclosure but do not constitute a limitation of the present disclosure.

[0023] Figure 1 is a block diagram of an example code of Embodiment 1 of the present disclosure;

[0024] Figure 2 is a schematic diagram of the structure of an abstract syntax tree of an example code of Embodiment 1 of the present disclosure;

[0025] Figure 3 is a control flow graph of the example code of Embodiment 1 of the present disclosure;

[0026] Figure 4 is a program dependency graph of the example code of Embodiment 1 of the present disclosure;

[0027] Figure 5 is a code property diagram of the example code of Embodiment 1 of the present disclosure;

[0028] Figure 6 is a model structure diagram of vulnerability detection in Example 1 of the present disclosure;

[0029] Figure 7 It is a flow chart of the vulnerability detection method of embodiment 1 of the present disclosure. DETAILED DESCRIPTION

[0030] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0031] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0032] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof. It should be noted that, in the absence of conflict, the various embodiments in the present disclosure and the features in the embodiments can be combined with each other. The embodiments will be described in detail below in conjunction with the accompanying drawings.

[0033] Technical data explanation:

[0034] Token: In language models, a token usually refers to a discrete unit of text, which can be translated as a word tag; a token can be a word, punctuation mark, number, or other language element, which is used as the basic unit for training and generating text. In NLP, tokenization technology is usually used to segment text into token sequences. Specifically, tokenization is the process of segmenting a continuous text string into discrete words or symbols. Each token is usually represented by a vector, which can be used to represent the elements of the text during model training and generation.

[0035] Example 1

[0036] In the technical solutions disclosed in one or more embodiments, Figures 1 to 7 As shown in FIG. 1 , a fine-grained vulnerability detection method based on semantic-aware sparse attention includes the following steps:

[0037] Step 1: Parse the source code to be tested and generate an abstract syntax tree (AST), a control flow graph (CFG) and a program dependency graph (PDG);

[0038] Step 2: Weightedly fuse the graph representations of the abstract syntax tree, control flow graph, and program dependency graph to form a code attribute graph;

[0039] Step 3: For the code attribute graph, a semantic-aware sparse attention method combining sparse attention, learnable Token pruning method, and Top-k interaction frequency is used to obtain the Token attention score.

[0040] Step 4: Based on the obtained Token attention score, accumulate the attention score of each Token in each line of the code to detect and determine the statement where the vulnerable code is located;

[0041] In this embodiment, by extracting the code attribute graph of the code, combining the global-local sparse attention method, the learnable marker pruning method and the Top-k selection method to construct a new structure-aware sparse attention method, which can fully understand the structural information and semantic information of the code, reduce the amount of calculation so that the model can process longer sequences, and calculate the sparse attention score for the code markers to reduce the amount of model calculation, highlight the statement information most relevant to the vulnerability, and perform more detailed code vulnerability detection.

[0042] In step 1, the abstract syntax tree (AST), control flow graph (CFG) and program dependency graph (PDG) are generated by parsing the source code. The example diagram is as follows: Figures 1 to 4 shown.

[0043] Finally, AST, CFGG and PDG are integrated into a unified code property graph (CPG). Figure 5 As shown;

[0044] Specifically, parse the source code to generate AST: use a compiler or code parsing tool to parse the source code, construct an abstract syntax tree, and ensure that the AST nodes cover the grammatical elements in the code, such as function declarations, variable declarations, conditional judgments, etc.

[0045] Specifically, parse the source code to generate CFG: By statically analyzing the code, a control flow graph is established to represent the logical path of program execution. The nodes of CFG represent code blocks, and the edges represent control flows, such as conditional branches and loops).

[0046] Specifically, analyze the code to generate PDG: analyze the data dependency and control dependency in the code to generate a program dependency graph, where data dependency describes the relationship between variables and control dependency describes the dependency conditions for statement execution.

[0047] In step 2, the method of integrating the abstract syntax tree, the control flow graph and the program dependency graph into a code property graph (CPG) includes the following steps:

[0048] Step 21: extract the mergeable nodes in the abstract syntax tree, control flow graph, and program dependency graph, and extract independent attribute information;

[0049] Determine which nodes in AST, CFG, and PDG can be merged. For example, a statement node in AST can be merged with a code block node in CFG. Mergeable nodes are nodes with the same identifier (such as a variable name or code block);

[0050] Optionally, retain independent attribute information, including the grammatical structure of AST, the control flow of CFG, and the dependencies of PDG.

[0051] Step 22: Edge integration: Integrate the edges of AST, CFG, and PDG, and annotate the edges of each relationship type (including syntax, control flow, and dependency) in the code property graph (CPG);

[0052] Furthermore, specific labels (such as grammatical relations, control relations, and data relations) can be assigned to different types of edges.

[0053] Step 23: Initialize the graph data structure, merge nodes with the same identifier (such as variable name or code block) to integrate the nodes; add edge relationships to obtain a code property graph (CPG);

[0054] Optionally, create a unified graph structure, which can be based on a graph database or graph processing framework to import nodes and edges of AST, CFG, and PDG.

[0055] Optionally, adding node attributes can preserve key information in the original graph (such as the type of AST node, the block type of CFG).

[0056] Specifically, add edge relationships, including:

[0057] Add the structural relationship edges in AST to CPG;

[0058] Map the control flow edges in CFG and the dependency edges in PDG into the unified graph respectively;

[0059] Label the type of each edge to distinguish AST edges, CFG edges, and PDG edges.

[0060] In this embodiment, by constructing a code property graph (CPG), the syntax, control flow, and data flow information of the code can be fully captured, helping the model understand the global structure and complex interactions of the code and improving the ability to capture the semantics of the code.

[0061] In step 3, the semantic-aware sparse attention method combines sparse attention, learnable token pruning method, and Top-k interaction frequency method. The fusion method includes:

[0062] Step 31, embed the nodes of the abstract syntax tree (AST), control flow graph (CFG) and program dependency graph (PDG), and use the attention mechanism to construct a shared attention matrix;

[0063] This embodiment extracts the abstract syntax tree, control flow graph and program dependency graph of the source code. The abstract syntax tree of the code can represent the grammatical structure information of the source code, the control flow graph can represent the control flow structure of the program, and the program dependency graph combines control dependency and data dependency to provide a comprehensive dependency representation that can fully capture the semantic relationship in the program, such as Figures 1 to 4 shown.

[0064] Step 311: embed the obtained abstract syntax tree (AST), control flow graph (CFG) and program dependency graph (PDG) to obtain corresponding node embedding matrices;

[0065] Specifically, after extracting different graph representations of the code, we first use the CodeBERT embedding layer to embed them, and obtain the corresponding node embedding matrix for each graph representation. For example, the AST node embedding is H AST , CFG node embedding is H CFG , PDG node embedding is H PDG ;

[0066] Step 312: setting weights and performing weighted fusion on the obtained node embedding matrix;

[0067] In order to fully capture the syntax, control flow, and data flow information of the code, this embodiment performs weighted fusion on different graph representations to obtain a code attribute graph. The specific formula is as follows:

[0068] H CPG =W AST H AST +W CFG H CFG +W PDG H PDG

[0069] Among them, W AST , W CFG , W PDG are the set weights for each graph embedding.

[0070] Step 313: Generate a shared attention matrix of CPG through a self-attention mechanism;

[0071] First, the fused node is represented by H CPG Perform linear transformation to obtain the Query, Key, and Value matrices:

[0072] Q=HW Q , K = HW K , V = HW V

[0073] Among them, W Q , W K With W V is the weight matrix of the attention mechanism

[0074] Next, we calculate the dot product of the query and the key and divide it by the scaling factor. Get the attention weight matrix of CPG

[0075] Finally, the softmax function is applied to the result to obtain the shared attention matrix A CPG =softmax(A CPG );

[0076] Step 32: Based on the shared attention matrix, a sparse attention matrix is ​​constructed using a global-local sparse attention method;

[0077] This embodiment adopts the global-local sparse attention method to construct a sparse attention matrix based on the shared attention matrix, aiming to reduce the computational complexity while maintaining the performance of the model.

[0078] Current shared attention matrix A CPG , whose dimension is [N,N], where N is the length of the sequence.

[0079] Specifically, global attention selects the token at a set position (such as CLStoken) and the tokens at all other positions based on the code attribute graph to perform attention calculations; local attention calculates the attention of each token only with a few positions adjacent to it.

[0080] A ij represents the attention score between position i and position j in the sequence.

[0081] Step 321: Create a sparse matrix S with the same dimension as the shared attention matrix, with an initial value of zero, S = 0 N×N ;

[0082] Step 322: perform global attention calculation, update the value of the initial sparse matrix S, and obtain a global attention matrix;

[0083] For global attention, the global attention index set is set to G, for example, G = {0, N-1}, indicating that the first position and the last position are global attention positions.

[0084] Then for i∈G, we have:

[0085]

[0086] Step 323: perform local attention calculation, update the value of the initial sparse matrix S, and obtain the local attention matrix;

[0087] For the local attention mechanism, each token only performs attention calculations on a few positions adjacent to it. The sliding window size is set to w, for example, w = 2. For i∈{0,1,…,N-1}, we have:

[0088]

[0089] Step 324: Combine the global attention matrix and the local attention matrix to construct a structure-aware sparse attention matrix.

[0090] Step 33, LTP matrix construction: For the source code to be detected, a learnable pruning matrix P is used to select important tokens and construct an LTP matrix; the elements of the matrix P represent the importance score of each token, and then the tokens are sorted and selected according to these scores;

[0091] Step 331: Initialize a pruning matrix P, P∈RN×1, and use a learnable function f to calculate the importance score of each token.

[0092] Assume that the input sequence is X, and its dimension is N×d, where d is the dimension of each token.

[0093] P = f(X), function f can be expressed as Pi = f(Xi; θ), where Xi is the i-th token and θ is the parameter of function f;

[0094] Specifically, the score of the pruning matrix P can be calculated using a multi-layer perceptron, namely:

[0095] Pi = σ(W2·ReLU(W1·X i +b1)+b2)

[0096] Among them, σ is the activation function softmax, W1 and W2 are weight matrices, b1 and b2 are bias vectors;

[0097] Step 331: Sort the tokens according to the importance score of each token calculated by the pruning matrix P, select the first k important tokens, set the pruning threshold k, and obtain the pruned token set L:

[0098] L = Top-k(X,P)

[0099] In order to keep the dimensions of the matrices the same, the k value in the top-k of LTP selects the sequence length N of the shared attention matrix.

[0100] Step 34, Top-k matrix construction: For the source code to be detected, the Top-k method is used on the basis of the shared attention matrix to obtain the Top-k matrix;

[0101] Step 341, the dimension of the shared attention matrix is ​​N×N, first sum the attention scores of each row of the shared attention matrix, and then take the average;

[0102]

[0103] Step 342: Sort all tokens according to the calculated attention scores, select the top k tokens with the highest scores, and set k to be the number of important tokens that you want to retain.

[0104] Specifically, in step 34, a sparse matrix T with the same dimension as the shared attention matrix is ​​first created, with an initial value of zero, T = 0 N×N ;

[0105] Set the Top-k index set to T, for example, T = {N k1 ,N k2 ,…,N k};

[0106] Then for i∈T, we have:

[0107]

[0108] The obtained matrix is ​​used as the Top-k matrix of the source code.

[0109] The top-k index is to filter the top k tokens with the highest attention scores under global attention as calculations (the subscripts k1, k2, k3..k here represent the top 1, 2, 3...k tokens with the highest scores). Therefore, the top-k matrix only calculates the adjacency matrix calculated between the top k tokens and all tokens.

[0110] For example, assuming that N1, N3, and N5 are the top three tokens with the highest scores, only the attention scores in Table 1 are calculated;

[0111] Table 1 Attention calculation details

[0112] N1 N2 N3 N4 N5 N1 1 1 1 1 1 N2 0 0 0 0 0 N3 1 1 1 1 1 N4 0 0 0 0 0 N5 1 1 1 1 1

[0113] In Table 1, 1 means that the attention score is calculated, and 0 means that the attention score is not calculated;

[0114] Step 35, constructing a semantic-aware sparse attention matrix: multiply the obtained shared attention matrix, sparse attention matrix, LTP matrix, and Top-k matrix element by element to obtain a semantic-aware sparse attention matrix;

[0115] This embodiment proposes a structure-aware sparse attention method composed of a shared attention matrix, a sparse attention matrix, an LTP matrix, and a Top-k matrix, which is implemented by element-by-element multiplication. The formula is as follows

[0116] A struct =A⊙S⊙L⊙T

[0117] By replacing the full self-attention in the traditional Transformer encoder with structure-aware sparse attention, it is possible to maximize the retention of the structural information of the code, improve computational efficiency, increase the flexibility and scalability of the model, and especially improve the ability to process long sequences of code. Figure 6 shown.

[0118] The score of each position in the semantic-aware sparse attention matrix is ​​the attention score of the corresponding token;

[0119] In step 4, fine-grained detection is performed to locate vulnerabilities in the code. Sparse attention mechanism is used to locate statement-level vulnerabilities, including the following steps:

[0120] Step 41: Based on the semantic-aware sparse attention matrix, filter out word tags with suspicious vulnerabilities for the function code;

[0121] The tokens with the highest attention scores in the code functions are most likely to have vulnerable vulnerabilities. Specifically, the sparse attention scores of each word token are calculated when processing the function, and these scores are aggregated from the 12 encoder modules. A vulnerability alert threshold is set to filter out word tokens with suspected vulnerabilities based on their attention scores.

[0122] The method proposed in this paper is an improvement on the CodeBERT pre-trained language model based on the Transformer architecture. The basic architecture of CodeBERT consists of twelve encoders, each of which includes an embedding layer, a multi-head attention layer (self-attention mechanism), a feedforward neural network, a residual connection, and normalization.

[0123] A semantically-aware sparse attention method designed in this embodiment replaces the self-attention mechanism in the original CodeBERT encoder with an attention method that combines sparse attention based on code attribute graphs, a learnable Token pruning method, and Top-k interaction frequency.

[0124] Step 42, use line breaks to split the function code into multiple token lists, each list representing a line of code;

[0125] Step 43: Based on the tag list, the attention scores of the word tags with higher scores in each line of code are summed up to obtain the summary attention score of the corresponding line;

[0126] Step 44: Select the line with the highest attention score as the code line where the vulnerability occurs;

[0127] Specifically, the aggregated sparse attention scores of all rows are compared and arranged in descending order to generate vulnerability detection results, and the code lines where the vulnerabilities occur are determined to issue early warning alerts.

[0128] A further technical solution is that the above process is implemented by building a vulnerability detection model, including improving the CodeBERT pre-trained language model based on the Transformer architecture. The basic architecture of CodeBERT includes multiple encoders, each of which includes a CPG unit, an embedding layer, a multi-head attention layer, a feedforward neural network, a residual connection and a normalization module.

[0129] The CPG unit is configured to build a code property graph;

[0130] The embedding layer is configured to embed the abstract syntax tree, control flow graph, and program dependency graph generated by parsing the source code to be detected;

[0131] Specifically, the multi-head attention layer includes:

[0132] The LTP unit is configured to adopt the Token pruning method to construct the LTP matrix;

[0133] The Top-k unit is configured to adopt the Top-k interaction frequencies and construct the Top-k matrix;

[0134] The semantic-aware sparse attention unit is configured to fuse the shared attention matrix, the sparse attention matrix, the LTP matrix, and the Top-k matrix to obtain the semantic-aware sparse attention matrix;

[0135] The attention score calculation unit is configured to accumulate the attention score of each token in each line of code;

[0136] The semantic-aware sparse attention unit is also connected to a residual connection and a normalization module, a feed-forward neural network, to implement the training process.

[0137] This embodiment uses the publicly available Big-Vul dataset, which is one of the largest datasets in the field of deep learning vulnerability detection. The Big-Vul dataset contains 3,754 code vulnerabilities, spanning 91 different vulnerability types. Before model training, the dataset is divided into training set, validation set, and test set according to the 8:1:1 standard. This embodiment mainly uses the pytorch framework. During the model training phase, the training set is propagated as the input of the network model to obtain the predicted probability of each sample. The label of the same dataset is used to calculate the loss value of the model through the cross entropy loss function. The parameter gradient of the network model is calculated by back propagation for the loss value, and the parameters of the network model are updated using the AdamW optimizer algorithm. Accuracy, F1, Precision, and Recall are used as indicators in this embodiment.

[0138] Accuracy=(TP+TN) / (TP+TN+FP+FN)

[0139] Recall=TP / (TP+FN)

[0140] Precision=TP / (TP+FP)

[0141] F1=2×Precision×Recall / (Precision+Recall)

[0142] Among them, TP represents true positive examples, which are the number of samples correctly predicted to have vulnerabilities; TN represents true negative examples, which are the number of samples correctly predicted to have no vulnerabilities; FP represents false positive examples, which are the number of samples incorrectly classified as having vulnerabilities; FN represents false negative examples, which are the number of samples incorrectly classified as having no vulnerabilities.

[0143] This embodiment will extract the code attribute graph of the code, and combine the global-local sparse attention method, the learnable tag pruning method and the Top-k selection method to build a new structure-aware sparse attention method, so that the model can fully understand the structural information and semantic information of the code, and can reduce the amount of calculation so that the model can process longer sequences. And by calculating the sparse attention score for the code tags, the amount of model calculation is reduced, and the statement information most related to the vulnerability is highlighted, helping security personnel to perform more detailed code vulnerability detection.

[0144] Example 2

[0145] Based on Example 1, this embodiment provides a fine-grained vulnerability detection system based on semantic-aware sparse attention, including:

[0146] Graph generation module: configured to parse the source code to be tested and generate an abstract syntax tree, a control flow graph, and a program dependency graph;

[0147] Fusion module: configured to weightedly fuse the graph representations of the abstract syntax tree, the control flow graph, and the program dependency graph into a code property graph;

[0148] Semantic-aware sparse attention module: It is configured to process the code attribute graph using a semantic-aware sparse attention method that combines sparse attention, learnable token pruning, and top-k interaction frequency to obtain a token attention score.

[0149] Vulnerability identification module: It is configured to accumulate the attention score of each token in each line of the code based on the obtained token attention score, and detect and determine the statement where the vulnerability code is located.

[0150] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation process is the same, which will not be repeated here.

[0151] Example 3

[0152] This embodiment provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the fine-grained vulnerability detection method based on semantic-aware sparse attention of Embodiment 1 are completed.

[0153] Example 4

[0154] This embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps in the fine-grained vulnerability detection method based on semantic-aware sparse attention in Embodiment 1 are completed.

[0155] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

[0156] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A fine-grained vulnerability detection method based on semantically aware sparse attention, characterized in that: The steps include: Parse the source code to be tested and generate an abstract syntax tree, control flow graph and program dependency graph; The graph representations of the abstract syntax tree, control flow graph, and program dependency graph are weightedly fused to form a code attribute graph. For the code attribute graph, a semantic-aware sparse attention method combining sparse attention, learnable Token pruning method and Top-k interaction frequency is used to obtain the Token attention score. Based on the obtained Token attention score, the attention score of each Token in each line of the code is accumulated to detect and determine the statement where the vulnerable code is located.

2. The semantically-aware sparse attention-based fine-grained vulnerability detection method according to claim 1, characterized in that: The method of integrating an abstract syntax tree, a control flow graph, and a program dependency graph into a code property graph comprises the following steps: Extract mergeable nodes from abstract syntax trees, control flow graphs, and program dependency graphs, and extract independent attribute information; Label the edges of each relationship type in the code property graph; Initialize the graph data structure, merge mergeable nodes to integrate nodes; add edge relationships to obtain the code attribute graph.

3. The semantically-aware sparse attention-based fine-grained vulnerability detection method according to claim 1, characterized in that: The semantic-aware sparse attention method combines sparse attention, learnable token pruning, and Top-k interaction frequency methods. The fusion method includes: Embed the nodes of the abstract syntax tree, control flow graph, and program dependency graph, and use the attention mechanism to build a shared attention matrix; Based on the shared attention matrix, a global-local sparse attention method is used to construct a sparse attention matrix; For the source code to be detected, a learnable pruning matrix P is used to select important tokens and construct the LTP matrix; For the source code to be detected, the Top-k method is used based on the shared attention matrix to obtain the Top-k matrix; The obtained shared attention matrix, sparse attention matrix, LTP matrix and Top-k matrix are multiplied element by element to obtain the semantic-aware sparse attention matrix.

4. The semantically-aware sparse attention-based fine-grained vulnerability detection method according to claim 3, characterized in that: The method for constructing a shared attention matrix includes: Embed the obtained abstract syntax tree, control flow graph and program dependency graph to obtain the corresponding node embedding matrix; Set weights and perform weighted fusion on the obtained node embedding matrix; Generate a shared attention matrix through the self-attention mechanism.

5. The semantically-aware sparse attention-based fine-grained vulnerability detection method according to claim 3, characterized in that: Constructing a sparse attention matrix includes the following steps: Create a sparse matrix S with the same dimension as the shared attention matrix, with an initial value of zero; Perform global attention calculation, update the value of the initial sparse matrix S, and obtain the global attention matrix; Perform local attention calculation, update the value of the initial sparse matrix S, and obtain the local attention matrix; The global attention matrix and the local attention matrix are combined to construct a structure-aware sparse attention matrix.

6. The semantically-aware sparse attention-based fine-grained vulnerability detection method according to claim 3, characterized in that: The LTP matrix construction method includes: Initialize a pruning matrix and use a learnable function f to calculate the importance score of each token; According to the pruning matrix P, the importance score of each token is calculated, the tokens are sorted, and the top k important tokens are selected. The pruning threshold k is set to obtain the pruned token set L as the LTP matrix. Alternatively, the Top-k matrix construction method includes the following steps: The shared attention matrix dimension is N×N. First, the attention scores of each row of the shared attention matrix are summed up and then the average is taken; According to the calculated attention scores, all tokens are sorted and the top k tokens with the highest scores are selected; Create a sparse matrix T with the same dimension as the shared attention matrix, calculate the retained token global attention, update the value of the sparse matrix T corresponding to the position of the token, and get the Top-k matrix.

7. The semantically-aware sparse attention-based fine-grained vulnerability detection method according to claim 1, characterized in that: The method for determining the statement where the vulnerable code is located uses a sparse attention mechanism to locate the vulnerability at the statement level, including the following steps: Based on the semantic-aware sparse attention matrix, we filter out word tags with suspicious vulnerabilities from the function code; Based on the semantic-aware sparse attention matrix, we filter out word tags with suspicious vulnerabilities from the function code; Based on the token list, the attention scores of the word tokens with higher scores in each line of code are summed up to get the summary attention score of the corresponding line; Select the line with the highest attention score as the code line where the vulnerability occurs.

8. A fine-grained vulnerability detection system based on semantically aware sparse attention, characterized by: include: Graph generation module: configured to parse the source code to be tested and generate an abstract syntax tree, a control flow graph, and a program dependency graph; Fusion module: configured to weightedly fuse the graph representations of the abstract syntax tree, the control flow graph, and the program dependency graph into a code property graph; Semantic-aware sparse attention module: It is configured to process the code attribute graph using a semantic-aware sparse attention method that combines sparse attention, learnable token pruning, and top-k interaction frequency to obtain a token attention score. Vulnerability identification module: It is configured to accumulate the attention score of each token in each line of the code based on the obtained token attention score, and detect and determine the statement where the vulnerability code is located.

9. An electronic device, characterized in that: The invention comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the fine-grained vulnerability detection method based on semantic-aware sparse attention as described in any one of claims 1 to 7 are completed.

10. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps in the semantically-aware sparse attention-based fine-grained vulnerability detection method described in any one of claims 1 to 7.

Images