A high-order vulnerability detection method and system based on graph representation and sparse Transformer

Abstract

The present disclosure provides a high-order vulnerability detection method and system based on graph representation and sparse Transformer, which relates to the field of information security detection technology, including: obtaining character stream data of module program source code; performing lexical analysis on the character stream data, and saving variable information of global variables or user-controlled input variables in a data table; generating a code attribute graph for each source code, and generating a persistent storage data flow relationship between programs by searching for variable information persistently stored in a data table; inputting the persistent storage data flow relationship into the information of nodes in a learning graph in a GNN module to obtain an embedding vector of the node; and inputting the embedding vector of the node into a Transformer module with sparse attention, using a threshold-based pruning sentence pruning algorithm to prune sentences layer by layer in the Transformer, and performing attention sparseness, adaptively deleting irrelevant sentences, and converting high-order vulnerabilities into low-order vulnerabilities.

Description

Technical Field

[0001] The present disclosure relates to the field of information security detection technology, and in particular to a high-order vulnerability detection method and system based on graph representation and sparse Transformer. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] In today's highly connected digital society, information security faces increasingly complex and widespread threats. Vulnerability detection is a vital part of information security management. Timely discovery and patching of vulnerabilities can effectively reduce the risk of system attacks and protect the security and integrity of critical data and systems. However, due to the complexity of software and the wide variety of vulnerabilities, vulnerability detection is a challenging task.

[0004] Most of the existing vulnerability detection methods detect low-level vulnerabilities, relying on the data flow, control flow and other relationships, and perform vulnerability detection by learning the structural relationship of the source code. During the propagation process, a piece of data is persistently stored. When it reaches a security-sensitive node, it lacks the dependency relationship with the source node data flow and control flow. The resulting vulnerability is called a high-order vulnerability. The process of detecting such vulnerabilities is cumbersome, and the methods for detecting them are limited and the accuracy is not high. Therefore, it is more convenient to convert high-order vulnerabilities into low-order vulnerabilities and detect them using low-order vulnerability detection methods. The method using the full-attention Transformer may take hours to weeks to reveal the contextual information required for vulnerability analysis, which greatly increases the time performance consumption, and the source code program that generates high-order vulnerabilities has a long sequence. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a high-order vulnerability detection method and system based on graph representation and sparse Transformer. Long sequence codes are processed by Transformer with sparse attention. At the same time, based on the Transformer using sparse attention, a sentence pruning algorithm is combined to improve the efficiency and interpretability of the model through the visualization of sentence pruning.

[0006] According to some embodiments, the present disclosure adopts the following technical solutions:

[0007] A high-level vulnerability detection method based on graph representation and sparse Transformer, including:

[0008] Get the character stream data of the module program source code;

[0009] Performing lexical analysis on the character stream data, and saving variable information of global variables or user-controlled input variables in a persistent storage into a data table;

[0010] Generate a code property graph for each source code, and generate persistent storage data flow relationships between programs by searching for variable information stored persistently in the data table;

[0011] Input the persistent storage data flow relationship into the GNN module to learn the information of the nodes in the graph and obtain the embedding vector of the node;

[0012] The embedding vector of the node is then input into the Transformer module with sparse attention. The threshold-based pruning sentence pruning algorithm is used to prune sentences layer by layer in the Transformer, and attention is sparsely processed to adaptively delete irrelevant sentences and convert high-order vulnerabilities into low-order vulnerabilities.

[0013] According to some embodiments, the present disclosure adopts the following technical solutions:

[0014] A high-level vulnerability detection system based on graph representation and sparse Transformer, including:

[0015] A data acquisition module is used to acquire character stream data of module program source code;

[0016] A preprocessing module, used for performing lexical analysis on the character stream data, and saving variable information of global variables or user-controlled input variables in a persistent storage into a data table;

[0017] The feature extraction module is used to generate a code attribute graph for each source code. By searching the variable information stored persistently in the data table, the persistent storage data flow relationship between programs is generated. The persistent storage data flow relationship is input into the GNN module to learn the information of the nodes in the graph and obtain the embedding vector of the node.

[0018] The pruning module is used to input the node's embedding vector into the Transformer module with sparse attention, prune sentences layer by layer in the Transformer using a threshold-based pruning sentence pruning algorithm, perform attention sparsification, adaptively delete irrelevant sentences, and convert high-order vulnerabilities into low-order vulnerabilities.

[0019] According to some embodiments, the present disclosure adopts the following technical solutions:

[0020] A non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, a high-order vulnerability detection method based on graph representation and sparse Transformer is implemented.

[0021] According to some embodiments, the present disclosure adopts the following technical solutions:

[0022] An electronic device comprises: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes and implements the high-order vulnerability detection method based on graph representation and sparse Transformer.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The present invention discloses a high-order vulnerability detection method based on graph representation and sparse Transformer. Existing vulnerability detection methods usually detect low-order vulnerabilities without considering the complex situation of high-order vulnerabilities. GNN and Transformer are used at the same time to combine the advantages of the two methods. GNN is good at processing graph structure data and can effectively capture the relationship and structural information between nodes. Transformer has a strong global information capture capability and can focus on the relationship between any two nodes in the graph. After combining the two, the characteristics of graph structure data can be better utilized. At the same time, the global modeling capability of Transformer can be used to improve the understanding of complex relationships and the accuracy of prediction, greatly reducing the complexity of detection and improving the accuracy of detection.

[0025] The present invention discloses a high-order vulnerability detection method based on graph representation and sparse Transformer, which uses Transformer with sparse attention to improve computational efficiency and processing power by reducing the number of elements that need to be calculated in the attention mechanism. This is particularly useful for processing long sequences, because the computational complexity of the standard attention mechanism is the square of the sequence length, and sparse attention can reduce this complexity. Sparse attention can reduce the interference of irrelevant information, improve the model's attention concentration and sensitivity to important information. This plays a positive role in improving the accuracy and stability of the model.

[0026] The present invention discloses a high-order vulnerability detection method based on graph representation and sparse Transformer. In the Transformer with sparse attention, sentence pruning is learned. Since not all sentences in the input sequence are necessary for model performance, the overall computation amount during model reasoning can be significantly reduced by removing less relevant tags. Sentence pruning can reduce the interference of irrelevant information and enhance the robustness of the model in the face of noisy data or incomplete data. This makes the model more stable and reliable in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0028] Figure 1 is a graph structure of an embodiment of the present disclosure;

[0029] Figure 2 This is the calculation process of sparse attention in the embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0031] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0032] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0033] Explanation of terms

[0034] 1. Lexical analysis: Lexical analysis is a stage in the compilation process, which is mainly responsible for dividing the source code into morphemes with independent meanings according to lexical rules. These morphemes are the basic units of subsequent processing by the compiler. The lexical analyzer generates these morphemes by identifying keywords, identifiers, operators, delimiters, etc.

[0035] 2. Code Property Graph: Code Property Graph (CPG) is a graphical data structure used to represent code structure and semantic information. It includes not only various entities in the code (such as functions, classes, variables, etc.), but also the relationships and dependencies between them. CPG is usually composed of multiple levels of graphs, each of which represents different abstract views and relationships, such as control flow graphs, data flow graphs, call graphs, etc. Through CPG, comprehensive static analysis of the code can be performed to support applications such as code understanding, refactoring, and security analysis.

[0036] 3. High-level vulnerabilities: If a piece of data is stored in a persistent storage during the transmission process, such as in a global variable, local file, database, etc. When the data is used again, it lacks the data flow and control flow relationship with the previous stage. Therefore, when the data (source) input by the user passes through the persistent storage and reaches a security-sensitive node, the resulting vulnerability is difficult to detect due to the lack of dependency with the source node. This type of vulnerability is called a high-level vulnerability.

[0037] 4. GNN: GNN (Graph Neural Network) is a type of neural network used to process graph structured data. Graph neural networks can capture complex relationships and dependencies between nodes in a graph by performing message passing and aggregation operations on the nodes and edges of the graph. The basic working principle of GNN includes the following steps:

[0038] (1) Message passing: A node transmits its own state (features) to adjacent nodes through edges.

[0039] (2) Aggregation: Each node collects and aggregates messages from its neighbor nodes.

[0040] (3) Update: Each node uses the aggregated message to update its own state.

[0041] 5. Transformer: Transformer is a deep learning model for natural language processing (NLP) tasks. It was first proposed by Vaswani et al. in their 2017 paper “Attention is All You Need”. The Transformer model introduces a new architecture that processes input sequences mainly through a self-attention mechanism, overcoming many limitations of traditional sequence models such as RNN and LSTM.

[0042] 6. Sparse Attention: Sparse Attention is an attention mechanism that is different from traditional fully connected attention (such as self-attention). In sparse attention, the model only pays attention to some positions or information in the input sequence, rather than fully connecting every position. This method can reduce computational costs and resource consumption, while also achieving good results in some tasks. Sparse attention can be implemented by selectively calculating attention weights through masks or specific sampling strategies, thereby improving the efficiency and scalability of the model.

[0043] Example 1

[0044] The present invention discloses a high-order vulnerability detection method based on graph representation and sparse Transformer. The Transformer with sparse attention can process long sequence codes well. At the same time, based on the Transformer with sparse attention, a learned sentence pruning algorithm is combined and implemented to improve the efficiency and interpretability of the model through the visualization of sentence pruning. The method includes:

[0045] Step 1: Get the character stream data of the module program source code;

[0046] Step 2: Perform lexical analysis on the character stream data, and save variable information of global variables or user-controlled input variables in a persistent storage into a data table;

[0047] Step 3: Generate a code property graph for each source code, and generate the persistent storage data flow relationship between programs by searching for variable information stored persistently in the data table;

[0048] Step 4: Input the persistent storage data flow relationship into the GNN module to learn the information of the nodes in the graph and obtain the embedding vector of the node;

[0049] Step 5: Input the node embedding vector into the Transformer module with sparse attention again, use the threshold-based pruning sentence pruning algorithm to prune sentences layer by layer in the Transformer, perform attention sparsification, adaptively delete irrelevant sentences, and convert high-order vulnerabilities into low-order vulnerabilities.

[0050] As an embodiment, the specific implementation process of a high-order vulnerability detection method based on graph representation and sparse Transformer disclosed in the present invention includes:

[0051] Step 1: Since global variables, user-controlled input variables and other persistently stored variables are easy to become tainted data, and global variables are usually at the beginning of the source code, and user-controlled input variables are usually defined in the module interface function, such variables are easy to find and record them in the data table.

[0052] Specifically, first find the variables that are persistently stored (data that are prone to become taint sources) such as global variables and user-controlled input variables. Global variables are generally in front of the module interface, and user-controlled inputs are generally defined in the interface function.

[0053] Step 2: Perform lexical analysis on each module program to convert the character stream of the source code into a series of meaningful morphemes (tokens). Keywords, identifiers, operators, constants, and separators are all tokens.

[0054] Specifically, the lexical analyzer is used to scan the source program character stream from left to right. During the scanning process, the lexical analyzer will ignore blank characters (such as spaces, tabs, and newlines) and comments, and will generate meaningful word symbols according to predefined lexical rules, usually including keywords, identifiers, operators, delimiters, and constants. First, a new data table is created. When an identifier is scanned, it is compared with the variables stored in the data table. If the data table already has information about the variable, just add it to the data table at the location where it appears.

[0055] If the variable information does not exist in the data table, store the name of this identifier and the position where it first appears in the data table. During the lexical analysis process, save its information in a table, which stores its name, the position where it first appears, and the position where it is used. Because this persistently stored variable may appear multiple times. Record the position of the first appearance to facilitate the subsequent generation of a data flow edge. If this variable appears in a security-sensitive node, a taint propagation chain is formed through these data flow edges, so that the graph structure information can be learned.

[0056] Step 3: Generate a code attribute graph inside each source code interface function, and connect the calling relationships between functions through calling edges;

[0057] Specifically, the open source tool joern is used to generate a code property graph for each source code, which includes an abstract syntax tree (AST), a control flow graph (CFG), a data flow graph (DFG) or a program dependency graph (PDG) and the calling relationships between internal functions.

[0058] Step 4: When the persistently stored variable appears for the first time in a node in the graph, search the data table to find the corresponding variable. If there are other locations in the data table that also use this variable, generate a data flow edge between the two variables in the graph structure based on the location where this variable first appears. In this way, the information of the taint chain of high-order vulnerabilities is saved in a graph, and then the information of the nodes in the graph is learned through the GNN module to obtain the embedding vector of the node. It is input into the Transformer, and through the Transformer with sparse attention, sentence pruning is learned to reduce trivial information in the code snippet and improve the performance of vulnerability detection;

[0059] Since variables are persistently stored, there is a lack of structural information about the variables propagating in the graph. Therefore, the information about persistently stored variables is obtained by searching the data table, and their execution order is obtained. A data flow edge is formed between every two variables. If there are security-sensitive operations, a taint propagation chain is formed. Then the information about the nodes in the graph can be learned through subsequent GNN.

[0060] Specifically, first, the source code needs to be converted into a graph structure. Each source code statement is taken as a node, and the edges between the nodes represent the dependencies or execution order between the statements. A random walk is performed on the graph to generate a series of node sequences. The depth and width of the walk can be controlled by parameters to explore different areas in the graph. Using the walk sequence as input, a model is trained to learn the vector representation of each node. The parameters of the model are adjusted through optimization algorithms (such as gradient descent) so that the generated vector can capture the semantic and structural information of the nodes in the graph. As for the representation of the initial vector, it is usually a randomly initialized vector. The initial vector of each node is generated by node2vec, and the adjacency matrix is ​​used to represent the graph structure, thereby capturing the correlation between nodes and edges in the graph. As Figure 1 As shown in the figure, this is a generated graph structure. Each number represents the number of lines of code in the program. The lines with arrows represent different types of edges. The line from 3 to 1 represents the taint propagation data flow of a persistent storage variable. Then the adjacency matrix of this graph is generated, as shown in Table 1, for subsequent node information updates. F11 represents the first line of code node in a source code.

[0061] Table 1 Adjacency matrix of the graph

[0062]

[0063] Furthermore, the graph structure is input into the GNN, and the adjacency matrix is ​​multiplied by the initial vector matrix to obtain the node vector representation updated after one layer of GNN message passing. A two-layer GNN is set, because too many layers of GNN will lead to over-smoothing problems, that is, the feature information of the nodes becomes similar after multiple layers, making it difficult to distinguish different nodes. It may also cause the gradient to disappear or explode, making the model difficult to train. Let H l is the information matrix of the l-layer nodes, H l +1=W*H l , W is the adjacency matrix of the graph.

[0064] Each sentence of the source code is represented by a node in the graph structure, and then the node vector after passing through the 2-layer GNN is input into the Transformer with sparse attention. Local attention is used in the sparse attention layer of the Transformer. Assume that a sliding window with a window size of 3 is given, and the vector representing the source code with n sentences is input. Figure 2 As shown in , starting from left to right, for each step from bottom to top, the sliding window connects each unit covered by the window size in the current step with the unit covered by the window in the next step. Therefore, each unit is paid attention to by its neighbors. When the window slides over the input sequence, as Figure 2 As shown in Table 2, the unit i (each unit in the sliding window, w is the window size) in the calculation of sparse attention at each step q (1<=q<=n) can not only be directly attended to by units from i−w to i+w, but also indirectly connected to more units because its direct neighbors are attended to by these units. In general, with sliding window attention, the sentence loses the information of a wide range of units in a single step, but regains it through depth by stacking multiple layers. As the depth of the layer increases, a single unit can obtain more and more information. The calculation of sparse attention is shown in Table 2.

[0065] Table 2 Calculation process of sparse attention

[0066]

[0067] Furthermore, after the attention matrix is ​​horizontally normalized by the softmax function in the Transformer model, the attention scores can be vertically accumulated to summarize the importance score of each token, as shown in Table 3. Then, sentence pruning can be adaptively performed using threshold-based pruning, which can significantly reduce the overall computation during model inference by removing less relevant sentences. In the pruning stage, the cumulative attention score of each sentence is compared with the threshold learned in each layer to adaptively remove the trial sentences. Here, the Transformer with sparse attention adopts learned sentence pruning, including soft pruning and hard pruning. It includes the following three steps:

[0068] 1) In soft pruning, by applying a fractional mask θ l The soft mask is used to train the model parameters and pruning threshold of each layer, where θ l is the soft pruning threshold of layer l.

[0069] 2) Binarize the decimal mask generated in soft pruning and fix the threshold, where S l (x i ) is the importance score of sentence i in layer l, Mask l (x i ) is the binary mask of sentence i in layer l.

[0070]

[0071] 3) During hard pruning, remove sentences with a binary mask of 0 and retain sentences with a mask of 1. Fine-tune the model parameters after hard pruning.

[0072] Finally, through this hierarchical and adaptive process, sentences that are not relevant to the source program will be pruned.

[0073] Table 3 Vertical cumulative attention scores

[0074]

[0075] Example 2

[0076] In one embodiment of the present disclosure, a high-order vulnerability detection system based on graph representation and sparse Transformer is provided, including:

[0077] A data acquisition module is used to acquire character stream data of module program source code;

[0078] A preprocessing module, used for performing lexical analysis on the character stream data, and saving variable information of global variables or user-controlled input variables in a persistent storage into a data table;

[0079] The feature extraction module is used to generate a code attribute graph for each source code. By searching the variable information stored persistently in the data table, the persistent storage data flow relationship between programs is generated. The persistent storage data flow relationship is input into the GNN module to learn the information of the nodes in the graph and obtain the embedding vector of the node.

[0080] The pruning module is used to input the node's embedding vector into the Transformer module with sparse attention, prune sentences layer by layer in the Transformer using a threshold-based pruning sentence pruning algorithm, perform attention sparsification, adaptively delete irrelevant sentences, and convert high-order vulnerabilities into low-order vulnerabilities.

[0081] Example 3

[0082] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the high-order vulnerability detection method based on graph representation and sparse Transformer is implemented.

[0083] Example 4

[0084] In one embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes and implements the high-order vulnerability detection method based on graph representation and sparse Transformer.

[0085] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0087] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A high-order vulnerability detection method based on graph representation and sparse Transformer, characterized in that: include: Get the character stream data of the module program source code; Performing lexical analysis on the character stream data, and saving variable information of global variables or user-controlled input variables in a persistent storage into a data table; Generate a code property graph for each source code, and generate persistent storage data flow relationships between programs by searching for variable information stored persistently in the data table; Input the persistent storage data flow relationship into the GNN module to learn the information of the nodes in the graph and obtain the embedding vector of the node; The embedding vector of the node is then input into the Transformer module with sparse attention. The threshold-based pruning sentence pruning algorithm is used to prune sentences layer by layer in the Transformer, and attention is sparsely processed to adaptively delete irrelevant sentences and convert high-order vulnerabilities into low-order vulnerabilities.

2. A high-order vulnerability detection method based on graph representation and sparse Transformer as claimed in claim 1, characterized in that: The character stream data is subjected to lexical analysis, and the source program character stream is scanned from left to right. During the scanning process, blank characters and comments are ignored and converted into meaningful word symbols according to predefined lexical rules, including keywords, identifiers, operators, separators and constants.

3. A high-order vulnerability detection method based on graph representation and sparse Transformer as claimed in claim 1, characterized in that: Create a new data table. When an identifier is scanned, compare it with the variables stored in the data table. If the data table already has information about the variable, just add it to the data table at the location where it appears. If the variable information does not exist in the data table, save the name of this identifier and the starting position in the data table.

4. The high-order vulnerability detection method based on graph representation and sparse Transformer as claimed in claim 1, characterized in that: Use the open source tool joern to generate code property graphs for each source code, including abstract syntax trees, control flow graphs, data flow graphs or program dependency graphs, and calling relationships between internal functions.

5. The high-order vulnerability detection method based on graph representation and sparse Transformer as claimed in claim 1, characterized in that: The obtained graph structure is used through node2vec to generate the initial vector of each node, and the adjacency matrix is ​​used to represent the graph structure, capturing the correlation between nodes and edges in the graph. The information of persistent storage variables is obtained by searching the data table, and the execution sequence relationship is obtained. A data flow edge is formed between every two variables. If there are security-sensitive operations, a taint propagation chain is formed. The taint propagation chain represents the taint propagation data flow of persistent storage variables, and the adjacency matrix of this graph is generated.

6. A high-order vulnerability detection method based on graph representation and sparse Transformer as claimed in claim 1, characterized in that: Input the graph structure into GNN, multiply the adjacency matrix by the initial vector matrix, and obtain the updated node vector representation after one layer of GNN message passing. Set up a 2-layer GNN, and set H l is the information matrix of the l-layer nodes, H l +1=W*H l , W is the adjacency matrix of the graph.

7. The high-order vulnerability detection method based on graph representation and sparse Transformer as claimed in claim 1, characterized in that: Each sentence of the source program is represented by a node in the graph structure. The node vector that passes through the two-layer GNN is input into the Transformer with sparse attention. Local attention is used in the sparse attention layer of the Transformer. Given a sliding window of a set window size, a vector represented by source code with n input sentences is input. The sliding window connects each unit covered by the window size in the current step with all units covered by the window in the next step to obtain the information of interest.

8. A high-order vulnerability detection system based on graph representation and sparse Transformer, characterized in that: include: A data acquisition module is used to acquire character stream data of module program source code; A preprocessing module, used for performing lexical analysis on the character stream data, and saving variable information of global variables or user-controlled input variables in a persistent storage into a data table; The feature extraction module is used to generate a code attribute graph for each source code, and to generate a persistent storage data flow relationship between programs by searching for variable information persistently stored in the data table; Input the persistent storage data flow relationship into the GNN module to learn the information of the nodes in the graph and obtain the embedding vector of the node; The pruning module is used to input the node's embedding vector into the Transformer module with sparse attention, prune sentences layer by layer in the Transformer using a threshold-based pruning sentence pruning algorithm, perform attention sparsification, adaptively delete irrelevant sentences, and convert high-order vulnerabilities into low-order vulnerabilities.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, a high-order vulnerability detection method based on graph representation and sparse Transformer as described in any one of claims 1-7 is implemented.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes a high-order vulnerability detection method based on graph representation and sparse Transformer as described in any one of claims 1-7.

Images