Code vulnerability detection method and device and electronic equipment

Abstract

The invention provides a code vulnerability detection method and device and electronic equipment, and relates to the technical field of computers. The method comprises the following steps: inputting an abstract syntax tree of a source code into a first network model, and outputting a first feature representation corresponding to syntax structure information of the source code; inputting the source code, the source code annotation and the variable sequence of the source code into a second network model, and outputting a second feature representation corresponding to semantic information of the source code; wherein a target hyper-parameter combination in the second network model is determined based on an adaptive grid search algorithm; and detecting whether the source code has vulnerabilities or not according to the first feature representation and the second feature representation. According to the scheme, the code vulnerability detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a code vulnerability detection method, device and electronic equipment. Background Art

[0002] Source code defect detection is a key link in the field of software security. Traditional detection methods often rely on a large amount of manual review, which is time-consuming and costly. In recent years, with the development of deep learning technology, especially the progress in the field of natural language processing, using deep learning models for code vulnerability detection has become a research hotspot.

[0003] In the related art, source code processing is usually regarded as special natural language processing, but this solution has low detection accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide a code vulnerability detection method, device and electronic device, which solve the problem of low accuracy of existing code vulnerability detection solutions.

[0005] To achieve the above object, an embodiment of the present invention provides a code vulnerability detection method, comprising:

[0006] Inputting an abstract syntax tree of the source code into a first network model, and outputting a first feature representation corresponding to the grammatical structure information of the source code;

[0007] Inputting the source code, source code annotations, and a variable sequence of the source code into a second network model, and outputting a second feature representation corresponding to the semantic information of the source code; wherein the target hyperparameter combination in the second network model is determined based on an adaptive grid search algorithm;

[0008] The source code is detected to determine whether there is a vulnerability according to the first feature representation and the second feature representation.

[0009] Optionally, the method for determining the target hyperparameter combination based on a grid search algorithm includes:

[0010] Repeat the following steps until the target hyperparameter combination is determined:

[0011] Performing model training on each grid point in the search grid to obtain a performance indicator for each grid point; wherein the search grid is a multidimensional space formed by a combination of different values ​​of multiple hyperparameters, and each dimension includes multiple values ​​of a hyperparameter;

[0012] The search grid is updated according to the performance indicators of adjacent grid points.

[0013] Optionally, updating the search grid according to the performance indicators of adjacent grid points includes:

[0014] Determining a performance gradient according to the performance indicators of adjacent grid points;

[0015] When the absolute value of the performance gradient is greater than or equal to a first preset gradient threshold, subdividing the grid area where the adjacent grid points are located;

[0016] When the absolute value of the performance gradient is greater than or equal to a second preset gradient threshold and less than a first preset gradient threshold, merging the adjacent grid points;

[0017] When the absolute value of the performance gradient is less than the second preset gradient threshold, the updating of the search grid is stopped.

[0018] Optionally, before inputting the abstract syntax tree of the source code into the first network model, the method further includes:

[0019] identifying adjacent code tokens in the abstract syntax tree;

[0020] Connecting the adjacent code tokens with natural code sequence NCS edges, and adding code metrics to each function node to obtain an enhanced abstract syntax tree;

[0021] The enhanced abstract syntax tree is input into the first network model, and the first feature representation is output.

[0022] Optionally, the code metric includes at least one of the following: function complexity, coupling, and cohesion.

[0023] Optionally, the first network model includes: a coding layer, a recursive neural network (RNN) layer and a pooling layer connected in sequence;

[0024] The encoding layer is used to convert a series of expression subtrees included in the abstract syntax tree into a series of first sub-feature representations;

[0025] The RNN layer processes the first sub-feature representation using a bidirectional GRU to obtain a series of second sub-feature representations;

[0026] By using the attention mechanism, the position information corresponding to the function nodes is integrated into the second sub-feature representation to obtain a series of third sub-feature representations;

[0027] The pooling layer is used to perform maximum pooling processing on a series of the third sub-feature representations to obtain the first feature representation.

[0028] To achieve the above object, an embodiment of the present invention provides a code vulnerability detection device, comprising:

[0029] A first processing module, configured to input an abstract syntax tree of a source code into a first network model, and output a first feature representation corresponding to grammatical structure information of the source code;

[0030] A second processing module is used to input the source code, source code annotations and variable sequences of the source code into a second network model, and output a second feature representation corresponding to the semantic information of the source code; wherein the target hyperparameter combination in the second network model is determined based on an adaptive grid search algorithm;

[0031] The third processing module is used to detect whether there is a vulnerability in the source code according to the first feature representation and the second feature representation.

[0032] To achieve the above object, an embodiment of the present invention provides an electronic device, including: a transceiver and a processor; the processor is used to:

[0033] Inputting an abstract syntax tree of the source code into a first network model, and outputting a first feature representation corresponding to the grammatical structure information of the source code;

[0034] Inputting the source code, source code annotations, and a variable sequence of the source code into a second network model, and outputting a second feature representation corresponding to the semantic information of the source code; wherein the target hyperparameter combination in the second network model is determined based on an adaptive grid search algorithm;

[0035] The source code is detected to determine whether there is a vulnerability according to the first feature representation and the second feature representation.

[0036] To achieve the above-mentioned purpose, an embodiment of the present invention provides an electronic device, comprising: a transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; when the processor executes the program or instruction, the code vulnerability detection method as described above is implemented.

[0037] To achieve the above object, an embodiment of the present invention provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps in the code vulnerability detection method as described above are implemented.

[0038] To achieve the above objective, an embodiment of the present invention provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the above-mentioned code vulnerability detection method.

[0039] The beneficial effects of the above technical solution of the present invention are as follows:

[0040] The method of the embodiment of the present invention inputs the abstract syntax tree of the source code into the first network model, and outputs the first feature representation corresponding to the syntax structure information of the source code; inputs the source code, source code annotations and the variable sequence of the source code into the second network model, and outputs the second feature representation corresponding to the semantic information of the source code; wherein the target hyperparameter combination in the second network model is determined based on the adaptive grid search algorithm; and detects whether the source code has a vulnerability based on the first feature representation and the second feature representation. In this way, in the process of code vulnerability detection, the syntax structure feature representation and semantic information feature representation of the source code are combined, and the two feature representations are combined, so that the detection method can take into account the syntax and semantic information of the code at the same time, thereby improving the accuracy of code vulnerability detection. Moreover, based on the target hyperparameter combination in the second network model determined by the adaptive grid search algorithm, it is realized by dynamically adjusting the grid division, which can improve the search efficiency and accuracy and reduce the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flowchart of a code vulnerability detection method according to an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of the architecture of a code vulnerability detection method according to an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of a processing flow of a code vulnerability detection method according to an embodiment of the present invention;

[0044] Figure 4 This is a second schematic diagram of the processing flow of the code vulnerability detection method according to an embodiment of the present invention;

[0045] Figure 5 This is a third schematic diagram of a processing flow of a code vulnerability detection method according to an embodiment of the present invention;

[0046] Figure 6 This is a fourth schematic diagram of a processing flow of a code vulnerability detection method according to an embodiment of the present invention;

[0047] Figure 7 This is a fifth schematic diagram of the processing flow of the code vulnerability detection method according to an embodiment of the present invention;

[0048] Figure 8 This is a sixth schematic diagram of the processing flow of the code vulnerability detection method according to an embodiment of the present invention;

[0049] Fig. 9 This is the seventh schematic diagram of the processing flow of the code vulnerability detection method according to the embodiment of the present invention;

[0050] Fig.10 is a structural block diagram of a code vulnerability detection device according to an embodiment of the present invention;

[0051] Fig.11 A schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention;

[0052] Fig.12 The figure is a schematic diagram of the hardware structure of an electronic device according to another embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0054] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present invention. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0055] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0056] In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0057] like Figure 1 As shown, a code vulnerability detection method according to an embodiment of the present invention comprises the following steps:

[0058] Step 101: input an abstract syntax tree of a source code into a first network model, and output a first feature representation corresponding to grammatical structure information of the source code.

[0059] In the specific implementation, the source code is parsed to generate an Abstract Syntax Tree (AST). As an intermediate representation of the source code, the AST displays lexical and grammatical information through a tree structure. Its internal nodes represent operations, and leaf nodes represent operands. The hierarchical relationship clearly reflects the branching and nested structure of the code. In addition, the AST also contains the control and data dependency information of the program.

[0060] It should be noted that in the abstract syntax tree, operations usually refer to various grammatical constructs and operators in programming languages. These operations may include: arithmetic operations, logical operations, comparison operations, assignment operations, control structures, function calls, object and class operations, etc. Operands refer to variables, constants, function names, etc.

[0061] The first network model is a neural network model. Optionally, the first network model is ASTPANN. The model is implemented based on ASTNN. After the source code is converted into AST, the structural information of the code is extracted by embedding AST nodes.

[0062] Step 102: Input the source code, source code annotations and variable sequence of the source code into a second network model, and output a second feature representation corresponding to the semantic information of the source code; wherein the target hyperparameter combination in the second network model is determined based on an adaptive grid search algorithm.

[0063] Optionally, the second network model is GraphCodeBERT, which is a model for graph neural network and code representation learning. Before the source code is represented by the pre-trained GraphCodeBERT model, the hyperparameters are parameters that need to be set before the training model, such as learning rate, batch size, number of layers, etc. GraphCodeBert represents source code information based on data flow, and this representation method can capture the intrinsic structure and semantic information of the code, thereby enhancing the understanding of the code. Therefore, the complexity and particularity of the model itself also make the hyperparameter tuning of the model more complicated. Different hyperparameter combinations may have a significant impact on the performance of the model, so an efficient search method is needed to find the optimal hyperparameter combination. To improve the search efficiency of the hyperparameter combination, the present application provides a method for determining the target hyperparameter combination based on an adaptive grid search algorithm, which can dynamically adjust the division of the grid according to the changes and needs of the solution, can improve the search efficiency and accuracy, and reduce the amount of calculation when processing multidimensional hyperparameter space.

[0064] like Figure 3 In the method of obtaining the variable sequence of the source code, the variable sequence is extracted from AST.

[0065] Step 103: Detect whether the source code has a vulnerability based on the first feature representation and the second feature representation.

[0066] When implementing it specifically, Figure 2In the above, the source code is preprocessed to obtain the grammatical structure feature representation (i.e., the first feature representation) and the semantic information feature representation (i.e., the second feature representation) of the source code; the grammatical structure feature representation and the semantic information feature representation of the source code are merged into a matrix to generate the final vector representation H corresponding to all source code functions, and the formula is: Among them, CONCAT is used to connect vectors and vector r, is the first feature representation, and r is the second feature representation; secondly, the vector H is input into the classification module and the detection label is output, where 0 indicates no vulnerability and 1 indicates a vulnerability.

[0067] like Figure 4 In the classification module, the classification module includes: a convolutional layer, a pooling layer, and a fully connected layer connected in sequence; wherein the classification process includes: sequentially using the convolutional layer and the pooling layer to perform convolution and pooling operations on the vector H to extract node features. The convolution and pooling operations can capture the local pattern and structural information between nodes, thereby generating a more representative feature vector. In order to detect vulnerabilities, a fully connected network is used as a classifier. Specifically, after the convolution and pooling operations, the feature vectors of all nodes are aggregated and fed into the fully connected neural network for classification; after being processed by the activation function, the network outputs the final prediction label, where 0 indicates no vulnerability and 1 indicates the presence of a vulnerability. The classifier consists of three parts: an input layer, a hidden layer, and an output layer, and can effectively process the code representation after feature extraction.

[0068] In the above embodiment, the grammatical structure feature representation and the semantic information feature representation are combined to perform code vulnerability detection. The combination of the two feature representations enables the detection method to take into account both the grammatical and semantic information of the code, thereby improving the accuracy of the detection. Moreover, the target hyperparameter combination in the second network model determined by the adaptive grid search algorithm can improve the search efficiency and accuracy and reduce the amount of calculation by dynamically adjusting the grid division.

[0069] In some embodiments, the first network model includes: a coding layer, a recursive neural network RNN ​​layer, and a pooling layer connected in sequence;

[0070] The encoding layer is used to convert a series of expression subtrees included in the abstract syntax tree into a series of first sub-feature representations;

[0071] The RNN layer processes the first sub-feature representation using a bidirectional GRU to obtain a series of second sub-feature representations;

[0072] By using the attention mechanism, the position information corresponding to the function nodes is integrated into the second sub-feature representation to obtain a series of third sub-feature representations;

[0073] The pooling layer is used to perform maximum pooling processing on a series of the third sub-feature representations to obtain the first feature representation.

[0074] Among them, the bidirectional GRU (Gated Recurrent Unit) is a variant of the recurrent neural network (RNN), which is often used to process sequence data, especially in natural language processing, time series prediction and other tasks. The main feature of the bidirectional GRU is that it considers both the forward and reverse information of the sequence at the same time, so as to better capture the context information.

[0075] Exemplarily, the first network model is ASTPANN. Figure 5 As shown, the first network model includes: a coding layer, a recursive neural network RNN ​​layer and a pooling layer connected in sequence. The processing process of each network layer is introduced below;

[0076] (1) Coding layer

[0077] The workflow of the encoding layer is as follows Figure 6 Through this process, these statement trees can be effectively processed, providing a basis for subsequent code understanding and vulnerability detection.

[0078] like Figure 6 In the ASTPANN network, before entering the coding layer, the pre-order traversal algorithm is first used to divide the pre-processed enhanced abstract syntax tree AST' into a series of expression subtrees (ST-trees). These subtrees are then used as input to the coding layer to capture and transform the grammatical structure information of the code into vector representation.

[0079] For each sentence tree, one-hot encoding is used to convert node n into the corresponding vector representation X n These vectors are based on the pre-trained embedding parameters W e ∈R |V|*d Initialize, the vocabulary vector representation V of non-leaf node n n as follows:

[0080]

[0081] Among them, X n is the vector representation of node n, |V| represents the size of the vocabulary, that is, the total number of all possible node types, and d represents the dimension of the embedding vector. W e The transposed matrix of .

[0082] Recursively update the vector corresponding to each node in the ST-tree, and the vector expression of node n is h:

[0083]

[0084] Among them, W n ∈R d*k is a weight matrix with encoding dimension k, whose dimensions are d rows and k columns, and R represents a set of real numbers; W n The transposed matrix of n is the bias vector of node n; C is the number of child nodes of node n; h represents the hidden state; σ is the activation function, which can generally use the tanh function; h i It refers to the hidden state of each child node i.

[0085] In order to obtain the most important features in the node vector, after completing the vector update of each node, use the following formula to update all node vectors h of the ST-tree i Perform maximum pooling to obtain the final vector representation of the ST-tree.

[0086] e t =[max(h i,1 ),…,max(h i,k )],i=1,…,N;

[0087] Where N refers to the total number of nodes in the ST-tree, i represents the number of each individual node in the tree, ranging from 1 to N, and k refers to the number of dimensions of each node vector, that is, for each node, considering its feature vector consists of k elements, h i,k Represents the k-th eigenvalue of the i-th node.

[0088] Next, each ST-tree is encoded into a vector by the sentence encoder using a recursive encoder, denoted as e1, ..., e t Finally, based on the order of the sentence vectors, the hidden states of the Bi-GRU are aggregated into a single vector through a max pooling operation as the overall representation of the code snippet.

[0089] (2) RNN layer

[0090] In order to further capture the hierarchical information between sentences, this layer uses a bidirectional GRU, which has the ability to transmit information in both forward and backward directions. The structure of this layer is shown in the figure below: Figure 7As shown. As an improved version of RNN, GRU effectively solves the long-term memory and gradient vanishing problems faced by RNN when processing long sequences, while maintaining high computational efficiency. Compared with the traditional unidirectional recurrent neural network, the bidirectional GRU structure can more comprehensively capture and understand the contextual information of text data, thereby improving the expressiveness and performance of the model. The calculation process of the bidirectional GRU includes forward and backward recurrent units, each of which updates and transmits information through a specific gating mechanism. In the previous encoding layer, each ST tree is encoded into a vector by the sentence encoder, denoted as e1,...,e t . Then the GRU calculation expression is as follows:

[0091]

[0092] The direction of the arrows above all vectors indicates the direction of bidirectional gating, GRU indicates gated recurrent unit network, T refers to the length of the entire input sequence, t refers to the time step or the position index in the sequence, and the value range is from 1 to T. and represent the hidden states of the forward GRU and the backward GRU at time step t, respectively, and h t Refers to the final hidden state at time step t, which is obtained by concatenating the forward hidden state and the backward hidden state.

[0093] The code syntax tree is a hierarchical representation of the source code, and its structural information is crucial for accurately understanding the syntax of the source code. Position information plays an important role in it. It reflects the relative position of the node in the tree and helps the model capture the hierarchical nature of the grammatical structure. Therefore, the introduction of the position attention mechanism (PA) can help the model consider the relative position of the node when processing the syntax tree, and enhance the model's perception of the syntax tree structure. Its calculation process includes:

[0094] (1) Query, key and value calculation

[0095] For each node i, given its bidirectional GRU output position embedding vector h i ,So Its query q i , key k i Sum value v i They are calculated as:

[0096]

[0097] Among them, W q , W k and W v is a learnable weight matrix.

[0098] It should be noted that when processing sequences, each node i usually corresponds to a time step t. Therefore, for node i, its corresponding hidden state can be expressed as h i That is h i In fact, h t , when i = t. This means h i It contains the local features of node i and its context information in the entire sequence.

[0099] (2) Position attention score calculation

[0100] For each node i, calculate the position attention score e between it and all other nodes j ij :

[0101]

[0102] Where, · represents the dot product operation, D k is the key vector k j The dimension is used to scale the dot product to prevent the gradient from disappearing, q i is the query vector.

[0103] (3) Normalize the position attention weight. Normalize the attention scores of all nodes to obtain the position attention weight:

[0104]

[0105] Where N is the total number of nodes and exp represents the exponential function. The purpose of this is to ensure that for any node i, the sum of the attention weights of all nodes j related to it is 1, e ik represents the unnormalized attention score between node i and node k, where k traverses all nodes.

[0106] Weighted fusion generates a new node representation. The output of the bidirectional GRU is weighted fused using the normalized position attention weights to obtain a new node representation h i ′ :

[0107]

[0108] Where j is the index variable used to traverse all nodes from 1 to N; v j Represents the value vector of node j. That is, the values ​​Vj of each node are weighted and summed according to their importance relative to node i.

[0109] Among them, h i ′ It combines the semantic information of node i and its position information in the syntax tree.

[0110] That is, the position attention mechanism enables the model to better understand the structure of the syntax tree by calculating an attention weight for each node based on its position. When calculating the attention weight, the semantic information of the node is also taken into account (through the hidden state h i ) and position attention weights. This combination enables the model to more accurately capture the key structures and patterns in the code, thereby improving the performance of tasks such as code understanding and vulnerability detection.

[0111] By introducing the positional attention mechanism, ASTPANN is able to not only process the semantic content of nodes but also fully consider their relative positions in the syntax tree, which is crucial for accurately understanding the syntax and structure of source code.

[0112] (3) Pooling layer

[0113] This layer uses the Max Pooling layer as the pooling model, which aims to convert vectors of different sizes into vector outputs of uniform size. Since different codes may contain different numbers of statements, the sizes of the generated vectors will also vary. Through the Max Pooling process, ASTPANN can effectively solve this problem and ensure the consistency and convenience of subsequent processing.

[0114] The implicit state list output by the previous layer position attention is max-pooled to obtain the final vector representation r of the abstract syntax tree and output it.

[0115] r∈R 2m ;

[0116] Among them, R represents the real number set and m is the vector dimension.

[0117] It should be pointed out that, unlike the existing methods that directly model the entire abstract syntax tree (AST), ASTPANN uses supervised learning to split the huge AST into a series of smaller statement trees. By encoding the statement tree into a vector and combining the bidirectional RNN model to exploit the naturalness of the statement, ASTPANN can generate a more accurate vector representation of code snippets. This model helps improve the accuracy of code understanding and vulnerability detection.

[0118] Taking the second network model as GraphCodeBERT as an example, the implementation process of inputting the source code, source code annotations and the variable sequence of the source code into the second network model and outputting the second feature representation corresponding to the semantic information of the source code in the above step 102 is introduced below.

[0119] First of all, it should be noted that the GraphCodeBERT pre-training model uses the semantic level information of the code, especially the data flow, to learn the representation of the code. Its structure is as follows Figure 8 As shown, it includes an input embedding layer, a Transfomer layer, and an output layer connected in sequence. Unlike many works that only treat source code as a sequence of characters for training, GraphCodeBERT takes into account the structural information inside the code, which is crucial for accurately understanding the semantics of the code. Since variable naming may not always reflect its semantics, it is difficult to infer its true meaning from the variable name alone. Therefore, it is an effective method to use the dependencies and data flow structures in the code to understand the semantics of variables. Among them, a data flow graph is a directed graph in which nodes represent variables and edges indicate the direction of "value flow". If a points to b, it means that b depends on the value of a. GraphCodeBERT uses this dependency to build its pre-trained model.

[0120] Secondly, the GraphCodeBert model uses Bert and a multi-layer bidirectional Transfomer as the basis, adding source code annotations and data flows as the input of the model.

[0121] The variables in the variable sequence usually correspond to the nodes in the data flow graph, and the data transfer relationship between the variables corresponds to the edges in the data flow graph. Each sequence element is a node in the data flow graph. The data flow graph is a graphical tool that represents the flow and change of data in the program. By extracting the variable sequence and constructing a data flow graph, you can clearly see the data transfer and change process in the code, making it easier to find potential vulnerabilities and errors.

[0122] Specifically, the processing of the GraphCodeBert model includes:

[0123] (1) The input consists of three parts: source code C, comments W (which identify the meaning of the source code fragment), and variable sequence V.

[0124] Among them, GraphCodeBERT connects these three sequences to obtain sequence X and takes sequence X as input.

[0125] X={[CLS],W,[SEP],C,[SEP],V};

[0126] Where C = {c1, c2, …, c n} is the source code, W={w1,w2,…,w n} is a comment, V = {v1,v2,…,v n} is a variable sequence, [CLS] is a special symbol for the start of input, and [SEP] is a special symbol used to separate two types of input.

[0127] (2) Convert the sequence X into an input vector H 0 , in the input vector H 0 Apply N Transformer layers to generate a context-rich representation H n .

[0128] H n =transformer n (H n-1 ),n∈[1,N];

[0129] Among them, H n is |X|×d h dimension, |X| is the length of the input sequence, d h is the dimension of the vector at each position, and h is the number of heads in the multi-head self-attention mechanism.

[0130] It should be noted that for an input sequence of length |X|, after being processed by n layers of Transformer, the element at each position is converted into a d h dimensional vectors, which together form the output matrix H n . And h is used to describe the number of attention heads working in parallel in the multi-head self-attention mechanism.

[0131] Each Transformer layer contains a Transformer module with the same structure, and the input H of the nth layer n-1 The multi-head self-attention mechanism is used in the above. This mechanism is connected with the feed-forward layer and together they form the core component of the Transformer layer. For the output of the nth Transformer layer It can be calculated by the following formula:

[0132]

[0133]

[0134] Among them, u is the number of attention heads, d k is the dimension of the head, is the model parameter, M is the mask matrix; Q i , K i , V i Represents the query, key, and value matrices of the i-th head, head i The attention output of the i-th head, T represents the transposed matrix, It is the direct output of the multi-head self-attention mechanism of the n-th layer Transformer.

[0135] In this way, for each function-level source code sample sequence, the pre-trained GraphCodeBERT encoder will generate corresponding vector matrices, which encapsulate the semantic meaning of the code and provide the basic conditions for subsequent code understanding and vulnerability classification.

[0136] In the above example, the model is based on a multi-layer bidirectional Transformer, and incorporates source code annotations and data flow information as input. Specifically, the input of the model consists of three parts: source code C, annotations W (explaining the meaning of the source code snippet), and variable sequence V. These input sequences are connected together to form the final input of the model. In this way, GraphCodeBERT is able to combine the text information of the code, annotations, and the data flow structure of the variable sequence to learn the representation of the code more comprehensively. This helps improve the performance of the model in related tasks such as code understanding and vulnerability detection.

[0137] In some embodiments, the method for determining the target hyperparameter combination based on an adaptive grid search algorithm includes:

[0138] Repeat the following steps until the target hyperparameter combination is determined:

[0139] Performing model training on each grid point in the search grid to obtain a performance indicator for each grid point; wherein the search grid is a multidimensional space formed by a combination of different values ​​of multiple hyperparameters, and each dimension includes multiple values ​​of a hyperparameter;

[0140] The search grid is updated according to the performance indicators of adjacent grid points.

[0141] In the specific implementation, first, determine the initial search range and initial grid size of the hyperparameters based on preliminary experiments. For each hyperparameter, set an initial search range [a, b] and an initial number of grid points N; second, train the model on each grid point and use the validation set to evaluate its performance, and obtain the performance index P corresponding to each grid point. i , where i is the index of the grid point; again, the search grid is updated according to the performance indicators of the adjacent grid points until the stop update condition is reached, and then the search grid is stopped from being updated; finally, a set of hyperparameter values ​​with the highest performance indicators is selected from the latest search grid as the target hyperparameter combination.

[0142] In some embodiments, updating the search grid according to the performance indicators of adjacent grid points includes:

[0143] Determining a performance gradient according to the performance indicators of adjacent grid points;

[0144] When the absolute value of the performance gradient is greater than or equal to a first preset gradient threshold, subdividing the grid area where the adjacent grid points are located;

[0145] When the absolute value of the performance gradient is greater than or equal to a second preset gradient threshold and less than a first preset gradient threshold, merging the adjacent grid points;

[0146] When the absolute value of the performance gradient is less than the second preset gradient threshold, the updating of the search grid is stopped.

[0147] For example, Fig. 9 In the method for determining the target hyperparameter combination based on the adaptive grid search algorithm, the method includes:

[0148] Step 901: Initialize the grid.

[0149] Determine the initial search range and initial grid size of the hyperparameters based on preliminary experiments. For each hyperparameter, set an initial search range [a, b] and an initial number of grid points N.

[0150] Step 902: Performance evaluation.

[0151] Secondly, the model is trained on each grid point and its performance is evaluated using the validation set to obtain the performance index P corresponding to each grid point. i , where i is the index of the grid point.

[0152] Step 903: Calculate the performance gradient.

[0153] For each hyperparameter dimension, the performance gradient can be calculated as the difference in performance of adjacent grid points divided by the difference in hyperparameter values: ΔP i =(P i+1 -P i ) / (x i+1 -x i ), where P i and P i+1 is the performance index of the adjacent grid points, x i and x i+1 is the hyperparameter value of the adjacent grid point, and i is the grid point index.

[0154] Step 904: adaptively adjust the grid.

[0155] A first preset gradient threshold is set. When the absolute value of the performance gradient exceeds the first preset gradient threshold, the grid is subdivided in the area; when the performance gradient is lower than the first preset gradient threshold, adjacent grid points are merged.

[0156] Among them, the algorithm for adaptively adjusting the search grid size includes:

[0157] S new =S old ×(1 / (|ΔP i |+ε)); where S old is the hyperparameter value of the grid point i before updating, S new is the hyperparameter value of the updated grid point i; ε is a positive number less than the preset value to prevent division by zero. In this algorithm, the grid size is adjusted according to the inverse of the performance gradient, so that the grid is divided finer in the area where the performance changes rapidly and coarser in the area where the change is slow.

[0158] Step 905 , determine whether the performance gradient is less than a threshold (ie, a second preset gradient threshold); if so, end; if not, execute step 902 .

[0159] In the above embodiment, the grid division can be adjusted according to the calculated performance gradient. In areas with large performance gradients (i.e., areas where performance changes faster), the density of the grid is increased for more detailed searches; in areas with small performance gradients, the density of the grid is reduced to save computing resources; until the stop condition is met (the performance improvement is less than the second preset gradient threshold). In each iteration, the grid division is adjusted according to the previous search results, and the optimal hyperparameter combination, i.e., the target hyperparameter combination, is gradually approached. In this way, the search focus is concentrated on the hyperparameter area with better performance, the efficiency of the hyperparameter search is improved, and the optimal hyperparameter combination is better approached.

[0160] Moreover, by fine-tuning the task-specific hyperparameters of the GraphCodeBERT model through the algorithm provided in the embodiment of the present application, it is possible to simplify the model training process while ensuring performance, thus providing strong support for code understanding and vulnerability detection tasks.

[0161] In some embodiments, before inputting the abstract syntax tree of the source code into the first network model, the method further includes:

[0162] identifying adjacent code tokens in the abstract syntax tree;

[0163] Connecting the adjacent code tokens with natural code sequence NCS edges, and adding code metrics to each function node to obtain an enhanced abstract syntax tree;

[0164] The enhanced abstract syntax tree is input into the first network model, and the first feature representation is output.

[0165] Optionally, the code metric includes at least one of the following: function complexity, coupling, and cohesion.

[0166] When implementing it specifically, Figure 3In the AST, adjacent code tokens in the abstract syntax tree can be identified by traversing the AST. After adjacent code tokens are identified, they are connected using natural code sequence (NCS) edges; further, attributes such as complexity, coupling, and cohesion are added to each function node to obtain an enhanced AST; after the code metrics are calculated, they are implemented by adding additional attributes or metadata to the nodes of the AST. For example, a "metrics" attribute can be added to each function node, which contains the measurement results of the complexity, coupling, and cohesion of the function, and finally an enhanced abstract syntax tree AST' is obtained.

[0167] Code metrics are a series of quantitative indicators used to evaluate the quality, complexity, and other characteristics of code. By calculating these indicators, we can have a more comprehensive understanding of the structure and characteristics of the code. Metrics include but are not limited to function complexity, coupling, cohesion, etc., which are obtained by analyzing code structure, call relationships, data flow, etc.

[0168] Specifically, the code metrics are obtained in the following ways:

[0169] ① Function complexity: Use metrics such as cyclomatic complexity to evaluate the complexity within each function. Code complexity is evaluated by analyzing the control flow nodes in the AST (such as conditional statements, loop statements, etc.) and counting the number of different paths.

[0170] ② Coupling degree: Evaluate the degree of interdependence between modules in the code. This includes the calling relationship between modules, data transfer methods, etc. The coupling degree of the code is evaluated by analyzing the parameter transfer between functions, the use of global variables, etc.

[0171] ③ Cohesion: Evaluate the degree of functional connection between the various parts of the code. Highly cohesive code means that related functions are organized together, while low cohesion may mean that the code has scattered functions or unclear responsibilities. Cohesion can be evaluated by analyzing the functional organization within the function, the division of code modules, etc.

[0172] Among them, adjacent code markers refer to code elements that are logically adjacent and have a specific association in the source code. These code elements may be variables, function calls, control statements, etc., which can together express a complete logical unit. By identifying adjacent code markers, you can better understand the logical relationship and dependency of the code.

[0173] Among them, the NCS edge is a special edge used to represent the natural order and dependencies between code tokens. By adding NCS edges, the logic presented by the source code sequence can be retained, providing rich contextual information for subsequent vulnerability detection. Therefore, when code elements with adjacent code tokens are identified, it uses NCS edges to connect these elements to build an enhanced AST containing more contextual information. This enhanced AST will be used by subsequent grammatical structure feature representation (i.e., the first feature representation) and semantic information feature representation (the second feature representation) for more accurate code vulnerability detection.

[0174] In the above embodiment, a natural code sequence (NCS) is added to the abstract syntax tree AST obtained by parsing, which can retain the programming logic contained in the source code sequence, so that the analysis process can accurately capture the key features of the code; at the same time, code metrics are introduced at each function node to help the model better understand and evaluate the characteristics of the code.

[0175] like Fig.10 As shown, a code vulnerability detection device according to an embodiment of the present invention includes:

[0176] The first processing module 1001 is used to input the abstract syntax tree of the source code into the first network model, and output the first feature representation corresponding to the grammatical structure information of the source code;

[0177] A second processing module 1002 is used to input the source code, source code annotations and variable sequences of the source code into a second network model, and output a second feature representation corresponding to the semantic information of the source code; wherein the target hyperparameter combination in the second network model is determined based on an adaptive grid search algorithm;

[0178] The third processing module 1003 is used to detect whether there is a vulnerability in the source code according to the first feature representation and the second feature representation.

[0179] Optionally, the apparatus 1000 further includes:

[0180] The fourth processing module is used to repeatedly perform the following steps until the target hyperparameter combination is determined:

[0181] Performing model training on each grid point in the search grid to obtain a performance indicator for each grid point; wherein the search grid is a multidimensional space formed by a combination of different values ​​of multiple hyperparameters, and each dimension includes multiple values ​​of a hyperparameter;

[0182] The search grid is updated according to the performance indicators of adjacent grid points.

[0183] Optionally, the fourth processing module is specifically configured to:

[0184] Determining a performance gradient according to the performance indicators of adjacent grid points;

[0185] When the absolute value of the performance gradient is greater than or equal to a first preset gradient threshold, subdividing the grid area where the adjacent grid points are located;

[0186] When the absolute value of the performance gradient is greater than or equal to a second preset gradient threshold and less than a first preset gradient threshold, merging the adjacent grid points;

[0187] When the absolute value of the performance gradient is less than the second preset gradient threshold, the updating of the search grid is stopped.

[0188] Optionally, the first processing module 1001 is further used for:

[0189] identifying adjacent code tokens in the abstract syntax tree;

[0190] Connecting the adjacent code tokens with natural code sequence NCS edges, and adding code metrics to each function node to obtain an enhanced abstract syntax tree;

[0191] The enhanced abstract syntax tree is input into the first network model, and the first feature representation is output.

[0192] Optionally, the code metric includes at least one of the following: function complexity, coupling, and cohesion.

[0193] Optionally, the first network model includes: a coding layer, a recursive neural network (RNN) layer and a pooling layer connected in sequence;

[0194] The encoding layer is used to convert a series of expression subtrees included in the abstract syntax tree into a series of first sub-feature representations;

[0195] The RNN layer processes the first sub-feature representation using a bidirectional GRU to obtain a series of second sub-feature representations;

[0196] By using the attention mechanism, the position information corresponding to the function nodes is integrated into the second sub-feature representation to obtain a series of third sub-feature representations;

[0197] The pooling layer is used to perform maximum pooling processing on a series of the third sub-feature representations to obtain the first feature representation.

[0198] like Fig.11 As shown, an electronic device 1100 according to an embodiment of the present invention includes a processor 1110 and a transceiver 1120, wherein the processor 1110 is used to:

[0199] Inputting an abstract syntax tree of the source code into a first network model, and outputting a first feature representation corresponding to the grammatical structure information of the source code;

[0200] Inputting the source code, source code annotations, and a variable sequence of the source code into a second network model, and outputting a second feature representation corresponding to the semantic information of the source code; wherein the target hyperparameter combination in the second network model is determined based on an adaptive grid search algorithm;

[0201] The source code is detected to determine whether there is a vulnerability according to the first feature representation and the second feature representation.

[0202] Optionally, the processor 1110 is further configured to:

[0203] Repeat the following steps until the target hyperparameter combination is determined:

[0204] Performing model training on each grid point in the search grid to obtain a performance indicator for each grid point; wherein the search grid is a multidimensional space formed by a combination of different values ​​of multiple hyperparameters, and each dimension includes multiple values ​​of a hyperparameter;

[0205] The search grid is updated according to the performance indicators of adjacent grid points.

[0206] Optionally, the processor 1110 is specifically configured to:

[0207] Determining a performance gradient according to the performance indicators of adjacent grid points;

[0208] When the absolute value of the performance gradient is greater than or equal to a first preset gradient threshold, subdividing the grid area where the adjacent grid points are located;

[0209] When the absolute value of the performance gradient is greater than or equal to a second preset gradient threshold and less than a first preset gradient threshold, merging the adjacent grid points;

[0210] When the absolute value of the performance gradient is less than the second preset gradient threshold, the updating of the search grid is stopped.

[0211] Optionally, the processor 1110 is further configured to:

[0212] identifying adjacent code tokens in the abstract syntax tree;

[0213] Connecting the adjacent code tokens with natural code sequence NCS edges, and adding code metrics to each function node to obtain an enhanced abstract syntax tree;

[0214] The enhanced abstract syntax tree is input into the first network model, and the first feature representation is output.

[0215] Optionally, the code metric includes at least one of the following: function complexity, coupling, and cohesion.

[0216] Optionally, the first network model includes: a coding layer, a recursive neural network (RNN) layer and a pooling layer connected in sequence;

[0217] The encoding layer is used to convert a series of expression subtrees included in the abstract syntax tree into a series of first sub-feature representations;

[0218] The RNN layer processes the first sub-feature representation using a bidirectional GRU to obtain a series of second sub-feature representations;

[0219] By using the attention mechanism, the position information corresponding to the function nodes is integrated into the second sub-feature representation to obtain a series of third sub-feature representations;

[0220] The pooling layer is used to perform maximum pooling processing on a series of the third sub-feature representations to obtain the first feature representation.

[0221] The electronic device of this embodiment combines the grammatical structure feature representation and the semantic information feature representation of the source code in the process of code vulnerability detection, and combines the two feature representations so that the detection method can take into account the grammatical and semantic information of the code at the same time, thereby improving the accuracy of detection. Moreover, based on the target hyperparameter combination in the second network model determined by the adaptive grid search algorithm, the grid division can be dynamically adjusted to improve the search efficiency and accuracy and reduce the amount of calculation.

[0222] An electronic device according to another embodiment of the present invention, such as Fig.12 As shown, it includes a transceiver 1210, a processor 1200, a memory 1220, and a program or instruction stored in the memory 1220 and executable on the processor 1200; when the processor 1300 executes the program or instruction, the above-mentioned method for code vulnerability detection is implemented.

[0223] The transceiver 1210 is used to receive and send data under the control of the processor 1200 .

[0224] Among them, Fig.12In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 1200 and various circuits of memory represented by memory 1220 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 1210 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. For different user devices, the user interface 1230 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0225] The processor 1200 is responsible for managing the bus architecture and general processing, and the memory 1220 can store data used by the processor 1300 when performing operations.

[0226] A computer-readable storage medium according to an embodiment of the present invention stores a program or instruction thereon, and when the program or instruction is executed by a processor, the steps in the code vulnerability detection method described above are implemented, and the same technical effect can be achieved. To avoid repetition, it is not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0227] To achieve the above objective, an embodiment of the present invention provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the above code vulnerability detection method.

[0228] It should be further explained that the terminals described in this specification include but are not limited to smart phones, tablet computers, etc., and many of the functional components described are called modules in order to more particularly emphasize the independence of their implementation methods.

[0229] In the embodiment of the present invention, module can be implemented with software so that it can be executed by various types of processors. For example, an executable code module of an identification can include one or more physical or logical blocks of computer instructions, for example, it can be constructed as an object, process or function. Nevertheless, the executable code of the identified module does not need to be physically located together, but can include different instructions stored in different positions, and when these instructions are logically combined together, it constitutes a module and realizes the specified purpose of the module.

[0230] In fact, executable code module can be a single instruction or many instructions, and can even be distributed on a plurality of different code segments, distributed among different programs, and distributed across a plurality of memory devices. Similarly, operating data can be identified in the module, and can be implemented and organized in the data structure of any appropriate type according to any appropriate form. The operating data can be collected as a single data set, or can be distributed in different locations (including on different storage devices), and can only be present on a system or network as an electronic signal at least in part.

[0231] When a module can be implemented by software, considering the level of existing hardware technology, a person skilled in the art can build a corresponding hardware circuit to implement the corresponding function of the module that can be implemented by software without considering the cost. The hardware circuit includes a conventional very large scale integration (VLSI) circuit or gate array and existing semiconductors such as logic chips, transistors, or other discrete components. The module can also be implemented by a programmable hardware device, such as a field programmable gate array, a programmable array logic, a programmable logic device, etc.

[0232] The above exemplary embodiments are described with reference to the accompanying drawings, and many different forms and embodiments are feasible without departing from the spirit and teachings of the present invention. Therefore, the present invention should not be constructed as a limitation of the exemplary embodiments proposed herein. More specifically, these exemplary embodiments are provided so that the present invention will be perfect and complete, and the scope of the present invention will be conveyed to those who are familiar with the technology. In these figures, the component sizes and relative sizes may be exaggerated for clarity. The terms used here are only based on the purpose of describing specific exemplary embodiments and are not intended to be limiting. As used herein, unless the text clearly indicates otherwise, the singular forms "one", "an" and "the" are intended to include these multiple forms. It will be further understood that the terms "including" and / or "comprising" when used in this specification indicate the presence of the features, integers, steps, operations, components and / or components, but do not exclude the presence or increase of one or more other features, integers, steps, operations, components, components and / or their groups. Unless otherwise indicated, when stated, a range of values ​​includes the upper and lower limits of that range and any subranges therebetween.

[0233] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A code vulnerability detection method, characterized in that: include: Inputting an abstract syntax tree of the source code into a first network model, and outputting a first feature representation corresponding to the grammatical structure information of the source code; Inputting the source code, source code annotations, and a variable sequence of the source code into a second network model, and outputting a second feature representation corresponding to the semantic information of the source code; wherein the target hyperparameter combination in the second network model is determined based on an adaptive grid search algorithm; The source code is detected to determine whether there is a vulnerability according to the first feature representation and the second feature representation.

2. The code vulnerability detection method according to claim 1, characterized in that: The method for determining the target hyperparameter combination based on an adaptive grid search algorithm includes: Repeat the following steps until the target hyperparameter combination is determined: Performing model training on each grid point in the search grid to obtain a performance indicator for each grid point; wherein the search grid is a multidimensional space formed by a combination of different values ​​of multiple hyperparameters, and each dimension includes multiple values ​​of a hyperparameter; The search grid is updated according to the performance indicators of adjacent grid points.

3. The code vulnerability detection method according to claim 2, characterized in that: The updating of the search grid according to the performance indicators of adjacent grid points comprises: Determining a performance gradient according to the performance indicators of adjacent grid points; When the absolute value of the performance gradient is greater than or equal to a first preset gradient threshold, subdividing the grid area where the adjacent grid points are located; When the absolute value of the performance gradient is greater than or equal to a second preset gradient threshold and less than a first preset gradient threshold, merging the adjacent grid points; When the absolute value of the performance gradient is less than the second preset gradient threshold, the updating of the search grid is stopped.

4. The code vulnerability detection method according to claim 1, characterized in that: Before inputting the abstract syntax tree of the source code into the first network model, the method further includes: identifying adjacent code tokens in the abstract syntax tree; Connecting the adjacent code tokens with natural code sequence NCS edges, and adding code metrics to each function node to obtain an enhanced abstract syntax tree; The enhanced abstract syntax tree is input into the first network model, and the first feature representation is output.

5. The code vulnerability detection method according to claim 4, characterized in that: The code metric includes at least one of the following: function complexity, coupling, and cohesion.

6. The code vulnerability detection method according to claim 1, characterized in that: The first network model includes: a coding layer, a recursive neural network RNN ​​layer and a pooling layer connected in sequence; The encoding layer is used to convert a series of expression subtrees included in the abstract syntax tree into a series of first sub-feature representations; The RNN layer processes the first sub-feature representation using a bidirectional GRU to obtain a series of second sub-feature representations; By using the attention mechanism, the position information corresponding to the function nodes is integrated into the second sub-feature representation to obtain a series of third sub-feature representations; The pooling layer is used to perform maximum pooling processing on a series of the third sub-feature representations to obtain the first feature representation.

7. A code vulnerability detection device, characterized in that: include: A first processing module, configured to input an abstract syntax tree of a source code into a first network model, and output a first feature representation corresponding to grammatical structure information of the source code; A second processing module is used to input the source code, source code annotations and variable sequences of the source code into a second network model, and output a second feature representation corresponding to the semantic information of the source code; wherein the target hyperparameter combination in the second network model is determined based on an adaptive grid search algorithm; The third processing module is used to detect whether there is a vulnerability in the source code according to the first feature representation and the second feature representation.

8. An electronic device, characterized in that: include: transceivers and processors; The processor is used to: Inputting an abstract syntax tree of the source code into a first network model, and outputting a first feature representation corresponding to the grammatical structure information of the source code; Inputting the source code, source code annotations, and a variable sequence of the source code into a second network model, and outputting a second feature representation corresponding to the semantic information of the source code; wherein the target hyperparameter combination in the second network model is determined based on an adaptive grid search algorithm; The source code is detected to determine whether there is a vulnerability according to the first feature representation and the second feature representation.

9. An electronic device, comprising: A transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; wherein the processor implements the code vulnerability detection method according to any one of claims 1 to 6 when executing the program or instruction.

10. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by the processor, the steps in the code vulnerability detection method according to any one of claims 1 to 6 are implemented.

11. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the code vulnerability detection method as claimed in any one of claims 1 to 6.

Images