Source code vulnerability prediction method and device, electronic equipment and medium

By introducing position coding layer and lightweight Transformer blocks into the source code vulnerability prediction model, combining the multi-head attention mechanism and loss function of dynamic weights, the problems of vulnerability data set processing difficulties and limited model performance in the existing technology are solved. High accuracy and efficient vulnerability prediction are achieved.

CN118981773AActive Publication Date: 2024-11-19GUANGDONG POLYTECHNIC NORMAL UNIV

Patent Information

Application Number
CN202411083393.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-19
Estimated Expiration
2044-08-08

AI Technical Summary

Technical Problem

In the process of highly unbalanced real-world vulnerability datasets, it is difficult to identify a few categories of vulnerabilities, traditional deep learning and machine learning methods cannot fully learn vulnerability features, and model performance is affected by negative impacts between different tasks and weight selection. Larger.

Method used

The source code vulnerability prediction model is adopted that is integrated into the Keras model, which is lightweight Transformer block, convolutional layer, global average pooling layer and fully connected layer. It is processed through the source code feature vector in the unique hot encoding format, combined with The multi-head attention mechanism and loss function of dynamic weights, feature extraction and vulnerability prediction of optimized models.

Benefits of technology

Accurate prediction of vulnerability categories is achieved, the generalization ability of the model and vulnerability feature extraction ability is improved, false positive rate is reduced, developers and security teams can efficiently discover vulnerabilities in the system, and potential security risks are reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118981773A_ABST
    Figure CN118981773A_ABST
Patent Text Reader

Abstract

The invention provides a source code vulnerability prediction method and device, electronic equipment and a medium, and the method comprises the steps: obtaining a to-be-predicted source code data set, carrying out the text marking processing of the to-be-predicted source code data set, obtaining a plurality of source code marking units, building a word frequency sorting vocabulary according to the source code text in the source code marking units, and converting the source code marking unit into a source code feature vector in a one-hot coding format according to the word frequency sorting vocabulary, inputting the source code feature vector in the one-hot coding format into the source code vulnerability prediction model, and outputting a vulnerability category prediction result. According to the method, the to-be-predicted source code data set is preprocessed, and the preprocessed to-be-predicted source code data set is processed through the trained source code vulnerability prediction model to obtain the vulnerability category prediction result, so that accurate prediction of vulnerability categories is realized; developers and security teams are helped to find vulnerabilities in the system accurately and efficiently, and potential security risks are reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of program analysis and artificial intelligence technology, and in particular to a source code vulnerability prediction method, a source code vulnerability prediction device, an electronic device and a computer-readable medium. Background Art

[0002] Source code vulnerability prediction is one of the key security measures in the software development process. During the software development process, the source code written by programmers may have various vulnerabilities, which may be exploited by hackers to cause security vulnerabilities and system crashes. Traditional source code vulnerability prediction methods are usually based on the use of various machine learning algorithms to try to predict potential vulnerabilities from a large number of software attributes, such as using traditional machine learning techniques such as decision trees, support vector machines, and neural networks. However, these existing methods often face difficulties in identifying minority class vulnerabilities when dealing with highly imbalanced real-world vulnerability datasets, and traditional deep learning and machine learning methods are also unable to fully learn vulnerability features. Although these minority class vulnerabilities are small in number, they are often the most dangerous.

[0003] Multi-task learning is a strategy in machine learning. It can improve the generalization ability of the model by learning multiple related tasks at the same time with a given model, and force the model to learn common features across tasks by sharing parameters between different tasks. Especially when there is less data for some tasks, they can gain a deeper understanding from tasks with larger data volumes, which helps the model better learn features from minority class samples. However, there may be negative interactions between different tasks, especially when the optimal representations of these tasks differ greatly, which may greatly reduce the performance of the model. In addition, the choice of weights for different tasks will also have a great impact on the MTL model. Improper weight selection may cause the model to favor certain tasks and ignore other tasks, resulting in some tasks not being able to learn feature data well, reducing accuracy and generalization ability.

[0004] Program source code is highly structured, and the characteristics of syntax and structure determine the execution flow of the code. For example, a simple C language code segment can consist of a logic block consisting of the declaration and initialization of the variable x, conditional judgment, and function call based on the conditional judgment. The security of this block may be affected by the function return value. Without considering positional encoding, the self-attention mechanism will consider all tokens equally in the entire sequence, and the model will find it difficult to understand the direct dependency between variable declaration and initialization and conditional judgment.

[0005] Since it was proposed by Vaswani et al. in 2017, the Transformer model has become the mainstream model for processing sequence data, especially in the field of natural language processing (NLP). It has the characteristics of high computational efficiency, ability to capture the relationship between any two elements, ability to learn complex representations, and easy model expansion. However, traditional Transformer models often have a large number of parameters and consume huge computing resources, which will become a bottleneck when processing large-scale or complex source code datasets such as real-world vulnerability datasets. Summary of the invention

[0006] In view of the above problems, the present invention is proposed to provide a source code vulnerability prediction method and a corresponding source code vulnerability prediction device, an electronic device and a computer-readable medium that overcome the above problems or at least partially solve the above problems.

[0007] The present invention discloses a source code vulnerability prediction method, which comprises:

[0008] Obtain the source code dataset to be predicted;

[0009] Performing text tokenization processing on the source code data set to be tested to obtain a plurality of source code token units;

[0010] Establishing a frequency-ordered vocabulary based on source code text in source code token units;

[0011] Converting source code token units into source code feature vectors in one-hot encoding format according to the frequency-ranked vocabulary;

[0012] The source code feature vector in the one-hot encoding format is input into a source code vulnerability prediction model, and a vulnerability category prediction result is output.

[0013] Optionally, the source code vulnerability prediction model includes a position encoding layer, a lightweight Transformer block, a convolution layer, a global average pooling layer, and a fully connected layer integrated into a Keras model; the source code feature vector in the one-hot encoding format is input into the source code vulnerability prediction model, and a vulnerability category prediction result is output, including:

[0014] Convert the source code feature vector mapping in one-hot encoding format to source code embedding vector.

[0015] Generate a corresponding position index for each source code embedding vector;

[0016] The source code embedding vector is input into the position encoding layer integrated into the Keras model. The position encoding layer integrated into the Keras model calculates the position encoding vector according to the total dimension and position index of the source code embedding vector, and adds the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing the position information.

[0017] Pass the source code embedding vector containing position information to the lightweight Transformer block, and output the source code feature deep enhanced fusion vector;

[0018] The source code feature deep enhancement fusion vector is processed by the convolution layer and the global average pooling layer and then passed to multiple fully connected layers. The outputs of the multiple fully connected layers are converted into the predicted probability of each vulnerability category through the sofmax function.

[0019] Optionally, the source code embedding vector is input into a position encoding layer integrated into the Keras model, the position encoding layer integrated into the Keras model calculates the position encoding vector according to the total dimension and position index of the source code embedding vector, and adds the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing the position information, including:

[0020] The source code embedding vector is input into the position encoding layer integrated into the Keras model. The position encoding layer integrated into the Keras model calculates the sine value and cosine value of each position in each dimension according to the total dimension and position index of the source code embedding vector;

[0021] For each position, the calculated sine and cosine values ​​are interleaved to form a position encoding vector with the same dimension as the total dimension of the source code embedding vector;

[0022] Add the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing position information.

[0023] Optionally, the source code embedding vector containing the position information is passed to a lightweight Transformer block, which outputs a source code feature deep enhanced fusion vector, including:

[0024] Input the source code embedding vector containing position information into the multi-head attention layer and output the multi-head attention fusion vector;

[0025] Perform residual connection between the multi-head attention fusion vector and the source code embedding vector containing position information and perform layer normalization to output the self-attention position enhancement vector;

[0026] The self-attention position enhancement vector is passed into the feed-forward network consisting of a fully connected layer, a dropout layer, and a relu activation function, and the feed-forward network enhancement vector is output;

[0027] The feedforward network enhancement vector is residually connected and layer normalized to output the source code feature depth enhancement fusion vector.

[0028] Optionally, the generation of the source code vulnerability prediction model includes:

[0029] Obtain a source code public data set; the source code public data set includes data with CWE vulnerability labels and data without CWE vulnerability labels;

[0030] Identifying data in the source code public data set that does not carry the CWE vulnerability label, randomly extracting a preset number of data from the data that does not carry the CWE vulnerability label, and recombining the extracted preset number of data with the data that carries the CWE vulnerability label to generate an under-sampled source code subset;

[0031] The under-sampled source code subset is divided into a training data set, a validation data set, and a test data set according to a preset ratio;

[0032] Performing text tokenization processing on the training data set, the verification data set, and the test data set to obtain corresponding multiple source code token units respectively;

[0033] Establishing a corresponding frequency-ranked vocabulary based on the source code text in the source code token unit;

[0034] According to the source code tagging unit, the training data set, the validation data set, and the test data set are converted into input matrices of equal length;

[0035] Convert the input matrices of the training dataset, validation dataset, and test dataset into source code feature vectors in one-hot encoding format based on the frequency-sorted vocabulary;

[0036] Build a source code vulnerability prediction model network;

[0037] The source code vulnerability prediction model network is trained using a training data set of source code feature vectors represented in a one-hot encoding format, and is verified and tested using a verification data set and a test data set of source code feature vectors represented in a one-hot encoding format to obtain the source code vulnerability prediction model.

[0038] Optionally, the source code vulnerability prediction model network is trained using a training data set of source code feature vectors expressed in a one-hot encoding format, and a verification data set and a test data set of source code feature vectors expressed in a one-hot encoding format are used for verification and testing to obtain the source code vulnerability prediction model, including:

[0039] Using a training data set of source code feature vectors represented in a one-hot encoding format to train the source code vulnerability prediction model network, using a loss function based on dynamic weights to calculate a total loss, and updating the source code vulnerability prediction model network parameters based on the total loss;

[0040] During the training iteration process, when the change trend of the loss function on the verification data set changes from a decrease in the previous iteration to an increase in the next iteration, a state change is recorded. If the number of records exceeds the preset early stopping threshold, the training is stopped, and the model parameters of the last time the loss function showed a downward trend in the current training process are saved to obtain the source code vulnerability prediction model.

[0041] The present invention also discloses a source code vulnerability prediction device, which comprises:

[0042] A module for obtaining data to be predicted, used to obtain a source code data set to be predicted;

[0043] A first tokenization processing module, configured to perform text tokenization processing on the source code data set to be tested to obtain a plurality of source code token units;

[0044] A first frequency-ranked vocabulary building module is used to build a frequency-ranked vocabulary according to the source code text in the source code tag unit;

[0045] a first one-hot encoding module, configured to convert source code token units into source code feature vectors in a one-hot encoding format according to the word frequency sorted vocabulary;

[0046] The vulnerability prediction module is used to input the source code feature vector in the one-hot encoding format into the source code vulnerability prediction model and output the vulnerability category prediction result.

[0047] Optionally, the source code vulnerability prediction model includes a position encoding layer, a lightweight Transformer block, a convolutional layer, a global average pooling layer, and a fully connected layer integrated into a Keras model; the vulnerability prediction module includes:

[0048] The embedding vector generation submodule is used to convert the source code feature vector mapping in the one-hot encoding format into the source code embedding vector.

[0049] A position index generation submodule is used to generate a corresponding position index for each source code embedding vector;

[0050] A position encoding submodule, used for inputting the source code embedding vector into the position encoding layer integrated into the Keras model, the position encoding layer integrated into the Keras model calculates the position encoding vector according to the total dimension and position index of the source code embedding vector, and adds the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing the position information;

[0051] The self-attention fusion submodule is used to pass the source code embedding vector containing position information to the lightweight Transformer block and output the source code feature deep enhanced fusion vector;

[0052] The vulnerability category probability prediction submodule is used to pass the source code feature depth enhancement fusion vector to multiple fully connected layers after processing through the convolution layer and the global average pooling layer, and convert the outputs of multiple fully connected layers into the prediction probability of each vulnerability category through the sofmax function.

[0053] Optionally, the position encoding submodule includes:

[0054] A position sine and cosine value calculation unit is used to input the source code embedding vector into the position encoding layer integrated into the Keras model. The position encoding layer integrated into the Keras model calculates the sine value and cosine value of each position in each dimension according to the total dimension and position index of the source code embedding vector.

[0055] a position encoding vector generating unit, configured to interleave the calculated sine values ​​and cosine values ​​for each position into a position encoding vector with the same dimension as the total dimension of the source code embedding vector;

[0056] The position information fusion unit is used to add the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing the position information.

[0057] Optionally, the self-attention fusion submodule includes:

[0058] A multi-head attention fusion vector generation unit, used to input the source code embedding vector containing position information into the multi-head attention layer and output the multi-head attention fusion vector;

[0059] A self-attention position enhancement vector generation unit is used to perform a residual connection between the multi-head attention fusion vector and the source code embedding vector containing the position information and perform layer normalization to output a self-attention position enhancement vector;

[0060] A feedforward network enhancement vector generation unit is used to pass the self-attention position enhancement vector into the feedforward network composed of a fully connected layer, a dropout layer, and a relu activation function, and output the feedforward network enhancement vector;

[0061] The source code feature deep enhancement fusion vector generation unit is used to perform residual connection and layer normalization on the feedforward network enhancement vector, and output the source code feature deep enhancement fusion vector.

[0062] Optionally, the device further comprises:

[0063] A source code public data set acquisition module is used to acquire a source code public data set; the source code public data set includes data with CWE vulnerability labels and data without CWE vulnerability labels;

[0064] an undersampling module, configured to identify data in the source code public data set that does not carry the CWE vulnerability label, randomly extract a preset number of data from the data that does not carry the CWE vulnerability label, and recombining the extracted preset number of data with the data that carries the CWE vulnerability label to generate an undersampled source code subset;

[0065] A data set segmentation module is used to segment the under-sampled source code subset into a training data set, a validation data set and a test data set according to a preset ratio;

[0066] A second tokenization processing module is used to perform text tokenization processing on the training data set, the verification data set, and the test data set to obtain corresponding multiple source code tag units respectively;

[0067] A second frequency-ranked vocabulary building module is used to build a corresponding frequency-ranked vocabulary according to the source code text in the source code tag unit;

[0068] An equal-length matrix construction module is used to convert a training data set, a validation data set, and a test data set into input matrices of equal length according to source code tag units;

[0069] A second one-hot encoding module, for converting input matrices of the training dataset, the validation dataset, and the test dataset into source code feature vectors in a one-hot encoding format according to a word frequency sorted vocabulary;

[0070] Model network building module, used to build source code vulnerability prediction model network;

[0071] The model training module is used to train the source code vulnerability prediction model network using a training data set of source code feature vectors represented in a one-hot encoding format, and to perform verification and testing using a verification data set and a test data set of source code feature vectors represented in a one-hot encoding format, so as to obtain the source code vulnerability prediction model.

[0072] Optionally, the model training module includes:

[0073] A model network parameter optimization submodule, for training the source code vulnerability prediction model network using a training data set of source code feature vectors represented in a one-hot encoding format, calculating a total loss using a loss function based on a dynamic weight, and updating the source code vulnerability prediction model network parameters based on the total loss;

[0074] The source code vulnerability prediction model generation submodule is used to record a state change when the change trend of the loss function on the verification data set changes from a decrease in the previous iteration to an increase in the next iteration during the training iteration process. If the number of records exceeds the preset early stopping threshold, the training is stopped, and the model parameters when the loss function showed a downward trend for the last time in the current training process are saved to obtain the source code vulnerability prediction model.

[0075] The present invention also discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0076] The memory is used to store computer programs;

[0077] The processor is used to implement the source code vulnerability prediction method of the present invention when executing the program stored in the memory.

[0078] The present invention also discloses one or more computer-readable media on which instructions are stored. When executed by one or more processors, the processors execute the source code vulnerability prediction method of the present invention.

[0079] The present invention includes the following advantages:

[0080] The source code vulnerability prediction method of the present invention obtains a source code data set to be predicted, performs text tokenization processing on the source code data set to be tested to obtain multiple source code token units, establishes a frequency-sorted vocabulary table according to the source code text in the source code token unit, converts the source code token unit into a source code feature vector in a one-hot encoding format according to the frequency-sorted vocabulary table, inputs the source code feature vector in the one-hot encoding format into a source code vulnerability prediction model, and outputs a vulnerability category prediction result. The present invention obtains a vulnerability category prediction result by preprocessing the source code data set to be predicted and processing the preprocessed source code data set to be predicted by a trained source code vulnerability prediction model, thereby achieving accurate prediction of vulnerability categories, helping developers and security teams to accurately and efficiently discover vulnerabilities in the system and reduce potential security risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 It is a flowchart of the steps of a source code vulnerability prediction method provided by an embodiment of the present invention;

[0082] Figure 2 is a flow chart of source code vulnerability prediction model training provided by an embodiment of the present invention;

[0083] Figure 3 It is a structural block diagram of a source code vulnerability prediction device provided by an embodiment of the present invention;

[0084] Figure 4 is a block diagram of an electronic device provided by an embodiment of the present invention;

[0085] Figure 5 It is a schematic diagram of a computer-readable medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0086] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0087] Reference Figure 1 , shows a flow chart of the steps of a source code vulnerability prediction method provided in an embodiment of the present invention, which may specifically include the following steps:

[0088] Step 101, obtaining a source code data set to be predicted;

[0089] Step 102, performing text tokenization processing on the source code data set to be tested to obtain a plurality of source code token units;

[0090] Step 103, establishing a frequency-ordered vocabulary based on the source code text in the source code tag unit;

[0091] Step 104, converting the source code token unit into a source code feature vector in a one-hot encoding format according to the word frequency sorted vocabulary;

[0092] Step 105: input the source code feature vector in the one-hot encoding format into a source code vulnerability prediction model, and output a vulnerability category prediction result.

[0093] In an embodiment of the present invention, after obtaining the source code data set to be predicted, some preprocessing can be performed on the source code data set to be predicted to accurately predict the vulnerability category. Specifically, the source code data set to be predicted can be cleaned and standardized to ensure the consistency and accuracy of the data, eliminate noise, and enable the model to better identify the vulnerability characteristics of the source code. Then, the cleaned and standardized source code data set to be predicted can be subjected to text tokenization, that is, the text data is converted into a series of source code tag unit tokens, which are used to construct numerical inputs that can be understood by the model, and a vocabulary sorted by word frequency is created based on all source code texts in the training set. Furthermore, according to the word frequency sorting vocabulary, the feature columns in the source code data set to be predicted after text tokenization can be converted into a one-hot encoding format using one-hot encoding, so that each category has a clear target vector representation, further enhancing the expressiveness of the data and laying the foundation for more accurate prediction of vulnerability categories.

[0094] Through these preprocessing of the source code data set to be predicted, model input data suitable for the source code vulnerability prediction model of the present invention that can improve the prediction accuracy is obtained. Finally, the source code feature vector in the one-hot encoding format obtained by preprocessing is input into the source code vulnerability prediction model. The source code vulnerability prediction model processes the source code feature vector in the one-hot encoding format and outputs a vulnerability category prediction result, thereby obtaining the probabilities of various vulnerability categories existing in the source code to be predicted, and accurately predicting the vulnerability category through the source code vulnerability prediction model of the present invention, helping developers and security teams to accurately and efficiently discover vulnerabilities in the system and reduce potential security risks.

[0095] In one embodiment of the present invention, the source code vulnerability prediction model includes a position encoding layer, a lightweight Transformer block, a convolution layer, a global average pooling layer, and a fully connected layer integrated into a Keras model; the source code feature vector in the one-hot encoding format is input into the source code vulnerability prediction model, and the vulnerability category prediction result is output, including:

[0096] Convert the source code feature vector mapping in one-hot encoding format to source code embedding vector.

[0097] Generate a corresponding position index for each source code embedding vector;

[0098] The source code embedding vector is input into the position encoding layer integrated into the Keras model. The position encoding layer integrated into the Keras model calculates the position encoding vector according to the total dimension and position index of the source code embedding vector, and adds the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing the position information.

[0099] Pass the source code embedding vector containing position information to the lightweight Transformer block, and output the source code feature deep enhanced fusion vector;

[0100] The source code feature deep enhancement fusion vector is processed by the convolution layer and the global average pooling layer and then passed to multiple fully connected layers. The outputs of the multiple fully connected layers are converted into the predicted probability of each vulnerability category through the sofmax function.

[0101] The source code vulnerability prediction model of an embodiment of the present invention may include a position encoding layer defined in a keras model, as well as a lightweight Transformer block, a convolution layer, a global average pooling layer, and a fully connected layer. The source code vulnerability prediction model first receives a source code feature vector in a one-hot encoding format, generates a source code embedding vector through a mapping transformation, and uses a position encoding layer to generate and add position information to each embedding vector to capture the sequence characteristics and long-distance dependencies in the source code. Subsequently, these source code embedding vectors containing position information are fed into a lightweight Transformer block, which further enhances and fuses the deep features of the source code through a self-attention mechanism, thereby extracting richer contextual information.

[0102] After feature extraction and fusion, the model uses convolutional layers and global average pooling layers to further process the source code feature depth enhancement fusion vector to reduce the number of parameters, prevent overfitting, and extract the most representative features. Finally, these features are passed to multiple fully connected layers, and the output of each fully connected layer is converted into the prediction probability of the corresponding vulnerability category through the softmax function. The vulnerability category predicted by the source code vulnerability prediction model of the embodiment of the present invention has high accuracy, high efficiency, low false positives, and high true negatives, which reduces the false alarm rate, helps developers and security teams accurately and efficiently discover vulnerabilities in the system and reduce potential security risks.

[0103] In one embodiment of the present invention, the source code embedding vector is input into a position encoding layer integrated into a Keras model, the position encoding layer integrated into the Keras model calculates the position encoding vector according to the total dimension and position index of the source code embedding vector, and adds the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing the position information, including:

[0104] The source code embedding vector is input into the position encoding layer integrated into the Keras model. The position encoding layer integrated into the Keras model calculates the sine value and cosine value of each position in each dimension according to the total dimension and position index of the source code embedding vector;

[0105] For each position, the calculated sine and cosine values ​​are interleaved to form a position encoding vector with the same dimension as the total dimension of the source code embedding vector;

[0106] Add the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing position information.

[0107] The position encoding layer of the embodiment of the present invention is integrated into the Keras model, and the floating point constant is used to improve the accuracy of the calculation. Specifically, the position encoding layer provides a unique position encoding PE for each position θ in the source code token sequence, where each position encoding PE is composed of the sine value H of each position θ. z and cosine value H y Combination generation, sine value H z and cosine value H y The formula is as follows:

[0108]

[0109] Among them, d i is the dimension index of the θ position in the embedding vector. The even index uses formula (1) to calculate the sine value, and the odd index uses formula (2) to calculate the cosine value. m is the total dimension of the embedding vector.

[0110] After obtaining the sine value H corresponding to each position θ z and cosine value H y Then, we calculate a 2×h-dimensional matrix consisting of alternating sine and cosine values. m Vector, that is, the complete position encoding vector PE θ , PE θ The formula is as follows:

[0111]

[0112] in, The sine value calculated for each position θ is The cosine is calculated for each position θ.

[0113] For each position index θ in the source code vulnerability prediction model, add the position encoding vector PE of the position θ The embedded vector X with position information added is obtained by adding the embedded vector at that position.

[0114] By adding a custom position encoding layer, the model can identify the relationship between the positions of code lines, thereby understanding the role of variables in conditional judgments and capturing syntax and structural dependencies. In addition, in a C language code segment, the semantics of each code line is closely related to its position in the code. Through the position encoding layer, the model can better understand the logical relationship between the code lines, reduce confusion in the model code logic understanding, and reduce the false alarm rate.

[0115] In one embodiment of the present invention, a source code embedding vector containing position information is passed to a lightweight Transformer block, and a source code feature deep enhanced fusion vector is output, including:

[0116] Input the source code embedding vector containing position information into the multi-head attention layer and output the multi-head attention fusion vector;

[0117] Perform residual connection between the multi-head attention fusion vector and the source code embedding vector containing position information and perform layer normalization to output the self-attention position enhancement vector;

[0118] The self-attention position enhancement vector is passed into the feed-forward network consisting of a fully connected layer, a dropout layer, and a relu activation function, and the feed-forward network enhancement vector is output;

[0119] The feedforward network enhancement vector is residually connected and layer normalized to output the source code feature depth enhancement fusion vector.

[0120] The lightweight Transformer block linearly transforms the embedded vector X with position information to obtain Q, K, and V, and then passes Q, K, and V through four attention heads to obtain the output Att of each attention head. i (i=1,2,3,4), then concatenate the outputs of all attention heads and perform a linear transformation to obtain the multi-head attention output MH(Q,K,V), and then perform a residual connection between the multi-head attention output MH(Q,K,V) and the input vector Q and perform layer normalization to obtain the output O attn , and finally the output O after layer normalization attn It is passed into the feedforward neural network (FFN), and then residual connection and layer normalization are performed to obtain the output O t , O t The formula is as follows:

[0121]

[0122] O attn =LayerNorm(Q+MH(Q,K,V)) (5)

[0123] O t =LayerNorm( attn +FFN·O attn ) (6)

[0124] Among them, W is the linear mapping matrix that maps the input to different representation spaces, W O It is the matrix used to output the concatenated mapping, LayerNorm is the layer normalization function, and FFN is the calculation function of the feedforward network.

[0125] In one embodiment of the present invention, the generation of the source code vulnerability prediction model includes:

[0126] Obtain a source code public data set; the source code public data set includes data with CWE vulnerability labels and data without CWE vulnerability labels;

[0127] Identifying data in the source code public data set that does not carry the CWE vulnerability label, randomly extracting a preset number of data from the data that does not carry the CWE vulnerability label, and recombining the extracted preset number of data with the data that carries the CWE vulnerability label to generate an under-sampled source code subset;

[0128] The under-sampled source code subset is divided into a training data set, a validation data set, and a test data set according to a preset ratio;

[0129] Performing text tokenization processing on the training data set, the verification data set, and the test data set to obtain corresponding multiple source code token units respectively;

[0130] Establishing a corresponding frequency-ranked vocabulary based on the source code text in the source code token unit;

[0131] According to the source code tagging unit, the training data set, the validation data set, and the test data set are converted into input matrices of equal length;

[0132] Convert the input matrices of the training dataset, validation dataset, and test dataset into source code feature vectors in one-hot encoding format based on the frequency-sorted vocabulary;

[0133] Build a source code vulnerability prediction model network;

[0134] The source code vulnerability prediction model network is trained using a training data set of source code feature vectors represented in a one-hot encoding format, and is verified and tested using a verification data set and a test data set of source code feature vectors represented in a one-hot encoding format to obtain the source code vulnerability prediction model.

[0135] In one embodiment of the present invention, the source code vulnerability prediction model network is trained using a training data set of source code feature vectors represented in a one-hot encoding format, and a verification data set and a test data set of source code feature vectors represented in a one-hot encoding format are used for verification and testing to obtain the source code vulnerability prediction model, including:

[0136] Using a training data set of source code feature vectors represented in a one-hot encoding format to train the source code vulnerability prediction model network, using a loss function based on dynamic weights to calculate a total loss, and updating the source code vulnerability prediction model network parameters based on the total loss;

[0137] During the training iteration process, when the change trend of the loss function on the verification data set changes from a decrease in the previous iteration to an increase in the next iteration, a state change is recorded. If the number of records exceeds the preset early stopping threshold, the training is stopped, and the model parameters of the last time the loss function showed a downward trend in the current training process are saved to obtain the source code vulnerability prediction model.

[0138] Reference Figure 2 The embodiment of the present invention uses a public real-world source code vulnerability dataset to train the constructed source code vulnerability prediction model network to obtain a source code vulnerability prediction model.

[0139] Specifically, the source code vulnerability prediction model network designed by the present invention is a multi-task learning model framework MTLPT based on a custom position encoding layer and a lightweight Transformer block, wherein the custom position encoding layer is integrated into the Keras model, and the floating point of constants is used to improve the accuracy of calculation; the lightweight Transformer block includes a multi-head self-attention layer with 4 heads to reduce the computational complexity, a Dropout layer for randomly discarding a part of the features during training to prevent the MTLPT model from over-relying on the sample features of certain specific vulnerabilities in the training data and reducing the overfitting phenomenon, a layer normalization layer to keep the internal covariate stable and accelerate the convergence of the MTLPT model, a feedforward network consisting of two fully connected layers, a Dropout layer and a relu activation function to ensure that the same fully connected neural network is applied at each position θ, thereby further enhancing the representation ability of the MTLPT model, and also including a residual connection to help alleviate the problem of gradient disappearance or explosion in deep networks; and adding convolutional layers and global average pooling layers to improve the ability to capture dependencies and reduce data dimensions and reduce the amount of calculation. Finally, multiple fully connected layers, Dense layers and softmax are used to make decisions for each task and obtain vulnerability prediction results. A dynamic weight-based loss function is defined to dynamically adjust the weights of each task in the total loss function according to the loss change rate of each task.

[0140] Before training the source code vulnerability prediction model network, the source code public dataset is preprocessed. Specifically, the source code public dataset is undersampled to alleviate the problem of data imbalance, that is, the data that does not contain CWE vulnerabilities in the original dataset is undersampled, and 10% of the data that does not contain CWE vulnerabilities is randomly selected and recombined with the data with CWE labels in the original dataset to generate an undersampled subset. The undersampled subset of the source code dataset is cleaned and standardized to ensure the consistency and accuracy of the data, eliminate noise, and enable the model to better learn the vulnerability characteristics of the source code. And the training dataset, verification dataset, and test dataset are randomly divided into a ratio of 8:1:1.

[0141] Furthermore, the training data set, validation data set, and test data set are tokenized: the text data is converted into a series of tokens for constructing numerical inputs that the model can understand, and a vocabulary is created based on all the source code text in the training set, sorted by word frequency. Then, each data set (training set, validation set, and test set) is converted into a lexical structure of length I according to the tokens of the text data. max The input matrix (D num , I max ), where D num It is the number of data, ensuring the consistency of model input. Use one-hot encoding to convert the feature columns in the dataset into one-hot encoding format, so that each category has a clear target vector representation, further enhancing the expressiveness of the data.

[0142] The one-hot encoded training set is used as the input of the model for training. At the end of each training iteration, the loss function based on dynamic weights is called to calculate the total loss, learn the correlation between tasks, reduce the difference interference between different tasks, improve model performance, and control the preservation of the best performance model through the change of the loss function of the model in the validation data set after each iteration. Set the early stopping function threshold to prevent overfitting. The early stopping function threshold can be set according to the actual situation, such as 5, 8, 10, etc. Loss function L based on dynamic weights t is defined as follows:

[0143]

[0144] Among them, L i For each task, the cross entropy loss of each task at the end of each iteration is M i , N is the number of tasks, S is a parameter used to control the degree of weight smoothing (S∈

[0145] (0, 5]), which is determined to be 0.7 by the experimental hyperparameter tuning of the present invention.

[0146] The source code vulnerability prediction method of the present invention combines a multi-task learning framework (MTL, multitasklearning), a custom position encoding layer (PE, position encoding layer) and a multi-category vulnerability prediction model of a Transformer block, and uses a dynamic weight allocation strategy to allocate task weights, thereby predicting vulnerabilities more accurately and efficiently. This method mainly includes three parts: First, using the correlation between different vulnerabilities, a model is used to simultaneously learn multiple related tasks to improve the generalization ability of the model, and by sharing parameters between different tasks, the model is forced to learn common features across tasks, thereby improving the feature extraction ability of a few classes of vulnerabilities. Second, through a custom position encoding layer and a lightweight Transformer block, the number of parameters is reduced and the computational efficiency is improved, while retaining the core advantages and performance of the Transformer, capturing syntax and structural dependencies, and effectively learning the complex features of source code vulnerabilities. Third, according to the loss change rate of each task, their weights in the total loss function are dynamically adjusted and a multi-head attention mechanism is used to allow the model to automatically learn the correlation between tasks, thereby reducing the difference interference between different tasks. The present invention uses a multi-task learning framework, combined with a customized lightweight Transformer block and a position encoding layer, to learn the potential relationship between different vulnerability types, and effectively capture the long-distance dependencies and complex context information of the source code, thereby improving the accuracy and efficiency of vulnerability prediction and enhancing the model's sensitivity to rare but serious vulnerabilities. In addition, the present invention introduces a loss function based on dynamic weights, dynamically adjusts the loss weights according to the prediction difficulty of different tasks, and effectively alleviates the challenges brought by unbalanced data, achieving high-accuracy, high-efficiency, low-false-positive, and high-true-negative vulnerability prediction, reducing the false alarm rate, and helping developers and security teams to accurately and efficiently discover vulnerabilities in the system and reduce potential security risks.

[0147] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0148] Reference Figure 3 , shows a structural block diagram of a source code vulnerability prediction device provided in an embodiment of the present invention, which may specifically include the following modules:

[0149] The to-be-predicted data acquisition module 301 is used to acquire a to-be-predicted source code data set;

[0150] A first tokenization processing module 302 is used to perform text tokenization processing on the source code data set to be tested to obtain a plurality of source code token units;

[0151] A first frequency-ranked vocabulary building module 303 is used to build a frequency-ranked vocabulary according to the source code text in the source code tag unit;

[0152] A first one-hot encoding module 304, configured to convert source code token units into source code feature vectors in a one-hot encoding format according to the frequency-ranked vocabulary;

[0153] The vulnerability prediction module 305 is used to input the source code feature vector in the one-hot encoding format into the source code vulnerability prediction model and output the vulnerability category prediction result.

[0154] Optionally, the source code vulnerability prediction model includes a position encoding layer, a lightweight Transformer block, a convolutional layer, a global average pooling layer, and a fully connected layer integrated into a Keras model; the vulnerability prediction module includes:

[0155] The embedding vector generation submodule is used to convert the source code feature vector mapping in the one-hot encoding format into the source code embedding vector.

[0156] A position index generation submodule is used to generate a corresponding position index for each source code embedding vector;

[0157] A position encoding submodule, used for inputting the source code embedding vector into the position encoding layer integrated into the Keras model, the position encoding layer integrated into the Keras model calculates the position encoding vector according to the total dimension and position index of the source code embedding vector, and adds the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing the position information;

[0158] The self-attention fusion submodule is used to pass the source code embedding vector containing position information to the lightweight Transformer block and output the source code feature deep enhanced fusion vector;

[0159] The vulnerability category probability prediction submodule is used to pass the source code feature depth enhancement fusion vector to multiple fully connected layers after processing through the convolution layer and the global average pooling layer, and convert the outputs of multiple fully connected layers into the prediction probability of each vulnerability category through the sofmax function.

[0160] Optionally, the position encoding submodule includes:

[0161] A position sine and cosine value calculation unit is used to input the source code embedding vector into the position encoding layer integrated into the Keras model. The position encoding layer integrated into the Keras model calculates the sine value and cosine value of each position in each dimension according to the total dimension and position index of the source code embedding vector.

[0162] a position encoding vector generating unit, configured to interleave the calculated sine values ​​and cosine values ​​for each position into a position encoding vector with the same dimension as the total dimension of the source code embedding vector;

[0163] The position information fusion unit is used to add the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing the position information.

[0164] Optionally, the self-attention fusion submodule includes:

[0165] A multi-head attention fusion vector generation unit, used to input the source code embedding vector containing position information into the multi-head attention layer and output the multi-head attention fusion vector;

[0166] A self-attention position enhancement vector generation unit is used to perform a residual connection between the multi-head attention fusion vector and the source code embedding vector containing the position information and perform layer normalization to output a self-attention position enhancement vector;

[0167] A feedforward network enhancement vector generation unit is used to pass the self-attention position enhancement vector into the feedforward network composed of a fully connected layer, a dropout layer, and a relu activation function, and output the feedforward network enhancement vector;

[0168] The source code feature deep enhancement fusion vector generation unit is used to perform residual connection and layer normalization on the feedforward network enhancement vector, and output the source code feature deep enhancement fusion vector.

[0169] Optionally, the device further comprises:

[0170] A source code public data set acquisition module is used to acquire a source code public data set; the source code public data set includes data with CWE vulnerability labels and data without CWE vulnerability labels;

[0171] an undersampling module, configured to identify data in the source code public data set that does not carry the CWE vulnerability label, randomly extract a preset number of data from the data that does not carry the CWE vulnerability label, and recombining the extracted preset number of data with the data that carries the CWE vulnerability label to generate an undersampled source code subset;

[0172] A data set segmentation module is used to segment the under-sampled source code subset into a training data set, a validation data set and a test data set according to a preset ratio;

[0173] A second tokenization processing module is used to perform text tokenization processing on the training data set, the verification data set, and the test data set to obtain corresponding multiple source code tag units respectively;

[0174] A second frequency-ranked vocabulary building module is used to build a corresponding frequency-ranked vocabulary according to the source code text in the source code tag unit;

[0175] An equal-length matrix construction module is used to convert a training data set, a validation data set, and a test data set into input matrices of equal length according to source code tag units;

[0176] A second one-hot encoding module, for converting input matrices of the training dataset, the validation dataset, and the test dataset into source code feature vectors in a one-hot encoding format according to a word frequency sorted vocabulary;

[0177] Model network building module, used to build source code vulnerability prediction model network;

[0178] The model training module is used to train the source code vulnerability prediction model network using a training data set of source code feature vectors represented in a one-hot encoding format, and to perform verification and testing using a verification data set and a test data set of source code feature vectors represented in a one-hot encoding format, so as to obtain the source code vulnerability prediction model.

[0179] Optionally, the model training module includes:

[0180] A model network parameter optimization submodule, for training the source code vulnerability prediction model network using a training data set of source code feature vectors represented in a one-hot encoding format, calculating a total loss using a loss function based on a dynamic weight, and updating the source code vulnerability prediction model network parameters based on the total loss;

[0181] The source code vulnerability prediction model generation submodule is used to record a state change when the change trend of the loss function on the verification data set changes from a decrease in the previous iteration to an increase in the next iteration during the training iteration process. If the number of records exceeds the preset early stopping threshold, the training is stopped, and the model parameters when the loss function showed a downward trend for the last time in the current training process are saved to obtain the source code vulnerability prediction model.

[0182] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0183] In addition, an embodiment of the present invention further provides an electronic device, such as Figure 4As shown, it includes a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0184] Memory 403, used for storing computer programs;

[0185] The processor 401 is used to implement the source code vulnerability prediction method described in the above embodiment when executing the program stored in the memory 403.

[0186] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0187] The communication interface is used for communication between the above terminal and other devices.

[0188] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0189] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0190] like Figure 5As shown, in another embodiment provided by the present invention, a computer-readable storage medium 501 is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the source code vulnerability prediction method described in the above embodiment.

[0191] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the source code vulnerability prediction method described in the above embodiment.

[0192] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0193] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0194] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0195] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A source code vulnerability prediction method, characterized in that: The method comprises: Obtain the source code dataset to be predicted; Performing text tokenization processing on the source code data set to be tested to obtain a plurality of source code token units; Establishing a frequency-ordered vocabulary based on source code text in source code token units; Converting source code token units into source code feature vectors in one-hot encoding format according to the frequency-ranked vocabulary; The source code feature vector in the one-hot encoding format is input into a source code vulnerability prediction model, and a vulnerability category prediction result is output.

2. The method according to claim 1, characterized in that The source code vulnerability prediction model includes a position encoding layer, a lightweight Transformer block, a convolutional layer, a global average pooling layer, and a fully connected layer integrated into the Keras model; The source code feature vector in the one-hot encoding format is input into the source code vulnerability prediction model, and the vulnerability category prediction result is output, including: Convert the source code feature vector mapping in one-hot encoding format to source code embedding vector. Generate a corresponding position index for each source code embedding vector; The source code embedding vector is input into the position encoding layer integrated into the Keras model. The position encoding layer integrated into the Keras model calculates the position encoding vector according to the total dimension and position index of the source code embedding vector, and adds the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing the position information. Pass the source code embedding vector containing position information to the lightweight Transformer block, and output the source code feature deep enhanced fusion vector; The source code feature deep enhancement fusion vector is processed by the convolution layer and the global average pooling layer and then passed to multiple fully connected layers. The outputs of the multiple fully connected layers are converted into the predicted probability of each vulnerability category through the sofmax function.

3. The method according to claim 2, characterized in that The source code embedding vector is input into the position encoding layer integrated into the Keras model. The position encoding layer integrated into the Keras model calculates the position encoding vector according to the total dimension and position index of the source code embedding vector, and adds the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing the position information, including: The source code embedding vector is input into the position encoding layer integrated into the Keras model. The position encoding layer integrated into the Keras model calculates the sine value and cosine value of each position in each dimension according to the total dimension and position index of the source code embedding vector; For each position, the calculated sine and cosine values ​​are interleaved to form a position encoding vector with the same dimension as the total dimension of the source code embedding vector; Add the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing position information.

4. The method according to claim 2, characterized in that: The source code embedding vector containing position information is passed to the lightweight Transformer block, and the source code feature deep enhancement fusion vector is output, including: Input the source code embedding vector containing position information into the multi-head attention layer and output the multi-head attention fusion vector; Perform residual connection between the multi-head attention fusion vector and the source code embedding vector containing position information and perform layer normalization to output the self-attention position enhancement vector; The self-attention position enhancement vector is passed into the feed-forward network consisting of a fully connected layer, a dropout layer, and a relu activation function, and the feed-forward network enhancement vector is output; The feedforward network enhancement vector is residually connected and layer normalized to output the source code feature depth enhancement fusion vector.

5. The method according to claim 1, characterized in that: The generation of the source code vulnerability prediction model includes: Obtain a source code public data set; the source code public data set includes data with CWE vulnerability labels and data without CWE vulnerability labels; Identifying data in the source code public data set that does not carry the CWE vulnerability label, randomly extracting a preset number of data from the data that does not carry the CWE vulnerability label, and recombining the extracted preset number of data with the data that carries the CWE vulnerability label to generate an under-sampled source code subset; The under-sampled source code subset is divided into a training data set, a validation data set, and a test data set according to a preset ratio; Performing text tokenization processing on the training data set, the verification data set, and the test data set to obtain corresponding multiple source code token units respectively; Establishing a corresponding frequency-ranked vocabulary based on the source code text in the source code token unit; According to the source code tagging unit, the training data set, the validation data set, and the test data set are converted into input matrices of equal length; Convert the input matrices of the training dataset, validation dataset, and test dataset into source code feature vectors in one-hot encoding format based on the frequency-sorted vocabulary; Build a source code vulnerability prediction model network; The source code vulnerability prediction model network is trained using a training data set of source code feature vectors represented in a one-hot encoding format, and is verified and tested using a verification data set and a test data set of source code feature vectors represented in a one-hot encoding format to obtain the source code vulnerability prediction model.

6. The method according to claim 5, characterized in that The source code vulnerability prediction model network is trained using a training data set of source code feature vectors represented in a one-hot encoding format, and a verification data set and a test data set of source code feature vectors represented in a one-hot encoding format are used for verification and testing to obtain the source code vulnerability prediction model, including: Using a training data set of source code feature vectors represented in a one-hot encoding format to train the source code vulnerability prediction model network, using a loss function based on dynamic weights to calculate a total loss, and updating the source code vulnerability prediction model network parameters based on the total loss; During the training iteration process, when the change trend of the loss function on the verification data set changes from a decrease in the previous iteration to an increase in the next iteration, a state change is recorded. If the number of records exceeds the preset early stopping threshold, the training is stopped, and the model parameters of the last time the loss function showed a downward trend in the current training process are saved to obtain the source code vulnerability prediction model.

7. A source code vulnerability prediction device, characterized in that: The device comprises: A module for obtaining data to be predicted, used to obtain a source code data set to be predicted; A first tokenization processing module, configured to perform text tokenization processing on the source code data set to be tested to obtain a plurality of source code token units; A first frequency-ranked vocabulary building module is used to build a frequency-ranked vocabulary according to the source code text in the source code tag unit; a first one-hot encoding module, configured to convert source code token units into source code feature vectors in a one-hot encoding format according to the word frequency sorted vocabulary; The vulnerability prediction module is used to input the source code feature vector in the one-hot encoding format into the source code vulnerability prediction model and output the vulnerability category prediction result.

8. The device according to claim 7, characterized in that The source code vulnerability prediction model includes a position encoding layer, a lightweight Transformer block, a convolutional layer, a global average pooling layer, and a fully connected layer integrated into the Keras model; the vulnerability prediction module includes: The embedding vector generation submodule is used to convert the source code feature vector mapping in the one-hot encoding format into the source code embedding vector. A position index generation submodule is used to generate a corresponding position index for each source code embedding vector; A position encoding submodule, used for inputting the source code embedding vector into the position encoding layer integrated into the Keras model, the position encoding layer integrated into the Keras model calculates the position encoding vector according to the total dimension and position index of the source code embedding vector, and adds the position encoding vector to the source code embedding vector to obtain the source code embedding vector containing the position information; The self-attention fusion submodule is used to pass the source code embedding vector containing position information to the lightweight Transformer block and output the source code feature deep enhanced fusion vector; The vulnerability category probability prediction submodule is used to pass the source code feature depth enhancement fusion vector to multiple fully connected layers after processing through the convolution layer and the global average pooling layer, and convert the outputs of multiple fully connected layers into the prediction probability of each vulnerability category through the sofmax function.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the source code vulnerability prediction method according to any one of claims 1 to 6 when executing a program stored in the memory.

10. One or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, enable the processors to execute the source code vulnerability prediction method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Intelligent contract code defect detection method based on structured word embedding network

    CN112416358A

  • A method for automatically identifying security-related code submission by using Attention coding representation

    CN113553052A

  • Software vulnerability automatic classification method based on weighted word vector and neural network

    CN114881172A

  • Intelligent contract vulnerability detection method based on local and sequence feature fusion

    CN115080981A

  • Transform-based source code vulnerability detection method and device, computer equipment and medium

    CN116595541A

Cited By

  • Intelligent contract code vulnerability detection system based on deep learning method

    CN119691752A