Vulnerability detection method based on code change large model

Abstract

The invention provides a vulnerability detection method based on a code change large model. Comprising the following steps: preprocessing a data set, deleting repeated samples and annotations, only reserving a code text, performing token processing on the code text, and removing repeated tokens; then, traversing the code text without the vulnerability feature, calculating the Jaccard similarity with the target code, and selecting the code text with the highest similarity with the code text without the vulnerability feature to form a data sample pair; and regarding the code text without the vulnerability feature as a pre-change fragment, regarding the target code as a post-change fragment, extracting an inter-code difference part, and marking the inter-code difference part as an added or deleted code fragment. Initializing a model by using a pre-training weight of a large model CCT5 based on code change, adjusting a model structure, setting key parameters, finely tuning the model on a training set, and updating the parameters for detecting code vulnerabilities; and inputting a section of code text into a CCT5 model which has the best performance in the fine tuning task, wherein the model outputs a predicted value of whether the model has a vulnerability or not.

Description

Technical Field

[0001] The present invention relates to an influencing factor analysis method, and in particular to a vulnerability detection method based on a large code change model. Background Art

[0002] With the widespread application of information technology in modern society, software vulnerabilities have caused great damage to individuals and enterprises. The personal information of many Internet users has been leaked due to security vulnerabilities. According to a report by IBM in 2021, the average cost of a data leak to the target company is 4.24 million US dollars. The number of disclosed security vulnerabilities has also shown an increasing trend year by year. According to a report released by the security database website CVE Details: a total of 20,153 security vulnerabilities were released in 2021, 25,082 security vulnerabilities were released in 2022, 29,066 security vulnerabilities were released in 2023, and 35,062 security vulnerabilities were released as of October 2024; in the past decade, more than 19.1% of the vulnerabilities released have a CVSS score of more than 9.0. In addition to the increase in quantity, the form of security vulnerabilities has also shown complexity and diversity, posing a great threat to the normal and safe operation of computer systems.

[0003] Traditional vulnerability detection technology is to discover potential vulnerabilities in the program by analyzing the lexical, grammatical and semantic information of the target program without executing the target program, combining the data flow and control flow information of the program, and using type inference, security rule checking, model detection and other technologies. With the booming development of the artificial intelligence industry, many methods based on deep neural network technology have achieved great success in image recognition, natural language processing and other fields. The vulnerability detection workflow based on deep learning mainly includes two key steps: training phase and detection phase. From the perspective of code representation, existing vulnerability detection methods directly encode a single code, and then predict whether the code has a vulnerability based on deep learning technology. However, these methods are less effective when processing long codes because many neural models have restrictions on input length, which limits their ability to process long vulnerability code fragments. Secondly, these methods find it difficult to distinguish vulnerability-related information from a large amount of irrelevant information, which affects their effectiveness in capturing vulnerability features. Summary of the invention

[0004] In view of the above problems existing in the prior art, the technical problem to be solved by the present invention is: how to more accurately predict whether a code has a vulnerability.

[0005] To solve the above technical problems, the present invention adopts the following technical solution: a vulnerability detection method based on a large code change model, comprising a component one for constructing differences between similar sample pairs and a component two for detecting whether a vulnerability exists.

[0006] Component 1 includes the following steps:

[0007] S101: Obtain labeled data samples as the original data set D = {d1, d2, d3, ..., d n}, where d n represents the nth original data sample. The label is whether the original data sample has a vulnerability. The original data sample is preprocessed, including deleting duplicate data samples and removing comments in the code. Only the code text is retained to obtain a new data set D″={d′1,d′2,d′3,…,d′ m}, where d′ m represents the mth code text, m≤n, and the labels of the samples in D″ and the corresponding samples in D remain unchanged.

[0008] Specifically, duplicate samples are deleted to ensure the uniqueness of samples in the data set. The original data set is D = {d1, d2, d3, ..., d n}, the dataset after deleting duplicate samples is recorded as D′. The code text in each data sample in D′ is parsed, the comments in the code are removed, and only the code text is retained. The processed dataset is represented as D″={d′1,d′2,d′3,…,d′ m}, where m≤n.

[0009] S102: The i-th code text d′ in D″ i Tokenize to get token list T i ={t1,t2,t3,…,t k}, where t k Indicates T i The kth element in T i Perform deduplication operation so that T i Only unique tokens are retained, and the token set after deduplication is recorded as in Represents T′ i The p-th element in , p≤k, ensures that each code text only retains a unique token list.

[0010] Specifically, the code text d′ i Tokenize d′ i Split into token list T i ={t1,t2,t3,…,t k}, where each t j Represents a code token (such as a variable, operator, keyword, etc.).

[0011] For each token list T i Perform deduplication operation so that Ti Only unique tokens are retained. The token set after deduplication is Where p≤k.

[0012] S103: For each T′ i , traverse all the samples T′ without loopholes in D″ j , calculate T′ i and T′ j The Jaccard similarity between them will find the most similar non-vulnerable code text and T′ i Composition sample pair

[0013] S104: The code snippet considered as “before change”, T′ i The code snippet considered as "changed" is calculated Relative to T′ i Added code snippet Δ add , deleted code snippet Δ delete and the difference set

[0014] Specifically, let the code snippet "before the change" be The "changed" code snippet is T after =T′ i .calculate Relative to T′ i Added and deleted segments Δ add and Δ delete :

[0015]

[0016] Get the difference code snippet set Δ=(Δ add ,Δ delete ).

[0017] S105: Construct the final data set D″′={d″1,d″2,d″3,…,d″ m},in Where L m Indicates label, L m The labels of the corresponding original data samples are the same.

[0018] Component 2 includes the following steps:

[0019] S106: As input, we fine-tune the large model CCT5 based on the code changes to optimize its parameters, θ prerepresents the pre-trained weights of CCT5, connects the [CLS] labeled representation of the last layer of the CCT5 encoder to the linear classifier, and uses θ pre Initialize CCT5 parameters.

[0020] S107: Divide D″ into a training set and a validation set, use the training set to fine-tune CCT5, randomly select a number of training samples from the training set to form a batch, calculate the loss after completing the training of a batch, and reversely update the model parameter θ of CCT5 according to the loss.

[0021] Specifically, the model parameters θ are updated through back propagation:

[0022]

[0023] Among them, α is the learning rate, is the loss function The gradient with respect to the model parameters θ.

[0024] S108: Input the test samples in the validation set into CCT5 after updating θ in S107, and calculate the F1 score. If the F1 score reaches the highest value and does not change, save the current model parameters as θ. best , and record the current CCT5 as the optimal CCT5, otherwise return to S107;

[0025] S109: For a piece of code text X test , X test Input optimal CCT5, optimal CCT5 output X test The predicted value y of whether there is a vulnerability test ∈{0,1}.

[0026] Furthermore, in S103, the calculation process of Jaccard similarity is as follows:

[0027]

[0028] Where |T i ′∩T j ′| represents the size of the intersection of two token sets, |T i ′∪T j ′| represents the union size of two token sets.

[0029] Furthermore, the step S106 includes the following steps: Before entering CCT5, and Use respectively <add>and <del>Mark connections;

[0030]

[0031] Among them, X add and X del Respectively represent the added items and deleted items after marking; x n and n They represent the nth added item and the nth deleted item respectively.

[0032] To adapt to the vulnerability detection task, the [CLS] tag output by the last layer of the encoder is represented by h CLS As a global representation, h CLS Connect to a linear classifier f linear , defined as:

[0033] h CLS =Encoder([X add ;X del ]) CLS

[0034] y=f linear (h CLS )

[0035] Among them, y∈(0,1) represents the final prediction result of the model.

[0036] Furthermore, in S107, the process of calculating the loss is as follows:

[0037] Define the loss function for:

[0038]

[0039] Among them, B represents a set of training samples in a batch, is the predicted probability of the i-th training sample (between 0 and 1), y i represents the true label (0 or 1) of the i-th sample.

[0040] Furthermore, in S108, at the end of each training step, the model is verified and the F1 score is calculated to evaluate its performance in the vulnerability detection task. The process of calculating the F1 score is as follows:

[0041]

[0042] Among them, Precision refers to the precision rate, which refers to the proportion of samples that are actually positive among all samples predicted by the model to be positive, and Recall refers to the recall rate, which refers to the proportion of samples that are correctly predicted as positive by the model among all samples that are actually positive.

[0043] Compared with the prior art, the present invention has at least the following advantages:

[0044] 1. The invention solves the technical problem by constructing two main components: differences between similar code pairs and detecting whether there are vulnerabilities. It aims to identify potential security vulnerabilities by analyzing the change process and semantic characteristics between codes with and without vulnerability features.

[0045] Component 1 is responsible for preprocessing the code data, which includes removing duplicates and comments and converting the code into token form. Then, by calculating the Jaccard similarity between the target code and the vulnerability-free code, the code pairs with the highest similarity are selected, and the change points between the two versions of the code are accurately identified, especially the newly added or deleted code segments, so that the model can focus on code changes that may have vulnerability characteristics.

[0046] Component 2 uses a large model based on code changes (CCT5) and further fine-tunes it to make it suitable for vulnerability detection tasks. In this process, the model is optimized using the training set, and the performance of the model is tested using the validation set. Finally, a set of optimal model parameter settings is determined to ensure that the model can perform best in real vulnerability detection scenarios.

[0047] 2. Performance improvement: Experimental results show that the present invention has significant improvements over other methods in key performance indicators such as Precision, Recall and F1-score, among which the F1-score reaches 55.62%, indicating that the model has achieved a better balance between precision and recall.

[0048] 3. Effectively identify vulnerabilities: The present invention is particularly outstanding in correctly identifying vulnerabilities, which shows that it can effectively identify potential security vulnerabilities from the code.

[0049] 4. Practical application value: The method of the present invention not only has superior performance in theory, but can also be applied in actual work scenarios to help developers detect and prevent security vulnerabilities during the software development process and reduce the risk of system failures and data leakage. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The figure is a simplified flow chart of the method of the present invention. DETAILED DESCRIPTION

[0051] The present invention is described in further detail below.

[0052] See also Figure 1 The present invention proposes a vulnerability detection method based on a large code change model. The method takes the semantic representation of the code editing sequence as the starting point, finds the difference between the code fragments without vulnerabilities, finds the changed code fragments with vulnerability characteristics, and then encodes the changed code fragments using a pre-trained model for classification prediction. The invention consists of a component one for constructing the difference between similar sample pairs and a component two for detecting whether the code has vulnerabilities.

[0053] Specifically, component one is responsible for constructing the difference between similar sample pairs. First, it obtains labeled data samples and preprocesses them, removes duplicate samples and comments, and retains the code text. Then, it tokenizes the code text and removes duplicate tokens to ensure that each code text contains only a unique token list. Then, it calculates the Jaccard similarity between the target code and the vulnerability-free code in the training set, and selects the most similar vulnerability-free code text to form a sample pair. Finally, it extracts the difference between the two code segments, including the added and deleted code segments, and marks these differences. Component two is used to detect vulnerabilities. First, it takes the difference as input, initializes the model CCT5 with pre-trained weights, adjusts the model structure, and sets key parameters. Then, it divides the data set into training set, validation set, and test set, fine-tunes CCT5 with the training set, and calculates the loss function to reversely update the model parameters. The model performance is evaluated on the validation set, and the optimal model parameters are selected by calculating the F1 score. Finally, the code is input into the optimal CCT5 model, and the prediction value of whether it has a vulnerability is output to achieve vulnerability detection.

[0054] A vulnerability detection method based on a large code change model includes a component one for constructing differences between similar sample pairs and a component two for detecting whether a vulnerability exists.

[0055] Component 1 includes the following steps:

[0056] S101: Obtain labeled data samples as the original data set D = {d1, d2, d3, ..., d n }, where d n represents the nth original data sample. The label is whether the original data sample has a vulnerability. The original data sample is preprocessed, including deleting duplicate data samples and removing comments in the code. Only the code text is retained to obtain a new data set D″={d′1,d′2,d′3,…,d′ m }, where d′ m represents the mth code text, m≤n, and the labels of the samples in D″ and the corresponding samples in D remain unchanged.

[0057] Specifically, duplicate samples are deleted to ensure the uniqueness of samples in the data set. The original data set is D = {d1, d2, d3, ..., d n}, the dataset after deleting duplicate samples is recorded as D′. The code text in each data sample in D′ is parsed, the comments in the code are removed, and only the code text is retained. The processed dataset is represented as D″={d′1,d′2,d′3,…,d′ m }, where m≤n.

[0058] S102: The i-th code text d′ in D″ i Tokenize to get token list T i ={t1,t2,t3,…,t k }, where t k Indicates T i The kth element in T i Perform deduplication operation so that T i Only unique tokens are retained, and the token set after deduplication is recorded as in Represents T′ i The p-th element in , p≤k, ensures that each code text only retains a unique token list.

[0059] Specifically, the code text d′ i Tokenize d′ i Split into token list T i ={t1,t2,t3,…,t k }, where each t j Represents a code token (such as a variable, operator, keyword, etc.).

[0060] For each token list T i Perform deduplication operation so that T i Only unique tokens are retained. The token set after deduplication is Where p≤k.

[0061] S103: For each T′ i , traverse all the samples T′ without loopholes in D″ j , calculate T′ i and T′ j The Jaccard similarity between them will find the most similar non-vulnerable code text and T′ i Composition sample pair

[0062] S104: The code snippet considered as "before change", T′ i The code snippet considered as "changed" is calculated Relative to T′ i Added code snippet Δ add , deleted code snippet Δ delete and the difference set

[0063] Specifically, assume that the code snippet "before the change" is The "changed" code snippet is T after =T′ i .calculate Relative to T′ i Added and deleted segments Δ add and Δ delete :

[0064]

[0065] Get the difference code snippet set Δ=(Δ add ,Δ delete ).

[0066] S105: Construct the final data set D″′={d″1,d″2,d″3,…,d″ m },in Where L m Indicates label, L m The labels of the corresponding original data samples are the same.

[0067] Component 2 includes the following steps:

[0068] S106: As input, we fine-tune the large model CCT5 based on the code changes to optimize its parameters, θ pre represents the pre-trained weights of CCT5, connects the [CLS] labeled representation of the last layer of the CCT5 encoder to the linear classifier, and uses θ pre Initialize CCT5 parameters.

[0069] S107: Divide D″ into a training set and a validation set, use the training set to fine-tune CCT5, randomly select a number of training samples from the training set to form a batch, calculate the loss after completing the training of a batch, and reversely update the model parameter θ of CCT5 according to the loss.

[0070] Specifically, the model parameters θ are updated through back propagation:

[0071]

[0072] Among them, α is the learning rate, is the loss function The gradient with respect to the model parameters θ.

[0073] S108: Input the test samples in the validation set into CCT5 after updating θ in S107, and calculate the F1 score. If the F1 score reaches the highest value and does not change, save the current model parameters as θ. best , and record the current CCT5 as the optimal CCT5, otherwise return to S107;

[0074] S109: For a piece of code text X test , X test Input optimal CCT5, optimal CCT5 output X test The predicted value y of whether there is a vulnerability test ∈{0,1}.

[0075] Furthermore, in S103, the calculation process of Jaccard similarity is as follows:

[0076]

[0077] Where |T i ′∩T j ′| represents the size of the intersection of two token sets, |T i ′∪T j ′| represents the union size of two token sets.

[0078] Furthermore, the step S106 includes the following steps: Before entering CCT5, and Use respectively <add>and <del>Mark connections;

[0079]

[0080] Among them, X add and X del Respectively represent the added items and deleted items after marking; x n and n They represent the nth added item and the nth deleted item respectively.

[0081] To adapt to the vulnerability detection task, the [CLS] tag output by the last layer of the encoder is represented by h CLS As a global representation, h CLS Connect to a linear classifier f linear , defined as:

[0082] h CLS =Encoder([X add ;X del ]) CLS

[0083] y=f linear (h CLS )

[0084] Among them, y∈(0,1) represents the final prediction result of the model.

[0085] Furthermore, in S107, the process of calculating the loss is as follows:

[0086] Define the loss function for:

[0087]

[0088] Among them, B represents a set of training samples in a batch, is the predicted probability of the i-th training sample (between 0 and 1), y i represents the true label (0 or 1) of the i-th sample.

[0089] Furthermore, in S108, at the end of each training step, the model is verified and the F1 score is calculated to evaluate its performance in the vulnerability detection task. The process of calculating the F1 score is as follows:

[0090]

[0091] Among them, Precision refers to the accuracy, which refers to the proportion of samples predicted by the model as positive that are actually positive; Recall refers to the recall rate, which refers to the proportion of samples correctly predicted by the model as positive among all samples that are actually positive.

[0092] Experiment and analysis

[0093] In order to verify the effectiveness of this method in vulnerability detection, the present invention uses a constructed merged dataset, which consists of three high-quality vulnerability datasets commonly used in vulnerability detection tasks, and all of them are established based on the actual C / C++ vulnerabilities in open source projects. Table 1 lists the data distribution of the training set, validation set, and test set used in the present invention.

[0094] Table 1 Dataset distribution statistics

[0095]

[0096] In order to improve the performance of the model in the code vulnerability detection task, the present invention pre-trains and fine-tunes the model, fully pre-processes and extracts features from data samples, and initializes the model using pre-trained weights. To ensure that the model can be stably trained and optimized on different data sets, the key parameters set for this pre-training and fine-tuning task are shown in Table 2.

[0097] Table 2 Key parameters

[0098]

[0099] After model training, under the optimal parameters, the performance of the present invention on the test data set is shown in Table 3. The experimental results show that the precision of this method reaches 47.37%, the recall reaches 67.36%, and the F1-score reaches 55.62%. Compared with other methods, this method has different degrees of improvement in various indicators, especially in correctly identifying vulnerabilities, while maintaining a good balance between precision and recall.

[0100] Table 3 Performance comparison of different methods

[0101]

[0102] The vulnerability detection method based on the code change large model proposed in the present invention shows remarkable performance in vulnerability detection and can effectively identify vulnerabilities in the code. At the same time, the present invention can be applied to actual work scenarios to help detect security vulnerabilities in the software development process, prevent potential security risks, and avoid system failures or data leakage caused by vulnerabilities.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.< / del> < / add> < / del> < / add>

Claims

1. A vulnerability detection method based on a large code change model, characterized in that: It includes component one for constructing differences between similar sample pairs and component two for detecting whether there is a vulnerability; Component 1 includes the following steps: S101: Obtain labeled data samples as the original data set D = {d1, d2, d3, ..., d n }, where d n represents the nth original data sample. The label is whether the original data sample has a vulnerability. The original data sample is preprocessed, including deleting duplicate data samples and removing comments in the code. Only the code text is retained to obtain a new data set D″={d′ i , d′2, d′3, ..., d′ m }, where d′ m represents the mth code text, m≤n, the labels of the samples in D″ and the corresponding samples in D remain unchanged; S102: The i-th code text d′ in D″ i Tokenize to get token list T i ={t1, t2, t3, ..., t k }, where t k Indicates T i The kth element in T i Perform deduplication operation so that T i Only unique tokens are retained, and the token set after deduplication is recorded as ,in Represents T′ i The p-th element in , p≤k, ensures that each code text only retains a unique token list; S103: For each T′ i , traverse all the samples T′ without loopholes in D″ j , calculate T′ i and T′ j The Jaccard similarity between them will find the most similar vulnerability-free code text T′ j* and T′ i The sample pair (T′ i , T′ j* ); S104: T′ j* The code snippet considered as "before change", T′ i Consider the code snippet "after the change" and calculate T' j* Relative to T′ i Added code snippet Δ add , deleted code snippet Δ delete and the difference set S105: Construct the final data set D″′={d″1,d″2,d″3,...,d″ m }, where d″ m =((T′ m , T′ n* ), Δ m , L m ), where L m Indicates label, L m The labels of the corresponding original data samples are the same; Component 2 includes the following steps: S106: As input, we fine-tune the large model CCT5 based on the code changes to optimize its parameters, θ pre represents the pre-trained weights of CCT5, connects the [CLS] labeled representation of the last layer of the CCT5 encoder to the linear classifier, and uses θ pre Initialize CCT5 parameters; S107: Divide D″ into a training set and a validation set, use the training set to fine-tune CCT5, randomly select a number of training samples from the training set to form a batch, calculate the loss after completing the training of a batch, and reversely update the model parameter θ of CCT5 according to the loss; S108: Input the test samples in the validation set into CCT5 after updating θ in S107, and calculate the F1 score. If the F1 score reaches the highest value and does not change, save the current model parameters as θ. best , and record the current CCT5 as the optimal CCT5, otherwise return to S107; S109: For a piece of code text X test , X test Input optimal CCT5, optimal CCT5 output X test The predicted value y of whether there is a vulnerability test ∈{0,1}.

2. A vulnerability detection method based on a code change model as claimed in claim 1, characterized in that: In S103, the calculation process of Jaccard similarity is as follows:

3. A vulnerability detection method based on a code change model as claimed in claim 2, characterized in that: The S106 includes the following steps: Before entering CCT5, and Use respectively <add>and <del> Mark connections;< / del> < / add> Among them, X add and X del Respectively represent the added items and deleted items after marking; x n and n They represent the nth added item and the nth deleted item respectively; The [CLS] tag output by the last layer of the encoder is represented as h CLS As a global representation, h CLS Connect to a linear classifier f linear , defined as: h CLS =Encoder([X add ;X del ]) CLS y=f linear (h CLS ) Among them, y∈(0,1) represents the final prediction result of the model.

4. A vulnerability detection method based on a code change model as claimed in claim 3, characterized in that: In S107, the process of calculating the loss is as follows: Define the loss function for: Among them, B represents a set of training samples in a batch, is the predicted probability y of the i-th training sample i represents the true label of the i-th sample.

5. A vulnerability detection method based on a code change model as claimed in claim 4, characterized in that: In S108, the process of calculating the F1 score is as follows: Among them, Precision refers to the accuracy, which refers to the proportion of samples predicted by the model as positive that are actually positive; Recall refers to the recall rate, which refers to the proportion of samples correctly predicted by the model as positive among all samples that are actually positive.

Images