Cross-project vulnerability detection model based on domain adaptation

Abstract

The present invention provides a cross-project vulnerability detection model based on domain adaptation. It learns project code features through a deep neural network and adopts a domain adaptation method to reduce the data distribution difference between the source project and the target project. The steps include: selecting the source project according to the software code metric similarity analysis; parsing the project source code to obtain the abstract syntax tree and performing code preprocessing; building a deep learning model for model pretraining; learning the deep feature representation of the source project and the target project based on a multi-domain dataset; using the semi-supervised metric transfer learning framework (Semi-Supervised Metric Transfer Learning, SSMTL) to perform domain adaptation processing on the deep features of the source project and the target project; training a classifier based on labeled data and performing vulnerability prediction on the data in the target project. This solution is applicable to source projects with sufficient labeled vulnerability data. By utilizing the vulnerability detection model, the vulnerability detection efficiency of the target project is improved, and labor costs are reduced.

Description

Technical Field

[0001] The present invention belongs to the field of software vulnerability detection, and more specifically, relates to a cross-project vulnerability detection model based on domain adaptation. Background Art

[0002] A software vulnerability refers to a flaw in the security of a computer system that poses a threat to the confidentiality, integrity, availability, and access control of the system or its application data. During system operation, cybersecurity incidents or data leaks caused by vulnerabilities occur frequently, and the number of vulnerabilities is increasing year by year. For example, the CVE vulnerability database, a well-known vulnerability database in the security field, registered approximately 4,500 vulnerabilities in 2010, but by 2021, the number of vulnerabilities had risen to over 17,000. These vulnerabilities affect the secure use of nearly 20,000 services or products, resulting in direct economic losses of over $200 billion annually. Although software developers have invested a lot of effort in the pursuit of secure programming, such as the use of static code analysis, dynamic code analysis, code review, and other technical and process improvements, software vulnerabilities are still difficult to avoid and become increasingly difficult to solve as software systems become more complex.

[0003] To identify vulnerabilities before software deployment, researchers have proposed a large number of techniques for software vulnerability detection. These techniques can be broadly categorized into two categories: learning-based and non-learning-based. Overall, learning-based vulnerability detection methods demonstrate superior performance compared to non-learning-based methods. Among existing learning-based vulnerability detection research, supervised learning techniques are the most widely used. Researchers train predictive models on a single dataset, assuming that the training and test datasets come from the same distribution. However, in practice, this idealized distribution assumption is often difficult to meet. Models trained on known vulnerability types and existing projects often fail to apply well to the target project, ultimately resulting in suboptimal model performance. This is primarily due to the fact that the target project cannot guarantee a uniform distribution of vulnerabilities, and new vulnerability types may exist. To address this issue, some research has begun exploring scenarios where training and test samples come from different projects, collectively referred to as cross-project vulnerability detection.

[0004] To address the data distribution discrepancies between source and target projects in cross-project vulnerability detection, many studies currently use transfer learning or domain adaptation methods. For example, adversarial generative transfer approaches are used for cross-project vulnerability detection. This approach uses a discriminator to shift the distribution of target project features to approximate that of the source project. However, the application of adversarial generative networks can lead to mode collapse, making model performance difficult to guarantee. Other studies use metric transfer learning to minimize the distribution discrepancy between the source and target domains to learn cross-domain representations. However, these approaches generally fail to consider the large amount of unlabeled data in the target domain, which compromises the effectiveness of domain adaptation. Summary of the Invention

[0005] The purpose of the present invention is to provide a cross-project vulnerability detection model based on domain adaptation to address the shortcomings of existing cross-project vulnerability detection models in code representation learning and reducing data distribution differences between projects. The model learns deep code representations from pre-trained multi-domain dataset models and merges them to obtain high-dimensional feature representations to obtain a wider range of coding patterns in project codes. The model uses a manifold semi-supervised metric transfer learning framework to perform domain adaptation on the source and target projects to reduce data distribution differences between projects, ultimately improving the performance of cross-project vulnerability detection.

[0006] To achieve the above objectives, the technical solution of the present invention is to provide a cross-project vulnerability detection model based on domain adaptation, and the implementation scheme includes the following steps:

[0007] S1: Obtain the code metrics of the source project set and the target project, use Euclidean distance to measure the code metric similarity between the projects, and select the project closest to the target project as the source project;

[0008] S2: Use code parsing tools to scan the project source code, parse the abstract syntax tree, traverse the abstract syntax tree nodes, generate a sequential token sequence, and normalize the sequence;

[0009] S3: Use the Word2vec model to embed the generated sequence, input the generated corresponding vector into the Bi-LSTM neural network model for training, and save the model with the best results;

[0010] S4: Use cross-project feature learning based on multi-domain datasets to learn deep feature representations of source and target project codes from pre-trained models;

[0011] S5: Use the SSMTL framework to perform domain adaptive learning on the deep features and labels in the source and target projects, generate a distance metric matrix, and transform the features to reduce the data distribution differences between the projects;

[0012] S6: Use linear discriminant analysis as a classifier to train the converted labeled code features and perform vulnerability prediction on the target project's data. Finally, output the function information predicted as the vulnerability and verify the classifier's prediction results. Compared to existing cross-project vulnerability detection models, this method innovatively incorporates feature learning based on multi-domain datasets into cross-project vulnerability prediction and considers the large amount of unlabeled data in the target project during domain adaptation processing.

[0013] Preferably, the S1 step includes: selecting a source project list, scanning and calculating code metrics of the source project set and the target project; calculating five features of each type of metric: maximum, minimum, average, median, and standard deviation, to generate a multidimensional feature set; calculating the Euclidean distance of the feature vectors between the source project and the target project to measure the similarity of data distribution between the projects, and selecting the project with the highest similarity as the source project for subsequent model training. The calculation formula is as follows:

[0014]

[0015] Where C1 represents the source item feature vector, C2 represents the target item feature vector, and p represents the dimension of the feature vector.

[0016] Preferably, the S2 step includes: obtaining the source code of the source project and the target project; using the code scanning tool CodeSensor to parse the source code and generate a serialized form of the abstract syntax tree for each function; traversing each node of the abstract syntax tree using a depth-first traversal method to obtain a sequential token sequence; deleting possible noise data in the sequence, including characters, string text, spaces, and semicolons, etc., and retaining only the code content; in order to avoid interference from custom content in the code in the deep learning algorithm, using the source code obfuscation tool Snob to obfuscate the names of user-defined functions, variables, and parameters, but the code structure, library / API functions, and keywords remain unchanged; using padding or truncation operations to ensure the uniformity of sequence length to facilitate subsequent use as model input.

[0017] Specifically, the abstract syntax tree generated in the S2 step consists of three types of nodes: placeholder nodes, API nodes, and syntax nodes. Placeholder nodes are used to connect function components. For example, "params" indicates that its leaf nodes are function parameters, and "stmts" indicates that its leaf nodes are various types of statements. API nodes refer to function return values ​​and function parameter types, and can also be variable declarations and function calls. For example, a function with a return type of "void" will have a "void" node, and a function that accepts an "int" parameter will have an "int" node. Syntax nodes are grammatical elements that contain control flow and operators. Control flow elements include while statements, if / else statements, etc. Operators such as "+", "-", or "=" remain unchanged in the abstract syntax tree.

[0018] Specifically, in step S2, the long sequence needs to be truncated, and the end of the short sequence is padded with 0 as padding.

[0019] Preferably, the S3 step includes: using the Word2vec model to perform embedding training on the input data and converting the corresponding token sequence into a high-dimensional vector feature; building a Bi-LSTM neural network model to train the input vector; and saving the model with the best performance during the training process to facilitate subsequent extraction of deep features of the code.

[0020] Specifically, the Bi-LSTM used in step S3 can learn vulnerability feature representations because vulnerable functions typically contain a large number of code statements. Although the truly vulnerable code often consists of only one line, the code before or after the vulnerable line can have a significant impact on it. This contextual information is crucial for revealing vulnerable code patterns. Bi-LSTM is able to effectively capture the long-term dependencies implicit in this contextual information, thereby learning vulnerability-related code patterns. When using a Bi-LSTM neural network for deep feature learning, the specific structure of the neural network can be customized. Feature extraction is performed by connecting Bi-LSTM layers, and the predicted output value of a layer of the neural network is then used as the deep feature representation of the input vector. The more Bi-LSTM layers connected, the more abstract the corresponding extracted features.

[0021] Preferably, the S4 step includes: inputting the token sequences of the source project and the target project into the pre-trained source project model and the Software Assurance Reference Dataset (SARD) project model respectively, and outputting the global maximum pooling layer prediction value in the model as the deep feature representation of the project code; and merging the deep features learned from the two independent models to obtain a higher-dimensional feature set.

[0022] Preferably, the S5 step includes: using the SSMTL framework to perform domain adaptive processing on the deep features of the source project and the target project, and learning the distance measurement matrix; and using the learned distance measurement matrix to transform the feature matrices of the source project and the target project.

[0023] Specifically, the SSMTL used in step S5 is a manifold semi-supervised metric learning framework. When dealing with cross-project vulnerability detection scenarios, SSMTL mainly performs domain adaptation on the deep feature representations in the source and target projects from the following three aspects:

[0024] (1) Learn a regularized distance metric to find an optimal distance metric that increases the distance between vulnerability and non-vulnerability data samples and decreases the distance between data samples that are both classified as vulnerability or non-vulnerability. In addition to considering the input feature vector, the learning of the regularized distance metric can also measure the appropriate distance function by providing external information or side information in the form of data.

[0025] (2) Use covariate shift adaptation to accurately minimize the distribution difference between source and target items. The regularization term that affects covariate shift is defined as follows:

[0026]

[0027] where ν0(x i ) is the data sample x i The initial weight under the Euclidean distance metric in the source project is obtained by ν0(x i ) can determine the value of the data sample x i The degree of similarity with the source project data or the target project data. If its value is high, x is higher than that of the source project. i Closer to the target project.

[0028] (3) Using the graph-based manifold hypothesis, based on the principle that if two samples have a high correlation with respect to a certain metric on the manifold, they remain similar when projected, the geometric shape of the data is modeled as a graph, and the distance between any sample and its neighboring nodes is measured. The data constitute the vertices of the graph, and the edges represent appropriate neighborhood relationships. Assuming G = (V, E) is a connected graph, V corresponds to N data samples of the source project and the target project respectively, and E represents the relationship between data samples, then any two samples x i and x j The similarity calculation formula between is defined as follows:

[0029]

[0030] where σ ij 2 =σi σ j ,σ i is the element x i The distance between it and its K nearest neighbors.

[0031] Preferably, the S6 step includes: selecting a linear discriminant analysis classifier and training it on labeled data in the source project and the target project; applying the trained classifier to the unlabeled data of the target project for classification prediction and outputting the vulnerability prediction result; and verifying the effect of the classifier based on the test set data.

[0032] The main idea of ​​the linear discriminant analysis classifier used in step S6 is to maximize the between-class mean and minimize the intra-class variance, that is, to project the data onto a low dimension, and after projection, the projection points of the data of the same category are as close as possible, and the center points of the projection points of the data of different categories are as far as possible.

[0033] In step S6, the performance of the classifier is evaluated using the following evaluation metrics: precision, recall, false positive rate (FPR), false negative rate (FNR), F1-measure (F1), and Matthews score (MCC). The definitions of each evaluation metric are as follows: TP is the number of correctly classified methods for vulnerable methods, TN is the number of correctly classified methods for non-vulnerable methods, FN is the number of incorrectly classified methods for vulnerable methods, and FP is the number of incorrectly classified methods for non-vulnerable methods.

[0034] Recall represents the ratio of the number of methods correctly predicted to be vulnerable to the number of methods with true labels as vulnerable, and is defined as:

[0035]

[0036] Precision represents the ratio of the number of methods correctly predicted to be vulnerable to the number of methods predicted to be vulnerable, and is defined as:

[0037]

[0038] FPR represents the ratio of the number of methods that are incorrectly predicted as non-vulnerable to the number of methods with true labels as non-vulnerable, and is defined as:

[0039]

[0040] FNR represents the ratio of the number of methods that are incorrectly predicted to be vulnerable to the number of methods with true labels as vulnerable, and is defined as:

[0041]

[0042] The F1 value is a comprehensive evaluation indicator that represents the harmonic average of recall and precision, and is defined as:

[0043]

[0044] MCC measures the similarity of two distributions by considering the predicted value and the actual value as two 0-1 distributions and is defined as:

[0045]

[0046] In general, the above technical solution conceived by the present invention has the following beneficial effects compared with the prior art:

[0047] (1) Compared with the existing cross-project vulnerability detection model, this paper innovatively uses a cross-project feature learning method based on multi-domain datasets in code representation learning, learning feature representations from two independent deep feature models and merging them into a more dimensional feature set to learn a wider range of coding patterns.

[0048] (2) Compared with the existing cross-project vulnerability detection model, the present invention uses the manifold semi-supervised metric transfer learning method to perform domain adaptive processing on the deep features of the source project and the target project, taking into account the large amount of unlabeled data in the target project, and achieves better results in reducing the data distribution differences between projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of a process flow of a domain-adaptive cross-project vulnerability detection model provided by an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of parsing an abstract syntax tree from source code disclosed in an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of cross-project feature learning based on multi-domain data disclosed in an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the process of performing cross-project vulnerability detection using a pre-trained model disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention.

[0054] Example 1: Please refer to Figures 1 to 4As shown, the present invention provides a cross-project vulnerability detection model based on domain adaptation, comprising the following steps:

[0055] S1: Obtain the code metrics of the source project set and the target project, use Euclidean distance to measure the code metric similarity between the projects, and select the project closest to the target project as the source project;

[0056] S2: Use a code parsing tool to scan the project source code, parse the abstract syntax tree for each function in the code, traverse the abstract syntax tree nodes, generate a sequential token sequence, and normalize the sequence;

[0057] S3: Use the Word2vec model to embed the generated sequence, input the generated corresponding vector into the Bi-LSTM neural network model for training, and save the model with the best results;

[0058] S4: Use cross-project feature learning based on multi-domain datasets to learn deep feature representations of source and target project codes from pre-trained models;

[0059] S5: Use the SSMTL framework to perform domain adaptive learning on the deep features and labels in the source and target projects, generate a distance metric matrix, and transform the features to reduce the data distribution differences between the projects;

[0060] S6: Use linear discriminant analysis as a classifier to train the converted labeled code features, and predict vulnerabilities on the target project data. Finally, output the function information predicted as the vulnerability and verify the prediction effect of the classifier.

[0061] The S1 step includes: selecting a source project list, scanning and calculating code metrics for the source project set and the target project. Plugins / code scanning tools including Source Code Metrics and C Metrics can be used to calculate source code metrics from multiple dimensions such as packages, classes, and files. The ratio metrics need to be calculated based on the scanning results; calculating the five features of each type of metric: maximum, minimum, average, median, and standard deviation to generate a multi-dimensional feature set; calculating the Euclidean distance of the feature vectors between the source project and the target project to measure the similarity of the data distribution between the projects, and selecting the project with the highest similarity as the source project for subsequent model training. The calculation formula is as follows:

[0062]

[0063] Where C1 represents the source item feature vector, C2 represents the target item feature vector, and p represents the dimension of the feature vector.

[0064] The S2 step includes: obtaining the source code of the source project and the target project; using the code scanning tool CodeSensor to parse the source code and generate a serialized form of the abstract syntax tree for each function; traversing each node of the abstract syntax tree using a depth-first traversal method to obtain a sequential token sequence; deleting noise data that may exist in each line of code, including characters, string text, spaces, and semicolons, etc., and retaining only the code content; in order to avoid interference from custom content in the code in the deep learning algorithm, using the source code obfuscation tool Snob to obfuscate the names of user-defined functions, variables, and parameters, but the code structure, library / API functions, and keywords remain unchanged; using padding or truncation operations to ensure the uniformity of sequence length to facilitate subsequent use as model input.

[0065] Specifically, the abstract syntax tree generated in the S2 step consists of three types of nodes: placeholder nodes, API nodes, and syntax nodes. Placeholder nodes are used to connect function components. For example, "params" indicates that its leaf nodes are function parameters, and "stmts" indicates that its leaf nodes are various types of statements. API nodes refer to function return values ​​and function parameter types, and can also be variable declarations and function calls. For example, a function with a return type of "void" will have a "void" node, and a function that accepts an "int" parameter will have an "int" node. Syntax nodes are grammatical elements that contain control flow and operators. Control flow elements include while statements, if / else statements, etc. Operators such as "+", "-", or "=" remain unchanged in the abstract syntax tree.

[0066] Specifically, in step S2, the long sequence needs to be truncated, and the end of the short sequence needs to be padded with 0.

[0067] The S3 step includes: using the Word2vec model to embed the input data and convert the corresponding token sequence into a high-dimensional vector feature; building a Bi-LSTM neural network model to train the input vector; and saving the model with the best performance during the training process to facilitate the subsequent extraction of deep features of the code.

[0068] Specifically, the Bi-LSTM used in step S3 can learn the feature representation of vulnerabilities because vulnerable functions typically contain a large number of code statements. Although the truly vulnerable code often consists of only one line, the code before or after the vulnerable line can have a significant impact on it. This contextual information is crucial for revealing vulnerable code patterns. Bi-LSTM can effectively capture the long-term dependencies implicit in this contextual information, thereby learning code patterns related to vulnerabilities. When using a Bi-LSTM neural network for deep feature learning, the specific structure of the neural network can be customized. Feature extraction is performed by connecting Bi-LSTM layers, and the predicted output value of a layer of the neural network is then used as the deep feature representation of the input vector. The more Bi-LSTM layers are connected, the more abstract the corresponding extracted features are.

[0069] The S4 step includes: learning deep feature representations of the source project and target project codes from the pre-trained source project model and SARD project model respectively; and merging the deep features learned from the two independent models to obtain a higher-dimensional feature set.

[0070] The S5 step includes: using the SSMTL framework to perform domain adaptive processing on the deep features of the source project and the target project, and learning the distance measurement matrix; using the learned distance measurement matrix to transform the feature matrices of the source project and the target project.

[0071] When dealing with cross-project vulnerability detection scenarios, SSMTL mainly performs domain adaptation on the deep feature representations in the source and target projects from the following three aspects:

[0072] (1) Learn a regularized distance metric to find an optimal distance metric that increases the distance between vulnerability and non-vulnerability data samples and decreases the distance between data samples that are both classified as vulnerability or non-vulnerability. In addition to considering the input feature vector, the learning of the regularized distance metric can also measure the appropriate distance function by providing external information or side information in the form of data.

[0073] (2) Use covariate shift adaptation to accurately minimize the distribution difference between source and target items. The regularization term that affects covariate shift is defined as follows:

[0074]

[0075] where v0(x i ) is the data sample x i The initial weight under the Euclidean distance metric in the source project is obtained by v0(x i ) can determine the value of the data sample x iThe degree of similarity with the source project data or the target project data. If its value is high, x is higher than that of the source project. i Closer to the target project.

[0076] (3) Using the graph-based manifold hypothesis, based on the principle that if two samples have a high correlation with respect to a certain metric on the manifold, they remain similar when projected, the geometric shape of the data is modeled as a graph, and the distance between any sample and its neighboring nodes is measured. The data constitute the vertices of the graph, and the edges represent appropriate neighborhood relationships. Assuming G = (V, E) is a connected graph, V corresponds to N data samples of the source project and the target project respectively, and E represents the relationship between data samples, then any two samples x i and x j The similarity calculation formula between is defined as follows:

[0077]

[0078] where σ ij 2 =σ i σ j ,σ i is the element x i The distance between it and its K nearest neighbors.

[0079] The S6 step includes: selecting a linear discriminant analysis classifier and training it on labeled data; applying the trained classifier to the unlabeled data of the target project to perform classification prediction and output vulnerability prediction results; and verifying the effectiveness of the classifier based on a test data set.

[0080] The main idea of ​​the linear discriminant analysis classifier used in step S6 is to maximize the between-class mean and minimize the intra-class variance, that is, to project the data onto a low dimension, and after projection, the projection points of the data of the same category are as close as possible, and the center points of the projection points of the data of different categories are as far as possible.

[0081] In step S6, the performance of the classifier is evaluated using the following evaluation metrics: precision, recall, false positive rate (FPR), false negative rate (FNR), F1-measure (F1), and Matthews score (MCC). The definitions of each evaluation metric are as follows: TP is the number of correctly classified methods for vulnerable methods, TN is the number of correctly classified methods for non-vulnerable methods, FN is the number of incorrectly classified methods for vulnerable methods, and FP is the number of incorrectly classified methods for non-vulnerable methods.

[0082] Recall represents the ratio of the number of methods correctly predicted to be vulnerable to the number of methods with true labels as vulnerable, and is defined as:

[0083]

[0084] Precision represents the ratio of the number of methods correctly predicted to be vulnerable to the number of methods predicted to be vulnerable, and is defined as:

[0085]

[0086] FPR represents the ratio of the number of methods that are incorrectly predicted as non-vulnerable to the number of methods with true labels as non-vulnerable, and is defined as:

[0087]

[0088] FNR represents the ratio of the number of methods that are incorrectly predicted to be vulnerable to the number of methods with true labels as vulnerable, and is defined as:

[0089]

[0090] The F1 value is a comprehensive evaluation indicator that represents the harmonic average of recall and precision, and is defined as:

[0091]

[0092] MCC measures the similarity of two distributions by considering the predicted value and the actual value as two 0-1 distributions and is defined as:

[0093]

[0094] Example 2:

[0095] In this embodiment, the implementation process and result analysis of cross-project vulnerability detection on a historical vulnerability dataset of open source projects are mainly described. The dataset contains vulnerabilities and non-vulnerability functions from four open source projects, including FFmpeg, LibTIFF, LibPNG, and Asterisk projects. Among them, the FFmpeg project provides a collection of libraries and tools for processing multimedia content, the LibTIFF project provides open source libraries and toolkits for operating TIFF images, the LibPNG project is a PNG image support library that can perform operations such as reading and writing PNG images, and the Asterisk project is an open source framework for building communication applications. These projects were selected because they have long iteration cycles and a relatively large number of vulnerabilities reported by developers or researchers and published in public vulnerability databases, which basically meet the requirements for deep learning model training. This embodiment parses the abstract syntax tree by scanning the project source code and trains the deep learning model based on the source project data labels. On this basis, the cross-project deep feature representation of the code is learned, and after domain adaptation processing, the classifier is trained. Then, vulnerability prediction is performed on the code in the target project, thereby improving the performance of vulnerability detection as much as possible and reducing the cost of manual inspection.

[0096] Step 1: Obtain code metric indicators of the source project set and the target project, and perform code metric similarity analysis.

[0097] The code metric scanning tool is used to scan the source code of the source project set and the target project, and the 20 code metric indicators shown in Table 1 are calculated based on the scanning results.

[0098] Table 1

[0099]

[0100] After obtaining the code metrics of the project, we calculate the five features of each metric: maximum, minimum, mean, median, and standard deviation, and generate a multi-dimensional feature set. We then use Euclidean distance to calculate the distance between the feature vectors of the source and target projects to measure the similarity of the data distribution between the projects. We then select the project with the highest similarity as the source project for subsequent model training. The calculation formula is as follows:

[0101]

[0102] Where C1 represents the source item feature vector, C2 represents the target item feature vector, and p represents the dimension of the feature vector.

[0103] Four projects are selected as target projects, and the selected source projects are shown in Table 2.

[0104] Table 2

[0105] Target Project Source Project FFmpeg LibTIFF LibTIFF FFmpeg LibPNG LibTIFF Asterisk LibTIFF

[0106] Step 2: Parse the project source code, generate an abstract syntax tree for each function, traverse the abstract syntax tree nodes, generate a sequential token sequence, and normalize the sequence.

[0107] Get the source code of the source project and the target project, use the code parsing tool CodeSensor to scan the project source code, parse the abstract syntax tree, and Figure 2 Take the C function shown in (a) as an example, its corresponding abstract syntax tree is as follows Figure 2 As shown in (b), after using CodeSensor to parse the code, the abstract syntax tree serialization form is obtained as follows Figure 2 (c) shown.

[0108] A depth-first traversal is used to traverse each node of the abstract syntax tree, resulting in a sequential token sequence. During the traversal, each node of the abstract syntax tree is mapped to an element in a sequence. All nodes in the function together form a text sequence representing the semantic information of the code. The token sequence after traversal is of the form {int, func_name, params, param, int, stmnts, decl,..., return}. This method converts the function-level source code in a project into a code sequence. Each sequence is then labeled as a vulnerability or non-vulnerability and aligned with the label of the original function.

[0109] Delete any noise data that may exist in the sequence, including characters, string text, spaces, and semicolons, and only keep the code content.

[0110] To prevent custom content in the code from interfering with deep learning algorithms, the source code obfuscation tool Snob is used to obfuscate the names of user-defined functions, variables, and parameters, but the code structure, library / API functions, and keywords remain unchanged.

[0111] Padding or truncation is used to ensure uniform sequence lengths for subsequent model input. Analysis of token sequence lengths across projects revealed that while sequence length distribution varies significantly, 95% of sequences are under 1000. Therefore, a sequence length threshold of 1000 is chosen to balance the number of sequence elements and sequence sparsity. Long sequences are truncated to 1000, while short sequences are padded with zeros.

[0112] Step 3: Use the Word2vec model to embed the input data and convert the corresponding token sequence into high-dimensional vector features. Use the Bi-LSTM neural network to build the model. The model is trained based on the data samples and labels in the source project. The Bi-LSTM model consists of an input layer, an embedding layer, two Bi-LSTM neural networks, a global maximum pooling layer, and three Dense layers.

[0113] Save the model with the best performance during training so that you can extract deep features of the code later.

[0114] Step 4: Use cross-project feature learning based on multi-domain datasets to learn deep feature representations of source and target project codes from the pre-trained model.

[0115] The processed token sequences of the source project and target project are input into the pre-trained source project model and SARD project model, and the prediction value output of the global maximum pooling layer in the model is used as the deep feature representation of the project code.

[0116] like Figure 3 As shown in Figure 2, the deep features learned from two independent models are merged to obtain a higher-dimensional feature set.

[0117] Step 5: Use the SSMTL framework to perform domain adaptive learning on the deep features and labels in the source and target projects, generate a distance metric matrix, and transform the features.

[0118] Step 6: Use linear discriminant analysis as a classifier to train the converted labeled code features and perform vulnerability prediction on the target project’s data.

[0119] A linear discriminant analysis classifier is selected and trained on labeled data.

[0120] Apply the trained classifier to the unlabeled data of the target project for classification prediction and output the vulnerability prediction results.

[0121] The deep feature representation of the target item learned in step 5 is input into the classifier, and the performance of the trained classifier is evaluated according to the evaluation indicators, and the results are shown in Table 3.

[0122] Table 3

[0123] Source Project Target Project Precision Recall FPR FNR F1 MCC FFmpeg LibTIFF 0.7053 0.7283 0.0943 0.2717 0.7166 0.6271 LibTIFF FFmpeg 0.6514 0.6961 0.129 0.3039 0.673 0.5552 LibTIFF LibPNG 0.6765 0.7188 0.1019 0.2812 0.697 0.6041 LibTIFF Asterisk 0.7368 0.4058 0.0439 0.5942 0.5234 0.4576

[0124] The domain-adaptive cross-project vulnerability detection model proposed in this invention has improved evaluation indicators compared with existing cross-project vulnerability detection work, verifying the effectiveness of the model.

[0125] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A cross-project vulnerability detection model based on domain adaptation, characterized by: The following steps are involved: S1: Obtain the code metrics of the source project set and the target project, use Euclidean distance to measure the code metric similarity between the projects, and select the project closest to the target project as the source project; S2: Use a code parsing tool to scan the project source code, generate an abstract syntax tree for each function in the source code, traverse all nodes in the abstract syntax tree, generate a sequential token sequence, and normalize the sequence. Label the function based on whether it has been reported in the Common Vulnerability and Exposure (CVE) database. The label indicates whether the function code has a vulnerability or not. S3: Use the Word2vec model to embed the generated sequence, input the generated corresponding vector into the Bi-directional Long Short-Term Memory (Bi-LSTM) neural network model for training, and save the model with the best results; S4: Use cross-project feature learning based on multi-domain datasets to learn deep feature representations of source and target project codes from pre-trained models; S5: Use the SSMTL framework to perform domain adaptive learning on the deep features and labels in the source and target projects, generate a distance metric matrix, and transform the features to reduce the data distribution differences between the projects; S6: Use linear discriminant analysis as a classifier to train the converted labeled code features and predict vulnerabilities on the target project data. Finally, output the function information predicted as the vulnerability and verify the prediction effect of the classifier. Wherein, the S5 comprises the following steps: S5-1: Use the SSMTL framework to perform domain adaptation on the deep features of the source and target items and learn the distance metric matrix; S5-2: Use the learned distance metric matrix to transform the feature matrices of the source and target items; The SSMTL framework used in this step is a manifold semi-supervised metric learning framework. When dealing with cross-project vulnerability detection scenarios, SSMTL performs domain adaptation on the deep feature representations in the source and target projects from the following three aspects: (1) Learning regularized distance metrics, thereby seeking an optimal distance metric that increases the distance between vulnerability and non-vulnerability data samples, and reduces the distance between data samples classified as both vulnerability and non-vulnerability. In addition to considering the input feature vector, the learning of regularized distance metrics measures the appropriate distance function by providing external information or side information in the form of data. (2) Use covariate shift adaptation to accurately minimize the distribution difference between source and target items. The regularization term that affects covariate shift is defined as follows: where v0(x i ) is the data sample x i The initial weight under the Euclidean distance metric in the source project is obtained by ν0(x i ) determines the value of the data sample x i The degree of similarity with the source project data or the target project data. If its value is higher, the x i Closer to the target project; (3) Using the graph-based manifold hypothesis, based on the principle that if two samples have a high correlation with respect to a certain metric on the manifold, they remain similar when projected, the geometric shape of the data is modeled as a graph, measuring the distance between any sample and its neighbor nodes, where the data constitute the vertices of the graph and the edges represent appropriate neighborhood relationships. Assuming G = (V, E) is a connected graph, V corresponds to N data samples of the source project and the target project respectively, and E represents the relationship between data samples, then any two samples x i and x j The similarity calculation formula between is defined as follows: where σ ij 2 =σ i σ j ,σ i is the element x i The distance between it and its K nearest neighbors.

2. The domain-adaptive cross-project vulnerability detection model according to claim 1 is characterized in that: The S1 comprises the following steps: S1-1: Select a source project list, scan and calculate code metrics for the source project set and target project. Use plugins / code scanning tools including Source Code Metrics and C Metrics to calculate source code metrics from multiple dimensions: packages, classes, and files. Ratio metrics are calculated based on the scan results. S1-2: Calculate the five characteristics of each type of metric: maximum, minimum, mean, median, and standard deviation, and generate a multidimensional feature set; S1-3: Calculate the Euclidean distance between the feature vectors of the source and target items to measure the data distribution similarity between the items, and select the item with the highest similarity as the source item for subsequent model training. The calculation formula is as follows: Where C1 represents the source item feature vector, C2 represents the target item feature vector, and p represents the dimension of the feature vector.

3. The domain-adaptive cross-project vulnerability detection model according to claim 1 is characterized in that: The S2 comprises the following steps: S2-1: Get the source code of the source project and target project; S2-2: Use the code scanning tool CodeSensor to parse the source code and generate a serialized form of the abstract syntax tree for each function; S2-3: Use depth-first traversal to traverse each node of the abstract syntax tree to obtain a sequential token sequence; S2-4: Delete the noise data in the sequence, including characters, string text, spaces, and semicolons, and only keep the code content; S2-5: To prevent custom content in the code from interfering with the deep learning algorithm, we use the source code obfuscation tool Snob to obfuscate the names of user-defined functions, variables, and parameters. However, the code structure, libraries, API functions, and keywords remain unchanged. S2-6: Use padding or truncation operations to ensure the uniformity of sequence length to facilitate subsequent use as model input.

4. The domain-adaptive cross-project vulnerability detection model according to claim 1 is characterized in that: The S3 includes the following steps: S3-1: Use the Word2vec model to embed the input data and convert the corresponding token sequence into high-dimensional vector features; S3-2: Build a Bi-LSTM neural network model and train the input vector; S3-3: Save the model with the best performance during training so that you can extract deep features of the code later.

5. The domain-adaptive cross-project vulnerability detection model according to claim 1 is characterized in that: The S4 comprises the following steps: S4-1: Input the token sequences of the source project and target project into the pre-trained source project model and the Software Assurance Reference Database (SARD) project model respectively, and output the prediction value of the global maximum pooling layer in the model as the deep feature representation of the project code; S4-2: Merge the deep features learned from two independent models to obtain a higher-dimensional feature set.

6. The domain-adaptive cross-project vulnerability detection model according to claim 1 is characterized in that: The S6 comprises the following steps: S6-1: Select a linear discriminant analysis classifier and train it on labeled data from the source and target projects. S6-2: Apply the trained classifier to the unlabeled data of the target project to perform classification prediction and output vulnerability prediction results; S6-3: Verify the effect of the classifier based on the test set data; The idea of ​​the linear discriminant analysis classifier used in step S6-1 is to maximize the between-class mean and minimize the within-class variance, that is, to project the data onto a low dimension, and after projection, the projection points of data of the same category are as close as possible, and the center points of the projection points of data of different categories are as far as possible; In step S6-3, the performance of the classifier is evaluated using evaluation indicators such as precision, recall, false positive rate (FPR), false negative rate (FNR), F1-measure, and Matthews-score.

Images