Method for eliminating false positives of vulnerabilities facing continuous integration platforms

Abstract

The application provides a vulnerability false alarm elimination method for a continuous integration platform, which captures vulnerability warning code features generated by program static analysis through a deep learning neural network, sorts and classifies the warnings, thereby eliminating false alarms in the warnings, and updates and optimizes the model using incremental warning data generated in the project continuous integration process. The steps include: obtaining historical warning data and corresponding warning label data of a Java project on a continuous integration platform; performing program slicing, tokenization and vectorization on the program corresponding to the warning; training a deep learning neural network to obtain a vulnerability false alarm elimination model; and adding the model to the continuous integration process of the project to enable the model to be continuously updated and optimized with project iterations. The application reduces the time cost of developers checking program static analysis vulnerability warnings in the project continuous integration process and improves the efficiency of processing warnings.

Description

Methods for eliminating false positives in continuous integration platforms Technical Field

[0001] This invention relates to the fields of continuous integration and static program analysis, and specifically to a method for eliminating false positives of vulnerabilities in continuous integration platforms. Background Technology

[0002] As software development iteration cycles become increasingly shorter, software needs to be tested and integrated more quickly, making continuous integration a popular software development practice. To promptly identify security vulnerabilities introduced during development in this high-speed development model, while reducing the cost of vulnerability discovery and remediation, incorporating static analysis techniques into the continuous integration process has become a promising solution for automated code security auditing. However, because static analysis tools often consider only a limited number of error cases when designing their analysis rules, the resulting rules may lack knowledge of how data flows through the system, inter-statement dependencies, and the overall software architecture. This leads to a large number of false positives. These false positives not only increase the time cost for developers to check reports generated by the tools but also cause abnormal build interruptions in continuous integration, thus causing developer resistance to using static analysis tools and limiting their widespread adoption in the software development process.

[0003] To address the issue of numerous false positives in static analysis tools, researchers have proposed several adjustments and improvements to reduce their impact on software development. Among these, warning ranking and classification fall under the category of post-processing related to this invention: Warning ranking calculates the probability that each warning is a genuine warning based on various characteristics of the warning and source code, warning fix history, and code change history, ranking the warning list so that warnings more likely to reflect real problems are prioritized; Warning classification categorizes warnings into genuine warnings and false positives, with false positives not reported to developers, thus eliminating false positives. Most existing research uses statistical or machine learning modeling for warning ranking and classification. However, these techniques require not only collecting warning-related information but also a large amount of information related to project details and developer commit history for modeling, making integration with real-world continuous integration platforms challenging. Therefore, current research typically aggregates warnings from different project versions into a single dataset to build a false positive elimination model all at once, without integrating the model into the continuous integration process. Furthermore, existing methods focus on the observable features of statically analyzed warnings, such as vulnerability type and warning priority. The definition of these features often relies on the experience of experts and time-consuming research, and often ignores the deep structure of the source code corresponding to the analyzed warning, inevitably leading to a loss of model accuracy. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing models, such as difficulty in integrating with continuous integration platforms and the neglect of the deep structure of source code in warnings. This invention provides a method for eliminating false positives in continuous integration platforms. The model uses the code representation corresponding to the warning as learning features, avoiding complex feature engineering. Furthermore, it uses a deep learning neural network to learn the long-term, bidirectional dependencies in the program, thereby capturing deep features at the source code level. In terms of presentation, this invention combines the advantages of existing warning ranking and warning classification models. It predicts the confidence level of each warning, representing the probability that it is a genuine warning, and then classifies the warnings into genuine warnings and false positives based on a threshold set by the developers.

[0005] To achieve the above objectives, the technical solution of the present invention is: a method for eliminating false positives of vulnerabilities in continuous integration platforms, comprising the following steps:

[0006] S1: Use the SpotBugs static analysis tool to perform static analysis on the project source code, obtain source code warnings for each commit version of the project, manually annotate the obtained warnings, and determine whether the code contains real vulnerabilities;

[0007] S2: Use a program slicing algorithm based on Program Dependence Graph (PDG) to slice the program corresponding to the warning, and obtain the statements that have data dependency or control dependency with the code line corresponding to the warning as program slices;

[0008] S3: Mark and normalize the program slices to obtain the Token sequence, embed the Token sequence into a vector form using Word2vec, and obtain the input of the neural network;

[0009] S4: Input the vector generated in S3 into a bidirectional long short-term memory (Bi-LSTM) neural network for training to build a vulnerability false alarm elimination model. Input the original warning to be predicted, and the model outputs the confidence score and classification label of the warning.

[0010] S5: The original model is incorporated into the project's continuous integration process. Each project commit generates new warning data on the platform, which serves as an incremental training set for incremental model training. This allows the model to continuously update and optimize during project iterations, adjusting its parameters based on developers' actual warning flagging behavior. Compared to traditional false alarm elimination models trained using software engineering metrics, this invention only requires the corresponding line number and source code as raw input, eliminating the need for complex feature engineering. This facilitates easy integration with the continuous integration platform. A program slicing algorithm based on a program dependency graph is used to slice the program corresponding to the warning, reducing the impact of irrelevant functions and variables on the model. A Bi-LSTM deep learning model is used to detect long-term, bidirectional dependencies in the program. The constructed model is then incorporated into the project's continuous integration process, allowing it to continuously update and optimize with project iterations.

[0011] S1 includes the following steps:

[0012] S1-1: Select projects that meet the modeling standards on the continuous integration platform;

[0013] The selection criteria for projects are as follows: Since SpotBugs can only perform static analysis on executable files compiled from Java projects, this invention selects Java projects that can obtain compilation configuration information; in order to obtain the types of vulnerabilities that SpotBugs can identify, the projects selected by this invention should have operations such as performing database and LDAP queries, using network connections, reading / writing files, or executing system commands; this invention needs to select projects that are under active development and are used intensively, because only when projects are constantly iterating can continuous static analysis warnings be generated, and the model can be updated and optimized as the project iterates.

[0014] S1-2: Obtain the necessary project compilation information, including the project's compilation environment, compilation tools and their versions, and project compilation commands, and compile the project to obtain executable bytecode files;

[0015] S1-3: Using the "Security" and "Malicious codevulnerability" analysis rules of the static analysis tool SpotBugs, static analysis was performed on the compiled binary executable files of each commit version of the project, and warnings were obtained from the project.

[0016] S1-4: Developers mark warnings. If the code corresponding to the warning has a real vulnerability, the warning is marked as a "real warning". If the code corresponding to the warning does not have the vulnerability indicated by the warning, the warning is marked as a "false alarm".

[0017] S2 includes the following steps:

[0018] S2-1: Obtain the program source code corresponding to each warning;

[0019] S2-2: Convert the program source code into a program dependency graph, where each node in the program dependency graph represents a line of code.

[0020] S2-3: Use the line numbers corresponding to the warnings generated by the static analysis tool as the program slicing criteria, and apply the graph reachability algorithm to perform backward slicing on the program dependency graph to obtain program dependency graph slices that have control or data dependencies with the slicing criteria.

[0021] S2-4: Map the program dependency graph slices to the source program to obtain the set of statements and control predicates corresponding to the slices, which serve as the program slices required for modeling.

[0022] S3 includes the following steps:

[0023] S3-1: Label program slices according to the manual labeling of warnings by developers. If a warning is labeled as a "real warning" by a developer, the corresponding program slice is labeled as 1. If a warning is labeled as a "false alarm", the corresponding program slice is labeled as 0.

[0024] S3-2: Tokenize program slices;

[0025] For the code snippets obtained through program slicing, firstly, for each line of code, remove characters, string text, spaces, and semicolons, while retaining quotation marks. Then, replace all functions and variables except system functions and system variables: replace function names with FUN, variable names with VAR, and class names with CLASS, and append different suffix numbers to represent different functions and variables. This method of symbolizing the program can highlight the constraints between variables in the program, while ignoring the impact of specific input data on the model.

[0026] S3-3: Vectorize the token sequence using Word2vec embedding training;

[0027] This invention embeds token sequences into vector form using Word2vec embedding technology, and implements Word2vec using the Skip-Gram model: First, each token word is encoded using a one-hot encoding scheme as w. i For each w i This yields a vector of length |V|. Except for the i-th position which is 1, all other positions are 0. Then, the Skip-Gram network is trained. When used as input, the output As close as possible to the sum of the one-hot embedding vectors in S (i.e. The weights of the hidden layer are close to those of the training layer. After training, the weights h = (h1, ..., h2) are set to... m As w i The embedding vector, which weights the original vocabulary w i The one-hot encoded vector form is uniquely converted to its neighboring form. This yields a vectorized representation of the token sequence in the code snippet;

[0028] S3-4: Padded or deleted from the vectorized representation of the token sequence to ensure consistency of the model input.

[0029] Since each code snippet may contain a different number of tokens, the corresponding vectors may have different lengths. Bi-LSTM requires vectors of equal length as input, so a parameter k is introduced as a fixed length for the vector corresponding to the code snippet. Because the code snippets are generated by backward slicing, when the vector length is less than the fixed value k, zeros need to be padded at the beginning of the vector; when the vector length is greater than the fixed value k, the beginning part of the vector is deleted, finally obtaining the input to the neural network model.

[0030] S4 includes the following steps:

[0031] S4-1: Construct a Bi-LSTM neural network model and train it using the vectors obtained in S3;

[0032] To capture bidirectional long-term dependencies in the context of warning codes, this invention uses LSTM units as basic building blocks and Bi-LSTM to detect forward and backward long-term dependencies. The vector representation obtained from S3 is used as the network input. The Bi-LSTM layer combines information from the input sequence in both forward and backward directions. Each layer contains 64 LSTM units. The Dense layer reduces the dimensionality of the vector received from the Bi-LSTM layer, converting the output vector into the dimension of the label vector. The Dense layer uses a Leaky ReLU function as its activation function, introducing a leaky value in the negative half-interval of the ReLU function to address the problem that the gradient of the traditional ReLU is zero when the input is less than 0, preventing weight updates. The SoftMax layer takes the low-dimensional vector received from the Dense layer as input and is responsible for normalizing the results, providing feedback for updating the neural network parameters during the learning phase. After training, a Bi-LSTM neural network with tuned model parameters is obtained as the false alarm elimination model.

[0033] S4-2: Set the threshold for the model;

[0034] S4-3: Apply the trained model to the unlabeled warning data of the project and predict the confidence level of each warning (with a value of [0,1]), which represents the probability that it is a real warning. When the confidence level is greater than the threshold set in S4-2, the warning is classified as a real warning by the model. When the confidence level is less than the threshold, the warning is classified as a false alarm by the model.

[0035] S4-4: Set the model threshold to 0.5 and use K-fold cross-validation to evaluate the model's warning classification performance;

[0036] Accuracy, F1 score, and AUC are used as evaluation metrics to assess the model's performance.

[0037] S5 includes the following steps:

[0038] S5-1: When developers modify the code repository or integrate new features, the continuous integration server automatically pulls the changed code.

[0039] S5-2: The continuous integration server performs dependency installation, compilation, etc., for the repository;

[0040] S5-3: The server performs static analysis on the source code and the compiled executable file to obtain the original warning list generated by the static analysis tool;

[0041] S5-4: Take the original warning list and the corresponding code as input, use the model built by S4 to make predictions, the model determines the probability that the code corresponding to each warning contains a real vulnerability, called the confidence level, and classifies the original warnings as real warnings or false alarms according to the threshold set by S4, thus obtaining the optimized warning list;

[0042] S5-5: Developers check and mark the optimized warning list, and verify whether the warnings are real warnings or false alarms. Through the developers' processing of the optimized warning list, warnings with tags are generated and exported.

[0043] S5-6: Input the labeled warnings and their corresponding codes as the incremental training set into the model, and execute S2, S3, and S4-1 to update the model.

[0044] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art:

[0045] (1) This invention uses the code representation corresponding to the warning as the learning feature. Only the code line number and warning label corresponding to the warning are needed to complete the modeling work, avoiding complex feature engineering and reducing the integration difficulty with the continuous integration platform. The PDG-based program slicing algorithm is used to extract the PDG slice corresponding to the code line number corresponding to the warning, so as to reduce the influence of irrelevant functions and variables on the model. The Bi-LSTM neural network is used instead of the convolutional neural network to detect the bidirectional long-term dependency information in the code context corresponding to the warning, thereby capturing deep features at the source code level.

[0046] (2) The present invention integrates the model into the continuous integration process of the project, and uses the newly generated warning data each time the project is submitted as an incremental training set to incrementally train the model, so that the model can be continuously updated and optimized during the project iteration process, and continuously adjust its own parameters according to the developers' marking of warnings, thereby realizing the adaptive improvement of the model to the project and obtaining increasingly accurate and more project-fitting results.

[0047] (3) This invention combines the advantages of warning ranking and warning classification to eliminate false alarms. It predicts the confidence level of each warning, which represents the probability that it is a real warning, and classifies the warnings into real warnings and false alarms according to the threshold set by the developer. The developer can customize the screening intensity of warnings by setting the threshold. Attached Figure Description

[0048] Figure 1 is a flowchart illustrating a method for eliminating false positives in a continuous integration platform.

[0049] Figure 2 is a schematic diagram of the process of obtaining model input from the source code corresponding to the warning disclosed in an embodiment of the present invention.

[0050] Figure 3 is a schematic diagram of the model training process disclosed in an embodiment of the present invention.

[0051] Figure 4 is a schematic diagram of the model update process disclosed in an embodiment of the present invention.

[0052] Figure 5 is a schematic diagram showing the changes in model indicators for different versions of the project disclosed in the embodiments of the present invention.

[0053] Figure 6 is a schematic diagram comparing the warning sorting effect before and after model optimization in the embodiments of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0055] Example 1: As shown in Figures 1 to 6, this invention provides a method for eliminating false positives in continuous integration platforms, comprising the following steps:

[0056] S1: Use the SpotBugs static analysis tool to perform static analysis on the project source code, obtain source code warnings for each commit version of the project, manually annotate the obtained warnings, and determine whether the code snippets contain real vulnerabilities;

[0057] S2: Use a program slicing algorithm based on program dependency graph to slice the program corresponding to the warning, and get the statements that have data dependency or control dependency with the code line corresponding to the warning as program slices;

[0058] S3: Mark and normalize the program slices to obtain the Token sequence, embed the Token sequence into a vector form using Word2vec, and obtain the input of the neural network;

[0059] S4: Input the vector generated in S3 into a bidirectional long short-term memory neural network for training to build a vulnerability false alarm elimination model. Input the original warning to be predicted, and the model outputs the confidence score and classification label of the warning.

[0060] S5: Add the original model to the project's continuous integration process. Each time the project is committed, new warning data will be generated on the platform. The newly generated warning data will be used as an incremental training set to incrementally train the model, so that the model will be continuously updated and optimized during the project iteration process, and its parameters will be continuously adjusted according to the actual marking of warnings by developers.

[0061] Step S1 includes: selecting projects that meet the modeling standards on the continuous integration platform; obtaining the required project compilation information, including the project's compilation environment, compilation tools and their versions, and the project's compilation commands, and compiling the project to obtain executable bytecode files; using the "Security" and "Malicious codevulnerability" analysis rules of the static analysis tool SpotBugs to perform static analysis on the compiled binary executable files of each submitted version of the project to obtain project warnings; and having developers mark the warnings. If the code corresponding to the warning has a real vulnerability, the warning is marked as a "real warning," and if the code corresponding to the warning does not have the vulnerability indicated by the warning, the warning is marked as a "false alarm."

[0062] Specifically, the selection criteria for projects in step S1 are as follows: Since SpotBugs can only perform static analysis on executable files compiled from Java projects, this invention selects Java projects that can obtain compilation configuration information; in order to obtain the vulnerability types that SpotBugs can identify, the project programs selected by this invention should have operations such as performing database and LDAP queries, using network connections, reading / writing files, or executing system commands; this invention needs to select projects that are under active development and are used intensively, because only when projects are continuously iterated and developed can continuous static analysis warnings be generated, and the model can be updated and optimized as the project iterates.

[0063] Step S2 includes: obtaining the program source code corresponding to each warning; converting the program source code into a program dependency graph, where each node in the program dependency graph represents a line of code; using the line number of the code corresponding to the warning generated by the static analysis tool as the program slicing criterion, applying a graph reachability algorithm to perform backward slicing on the program dependency graph to obtain program dependency graph slices that have control or data dependencies with the slicing criterion; mapping the program dependency graph slices to the source program to obtain the set of statements and control predicates corresponding to the slices, which are used as the program slices required for modeling.

[0064] The S3 step includes: labeling program slices according to the developers' manual labeling of warnings; if a warning is labeled as a "real warning" by the developers, the corresponding program slice is labeled as 1; if a warning is labeled as a "false alarm", the corresponding program slice is labeled as 0; tokenizing the program slices; using Word2vec for embedding training to vectorize the token sequence; padding or deleting the vectorized representation of the token sequence to ensure the consistency of the model input.

[0065] Specifically, in step S3, for the code fragments obtained after program slicing, firstly, for each line of code, characters, string text, spaces, and semicolons are deleted, while quotation marks are retained. Then, all functions and variables except system functions and system variables are uniformly replaced: function names are replaced with FUN, variable names are replaced with VAR, and class names are replaced with CLASS, with different suffix numbers appended to represent different functions and variables. This method of symbolizing the program can highlight the constraints between variables in the program, while ignoring the impact of specific input data on the model.

[0066] Specifically, in step S3, the Word2vec embedding technique is used to learn the vector embeddings of words in a given corpus. Essentially, it is a neural network with a single hidden layer and N nodes. There are two implementations of Word2vec: Skip-Gram and CBOW. Skip-Gram predicts the context given a center word, while CBOW predicts the center word given the context. This invention uses the Skip-Gram model to embed words from the corpus into vectors, ensuring that each word is recognized by its neighboring words. A one-hot encoding scheme is used to encode each token word as a w. i For each w i This yields a vector of length |V|. Except for the i-th position which is 1, all other positions are 0. Then, the Skip-Gram network is trained. When used as input, the output As close as possible to the sum of the one-hot embedding vectors in S (i.e. The values ​​are close to those of the hidden layer. After training, the weights h = (h1, ..., h2) of the hidden layer are set to... m As w i The embedding vector, which weights the original vocabulary w i The one-hot encoded vector form is uniquely converted to its neighboring form. This yields the vectorized representation of the token sequence in the code snippets. Finally, since each code snippet may contain a different number of tokens, the corresponding vectors may have different lengths. Bi-LSTM requires vectors of equal length as input, so a parameter k is introduced as the fixed length of the vector corresponding to each code snippet. Because the code snippets are generated by backward slicing, when the vector length is less than the fixed value k, zeros are padded at the beginning of the vector; when the vector length is greater than the fixed value k, the beginning of the vector is deleted, ultimately yielding the input to the neural network model.

[0067] Step S4 includes: constructing a Bi-LSTM neural network model and training it using the vectors obtained in S3; setting a threshold for the model; applying the trained model to the unlabeled warning data of the project, predicting the confidence level (with values ​​of [0,1]) of each warning, representing the probability that it is a real warning; when the confidence level is greater than the threshold set in S4, the warning is classified as a real warning by the model, and when the confidence level is less than the threshold, the warning is classified as a false alarm by the model; setting the model threshold to 0.5, and using K-fold cross-validation to evaluate the warning classification effect of the model.

[0068] Specifically, the Bi-LSTM neural network used in step S4 can capture bidirectional long-term dependency information in the context of the warning code. This invention uses LSTM units as basic building blocks and Bi-LSTM to detect forward and backward long-term dependencies. The vector representation obtained in S3 is used as the network input. The Bi-LSTM layer combines information from the input sequence in both forward and backward directions, and each layer contains 64 LSTM units. The Dense layer reduces the dimensionality of the vector received from the Bi-LSTM layer, converting the output vector into the dimension of the label vector. The Dense layer uses a Leaky ReLU function as the activation function, which introduces a leaky value in the negative half-interval of the ReLU function to solve the problem that the gradient is also 0 when the input is less than 0, causing the weights to fail to be updated. The SoftMax layer takes the low-dimensional vector received from the Dense layer as input and is responsible for normalizing the results, providing feedback for updating the neural network parameters during the learning phase. After the training phase is completed, a Bi-LSTM neural network with tuned model parameters is obtained as a false alarm elimination model.

[0069] Specifically, the reason why accuracy, F1 score, and AUC score are used as evaluation metrics in step S4 is that there is usually a class imbalance problem in the static analysis of warnings. Performance metrics such as precision and recall are greatly affected by the class ratio. Therefore, a comprehensive metric commonly used in previous studies is adopted. The definitions of each metric are as follows:

[0070] Accuracy: One of the most commonly used evaluation metrics in classification tasks. It is based on a single model M built on a dataset D, where S is the size of D, and M(x) = ... i ) and y i These represent the results of model prediction and manual inspection, respectively, and are defined as follows:

[0071]

[0072] F1 Score: For warning classification problems, instances can be categorized based on the model's predictions and the actual results of manual inspection: True Positives (TP): The number of correctly classified true warnings; False Negatives (FN): The number of misclassified false alarms; False Positives (FP): The number of misclassified true warnings; True Negatives (TN): The number of correctly classified false alarms. These categories form a confusion matrix. Precision (P), Recall (R), and F1 score are derived from the confusion matrix. Precision and recall are contradictory metrics. Generally, higher precision means more true vulnerabilities can be classified as positive, while higher recall reveals more true vulnerabilities. Considering both precision and recall, the F1 score is a harmonic mean that combines both.

[0073]

[0074]

[0075]

[0076] AUC (Average Acceptance Value): The receiver operating characteristic curve (ROC) is widely used in machine learning and is an effective tool for visualizing the generalization performance of classifiers. However, the ROC curve has a drawback: when the curves of two classifiers intersect, it is difficult to predict which classifier is superior. Therefore, the area under the ROC curve, i.e., the sum of the areas of all parts under the ROC curve, has been introduced as an evaluation metric. Assume the ROC curve is formed by a series of points {(x1,y1),(x2,y2),….,(x... n ,y n Connecting these sequentially, AUC can be calculated using the following formula:

[0077]

[0078] Step S5 includes: developers modifying the code repository or integrating new features; the continuous integration server automatically pulling the changed code; the continuous integration server performing dependency installation and compilation on the repository; the server performing static analysis on the source code and the compiled executable file to obtain an initial warning list generated by the static analysis tool; using the initial warning list and corresponding code as input, the model built in S4 is used for prediction, and the model determines the probability that the code corresponding to each warning contains a real vulnerability, called the confidence level. Based on the threshold set in S4, the initial warnings are classified as real warnings or false alarms, thus obtaining an optimized warning list; developers check and mark the optimized warning list, verifying that the warnings are real warnings or false alarms; through the developers' processing of the optimized warning list, marked warnings are generated and exported; the marked warnings and their corresponding code are used as an incremental training set, input into the model, and steps S2, S3, and S4 are executed to update the model.

[0079] Example 2: This example mainly describes the implementation process and result analysis of vulnerability false positive elimination on the Juliet 1.3 vulnerability dataset from the open-source project. Juliet is a collection of test cases for C / C++ and Java, composed of real-world applications, artificially generated test cases, and academic research. Juliet Test Suite V1.3 contains 28,888 Java test cases, covering 112 different software security-related vulnerability categories. Each test case consists of a method with a specific vulnerability instance (used to capture real vulnerabilities) and an additional non-vulnerable method (used to capture false positives). For each test case, Juliet annotates the methods within it; a method name containing the "good" field indicates that the method is vulnerability-free, and a method name containing the "bad" field indicates that the method is vulnerable. These naming rules help researchers easily categorize warnings generated by static analysis tools. This embodiment uses SpotBugs to perform static analysis on all test cases of the project to obtain a warning list. The warnings are marked according to the method name corresponding to the warning to obtain a data source. The data source is used to train a deep learning neural network to obtain a false alarm elimination model. The model predicts the type of warning and eliminates false alarms, thereby reducing the number of false alarms in the warning list and improving the efficiency of developers in handling and checking warnings.

[0080] Step 1: Use the "Security" and "Malicious code vulnerability" analysis rules related to software security in the SpotBugs static analysis tool to perform static analysis on the compiled source code, obtain the original warnings, and mark them.

[0081] This embodiment focuses on four vulnerability datasets in Juliet: CWE23, CWE78, CWE89, and CWE113. Compared to other vulnerability datasets, SpotBugs generates more warnings for these types of vulnerabilities. The warnings are labeled according to the method name (a "good" field in the method name indicates that the method has no vulnerability and the corresponding warning is a false alarm; a "bad" field in the method name indicates that the method has a vulnerability and the corresponding warning is a real warning). Table 1 shows the number of warnings and the number of false alarms generated by the four types of vulnerability datasets.

[0082] Table 1

[0083]

[0084] Step 2: Slice the program corresponding to the warning to obtain the program slice related to the statement corresponding to the warning.

[0085] Step 3: Normalize the program slices to obtain the Token sequence and embed it into a vector form.

[0086] Taking the warning actually detected by SpotBugs in this embodiment as an example, the process of obtaining program slices from the program source code and performing preprocessing in steps 2 and 3 is explained in detail. The program source code corresponding to a warning in this embodiment is shown in Figure 2a. A PDG is generated, and the line number 28 corresponding to the warning detected by SpotBugs is used as the criterion for program slicing, as shown in Figure 2b. Each ellipse represents a single line of code. Solid arrows indicate statements with data dependencies on the code statements corresponding to the warning line number, including lines 9, 14, 20, and 25. Dashed arrows indicate statements with control dependencies on the code statements corresponding to the warning line number, including lines 2, 4, 5, 9, 14, 16, 17, 20, and 25. By extracting all code statements with data and control dependencies on the line number corresponding to the warning, the program slice corresponding to this warning is obtained. After obtaining the sliced ​​code snippets, preprocessing is required. The entire process is shown in Figure 2c. For each line of code, characters or strings are deleted while quotation marks are retained. Then, all functions and variables except system functions and variables are uniformly replaced, converting the program slice into a token sequence. The generated token sequence embedding vector is shown in Figure 2d. Vectorization is implemented using the Skip-Gram model of Word2vec embedding technology. Since each code snippet may contain a different number of tokens, and Bi-LSTM requires vectors of equal length as input, analysis of the token sequence length of the warning program shows that 95% of the lengths are below 1000. Therefore, 1000 is used as the fixed length of the vector corresponding to the program slice. When the vector length is less than 1000, zeros are padded at the beginning of the vector; when the vector length is greater than 1000, the beginning part of the vector is deleted.

[0087] Step 4: Train the Bi-LSTM neural network using the vectorized token sequence to build a vulnerability false alarm elimination model, set the model threshold, eliminate false alarms of the warnings to be predicted, and evaluate the model performance using K-fold cross-validation.

[0088] The structure of the Bi-LSTM neural network in this embodiment is shown in Figure 3. Each token vectorized by Word2vec in step 3 is embedded into a vector as the network input. The Bi-LSTM layer combines information from both the forward and backward directions of the input sequence, and each layer contains 64 LSTM units. The Dense layer reduces the dimensionality of the vector received from the Bi-LSTM layer, converting the output vector into the dimension of the label vector. The Dense layer uses the Leaky ReLU function as the activation function, which introduces a leaky value in the negative half-interval of the ReLU function, solving the problem that the gradient of the traditional ReLU is also 0 when the input is less than 0, resulting in the weights not being updated. The SoftMax layer takes the low-dimensional vector received by the Dense layer as input and is responsible for normalizing the results, providing feedback for updating the neural network parameters during the learning phase. After the training phase is completed, a Bi-LSTM neural network with tuned model parameters is obtained as a false positive elimination model. Before performing false alarm elimination on predicted warnings, a model threshold (with a value of [0,1]) needs to be set. This threshold represents the model's filtering strength for false alarms. The model predicts the confidence level (with a value of [0,1]) for each warning, representing the probability that it is a genuine warning. Then, warnings are classified according to the threshold set by the developers. Warnings with a confidence level greater than the threshold are classified as genuine warnings and retained, while warnings with a confidence level less than the threshold are classified as false alarms and eliminated. In this embodiment, the model threshold is set to 0.5, meaning that if the confidence level of a warning predicted by the model is greater than 0.5, it is classified as a genuine warning; if it is less than 0.5, it is classified as a false alarm.

[0089] In this embodiment, ten-fold cross-validation is used to evaluate the performance of the trained vulnerability false positive elimination model. The evaluation metrics are accuracy, F1-measure, and AUC value, and the results are shown in Table 2.

[0090] Table 2

[0091] Dataset Accuracy F1-measure AUC CWE23 88.27% 91.83% 87.34% CWE78 84.72% 88.34% 82.02% CWE89 83.68% 87.87% 82.85% CWE113 82.35% 86.57% 81.26% surface

[0092] The vulnerability false positive elimination method for continuous integration platforms proposed in this invention improves upon existing false positive elimination methods in terms of evaluation metrics, verifying the effectiveness of the model.

[0093] Example 3: This example focuses on a business project undergoing development and iteration within an enterprise's continuous integration platform. Unlike manually synthesized vulnerability datasets, real-world business projects in iteration continuously generate new warning data. This invention uses this newly generated warning data as an incremental training set to incrementally train the model. This allows the model to continuously update and optimize during project iterations, adjusting its parameters based on the actual warning labeling by developers. This enables the model to adapt to the project, resulting in increasingly accurate and project-specific results. Table 3 provides a brief overview of the project, which is an internal business project on an enterprise's continuous integration platform.

[0094] Table 3

[0095] Project Area: Software Development; Size (Lines of Code): 192,190; Project Time: 2020.06.11—2022.03.20 (at the time of data retrieval); Number of Submissions: 221; Language: Java; Latest Version; Number of Warnings: 1207 surface

[0096] This embodiment first uses the project's historical warning data to train the model (the specific training steps are as described in Embodiment 1), and then directly executes step 5 to add the obtained initial model into the project's continuous integration process, so that the model is updated synchronously with the project iteration. The model update process of this embodiment is shown in Figure 4.

[0097] Tagged commits were used as the evaluation targets. Tagded versions represent project releases or phased archives by developers. Since there are significant differences in repositories and time spans between versions, changes in model performance throughout the project lifecycle can be observed. The number of warnings for each tagged version was statistically analyzed, and the results are shown in Table 4. It can be seen that the total number of warnings detected across versions is relatively similar, with some even showing a decrease. Analysis of the project source code suggests this may be due to a streamlined project structure resulting from refactoring. Furthermore, an average of 25.4 warnings that had never appeared before were reported per version. These warnings, after being tagged by developers, are used as incremental input to update the model. That is, the model for a version is updated from the model of the previous version, and the incremental training set required for updating the model consists of the newly added warnings tagged by developers in this version and their corresponding source code. Assuming there are versions 1, 2, and 3 in the commit order, the model for version 3 is updated based on the model for version 2, using the newly added warnings in version 3 as the incremental training set. The changes in model performance metrics for each commit version are shown in Figure 5. It can be seen that the performance indicators of the various versions of the model generally show an upward trend, proving that in the context of continuous integration, the vulnerability false alarm elimination model of this invention can be continuously updated and optimized with project iterations, and continuously adjusted according to the developers' marking of new warnings, thereby achieving increasingly accurate results.

[0098] Table 4

[0099] Version Number | Historical Warnings | New Warnings | Total Warnings 20200611106001060 202006171052451097 202006281085491134 202007311070231093 20200911108181089 202012291078241102 202104131158321190 202107161173141187 202109241168231191 202203 201171361207 surface

[0100] The newly added warnings in the latest project submission were optimized by setting the model threshold to 0.5. The results before and after optimization are shown in Table 5. As can be seen, of the original 36 newly added warnings, only 19 remained after model optimization (19 warnings had a confidence level greater than 0.5 and were classified as genuine warnings; the other 17 warnings had a confidence level less than 0.5 and were classified as false alarms and not displayed). This reduced the number of warnings requiring review and confirmation for developers by nearly half, demonstrating that the model can help developers save time in checking warnings and improve the efficiency of warning handling.

[0101] Table 5

[0102] False alarms require review. Warning count: 92,736 before model optimization; 71,219 after optimization. surface

[0103] Furthermore, the development team can set the model threshold according to the needs of the team and project. The higher the threshold, the greater the filtering of false positives, but the risk of missing real warnings also increases. To avoid missing any possible real warnings, the model threshold can be set to 0. In this case, the model does not filter or classify any warnings, and all raw warnings will be displayed. However, the model can sort the warnings in descending order of confidence, placing the warnings most likely to contain vulnerabilities at the top of the warning list for developers to review first. The effect of warning ranking on developers' actual work is shown in Figure 6. The y-axis represents the number of real warnings reviewed, and the x-axis represents the number of warnings reviewed in descending order of ranking. It can be seen that the confidence-based sorting method of this invention is better than the platform's default method of sorting by warning priority. After 30 checks, all real warnings were identified, and in the first 5 checks, 4 of them were real warnings. This shows that when ranking warnings, the model can place warnings with a higher probability of vulnerabilities at the top of the list, allowing developers to review more vulnerabilities in a shorter time.

[0104] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for eliminating false positives in vulnerability reporting for continuous integration platforms, characterized in that, Includes the following steps: S1: Use the SpotBugs static analysis tool to perform static analysis on the project source code, obtain source code warnings for each commit version of the project, manually annotate the obtained warnings, and determine whether the code contains real vulnerabilities; S2: Use a program slicing algorithm based on Program Dependence Graph (PDG) to slice the program corresponding to the warning, and obtain the statements that have data dependency or control dependency with the code line corresponding to the warning as program slices; S3: Label and normalize the program slices to obtain a token sequence. Embed the token sequence into a vector form using Word2vec to obtain the input of the neural network. S4: Input the vector generated in S3 into a bidirectional long short-term memory (Bi-LSTM) neural network for training to build a vulnerability false alarm elimination model. Input the original warning to be predicted, and the model outputs the confidence score and classification label of the warning. S5: Add the original model to the project's continuous integration process. Each time the project is committed, new warning data will be generated on the platform. The newly generated warning data will be used as an incremental training set to incrementally train the model, so that the model is continuously updated and optimized during the project iteration process, and its parameters are continuously adjusted according to the actual labeling of warnings by the developers.

2. The method for eliminating false positives in a continuous integration platform according to claim 1, characterized in that, S1 includes the following steps: S1-1: Select projects on the continuous integration platform that meet the modeling standards; the selection criteria for projects are as follows: since SpotBugs can only perform static analysis on the compiled executable files of Java projects, select Java projects for which compilation configuration information can be obtained; in order to obtain the vulnerability types that SpotBugs can identify, the selected project programs should have operations such as performing database and LDAP queries, using network connections, reading / writing files, or executing system commands; select projects that are under active development and are being used intensively; S1-2: Obtain the required project compilation information, including the project's compilation environment, compilation tools and their versions, and the project's compilation commands, and compile the project to obtain executable bytecode files; S1-3: Use the "Security" and "Malicious code vulnerability" analysis rules of the static analysis tool SpotBugs to perform static analysis on the compiled binary executable files of each submitted version of the project to obtain project warnings; S1-4: Developers mark the warnings. If the code corresponding to the warning has a real vulnerability, the warning is marked as a "real warning". If the code corresponding to the warning does not have the vulnerability indicated by the warning, the warning is marked as a "false alarm".

3. The method for eliminating false positives in a continuous integration platform according to claim 1, characterized in that, S2 includes the following steps: S2-1: Obtain the program source code corresponding to each warning; S2-2: Convert the program source code into a program dependency graph, where each node in the program dependency graph represents a line of code; S2-3: Use the line number corresponding to the warning generated by the static analysis tool as the program slicing criterion, and apply a graph reachability algorithm to perform backward slicing on the program dependency graph to obtain program dependency graph slices that have control or data dependencies with the slicing criterion; S2-4: Map the program dependency graph slices to the source program to obtain the set of statements and control predicates corresponding to the slices, which are used as the program slices required for modeling.

4. The method for eliminating false positives in a continuous integration platform according to claim 1, characterized in that, S3 includes the following steps: S3-1: Label program slices according to the developers' manual marking of warnings. If a warning is marked as a "real warning" by the developers, the corresponding program slice is marked as 1; if a warning is marked as a "false alarm," the corresponding program slice is marked as 0. S3-2: Tokenize the program slices. For the code snippets obtained after program slicing, first, for each line of code, delete characters, string text, spaces, and semicolons, while retaining quotation marks. Then, replace all functions and variables except system functions and system variables: replace function names with FUN, variable names with VAR, and class names with CLASS, and append different suffix numbers to represent different functions and variables. S3-3: Vectorize the token sequence using Word2vec embedding training. Embed the token sequence into a vector form using Word2vec embedding technology, and implement Word2vec using the Skip-Gram model: First, encode each token word using a one-hot encoding scheme as w. i For each W i This yields a vector of length |V|. Except for the i-th position which is 1, all other positions are 0. Then, the Skip-Gram network is trained. When used as input, the output As close as possible to the sum of the one-hot embedding vectors in S (i.e. The weights of the hidden layer are close to those of the training layer. After training, the weights h = (h1, ..., h2) are set to... m As w i The embedding vector, which weights the original vocabulary w i The one-hot encoded vector form is uniquely converted to its neighboring form. This yields the vectorized representation of the code snippet token sequence; S3-4: The vectorized representation of the token sequence is padded or deleted to ensure the consistency of the model input. A parameter k is introduced as the fixed length of the vector corresponding to the code snippet. Since the code snippet is generated by backward slicing, when the vector length is less than the fixed value k, zeros need to be padded at the beginning of the vector; when the vector length is greater than the fixed value k, the beginning part of the vector is deleted, and finally the input of the neural network model is obtained.

5. The method for eliminating false positives in a continuous integration platform according to claim 1, characterized in that, S4 includes the following steps: S4-1: Construct a Bi-LSTM neural network model and train it using the vectors obtained in S3; use LSTM units as basic building blocks and use Bi-LSTM to detect long-term dependencies in the forward and backward directions. The vector representation obtained in S3 is used as the network input. The Bi-LSTM layer combines information from the input sequence in both the forward and backward directions. Each layer contains 64 LSTM units. The Dense layer reduces the dimensionality of the vectors received from the Bi-LSTM layer, converting the output vector into the dimension of the label vector. The Dense layer uses a Leaky ReLU function as the activation function, which introduces a leaky value in the negative half-interval of the ReLU function to address the limitations of traditional Rectified Linear Units (Rectified Linear Units). The ReLU unit has a gradient of 0 when the input is less than 0, which leads to the problem that the weights cannot be updated. The SoftMax layer takes the low-dimensional vector received by the Dense layer as input and is responsible for normalizing the result, providing feedback for updating the neural network parameters during the learning phase. After the training phase is completed, a Bi-LSTM neural network with tuned model parameters is obtained as a false alarm elimination model. S4-2: Set the threshold of the model; S4-3: Apply the trained model to the unlabeled warning data of the project and predict the confidence of each warning (with a value of [0,1]), which represents the probability that it is a real warning. When the confidence is greater than the threshold set in S4-2, the warning is classified as a real warning by the model. When the confidence is less than the threshold, the warning is classified as a false alarm by the model; S4-4: Set the model threshold to 0.5 and use K-fold cross-validation to evaluate the warning classification effect of the model; use accuracy, F1 score, and AUC score as evaluation metrics to evaluate the performance of the model.

6. The method for eliminating false positives in a continuous integration platform according to claim 1, characterized in that, S5 includes the following steps: S5-1: Developers modify the code repository or integrate new features, and the continuous integration server automatically pulls the changed code; S5-2: The continuous integration server installs dependencies and compiles the repository; S5-3: The server performs static analysis on the source code and the compiled executable file to obtain the original warning list generated by the static analysis tool; S5-4: The original warning list and the corresponding code are used as input, and the model built in S4 is used for prediction. The model determines the probability that the code corresponding to each warning contains a real vulnerability, called the confidence level, and classifies the original warnings as real warnings or false alarms according to the threshold set in S4, thereby obtaining the optimized warning list; S5-5: Developers check and mark the optimized warning list, and verify that the warnings are real warnings or false alarms. Through the processing of the optimized warning list by developers, marked warnings are generated and exported; S5-6: The marked warnings and the corresponding code are used as an incremental training set, input into the model, and S2, S3, and S4-1 are executed to update the model.

Images