Multi-programming language vulnerability influence range detection method based on semantic analysis

Abstract

The invention discloses a multi-programming language vulnerability influence range detection method based on semantic analysis, and the method comprises the steps: building a multi-programming language vulnerability influence range detection model based on a large language model (LLM), completing the training of the vulnerability influence range detection model through employing a training data set covering a plurality of programming languages and vulnerability types, and obtaining a multi-programming language vulnerability influence range detection model. Metadata, deletion line numbers and difference information submitted by vulnerability repair are used as model input to obtain root cause deletion lines of submitted files, a line mapping algorithm is used for backtracking according to the root cause deletion lines to obtain candidate lines, then the candidate lines are used as model input to screen vulnerability root cause lines, and therefore vulnerability introduction and submission are found. Therefore, the vulnerability influence range is determined, the introduction and submission of candidate vulnerabilities are effectively reduced while the cross-language applicability is improved, the vulnerability tracking capability is improved, and the false positive rate is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of vulnerability impact range identification, and in particular relates to a multi-programming language vulnerability impact range detection method based on semantic analysis. Background Art

[0002] The present invention mainly involves the following professional terms:

[0003] Bug-inducing Commit: refers to the submission record in the software version control process, in which a code submission leads to the introduction of new vulnerabilities or defects in the software.

[0004] Bug-fixing Commit: refers to the submission record of a code submission aimed at fixing discovered vulnerabilities or defects during the software version control process.

[0005] Previous Commit: refers to the intermediate commit record between the vulnerability introduction commit and the vulnerability repair commit. These commits usually do not directly introduce or repair vulnerabilities, but changes to the code may affect the generation or repair of vulnerabilities.

[0006] Commit Metadata: refers to the relevant information attached to each code submission in the software version control system, including but not limited to submission time, submitter identity, submission message, etc.

[0007] Vulnerability root cause lines: refers to the specific lines of code that cause the vulnerability. These lines of code are the direct cause of software defects or vulnerabilities.

[0008] The existing vulnerability impact range identification methods are mainly SZZ series algorithms. The B-SZZ algorithm was first proposed in 2005. The algorithm identifies the corresponding vulnerability fix submissions by analyzing the error reports in the issue tracking system, and uses the reference information clearly related to the vulnerability ID in the version control system (VCS) and the vulnerability description in the submission information to confirm the relationship between the vulnerability and its fix from the grammatical and semantic levels. After identifying the vulnerability fix submission, the diff command is used to locate the specific lines that have been modified during the error repair. Finally, the annotate command is used to track the history of the code in chronological order to determine the potential origin of the vulnerability. AG-SZZ is an improvement on the B-SZZ algorithm. It solves the problem that the B-SZZ algorithm takes into account non-semantic lines and changes in code format when processing code changes, resulting in the processing of a large amount of content that is irrelevant to the actual error, thereby missing the real cause of the error. In addition, algorithms such as MA-SZZ, RA-SZZ, V-SZZ and NEURAL-SZZ have improved the B-SZZ algorithm.

[0009] Among them, V-SZZ places special emphasis on the identification of vulnerabilities that may be introduced in earlier versions of the software. Its workflow diagram is shown in Figure 1. Traditional SZZ algorithms often only focus on the last modification and cannot effectively trace back to possible vulnerabilities in earlier versions. The V-SZZ algorithm can be reversed multiple times by combining line mapping technology. For Java projects, the abstract syntax tree (AST) mapping algorithm is used, and for C / C++ projects, the edit distance, that is, the Levenshtein distance, is used to map similar lines. Through reverse backtracking, V-SZZ identifies and tracks the earliest submissions that modified the vulnerability-related code lines, and believes that these submissions may be the original submissions that introduced the vulnerability. This method significantly improves the accurate understanding of the causes of vulnerabilities, especially vulnerabilities that were introduced early and have existed for a long time.

[0010] However, existing SZZ algorithms all have certain defects. For example, RA-SZZ and NEURAL-SZZ are mainly designed for the Java language, so they face many challenges when extended to other programming languages. These algorithms rely on parsing abstract syntax trees of different languages. However, it is very difficult to maintain an abstract syntax tree parser for each programming language in practical applications. Therefore, the current SZZ algorithm has significant limitations in cross-language applicability and scalability, and a more general method is urgently needed to solve this problem.

[0011] In addition, the current SZZ methods represented by B-SZZ, including B-SZZ, AG-SZZ, RA-SZZ and MA-SZZ, can only perform a single backtracking. Once the cause of the vulnerability exists in an earlier version, this type of method cannot detect it at all. The process of several SZZ algorithm variants in handling the CVE-2016-10049 vulnerability, such as Figure 2 As shown in the figure, this CVE is a buffer overflow vulnerability, and the vulnerability fix commit uses the magickmax() function to filter the input that may cause the buffer overflow. For the B-SZZ algorithm, after performing a blame operation on the deleted lines A and B, they are traced back to the early commit 2174484, causing B-SZZ to finally identify the commit where the vulnerability was introduced as 2174484. However, this identification is obviously wrong. The real defective code is hidden in earlier commits, and a single backtracking cannot accurately reveal the root cause of the vulnerability. The limitation of the B-SZZ algorithm is that it only relies on the most recent code change to determine the source of the error. Although this method is efficient in dealing with some simple errors, its usability is significantly reduced when facing complex code vulnerabilities, especially those that have been introduced in early versions of the code and have not been discovered for a long time.

[0012] V-SZZ first introduced the ability of multiple backtracking, but it only matches and backtracks based on code similarity and stops backtracking when the Levenshtein distance is less than 0.75, which may result in insufficient backtracking. This method cannot fully utilize the semantic information of the code to determine whether there are vulnerabilities in early versions, but only relies on code similarity. In fact, complex vulnerabilities may go through multiple evolution processes, and low similarity does not mean that the vulnerability does not exist. Figure 2 As shown in the figure, when processing the deletion of line A, the first backtracking obtained 2174484, and further backtracking was terminated because no matching line that met the conditions for continued backtracking was found. When tracking the deletion of line B, V-SZZ backtracked 5 times in a row until the tracking identified 7131d8f. Therefore, the vulnerability introduction commits finally identified by V-SZZ were 2174484 and 7131d8f. Despite this, the backtracking strength of V-SZZ is still insufficient, and it failed to identify the real vulnerability introduction commit, namely 3ed852. Although V-SZZ provides certain vulnerability tracking capabilities, the line mapping algorithm it uses shows obvious limitations when facing more complex or deeper hidden code vulnerabilities.

[0013] The above algorithms will backtrack all deleted lines in the vulnerability fix submission and regard all results as vulnerability introduction submissions. However, if the vulnerability fix submission involves multiple files, and each file contains multiple deleted lines, the algorithm will generate a large number of redundant and irrelevant vulnerability introduction submissions, resulting in a sharp increase in the false positive rate. NEURAL-SZZ proposed a method to identify the root cause of deleted lines to solve this problem. However, its deep learning method is not very accurate in identifying the root cause of deleted lines when solving this problem. Summary of the invention

[0014] In view of this, the present invention provides a multi-programming language vulnerability impact range detection method based on semantic analysis, which realizes cross-language, high-traceability vulnerability impact range detection.

[0015] The present invention provides a method for detecting the impact range of multiple programming language vulnerabilities based on semantic analysis, comprising the following steps:

[0016] Step 1: Collect CVE vulnerabilities in various programming languages ​​and their corresponding vulnerability fix submissions and vulnerability introduction submissions, as well as natural language information in the submission metadata, and perform data cleaning and normalization processing; use the collected data to build training data sets, verification data sets, and test data sets;

[0017] Step 2: Build a multi-programming language vulnerability impact range detection model based on the large language model. Use the training data set, verification data set, and test data set built in step 1 to complete the training of the multi-programming language vulnerability impact range detection model. Use parsing tools to extract submission metadata from the code to be detected, delete line numbers and difference information,

[0018] Step 3: Divide the extracted submission metadata, deletion line numbers, and difference information into multiple vulnerability-related repair information based on the files involved in the vulnerability. Each vulnerability-related repair information corresponds to multiple files involved in the code to be detected. Input each file into the multi-programming language vulnerability impact range detection model trained in step 2 to obtain the line number of the root cause deletion line of each file related to each vulnerability.

[0019] Step 4: Take the root cause deletion line obtained in step 3 as the starting line, obtain the most recent previous commit before the starting line, and stop backtracking if the previous commit does not contain the deletion line, otherwise calculate the Levenshtein distance between each deletion line in the previous commit and the starting line; input the deletion line whose Levenshtein distance is not less than the threshold as the candidate line into the multi-programming language vulnerability impact range detection model, and the multi-programming language vulnerability impact range detection model determines whether the candidate line is related to the vulnerability based on the submission metadata and difference information. If it is not related to the vulnerability, stop backtracking, otherwise output the candidate line with the highest possibility as the vulnerability root cause line of this round; traverse the root cause deletion lines obtained in step 3 to obtain a set of vulnerability root cause lines, and use the commit with the earliest vulnerability root cause line as the candidate vulnerability of the file for submission;

[0020] Step 5: Use the diff information of the vulnerability fix submission in the candidate vulnerability introduction submission obtained in step 5 and the CVE submission information as the input of the large language model. After calculation by the large language model, select the introduction submission that is most relevant to the vulnerability from multiple candidate vulnerability introduction submissions.

[0021] Furthermore, the submission metadata in step 2 is submission information related to the vulnerability repair submission in the code to be detected.

[0022] Furthermore, the submission information is obtained by extracting the vulnerability repair submission using the GitHub parsing tool PyDriller.

[0023] Furthermore, the deleted line numbers in step 2 are the line numbers of all deleted lines of the code to be detected.

[0024] Furthermore, the difference information in step 2 is patch block information where each deleted line is located.

[0025] Furthermore, thought chain prompts are used to guide the multi-programming language vulnerability impact range detection model to complete the analysis.

[0026] Furthermore, step 3 also includes inputting each file and the common vulnerability disclosure description into the multi-programming language vulnerability impact range detection model trained in step 2 to obtain the line number of the root cause deletion line of each file related to each vulnerability.

[0027] Furthermore, in step 4, the method of obtaining the most recent previous commit before the starting line is to use gitblame to obtain it.

[0028] Beneficial effects:

[0029] The present invention builds a multi-programming language vulnerability impact range detection model based on a large language model (LLM), uses a training data set covering multiple programming languages ​​and vulnerability types to complete the training of the vulnerability impact range detection model, uses the submission metadata, deletion line numbers and difference information of the vulnerability repair submission as model input to obtain the root cause deletion lines of each file in the submission, uses a row mapping algorithm to backtrack according to the root cause deletion lines to obtain candidate lines, and then uses the candidate lines as model input to screen to obtain the vulnerability root cause lines, thereby finding the vulnerability introduction submission, thereby determining the vulnerability impact range, effectively reducing the candidate vulnerability introduction submissions while improving cross-language applicability, improving the vulnerability tracking capability, and reducing the false positive rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the workflow of the existing V-SZZ algorithm.

[0031] Figure 2 Schematic diagram of the performance of different existing SZZ algorithms on CVE-2016-10049.

[0032] Figure 3 A flowchart of a method for detecting the impact range of multiple programming language vulnerabilities based on semantic analysis provided by the present invention.

[0033] Figure 4 A schematic diagram of a root cause deletion line identification process in a multi-programming language vulnerability impact range detection method based on semantic analysis provided by the present invention.

[0034] Figure 5 A schematic diagram of a vulnerability backtracking decision process in a multi-programming language vulnerability impact range detection method based on semantic analysis provided by the present invention. DETAILED DESCRIPTION

[0035] The present invention is described in detail with reference to the following embodiments.

[0036] The present invention provides a method for detecting the impact range of multiple programming language vulnerabilities based on semantic analysis, and the processing flow is as follows: Figure 3As shown, the specific steps include:

[0037] Step 1: Collect CVE vulnerabilities, vulnerability repair submission codes, related natural language information, and vulnerability introduction submissions in various programming languages, and ensure the quality and consistency of the data through data cleaning and normalization. Use the collected data to build training data sets, verification data sets, and test data sets. The training data set covers multiple programming languages ​​and vulnerability types to enhance the generalization ability of the model.

[0038] Step 2: Build a multi-programming language vulnerability impact range detection model based on the Large Language Model (LLM), and use the training data set, verification data set, and test data set built in step 1 to complete the training of the multi-programming language vulnerability impact range detection model.

[0039] The present invention is aimed at vulnerabilities in various programming languages, so special training and optimization are required for different programming languages ​​and vulnerability types. Using the training data set, verification data set and test data set constructed by the present invention to train LLM can enable the large model to perform tasks better.

[0040] Step 3. Use a parsing tool to extract submission metadata, deletion line numbers and difference information (diff information) from the code to be tested, where the submission metadata is the submission information related to the vulnerability repair submission in the code to be tested. The submission information is the prompt information related to the vulnerability repair refined by the project developer and has a strong reference for understanding the vulnerability; the deletion line number is all the deleted lines of the code to be tested, and the line number of each deleted line is extracted, thereby limiting the selection of LLM to a certain range; the difference information is the patch block (hunk) information where each deleted line is located, so as to ignore irrelevant hunks, reduce the total amount of difference information, and increase the ratio of effective information.

[0041] like Figure 4 As shown, there are three items of information that together constitute the input basis of LLM. Submission metadata: The present invention uses the GitHub parsing tool PyDriller to extract the submission information of the vulnerability fix submission. This information is the prompt information related to the vulnerability fix refined by the project developer, and it has a strong reference for understanding the vulnerability. Deleted line number: LLM-SZZ will consider all deleted lines of the file and extract the line number of each line, thereby limiting the selection of LLM to a certain range. Difference information: This article extracts the hunk information of each deleted line as a diff set, so that irrelevant hunks can be omitted, reducing the total amount of diff information to increase the ratio of effective information.

[0042] Step 4: Divide the submission metadata, deleted line numbers, and difference information of the code to be detected extracted in step 3 into multiple vulnerability-related repair information based on each file involved in the vulnerability. Each vulnerability-related repair information corresponds to multiple files involved in the vulnerability in the code to be detected. Input each file into the multi-programming language vulnerability impact range detection model trained in step 2. The output of the multi-programming language vulnerability impact range detection model is the line number of the deleted line of each file related to each vulnerability.

[0043] SZZ-type algorithms believe that statements involving vulnerabilities will be deleted, and statements that fix the vulnerabilities will be added. Therefore, in order to trace the source of the vulnerability, the evolution of the vulnerability should be studied with clues from the deletion behavior. Based on this purpose, the present invention first identifies the root cause deletion lines. A vulnerability fix submission may involve multiple files, and each file may contain multiple deletion lines. If all deletion lines of all files are considered at once, the input of LLM will be too large, resulting in a decrease in the accuracy of the results. Therefore, the present invention independently analyzes each file to extract their respective root cause deletion lines. CVE description is CVE description information of the vulnerability related to the code to be detected.

[0044] At present, relevant research has proved that code is not a natural language, and the application of closed-source LLM to real-world security management scenarios is questionable. Therefore, in order to further leverage the advantages of LLM, this paper uses natural language information and Common Vulnerabilities & Exposures (CVE) in the submission metadata to assist LLM in understanding the code in the vulnerability fix submission, thereby capturing the real cause of the vulnerability.

[0045] Furthermore, when guiding LLM to perform tasks, the present invention uses thought chain prompts to guide LLM to conduct in-depth analysis, and divides the analysis task of each file into multiple subtasks to construct thought chain prompts, for example:

[0046] 1. Analyze the commit metadata and diff information to understand the nature of the vulnerability;

[0047] 2. Evaluate the context of each candidate row to determine whether it is likely to contain a vulnerability, classify the likelihood as unlikely, possible, or highly likely, and give reasons;

[0048] 3. Directly output the possible root cause deletion line number.

[0049] An example of LLM's analysis output is shown in the following table:

[0050]

[0051] Step 5. Take the root cause deletion behavior obtained in step 4 as the starting line, use gitblame to obtain the most recent previous commit before the starting line, if the previous commit does not contain the deleted line, stop backtracking, otherwise calculate the Levenshtein distance between each deleted line in the previous commit and the starting line; input the deleted lines whose Levenshtein distance is not less than the threshold as candidate lines into the multi-programming language vulnerability impact range detection model, the multi-programming language vulnerability impact range detection model determines whether the candidate line is related to the vulnerability based on the submission metadata and difference information, if it is not related to the vulnerability, stop backtracking, otherwise output the candidate line with the highest possibility as the vulnerability root cause line of this round; traverse the root cause deletion lines obtained in step 4 in the above way to obtain a set of vulnerability root cause lines, sort all vulnerability root cause lines in the set according to the usage time, and introduce the commit with the earliest usage time into the commit as the candidate vulnerability of the file, the processing process is as follows: Figure 5 shown.

[0052] The vulnerability backtracking decision determined by the present invention is to combine the analysis results of the row mapping algorithm and the large language model to jointly decide whether to continue backtracking. If it is decided to continue backtracking, it is necessary to select the vulnerability root cause row in the vulnerability repair submission to determine the backtracking path, and repeat this process until the backtracking stops.

[0053] In the process of software version iteration and update, the vulnerability root cause line is the vulnerability carrying line, which may remain unchanged or change to a certain extent. The prior art usually uses a line mapping algorithm to match the changed deleted lines. However, these algorithms are mainly based on similarity calculation and fail to make full use of the contextual semantic information of the code. Therefore, the present invention proposes to use LLM for auxiliary selection while combining the Levenshtein distance judgment.

[0054] Specifically, after obtaining the root cause deletion line, use this as the starting point to continue with a deeper backtracking. Execute the git blame command to find the most recent commit. If it does not contain any root cause deletion lines, stop backtracking; otherwise, further calculate the Levenshtein distance between each deletion line in the commit and the root cause line at the starting point. If the Levenshtein distance is less than 0.5, it indicates that the deletion line is very different from the root cause deletion line and the possibility of containing a vulnerability is low, then stop backtracking; if the Levenshtein distance is not less than 0.5, it is used as a valid candidate line and submitted to LLM. If LLM determines that the candidate lines are not related to the vulnerability based on the commit metadata, the diff information of the vulnerability repair commit, its own diff information, and the candidate line number, then stop backtracking; otherwise, select the most likely root cause line of the vulnerability in the commit. This process will eventually get the set of previous commits. After sorting by time, the earliest commit will be used as the real introduction commit.

[0055] The Levenshtein distance is an edit distance that measures the difference between two strings. This distance can help eliminate some root cause deletion lines that are highly likely not related to the vulnerability.

[0056] Step 6: Use a large language model (LLM) to screen the introduction submissions related to each file in the vulnerability repair submission obtained in step 5. That is, the diff information of the vulnerability repair submission and the CVE submission information are used as the input of the large language model. After calculation by the large language model, an introduction submission that is most relevant to the vulnerability is intelligently screened out from multiple candidate introduction submissions.

[0057] After processing from step 1 to step 5, each file in the vulnerability fix submission will be associated with a specific introduction submission. In order to further reduce the false positive rate, the present invention uses LLM to select the submission that is most related to the vulnerability based on the relevant information of the vulnerability fix. Still taking CVE-2016-10049 as an example, Figure 2 As shown, the LLM-SZZ algorithm simultaneously tracks and deletes rows A and B to obtain two previous submissions, 2174484 and 3ed852. Then this step will further screen out 2174484 that is irrelevant to the problem, and finally confirm that 3ed852 is the real vulnerability introduction submission. Compared with V-SZZ, the LLM-SZZ algorithm pays great attention to the exclusion of irrelevant candidates. The present invention will eventually obtain a real vulnerability introduction submission, which not only improves the matching accuracy, but also greatly reduces the possible false positive rate.

[0058] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting the impact range of multiple programming language vulnerabilities based on semantic analysis, characterized in that: The following steps are involved: Step 1: Collect CVE vulnerabilities in various programming languages ​​and their corresponding vulnerability fix submissions and vulnerability introduction submissions, as well as natural language information in the submission metadata, and perform data cleaning and normalization processing; Use the collected data to build training datasets, validation datasets, and test datasets; Step 2: Build a multi-programming language vulnerability impact range detection model based on the large language model. Use the training data set, verification data set, and test data set built in step 1 to complete the training of the multi-programming language vulnerability impact range detection model. Use parsing tools to extract submission metadata from the code to be detected, delete line numbers and difference information, Step 3: Divide the extracted submission metadata, deletion line numbers, and difference information into multiple vulnerability-related repair information based on the files involved in the vulnerability. Each vulnerability-related repair information corresponds to multiple files involved in the code to be detected. Input each file into the multi-programming language vulnerability impact range detection model trained in step 2 to obtain the line number of the root cause deletion line of each file related to each vulnerability. Step 4: Take the root cause deletion behavior obtained in step 3 as the starting line, obtain the most recent previous commit before the starting line, and stop backtracking if the previous commit does not contain the deleted line. Otherwise, calculate the Levenshtein distance between each deleted line in the previous commit and the starting line; input the deleted lines whose Levenshtein distance is not less than the threshold into the multi-programming language vulnerability impact range detection model as candidate lines. The multi-programming language vulnerability impact range detection model determines whether the candidate line is related to the vulnerability based on the submission metadata and difference information. If it is not related to the vulnerability, stop backtracking. Otherwise, output the candidate line with the highest possibility as the root cause line of the vulnerability in this round; Traverse the root cause obtained in step 3 and delete the rows to get the set of vulnerability root cause rows, and use the commit with the earliest vulnerability root cause row as the candidate vulnerability introduction commit for the file; Step 5: Use the diff information of the vulnerability fix submission in the candidate vulnerability introduction submission obtained in step 5 and the CVE submission information as the input of the large language model. After calculation by the large language model, select the introduction submission that is most relevant to the vulnerability from multiple candidate vulnerability introduction submissions.

2. The method for detecting the impact range of multiple programming language vulnerabilities according to claim 1, characterized in that: The submission metadata in step 2 is submission information related to the vulnerability repair submission in the code to be detected.

3. The method for detecting the impact range of multiple programming language vulnerabilities according to claim 2, characterized in that: The commit information is obtained by extracting the vulnerability repair commit using the GitHub parsing tool PyDriller.

4. The method for detecting the impact range of multiple programming language vulnerabilities according to claim 1, characterized in that: The deleted line numbers in step 2 are the line numbers of all deleted lines of the code to be detected.

5. The method for detecting the impact range of multiple programming language vulnerabilities according to claim 1, characterized in that: The difference information in step 2 is the patch block information where each deleted line is located.

6. The method for detecting the impact range of multiple programming language vulnerabilities according to claim 1, characterized in that: The thinking chain prompt is used to guide the multi-programming language vulnerability impact range detection model to complete the analysis.

7. The method for detecting the impact range of multiple programming language vulnerabilities according to claim 1, characterized in that: The step 3 also includes inputting each file and the common vulnerability disclosure description into the multi-programming language vulnerability impact range detection model trained in step 2 to obtain the line number of the root cause deletion line of each file related to each vulnerability.

8. The method for detecting the impact range of multiple programming language vulnerabilities according to claim 1, characterized in that: The method of obtaining the most recent previous commit before the starting line in step 4 is to use gitblame to obtain it.

Images