A vulnerability patch sequencing method based on decomposition analysis and progressive comparison

Abstract

This invention provides a vulnerability patch sorting method based on decomposition analysis and progressive comparison. The method includes the following steps: For each candidate submission, a structured decomposition analysis report containing vulnerability-related and irrelevant changes is generated using a large language model; using this report and historical remediation cases, each candidate submission is binary-classified, with submissions classified as "yes" forming a set Y and submissions classified as "no" forming a set N, and set Y is concatenated before set N to form an initial sorting list L; for the submissions in the initial sorting list L, an insertion sort strategy is used, and pairwise comparisons are performed using the large language model to determine the final position of each submission in the ordered result list R, outputting the final sorting list. The technical solution of this invention filters out code noise, solves the problem of inability to sort multiple candidate submissions internally, and significantly improves the accuracy of vulnerability patch location and the robustness of sorting.

Description

Technical Field

[0001] This invention relates to the fields of network security and software security technology, and in particular to a vulnerability patch sorting method based on decomposition analysis and progressive comparison. Background Technology

[0002] In open-source software security maintenance, once a publicly disclosed vulnerability (CVE) is revealed, it is crucial to quickly and accurately locate the patch submission that actually fixes the vulnerability from a large number of code commits to aid in security analysis, vulnerability verification, and risk assessment. Existing vulnerability patch location methods are mainly divided into three categories, all of which achieve this by sorting candidate commits.

[0003] The first category is learning-based ranking methods. These methods treat ranking as a learning task, using a learning-based ranking model and relying on manually designed features such as code change statistics, author history, and commit message keywords. This requires a large amount of manually labeled training data, resulting in complex feature engineering and limited generalization ability to new repositories or new vulnerability types. The second category is ranking methods based on fine-tuning pre-trained language models. These methods use a pre-trained model based on code and fine-tune it on labeled data to determine the matching relationship between commits and CVEs. However, they still rely on high-quality labeled data, and the model is prone to overfitting to specific vulnerability patterns in the training set, resulting in insufficient cross-project generalization ability. The third category is classification-based ranking methods based on large language models. These methods perform independent binary classification judgments on each candidate commit and then rank all commits judged as "yes" before those judged as "no" to complete the ranking.

[0004] In summary, the existing technology has the following defects: (1) Independent judgment of single candidates leads to inconsistent ranking: The existing large language model method performs isolated binary classification on the candidate set, and then roughly sorts them in the manner of "yes group first, no group last", while the original disordered state is maintained within the group. When multiple submissions are judged as "yes", it is impossible to further distinguish which one is more likely to be the real fix, which may cause the real fix patch to be submerged in false positive results. (2) Lack of deconstructive analysis of the submission content, the entire submission difference and message are input into the large language model as a black box, without guiding the model to distinguish between "code changes related to the vulnerability" and "code changes unrelated to the vulnerability". This makes the model easily misled by irrelevant changes in the submission, resulting in misclassification. Summary of the Invention

[0005] To address the above technical issues, this invention discloses a vulnerability patch ranking method based on decomposition analysis and progressive comparison. This method overcomes the shortcomings of existing vulnerability patch ranking methods, such as reliance on manual annotation, weak generalization ability, isolated binary classification leading to disorder within groups, and susceptibility to being misled by irrelevant code changes. It does not require supervised training data, has stronger cross-project generalization ability and anti-interference ability, and can accurately rank real vulnerability patch repairs to the top. It is suitable for scenarios such as automated code review, vulnerability verification script generation, and cybersecurity teaching range.

[0006] The technical solution adopted by this invention is as follows:

[0007] A vulnerability patch sorting method based on decomposition analysis and progressive comparison includes the following steps:

[0008] Step S1, Decomposition and Analysis Phase: The candidate commit set to be sorted and the pre-collected historical real patch commit set are respectively structured and decomposed. For each candidate commit, a structured decomposition and analysis report is generated using a large language model based on the target CVE description, its code differences and commit message. The decomposition and analysis report includes at least vulnerability-related change analysis and vulnerability-independent change analysis.

[0009] Step S2, retrieval of the enhanced initial binary classification stage: using the decomposition analysis report and historical repair cases, perform binary classification on each candidate submission, form a set Y of submissions classified as "yes" and a set N of submissions classified as "no", and concatenate set Y before set N to form an initial sorted list L;

[0010] Step S3, Pairwise Comparison and Insertion Sort Stage: For the submissions in the initial sorted list L, the insertion sort strategy is adopted, and pairwise comparisons are performed using the large language model to determine the final position of each submission in the ordered result list R, and the final sorted list is output.

[0011] This technical solution employs forced decomposition analysis, utilizing a large language model to generate a structured decomposition report for each submission to distinguish between relevant and irrelevant changes. This effectively differentiates between vulnerability-related and irrelevant changes, reduces noise interference, and improves judgment accuracy. Search enhancement is used to generate historical repair examples for initial binary classification, providing the model with a concrete comparison benchmark and compensating for the shortcomings of isolated judgments. Finally, pairwise comparisons and insertion sort are used to relatively rank all candidates, resolving the issue of the inability to sort within the "yes" set, ensuring that truly patched changes are prioritized, thereby improving the accuracy and robustness of the ranking.

[0012] As a further improvement of the present invention, in step S1, the structured decomposition analysis report is generated in the following way: the target CVE description, complete code differences and submission messages are combined according to a preset template and input into a large language model, and the model is forced to output structured text including three dimensions: code change analysis, vulnerability-related reasoning and vulnerability-irrelevant reasoning.

[0013] As a further improvement of the present invention, step S2, utilizing historical repair cases includes: converting the decomposition analysis report of the candidate submission generated in step S1 into a vector representation, using the vector as a query vector, and retrieving one or more of the most similar historical real repair patch cases from the pre-built historical repair case vector index.

[0014] As a further improvement of the present invention, the construction method of the historical repair case vector index is as follows: a set of historical real repair patch submissions is collected in advance, step S1 is executed for each historical submission to generate its decomposition analysis report, and the report text is converted into a vector through an embedding model and then stored in the index.

[0015] As a further improvement of the present invention, the binary classification in step S2 includes: constructing a classification prompt by combining the decomposition analysis report and code differences of the retrieved historical repair cases with the target CVE description, the decomposition analysis report and code differences of the current candidate submission, inputting the prompt into a large language model, and having the model output the judgment result of whether the current candidate submission is a patch to repair the CVE.

[0016] As a further improvement of the present invention, step S3 includes:

[0017] Sub-step S301: Initialize the sorted list. Create an empty sorted result list R, and put the first element of the initial sorted list L into R;

[0018] Sub-step S302: Pairwise comparison and position determination. For each remaining element c_new in L, starting from the last element of R and proceeding backwards, perform pairwise comparisons with c_new. Use the large language model to determine "which commit is more likely to be the real fix patch", and determine the insertion position of c_new based on the comparison results.

[0019] Sub-step S303: Insertion and list update. Insert c_new at the insertion position determined in sub-step S302, and update the ordered list R;

[0020] Sub-step S304: Output the sorting result. After all elements in L have been processed, output R as the final sorted list.

[0021] As a further improvement of the present invention, during the pairwise comparison, a prompt is generated to input the target CVE description, c_new and the decomposition analysis report and code differences of the currently compared element, requiring the large language model to force an answer of "new submission is better" or "currently sorted submission is better"; if c_new is determined to be better, the comparison continues forward; otherwise, the comparison stops, and the insertion position of c_new is determined to be after the currently compared element.

[0022] As a further improvement of the present invention, the submission ranked first in the final sorting list is considered to be the vulnerability patch most likely to actually fix the target CVE.

[0023] This invention also discloses a vulnerability patch sorting system based on decomposition analysis and progressive comparison, comprising:

[0024] The decomposition analysis module is used to generate a structured decomposition analysis report for each candidate submission based on the target CVE description, its code differences, and the submission message, using a large language model. The decomposition analysis report includes at least vulnerability-related change analysis and vulnerability-independent change analysis.

[0025] The retrieval enhancement initial classification module uses the decomposition analysis report and historical repair cases to perform binary classification on each candidate submission. Submissions classified as "yes" are grouped into set Y, and submissions classified as "no" are grouped into set N. Set Y is then concatenated before set N to form an initial sorted list L.

[0026] The pairwise comparison insertion sort module is used to perform pairwise comparisons on the submissions in the initial sorted list L using an insertion sort strategy and a large language model to determine the final position of each submission in the ordered result list R, and output the final sorted list.

[0027] As a further improvement of the present invention, the structured decomposition analysis report is generated in the following way: the target CVE description, complete code differences and commit messages are combined according to a preset template and input into a large language model, and the model is forced to output structured text including three dimensions: code change analysis, vulnerability-related reasoning and vulnerability-irrelevant reasoning.

[0028] As a further improvement of the present invention, utilizing historical repair cases includes: converting the candidate submission decomposition analysis report generated by the decomposition analysis module into a vector representation, using the vector as a query vector, and retrieving one or more of the most similar historical real repair patch cases from a pre-built historical repair case vector index.

[0029] As a further improvement of the present invention, the construction method of the historical repair case vector index is as follows: a set of historical real repair patch submissions is collected in advance, the decomposition analysis module is executed for each historical submission to generate its decomposition analysis report, and the report text is converted into a vector through an embedding model and then stored in the index.

[0030] As a further improvement of the present invention, the binary classification includes: constructing a classification prompt by combining the decomposition analysis report and code differences of the retrieved historical repair cases with the target CVE description, the decomposition analysis report and code differences of the current candidate submission, inputting the prompt into a large language model, and having the model output a judgment result on whether the current candidate submission is a patch to repair the CVE.

[0031] As a further improvement of the present invention, the pairwise comparison insertion sort module performs the following steps:

[0032] Sub-step S301: Initialize the sorted list. Create an empty sorted result list R, and put the first element of the initial sorted list L into R;

[0033] Sub-step S302: Pairwise comparison and position determination. For each remaining element c_new in L, starting from the last element of R and proceeding backwards, perform pairwise comparisons with c_new. Use the large language model to determine "which commit is more likely to be the real fix patch", and determine the insertion position of c_new based on the comparison results.

[0034] Sub-step S303: Insertion and list update. Insert c_new at the insertion position determined in sub-step S302, and update the ordered list R;

[0035] Sub-step S304: Output the sorting result. After all elements in L have been processed, output R as the final sorted list.

[0036] As a further improvement of the present invention, during the pairwise comparison, a prompt is made to input the target CVE description, c_new and the decomposition analysis report and code differences of the currently compared element, requiring the large language model to force an answer of "new submission is better" or "currently sorted submission is better"; if c_new is determined to be better, the comparison continues forward; otherwise, the comparison stops, and the insertion position of c_new is determined to be after the currently compared element.

[0037] As a further improvement of the present invention, the submission ranked first in the final sorting list is considered to be the vulnerability patch most likely to actually fix the target CVE.

[0038] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vulnerability patch sorting method based on decomposition analysis and progressive comparison as described above.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] By adopting the technical solution of this invention, the vulnerability-related changes and irrelevant changes can be effectively distinguished through forced decomposition analysis, reducing noise interference and improving the accuracy of judgment. The retrieval enhancement of historical repair cases is introduced to provide specific comparison benchmarks for the model and make up for the defects of isolated judgment. The progressive mechanism of pairwise comparison and insertion sort is adopted to solve the problem that the "yes" set cannot be sorted, so that the real repair patch is placed at the top. Attached Figure Description

[0041] Figure 1 This is an overall flowchart of a vulnerability patch sorting method based on decomposition analysis and progressive comparison, according to Embodiment 1 of the present invention.

[0042] Figure 2 This is a detailed flowchart of the pairwise comparison and insertion sort stage in step S3 of Embodiment 1 of the present invention. Detailed Implementation

[0043] The preferred embodiments of the present invention will be described in further detail below.

[0044] Example 1

[0045] A vulnerability patch sorting method based on decomposition analysis and progressive comparison, the overall process is as follows: Figure 1 As shown, the process is divided into three sequentially executed phases: First, through a decomposition and analysis phase, each candidate submission is broken down into vulnerability-related changes and irrelevant changes; second, through an enhanced initial binary classification phase, candidate submissions are categorized into priority and backup groups using historical remediation cases, achieving initial ordering; finally, through a pairwise comparison and insertion sort phase, a fine-grained sort is performed, ultimately placing submissions that truly remediate the target CVE at the top of the list. The specific steps are as follows:

[0046] Step S1, decomposition and analysis stage.

[0047] This stage forms the basis for subsequent progressive ranking and aims to overcome the shortcomings of existing technologies that treat commit differences as a black box. Existing methods directly input raw code differences and commit information into the model, which cannot distinguish which code changes are related to vulnerability fixes and which are irrelevant interferences, leading to subsequent judgments being easily overwhelmed by a large amount of noise. This stage forces a large language model to perform a structured decomposition of each commit, transforming implicit code changes into explicit three-dimensional analysis reports, providing a denoised, high-quality semantic foundation for subsequent retrieval matching and pairwise comparisons.

[0048] This phase processes two types of submissions: the current set of candidate submissions to be sorted, and a pre-collected set of historical real patch submissions. The specific processing flow is as follows: For each submission, the system combines its corresponding CVE description text, complete code differences, and submission message according to a template and inputs them into the large language model. This template requires the model to decompose the data along three dimensions and output a structured report: ① Code change analysis, objectively describing which files were modified and the addition or deletion of key code blocks; ② Vulnerability-related reasoning, focusing on code changes and logic adjustments that may directly fix the CVE and explaining how they block attack paths; ③ Vulnerability-unrelated reasoning, explicitly identifying changes unrelated to the vulnerability, such as code refactoring, formatting, and the addition or deletion of comments. This forced separation design compels the model to actively compare the semantic relationship between the CVE description and code changes during reasoning, rather than making a general judgment on the relevance of the entire code difference, thereby effectively filtering out noise.

[0049] After completing the decomposition analysis of all submissions, the decomposition report text of historical patch submissions is converted into vectors using an embedding model and stored in a vector index, providing knowledge support for historical case retrieval in stage S102. Decomposition reports of candidate submissions are also generated as vector representations, but are only used as query vectors in stage S102 to retrieve historical cases from the index. This stage outputs a structured decomposition analysis report and its corresponding vector representation for each candidate submission.

[0050] Step S2, Retrieve Enhanced Initial Binary Classification Stage

[0051] This stage utilizes the decomposition analysis report and historical repair cases generated in step S1 to perform preliminary binary classification judgment on each candidate submission, generating an initial sequence that is nearly ordered. The core logic is: for each candidate submission, using the vector generated in step S1 as the query vector, the most similar historical real repair patch case is retrieved from the historical case vector index constructed in step S1, serving as a reliable positive example reference.

[0052] The retrieved historical cases, including decomposition analysis reports and code differences, are used together with the target CVE description, the decomposition analysis reports of candidate submissions, and code differences to construct classification suggestions for input into the large language model. The role of historical cases is to provide the large language model with a concrete reference of "what a real fix patch for this type of CVE typically looks like." This reference mechanism effectively compensates for the shortcomings of isolated judgments, where the model relies solely on current submission information and lacks a comparison benchmark.

[0053] After traversing all candidate commits and classifying them, all commits marked "yes" are grouped into a set Y, and all commits marked "no" are grouped into a set N. Sets Y and N maintain their original input order. Y is concatenated before N to form an initial sorted list L. This list is nearly ordered, and most of the actual fix commits are now near the top. This stage outputs a preliminary sorted list of candidate commits L.

[0054] Step S3, pairwise comparison and insertion sort stage.

[0055] This stage is the second step of the progressive sorting process, involving fine-tuning. By introducing pairwise comparisons and insertion sort mechanisms, it addresses the issue of confusion between true and false submissions within the "yes" set that might occur in stage S102. It gradually differentiates the merits of multiple seemingly similar candidates, prioritizing truly corrective submissions. The specific process is as follows: Figure 2 As shown, it includes the following sub-steps:

[0056] Sub-step S301: Initialize the ordered list. Create an empty ordered result list R, and put the first element c1 of the initial list L obtained in stage S102 into R.

[0057] Sub-step S302: Compare pairs and determine positions.

[0058] For each remaining element c_new in L, an insertion sort strategy is used, starting from the last element of R and comparing it with c_new in pairs from back to front. During each comparison, a prompt is generated to input the target CVE description, a decomposition analysis report of c_new and the currently compared element R[j], and code differences. The large language model is then asked to determine: "Which commit is more likely to be the real fix patch?" and is forced to answer either "The new commit is better" or "The currently sorted commit is better". If c_new is considered better, the comparison continues forward until the beginning of the sequence; otherwise, it stops, and the insertion position of c_new is determined to be after the currently compared element.

[0059] Sub-step S303, Insertion and list update.

[0060] Insert c_new into the position determined in S103-2, and update the ordered list R. After all elements in L have been processed and inserted in sequence, R is the complete ordered list after pairwise comparisons.

[0061] Sub-step S304: Output the sorting results.

[0062] Depending on downstream requirements, a complete ordered list R can be output directly, or the top K commits can be truncated as the final output. The commit ranked first in the list is the one most likely to actually fix the CVE, as determined by this method after decomposition analysis and progressive sorting. This commit is available for direct confirmation by security analysts or for use in subsequent automated processes.

[0063] This stage outputs the final sorted candidate submission list, completing the sorting of vulnerability patches.

[0064] Example 2

[0065] Based on Example 1, this example discloses a vulnerability patch sorting system based on decomposition analysis and progressive comparison, including:

[0066] The decomposition analysis module is used to execute step S1 of Example 1;

[0067] The retrieval enhances the initial classification module, which is used to perform step S2 of Example 1;

[0068] Pairwise comparison insertion sort module: used to perform step S3 of embodiment 1.

[0069] Example 3

[0070] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the vulnerability patch sorting method based on decomposition analysis and progressive comparison of Embodiment 1.

[0071] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A vulnerability patch sorting method based on decomposition analysis and progressive comparison, characterized in that, Includes the following steps: Step S1: For each candidate submission, based on the target CVE description, its code differences and submission message, a structured decomposition analysis report is generated using a large language model. The decomposition analysis report includes vulnerability-related change analysis and vulnerability-independent change analysis. Step S2: Using the decomposition analysis report and historical repair cases, each candidate submission is classified into two categories. Submissions classified as "yes" are grouped into set Y, and submissions classified as "no" are grouped into set N. Set Y is then concatenated before set N to form an initial sorted list L. Step S3: For the submissions in the initial sorted list L, use the insertion sort strategy, perform pairwise comparisons using the large language model, determine the final position of each submission in the ordered result list R, and output the final sorted list. Step S3 includes: Sub-step S301: Create an empty ordered result list R, and put the first element of the initial sorted list L into R; Sub-step S302: For each remaining element c_new in L, starting from the last element of R, compare it with c_new in pairs from back to front. Use the large language model to determine "which commit is more likely to be the real patch", and determine the insertion position of c_new based on the comparison results. Sub-step S303: Insert c_new into the insertion position determined in sub-step S302, and update the ordered list R; Sub-step S304: After all elements in L have been processed, output R as the final sorted list.

2. The vulnerability patch sorting method based on decomposition analysis and progressive comparison according to claim 1, characterized in that: In step S1, the structured decomposition analysis report is generated in the following way: the target CVE description, complete code differences and commit messages are combined according to a preset template and input into the large language model, and the model is forced to output structured text including three dimensions: code change analysis, vulnerability-related reasoning and vulnerability-irrelevant reasoning.

3. The vulnerability patch sorting method based on decomposition analysis and progressive comparison according to claim 2, characterized in that: In step S2, utilizing historical remediation cases includes: converting the decomposition analysis report of the candidate submission generated in step S1 into a vector representation, using this vector as a query vector, and retrieving one or more of the most similar historical real remediation patch cases from the pre-built historical remediation case vector index; The construction method of the historical repair case vector index is as follows: a set of historical real repair patch submissions is collected in advance, step S1 is executed for each historical submission to generate its decomposition analysis report, and the report text is converted into a vector through an embedding model and then stored in the index; The binary classification in step S2 includes: constructing classification hints by combining the decomposition analysis report and code differences of the retrieved historical remediation cases with the target CVE description and the decomposition analysis report and code differences of the current candidate submission, inputting them into the large language model, and having the model output the judgment result of whether the current candidate submission is a patch to fix the CVE.

4. The vulnerability patch sorting method based on decomposition analysis and progressive comparison according to claim 1, characterized in that: During the pairwise comparison, the system prompts the input of the target CVE description, c_new, and the decomposition analysis report and code differences of the currently compared element, requiring the large language model to force an answer of "new submission is better" or "currently sorted submission is better"; If c_new is determined to be better, the comparison continues forward; otherwise, the comparison stops, and the insertion position of c_new is determined to be after the currently compared element. The submission ranked first in the final sorted list is considered the most likely patch to actually fix the target CVE.

5. A vulnerability patch sorting system based on decomposition analysis and progressive comparison, characterized in that, include: The decomposition analysis module is used to generate a structured decomposition analysis report for each candidate submission based on the target CVE description, its code differences, and the submission message, using a large language model. The decomposition analysis report includes vulnerability-related change analysis and vulnerability-independent change analysis. The retrieval enhancement initial classification module uses the decomposition analysis report and historical repair cases to perform binary classification on each candidate submission. Submissions classified as "yes" are grouped into set Y, and submissions classified as "no" are grouped into set N. Set Y is then concatenated before set N to form an initial sorted list L. The pairwise comparison insertion sort module is used to perform pairwise comparisons on the submissions in the initial sorted list L using an insertion sort strategy and a large language model to determine the final position of each submission in the ordered result list R, and output the final sorted list. The pairwise comparison insertion sort module performs the following steps: Sub-step S301: Create an empty ordered result list R, and put the first element of the initial sorted list L into R; Sub-step S302: For each remaining element c_new in L, starting from the last element of R, compare it with c_new in pairs from back to front. Use the large language model to determine "which commit is more likely to be the real patch", and determine the insertion position of c_new based on the comparison results. Sub-step S303: Insert c_new into the insertion position determined in sub-step S302, and update the ordered list R; Sub-step S304: After all elements in L have been processed, output R as the final sorted list.

6. The vulnerability patch sorting system based on decomposition analysis and progressive comparison according to claim 5, characterized in that: The structured decomposition analysis report is generated in the following way: the target CVE description, complete code differences and commit messages are combined according to a preset template and input into a large language model, and the model is forced to output structured text including three dimensions: code change analysis, vulnerability-related reasoning and vulnerability-irrelevant reasoning.

7. The vulnerability patch sorting system based on decomposition analysis and progressive comparison according to claim 5, characterized in that: Utilizing historical repair cases includes: converting the candidate submission decomposition analysis report generated by the decomposition analysis module into a vector representation, using this vector as a query vector, and retrieving one or more of the most similar historical real repair patch cases from the pre-built historical repair case vector index; The historical repair case vector index is constructed as follows: a set of historical real repair patch submissions is collected in advance, the decomposition analysis module is executed on each historical submission to generate its decomposition analysis report, and the report text is converted into a vector through an embedding model and then stored in the index; The binary classification includes: constructing classification prompts by combining the decomposition analysis reports and code differences of retrieved historical remediation cases with the target CVE description and the decomposition analysis reports and code differences of the current candidate submission, inputting them into a large language model, and having the model output the judgment result of whether the current candidate submission is a patch to fix the CVE.

8. The vulnerability patch sorting system based on decomposition analysis and progressive comparison according to claim 5, characterized in that: During the pairwise comparison, the system prompts the input of the target CVE description, c_new, and the decomposition analysis report and code differences of the currently compared element, requiring the large language model to force an answer of "new submission is better" or "currently sorted submission is better"; If c_new is determined to be better, the comparison continues forward; otherwise, the comparison stops, and the insertion position of c_new is determined to be after the currently compared element. The submission ranked first in the final sorted list is considered the most likely patch to actually fix the target CVE.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the vulnerability patch sorting method based on decomposition analysis and progressive comparison as described in any one of claims 1 to 4.

Images