Version-level open vulnerability influence library identification method based on multi-source data enhanced LLM in cross-ecological scene

Abstract

The invention discloses a version-level open vulnerability influence library identification method based on multi-source data enhanced LLM in a cross-ecological scene, and the method comprises the steps: S1, constructing a cross-ecological system multi-source vulnerability data knowledge base, comprising a version-level vulnerability library annotation data set, a multi-source vulnerability information data set, a real library name and version data set, a patch submission annotation data set and a vulnerability information abstract annotation data set; s2, performing input enhancement, performing vulnerability data quality evaluation by adopting an improved CodeBERT model, and realizing long vulnerability text key information extraction in combination with a BART model, thereby effectively improving the credibility and the structured degree of multi-source vulnerability data; s3, vulnerability information identification: constructing an Alpaca instruction template, and performing fine tuning training on a large language model by using a labeling data set through a multi-source data context fusion and low rank adaptation (LoRA) parameter optimization strategy, so as to strengthen the understanding ability of the model for vulnerability features; and S4, result post-processing: based on similarity matching and patch submission association analysis, remarkably reducing hallusion deviation output by the large language model.

Description

Technical Field

[0001] This invention relates to the field of open vulnerability impact library identification, specifically a method for identifying version-level vulnerability impact libraries using deep learning technology to enhance a large language model in a cross-third-party library ecosystem scenario. Background Technology

[0002] Third-party library dependencies have become a critical infrastructure of the modern software engineering ecosystem. By integrating proven functional modules, developers can effectively reduce the duplication of underlying logic implementation. Over the past two decades, the collaborative evolution mechanism of the open-source community has led to an exponential increase in the use of third-party libraries, making applications without integrated third-party libraries marginal cases. However, while developers achieve rapid feature iteration through integrating third-party libraries, the attack surface of the software supply chain is expanding significantly. Because third-party libraries typically have cross-project reusability, components with known vulnerabilities can easily become attack propagation multipliers, especially in scenarios lacking systematic security review mechanisms. The spread rate and impact of vulnerabilities often exhibit non-linear, exponential growth characteristics.

[0003] To address these security threats, software maintainers typically utilize standardized vulnerability disclosure systems for vulnerability management, such as Common Vulnerabilities and Exposures (CVEs). This system uses a consistent identifier mechanism to map vulnerability remediation information to public vulnerability databases (such as the National Vulnerability Database, NVD), thereby building a basic protection knowledge base. Industry practice dictates that maintainers regularly check whether project dependency libraries contain CVE entries recorded in the NVD; this is the first line of defense in supply chain security. However, empirical research also indicates that this protection system is prone to failure: firstly, there are defects in the source data, such as missing affected library identifiers and inaccurate vulnerability feature descriptions in NVD vulnerability reports; secondly, there is a bottleneck in manual processing, as manually tracing affected dependency libraries based on NVD entries carries a high risk of misjudgment.

[0004] Given the limitations of the traditional protection models mentioned above, academia and industry are working to develop automated vulnerability impact databases and affected version identification.

[0005] The current mainstream methods for automatically identifying the library names of affected libraries can be divided into three major technical routes based on their core mechanisms: (1) Based on extreme multi-label classification: represented by Chronos, which establishes a mapping relationship between vulnerability reports and third-party libraries by predefining the affected library pool; (2) Based on deep learning / machine learning: typically VulLibMiner, which uses neural networks to extract library features from vulnerability reports; (3) Based on large language models: the typical solution VulLibGen generates the names of potentially affected libraries through LLM.

[0006] Regarding the identification of vulnerability-affected versions, existing research has not yet formed a systematic research framework starting from vulnerability report text. The mainstream methods still heavily rely on reverse derivation of patch submissions, with representative methods including AFV and Vision.

[0007] Existing research still faces the following core problems:

[0008] Problem 1—Limitations of Ecosystem Adaptability: Existing methods are mainly developed for specific ecosystems such as Maven and NPM, and their performance declines significantly in cross-ecosystem scenarios;

[0009] Question 2—Data Source Reliability: This issue manifests in two ways. First, there is the limitation of the input source. LLM-based methods typically rely solely on NVD vulnerability reports as input, which differs from the practice of security experts who rely on multiple sources of information (vulnerability information aggregation websites, security announcements from ecosystem vendors, etc.) for cross-validation. Second, there is the issue of data quality. Existing research on vulnerability datasets mainly relies on vulnerability information aggregation websites such as GitHub Advisory and Snyk. Due to the inconsistent quality of the original data, the accuracy of the identification methods is limited.

[0010] Issue 3—Insufficient Version-Level Vulnerability Detection and Dependence on Patch Location: Most methods can only identify vulnerabilities at the library name level, lacking the ability to detect affected library versions. Existing version identification schemes (such as methods based on weighted program dependency graphs) heavily rely on the accurate location of patch submissions, and their reliability decreases when a complete code change record is lacking.

[0011] Therefore, it is necessary to propose a method for identifying the impact of version-level open vulnerabilities on libraries based on multi-source data-enhanced LLM in cross-ecosystem scenarios to solve the above problems. Summary of the Invention

[0012] The purpose of this invention is to address the shortcomings of existing methods by providing a cross-ecosystem version-level vulnerability impact library identification method based on a large language model. By constructing a fine-grained vulnerability dataset covering 12 ecosystems including Maven, NPM, and PyPI, developing a multi-source heterogeneous data fusion engine, and combining it with a domain knowledge-enhanced instruction fine-tuning strategy, multi-granularity correlation analysis from vulnerability reports to affected libraries and version ranges is achieved.

[0013] like Figure 1 To address the aforementioned technical problems, this invention provides the following technical solution: a method for identifying version-level open vulnerability impact libraries based on multi-source data-enhanced LLM in cross-ecosystem scenarios, comprising the following steps:

[0014] S1. Construct a cross-ecosystem multi-source vulnerability data knowledge base, including version-level vulnerability database annotation datasets, multi-source vulnerability information datasets, real database name and version datasets, patch submission annotation datasets, and vulnerability information summary annotation datasets;

[0015] S2. Input enhancement: An improved CodeBERT model is used to assess the quality of vulnerability data, and the BART model is combined to extract key information from long vulnerability texts, effectively improving the credibility and structure of multi-source vulnerability data.

[0016] S3. Vulnerability information identification: Construct Alpaca instruction templates, and use labeled datasets to fine-tune the large language model through multi-source data context fusion and low-rank adaptation (LoRA) parameter optimization strategies to enhance the model's ability to understand vulnerability features.

[0017] S4. Post-processing of results: Based on similarity matching and patch submission correlation analysis, the illusion bias of the output of large language models is significantly reduced.

[0018] Preferably, the construction of the version-level vulnerability database annotation dataset in S1 adopts a strict manual review process:

[0019] First, candidate vulnerability reports were collected from NVD and filtered for validity. Then, automated tools combined with manual verification were used to accurately locate affected libraries and their respective ecosystems (such as Maven, NPM, PyPI, etc.), and to extract the affected version ranges and patch information. Finally, multiple researchers independently labeled and cross-validated discrepancies (e.g., using Kappa coefficients to quantify consistency), resulting in a high-quality labeled dataset containing structured fields such as ecosystem, package name, and semantic version constraints. This process ensures the high accuracy and reliability of the dataset across ecosystems, providing a solid benchmark for subsequent model training and evaluation.

[0020] Preferably, the vulnerability data quality assessment model in S2 is based on an improved CodeBERT architecture:

[0021] For affected library records extracted from vulnerability information aggregation websites (such as GitHub Advisory and Snyk), the model simultaneously inputs the record information, the corresponding NVD vulnerability description, and the vulnerability description text from the website where the record originated. After lexical concatenation, the input is fed into the encoder, fusing the [CLS] tag vector with the pooling layer output to form a hybrid feature. A binary classification layer is then used to output the confidence score of the record. This model can effectively identify and filter low-quality or mislabeled records (such as the incorrectly associated org.apache.mesos / mesos library record in GitLabAdvisory), retaining only high-quality records with a confidence score higher than a set threshold (e.g., 0.5) for subsequent context construction, significantly improving the credibility of the input data.

[0022] Preferably, the multi-source data context fusion strategy in S3 integrates three types of key information sources:

[0023] 1) Core metadata of the NVD report after digest processing (CVE ID, concise description, structured CPE / reference link keywords);

[0024] 2) High-confidence vulnerability information compiled from various websites, selected using a quality assessment model;

[0025] 3) Standardized vulnerability information extracted from security bulletins of ecosystem vendors.

[0026] This information is structured and injected into predefined Alpaca instruction templates.

[0027] Preferably, the low-rank adaptation (LoRA) fine-tuning strategy employs specific hyperparameter configurations (e.g., rank lora_r=8, lora_alpha=16). While freezing the original weights of the large language model (e.g., Llama3.1-8B), it optimizes only the low-rank adaptation matrix added to the attention layer (matrix A is Gaussian initialized, matrix B is zero initialized). This strategy efficiently injects cross-ecosystem vulnerability knowledge (e.g., library name specifications, version constraint syntax, ecosystem identifiers) into the model, significantly improving its ability to jointly identify "ecosystem + library name + version range" with limited computational overhead.

[0028] Preferably, the specific details of generating the target code vulnerability detection thought chain in S3 include:

[0029] Based on the mapping from the first component of the initial judgment information in S2, namely the static tool analysis results, to the operating system kernel classification hierarchy tree, the corresponding basic guidance prompts of the thinking chain are extracted. Then, based on the second component of the initial judgment information, namely the test results of the deep learning small model, the strictness of the prompt template is adjusted. Combining the content of the two and generating high-quality input that stimulates the classification potential of the large language model with strict output format requirements.

[0030] Preferably, the similarity-matching-based illusion elimination algorithm targets fictitious library names or version numbers that may be generated by a large model:

[0031] This algorithm utilizes a constructed dataset of real library names and versions as a knowledge base. For unmatched library names, it finds and replaces them with the most similar real library names by tokenization, removal of common words, and calculation of a comprehensive similarity score based on the longest common subsequence (LCS) and tail token overlap. For unmatched version numbers, it parses the semantic version number (major.minor.revision.modifier), removes common prefixes, and calculates a hierarchically weighted numerical matching score and sequence similarity score (major version has the highest weight) to locate the closest real version. This algorithm effectively solves common illusion problems such as mistakenly writing "springboot" instead of "spring-boot" or "v2.1.4" instead of "2.1.4".

[0032] Preferably, the patch submission correlation analysis is used to correct and fix the version:

[0033] When the NVD report is not associated with a patch commit, the code repository of the affected library is located, and candidate code commits are screened within the time window before and after the vulnerability disclosure. The commit information and code differences (git diff) of each candidate commit are extracted. A pre-trained binary model is used to calculate the probability of it being a general patch commit (vfc_prob) and the probability of it being a specific CVE fix commit (vfc_prob_for_cve). This is combined with six-dimensional features, including semantic similarity between the commit information and the vulnerability description, path similarity, whether the CVE ID appears in the commit information, and whether the vulnerability type belongs to the Top 10. The final commit ranking is generated by inputting the logistic regression ranking model. The version tag associated with the Top 1 commit is used to correct the patch version number output by the large model (e.g., correcting the model output nightly-2024-05-12 to nightly-2024-06-06-02-00-53, which includes the actual fix). This method can still accurately correct the version range through code change analysis even in the absence of explicit patch links.

[0034] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method for identifying version-level open vulnerability impact libraries based on multi-source data augmentation LLM in cross-ecosystem scenarios.

[0035] A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method for identifying version-level open vulnerability impact libraries based on multi-source data augmentation LLM in cross-ecosystem scenarios.

[0036] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: To address the issue of cross-ecosystem universality, a fine-grained labeled dataset covering 12 ecosystems, including Maven, NPM, and PyPI, is constructed. Augmented data is used to construct context during the fine-tuning training phase of the large model, enhancing the cross-ecosystem vulnerability impact database and version identification capabilities of the large language model. To address the limitations of single-source information, a multi-source heterogeneous data integration engine is developed, integrating vulnerability data from comprehensive vulnerability information websites, security announcements from ecosystem vendors, and patch submissions from repositories, among other security data sources. Specifically, to address the data quality issues of comprehensive vulnerability information websites, automated data quality selection and noise filtering are implemented. To address the insufficient version-level vulnerability detection, contextual semantic information about affected and patched versions in vulnerability reports is injected into the large language model, enhancing its fine-grained understanding of the version impact range. Furthermore, feature engineering-driven patch submission localization technology is combined to further refine the version identification results of the large model using patch submissions, reducing dependence on patch submissions while improving version identification accuracy.

[0037] The main contributions of this invention can be summarized in the following three aspects: First, by constructing a vulnerability impact database dataset covering 12 ecosystems through a manual review process, a precise mapping from vulnerability reports to affected libraries (library name + version number) is achieved, overcoming the limitations of existing datasets with only a single granularity. Second, by fully exploring the potential of Large Language Models (LLM), a two-level identification architecture is proposed, simultaneously achieving the dual goals of library name identification and version identification. Third, a multi-source information extraction pipeline is established, integrating vulnerability features from multiple sources such as NVD reports, vulnerability information websites, and vendor security announcements, providing LLM with a more dense and higher-quality context of vulnerability information. Fourth, a machine learning-based patch submission localization technology is developed, automatically identifying code changes associated with vulnerabilities through feature engineering, assisting in correcting version identification in affected libraries. Fifth, through cross-ecosystem scenario comparison experiments, ablation experiments, and user experiments, the comprehensive effectiveness of this method is systematically verified from dimensions such as accuracy and practicality. Attached Figure Description

[0038] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0039] Figure 1 This is a general flowchart of the present invention;

[0040] Figure 2 This is a detailed flowchart of the version correction in step S4 of the present invention;

[0041] Figure 3 This is a large model prompt template example from the embodiments;

[0042] Figure 4 This is a pseudocode description of the library name illusion elimination algorithm in step S4 of the present invention;

[0043] Figure 5 This is a pseudocode description of the version illusion elimination algorithm in step S4 of the present invention;

[0044] Figure 6 This is a pseudocode description of the code change diagram generation algorithm in step S4 of the present invention;

[0045] Figure 7 This is a pseudocode description of the path similarity calculation algorithm in step S4 of the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] The term "P@k" (Precision at k) used in this paper refers to the proportion of correctly predicted samples among the top k samples in the prediction results. It is calculated using the following formula: P@k = |Top-k Predicted Results| / min(k, |True Labels|). P@k measures the model's accuracy in the Top-k results. The denominator is the minimum of k and the number of true labels to avoid metric distortion caused by insufficient true labels (k).

[0048] The term "R@k" (Recall at k) used in this paper refers to the proportion of all true labels included in the Top-k predicted results. It is calculated using the following formula: R@k = |Top-k predicted results| / |true labels|. R@k measures the model's ability to cover the true labels in the Top-k results; a higher value indicates a lower false negative rate.

[0049] The term "F1@k" used in this paper is the harmonic mean of P@k and R@k, used to comprehensively evaluate the model's performance in Top-k results. It is calculated using the following formula: F1@k = 2 × P@k × R@k / (P@k + R@k). F1@k balances precision and recall, and is a core metric in ranking tasks, especially suitable for multi-label classification scenarios.

[0050] The term "Avg F1" used in this paper refers to the average F1@k value of the model across multiple k values ​​(e.g., k=1,2,3), used to evaluate overall performance. The calculation formula is: Avg F1=1 / m * k∈K∑F1@k(K={1,2,3},m=|K|). AvgF1 comprehensively reflects the model's stability at different retrieval depths, avoiding the random bias caused by a single k value.

[0051] A method for identifying the impact of version-level open vulnerabilities on libraries in cross-ecosystem scenarios based on multi-source data-enhanced LLM, including the following steps:

[0052] S1. Construct a cross-ecosystem multi-source vulnerability data knowledge base, including version-level vulnerability database annotation datasets, multi-source vulnerability information datasets, real database name and version datasets, patch submission annotation datasets, and vulnerability information summary annotation datasets;

[0053] S2. Input enhancement: An improved CodeBERT model is used to assess the quality of vulnerability data, and the BART model is combined to extract key information from long vulnerability texts, effectively improving the credibility and structure of multi-source vulnerability data.

[0054] S3. Vulnerability information identification: Construct Alpaca instruction templates, and use labeled datasets to fine-tune the large language model through multi-source data context fusion and low-rank adaptation (LoRA) parameter optimization strategies to enhance the model's ability to understand vulnerability features.

[0055] S4. Post-processing of results: Based on similarity matching and patch submission correlation analysis, the illusion bias of the output of large language models is significantly reduced.

[0056] The flowchart of S1 includes the following steps:

[0057] S1-1: Constructing a version-level vulnerability database annotation dataset. First, 2,217 vulnerability reports were collected from NVD. After filtering out invalid entries, three researchers independently annotated the ecosystem, database name, affected version range, and patched version of the affected databases. Finally, the dataset was stored in a structured JSON format (including fields such as ecosystem / package_name / vulnerable_version_range). The dataset was divided into a training set (1,551 entries) and a test set (666 entries) in a 7:3 ratio.

[0058] S1-2: Construct a multi-source vulnerability information knowledge base, integrating comprehensive vulnerability websites (GitHub Advisory / GitLabAdvisory / Snyk / Mend.io) and security announcements from 12 ecosystem vendors (such as Maven / NPM / PyPI), and acquiring data through differentiated collection strategies: direct RESTful collection is used for API open platforms, DOM parsing is used for structured pages, and regular expression + DFS parsing is used for unstructured data (such as email announcements), and noise is filtered based on keywords (fix / vulnerability / version, etc.), ultimately forming a cross-ecosystem vulnerability database containing 144,694 records;

[0059] S1-3: Construct a dataset of real library names and versions, collect full library information through a custom crawler for the ecosystem, and finally include 93,719 library records (including 857,442 version numbers) to eliminate the illusion of library names and versions in the output of the large model;

[0060] S1-4: Construct a patch submission dataset, which is divided into a general patch identification dataset (1,282 entries, including 624 positive samples) and a CVE-targeted patch dataset (3,718 entries, including 615 positive samples). The former randomly samples non-vulnerability submissions as negative samples, while the latter constructs negative samples through spatiotemporal proximity sampling (30 days before and after vulnerability patching). Both are divided into training and test sets in a 7:3 ratio.

[0061] S1-5: Construct a vulnerability information summary dataset. Extract 500 long text descriptions from multi-source vulnerability reports. The annotation team manually annotates semantic units such as vulnerability type, affected components, and remediation solutions to generate a summary covering the core elements. This summary is used to train the BART summary model. The dataset is divided in a 7:3 ratio to support model optimization.

[0062] The flowchart of S2 is as follows Figure 4 As shown, it includes the following steps:

[0063] S2-1: Train a vulnerability data quality assessment model to address the data quality issues existing in vulnerability information aggregation websites. Inputs include an NVD report, an affected database record from the vulnerability information aggregation website, and the vulnerability report on the website to which the data record belongs. The output is the confidence level of this affected database record.

[0064] S2-2: Train the BART model to perform summary processing on lengthy vulnerability descriptions, extract key information related to the vulnerability-affected database, affected version range, and patch version, in order to remove noisy data and prevent exceeding the maximum token length of the large language model.

[0065] The flowchart of S3 is as follows: Figure 5 As shown, it includes the following steps:

[0066] S3-1: Multi-source data context construction - integrating three types of heterogeneous data sources: (1) NVD vulnerability report metadata: extract CVE number, description summary (long text compressed to 256 tokens by BART model), CPE structured triple (⟨vendor,product,version>), and parse reference link paths through RFC 3986 standard (e.g., extract owner / project from GitHub submission links, filter common words tree / blob and retain key path words); (2) Trusted records of vulnerability information comprehensive websites: extract affected library records from GitHub / GitLab / Snyk / Mend.io, filter entries with confidence > 0.5 through quality assessment model (CodeBERT fine-tuning), and standardize them into a unified format; (3) Security announcements from ecosystem vendors: retrieve official announcements from 12 types of ecosystems (e.g. Maven / NPM), directly incorporate them into unstructured vulnerability descriptions (long text is also summarized); finally, construct a multi-source context according to priority (vendor announcements > comprehensive websites > NVD).

[0067] S3-2: Construction of Alpaca instruction template: (1) Instruction part: Define task objectives (extract affected libraries and versions), conflict resolution rules (arbitrate according to data source priority), output format constraints (JSON array [ecosystem, library name, version range]) and entry limit (Top-10 sorted by popularity); (2) Input part: Inject the multi-source context generated by S1-1; (3) Output part: Force the model to generate standardized responses.

[0068] S3-3: Fine-tuning of large language model instructions - LoRA parameter optimization strategy. (1) Initialize low-rank matrix: Adaptation matrix A (Gaussian distribution initialization) and B (zero matrix) are superimposed on LLM attention layer (original parameters are frozen); (2) Use mixed precision training: batch size 128, learning rate 3e-4, LoRA rank r=8 / scaling factor α=16; (3) Optimize generation strategy: beam search width 4, temperature coefficient 1.0, maximum number of newly generated tokens 512, and supervise output format compliance through cross-entropy loss.

[0069] S4 includes the following steps:

[0070] S4-1: Library Name Illusion Removal: Verify that the library names output by the large model exist in the real library name dataset. If missing, match the knowledge base using word segmentation (split by [-.: / @]), general word filtering (such as lib / core), and semantic similarity calculation (LCS sequence matching degree + tail word overlap degree), and replace it with the closest real library name with a similarity > 0.85. The specific library name illusion removal algorithm is as follows: Figure 4 As shown;

[0071] S4-2: Version Illusion Elimination: Verify the validity of the version number. If invalid, parse the semantic version (remove the v / ver prefix and extract the major, minor, and minor number segments), calculate the hierarchical similarity (70% weighted number segment matching + 30% LCS sequence matching), and replace it with the closest true version with a similarity > 0.8. The specific version illusion elimination algorithm is as follows: Figure 5 As shown;

[0072] S4-3: Patch submission for driver correction:

[0073] (1) Candidate submission filtering: Submissions are filtered based on the time window (2 years before and 100 days after the vulnerability disclosure) and file type (code extension such as .java / .py);

[0074] (2) Feature extraction and ranking: Extract six-dimensional features (including patch probability, semantic / path similarity, etc.), and generate a ranked list of patch submission probabilities through a logistic regression model. The six-dimensional feature extraction strategy is as follows:

[0075] Feature 1: Submitted information includes a CVE identifier (cve_in_message): The regular expression CVE-\d{4}-\d+ achieves precise matching across case and context.

[0076] Feature 2: The probability of patch submission (vfc_prob): The first feature is used to determine whether a submission fixes a vulnerability. This process is based on a fine-tuned CodeBERT model, which is adaptively trained to recognize patch submissions.

[0077] Feature 3: Semantic Similarity: This third feature is obtained by calculating the semantic similarity between the submitted information and the original NVD vulnerability description. This mechanism is based on the Sentence Transformers framework and uses a pre-trained all-mpnetbase-v2 model to generate text embedding vectors.

[0078] Feature 4: The probability of a patch submission for the corresponding vulnerability (vfc_prob_for_nvd): This second feature is used to determine whether the submission fixes a specific CVE vulnerability. This paper proposes a vulnerability remediation submission identification framework based on multimodal deep learning. By jointly analyzing the graph structure features of code changes and textual semantic features, it achieves the matching of CVE reports and patch submissions. For details on the specific logic of code change graph generation, please refer to... Figure 6 The pseudocode description.

[0079] Feature 5: Path Similarity: First, this paper extracts path information from the vulnerability description. When a path is found, it is separated from all modified file paths using dots and forward slashes as delimiters, and the path component list is reversed. Depending on whether the path extracted from the description contains a file extension, the extension portion is removed from these lists. For each modified file, the length of the longest similar sublist is calculated starting from the head of the list. For example, if the path "example / file.py" is extracted from the description, and a file path is "main / example / file.py", the length of the longest matching sublist is 3 (["py", "file", "example"]). It should be noted that if the extracted path contains an extension and the length of the longest matching sublist is only 1 (i.e., only the extension matches), this match is not included in the score. For details on the specific calculation logic of path similarity, please refer to [link to documentation]. Figure 7 The pseudocode description.

[0080] Feature 6: Is it a Top 10 vulnerability type (vfc_type_top_10): The vulnerability remediation submission type strictly follows the OWASP Top 10 official vulnerability classification standard. By referencing the "Top 10 Vulnerability Type and CWEID Mapping Table" published on the OWASP official website (e.g., A01 privilege vulnerability is mapped to CWE-284, CWE-863, etc.), each vulnerability is accurately associated with the corresponding general vulnerability enumeration (CWE ID), ensuring that the remediation measures are aligned with industry standards; at the same time, an independent 'Other' category is added to classify vulnerability types not covered by OWASP Top 10.

[0081] (3) Fix version binding: associate the Git tag of the top 1 commit and select the smallest valid version number to correct the original output of the model.

[0082] Example: This example uses the "Version-Level Vulnerability Database Annotation Dataset" built with S-1, covering 12 mainstream ecosystems including Maven, NPM, and PyPI, containing 2,217 vulnerability samples. The dataset is randomly divided into a training set (1,551 samples) and a test set (666 samples) in a 7:3 ratio to ensure unbiased data distribution. This implementation uses the Llama 3.1-8B large language model. The steps of this invention are then run using the dataset as input.

[0083] To investigate the effectiveness of this invention in identifying vulnerability impact libraries, the experiment compared the experimental metrics of existing benchmark methods for identifying vulnerability impact library names (including CHRONOS, VulLibMiner, VulLibGen, and Holmes) with the method proposed in this paper at the "ecosystem + library name" identification granularity (i.e., the task of identifying impacted libraries). The results are shown in Table 1. The F1@3 score of the method proposed in this invention reaches 95.87%, which is 32.8% higher than the best baseline, Holmes.

[0084] Table 1. Comparison of the effectiveness of the proposed method and the baseline method in cross-ecosystem scenarios (vulnerability impact library identification):

[0085]

[0086] Note: P = Precision (%), R = Recall (%), F1 = F1 score (%); Best results are marked in bold.

[0087] Next, to explore the effectiveness of the method in the affected version range inference task, the experiment first needs to evaluate the overall performance of the method in identifying "ecosystem + repository name + version range (including patched versions)" in a cross-ecosystem scenario. Secondly, the experiment needs to compare the proposed method with existing baseline methods (AFV and VISION) based on patch commit inference to evaluate the performance metrics at the granularity of "ecosystem + repository name + version range (including patched versions)" identification level (i.e., the affected version inference task). Since AFV is only applicable to the Packagist ecosystem, the experiment focuses on use cases related to the Packagist ecosystem in the test set to compare AFV and the proposed method; similarly, since VISION is only applicable to the Maven ecosystem, the experiment selects use cases related to the Maven ecosystem in the test set to compare VISION and the proposed method. The comparison results are shown in Tables 2, 3, and 4.

[0088] Table 1: Recognition performance of this method in cross-ecosystem scenarios (affected version range derivation task)

[0089]

[0090] Note: The values ​​represent precision (P) / recall (R) / F1 score (%)

[0091] Table 2: Comparison of the performance of this method and AFV on the Packagist ecosystem (affected version range derivation task)

[0092]

[0093] Note: Values ​​represent precision (P), recall (R), and F1 score (%). Bold text indicates results from the method described in this paper. * indicates that only test cases associated with patch submissions in the vulnerability report are considered.

[0094] Table 3: Comparison of the performance of this method and VISION in the Maven ecosystem (Affected version range derivation task)

[0095]

[0096] Note: Values ​​represent precision (P), recall (R), and F1 score (%). Bold text indicates results from the method described in this paper. * indicates that only test cases associated with patch submissions in the vulnerability report are considered.

[0097] Table 2 clearly shows that on a mixed test set including cross-ecosystems, the proposed method achieves a Top-1 identification F1 score of 69.39%, and this score further improves to 77.47% as the number of candidate results expands to Top-3, validating the adaptability of the version correction module to multi-version scenarios. As shown in Tables 3 and 4, the baseline methods AFV and VISION perform exceptionally well under ideal experimental conditions (i.e., test cases where vulnerability reports are associated with patch submissions) (F1@3 reaches 98.56% and 98.67%, respectively), but they heavily rely on the strong assumption of "vulnerability reports associated with patch submissions." Under non-ideal conditions, they still achieve F1 scores of 76.70% and 84.02% in the Packagist and Maven ecosystems, significantly outperforming the baseline methods (AFV improves by 34.11 percentage points, and VISION by 69.61 percentage points). This performance stems from a breakthrough in patch submission dependency—in the samples included in the test set, 70.1% of vulnerability reports were not associated with valid patch submissions. In this case, the baseline methods AFV and VISION failed to identify version changes completely because they could not locate patch submissions (NVD vulnerability reports did not contain associated patch submissions). However, this method utilizes multi-source data (such as version constraint expressions from PyPI announcements).

[0098] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0099] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0100] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention. Equivalent transformations or substitutions made based on the above technical solutions all fall within the scope of protection of the claims of the present invention.

Claims

1. A method for identifying version-level open vulnerability impact libraries based on multi-source data-enhanced LLM in cross-ecosystem scenarios, characterized by: By collaboratively optimizing input enhancement, vulnerability information identification, and result post-processing, a complete general technical system for identifying version-level vulnerability impact databases has been constructed. The identification method includes the following steps: S1. Construct a cross-ecosystem multi-source vulnerability data knowledge base, including version-level vulnerability database annotation datasets, multi-source vulnerability information datasets, real database name and version datasets, patch submission annotation datasets, and vulnerability information summary annotation datasets; S2. Input enhancement: An improved CodeBERT model is used to assess the quality of vulnerability data, and the BART model is combined to extract key information from long vulnerability texts, effectively improving the credibility and structure of multi-source vulnerability data. S3. Vulnerability information identification: Construct Alpaca instruction templates, and use labeled datasets to fine-tune the large language model through multi-source data context fusion and low-rank adaptation (LoRA) parameter optimization strategies to enhance the model's ability to understand vulnerability features. S4. Post-processing of results: Based on similarity matching and patch submission correlation analysis, the illusion bias of the output of large language models is significantly reduced.

2. The method for identifying version-level open vulnerability impact libraries based on multi-source data augmented LLM in cross-ecosystem scenarios according to claim 1, characterized in that: S1 includes the following steps: S1-1: Construct a version-level vulnerability database annotation dataset. First, 2,217 vulnerability reports were collected from NVD. After filtering out invalid entries, three researchers independently annotated the ecosystem, database name, affected version range, and patch version of the affected database. Finally, the dataset was stored in structured JSON format and divided into training and test sets in a 7:3 ratio. S1-2: Construct a multi-source vulnerability information knowledge base, integrate vulnerability comprehensive websites and security announcements from 12 ecosystem vendors, and acquire data through differentiated collection strategies: adopt RESTful direct collection for API open platforms, use DOM parsing for structured pages, use regular expression + DFS parsing for unstructured data, and filter noise based on keywords, ultimately forming a cross-ecosystem vulnerability database containing 144,694 records. S1-3: Construct a dataset of real library names and versions, collect full library information through a custom crawler of the ecosystem, and finally collect 93,719 library records to eliminate the illusion of library names and versions output by the large model; S1-4: Construct a patch submission dataset, which is divided into a general patch identification dataset and a CVE-targeted patch dataset. The former randomly samples non-vulnerability submissions as negative samples, while the latter constructs negative samples through spatiotemporal proximity sampling. Both are divided into training and test sets in a 7:3 ratio. S1-5: Construct a vulnerability information summary dataset. Extract 500 long text descriptions from multi-source vulnerability reports. The annotation team manually annotates semantic units such as vulnerability type, affected components, and remediation solutions to generate a summary covering the core elements. This summary is used to train the BART summary model. The dataset is divided in a 7:3 ratio to support model optimization.

3. The method for identifying version-level open vulnerability impact libraries based on multi-source data augmented LLM in cross-ecosystem scenarios according to claim 1, characterized in that, S2 includes the following steps: S2-1: Train a vulnerability data quality assessment model to address the data quality issues present in vulnerability information aggregation websites. Inputs include an NVD report, an affected database record from the vulnerability information aggregation website, and the vulnerability report from that website. Output is the confidence score of that affected database record. S2-2: Train the BART model to perform summary processing on lengthy vulnerability descriptions, extract key information related to the vulnerability-affected database, affected version range, and patch version, in order to remove noisy data and prevent exceeding the maximum token length of the large language model.

4. The method for identifying version-level open vulnerability impact libraries based on multi-source data augmented LLM in cross-ecosystem scenarios according to claim 1, characterized in that, S3 includes the following steps: S3-1: Multi-source data context construction - integrating three types of heterogeneous data sources: (1) NVD vulnerability report metadata: extracting CVE number, description summary, and CPE structured triples, and parsing the reference link path through RFC 3986 standard; (2) Trusted records of vulnerability information comprehensive websites: extracting affected library records from GitHub / GitLab / Snyk / Mend.io, filtering entries with confidence > 0.5 through a quality assessment model, and standardizing them into a unified format; (3) Security announcements from ecosystem vendors: retrieving official announcements from 12 types of ecosystems and directly incorporating them into unstructured vulnerability descriptions; finally, constructing a multi-source context according to priority: vendor announcements > comprehensive websites > NVD. S3-2: Alpaca instruction template construction: (1) Instruction part: Define task objectives, conflict resolution rules, output format constraints and item limit; (2) Input part: Inject the multi-source context generated in S1-1; (3) Output part: Force the model to generate a standardized response. S3-3: Fine-tuning of large language model instructions - LoRA parameter optimization strategy, (1) Initialize low-rank matrix: Gaussian distribution initialization of adaptation matrix A and zero matrix B, superimposed on LLM attention layer; (2) Use mixed precision training: batch size 128, learning rate 3e-4, LoRA rank r=8 / scaling factor α=16; (3) Optimize generation strategy: beam search width 4, temperature coefficient 1.0, maximum number of newly generated tokens 512, supervise output format compliance through cross-entropy loss.

5. The method for identifying version-level open vulnerability impact libraries based on multi-source data augmented LLM in cross-ecosystem scenarios according to claim 1, characterized in that, S4 includes the following steps S4-1: Library Name Illusion Elimination: Verify whether the library name output by the large model exists in the real library name dataset. If it is missing, match the knowledge base through word segmentation, general word filtering, and semantic similarity calculation (LCS sequence matching degree + tail word overlap degree), and replace it with the closest real library name with similarity > 0.

85. S4-2: Version illusion elimination: Verify the validity of the version number; if invalid, parse the semantic version, calculate the hierarchical similarity, and replace it with the closest real version with a similarity > 0.8; S4-3: Patch Submission Driven Correction: (1) Candidate Submission Screening: Filter submissions based on time window (2 years before and 100 days after vulnerability disclosure) and file type (code extension such as .java / .py); (2) Feature Extraction and Ranking: Extract six-dimensional features (including patch probability, semantic / path similarity, etc., detailed feature extraction strategy is described below), and generate a patch submission probability ranking list through logistic regression model; (3) Fix Version Binding: Associate the Git tag of the Top 1 submission and select the smallest valid version number to correct the original output of the model.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for identifying the impact library of version-level open vulnerabilities based on multi-source data augmented LLM in cross-ecosystem scenarios as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, the computer instruction implements the method for identifying the impact library of version-level open vulnerabilities based on multi-source data augmentation LLM in cross-ecosystem scenarios as described in any one of claims 1-4.

Images