Method and system for automatically positioning vulnerability-related file

Abstract

The invention discloses a method and a system for automatically positioning vulnerability-related files, and belongs to the technical field of software security. The method comprises the following steps: acquiring supplementary description information corresponding to a vulnerability entry; performing enhancement processing on the original description information in the vulnerability entry based on the supplementary description information; determining an open source storage library corresponding to the vulnerability entry; matching the vulnerability entries with all files in a target storage library, and determining the files matched with the vulnerability entries as candidate vulnerability files; semantic analysis is conducted on the content of all the candidate vulnerability files, and finally the vulnerability files related to the vulnerability items are determined. The problem that the vulnerability file cannot be accurately identified based on the low-quality vulnerability entry is solved, the quality of the vulnerability entry is improved, and the vulnerability file is accurately identified based on the effective vulnerability information in the vulnerability entry.

Description

Technical Field

[0001] The present invention relates to the field of software security technology, and in particular to an automated vulnerability file locating method based on a multi-agent architecture and CVE description enhancement, and specifically to a method and system for automatically locating vulnerability-related files. Background Art

[0002] Open Source Software (OSS) refers to software whose source code is publicly available, allowing users to freely use, modify, and distribute it. With the widespread adoption of OSS in modern software development, software vulnerabilities are increasing at an unprecedented rate and are widely disseminated to downstream projects through the software supply chain. The management of OSS vulnerabilities typically utilizes public consultations. For example, the National Vulnerability Database (NVD) records vulnerabilities through Common Vulnerabilities and Exposures (CVE) entries. However, many original vulnerability entries are of low quality, meaning that the entry descriptions lack valid vulnerability information. This makes it difficult to accurately identify vulnerable files based on low-quality vulnerability entries.

[0003] Currently, no effective solution has been proposed to the problem in related technologies that vulnerable files cannot be accurately identified based on low-quality vulnerability entries. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a method and system for automatically locating vulnerability-related files to solve the problem of being unable to accurately identify vulnerable files based on low-quality vulnerability entries.

[0005] The object of the present invention is achieved through the following technical solution: a method for automatically locating vulnerability-related files, the method comprising:

[0006] Obtaining supplementary description information corresponding to each vulnerability entry; the supplementary description information includes key elements for describing the vulnerability corresponding to the vulnerability entry, the key elements including the product and version where the vulnerability occurs, the vulnerability type, the vulnerability component, the attack method, the fatal cause, and the impact;

[0007] Based on the supplementary description information, the original description information in the vulnerability entry is enhanced to obtain a supplementary vulnerability entry;

[0008] Determine a target repository corresponding to the vulnerability supplement entry; match the vulnerability supplement entry with all files in the target repository, and use the file that matches the vulnerability supplement entry in the matching result as a target vulnerability file.

[0009] Furthermore, obtaining the supplementary description information corresponding to each vulnerability entry includes: obtaining all reference links appearing in each vulnerability entry; selecting multiple target reference links from each reference link based on preset screening rules; and collecting information on the websites pointed to by each target reference link according to each vulnerability entry to obtain the corresponding supplementary description information.

[0010] Furthermore, based on the supplementary description information, the original description information in the vulnerability entry is enhanced to obtain the supplementary vulnerability entry, including: extracting key elements of the supplementary description information through a large language model; based on a preset description template, the original description information in the vulnerability entry is enhanced according to the extraction result to obtain the supplementary vulnerability entry.

[0011] Furthermore, the enhancing the original description information in the vulnerability entry based on the supplementary description information to obtain the supplementary vulnerability entry further includes: when a target repository corresponding to the supplementary vulnerability entry is not obtained, inputting the supplementary vulnerability entry as context information into a large language model to obtain a target vulnerability file that matches the supplementary vulnerability entry.

[0012] Furthermore, determining the target repository corresponding to the vulnerability supplement entry includes: determining a repository set corresponding to the vulnerability supplement entry; the repository set includes the same open source repository under different modification versions; determining the difference between the submission time of each of the modification versions and the release time of the vulnerability supplement entry; using the modification version corresponding to the minimum difference as the target modification version; and determining the open source repository under the target modification version as the target repository corresponding to the vulnerability supplement entry.

[0013] Furthermore, the method of matching the supplementary vulnerability entry with all files in the target repository and taking the file that matches the supplementary vulnerability entry in the matching result as the target vulnerability file includes: extracting key components from the description information in the supplementary vulnerability entry to obtain a corresponding plurality of vulnerability components; matching each of the vulnerability components with all files in the target repository and taking the file that matches the vulnerability component in the matching result as a candidate vulnerability file; matching the description information in the supplementary vulnerability entry with relevant information of each of the vulnerability files and taking the candidate vulnerability file that matches the supplementary vulnerability entry in the matching result as the target vulnerability file; the relevant information includes the file content and file path of the candidate vulnerability file.

[0014] Furthermore, matching the description information in the supplementary vulnerability entry with the relevant information of each candidate vulnerability file, and selecting the candidate vulnerability file that matches the supplementary vulnerability entry in the matching result as the target vulnerability file, includes: inputting the description information in the supplementary vulnerability entry and the relevant information of each candidate vulnerability file into a large language model to obtain a confidence score corresponding to each candidate vulnerability file; the confidence score is used to indicate the degree of match between the candidate vulnerability file and the supplementary vulnerability entry; and selecting the candidate vulnerability file with the confidence score greater than a preset threshold as the target vulnerability file.

[0015] The present invention also provides a system for automatically locating vulnerability-related files, comprising:

[0016] An acquisition module, configured to acquire supplementary description information corresponding to each vulnerability entry; the supplementary description information includes key elements for describing the vulnerability corresponding to the vulnerability entry;

[0017] an enhancement module, configured to enhance the original description information in the vulnerability entry based on the supplementary description information to obtain a supplementary vulnerability entry;

[0018] A search module, configured to determine a target repository corresponding to the vulnerability supplement entry;

[0019] The matching module is used to match the supplementary vulnerability entry with each vulnerability file in the target repository, and use the vulnerability file that matches the supplementary vulnerability entry in the matching result as the target vulnerability file.

[0020] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the method for automatically locating vulnerability-related files.

[0021] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for automatically locating vulnerability-related files is implemented.

[0022] The beneficial effects of this invention are as follows: It utilizes a large language model to supplement vulnerability entry information, generating high-quality vulnerability entries for further vulnerability research. Vulnerable components are extracted from the supplemented vulnerability entries and matched against repository files by name and content to generate candidate vulnerable files, effectively reducing the cost of locating target vulnerable files. Leveraging the code understanding capabilities of the large oracle model, the final target vulnerable file is precisely located. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1It is a flow chart of the overall architecture of the method of the present invention. DETAILED DESCRIPTION

[0024] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, other implementation methods obtained by those skilled in the art without creative work also fall within the scope of protection of the present invention.

[0025] like Figure 1 As shown, an embodiment of the present invention provides a method and system for automatically locating vulnerability-related files to solve the problem in related technologies that vulnerability files cannot be accurately identified based on low-quality vulnerability entries. The method includes the following steps:

[0026] Step 1: CVE description enhancement

[0027] 1.1. Information Collection: Based on the acquisition module, supplementary information is collected from NVD reference links and the Internet to fill in the gaps and incomplete details in the original CVE description.

[0028] Due to the heterogeneity of websites, it is unrealistic to write crawlers for all websites. As a solution, this paper manually constructs a set of high-quality domain names based on the frequency and importance of domain names in vulnerability reference links. The sources of these domain names include security advisories, vendor advisories, and vulnerability databases (such as the CERT vulnerability database).

[0029] The crawl_web_pages function is called to crawl the reference URLs provided in the NVD CVE entries, focusing on frequently occurring and reliable domains. However, while NVD reference links can provide additional vulnerability information, they can be limited or even missing. Furthermore, the content crawled from reference links often contains other links that are also valuable. To obtain more of this dynamic information, a function is used to call external tools to search for more relevant information.

[0030] Specifically, google_search is called to perform an online search using the original CVE description as the query, and the top five URLs are returned for expansion. Next, crawl_web_page is called to crawl the content of the URLs. Furthermore, URLs found in the reference links are accessed in the same manner to gather more information.

[0031] 1.2. Description expansion: Based on the collected information, the enhancement module generates a more detailed and structured CVE description, including the vulnerability component, vulnerability type, attack vector, root cause, and impact. The CVE description helps localize the vulnerability-related files and provides accurate guidance for subsequent steps. The CVE analyst then automatically integrates the extracted CVE description into the original CVE description to generate an enhanced CVE description, providing comprehensive context for repository retrieval and file location tasks.

[0032] Step 2: Filter candidate files

[0033] 2.1. Repository Retrieval: Based on the search module, the repository where the vulnerability is located is identified. If the CVE entry contains a patch URL in its reference link, the repository name is extracted from the reference link. For CVEs that lack a clear repository reference, the official product name obtained from the Common Platform Enumeration (CPE) is queried by a large language model to retrieve the open source repository.

[0034] Furthermore, to ensure the accuracy of repository identification and prevent hallucinations caused by large models, a check_repository_existence function is used, which queries the official GitHub API to verify that the suggested repository exists and is accessible. If no valid repository is found, the search strategy will be iteratively retried, optimizing keywords and search parameters until a preset threshold is reached.

[0035] 2.2 Vulnerability Keyword Extraction: After identifying vulnerability-related open source repositories, get_vulnerability_keywords is called to extract vulnerability keywords from the enhanced CVE descriptions. Based on the CVE descriptions, keywords that help identify vulnerability-related files are independently selected. Extracted keywords typically include vulnerable components, including module names, file names, and function names, as well as other contextual clues that are semantically closer to the target name or vulnerability context. These keywords are then used to filter candidate files in the repository.

[0036] 2.3,Candidate file filtering: After obtaining the repository and vulnerability keywords, two filtering strategies are used to filter out irrelevant files from the repository, thereby narrowing the scope of vulnerability-related file location.

[0037] Specifically, filter_files_by_name is called, which iterates over all files in the repository and compares each file name with the extracted keyword list. If the keyword is found to be a suffix or substring of the file name, the file is retained, effectively identifying files explicitly mentioned in the CVE description or external reference.

[0038] Next, using filter_files_by_content, the contents of all files are scanned for occurrences of the keyword by calling view_file_content. To improve precision and reduce noise from accidental matches, this function only retains files where at least two different keywords were found. This conservative threshold helps ensure that the retained files are closely related to the vulnerability context. If neither filtering strategy identifies any relevant files, the entire repository is retained to avoid missing potentially vulnerable files. The identified files include a smaller set of candidate files.

[0039] Step 3: Locate vulnerability-related files

[0040] Based on the matching module, the context clues in the enhanced CVE description are used to finally locate the files related to the vulnerability from the vulnerability candidate files.

[0041] Specifically, the prioritize_suspicious_files function is first called to analyze the repository's file hierarchy and identify files whose names or paths contain keywords related to the vulnerability. This step generates a prioritized list of suspicious files that may be related to the vulnerability. The view_file_content function is then called to retrieve the contents of each file in the list and perform semantic analysis using analyze_file_relevance to detect the vulnerability. This function uses the enhanced CVE description to guide a large language model-based assessment to determine whether the file is related to the described vulnerability. If the file is confirmed to be relevant, it is included in the final output. Otherwise, analysis continues with the next most likely candidate file. When reviewing file content, the main challenge at this stage is managing large files that exceed the input limits of the large language model. To address this, the simply_large_file function is called. This function compresses the contents of files larger than 10,000 tokens using LLM-based summaries, reducing them to a manageable size while retaining key information related to the vulnerability. This ensures that the contents of multiple large files can be reviewed simultaneously without exceeding the model's limits.

[0042] An embodiment of the present invention also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the method for automatically locating vulnerability-related files.

[0043] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for automatically locating vulnerability-related files is implemented.

[0044] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0045] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only.

[0046] It will be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A method for automatically locating vulnerability-related files, characterized in that: The steps include: Obtaining supplementary description information corresponding to each vulnerability entry; the supplementary description information includes key elements for describing the vulnerability corresponding to the vulnerability entry, the key elements including the product and version where the vulnerability occurs, the vulnerability type, the vulnerability component, the attack method, the fatal cause, and the impact; Based on the supplementary description information, the original description information in the vulnerability entry is enhanced to obtain a supplementary vulnerability entry; Determine a target repository corresponding to the vulnerability supplement entry; match the vulnerability supplement entry with all files in the target repository, and use the file that matches the vulnerability supplement entry in the matching result as a target vulnerability file.

2. The method according to claim 1, characterized in that The method of obtaining the supplementary description information corresponding to each vulnerability entry includes: obtaining all reference links appearing in each vulnerability entry; selecting multiple target reference links from each reference link based on preset screening rules; and collecting information on the websites pointed to by each target reference link according to each vulnerability entry to obtain the corresponding supplementary description information.

3. The method according to claim 1, characterized in that The method of enhancing the original description information in the vulnerability entry based on the supplementary description information to obtain the supplementary vulnerability entry includes: extracting key elements from the supplementary description information through a large language model; and enhancing the original description information in the vulnerability entry based on the extraction result based on a preset description template to obtain the supplementary vulnerability entry.

4. The method according to claim 1, wherein The enhancing the original description information in the vulnerability entry based on the supplementary description information to obtain the supplementary vulnerability entry further includes: when a target repository corresponding to the supplementary vulnerability entry is not obtained, inputting the supplementary vulnerability entry as context information into a large language model to obtain a target vulnerability file that matches the supplementary vulnerability entry.

5. The method according to claim 1, wherein Determining the target repository corresponding to the vulnerability supplementation entry includes: determining a repository set corresponding to the vulnerability supplementation entry; the repository set includes the same open source repository under different modification versions; determining the difference between the submission time of each of the modification versions and the release time of the vulnerability supplementation entry; using the modification version corresponding to the minimum difference as the target modification version; and determining the open source repository under the target modification version as the target repository corresponding to the vulnerability supplementation entry.

6. The method according to claim 1, characterized in that The method of matching the supplementary vulnerability entry with all files in the target repository and using the file matching the supplementary vulnerability entry in the matching result as the target vulnerability file includes: extracting key components from the description information in the supplementary vulnerability entry to obtain a plurality of corresponding vulnerability components; matching each vulnerability component with all files in the target repository and using the file matching the vulnerability component in the matching result as a candidate vulnerability file; matching the description information in the supplementary vulnerability entry with relevant information of each candidate vulnerability file and using the candidate vulnerability file matching the supplementary vulnerability entry in the matching result as the target vulnerability file; the relevant information includes the file content and file path of the candidate vulnerability file.

7. The method according to claim 6, characterized in that Matching the description information in the supplementary vulnerability entry with the relevant information of each candidate vulnerability file, and selecting the candidate vulnerability file that matches the supplementary vulnerability entry in the matching result as the target vulnerability file, includes: inputting the description information in the supplementary vulnerability entry and the relevant information of each vulnerability file into a large language model to obtain a confidence score corresponding to each vulnerability file; the confidence score is used to indicate the degree of match between the vulnerability file and the supplementary vulnerability entry; and selecting the vulnerability file with the confidence score greater than a preset threshold as the target vulnerability file.

8. A system for automatically locating vulnerability-related files, characterized in that: include: An acquisition module, configured to acquire supplementary description information corresponding to each vulnerability entry; the supplementary description information includes key elements for describing the vulnerability corresponding to the vulnerability entry; an enhancement module, configured to enhance the original description information in the vulnerability entry based on the supplementary description information to obtain a supplementary vulnerability entry; A search module, configured to determine a target repository corresponding to the vulnerability supplement entry; The matching module is used to match the supplementary vulnerability entry with each vulnerability file in the target repository, and use the vulnerability file that matches the supplementary vulnerability entry in the matching result as the target vulnerability file.

9. An electronic device comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement a method for automatically locating vulnerability-related files as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for automatically locating vulnerability-related files according to any one of claims 1 to 7 is implemented.

Images