An automated cross-project vulnerability verification method and apparatus based on a large language model

Abstract

This invention provides an automated cross-project vulnerability verification method and apparatus based on a large language model, belonging to the field of vulnerability verification technology. It aims to solve the problem that existing technologies struggle to efficiently confirm whether upstream vulnerabilities can still be triggered in scenarios where there are significant differences in parameter semantics and input formats between the source and target projects. The method constructs a multi-agent parallel target-aware parameter transfer framework, combining RAG retrieval enhancement technology to automatically generate legal command-line parameter combinations related to the vulnerability function from the target software manual. Furthermore, it introduces the ReAct agent to embed the source PoC file into an input shell acceptable to the target software across formats without manual intervention, forming an initial cross-format seed. Subsequently, using function-level trajectory similarity as a feedback indicator, the parameter-seed pair is iteratively optimized, driving parameter-sensitive gray-box fuzzing to quickly converge to the precise input that can trigger the vulnerability.

Description

Technical Field

[0001] This invention relates to the field of vulnerability verification technology, specifically to an automated cross-project vulnerability verification method and apparatus based on a large language model. Background Technology

[0002] In recent years, with the explosive growth of the open-source ecosystem, code reuse has become commonplace in software development. To improve development efficiency, software developers tend to reuse proven open-source project components. While this practice accelerates the software development process, it also introduces potential security vulnerabilities. Because specific code snippets may be widely reused across multiple projects, the spread of a single vulnerability can expand exponentially, posing a serious challenge to the security of the entire open-source project ecosystem.

[0003] To address this issue, existing technologies have proposed vulnerability propagation analysis techniques based on code clone detection. This technology, by constructing a code feature database and combining it with similarity comparison algorithms, can effectively identify potential vulnerability clone instances in software projects. Its application allows developers to quickly locate known vulnerabilities that have propagated into projects and formulate corresponding remediation strategies accordingly. However, existing detection technologies still have significant limitations in verifying triggerable paths. Specifically, while current mainstream methods can accurately identify code cloning phenomena, they cannot effectively determine whether the detected vulnerable code possesses actual triggerability. This problem mainly stems from three influencing factors: 1. Code regions identified as vulnerability clones may not actually be executed by the program; 2. The compilation and optimization process may exclude some vulnerable code segments, preventing them from being executed during program execution; 3. Developers may insert patches to partially fix the vulnerability, which static detection tools often fail to detect.

[0004] Third-party libraries, frameworks, and even complete modules are indiscriminately introduced into various projects, significantly shortening delivery cycles. However, this also leads to a rapid amplification of systemic risks, such as "a single defect can cause a global contagion." A vulnerability disclosed in an upstream library can often silently spread to hundreds or thousands of downstream products through copy-paste, dependency references, or binary packaging. While traditional static clone detection can quickly locate similar code snippets, it can only provide a vague conclusion that "there may be hidden dangers." It cannot answer whether the snippet will actually be executed in the target program, nor can it determine whether the developer has implanted targeted patches during the porting process. As a result, security teams have to invest heavily in verifying each false positive, while development teams are forced to accept deployments with vulnerabilities due to the overwhelming number of fixes, ultimately creating a stalemate where "the vulnerability database is expanding daily, but the fix rate is stagnating."

[0005] While comprehensively patching all identified vulnerabilities is theoretically feasible, significant resource constraints exist in practice. Over-patching not only consumes substantial development resources but can also lead to code compatibility issues and even functional abnormalities. To address these challenges, establishing a triggerable verification mechanism is crucial for optimizing vulnerability patching priorities. Existing solutions primarily follow two technical routes: one utilizes historical knowledge from third-party library Proof of Concept (PoC) libraries to guide dynamic testing engines in migrating existing PoCs to vulnerability PoCs suitable for the target software. However, this method can only migrate file portions of the PoC, not the entire PoC. Furthermore, it struggles to handle situations where PoC files have undergone significant changes. The second approach uses parameter-sensitive fuzzing to automatically infer parameters for PoC generation; however, existing methods primarily generate parameters to increase code coverage, performing poorly in verifying vulnerability propagation.

[0006] In short, to overcome the bottleneck of "easy detection, difficult confirmation," the academic community has proposed two technical approaches. Please refer to [link / reference]. Figure 1 One approach is Proof-of-Concept (PoC) migration: This involves modifying a Proof-of-Concept (PoC) sample that has already triggered a vulnerability in the source project into an input suitable for the target program through gray-box fuzzing using symbolic execution, path alignment, or taint-guided methods, thus directly observing whether the vulnerability can still be triggered. The other approach is parameter-sensitive fuzzing: This involves jointly mutating the program's command-line options, configuration files, environment variables, and other parameter spaces with file input to increase code coverage and improve the probability of crashes. Both approaches have achieved some success in their respective scenarios, but both expose shortcomings in real-world "cross-project" verification tasks. PoC migration schemes generally assume that the source and target programs are "from the same source." Once their input formats differ (e.g., the upstream is the original image encoding / decoding library, and the downstream is a PDF reader integrating that library), the original PoC is directly rejected due to format verification failure. Existing symbolic execution engines lack scalability for large-scale binary programs, and gray-box mutation lacks precise location of "keyword sections," leading to a sharp drop in migration success rates. While parameter-sensitive fuzz testing can automatically generate massive parameter combinations, it only aims to improve overall coverage and lacks the awareness to converge "directly" to upstream vulnerability code. The computational overhead caused by blind searching creates a sharp conflict with the business delivery cycle. More seriously, modern large-scale software often has hundreds of parameters, and the combination space explodes exponentially. It is impossible for humans to write sufficient and legitimate configurations, and existing algorithms also lack prior knowledge of whether "the parameter is associated with the vulnerability under test," ultimately falling into the predicament of "running a lot, triggering few."

[0007] In view of the above, this application is hereby submitted. Summary of the Invention

[0008] This invention provides an automated cross-project vulnerability verification method and apparatus based on a large language model, which can at least partially improve the above-mentioned problems.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] An automated cross-project vulnerability verification method based on a large language model includes:

[0011] S1: Obtain multiple working nodes to be processed, call multiple agents to process them in parallel, construct prompt words, and query the target software manual based on RAG technology to generate candidate parameter combinations;

[0012] S2, construct cross-format seed prompts based on candidate parameter combinations, call the ReAct agent to process the cross-format seed prompts with code, and generate cross-format seeds;

[0013] S3 performs corpus optimization on candidate parameter combinations and cross-format seeds to obtain corpus information of downstream software with functions most likely to trigger upstream vulnerabilities, and generates PoC for the target vulnerability by performing parameter-sensitive fuzz testing based on the corpus information.

[0014] This invention also provides an automated cross-project vulnerability verification device based on a large language model, comprising:

[0015] The command-line parameter combination generation unit is used to obtain multiple working nodes to be processed, call multiple intelligent agents to process them in parallel, construct prompt words, and query the target software manual based on RAG technology to generate candidate parameter combinations.

[0016] The cross-format seed generation unit is used to construct cross-format seed prompts based on candidate parameter combinations, and call the ReAct agent to process the cross-format seed prompts with code to generate cross-format seeds.

[0017] The corpus optimization unit is used to optimize the candidate parameter combinations and cross-format seeds to obtain the corpus information of the downstream software with the relevant functions most likely to trigger the upstream vulnerability, and to generate the PoC of the target vulnerability by performing parameter-sensitive fuzz testing based on the corpus information.

[0018] In summary, the automated cross-project vulnerability verification method based on a large language model addresses the governance pain point of "one vulnerability, multiple projects spreading" in the open-source era. It proposes an automated cross-project vulnerability verification scheme driven by a large language model. Through a three-stage architecture of "automatic parameter generation—cross-format seed generation—runtime feedback iteration," the system can autonomously complete the discovery of legitimate parameter combinations, PoC format migration, and vulnerability triggerability confirmation even when the source and target programs have completely different parameter semantics and input formats. This reduces the traditional manual investigation process, which requires weeks, to hours or even minutes. The method's PoC migration success rate in real-world open-source software scenarios is significantly better than existing symbolic execution and gray-box fuzzing testing schemes. Furthermore, it requires no manual debugging or domain expert experience and can stably output vulnerability trigger inputs that can be directly used for remediation decisions. Based on these technical effects, this invention provides a scalable, low-cost, and high-precision vulnerability propagation verification method for large open-source ecosystems. It can be directly embedded into continuous integration / continuous delivery pipelines to achieve the proactive defense goal of "once an upstream vulnerability is disclosed, the downstream risk is immediately eliminated."

[0019] Compared with existing technologies, the advantages of this invention are as follows: 1. Strong cross-project adaptability: It can automatically migrate vulnerability verification conditions even when there are differences in parameter semantics and input format between the source program and the target program. 2. High degree of automation: Relying on intelligent agents and dynamic feedback mechanisms, it eliminates the need for manual parameter debugging or seed file writing, significantly reducing verification costs. 3. Improved vulnerability verification efficiency: Through corpus optimization driven by function trajectory similarity and parameter-sensitive fuzzing, it can quickly generate PoCs that can trigger vulnerabilities in the target software. 4. Strong versatility: It is applicable to various types of software projects, especially showing significant advantages in cross-library and cross-framework vulnerability propagation verification scenarios. It effectively solves the problems of difficult parameter migration and format adaptation in cross-project vulnerability verification, providing a feasible technical solution for large-scale automated vulnerability propagation verification. Attached Figure Description

[0020] Figure 1 This is an example diagram of cross-project vulnerability verification in the prior art provided by the present invention.

[0021] Figure 2 This is a flowchart illustrating the automated cross-project vulnerability verification method based on a large language model provided in the first embodiment of the present invention.

[0022] Figure 3 This is a summary diagram of the automated cross-project vulnerability verification method based on a large language model provided in this embodiment of the invention.

[0023] Figure 4 This is a flowchart of the candidate parameter combination generation process provided in the embodiments of the present invention.

[0024] Figure 5 This is a schematic diagram of the initial parameter generation prompt word provided in the embodiment of the present invention.

[0025] Figure 6 This is a diagram of the domain knowledge query architecture based on RAG provided in an embodiment of the present invention.

[0026] Figure 7 This is a diagram of an event-driven parallel agent collaboration model provided in an embodiment of the present invention.

[0027] Figure 8 This is a schematic diagram of cross-format seed generation provided in an embodiment of the present invention.

[0028] Figure 9 This is a prompt word for seed generation provided in the embodiments of the present invention.

[0029] Figure 10 This is a schematic diagram of a module of an automated cross-project vulnerability verification device based on a large language model provided in the second embodiment of the present invention. Detailed Implementation

[0030] The core challenge in verifying whether a disclosed vulnerability exists in other projects lies in how to locate suspected vulnerabilities from massive amounts of code and prove their triggerability. The technical process can be divided into two stages: (1) candidate vulnerability location based on vulnerability code cloning detection; (2) triggerability confirmation combined with dynamic verification methods. First, let's introduce the relevant technologies for cross-project verification, including PoC migration technology and parameter-sensitive fuzzing technology.

[0031] The first type, PoC migration technology aims to achieve the adaptive adjustment of existing PoC to the target software system. At present, this technology is mainly applied to two scenarios: cross-version migration and cross-project migration, and their technical implementation paths are significantly different. In the cross-version vulnerability verification scenario, when a vulnerability is known to exist in a specific version of the software, it is necessary to verify the existence of the same source vulnerability in its historical version or the updated version. Empirical studies show that although the original PoC can directly trigger the vulnerability in most cases, 21.17% of the cases still require byte-level adjustment to adapt to the target version. In response to this requirement, VulScope[3] proposed a path alignment algorithm, which establishes a mapping relationship between the cross-version vulnerability execution trajectory and combines it with the targeted gray box fuzzing test technology to achieve the automatic migration of PoC. However, this method has significant limitations in the cross-project migration scenario. The root cause of its failure is that when the version difference between programs is too large, the basic assumption of the path alignment algorithm fails, and the target system in the cross-project scenario and the program to which the source PoC belongs often belong to completely different code systems. In order to break through the technical bottleneck of cross-project migration, Kwon et al. proposed the OCTOPOCS framework, which for the first time realized cross-project PoC migration. The innovation of this technology is reflected in: 1. Crash primitive extraction: extracting key constraints from the source PoC (such as 0xffff1111==AABBCCDD); 2. Constraint-guided execution: realizing path exploration of the target system through symbolic execution engine; 3. Dynamic constraint solving: generating adapted PoC by performing instant constraint solving at the target execution point.

[0032] like Figure 1The typical case shown illustrates a vulnerability originating from the `opj_dump` component (CVE-2020-27823) of the OpenJPEG library, whose Proof-of-Concept (PoC) is a specially crafted j2k file. While the target project `mutool` reuses OpenJPEG library code, the original PoC fails to trigger the vulnerability due to differences in file format verification mechanisms (only accepting PDF input). OCTOPOCS achieves migration through the following technical path: parsing crash primitive constraints in the j2k file; constructing a PDF shell to wrap the core constraints; and triggering a null pointer exception at the decoding function. This case validates the framework's effectiveness in cross-project scenarios, successfully generating a vulnerability-triggered PoC conforming to the `mutool` input specification. However, the OCTOPOCS framework faces significant constraints in engineering practice: its symbolic execution engine is limited by computational complexity, posing scalability challenges in large binary program applications. To address this bottleneck, TransferFuzz proposes a method based on directed gray-box fuzzing. Its core idea is consistent with OCTOPoCs, namely, fully utilizing the information from the source PoC to guide the directed testing engine to efficiently verify the triggerability of the source project vulnerability in the target software. However, unlike OCTOPoCs, TransferFuzz extracts key bytes of the PoC through taint analysis and uses them as a mutation dictionary for fuzzing to guide the dynamic testing engine to verify the target vulnerability.

[0033] The second type involves software parameters defining the program's behavior and functionality, providing users with the ability to customize settings. This configurable design enhances the software's adaptability to different environments, with configurations covering network parameter settings, user interface customization, and more. While flexible configuration mechanisms improve software usability and user experience, they also increase system complexity and the difficulty of software testing. Since each configuration parameter can potentially change the program's path, multiple parameter combinations may cause unexpected behavior. Therefore, in-depth analysis of the relationships and combined effects between options can effectively ensure software quality and security. To address this need, parameter-sensitive fuzz testing has emerged. Its core idea is to dynamically adjust parameter combinations and explore code coverage paths under different parameters in real time, providing support for system security verification.

[0034] There are two main types of work in this field: mutation-based testing and filtering-based testing. The core idea of ​​the former is to generate option combinations through mutation strategies and discover unexpected vulnerabilities by constructing invalid options. The earliest work, AFL-argv, directly used parameters as test case bytes for random mutation, but this could introduce a large number of invalid parameters, making in-depth testing of the program impossible. To address this, ToFo adopted a structured mutation strategy to try to generate more valid option combinations. ConfigFuzz achieved collaborative testing of configuration and input by encoding program options into input files and reusing the mutation strategies of existing fuzzers. Power significantly improved crash detection capabilities in 30 real programs by actively selecting the option configurations with the greatest "difference" and designing different mutation strategies for the option domain and file domain. Its core idea is to expand the scope of code execution through configuration diversity. CarpetFuzz first proposed automatically extracting option constraint relationships from program documents using natural language processing to filter invalid combinations. Its core contribution was using NLP technology to reduce invalid option combinations by 67.91%, improving AFL's path coverage by 45.97%, and discovering 57 vulnerabilities in 20 open-source projects. This work provides a basic framework for automated constraint extraction in subsequent research, but its reliance on document quality has spurred the development of a more intelligent prediction method, ProphetFuzz. ProphetFuzz is the first to introduce a large language model, automatically predicting high-risk option combinations through prompt engineering and performing targeted fuzz testing. In experiments with 52 programs, 12.30% of the predicted high-risk combinations actually contained vulnerabilities, achieving a 32.85% improvement in detection efficiency compared to traditional methods, with an average cost of only $8.69 per program. ZigZagFuzz further proposes an alternating mutation strategy, decoupling the mutation process of command-line options from file input and dynamically reducing the test set based on function-level coverage information, thereby improving testing efficiency and effectiveness. Experimental results show that ZigZagFuzz significantly outperforms existing tools such as AFL++, CarpetFuzz, and POWER in vulnerability detection capabilities, detecting 1.9 to 10.6 times more vulnerabilities. The core idea of ​​the latter is to filter invalid combinations through constraint conditions. For example, CrFuzz addresses the input validation problem in multi-purpose programs (such as FFmpeg) by predicting input validity through cluster analysis, thereby enhancing the path coverage of fuzzers such as AFL and QSYM (by 19.3%).

[0035] In summary, current technologies face two main challenges that hinder their application in large-scale programs and real-world cross-project vulnerability verification tasks: 1. Extremely large parameter combination search space: Given a vulnerability in source software, verifying whether it can be triggered in target software requires finding command-line parameters related to the vulnerability's functionality within the target software. However, current PoC migration techniques rely on manual configuration of these parameters. Modern software often has complex parameters, requiring multiple parameter combinations to trigger a program's function. This task is difficult to perform manually. While parameter-sensitive fuzzing can automate parameter generation for testing, it currently lacks target-oriented capabilities, only improving software test coverage. It cannot efficiently and quickly verify whether a source vulnerability is triggerable in the target software, resulting in poor performance in cross-project vulnerability verification tasks. 2. Cross-format file migration: Given a vulnerability in source software, verifying whether it can be triggered in target software faces situations where the source and target software accept different input formats. For example, the source software might be an image library, while the target software might be a PDF reader that uses the source software as a third-party library. Existing work has not considered this situation, and it is difficult to mutate the PoC of the source vulnerability into the PoC of the target software vulnerability in another format using only the mutation function of fuzzing.

[0036] Based on this, the method mentioned in this invention was designed. To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0037] refer to Figure 2 , Figure 3 As shown, the first embodiment of the present invention discloses an automated cross-project vulnerability verification method based on a large language model, which can be executed by an automated cross-project vulnerability verification device based on a large language model (hereinafter referred to as the verification device), specifically, by one or more processors within the verification device, to implement the following method:

[0038] S1: Obtain multiple working nodes to be processed, call multiple agents to process them in parallel, construct prompt words, and query the target software manual based on RAG technology to generate candidate parameter combinations;

[0039] Specifically, step S1 further includes: acquiring multiple worker nodes to be processed, initializing them, creating a corresponding ProcessEvent event for each node, and setting the corresponding maximum number of retries for it;

[0040] Based on the event loop mechanism, multiple agents are used for parallel processing. Different event types will trigger different processing functions. When processing the ProcessEvent, prompt words are constructed and the large language model is called to generate parameter combinations. When the GenerationDone event is received, format verification is performed. If it fails, the error message is passed back to the large language model for regeneration. Successful results are collected in a distributed manner through ResultEvent.

[0041] Successful results from multiple events are collected and integrated to generate a final output of candidate parameter combinations.

[0042] Preferably, when processing the ProcessEvent event, a prompt word is constructed, and a large language model is called to generate parameter combinations. Specifically, the large model is driven to query the description in the target software manual to understand the core functions of the target program, and the large model is driven to search for the options of possible source projects in the target software manual.

[0043] Ensure that conflicting options are not used when building commands, use preset placeholders for input files, use preset requirements when the target software manual does not specify output file requirements, and follow the requirements of the target software manual when the target software manual specifies a file extension.

[0044] The fmt_convert attribute is determined based on the compatibility of the PoC format. The fmt_convert attribute is used to confirm whether the command directly supports the PoC format. If it does, the PoC format is used; if it does not, the format to be converted is specified to confirm the format of the generated file and construct the prompt word.

[0045] The prompt words are used as input to the big model, which drives the agent to query the target software manual through RAG technology. The big model infers the core query content to be retrieved through semantic understanding. Based on this semantic query, the query engine uses the vector distance index in a pre-generated vector database to select the text fragments that are closest to the query semantics as enhanced context information returned to the big model.

[0046] The vector distance metrics include Euclidean distance and cosine similarity. The database is constructed from the target software manual using vector embedding technology (that is, converting the text in the original document library into a numerical semantic representation in a high-dimensional space).

[0047] The large model outputs a text response based on enhanced contextual information and given prompt words, thus obtaining the parameter combination.

[0048] Please see Figure 4In this embodiment, the command-line parameter combination generation aims to generate legitimate parameters for the program to be verified, which are associated with the target (source vulnerability). This step is based on the user manual of the target software under test, and uses RAG (Retrieval-Augmented Generation) technology combined with a large model to generate legitimate command-line parameter combinations that may trigger upstream vulnerability functions.

[0049] Specifically, given upstream vulnerability information, it is structured into prompts generated by parameter combinations. Multiple agents capable of invoking tools are constructed to generate candidate parameter combinations in parallel. The target software manual is vectorized into a knowledge base, and the agents ensure the validity of the generated parameter combinations by invoking this tool (querying the document). In other words, the parameter combinations are generated through a large model's understanding of the target software manual.

[0050] First, structured prompts are designed, serving as input to the parameter-generating agent. Parameter combinations need to satisfy constraints; therefore, the prompts are designed to improve the quality of the generated parameter combinations. The agent obtains these parameter combinations by querying the target software's user manual using RAG technology. Since generating initial parameter combinations requires satisfying dual constraints—ensuring the combinations pass the target project's validity check and trigger relevant source project functions—this method designs the following... Figure 4 The parameter combination shown generates a prompt word framework. The prompt word first sets two constraints to guide the agent in generating appropriate parameter combinations, emphasizing the use of a RAG-based domain knowledge query tool to achieve this. Secondly, for ease of subsequent processing, a strict output format is defined. Finally, the prompt word suggests that the large model thinks step-by-step, constructing a thought process chain to answer the question.

[0051] This is because constructing a parameter combination that satisfies the constraints is a relatively complex problem. Using a chain-of-thought (CoT) approach is a feasible method. A chain of thought involves a series of sequential reasoning processes, and research has shown that it can improve the ability of large models to handle complex problems. As... Figure 5 The presented prompts are broken down into five steps using a thought chain to guide the large language model through the task. Initially, this method allows the large model to understand the core functionality of the target program by consulting the manual's description. Figure 5 Step 1 in the process. The next step is to have the large model look for options in the man page that might call the source project ( Figure 5Step 2 in the previous section. Subsequent steps 3 and 4 are to remind the large model of the format of the generated parameter combinations, facilitating subsequent dynamic execution verification. Finally, the generated "fmt_convert" attribute is designed for subsequent file generation. When generating files for a given parameter combination, this method needs to refer to this attribute to confirm the format of the generated file. Simultaneously, if the large model determines that the generated parameter combination directly supports the PoC format, this study will directly execute that parameter combination and the source PoC together on the target program in the subsequent verification stage. Otherwise, the dynamic execution verification stage is skipped. After these five steps, this study uses the instruction "Let's take a deep breath and think step by step. Please demonstrate your thoughts in each step" to encourage the large model to reason carefully and meticulously.

[0052] Secondly, a domain knowledge query tool based on RAG technology was designed. RAG is a callable tool for the agent. Prompt words serve as input to the agent, which uses RAG to query the target software's manual, obtaining textual descriptions of parameters related to the vulnerability's functionality. Based on this textual response, the agent generates parameter combinations and outputs them in a structured format (such as JSON). Specifically, RAG is a lightweight method that allows adding one's own data to a large language model without using time-consuming and hardware-intensive techniques like large model fine-tuning to make the large model understand the target software's parameter information. Furthermore, another reason for using RAG instead of directly feeding the target software's manual to the large model is the limited context window length of the large model; some software manuals are too long, exceeding the current large model's context window limit. For example, the GraphicsMagick manual contains over 400K characters.

[0053] Please see Figure 6 In this invention, given a prompt word as input, the agent constructs a query string based on the prompt word content, inputs it into the RAG engine, and obtains parameter text descriptions related to the vulnerability function. The agent then responds based on this text and outputs the parameter combination in a structured form. This parameter combination is also similar to text, in the form of (mutool draw-f 100 @@, where @@ represents an input placeholder and is also the cross-format seed that needs to be generated in this step). To illustrate how this method generates parameter combinations based on RAG, the following example is used:

[0054] Before generating parameters, a vector database is constructed based on the target software's parameter manual and the help information output by the binary program. These two sources are chosen for two reasons: First, the target software's parameter manual provides detailed explanations of options, but some options may be missing. During our experiments, we found that the `run` option was not mentioned in the `mupdf` 1.9 manual, while it was present in the help information of the compiled binary. Using only the manual as the source for constructing the vector database would make it difficult to deduce the source vulnerability related to the `run` option. Second, while the help information provides relatively complete options, the explanations of each option are rather rudimentary. Therefore, this method combines these two information sources to construct the Vulcan vector database. In this example, the prompt input will be given the source project name `openjpeg`, the target software name `mupdf`, and the format of the source vulnerability PoC. The large language model then infers the query text "mutool" and sends it to the query engine to search the vector database for similar text information. The returned context includes explanations of several options for `mutool`, such as "draw", "convert", and "run". The large model determines whether a further query is needed based on contextual information. If a further query is required, the query text will be more refined, such as "mutool draw". Finally, based on the given context and prompts, it generates parameter combinations such as "mutool draw @@ -f pdf".

[0055] Finally, the design of parallel intelligent agents is carried out. Figure 7 This paper presents a parameter generation framework for a parallel agent. Multiple worker nodes are initialized, each with a ProcessEvent event and a maximum retry count. Each worker node uses a different large language model because the content generated by these models is random, and different models exhibit performance differences across different software. Therefore, to improve the quality of each generated content, this study uses multiple large language models to generate content concurrently. An event loop mechanism is employed, triggering different processing logic based on event type to achieve parallel processing by the agent. After collecting all multiple results, they are integrated to generate the final output, which will be further optimized through dynamic execution.

[0056] It should be noted that the above-mentioned structured prompt design and RAG query manual steps are only executed when processing ProcessEvent.

[0057] S2, construct cross-format seed prompts based on candidate parameter combinations, call the ReAct agent to process the cross-format seed prompts with code, and generate cross-format seeds;

[0058] Specifically, step S2 further includes: constructing a cross-format seed prompt word based on the candidate parameter combination and the preset prompt word template, wherein the cross-format seed prompt word is an input constraint;

[0059] The cross-format seed prompts are used as input to the ReAct agent. The task requirements are analyzed to determine the two formats that need to be generated. The code implementation ideas are planned, and the code program is written and executed based on the code implementation ideas to generate candidate cross-format seeds.

[0060] The system monitors the output of code execution. When code execution fails, the system adjusts the code based on error feedback. When code execution succeeds, a cross-format seed is generated. The error feedback includes syntax errors that occur when executing the code generated by the agent, which can be used to correct the code, as well as information to check whether the generated file meets the format requirements.

[0061] In this embodiment, based on the format of the vulnerability PoC and the input sample format of the target software under test, a ReAct agent is constructed based on a large model. Python code is then used to generate a cross-format seed, which contains a file that conforms to both the source vulnerability PoC format and the input format of the target software under test. For example, for Figure 1 In the cross-project vulnerability verification example, this step aims to generate a PDF containing a j2k image.

[0062] Please see Figure 8 , Figure 8 This demonstrates the workflow of a cross-format seed generation agent. The core of this process lies in prompt-driven seed generation: first, given prompts for seed generation, the large language model attempts to write code guided by these prompts; then, it executes the code by calling an external tool and observes the results. If execution is successful, the initial seed is output; otherwise, the code is modified based on feedback information until successful generation or the maximum number of iterations is reached. The prompts for seed generation are as follows: Figure 9 As shown, the prompt words employ a thought chain design, breaking down the complex problem of cross-format seed generation into several steps. The goal is to guide a large language model to write a piece of code for each parameter combination, thereby generating a cross-format hybrid seed (e.g., a PDF file embedding PNG images). Therefore, the prompt words are not the final product, but rather serve as input constraints, driving the agent to generate cross-format seeds.

[0063] The reason for using the ReAct (Reasoning and Acting) agent in this method is that large language models cannot guarantee that the code is completely correct in a single call. However, the success rate and robustness of seed generation can be significantly improved through the cyclical mechanism of "writing code - executing - feedback correction".

[0064] S3 performs corpus optimization on candidate parameter combinations and cross-format seeds to obtain corpus information of downstream software with functions most likely to trigger upstream vulnerabilities, and generates PoC for the target vulnerability by performing parameter-sensitive fuzz testing based on the corpus information.

[0065] Specifically, step S3 further includes: dynamically executing the program based on candidate parameter combinations and cross-format seeds, obtaining execution feedback information, and performing corpus optimization processing on the candidate parameter combinations and cross-format seeds according to the execution feedback information until the optimization end criteria are met, thereby obtaining the corpus information of the downstream software with the most likely functions to trigger the upstream vulnerability. The execution feedback information includes source item function coverage (used to determine which source item functions are covered by the corpus; if no source item is covered, it means that the source item-related functions have not been triggered, guiding the agent to generate parameters associated with the source item or embed seeds of the source item-related formats), program execution output results (during program execution, the terminal will return useful prompts to provide the agent with optimization directions. For example, if the returned result indicates that the given format is not supported, the agent is guided to try to correct the seed to the correct file format. If the returned result indicates parameter contradiction, the agent is guided to modify the parameters), and historical attempt records (guiding the agent to reflect on historical error records and infer reasonable directions for corpus optimization).

[0066] The optimization termination criteria include exceeding the maximum number of iterations and the execution of a set of functions containing a set of vulnerable functions exceeding a preset size. The set of vulnerable functions is a set of function names obtained by dynamically executing the PoC of the vulnerability and recording the functions executed.

[0067] Preferably, the quality of the candidate parameter combinations and cross-format seeds is evaluated based on function trajectory similarity, using the following formula: s represents the corpus, which consists of candidate parameter combinations and cross-format seeds. Let be the set of functions on the execution path of corpus s in target program T, and let P be the set of function calls on the source PoC's call stack in the source program. For a corpus, the higher the similarity of its function trajectories, the closer the corpus is to the core functionality of the source PoC vulnerability.

[0068] In this embodiment, since the seeds generated in the first two steps are only based on existing vulnerability information and the target software manual, the quality of the generated corpus (parameter combinations + files) may not be very high. Therefore, in this step, the initial corpus generated in the aforementioned steps is input into the target software to be tested to obtain runtime feedback information. This feedback information is used to update the prompt words of the first two steps of the large model, generating a higher-quality corpus, which is ultimately used for parameter-sensitive fuzzing to generate a Proof-of-Concept (PoC) for the target vulnerability. It should be noted that the corpus here refers to candidate parameter combinations and cross-format seeds. The purpose of corpus optimization is to use the corpus to dynamically execute the program, continuously optimizing the corpus based on the program's feedback information.

[0069] After collecting the program's execution feedback information, it is integrated into the prompt words and used as input for the agent to guide it in generating the next batch of corpus until the function trajectory similarity threshold is met or the maximum number of iterations is exceeded. This method sets the function trajectory similarity threshold to |P| / 3 because some vulnerabilities have short call stack functions; setting an excessively high threshold might result in the loss of high-quality corpus data. The maximum number of iterations is set to 10 to avoid excessive overhead in LLM corpus generation.

[0070] In summary, the main purpose of this method is to verify whether a known vulnerability in the source software (or upstream software, such as a third-party library) will be triggered in the target software (e.g., software using the source software as a component). If it can be triggered, an input sample (i.e., a Proof-of-Concept) that triggers the vulnerability is generated. This addresses the problem that existing technologies cannot generate parameters associated with the source vulnerability, requiring significant time to search for suitable parameter combinations, leading to low verification efficiency. This method employs a target-aware parameter transfer algorithm, utilizing a large language model to construct multiple agents that infer initial parameter combinations associated with the vulnerability in parallel. Instrumentation is used to obtain vulnerability call stack function coverage information, driving the agents to optimize parameter combinations, thereby reducing the time overhead caused by traditional parameter search space traversal. Furthermore, it addresses the problem that existing technologies struggle to generate high-quality cross-format seeds, resulting in low verification efficiency when upstream and downstream software input formats differ. This method uses the ReAct (Reasoning and Acting) agent to generate an initially usable cross-format seed, which is then executed on the instrumentation program. The cross-format seed is further optimized by combining the vulnerability call stack function coverage, program execution output, and historical attempt records, ultimately improving the efficiency of PoC migration.

[0071] Please see Figure 10 The second embodiment of the present invention provides an automated cross-project vulnerability verification device based on a large language model, which includes:

[0072] Command line parameter combination generation unit 101 is used to obtain multiple working nodes to be processed, call multiple intelligent agents to process them in parallel, construct prompt words, and query the target software manual based on RAG technology to generate candidate parameter combinations.

[0073] The cross-format seed generation unit 102 is used to construct cross-format seed prompts based on candidate parameter combinations, call the ReAct agent to process the cross-format seed prompts with code, and generate cross-format seeds.

[0074] The corpus optimization unit 103 is used to perform corpus optimization processing on candidate parameter combinations and cross-format seeds to obtain corpus information of downstream software with relevant functions that are most likely to trigger upstream vulnerabilities, and to generate the PoC of the target vulnerability by performing parameter-sensitive fuzz testing based on the corpus information.

[0075] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A large language model-based automated cross-project vulnerability verification method, characterized in that, The method comprises the following steps: S1, obtaining a plurality of work nodes to be processed, calling a plurality of agents to perform parallel processing, constructing a prompt word, and querying a target software manual based on RAG technology to generate a candidate parameter combination; S2, constructing a cross-format seed prompt word based on the candidate parameter combination, calling a ReAct agent to perform code writing processing on the cross-format seed prompt word, and generating a cross-format seed; S3, performing corpus optimization processing on the candidate parameter combination and the cross-format seed to obtain corpus information of a downstream software related to a function that is most likely to trigger an upstream vulnerability, and performing parameter sensitive fuzz testing based on the corpus information to generate a PoC of the target vulnerability; The step S1 is specifically: Obtaining a plurality of work nodes to be processed, initializing the work nodes, creating a corresponding ProcessEvent event for each node, and setting a corresponding maximum number of retries for each node; Based on the event loop mechanism, a plurality of agents are used for parallel processing, different event types trigger different processing functions, wherein when the ProcessEvent event is processed, the prompt word is constructed, the large language model is called to generate the parameter combination, when the GenerationDone event is received, the format verification is performed, if the format verification fails, the error information is transmitted back to the large language model for re-generation, and the successful result is collected through the ResultEvent for distributed collection; The successful results of a plurality of events are integrated to generate the final output candidate parameter combination; The step S2 is specifically: According to the candidate parameter combination and the preset prompt word template, a cross-format seed prompt word is constructed, and the cross-format seed prompt word is an input constraint; The cross-format seed prompt word is used as the input of the ReAct agent, the task demand in the cross-format seed prompt word is analyzed, two formats to be generated are determined, a code implementation idea is planned, code programs are written and executed based on the code implementation idea, and a candidate cross-format seed is generated; The output result of the code execution is monitored, the code is adjusted according to the error feedback information when the code execution fails, and the cross-format seed is generated when the code execution succeeds, wherein the error feedback information includes a syntax error that occurs when the code generated by the execution agent is executed, the code can be corrected according to the syntax error information, and information about whether the generated file meets the format requirement is detected.

2. The automatic cross-project vulnerability verification method based on a large language model according to claim 1, characterized in that, When the ProcessEvent event is processed, the prompt word is constructed, and the large language model is called to generate the parameter combination, which is specifically: The large model is driven to query the description in the target software manual to understand the core function of the target program, and the large model is driven to find the options of the possible calling source project in the target software manual; When constructing the command, it is ensured that no conflicting options are used, the input file uses a preset placeholder, when it is judged that the output file requirement is not specified in the target software manual, the preset requirement is used, and when it is judged that there is a specified suffix in the target software manual, the requirement of the target software manual is followed; The fmt_convert attribute is determined according to compatibility of the PoC format, the fmt_convert attribute is used to confirm whether the command directly supports the PoC format, if yes, the PoC format is used, if not, the format to be converted is specified, to confirm the format of the generated file, and the prompt word is constructed.

3. The automated cross-project vulnerability verification method based on a large language model according to claim 2, characterized in that, Also includes: The prompt word is taken as an input of the large model, and the agent is driven to query the target software manual through the RAG technology, wherein the large model infers the core query content to be searched through semantic understanding, the query engine filters out the text segment closest to the query semantics in the pre-generated vector database as enhanced context information returned to the large model based on the semantic query, and the vector distance index is used; The vector distance index includes the Euclidean distance and the cosine similarity, and the database is constructed by the vector embedding technology on the target software manual; The large model outputs a text reply according to the enhanced context information and the given prompt word input, and obtains a parameter combination.

4. The automated cross-project vulnerability verification method based on a large language model according to claim 1, characterized in that, The step S3 is specifically: Based on the candidate parameter combination and the cross-format seed, a dynamic execution program is dynamically executed to obtain execution feedback information, and the candidate parameter combination and the cross-format seed are corpus optimized according to the execution feedback information until an optimization end standard is reached, to obtain corpus information of a downstream software of a related function of an upstream vulnerability, wherein the execution feedback information includes source project function coverage, program execution output result and historical attempt record; The optimization end standard includes exceeding a maximum iteration number and containing a vulnerability function set exceeding a preset size in a function set executed, and the vulnerability function set is obtained by dynamically executing a PoC of a vulnerability, recording functions executed, and obtaining a function name set.

5. The automatic cross-project vulnerability verification method based on a large language model according to claim 4, characterized in that, The quality of the candidate parameter combination and cross-format seed is evaluated based on a function trace similarity, which is formulated as: s is a corpus, which is composed of the candidate parameter combination and the cross-format seed, is a set of functions of the corpus s on the execution trace of the target program T, and P is a set of call stack functions of the source PoC in the source program.

6. A large language model-based automated cross-project vulnerability verification device, characterized by, The method comprises the following steps: A command line parameter combination generation unit is configured to obtain a plurality of work nodes to be processed, call a plurality of agents to perform parallel processing, construct a prompt word, and query a target software manual based on the RAG technology to generate a candidate parameter combination; A cross-format seed generation unit is configured to construct a cross-format seed prompt word based on the candidate parameter combination, call a ReAct agent to perform code writing processing on the cross-format seed prompt word, and generate a cross-format seed; A corpus optimization unit is configured to perform corpus optimization processing on the candidate parameter combination and the cross-format seed to obtain corpus information of a downstream software of a related function of an upstream vulnerability, and perform parameter sensitive fuzzy testing based on the corpus information to generate a PoC of a target vulnerability.

Images