LLM-based multi-tool collaborative vulnerability verification method and device, medium and product

Abstract

The method, device, medium and product for verifying vulnerabilities based on LLM involve the field of network security. In the method, original vulnerability data from multiple security detection tools is obtained, and the original vulnerability data is converted into standard vulnerability data entries; the standard vulnerability data entries are subjected to field merging and vulnerability deduplication according to a preset field priority rule, to generate target vulnerability data entries; the target vulnerability data entries are subjected to semantic analysis based on a preset LLM model, to generate a vulnerability verification configuration package; the vulnerability verification configuration package is parsed by an enhanced executor, and a verification request is actively sent to a target application according to API entry information and attack payloads, and a request response result is obtained, the target application being an application to be subjected to vulnerability verification; and the request response result is analyzed based on a verification strategy, to generate a vulnerability exploitability conclusion. The technical solution provided by the present application realizes accurate determination of vulnerability authenticity.

Description

Technical Field

[0001] This application relates to the field of cybersecurity, specifically to a multi-tool collaborative vulnerability verification method, device, medium, and product based on LLM. Background Technology

[0002] With the rapid popularization and application of AI coding technology, the code production efficiency of software development has been improved like never before. However, at the same time, the surge in the complexity of software systems and the amount of code has also led to an exponential increase in security risks. In order to effectively manage these risks throughout the entire software development chain, the existing security testing system mainly relies on various technical tools such as Static Application Security Testing (SAST), Dynamic Application Security Testing (DAST), and Interactive Application Security Testing (IAST).

[0003] Currently, this traditional security detection system, which relies on multiple detection tools operating independently, suffers from core pain points such as tool silos, high false positive rates, lack of verification, and human resource bottlenecks. First, tools like SAST, IAST, and DAST operate independently, with incompatible vulnerability data formats, hindering effective information linkage and analysis. The same vulnerability is often repeatedly reported by different tools, generating a large amount of noise in work orders and severely interfering with security operation efficiency. Second, the verification mechanisms of existing tools are relatively simple. For example, DAST mainly relies on HTTP response characteristics for matching, resulting in a persistently high false positive rate for vulnerabilities such as SQL injection. The industry-wide false positive rate is generally above 50%, requiring significant manpower for verification. Furthermore, whether it's SAST, DAST, or SCA (Software Component Analysis) which only judges based on version numbers, various detection tools typically only report the discovery of vulnerabilities but cannot confirm whether the vulnerability is actually exploitable. Summary of the Invention

[0004] This application provides a multi-tool collaborative vulnerability verification method, device, medium, and product based on LLM, which enables accurate determination of vulnerability authenticity.

[0005] The first aspect of this application provides a multi-tool collaborative vulnerability verification method based on LLM. The method includes: acquiring raw vulnerability data from multiple security detection tools and converting the raw vulnerability data into standard vulnerability data entries; when standard vulnerability data entries for the same vulnerability exist from multiple security detection tools, performing field merging and vulnerability deduplication on the standard vulnerability data entries according to a preset field priority rule to generate a target vulnerability data entry, the target vulnerability data entry including vulnerability type, API entry information, code location information, and code reachability information; performing semantic analysis on the target vulnerability data entry based on a preset LLM model to generate a vulnerability verification configuration package, the vulnerability verification configuration package including multiple attack payloads and verification strategies; parsing the vulnerability verification configuration package through an enhanced executor, and actively sending a verification request to a target application according to the API entry information and the attack payload, and obtaining the request response result, the target application being the application to be verified, the request response result including an HTTP status code, response headers, response body, and response time; and analyzing the request response result based on the verification strategy to generate a vulnerability exploitability conclusion.

[0006] By adopting the above technical solutions, raw vulnerability data from multiple security detection tools is acquired and converted into standard vulnerability data entries, achieving unified processing of the output formats of different tools. This allows vulnerability data with different formats and structures to be processed and analyzed within the same system. By using preset field priority rules to merge fields and remove duplicate vulnerabilities from standard vulnerability data entries, the problem of multiple tools repeatedly reporting the same vulnerability is effectively eliminated. The generated target vulnerability data entries contain complete information such as vulnerability type, API entry information, code location information, and code reachability information, providing a comprehensive data foundation for subsequent verification. Based on a preset LLM model, semantic analysis of the target vulnerability data entries is performed to generate vulnerability verification configuration packages, realizing intelligent and automated verification processes. The generated configuration packages contain various attack payloads and verification strategies, improving the comprehensiveness and accuracy of verification. By enhancing the executor to actively send verification requests and obtain request response results, proactive vulnerability verification capabilities are achieved. The obtained response results include multi-dimensional information such as HTTP status codes, response headers, response bodies, and response times. Based on the verification strategies, the request response results are analyzed to generate vulnerability exploitability conclusions, achieving accurate determination of vulnerability authenticity and providing reliable verification results for the security team. This method effectively solves the problems of tool silos and duplicate error reports by standardizing, merging fields, and deduplicating the raw vulnerability data from various security detection tools. It generates vulnerability verification configuration packages based on semantic analysis of the LLM model and combines them with the active verification capabilities of the enhanced executor, which effectively reduces the high false positive rate and missing verification phenomenon in existing methods. Through the linkage analysis of request response results and verification strategies, it accurately generates vulnerability exploitability conclusions, thereby realizing the automation and intelligence of vulnerability verification and significantly improving the efficiency of security detection and the accuracy of vulnerability verification.

[0007] Optionally, the step of performing semantic analysis on the target vulnerability data entries based on a preset LLM model to generate a vulnerability verification configuration package specifically includes: extracting a vulnerability type field from the target vulnerability data entries, and matching a corresponding prompt word template from a preset prompt word strategy library based on the vulnerability type field, wherein the prompt word template is used to guide the generation of attack payloads; filling the prompt word template with the vulnerability type, the API entry information, and the code location information as context to form a generation instruction; and inputting the generation instruction into the preset LLM model to generate a vulnerability verification configuration package containing multiple attack payloads and verification strategies.

[0008] By employing the above technical solution, vulnerability type fields are extracted from target vulnerability data entries and matched with corresponding hint templates, enabling customized processing strategies for different vulnerability types and improving the targeting and effectiveness of attack payload generation. The hint templates in the preset hint strategy library contain feature descriptions of various vulnerabilities and attack vector construction rules, ensuring the professionalism and accuracy of the generated instructions. By filling the hint templates with vulnerability type, API entry information, and code location information as context, the generated instructions contain complete vulnerability context information, improving the accuracy of LLM model understanding and generation. After inputting the generation instructions into the preset LLM model, a vulnerability verification configuration package containing multiple attack payloads and verification strategies can be automatically generated, achieving intelligent conversion from vulnerability information to verification configuration. This differentiated processing method based on vulnerability type allows different types of vulnerabilities, such as SQL injection, XSS, and SSRF, to obtain the most suitable verification scheme, greatly improving the success rate and accuracy of vulnerability verification.

[0009] Optionally, the step of actively sending a verification request to the target application based on the API entry information and the attack payload, and obtaining the request response result, specifically includes: constructing an HTTP request based on the API entry information, and injecting the attack payload into the HTTP request based on the injection point location included in the API entry information to obtain an attack request; sending the attack request to the target application, and receiving a complete response from the target application to form the request response result.

[0010] By employing the above technical solution, an HTTP request is constructed based on the API entry information, enabling precise positioning of the verification request and ensuring that the attack payload is accurately delivered to the target interface. The attack payload is injected into the HTTP request based on the injection point location included in the API entry information, achieving precise payload implantation and avoiding verification failure caused by blind injection. Sending the constructed attack request to the target application enables proactive detection of the target application, overcoming the limitations of traditional passive monitoring. Receiving the complete response from the target application forms a request-response result, providing comprehensive data support for subsequent vulnerability assessment. This proactive method of sending verification requests can securely verify the existence of vulnerabilities in a controlled environment, obtaining more realistic and reliable verification results compared to traditional static analysis or passive monitoring. The automated execution of the entire process greatly improves the efficiency of vulnerability verification, simplifying the tedious process of manually constructing and sending requests into an automated workflow.

[0011] Optionally, the step of analyzing the request response result based on the verification strategy to generate a vulnerability exploitability conclusion specifically includes: performing corresponding strategy analysis on the request response result according to the verification strategy in the vulnerability verification configuration package to obtain the individual verification judgment result of each verification strategy; calculating the vulnerability confidence score based on the individual verification judgment result of each verification strategy, combined with the number of sources of security detection tools reporting the same vulnerability and the code reachability information; and generating the vulnerability exploitability conclusion based on the comparison result of the vulnerability confidence score and a preset confidence threshold.

[0012] By adopting the above technical solution, and performing corresponding policy analysis on the request response results according to the verification strategies in the vulnerability verification configuration package, multi-dimensional vulnerability verification and judgment are achieved. Each verification strategy can generate an independent single verification judgment result, improving the comprehensiveness of the verification. Based on the single verification judgment results of each verification strategy, combined with the number of security detection tools reporting the same vulnerability and code reachability information, a vulnerability confidence score is calculated, realizing a multi-factor comprehensive evaluation and avoiding misjudgments that may be caused by a single judgment criterion. The consideration of the number of sources gives higher weight to vulnerabilities reported by multiple tools simultaneously, and the introduction of code reachability information ensures that only truly reachable vulnerabilities are judged as high-risk. The vulnerability exploitability conclusion is generated based on the comparison result of the vulnerability confidence score and the pre-set confidence threshold, realizing a quantitative judgment mechanism, making the vulnerability exploitability judgment more scientific and objective. This judgment method based on the confidence model can effectively distinguish between real vulnerabilities and false alarms, providing accurate guidance for the security team on remediation priority.

[0013] Optionally, the verification strategy includes an error string matching strategy, a time delay detection strategy, an out-of-band callback verification strategy, and a response content matching strategy. The step of performing corresponding strategy analysis on the request-response result according to the verification strategy in the vulnerability verification configuration package to obtain the individual verification judgment result of each verification strategy specifically includes: when the verification strategy is an error string matching strategy, filtering for identically named strings in the response body of the request-response result using a keyword whitelist to exclude false matches, and then detecting whether it contains a preset error feature string; if it contains the preset error feature string, then the individual verification judgment result of the verification strategy is determined as a hit; when the verification strategy is a time delay detection strategy, the response time of the request-response result is... The time delay detection strategy is compared with a preset response time threshold. When any two consecutive response times exceed the preset response time threshold, the single-item verification result of the time delay detection strategy is determined to be a hit. When the verification strategy is an out-of-band callback verification strategy, it is determined whether a callback request associated with the vulnerability verification configuration package is received. If a callback request associated with the vulnerability verification configuration package is received, the single-item verification result of the out-of-band callback verification strategy is determined to be a hit. When the verification strategy is a response content matching strategy, it is detected whether the response body of the request response result contains expected feature content corresponding to the attack payload. If the expected feature content is detected, the single-item verification result of the content matching strategy is determined to be a hit.

[0014] By adopting the above technical solutions, the error string matching strategy, combined with keyword whitelisting to filter for identically named strings, effectively eliminates common mismatches, improves the accuracy of error detection, and avoids misjudging identically named characters in normal business responses as evidence of vulnerabilities. The time delay detection strategy, through verification of two consecutive response times, ensures the reliability of time-based vulnerability assessments and avoids misjudgments caused by accidental factors such as network jitter. The out-of-band callback verification strategy, by detecting callback requests associated with the vulnerability verification configuration package, provides the most credible proof of vulnerability existence; this physical-level verification has virtually no possibility of false positives. The response content matching strategy, by detecting expected characteristic content, can verify whether the attack payload was successfully executed and reflected, providing direct evidence for vulnerability assessment. The combined use of these four verification strategies forms a comprehensive verification system, with different strategies complementing and verifying each other, greatly improving the accuracy of vulnerability assessment. Each strategy has clear hit criteria, making the verification results highly reproducible and interpretable.

[0015] Optionally, before analyzing the request response results based on the verification strategy and generating a vulnerability exploitability conclusion, the method further includes: sending multiple baseline requests without attack payloads to the target application through the enhanced executor, and measuring the average and standard deviation of the response time; determining a preset response time threshold based on the average and the standard deviation, wherein the preset response time threshold is the average plus the standard deviation by a preset multiple.

[0016] By employing the above technical solution, the enhanced executor sends multiple baseline requests without attack payloads to the target application, establishing a baseline for the target application's response time under normal conditions. This provides a scientific reference standard for subsequent time-based vulnerability assessment. Measuring the average and standard deviation of the response time enables statistical modeling of the target application's response characteristics, taking into account the natural fluctuation range of application response time. A preset response time threshold is determined based on the average and standard deviation, using the average plus a preset multiple of the standard deviation, adhering to statistical principles and ensuring the rationality of the threshold setting. This dynamic threshold determination method can adapt to the performance characteristics of different applications, avoiding false positives that may result from using a fixed threshold. For applications with inherently long response times, the threshold will be increased accordingly; for applications with fast responses, the threshold will be decreased accordingly, achieving personalized judgment criteria. Through this baseline establishment mechanism, the detection accuracy of time-based blind injection vulnerabilities is significantly improved, effectively reducing false positives caused by network latency or server load variations.

[0017] Optionally, before performing semantic analysis on the target vulnerability data entry based on a preset LLM model to generate a vulnerability verification configuration package, the method further includes: when the API address field of the target vulnerability data entry is incomplete, obtaining supplementary API entry information from multiple supplementary sources according to a preset priority order; when the supplementary API entry information is obtained from any of the supplementary sources, filling the supplementary API entry information back into the API address field of the target vulnerability data entry, and updating the status of the target vulnerability data entry from the pending completion status to the verifiable status.

[0018] By adopting the above technical solution, when the API address field is incomplete, supplementary API entry information is obtained from multiple supplementary sources according to a preset priority order. This solves the problem of incomplete API identification by static analysis tools in scenarios such as dynamic routing, ensuring the executability of vulnerability verification. The priority design of multiple supplementary sources fully considers the reliability and accuracy of the data. The API list captured by IAST runtime is given the highest priority, providing the most realistic runtime data; the OpenAPI specification document is given the second priority, providing standard interface information defined by developers; traffic mirroring records, LLM inference, and manual annotation are used as subsequent supplementary means, forming a complete information acquisition chain. When supplementary API entry information is obtained from any supplementary source, it is immediately backfilled, realizing timely data updates and avoiding verification interruptions caused by missing information. The status of the target vulnerability data entry is updated from the pending completion state to the verifiable state, realizing precise status management and ensuring that only vulnerabilities with complete information enter the verification process. This multi-source supplementation mechanism greatly improves the automation level of the system, reduces the need for manual intervention, and enables vulnerabilities that could not be verified due to incomplete information to be successfully verified automatically.

[0019] Secondly, embodiments of this application provide an LLM-based multi-tool collaborative vulnerability verification device, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, which includes computer instructions, and the one or more processors call the computer instructions to cause the LLM-based multi-tool collaborative vulnerability verification device to perform the method described in the first aspect and any possible implementation thereof.

[0020] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on an LLM-based multi-tool collaborative vulnerability verification device, cause the LLM-based multi-tool collaborative vulnerability verification device to perform the method described in the first aspect and any possible implementation thereof.

[0021] Fourthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on an LLM-based multi-tool collaborative vulnerability verification device, cause the LLM-based multi-tool collaborative vulnerability verification device to execute the method described in the first aspect and any possible implementation thereof.

[0022] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages: 1. By using semantic analysis based on the LLM model and a multi-tool collaboration mechanism, not only are the problems of tool silos and information inconsistency in traditional vulnerability verification solved, but the accuracy of verification requests and vulnerability matching can also be improved by generating vulnerability verification configuration packages. Then, the enhanced executor can be used to initiate active verification of the target application and analyze the request response results, which greatly reduces the false positive rate and manual verification cost, and effectively improves the accuracy and efficiency of vulnerability verification.

[0023] 2. By combining verification strategies (such as error string matching strategy, time delay detection strategy, out-of-band callback verification strategy, and response content matching strategy) with multi-dimensional analysis of request and response results, the system can automatically generate vulnerability exploitability conclusions. Furthermore, by introducing a quantitative evaluation mechanism for vulnerability confidence, and combining code accessibility information and the contribution of detection tools, the system can further improve the credibility of vulnerability verification results and decision support capabilities.

[0024] 3. By supplementing the API entry information of the target vulnerability data entries and dynamically calculating the response time threshold, the inaccuracy issues in incomplete data and time delay detection are resolved, enhancing the accuracy and effectiveness of verification requests and providing comprehensive technical assurance and support for realizing a multi-tool collaborative vulnerability verification method. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the multi-tool collaborative vulnerability verification method based on LLM disclosed in an embodiment of this application; Figure 2 This is another flowchart illustrating the multi-tool collaborative vulnerability verification method based on LLM disclosed in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the multi-tool collaborative vulnerability verification device based on LLM provided in the embodiments of this application.

[0026] Explanation of reference numerals in the attached drawings: 301, Central Processing Unit; 302, Read-Only Memory; 303, Random Access Memory; 304, Bus; 305, Input / Output Interface; 306, Input Section; 307, Output Section; 308, Storage Section; 309, Communication Section; 310, Driver; 311, Removable Media. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0028] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0029] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple system devices refer to two or more system devices, and multiple screen terminals refer to two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0030] This application provides a multi-tool collaborative vulnerability verification method based on LLM, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the LLM-based multi-tool collaborative vulnerability verification method provided in this application embodiment. The method is applied to a server, which can execute an LLM-based multi-tool collaborative vulnerability verification program. The method includes steps S101 to S105, as follows: Step S101: Obtain raw vulnerability data from multiple security detection tools and convert the raw vulnerability data into standard vulnerability data entries.

[0031] In step S101, the multiple security testing tools refer to a set of tools including but not limited to Static Application Security Testing (SAST), Interactive Application Security Testing (IAST), and Dynamic Application Security Testing (DAST); the raw vulnerability data refers to vulnerability reports generated by the aforementioned multiple security testing tools that are incompatible in format; the standard vulnerability data entry refers to a unified vulnerability format UVF, which includes a set of fields such as basic identifier, vulnerability semantics, code location, API entry, verification results, and raw data retention, used to provide a standardized data structure for subsequent processing.

[0032] Specifically, the server first acquires raw vulnerability data by executing different security detection tools. For example, the server drives the SAST tool to scan the application's source code to obtain static analysis vulnerability data containing code locations and data flow paths. Simultaneously, it drives the IAST tool to capture the application's runtime API list and request structure under real traffic through instrumentation and listening techniques, and drives the DAST tool to dynamically scan the application's exposed interfaces to obtain response anomaly records. After acquiring this raw vulnerability data from different sources and in various formats, the server performs a standardization transformation process, uniformly mapping and converting the SAST, IAST, and DAST raw vulnerability data into a preset standard vulnerability data entry format, thereby forming a standard vulnerability data entry list. This lays the foundation for subsequent cross-tool data fusion and analysis.

[0033] Step S102: When there are standard vulnerability data entries for the same vulnerability from multiple security detection tools, the standard vulnerability data entries are merged and deduplicated according to preset field priority rules to generate target vulnerability data entries. The target vulnerability data entries include vulnerability type, API entry information, code location information, and code reachability information.

[0034] In step S102, the preset field priority rule refers to assigning a source priority order to fields with the same semantics when merging vulnerability data from different tools. The order from high to low is IAST, DAST, and SAST. The target vulnerability data entry refers to a standard vulnerability data entry that represents a unique vulnerability after merging and deduplication, and whose information has been supplemented and optimized.

[0035] Specifically, the server iterates through the list of standard vulnerability data entries generated in step S101. When the server identifies multiple standard vulnerability data entries pointing to the same actual vulnerability, it initiates a field merging and vulnerability deduplication process. The server correlates and deduplicates these entries based on information such as vulnerability type and code location, ensuring that each vulnerability is represented by only one data entry. When merging field content, the server strictly follows preset field priority rules. For example, for the API entry information field, if entries from IAST, DAST, and SAST all provide information for this field, the server will prioritize the information provided by IAST, followed by DAST, and finally SAST, to ensure the highest accuracy for this field. After this processing, the server finally generates a target vulnerability data entry containing the precise vulnerability type, the most reliable API entry information, code location information, and code reachability information.

[0036] Step S103: Perform semantic analysis on the target vulnerability data entries based on the preset LLM model to generate a vulnerability verification configuration package, which includes various attack payloads and verification strategies.

[0037] In step S103, the preset LLM model refers to a large language model deployed in the server's internal environment, such as the Qwen or DeepSeek model, used for vulnerability semantic analysis and generation tasks; the vulnerability verification configuration package refers to a structured set of configuration data, which includes fields such as poc_id (vulnerability verification test case ID), uvf_ref (universal vulnerability format reference), verification strategy, main request, attack payload variant, out-of-band callback OOB configuration, and retry timeout parameters, used to guide subsequent automated verification operations; the attack payload refers to a carefully constructed data fragment to trigger a specific vulnerability; the verification strategy refers to the specific detection method used to determine whether a vulnerability exists, including error string matching strategy, time delay detection strategy, out-of-band callback verification strategy, and response content matching strategy.

[0038] Specifically, the server extracts the vulnerability type, API entry point information, and code location information from the target vulnerability data entries. Based on the extracted vulnerability type, the server matches and selects a corresponding prompt template from a pre-defined prompt strategy library. Subsequently, the server uses the extracted vulnerability type, API entry point information, and code location information as context content to populate the selected prompt template, thus forming a complete generation instruction. The server inputs this generation instruction into a pre-defined LLM model. Upon receiving the instruction, the pre-defined LLM model performs semantic analysis and reasoning to generate a structured vulnerability verification configuration package. This vulnerability verification configuration package contains multiple attack payloads targeting the vulnerability and explicitly specifies the verification strategy used to determine whether the attack was successful.

[0039] In one possible implementation, semantic analysis is performed on the target vulnerability data entries based on a preset LLM model to generate a vulnerability verification configuration package, specifically including steps S1031-S1033, as follows: Step S1031: Extract the vulnerability type field from the target vulnerability data entry, and match the corresponding prompt word template from the preset prompt word policy library according to the vulnerability type field. The prompt word template is used to guide the generation of attack payload.

[0040] In step S1031, the vulnerability type field represents the field in the target vulnerability data entry used to indicate the specific classification of the vulnerability, such as SQL injection, XSS, or SSRF; the preset prompt word policy library refers to a storage set pre-established in the server, which contains exclusive prompt word templates designed for different vulnerability types; the prompt word template refers to a text structure containing placeholders customized for a specific vulnerability type, which is used to guide the preset LLM model to generate attack payloads with specific format and content.

[0041] Specifically, the server first parses the target vulnerability data entry to accurately extract the value of the vulnerability type field. For example, the server reads the value of the vulnerability type field as "SSRF". Then, the server uses this value as a query index to search in a pre-defined hint policy library to match and retrieve a hint template specifically designed for SSRF vulnerabilities. This hint template has predefined core logic and requirements for generating the attack payload, such as requiring the generated payload to attempt to access an externally controlled address.

[0042] Step S1032: Using the vulnerability type, the API entry information, and the code location information as context, fill in the prompt word template to form a generation instruction.

[0043] In step S1032, the context refers to a series of information that provides a specific context for the preset LLM model, including vulnerability type, API entry information, and code location information; the generated instruction refers to a complete instruction that can be directly understood and executed by the preset LLM model by filling the context information into the prompt word template.

[0044] Specifically, the server extracts API entry information from the currently processed target vulnerability data entries. This API entry information includes the request URL, request method, and related parameters. It also extracts code location information, including the file path and line number where the vulnerability is located. The server combines this extracted information with the vulnerability type obtained in step S1031, using it as context content, and fills it into the corresponding placeholder positions of the prompt word template matched from the preset prompt word policy library. This process transforms a general template into a detailed generation instruction specific to this particular vulnerability instance.

[0045] Step S1033: Input the generation command into the preset LLM model to generate a vulnerability verification configuration package containing multiple attack payloads and verification strategies.

[0046] In step S1033, the server takes the generation instruction generated in step S1032 as input and sends it to the preset LLM model deployed within the server. Upon receiving the generation instruction, the preset LLM model performs semantic analysis and inference based on the contextual information in the instruction and its built-in vulnerability knowledge. The model's output is strictly limited to a structured data format, which is the vulnerability verification configuration package. The vulnerability verification configuration package generated by the server is complete, containing not only one attack payload but also multiple attack payload variants designed to improve the verification success rate, and explicitly specifies the subsequent verification strategies used to determine whether the vulnerability truly exists for these attack payloads.

[0047] To facilitate understanding, the following concrete example will be used to explain steps S1031-S1033: The server obtains a target vulnerability data entry indicating an SSRF vulnerability in an image transfer function. First, the server executes step S1031, extracting the vulnerability type field as SSRF from the entry and matching it with a specific SSRF-specific prompt template from the preset prompt policy library. The core instruction of this template is to generate a URL that can trigger an out-of-band network request as the attack payload. Next, the server executes step S1032, extracting the API entry information (POST / api / image / fetch) and the code location information (Line 58 of ImageFetcher.java) from the target vulnerability data entry, and filling this information along with the vulnerability type SSRF into the prompt template to form a complete generation instruction. Finally, the server executes step S1033, sending this generation instruction to the preset LLM model. After analyzing the instruction, the preset LLM model generates a vulnerability verification configuration package. The package contains various attack payloads, such as http: / / unique-id.oob.domain and https: / / unique-id.oob.domain, and specifies an out-of-band callback verification strategy, which means that it is necessary to monitor whether the oob.domain domain has received a DNS query or HTTP request from the target application server.

[0048] In one possible implementation, before performing semantic analysis on the target vulnerability data entry based on a preset LLM model to generate a vulnerability verification configuration package, the method further includes: when the API address field of the target vulnerability data entry is incomplete, obtaining supplementary API entry information from multiple supplementary sources according to a preset priority order; (wherein the multiple supplementary sources are, in order of priority from high to low: API list captured by IAST runtime, OpenAPI specification document, traffic mirroring record, API path inferred by LLM based on the code location information, and manual annotation) when the supplementary API entry information is obtained from any of the supplementary sources, the supplementary API entry information is backfilled into the API address field of the target vulnerability data entry, and the status of the target vulnerability data entry is updated from the pending completion status to the verifiable status.

[0049] Specifically, before executing the core task of generating a vulnerability verification configuration package based on a preset LLM model, the server first performs a critical data preprocessing and enhancement process to ensure that all target vulnerability data entries used for analysis have the complete information necessary for effective verification. The core purpose of this process is to address the insufficient accuracy of sources such as Static Analysis Tools (SAST) in identifying API entry points in dynamic routing, complex frameworks, or microservice architectures. When the server's standardization layer component processes a target vulnerability data entry from a SAST tool, if it detects that the API address field in the data entry is empty, incomplete, or improperly formatted, the server will not immediately discard it or pass it to subsequent LLM models. Instead, it will mark the target vulnerability data entry as pending completion and initiate a structured API entry information supplementation procedure executed in a preset priority order.

[0050] The program strictly follows a waterfall query strategy, moving from high-reliability to low-reliability sources, to maximize the accuracy of supplementary information. The server will attempt to retrieve supplementary API entry information from multiple sources in the following order: First, the server queries the highest-priority supplementary source: the list of APIs captured by IAST at runtime. This source is given the highest priority because the IAST tool, through instrumentation at application runtime, can capture every network request actually processed by the application and its corresponding backend code execution path. The server uses existing code location information in the target vulnerability data entries, such as class names, method names, or line numbers, to retrieve the list of APIs captured and stored by the IAST tool. If a record is found in this list that exactly matches the code location information, the server extracts the complete and accurate API address field information contained in that record, including the HTTP method and URL path.

[0051] If no match is found in the API list captured during IAST runtime (e.g., IAST does not cover the code module), the server will turn to a second-priority supplementary source: the OpenAPI specification document. Applications that follow the API-first design principle typically maintain an OpenAPI (or Swagger) format API specification document. The server loads and parses the OpenAPI specification document associated with the target application, then again uses the code location information from the target vulnerability data entry to attempt matches against the various API operations defined in the document. By comparing controller names, method signatures, and other information, the server can locate the corresponding API definition and extract the authoritative API address field.

[0052] If the OpenAPI specification documentation is unavailable or does not contain the relevant information, the server will continue to try a third-priority supplementary source: traffic mirroring records. Traffic mirroring records all real requests flowing through the network in a specific environment (such as a test environment). The server accesses the database or log system storing these records and performs a more complex matching logic. The server may search and correlate massive amounts of traffic records based on parameter names, code path fragments, or other characteristics in the target vulnerability data entry to infer API address fields related to the vulnerable code. This is a costly but still effective supplementary method based on real-world data.

[0053] When all attempts based on structured or semi-structured data fail, the server activates a fourth priority supplementary source: inferring the API path using the LLM based on the code location information. At this stage, the server constructs a specific analysis task, providing the code location information from the target vulnerability data entry, particularly code snippets containing routing annotations, as input to a pre-defined LLM model. The server instructs the LLM model to act as an expert familiar with the development framework, inferring the most probable complete API address based on the code context. The powerful pattern recognition and code understanding capabilities of the LLM model enable it to handle many complex situations that traditional rules cannot cover.

[0054] Finally, if even the LLM model cannot provide a high-confidence inference, the server will resort to the last, lowest-priority supplementary source: manual annotation. At this point, the target vulnerability data entry will remain in a pending state and will be automatically pushed to a dedicated manual review queue for security engineers. Security engineers will intervene, manually analyzing the code or reproducing the vulnerability to determine its precise API entry point, and then manually enter this supplementary API entry point information into the system.

[0055] Upon successfully obtaining supplementary API entry information from any of the aforementioned sources, the server will immediately interrupt the subsequent query process. The server will then populate the API address field of the original target vulnerability data entry with the obtained supplementary API entry information, whether it comes from precise captures by IAST, OpenAPI definitions, traffic mirroring records, LLM inferences, or manual annotations. After population, the server will immediately update the status of the target vulnerability data entry from "Pending Completion" to "Verifiable." This status change signifies that the data entry has transformed from an incomplete original alert into a complete and valid vulnerability intelligence that can be processed by downstream automated processes and can be securely fed into the LLM model for the next step of vulnerability verification configuration package generation.

[0056] Step S104: The enhanced executor parses the vulnerability verification configuration package and actively sends a verification request to the target application based on the API entry information and the attack payload, and obtains the request response result. The target application is the application to be verified for vulnerability. The request response result includes the HTTP status code, response header, response body and response time.

[0057] In step S104, the enhanced executor refers to an execution entity in the server that has the ability to actively send HTTP requests, analyze responses, and listen for out-of-band callbacks. This entity is an upgrade from the traditional passive observation capability. The target application refers to a running application instance that is to be verified for vulnerabilities. The request response result has been defined in step S104 as complete response data that includes the HTTP status code, response header, response body, and response time.

[0058] Specifically, the server uses an enhanced executor to parse the vulnerability verification configuration package generated in step S103. The enhanced executor first extracts the entry information of the target API from the configuration package, including the request method, URL, and the injection point location of the attack payload. Based on this information, the enhanced executor constructs one or more HTTP requests and automatically obtains a valid authentication token from a runtime session pool and injects it into the request header. Next, based on the injection point location, the enhanced executor injects the attack payloads provided in the vulnerability verification configuration package into the corresponding parts of the HTTP requests one by one, forming the final attack requests. The enhanced executor sends these attack requests to the target application running in an isolated environment and accurately captures the complete response returned by the target application, forming a request-response result containing the HTTP status code, response headers, response body, and response time for subsequent analysis.

[0059] In one possible implementation, based on the API entry information and the attack payload, a verification request is actively sent to the target application, and the request response result is obtained. Specifically, this includes steps S1041-S1042, as follows: Step S1041: Construct an HTTP request based on the API entry information, and inject the attack payload into the HTTP request according to the injection point location included in the API entry information to obtain an attack request.

[0060] In step S1041, an HTTP request represents a raw request structure that follows the Hypertext Transfer Protocol format and is used to interact with the target application; the injection point location refers to the specific location in the HTTP request where the attack payload is to be inserted, such as a URL parameter, a request header field, or a key-value pair in the request body; and the attack request refers to the final request for vulnerability verification formed after the attack payload has been successfully injected into the HTTP request.

[0061] Specifically, the server executes this step through an enhancement executor. The enhancement executor first constructs a basic HTTP request based on the API entry information provided in the vulnerability verification configuration package. This HTTP request includes the correct request method, request URL, and necessary request headers. Simultaneously, the enhancement executor automatically matches and extracts a valid authentication token from a runtime session pool captured via IAST instrumentation, and injects this authentication token into the authentication field of the HTTP request to pass the target application's authentication. Subsequently, based on the injection point locations explicitly stated in the API entry information, the enhancement executor sequentially and precisely inserts one or more attack payloads provided in the vulnerability verification configuration package into the specified positions of the HTTP request, thereby generating one or more attack requests to be sent.

[0062] Step S1041: Send the attack request to the target application and receive a complete response from the target application to form the request response result.

[0063] In step S1042, a complete response refers to all the data returned by the target application after receiving the attack request. This complete data includes the HTTP status code, response headers, response body, and the total response time from the sending of the request to the complete reception of the response. The request response result refers to the data record formed by structurally encapsulating the complete response, which is used for subsequent analysis steps.

[0064] Specifically, the server, through the enhanced executor, sends the attack request generated in step S1041 to the target application in a separate and isolated staging environment used for testing. The enhanced executor strictly controls packet sending behavior to ensure that it does not affect the production environment. After sending the attack request, the enhanced executor enters a listening state, waiting for and receiving responses from the target application. The enhanced executor captures all data throughout the entire response process, including the final HTTP status code, all response header fields, the complete response body content, and accurately records the response time consumed from sending the attack request to receiving the complete response. Finally, the server combines and formats this captured data to form a well-organized request-response result for analysis in subsequent steps.

[0065] To facilitate understanding, the following concrete example will be used to explain steps S1041-S1042: The server's enhanced executor parsed a vulnerability verification configuration package aimed at verifying the existence of an XSS vulnerability in a user comment interface. The API entry point information specified a POST request method, a URL of / api / comments, and the injection point as the comment_text field in the request body. The attack payload was a piece of JavaScript code. <script>alert(document.domain)< / script> The enhanced executor first constructs a POST request to ` / api / comments`, automatically retrieving a user authentication token from the session pool and adding it to the request header. Next, the enhanced executor sends the attack payload... <script>alert(document.domain)< / script> The `comment_text` field in the request body is filled in to form the final attack request. The enhanced executor then sends this attack request to the target application running in the staging environment and receives the response from the target application. The enhanced executor captures an HTTP status code of 200, a response body containing the submitted comment, and a response time of 85 milliseconds. The server combines this information—including the status code, response headers, response body containing the attack payload, and response time—into a single request-response result, which is then passed to the next step for analysis to determine whether the attack payload was successfully executed.

[0066] Step S105: Based on the verification strategy, analyze the request response results and generate a vulnerability exploitability conclusion.

[0067] In step S105, the vulnerability exploitability conclusion refers to a final determination of whether a vulnerability actually exists and can be exploited. This conclusion is divided into three levels: confirmed exploitability, suspected exploitability, and unexploitable exploitability.

[0068] Specifically, the server first performs corresponding analysis on the request-response result obtained in step S104 according to the verification strategy specified in the vulnerability verification configuration package. If the verification strategy is an error string matching strategy, the server checks whether the response body contains a preset error characteristic string, and combines keyword whitelist filtering to exclude false positives. If the verification strategy is a time delay detection strategy, the server compares the response time with a threshold dynamically determined based on the average and standard deviation of multiple benchmark requests. If the verification strategy is an out-of-band callback verification strategy, the server determines whether its deployed listening service has received the unique callback request associated with this verification. Each verification strategy yields a single verification judgment result. Subsequently, based on all single verification judgment results, combined with the number of security detection tools reporting the vulnerability and code reachability information, the server calculates a comprehensive vulnerability confidence score through a weighted calculation model. Finally, the server compares this vulnerability confidence score with two preset confidence thresholds, such as 0.7 and 0.4, to generate the final vulnerability exploitability conclusion.

[0069] To facilitate understanding, the following example of the discovery and verification process of an SQL injection vulnerability will be used to explain steps S101-S105: After developers submitted code containing a user query function, the SAST tool detected a potential SQL injection risk and generated raw vulnerability data, but this data lacked a complete API access path. Simultaneously, the DAST tool, while scanning the application, also sent a probe request to a user query interface and received a database error response, similarly generating raw vulnerability data. After obtaining these two raw vulnerability data sets, the server first converted them into standard vulnerability data entries. Next, the server identified that both sets of data pointed to the same vulnerability and merged and deduplicated them according to a predefined field priority rule: IAST > DAST > SAST. Because the DAST data contained explicit API entry information, the final target vulnerability data entry contained the accurate API path, request method, and parameter information.

[0070] Subsequently, the server submits this target vulnerability data entry regarding SQL injection to the preset LLM model. The LLM model, based on its built-in SQL injection vulnerability hint template and combining the API information and code context in the entry, generates a vulnerability verification configuration package containing various attack payloads and verification strategies. This configuration package may contain a time-based blind SQL injection attack payload, such as 'AND (SELECT 1 FROM (SELECT(SLEEP(5)))a)--', and specifies the verification strategy as a time-delay detection strategy.

[0071] The enhanced executor parses the vulnerability verification configuration package and first sends several baseline requests without attack payloads to the target application's API, measuring an average response time of 150 milliseconds. Then, the enhanced executor injects a time-based blind injection attack payload into the request parameters and sends the attack request to the target application. The server's response shows a response time of 5180 milliseconds. The enhanced executor may resend the request for confirmation, obtaining a similar delay result.

[0072] Finally, the server analyzes the request-response result. Since 5180 milliseconds far exceeds the response time threshold determined based on the baseline request, the single-item verification result of the time latency detection strategy is a hit. Combining this hit result with the fact that there are two vulnerability sources, the server calculates a vulnerability confidence score of 0.85. Because 0.85 is higher than the preset threshold of 0.7, the server ultimately generates a vulnerability exploitability conclusion of "confirmed exploitability." This conclusion, along with the evidence chain containing the specific request and response time, is automatically generated into a security report, and a remediation ticket is created and pushed to the relevant development team.

[0073] Please refer to Figure 2 Based on the verification strategy, the request response result is analyzed to generate a vulnerability exploitability conclusion, specifically including steps S201-S210, as follows: Step S201: When the verification strategy is an error string matching strategy, after filtering out false matches by combining the keyword whitelist in the response body of the request response result, it is detected whether the preset error feature string is included.

[0074] In step S201, the error string matching strategy refers to a verification method that determines the existence of a vulnerability by detecting specific database or server error information in the application response; the keyword whitelist refers to a predefined set of strings used to exclude strings that contain error keywords but belong to normal business content during the error detection process to prevent false alarms; the preset error feature string refers to a database containing various typical vulnerability error messages, such as specific strings containing SQL syntax errors, XML parsing errors, etc.

[0075] Specifically, after receiving the request-response result, the server's response analyzer first determines whether the verification strategy specified in the vulnerability verification configuration package is an error string matching strategy. If so, the response analyzer will scan the response body content in the request-response result. During the scanning process, the response analyzer will first apply a keyword whitelist to filter out normal business strings in the response body that match the whitelist content. After filtering, the response analyzer will then compare the remaining response body content with a preset error signature string library to detect whether there is any real error information.

[0076] Step S202: If the preset error feature string is included, the single-item verification judgment result of the verification strategy is determined as a hit; In step S202, the single-item verification judgment result represents an independent judgment conclusion for a specific verification strategy, which is usually a hit or a miss.

[0077] Specifically, if the server's response analyzer successfully matches a preset error feature string during the detection process in step S201, the server records the single-item verification judgment result of this error string matching strategy as a hit.

[0078] Step S203: When the verification strategy is a time delay detection strategy, the response time of the request response result is compared with a preset response time threshold. In step S203, the time delay detection strategy refers to a verification method that uses an attack payload to inject an application into performing a time-consuming operation and measures whether the response time increases significantly to determine whether a vulnerability exists; the preset response time threshold refers to a critical value used to determine whether the response time is abnormal, and this critical value is dynamically calculated using statistical methods.

[0079] Specifically, before initiating an active packet attack, the server's response analyzer sends at least three normal requests without attack payloads to the target application to establish a statistical baseline for response time. Subsequently, the response analyzer calculates the average response time and standard deviation based on this baseline, and generates a preset response time threshold for this verification according to a rule such as the average plus twice the standard deviation. Upon receiving a response to a request containing a time-delay attack payload, the response analyzer compares the actual response time of that request with the calculated preset response time threshold.

[0080] Step S204: When the response times of any two consecutive responses exceed the preset response time threshold, the single-item verification judgment result of the time delay detection strategy is determined to be a hit; In step S204, to eliminate the possibility of increased response time due to accidental factors such as network fluctuations, the server's response analyzer requires the enhanced executor to send at least two consecutive attack requests containing time-delay attack payloads. The response analyzer performs a time comparison after receiving the response to each request. Only when the response times of two consecutive attack requests both exceed the preset response time threshold determined in step S203 will the server ultimately determine that the single-item verification result of the time delay detection strategy is a hit.

[0081] Step S205: When the verification strategy is an out-of-band callback verification strategy, determine whether a callback request containing the vulnerability verification configuration package has been received; In step S205, the out-of-band callback verification strategy refers to a verification method that injects an attack payload that can trigger the target application to make a network request to an external server, and determines whether a vulnerability exists by listening to whether the external server receives the request; the callback request refers to a network request sent by the target application to the external listening server.

[0082] Specifically, the server deploys an internally self-hosted out-of-band callback listening service. When generating the vulnerability verification configuration package, a globally unique subdomain identifier is assigned to each verification task using the out-of-band callback verification strategy. When the verification strategy is determined to be an out-of-band callback verification strategy, the server's out-of-band callback listening service begins asynchronously listening for DNS, HTTP, and other protocol requests sent to that unique subdomain. This listening process lasts for a preset period of time, such as 30 seconds.

[0083] Step S206: If a callback request containing the vulnerability verification configuration package is received, the single verification judgment result of the out-of-band callback verification strategy is determined to be a hit; In step S206, if the server's out-of-band callback listening service successfully receives a network request within the preset listening time, and the target address of the request contains the globally unique identifier associated with the current vulnerability verification configuration package, the server confirms that the callback request was triggered by this vulnerability verification action. Subsequently, the server determines the single verification result of the out-of-band callback verification strategy as a hit, and archives the received original callback request message as evidence.

[0084] Step S207: When the verification strategy is a response content matching strategy, detect whether the response body of the request response result contains expected feature content corresponding to the attack payload; In step S207, the response content matching strategy refers to a verification method that determines the existence of a vulnerability by checking whether the response content contains an attack payload or the expected output generated by the attack payload; the expected feature content refers to a specific string or data pattern that is expected to appear in the response body after a successful attack.

[0085] Specifically, when the server's response analyzer determines that the verification policy is a response content matching policy, the analyzer retrieves the attack payload itself or the specific result expected to be generated by the attack payload from the vulnerability verification configuration package as the expected characteristic content. The analyzer then carefully examines the response body of the requested response result to confirm whether the expected characteristic content exists in the response body.

[0086] Step S208: If the expected feature content is detected, the single-item verification judgment result of the content matching strategy is determined to be a hit.

[0087] In step S208, if the server's response analyzer finds the expected feature content corresponding to the attack payload in the response body during the detection process in step S207, the server records the single-item verification judgment result of this content matching strategy as a hit.

[0088] Step S209: Based on the individual verification judgment results of each verification strategy, combined with the number of sources of security detection tools reporting the same vulnerability and the code reachability information, calculate the vulnerability confidence score.

[0089] In step S209, the vulnerability confidence score is a numerical value used to quantify the degree of vulnerability exploitability by calculating a weighted average of multi-dimensional information; the number of sources of security detection tools refers to the number of different types of tools such as SAST, IAST, and DAST that report the same vulnerability; and the code reachability information refers to the analytical conclusion on whether the code path where the vulnerability is located can be accessed in the actual operation of the application.

[0090] Specifically, after determining all individual verification strategies, the server initiates a multi-dimensional cross-validation confidence model for calculation. The server first aggregates all strategies that are confirmed to be successful in each individual verification and assigns a preset weight to each strategy; for example, the out-of-band callback verification strategy has the highest weight. Simultaneously, the server counts the number of security detection tools reporting the vulnerability; the more sources, the higher the weight. The server also incorporates code reachability information; if code reachability is confirmed, the weight is increased. Finally, the server weights all these dimensions and sums them to obtain the final vulnerability confidence score.

[0091] Step S210: Based on the comparison result between the vulnerability confidence score and the preset confidence threshold, generate the vulnerability exploitability conclusion.

[0092] In step S210, the preset confidence threshold refers to two predefined score lines used to divide the vulnerability exploitability level; the vulnerability exploitability conclusion refers to the final judgment result on whether the vulnerability is truly exploitable, which is divided into three levels: confirmed exploitable, suspicious, and unexploitable.

[0093] Specifically, the server compares the vulnerability confidence score calculated in step S209 with two preset confidence thresholds, such as a high threshold of 0.7 and a low threshold of 0.4. If the vulnerability confidence score is greater than the high threshold of 0.7, the server generates a vulnerability exploitability conclusion of "confirmed exploitability." If the score is between the low threshold of 0.4 and the high threshold of 0.7, the conclusion is "suspected." If the score is lower than the low threshold of 0.4, the conclusion is "unexploitable." This conclusion will be the final output of the vulnerability verification.

[0094] The following uses the verification process of a time-based blind SQL injection vulnerability as an example to explain steps S201-S2010.

[0095] The server's response analyzer received a request-response result. The vulnerability verification configuration package corresponding to this result specified a time-delay detection strategy, and the attack payload was an SQL query that would delay the database response by 5 seconds. This vulnerability was initially discovered simultaneously by two security detection tools, SAST and DAST, and IAST confirmed the code reachability information. The response analyzer first established a response time baseline, determining the preset response time threshold to be 350 milliseconds. The enhanced executor sent two consecutive attack requests, receiving response times of 5352 milliseconds and 5360 milliseconds respectively. The response analyzer found that both response times far exceeded the threshold, therefore determining the single-item verification result of the time-delay detection strategy as a hit. Subsequently, the server entered the confidence calculation phase. Considering the number of sources from two security detection tools, the true code reachability information, and the hit determination of the time-delay detection strategy, the vulnerability confidence score calculated through weighted average was 0.8. Finally, the server compares the score with the preset confidence thresholds of 0.7 and 0.4. Since 0.8 is greater than 0.7, the server finally generates the vulnerability exploitability conclusion as confirmed exploitability and archives the evidence chain containing records of excessively long response times.

[0096] In one possible implementation, before analyzing the request response results based on the verification strategy to generate a vulnerability exploitability conclusion, the method further includes: sending multiple baseline requests without attack payloads to the target application through the enhanced executor, and measuring the average and standard deviation of the response times; determining a preset response time threshold based on the average and the standard deviation, wherein the preset response time threshold is the average plus the standard deviation by a preset multiple.

[0097] Specifically, when the server determines that a time delay detection strategy needs to be implemented, the server will first start a preprocessing process to establish a response time baseline. The purpose of this process is to provide a dynamic and accurate reference standard for subsequent delay judgment, so as to eliminate interference from non-vulnerability factors such as network environment fluctuations.

[0098] The server instructs the enhanced executor to continuously send multiple baseline requests without attack payloads to the target application. To ensure the validity of the statistical sample, the number of transmissions is set to a preset value, such as at least three. These baseline requests are consistent with subsequent attack requests in all aspects except for the attack payload, such as request method, path, headers, and parameters.

[0099] After each baseline request is sent, the server precisely records the time elapsed from the time the request is sent to the time a complete response is received. After all baseline requests are completed, the server aggregates all recorded response time data and calculates the average and standard deviation of the response times based on this data.

[0100] Subsequently, the server determines the preset response time threshold according to a pre-defined formula. This formula is: the preset response time threshold equals the average value plus the standard deviation by a preset multiple. The preset multiple is a configurable parameter, typically set to 2, to establish a judgment boundary based on the 2σ statistical principle, effectively filtering out most normal network latency fluctuations. The final calculated preset response time threshold will be used in subsequent steps to compare with the actual response time of requests containing attack payloads.

[0101] The following describes the LLM-based multi-tool collaborative vulnerability verification method and apparatus in the embodiments of this invention from the perspective of hardware processing. Please refer to [link to relevant documentation]. Figure 3 This is a schematic diagram of the structure of the multi-tool collaborative vulnerability verification method device based on LLM in the embodiments of this application.

[0102] It should be noted that, Figure 3 The structure of the LLM-based multi-tool collaborative vulnerability verification method apparatus shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0103] like Figure 3 As shown, the LLM-based multi-tool collaborative vulnerability verification method apparatus includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in Read-Only Memory (ROM) 302 or a program loaded from storage portion 308 into Random Access Memory (RAM) 303, such as executing the method described in the above embodiments. The RAM 303 also stores various programs and data required for apparatus operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An Input / Output (I / O) interface 305 is also connected to the bus 304.

[0104] The following components are connected to I / O interface 305: input section 306 including audio input devices, push-button switches, etc.; output section 307 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 308 including a hard disk, etc.; and communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0105] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs the various functions defined in the present invention.

[0106] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0108] Specifically, the LLM-based multi-tool collaborative vulnerability verification method apparatus of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the LLM-based multi-tool collaborative vulnerability verification method provided in the above embodiment.

[0109] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the LLM-based multi-tool collaborative vulnerability verification method apparatus described in the above embodiments; or it may exist independently and not assembled into the LLM-based multi-tool collaborative vulnerability verification method apparatus. The storage medium carries one or more computer programs, which, when executed by a processor of the LLM-based multi-tool collaborative vulnerability verification method apparatus, cause the LLM-based multi-tool collaborative vulnerability verification method apparatus to implement the LLM-based multi-tool collaborative vulnerability verification method provided in the above embodiments.

Claims

1. A multi-tool collaborative vulnerability verification method based on LLM, characterized in that, The method includes: Obtain raw vulnerability data from multiple security detection tools and convert the raw vulnerability data into standard vulnerability data entries; When there are standard vulnerability data entries for the same vulnerability from multiple security detection tools, the standard vulnerability data entries are merged and deduplicated according to preset field priority rules to generate target vulnerability data entries. The target vulnerability data entries include vulnerability type, API entry information, code location information, and code reachability information. Based on a preset LLM model, semantic analysis is performed on the target vulnerability data entries to generate a vulnerability verification configuration package, which includes various attack payloads and verification strategies. The enhanced executor parses the vulnerability verification configuration package and actively sends a verification request to the target application based on the API entry information and the attack payload, and obtains the request response result. The target application is the application to be verified for vulnerability. The request response result includes the HTTP status code, response header, response body and response time. Based on the verification strategy, the request response results are analyzed to generate a conclusion on vulnerability exploitability.

2. The method according to claim 1, characterized in that, The step of performing semantic analysis on the target vulnerability data entries based on a preset LLM model to generate a vulnerability verification configuration package specifically includes: Extract the vulnerability type field from the target vulnerability data entry, and match the corresponding prompt word template from the preset prompt word policy library based on the vulnerability type field. The prompt word template is used to guide the generation of attack payload. Using the vulnerability type, the API entry information, and the code location information as context, fill the prompt word template to form a generation instruction; Input the generation command into the preset LLM model to generate a vulnerability verification configuration package containing various attack payloads and verification strategies.

3. The method according to claim 1, characterized in that, The step of actively sending a verification request to the target application based on the API entry information and the attack payload, and obtaining the request response result, specifically includes: An HTTP request is constructed based on the API entry information, and the attack payload is injected into the HTTP request based on the injection point location included in the API entry information to obtain the attack request; The attack request is sent to the target application, and a complete response is received from the target application, forming the request-response result.

4. The method according to claim 1, characterized in that, The step of analyzing the request-response result based on the verification strategy to generate a vulnerability exploitability conclusion specifically includes: Based on the verification strategy in the vulnerability verification configuration package, perform corresponding strategy analysis on the request response result to obtain the individual verification judgment result of each verification strategy; Based on the individual verification judgment results of each verification strategy, combined with the number of sources of security detection tools reporting the same vulnerability and the code reachability information, the vulnerability confidence score is calculated. Based on the comparison between the vulnerability confidence score and the preset confidence threshold, a conclusion on the vulnerability exploitability is generated.

5. The method according to claim 4, characterized in that, The verification strategy includes an error string matching strategy, a time delay detection strategy, an out-of-band callback verification strategy, and a response content matching strategy. The step involves performing corresponding strategy analysis on the request-response result based on the verification strategy in the vulnerability verification configuration package to obtain the individual verification judgment result for each of the verification strategies, specifically including: When the verification strategy is an error string matching strategy, after filtering out false matches by combining keyword whitelist in the response body of the request response result, it is detected whether a preset error feature string is included. If the preset error feature string is included, the single-item verification result of the verification strategy is determined as a hit; When the verification strategy is a time delay detection strategy, the response time of the request response result is compared with a preset response time threshold. When the response times of any two consecutive responses both exceed the preset response time threshold, the single-item verification result of the time delay detection strategy is determined to be a hit. When the verification strategy is an out-of-band callback verification strategy, determine whether a callback request containing the vulnerability verification configuration package has been received; If a callback request containing the vulnerability verification configuration package is received, the single verification result of the out-of-band callback verification strategy is determined to be a hit. When the verification strategy is a response content matching strategy, it detects whether the response body of the request response result contains expected feature content corresponding to the attack payload. If the expected feature content is detected, the single-item verification result of the content matching strategy is determined to be a hit.

6. The method according to claim 5, characterized in that, Before analyzing the request response result based on the verification strategy and generating a vulnerability exploitability conclusion, the method further includes: The enhanced actuator sends multiple baseline requests without attack payloads to the target application and measures the average and standard deviation of the response time. The preset response time threshold is determined based on the average value and the standard deviation, wherein the preset response time threshold is the average value plus the standard deviation by a preset multiple.

7. The method according to claim 1, characterized in that, Before performing semantic analysis on the target vulnerability data entries based on a preset LLM model to generate a vulnerability verification configuration package, the method further includes: When the API address field of the target vulnerability data entry is incomplete, supplementary API entry information is obtained from multiple supplementary sources according to a preset priority order; When the supplementary API entry information is obtained from any of the supplementary sources, the supplementary API entry information is filled back into the API address field of the target vulnerability data entry, and the status of the target vulnerability data entry is updated from the pending completion status to the verifiable status.

8. A multi-tool collaborative vulnerability verification device based on LLM, characterized in that, The LLM-based multi-tool collaborative vulnerability verification device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the LLM-based multi-tool collaborative vulnerability verification device to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is run on an LLM-based multi-tool collaborative vulnerability verification device, the LLM-based multi-tool collaborative vulnerability verification device performs the method as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program runs on the LLM-based multi-tool collaborative vulnerability verification device, the LLM-based multi-tool collaborative vulnerability verification device performs the method as described in any one of claims 1-7.

Images