A web application vulnerability PoC generation method based on a large language model

Abstract

The application discloses a Web application vulnerability PoC generation method based on a large language model, first, a vulnerability benchmark set is constructed, the vulnerability information includes a vulnerability CVE number, a vulnerability description, a CVSS3 score, a patch link, patch modification information corresponding to the patch link, a CWE number and an affected software configuration; for each vulnerability in the constructed vulnerability benchmark set, function-level vulnerability information and navigation information are extracted as input for subsequent PoC generation; the PoC generation process is divided into multiple subtasks; then, based on the divided subtasks, CoT guides LLM to perform step-by-step reasoning, and the content of each subtask is output, and finally, a usable PoC is constructed. The application can automatically identify vulnerabilities from vulnerability descriptions and codes, overcoming the limitations of manual pre-defined rules; at the same time, relying on the code understanding ability of LLM, the application can handle semantic and grammatical constraints, generate effective attack payloads, and finally construct a usable PoC.

Description

Technical Field

[0001] The present invention belongs to the field of software security technology, and in particular relates to a method for generating PoC (PoC) of Web application program vulnerabilities based on a large language model. Background Art

[0002] In software security, vulnerabilities can be exploited to compromise the confidentiality, integrity, and availability of information. Web applications, as the primary vehicle for providing information and services on the modern internet, are often the preferred targets of attackers and are frequently threatened by cross-site scripting (XSS) and SQL injection (SQLi) vulnerabilities. Proof of Concept (PoC) provides a method for demonstrating the feasibility of a specific vulnerability. However, statistics show that of the approximately 250,000 vulnerabilities disclosed by the U.S. National Vulnerability Database (NVD), only 34% have a link to a PoC. In the absence of a PoC, security experts often need to expend significant manpower and time resources manually inferring and re-constructing a PoC, severely hindering the process of mitigating web vulnerabilities.

[0003] Traditional PoC generation for XSS and SQL injection vulnerabilities typically relies on techniques such as static analysis, dynamic analysis, or symbolic execution. For example, static analysis can be combined with dynamic analysis, while attack templates can be defined to automatically generate PoCs for identified vulnerabilities. Application execution paths can be systematically inferred through static analysis, directing the application execution flow to vulnerable sinks and generating PoCs that span multiple HTTP requests. Taint propagation can be performed using symbolic sockets, while constraints can be solved using concrete symbolic execution. Symbolic execution and dynamic taint tracking can be combined to generate PoCs for vulnerabilities using predefined attack payloads. A goal-oriented model checking system can be designed to automatically generate PoCs for XSS and SQL injection vulnerabilities using model checking techniques by accepting vulnerability specifications.

[0004] However, existing PoC generation efforts have numerous limitations. First, they are designed to identify and verify new vulnerabilities at scale through rules, but are unable to handle situations outside the rules, where false negatives are acceptable. For PoC generation tasks targeting specific n-day vulnerabilities, more flexible vulnerability understanding and constraint-solving capabilities are required. Second, traditional PoC generation efforts primarily rely on predefined rules and heuristics to model vulnerabilities, including sources, sinks, sanitization functions, and attack payloads. Consequently, they lack the ability to automatically update vulnerability knowledge and struggle to extract vulnerability information from disclosed vulnerability descriptions and patches. Finally, modern web applications typically have complex page dependencies and intricate data transfer. Traditional PoC generation efforts are insufficiently capable of resolving constraints related to code semantics. When dealing with complex constraints related to program logic and semantics, their performance is poor, and they are unable to generate effective attack payloads.

[0005] In order to overcome the above limitations, the present invention proposes a Web application vulnerability PoC generation method based on a large language model. Summary of the Invention

[0006] To address the limitations of traditional PoC generation, which is limited to specific rule-based vulnerabilities, lacks the ability to automatically update vulnerability knowledge, and lacks the ability to solve semantic constraints, this paper proposes a PoC generation method for Web application vulnerabilities based on a large language model. By leveraging the security domain knowledge embedded in the LLM, this method can automatically identify vulnerabilities from vulnerability descriptions and code, overcoming the limitations of manually predefined rules. Furthermore, relying on the LLM's code understanding capabilities, this method can process semantic and syntactic constraints, generate effective attack payloads, and ultimately construct a usable PoC.

[0007] The present invention is achieved through the following technical solutions:

[0008] A method for generating PoC for web application vulnerabilities based on a large language model, comprising the following steps:

[0009] Step 1: Build a vulnerability benchmark set. Vulnerability information includes the vulnerability CVE number, vulnerability description, CVSS3 score, patch link, patch modification information corresponding to the patch link, CWE number, and affected software configuration.

[0010] Step 2: For each vulnerability in the vulnerability benchmark set constructed in step 1, extract function-level vulnerability information and navigation information as input for subsequent PoC generation;

[0011] Step 3: Split the PoC generation process into multiple subtasks;

[0012] In step 4, based on the subtasks divided in step 3, the Chain of Thought (CoT) is used to guide the LLM to perform step-by-step reasoning and output the content of each subtask.

[0013] In the above technical solution, step 1 includes the following steps:

[0014] Step 1.1: Access NVD to obtain all publicly available vulnerability information, including the CVE number, vulnerability description, CVSS3 score, patch link, CWE number, and affected software configuration. Obtain the corresponding patch modification information through the patch link.

[0015] Step 1.2: Based on the CVE number, filter vulnerabilities with a release date after 2018, a CVSS3 score greater than 7.0, and CWE numbers CWE-79 (XSS) and CWE-89 (SQLi). Then, randomly select a vulnerability from these vulnerabilities and build a local vulnerability environment based on the affected software configuration to reproduce the vulnerability. If the vulnerability can be reproduced successfully, add it to the vulnerability baseline set. Otherwise, discard the vulnerability and re-sample vulnerabilities randomly. Repeat the above process until the number of vulnerability baseline sets reaches the set number.

[0016] In the above technical solution, step 2 includes the following steps:

[0017] Step 2.1, extract vulnerability information

[0018] First, we design a prompt word. For each CVE-numbered vulnerability, the LLM extracts key information from the vulnerability description corresponding to the CVE number, including variables and the file names of the files they are located in. The variables extracted from the vulnerability description are considered potential vulnerability variables. All variables involved in the patch information corresponding to the patch link corresponding to the CVE number are also considered potential vulnerability variables.

[0019] The vulnerable application source code is then parsed into a code property graph, tracking the assignment statements and data dependency statements of the potential vulnerable variables in the propagation path until they reach the sink. The variables in the sink are considered the final vulnerable variables and backward taint analysis is performed until the source point controlled by the attacker is reached. All related functions in the path are extracted. If the code does not exist in a specific function, the entire file content is extracted.

[0020] Step 2.2, extract navigation information

[0021] Starting from the vulnerability information module, we perform backward analysis to locate the first publicly accessible page and extract the navigation information in the path.

[0022] In the above technical solution, in step 3, the PoC generation process is divided into 14 subtasks, including: ① identifying the sink, ② identifying the vulnerable variable, ③ identifying the source, ④ identifying the data flow constraints encountered in the process of propagating the vulnerable variable from the source to the sink, ⑤ identifying the control flow constraints encountered in the process of propagating the vulnerable variable from the source to the sink, ⑥ identifying the grammatical constraints at the sink, ⑦ solving the attack payload that generates the vulnerable variable, ⑧ identifying the file navigation chain, ⑨ identifying the file navigation code, ⑩ identifying the path constraint code, Solution path constraint variables and values, Determine the request parameters required to build PoC, Determine the request method required to build PoC and Determine the request URL required to build the PoC. These subtask contents all require LLM to identify or solve.

[0023] In the above technical solution, the specific process of step 4 is as follows:

[0024] First, identify the sink, vulnerable variables, and source;

[0025] Then, analyze the data flow constraints, control flow constraints, and grammatical constraints at the sink encountered during the propagation of the vulnerability variable from the source to the sink;

[0026] Then, the attack payload of the vulnerability variable is generated based on the above three constraint information;

[0027] Then, the global path of the application execution flow is analyzed to identify the file navigation chain and file navigation code;

[0028] Then, analyze the path constraint code that each file needs to satisfy in order to reach the navigation code, and solve the path constraint variables and values ​​based on the path constraint code;

[0029] Finally, determine the request parameters, request method, and request URL required to build the PoC.

[0030] In the above technical solution, prompt words are designed for each subtask, and the content includes four parts: ① input information, ② subtask explanation, ③ few-sample prompt cases, and ④ subtask output template.

[0031] In the above technical solution, in step 4, an attack payload verifier and an execution trajectory verifier are designed to collect feedback information and combine it with CoT to optimize PoC generation;

[0032] For the attack payload verifier, the identified sink, vulnerable variable, source, data flow constraints, control flow constraints, syntax constraints, and vulnerability information are collected and prompt words are designed to allow the LLM to synthesize a local vulnerability execution environment. At the same time, to enable attack payload debugging, the LLM inserts code that collects the three constraint information into the local execution environment. If the attack payload generation fails, the actual representation of the three constraints is fed back to the LLM, allowing it to regenerate the attack payload.

[0033] For the execution trace verifier, run the generated PoC and obtain the log file of the program execution trace; if the PoC execution fails, extract the function call and file call information in the log file, compare it with the identified file navigation chain nodes, find the navigation file nodes that have not been triggered, and feed it back to the LLM, allowing it to re-solve the path constraint variables and values ​​and rebuild the PoC.

[0034] The advantages and beneficial effects of the present invention are:

[0035] First, the present invention splits the PoC generation subtasks of XSS vulnerabilities and SQL injection vulnerabilities to form a standardized PoC generation method; second, the present invention constructs a PoC generation prompt word system based on LLM, which significantly improves the PoC generation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the present invention for constructing a vulnerability benchmark set.

[0037] Figure 2 It is an overall flow chart of the present invention.

[0038] For ordinary technicians in this field, other relevant drawings can be obtained based on the above drawings without any creative work. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention are further described below with reference to specific embodiments.

[0040] A method for generating PoC for web application vulnerabilities based on a large language model, comprising the following steps:

[0041] Step 1: Build a vulnerability benchmark set.

[0042] Step 1.1: Access the National Vulnerability Database (NVD) to obtain all publicly available vulnerability information, including the vulnerability's CVE number, vulnerability description, CVSS3 score, patch link, CWE number, and affected software configuration. Obtain the corresponding patch modification information through the patch link.

[0043] CVE number: refers to the unique identifier assigned to each vulnerability (such as CVE-2021-44228).

[0044] Vulnerability description: including the vulnerability's impact, attack vector, affected software / hardware, etc.

[0045] CVSS3 score: It is a severity score of the vulnerability, using CVSS (Common Vulnerability Scoring System) to quantify the risk level of the vulnerability.

[0046] CWE numbers stand for Common Weakness Enumeration. This is a community-maintained list used to systematically identify, describe, and categorize security weaknesses (potential vulnerabilities) in software, hardware, and other systems. A CWE number uniquely identifies each security weakness, helping developers, security researchers, and tools to uniformly describe security issues.

[0047] Step 1.2: Based on the CVE number, filter out vulnerabilities with a release date after 2018, a CVSS3 score greater than 7.0, and CWE numbers CWE-79 (XSS) and CWE-89 (SQLi). Then, randomly select a vulnerability from these vulnerabilities and build a local vulnerability environment based on the affected software configuration to reproduce the vulnerability. If the vulnerability can be reproduced successfully, add it to the vulnerability benchmark set. Otherwise, discard the vulnerability and re-sample the vulnerabilities randomly. Repeat the above process until the number of vulnerability benchmark sets reaches the set number (the set number in this embodiment is 100).

[0048] Step 2: For each vulnerability in the vulnerability benchmark set constructed in step 1, extract the function-level vulnerability information and navigation information as the input for subsequent PoC generation.

[0049] Step 2.1, extract vulnerability information.

[0050] First, a prompt word is designed. For each CVE numbered vulnerability prompt, the Large Language Model (LLM) extracts key information from the vulnerability description corresponding to the CVE number, including variables and the file names of the files they are located in. The variables extracted from the vulnerability description are considered potential vulnerability variables, and all variables involved in the patch information corresponding to the patch link corresponding to the CVE number are also considered potential vulnerability variables.

[0051] Then, with the help of tools such as phpjoern, the vulnerable application source code is parsed into a code property graph (CPG) and queried in conjunction with the Neo4j database to track the assignment statements and data dependency statements of the above-mentioned potential vulnerable variables in the propagation path until they reach the sink. The variables in the sink are regarded as the final vulnerable variables, and backward taint analysis is performed until the source point (source) controlled by the attacker is reached. All related functions in the path are extracted. If the code does not exist in a specific function, the contents of the entire file are extracted.

[0052] Step 2.2, extract navigation information.

[0053] Starting from the vulnerability information module, we perform backward analysis to locate the first publicly accessible page and extract the navigation information in the path.

[0054] Specifically, starting from the vulnerability information module, add it to the navigation information and check whether it is publicly accessible. If the conditions are met, directly output the navigation information; otherwise, combine the HTML parser and CPG to locate the code responsible for navigation, such as redirection, form, include statement and function call, etc., identify the files that can reach the vulnerability information module, add them to the navigation information, and continue to check their accessibility; repeat the above process until the first publicly accessible page is identified. During the path tracing process, record all navigation-related functions.

[0055] Step 3: Split the PoC generation process into multiple subtasks.

[0056] PoC generation is a complex task. To improve the accuracy of LLM generation, the complete PoC generation process is split into multiple subtasks, which are solved one by one by LLM and finally assembled into a complete PoC.

[0057] The present invention divides the generation process of PoC into 14 subtasks, including ① identifying sink, ② identifying vulnerability variables, ③ identifying source, ④ identifying data flow constraints encountered during the propagation of vulnerability variables from source to sink, ⑤ identifying control flow constraints encountered during the propagation of vulnerability variables from source to sink, ⑥ identifying syntax constraints at sink, ⑦ solving the attack payload that generates vulnerability variables, ⑧ identifying file navigation chain, ⑨ identifying file navigation code, ⑩ identifying path constraint code, Solution path constraint variables and values, Determine the request parameters required to build PoC, Determine the request method required to build PoC and Determine the request URL required to build the PoC. These subtask contents all require LLM to identify or solve.

[0058] In step 4, based on the subtasks divided in step 3, the Chain of Thought (CoT) is used to guide the LLM to perform step-by-step reasoning and output the content of each subtask.

[0059] The specific process is as follows:

[0060] First, identify the sink, vulnerable variables, and source.

[0061] Then, the data flow constraints, control flow constraints, and grammatical constraints at the sink encountered during the propagation of the vulnerability variable from the source to the sink are analyzed.

[0062] Then, the attack payload of the vulnerability variable is generated based on the above three constraint information.

[0063] Then, the global path of the application execution flow is analyzed, that is, the file navigation chain and file navigation code are identified.

[0064] Then, the path constraint code that each file needs to satisfy in order to reach the navigation code is analyzed. Based on the path constraint code, the path constraint variables and values ​​are solved.

[0065] Finally, determine the request parameters, request method, and request URL required to build the PoC.

[0066] Furthermore, to improve LLM's information recognition and extraction capabilities, we collected examples for each subtask from related work, GitHub, Google, and other sources to create few-shot prompts. We designed prompts for each subtask, each consisting of four parts: ① the input information, ② the subtask explanation, ③ a few-shot prompt example, and ④ a subtask output template.

[0067] Furthermore, in order to mitigate the hallucination phenomenon of LLM, two verifiers are designed: attack payload verifier and execution trace verifier, which are used to collect feedback information and combine it with CoT to optimize PoC generation.

[0068] The attack payload verifier collects identified sinks, vulnerable variables, sources, data flow constraints, control flow constraints, syntax constraints, and vulnerability information. Hints are then designed to allow the LLM to synthesize a local vulnerability execution environment. Furthermore, to enable attack payload debugging, the LLM is required to insert code that collects the three constraint information into the local execution environment. If attack payload generation fails, the actual representation of the three constraints is fed back to the LLM, allowing it to regenerate the attack payload.

[0069] For the execution trace verifier, the generated PoC is run and a log file of the program execution trace is obtained. If the PoC fails, the function call and file call information in the log file is extracted and compared with the identified file navigation chain nodes to identify the untriggered navigation file nodes. This information is fed back to the LLM, which re-solves the path constraint variables and values ​​and rebuilds the PoC. The maximum number of feedbacks for both verifiers is set to 5.

[0070] The above is an exemplary description of the present invention. It should be noted that, without departing from the core of the present invention, any simple deformation, modification or other equivalent replacement that can be made by other skilled in the art without expending creative labor falls within the scope of protection of the present invention.

Claims

1. A method for generating PoC of web application vulnerabilities based on a large language model, characterized in that: The following steps are involved: Step 1: Build a vulnerability benchmark set. Vulnerability information includes the vulnerability CVE number, vulnerability description, CVSS3 score, patch link, patch modification information corresponding to the patch link, CWE number, and affected software configuration. Step 2: For each vulnerability in the vulnerability benchmark set constructed in step 1, extract function-level vulnerability information and navigation information as input for subsequent PoC generation; Step 3: Split the PoC generation process into 14 subtasks, including: ① identifying the sink, ② identifying the vulnerable variable, ③ identifying the source, ④ identifying the data flow constraints encountered during the propagation of the vulnerable variable from the source to the sink, ⑤ identifying the control flow constraints encountered during the propagation of the vulnerable variable from the source to the sink, ⑥ identifying the syntax constraints at the sink, ⑦ solving the attack payload that generates the vulnerable variable, ⑧ identifying the file navigation chain, ⑨ identifying the file navigation code, ⑩ identifying the path constraint code, ⑪ solving the path constraint variables and values, ⑫ determining the request parameters required to build the PoC, ⑬ determining the request method required to build the PoC, and ⑭ determining the request URL required to build the PoC. All of these subtasks require LLM to identify or solve; Step 4: Based on the subtasks divided in step 3, use CoT to guide LLM to perform step-by-step reasoning and output the content of each subtask.

2. The method for generating PoC for a Web application vulnerability based on a large language model according to claim 1, wherein: Step 1 includes the following steps: Step 1.1: Access NVD to obtain all publicly available vulnerability information, including the vulnerability CVE number, vulnerability description, CVSS3 score, patch link, CWE number, and affected software configuration. Obtain the corresponding patch modification information through the patch link. Step 1.2: Based on the CVE number, filter vulnerabilities with a release date after 2018, a CVSS3 score greater than 7.0, and CWE numbers CWE-79 and CWE-89. Then, randomly select a vulnerability from these vulnerabilities and build a local vulnerability environment based on the affected software configuration to reproduce the vulnerability. If the vulnerability can be reproduced successfully, add it to the vulnerability baseline set. Otherwise, discard the vulnerability and re-sample vulnerabilities randomly. Repeat the above process of filtering vulnerabilities and adding them to the vulnerability baseline set until the number of vulnerability baseline sets reaches the set number.

3. The method for generating PoC for a Web application vulnerability based on a large language model according to claim 1, wherein: Step 2 includes the following steps: Step 2.1, extract vulnerability information First, we design a prompt word. For each CVE-numbered vulnerability, the LLM extracts key information from the vulnerability description corresponding to the CVE number, including variables and the file names of the files they are located in. The variables extracted from the vulnerability description are considered potential vulnerability variables. All variables involved in the patch information corresponding to the patch link corresponding to the CVE number are also considered potential vulnerability variables. The vulnerable application source code is then parsed into a code property graph, tracking the assignment statements and data dependency statements of the potential vulnerable variables in the propagation path until they reach the sink. The variables in the sink are considered the final vulnerable variables and backward taint analysis is performed until the source point controlled by the attacker is reached. All related functions in the path are extracted. If the code does not exist in a specific function, the entire file content is extracted. Step 2.2, extract navigation information Starting from the vulnerability information module, we perform backward analysis to locate the first publicly accessible page and extract the navigation information in the path.

4. The method for generating PoC for a Web application vulnerability based on a large language model according to claim 1, wherein: The specific process of step 4 is as follows: First, identify the sink, vulnerable variables, and source; Then, analyze the data flow constraints, control flow constraints, and grammatical constraints at the sink encountered during the propagation of the vulnerability variable from the source to the sink; Then, the attack payload of the vulnerability variable is generated based on the data flow constraints, control flow constraints and grammatical constraints at the sink; Then, the global path of the application execution flow is analyzed to identify the file navigation chain and file navigation code; Then, analyze the path constraint code that each file needs to satisfy in order to reach the navigation code, and solve the path constraint variables and values ​​based on the path constraint code; Finally, determine the request parameters, request method, and request URL required to build the PoC.

5. The method for generating PoC for a Web application vulnerability based on a large language model according to claim 1, wherein: A prompt word is designed for each subtask, which includes four parts: ① input information, ② subtask explanation, ③ few-sample prompt case, and ④ subtask output template.

6. The method for generating PoC for a Web application vulnerability based on a large language model according to claim 1, wherein: In step 4, an attack payload verifier and an execution trace verifier are designed to collect feedback information and combine it with the CoT to optimize PoC generation; For the attack payload verifier, the identified sinks, vulnerable variables, sources, data flow constraints, control flow constraints, syntax constraints, and vulnerability information are collected and prompt words are designed to allow the LLM to synthesize a local vulnerability execution environment. At the same time, to enable debugging of the attack payload, the LLM inserts code that collects data flow constraint, control flow constraint, and syntax constraint information into the local execution environment. If the attack payload generation fails, the actual representation of the data flow constraint, control flow constraint, and syntax constraint is fed back to the LLM, allowing it to regenerate the attack payload. For the execution trace verifier, run the generated PoC and obtain the log file of the program execution trace; If the PoC fails to execute, the function call and file call information in the log file is extracted and compared with the identified file navigation chain nodes to find the navigation file nodes that have not been triggered. The information is fed back to the LLM to let it re-solve the path constraint variables and values ​​and rebuild the PoC.

Images