A vulnerability entry generation method and device containing vulnerability details, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a vulnerability entry generation method and device containing vulnerability details, electronic equipment and storage medium, to collect high-quality vulnerability library containing vulnerability details. The method comprises: performing feature extraction on original vulnerability information to obtain core attack features contained in the original vulnerability information; filtering hyper text transfer protocol (HTTP) requests in a web application firewall (WAF) traffic log based on the core attack features to obtain at least one candidate vulnerability entry; the candidate vulnerability entry is an HTTP request containing a trigger element meeting the core attack features; for each candidate vulnerability entry, based on the original vulnerability information, if the attack type of the candidate vulnerability entry is the same as that of the original vulnerability information, the candidate vulnerability entry is stored as a target vulnerability entry in the vulnerability library.

Description

Technical Field

[0001] This application relates to the field of cybersecurity technology, and in particular to a method, apparatus, electronic device, and storage medium for generating vulnerability entries that include details of vulnerability exploitation. Background Technology

[0002] In today's digital age, the security of web applications has become a critical issue in the field of cybersecurity. With the widespread deployment and increasing complexity of web applications, various vulnerabilities such as Structured Query Language (SQL) injection, Cross-Site Scripting (XSS), and Cross-Site Request Forgery (CSRF) are frequently emerging. These vulnerabilities not only lead to data breaches and system crashes but can also cause significant economic losses.

[0003] To address such threats, existing vulnerability databases have accumulated a wealth of vulnerability information, providing a reference for security researchers and developers through standardized descriptions and classifications. However, these databases almost never disclose the exploit details of vulnerabilities to prevent malicious attackers from abusing them. Security vendors need to know these exploit details to develop targeted protection solutions; therefore, vulnerability exploit details (Proof of Concept, PoC / Exploit, EXP) have become a scarce resource.

[0004] Therefore, how to collect a high-quality vulnerability database containing details of vulnerability exploitation is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for generating vulnerability entries that include vulnerability exploitation details, in order to collect a high-quality vulnerability database containing vulnerability exploitation details.

[0006] In a first aspect, embodiments of this application provide a method for generating vulnerability entries that include details of vulnerability exploitation, comprising: Feature extraction is performed on the original vulnerability information to obtain the core attack features contained in the original vulnerability information; Based on the core attack characteristics, the Hypertext Transfer Protocol (HTTP) requests in the Web Application Firewall (WAF) traffic logs are filtered to obtain at least one candidate vulnerability entry; the candidate vulnerability entry is an HTTP request containing triggering elements that conform to the core attack characteristics. For each candidate vulnerability entry, if the attack type of the candidate vulnerability entry is the same as that of the original vulnerability information, the candidate vulnerability entry is stored in the vulnerability database as a target vulnerability entry.

[0007] Secondly, embodiments of this application provide a vulnerability entry generation apparatus that includes vulnerability exploitation details, comprising: The feature extraction unit is used to extract features from the original vulnerability information and obtain the core attack features contained in the original vulnerability information. The filtering unit is used to filter the Hypertext Transfer Protocol (HTTP) requests in the Web Application Firewall (WAF) traffic logs based on the core attack characteristics to obtain at least one candidate vulnerability entry; the candidate vulnerability entry is an HTTP request containing triggering elements that conform to the core attack characteristics. The generation unit is configured to, for each candidate vulnerability entry, based on the original vulnerability information, determine that when the attack type of the candidate vulnerability entry is the same as that of the original vulnerability information, store the candidate vulnerability entry as a target vulnerability entry in the vulnerability database.

[0008] In some embodiments, the core attack features include: a path entity and a parameter entity; the path entity represents a path accessible by an HTTP request corresponding to the vulnerability description information; the parameter entity represents the name or field of the parameter being attacked. The feature extraction unit is specifically used for: Based on the security semantic model, non-technical noise and redundant descriptive information in the original vulnerability information are filtered out to obtain vulnerability description information; By using a predefined set of security semantic rules, feature extraction is performed on the vulnerability description information to obtain the path entity and the parameter entity.

[0009] In some embodiments, the filtering unit is specifically used for: Based on the path entity, the HTTP requests in the WAF traffic log are filtered to obtain the HTTP requests in the WAF traffic log that contain the path entity. If the parameter name contained in the HTTP request matches the parameter entity, and the HTTP request also contains a preset special character sequence, then the HTTP request is considered a candidate vulnerability entry.

[0010] In some embodiments, the filtering unit is specifically used for: Using the path entity as a query condition, an inverted index is created on the HTTP requests in the WAF traffic log to obtain a candidate HTTP request set. Obtain the public disclosure time of the original vulnerability information, and trace back a preset time period from the public disclosure time to determine the attack time window; In the candidate HTTP request set, HTTP requests whose timestamp attributes fall within the attack time window are identified, wherein the duration of the attack time window is the preset duration and the end time is the public disclosure time.

[0011] In some embodiments, before the generation unit determines, based on the original vulnerability information, that the candidate vulnerability entry has the same attack type as the original vulnerability information, and stores the candidate vulnerability entry as a target vulnerability entry in the vulnerability database, the apparatus further includes: A matching unit is used to obtain the request payload of the candidate vulnerability entries; The request payload is matched with a predefined set of attack features to determine if the syntax structure of the request payload is consistent with the syntax structure of the malicious attack features contained in the set of attack features. The attack type corresponding to the malicious attack feature is then used as the attack type of the candidate vulnerability entry.

[0012] In some embodiments, the matching unit is further configured to: Extract the constraint entity from the original vulnerability information. The constraint entity represents the pre-triggering conditions required to exploit the vulnerability for an attack. Based on the language model, the session state information and request header features in the candidate vulnerability entries are obtained, and based on the session state information and the request header features, it is determined whether the candidate vulnerability entries contain the pre-triggered conditions. When the attack type of the candidate vulnerability entry is the same as the attack type of the original vulnerability information, and the candidate vulnerability entry contains the pre-triggered condition, the candidate vulnerability entry is stored in the vulnerability database as a target vulnerability entry.

[0013] Thirdly, embodiments of this application provide an electronic device, including: Memory, used to store program instructions; The processor is configured to invoke program instructions stored in the memory and execute the vulnerability entry generation method containing exploit details according to the obtained program instructions.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned vulnerability entry generation method containing vulnerability exploitation details.

[0015] Fifthly, embodiments of this application provide a computer program product, including a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to execute the aforementioned vulnerability entry generation method containing vulnerability exploitation details.

[0016] This application provides a method, apparatus, electronic device, and storage medium for generating vulnerability entries that include vulnerability exploitation details. This application converts the core attack characteristics in the original vulnerability information into retrieval logic, filters vulnerability exploitation details based on the original vulnerability information and massive HTTP requests in WAF traffic logs, maps the exploitation details to the original vulnerability information, and produces a high-quality vulnerability database.

[0017] Many security products require details of vulnerability exploitation to provide better protection for users. The high-quality vulnerability database provided in this application embodiment can provide high-quality corpus for training large security vertical models and increase the dimension of threat intelligence data. Attached Figure Description

[0018] Figure 1 A schematic diagram illustrating an application scenario of a vulnerability entry generation method including vulnerability exploitation details, provided in an embodiment of this application; Figure 2 The diagram shown is a schematic of a high-quality Web vulnerability database generation system provided in an embodiment of this application; Figure 3 A flowchart illustrating a vulnerability entry generation method including vulnerability exploitation details, provided for embodiments of this application; Figure 4 A logic diagram for large language model analysis provided in this application embodiment; Figure 5 A flowchart illustrating another vulnerability entry generation method including vulnerability exploitation details provided for embodiments of this application; Figure 6 A vulnerability entry generation apparatus including vulnerability exploitation details is provided for embodiments of this application; Figure 7 This is a schematic diagram of the structure of an electronic device that includes a vulnerability entry generation method with exploit details, according to an embodiment of this application. Figure 8 This is a schematic diagram of the hardware structure of an electronic device using an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] The following describes some of the concepts involved in the embodiments of this application.

[0021] 1. Vulnerability databases: These are standardized databases that centrally store and manage information on known security vulnerabilities. They record in detail the core attack characteristics of each vulnerability, such as the method of harm (i.e., parameter entity), the scope of impact (i.e., path entity), the severity level (i.e., constraint entity), the remediation plan (patch or mitigation measure), and a unique identification number, such as the Common Vulnerabilities and Exposures (CVE) number. Major vulnerability databases include the officially maintained U.S. National Vulnerability Database (NVD) and various commercial / open-source vulnerability platforms.

[0022] 2. CVE: is a public vulnerability identification system that assigns a unique number to known information security vulnerabilities to facilitate tracking, sharing, and remediation.

[0023] 3. China National Vulnerability Database (CNVD): This is China's national non-profit vulnerability management platform. CNVD-ID serves as the Chinese standard vulnerability number and is linked and complementary to the globally used CVE numbering.

[0024] 4. Web Vulnerability Proof-of-Concept (PoC): A piece of code or HTTP request that proves the existence of a vulnerability without causing substantial damage. For example, creating a file in the / tmp directory of a Linux server using a PoC.

[0025] 5. Web Vulnerability Exploit Code (EXP): This is exploit code that checks for and utilizes vulnerabilities, resulting in substantial impact. For example, executing an EXP can grant a reverse shell connection.

[0026] 6. Security Company Data Warehouse: A data warehouse established by security-related enterprises, or simply a data warehouse, centrally stores and analyzes massive amounts of security data (such as WAF traffic logs, threat intelligence, etc.) to support security operations such as threat detection, incident response, and risk prediction.

[0027] 7. Artificial Intelligence (AI) Agent: An autonomous intelligent system with a large language model as its core brain, capable of actively perceiving the environment (text, speech, images, etc.), calling up tools, and independently completing complex goals like a digital employee by reasoning and planning multi-step tasks.

[0028] 8. Zero-day vulnerability: This refers to a security vulnerability discovered in software or a system but not yet officially patched. Because software vendors or developers do not have "one day" (0day) to release a patch or fix after the vulnerability is disclosed or exploited by attackers, it is called a zero-day vulnerability.

[0029] like Figure 1 The diagram illustrates an application scenario of a vulnerability entry generation method containing vulnerability exploitation details, as provided in an embodiment of this application. The application scenario diagram includes a client 110 and a server 120.

[0030] In one optional implementation, when executed by server 120 alone, server 120 performs feature extraction on the original vulnerability information to obtain the core attack features contained in the original vulnerability information; based on the core attack features, server 120 filters the Hypertext Transfer Protocol (HTTP) requests in the Web Application Firewall (WAF) traffic logs to obtain at least one candidate vulnerability entry; for each candidate vulnerability entry, based on the original vulnerability information, if it is determined that the attack type of the candidate vulnerability entry is the same as that of the original vulnerability information, the candidate vulnerability entry is stored as a target vulnerability entry in the vulnerability database.

[0031] In one alternative implementation, the terminal device 110 and the server 120 can communicate via a communication network.

[0032] In one alternative implementation, the communication network is a wired network or a wireless network.

[0033] In one alternative implementation, such as Figure 2 As shown, the vulnerability entry generation method including vulnerability exploitation details provided in this application embodiment can be applied to a high-quality Web vulnerability database generation system that includes the following inputs and outputs: Figure 2 The high-quality web vulnerability generation system shown takes raw vulnerability information and a data warehouse containing WAF traffic logs as input, and outputs a high-quality web vulnerability database containing vulnerability entries.

[0034] The following is combined Figure 3 This application provides a method for generating vulnerability entries that includes details of vulnerability exploitation.

[0035] Figure 3The flowchart of a vulnerability entry generation method including vulnerability exploitation details provided in the embodiments of this application is shown, and the following iterative steps S301~S303 are executed: S301: Extract features from the original vulnerability information to obtain the core attack features contained in the original vulnerability information.

[0036] The original vulnerability information consists of unstructured or semi-structured text data, which usually only contains theoretical proof of the existence of the vulnerability (e.g., an interface has SQL injection), lacking specific attack paths, injection points, and exploit prerequisites and other executable technical details, and cannot be directly used for vulnerability detection or protection in a production environment.

[0037] In one optional implementation, the original vulnerability information in this application embodiment is the basic vulnerability information (including CVE / CNVD-ID, vulnerability description information, affected components, reference links, etc.) from a public vulnerability database (such as NVD, CNVD).

[0038] The core attack characteristics are a set of structured technical fingerprints that uniquely identify the attack behavior of the vulnerability, obtained through deep semantic analysis and feature extraction of the original vulnerability information. Non-technical noise (such as vendor promotional language and remediation suggestions) has been removed from the original vulnerability information, retaining the most critical features used to match real HTTP requests.

[0039] In one optional implementation, the core attack features in this application embodiment include: path entity and parameter entity.

[0040] The path entity represents the path accessible via HTTP requests corresponding to the vulnerability description information, i.e., the Uniform Resource Identifier (URI) path or filename contained in the original vulnerability information. It also determines the scope of the vulnerability's impact. For example, if the vulnerability description information in the original vulnerability information mentions "the / filesrv / NGInterface / Index interface has a flaw," then that URI path is the path entity.

[0041] A parameter entity represents the name or field of the parameter that is under attack; that is, the name or field of the HTTP parameter that is being attacked, extracted from the vulnerability description information and located at a specific position in the HTTP request. It also represents the manner in which the vulnerability causes harm. For example, if the vulnerability description mentions "inadequate filtering of the guid parameter," then guid is the parameter entity.

[0042] In one alternative implementation, the core attack characteristics also include constraint entities, which are specific conditions that identify the vulnerability triggering, such as "login required", "administrator privileges required", or specific environment configurations.

[0043] In one optional implementation, the core attack characteristics of this application can be obtained in the following manner: Based on the security semantic model, non-technical noise and redundant descriptive information in the original vulnerability information are filtered out to obtain vulnerability description information; through a predefined set of security semantic rules, feature extraction is performed on the vulnerability description information to obtain path entities and parameter entities.

[0044] Among them, security semantic models are algorithmic models specifically designed for understanding and processing textual information in the field of cybersecurity. Examples include Natural Language Processing (NLP) models.

[0045] A security semantic rule set is a collection of rules built upon cybersecurity knowledge to extract structured features from raw vulnerability information. It typically includes pattern matching rules, a lexical analysis dictionary, and contextual logic.

[0046] Specifically, this step focuses on cleaning, deduplicating, semantically normalizing, and extracting core attack features from the original vulnerability information.

[0047] For multiple original vulnerability entries, redundant information is merged using a deduplication algorithm (based on CVE-ID or fingerprint hash). Furthermore, for each original vulnerability entry, a security semantic model (NLP) is used to identify and remove meaningless descriptive statements, retaining the attack type, affected components, and version information.

[0048] Furthermore, this application embodiment applies a set of security semantic rules to perform pattern matching and entity extraction on the vulnerability description information obtained after filtering by the security semantic model, thereby transforming the natural language description into machine-executable core attack features (including path entities, parameter entities, and constraint entities, etc.).

[0049] S302: Based on core attack characteristics, filter the Hypertext Transfer Protocol (HTTP) requests in the Web Application Firewall (WAF) traffic logs to obtain at least one candidate vulnerability entry.

[0050] Among them, candidate vulnerability entries are HTTP requests that contain triggering elements that match the core attack characteristics.

[0051] First, for each original vulnerability information, the path entity, parameter entity, and constraint entity are converted into structured vulnerability units, for example: path=" / api / upload" AND param="filename", and the structured vulnerability units are combined into a vulnerability feature file.

[0052] In one optional implementation, embodiments of this application obtain at least one candidate vulnerability entry through the following steps: A1: Filter the HTTP requests in the WAF traffic log based on the path entity to obtain the HTTP requests in the WAF traffic log that contain the path entity.

[0053] Given the massive scale of WAF traffic log data and the presence of a large amount of historical noise, in order to ensure that the obtained HTTP requests are highly correlated with the original vulnerability information, this application introduces an inverted index mechanism to accelerate retrieval and defines an attack time window based on the public disclosure time of the vulnerability for precise filtering.

[0054] In one optional implementation, the path entity is used as the query condition to perform an inverted index on the HTTP requests in the WAF traffic log to obtain a candidate HTTP request set; the public disclosure time of the original vulnerability information is obtained, and the attack time window is determined by tracing back a preset time from the public disclosure time as the endpoint; in the candidate HTTP request set, the HTTP requests whose timestamp attributes fall within the attack time window are identified.

[0055] The attack window has a preset duration and ends when the attack is publicly disclosed.

[0056] This application's embodiments take into account that it's impossible to perform a full scan of all HTTP requests in WAF traffic logs. By correlating the vulnerability disclosure time with the time distribution of vulnerability exploitation in the wild, a large number of invalid HTTP requests can be filtered out. For example, log information from the 90 days prior to the vulnerability information disclosure (i.e., a preset duration) can be queried to avoid querying a large number of logs and improve efficiency.

[0057] Specifically, by using an inverted index mechanism and taking path entities as the primary filtering condition, the query scope in the data warehouse's WAF traffic logs can be quickly narrowed down to the potential attack time window.

[0058] A2: If the parameter names in the HTTP request match the parameter entities, and the HTTP request also contains a preset special character sequence, then the HTTP request will be considered a candidate vulnerability entry.

[0059] Specifically, for each HTTP request after path entity filtering, the following logical combination is used for fine-grained matching to achieve secondary filtering: B1: Matches whether the parameter names contained in the HTTP request are consistent with the parameter entities.

[0060] B2: Scan the HTTP request payload for special character sequences (such as closure characters ', -- in SQL injection, or pipe characters |, ; in command injection) and compare them with the attack types in the WAF regular expression. That is, if a special character sequence matches a special character sequence in the WAF regular expression, the HTTP request may contain a valid exploit attack; if the special character sequence does not match a special character sequence in the WAF regular expression, the HTTP request is unlikely to contain a valid exploit attack.

[0061] When an HTTP request that meets the requirements of A1 or A2 is found, the complete data packet information of the HTTP request (including Header, Body, Method, etc.) is extracted to form a candidate "PoC / EXP sample", which is used as a candidate vulnerability entry and automatically labeled with the associated CVE-ID.

[0062] S303: For each candidate vulnerability entry, based on the original vulnerability information, if the attack type of the candidate vulnerability entry is the same as that of the original vulnerability information, the candidate vulnerability entry will be stored in the vulnerability database as the target vulnerability entry.

[0063] In this step, the embodiments of this application automatically analyze the retrieved candidate vulnerability entries that are suspected of containing effective exploit attacks, determine whether they do indeed contain effective exploit attacks, and associate them with known original vulnerability information or identify them as potential zero-day attacks.

[0064] In an optional implementation, embodiments of this application can construct the above-mentioned judgment process based on a large language model: Specifically, multi-turn dialogue prompt templates can be preset to guide the large language model in its step-by-step analysis. For example: The first prompt message is: "Please analyze the following HTTP request and determine whether it exhibits characteristics of a malicious attack." If the large language model determines that the HTTP request exhibits malicious attack characteristics, then the HTTP request can be considered to contain a valid exploit attack.

[0065] The second round of warning messages (if the request is deemed to exhibit malicious attack characteristics): "Please determine the specific attack type of this HTTP request (such as SQL injection, XSS, etc.) and assess its potential harm."

[0066] In one alternative implementation, the large language model can determine the attack type in the following manner: Obtain the request payload of the candidate vulnerability entry; match the request payload with a predefined attack feature set, determine that the syntax structure of the request payload is consistent with the syntax structure of the malicious attack features contained in the attack feature set, and take the attack type corresponding to the malicious attack feature as the attack type of the candidate vulnerability entry.

[0067] Specifically, the request payload is used to obtain candidate vulnerability entries; the request payload is the specific value of the parameters carried in the HTTP request, which is the input content that an attacker may use to implement vulnerability exploitation.

[0068] The request payload is matched against a predefined set of attack signatures to determine whether the syntax of the request payload matches the syntax of the malicious attack signatures contained in the attack signature set. The specific matching process includes: First, extract the special control character sequence contained in the request payload, as well as the encoding variants of the above characters (e.g., URL encoding %27, Unicode encoding %u0027, or dual encoding). Secondly, verify whether the overall syntax of the request payload conforms to the attack paradigm of the corresponding attack type.

[0069] Finally, by verifying the context of the candidate vulnerability entries, it was confirmed that the syntax structure of the request payload not only matched in form, but also in substance conformed to the execution logic of the corresponding attack type. That is, the payload could be parsed by the backend service and trigger the expected security flaw.

[0070] When the above matching and verification confirm that the syntax structure of the request payload is consistent with the syntax structure of the malicious attack features contained in the attack feature set, the attack type corresponding to the malicious attack feature is determined as the attack type of the candidate vulnerability entry.

[0071] The third round of prompts reads: "Please compare the above attack type with the vulnerability description information '[Insert vulnerability description information]' to determine whether the two match."

[0072] In one optional implementation, embodiments of this application determine whether a candidate vulnerability entry can be used as a target vulnerability entry in the following manner: Extract constraint entities from the original vulnerability information; based on the language model, obtain session state information and request header features from the candidate vulnerability entries, and determine whether the candidate vulnerability entries contain pre-triggered conditions based on the session state information and request header features; when the attack type of the candidate vulnerability entry is the same as the attack type of the original vulnerability information, and the candidate vulnerability entry contains pre-triggered conditions, store the candidate vulnerability entry as a target vulnerability entry in the vulnerability database.

[0073] Among them, constraint entities represent the pre-triggered conditions required for exploiting vulnerabilities to launch attacks. They can be extracted from the original vulnerability information along with path entities and parameter entities, and serve as core attack features.

[0074] Specifically, based on the trained large language model, deep analysis is performed on candidate vulnerability entries to obtain the session state information and request header features contained therein. The session state information mainly refers to the authentication fields in the HTTP request header, such as the session ID in the Cookie field and the token or JWT token in the Authorization field, used to determine the requester's identity and permissions. Request header features include the User-Agent, Referer, or custom business header fields, used to assist in determining the legitimacy of the request and the operating environment.

[0075] Subsequently, based on the obtained session state information and request header characteristics, it is determined whether the candidate vulnerability entries meet the pre-triggered conditions defined by the constraint entity. Specifically, if the constraint entity is defined as "requires administrator privileges," it verifies whether the Cookie or Authorization field in the session state information contains identification information representing the administrator role; if the constraint entity is defined as "requires a specific request method," it verifies whether the request method field in the request header characteristics is a specified method such as POST or PUT; if the constraint entity is defined as "requires a specific runtime environment," it verifies whether there is a corresponding environment identifier field in the request header characteristics.

[0076] When the attack type of a candidate vulnerability entry is determined to be the same as that of the original vulnerability information, and the candidate vulnerability entry is verified to indeed contain the pre-triggered conditions (i.e., meet the requirements of the constraint entity), it is confirmed that the candidate vulnerability entry is not a random probe or invalid attack by the scanner, but a vulnerability exploitation attack with real exploitability potential. At this time, the candidate vulnerability entry is stored in the vulnerability database as a target vulnerability entry for subsequent generation of protection rules or threat analysis.

[0077] In the three rounds of assessment mentioned above, if the HTTP request does not exhibit malicious attack characteristics in the first round, it is marked as a false alarm and reported to the relevant product line to optimize the protection strategy. If the attack type of the HTTP request does not match the attack type in the vulnerability description information (i.e., the original vulnerability information) in the third round, it is determined to be a potential unknown attack. The HTTP request is then generated as a 0-day vulnerability warning entry containing attack characteristics and marked for review by security experts.

[0078] Only when the attack type of the HTTP request matches the attack type of the vulnerability description information (i.e., the original vulnerability information) in the third round of judgment, the HTTP request or the HTTP request is associated with the vulnerability description information to form a high-quality target vulnerability entry. This target vulnerability entry may also contain a CVE-ID and be stored in the vulnerability database.

[0079] like Figure 4 As shown, it is a logic diagram for large language model analysis provided in an embodiment of this application.

[0080] S401: The large language model receives vulnerability description information and candidate vulnerability entries; S402: The large language model determines whether candidate vulnerability entries exhibit malicious attack characteristics; Specifically, if the attack exhibits malicious characteristics, execute S403; otherwise, execute S404.

[0081] S403: The large language model determines the attack type of candidate vulnerability entries and compares it with the attack type in the vulnerability description information; Specifically, if the attack type of the candidate vulnerability entry matches the attack type in the vulnerability description information, then execute S405; otherwise, execute S406.

[0082] S404: Mark the candidate vulnerability entry as "non-attack"; S405: Store the candidate vulnerability entry as a target vulnerability entry in the vulnerability database. The type of the target vulnerability entry is a known vulnerability. S406: Store the candidate vulnerability entry as a target vulnerability entry in the vulnerability database. The type of the target vulnerability entry is a 0-day vulnerability. S407: Manually determine whether candidate vulnerability entries contain valid exploit attacks; Specifically, if the attack contains a valid exploit, execute S408; otherwise, execute S409.

[0083] S408: Return to the large language model for re-evaluation; S409: Discard the HTTP request. Specific implementation examples: Take "SQL injection vulnerability in XXXX Engineering Enterprise Management Software" as an example.

[0085] Input: The vulnerability description is "The XXXX Engineering Enterprise Management Software / filesrv / NGInterface / Index interface has an SQL injection vulnerability".

[0086] Solution: Extract the core attack characteristics from the vulnerability description information: URL path / filesrv / NGInterface / Index.

[0087] Search: Using this URL path as a condition, a matching HTTP request was found in the data warehouse. The complete request content is as follows: POST / filesrv / NGInterface / Index?_Hash_ID=0858e92eb4b7fc12 HTTP / 1.1 Host:

[0088] Accept: /

[0089] Accept-Encoding: gzip, deflate, br

[0090] Connection: keep-alive

[0091] User-Agent: Mozilla / 5.0 (Windows NT 10.0; Win64; x64) AppleWebKit / 537.36 (KHTML, like Gecko) Chrome / 70.0.3538.77 Safari / 537.36

[0092] Content-Type: application / x-www-form-urlencoded

[0093] Content-Length: 47

[0094] type=getfilelist&guid=';WAITFOR+DELAY+'0:0:4'--

[0095] Input the following into the large language model: Vulnerability description: "The XXXX Engineering Enterprise Management Software / filesrv / NGInterface / Index interface contains an SQL injection vulnerability." HTTP request: POST / filesrv / NGInterface / Index?_Hash_ID=0858e92eb4b7fc12 HTTP / 1.1 Host:

[0096] Accept: /

[0097] Accept-Encoding: gzip, deflate, br

[0098] Connection: keep-alive

[0099] User-Agent: Mozilla / 5.0 (Windows NT 10.0; Win64; x64) AppleWebKit / 537.36 (KHTML, like Gecko) Chrome / 70.0.3538.77 Safari / 537.36

[0100] Content-Type: application / x-www-form-urlencoded

[0101] Content-Length: 47

[0102] type=getfilelist&guid=';WAITFOR+DELAY+'0:0:4'--

[0103] How do you determine if an HTTP request matches the attack described in the vulnerability description?

[0104] The large language model receives the HTTP request and analysis task, and concludes that "yes, the HTTP request matches the attack described in the vulnerability description information".

[0105] In the above embodiments, the present application introduces a large language model to improve efficiency. The large language model automatically judges candidate vulnerability requests containing effective vulnerability exploitation attacks, classifies them, and then checks them against the attack type in the vulnerability description information, which greatly improves efficiency.

[0106] like Figure 5 As shown, it is a logical schematic diagram of a vulnerability database production system provided in an embodiment of this application. Figure 5 Raw vulnerability information can be obtained from CVE, CNVD, vulnerability boxes, open-source software, etc. The raw vulnerability information processing module processes the raw vulnerability information into vulnerability description information and extracts core attack characteristics. The filtering module filters candidate vulnerability entries from each HTTP request in the WAF traffic log based on these core attack characteristics. The large language model analysis module inputs the vulnerability description information and candidate vulnerability entries into a large language model, stores the target vulnerability entries in a high-quality web vulnerability database, and sends suspected false positives to human review. If a false positive is confirmed, the candidate vulnerability entry is stored as a target vulnerability entry in the high-quality web vulnerability database.

[0107] Unlike existing technologies that only process log behavior or organizational characteristics, this application embodiment uses a raw vulnerability information processing module to transform the core attack features in the raw vulnerability information into retrieval logic, thereby achieving a high-precision mapping between vulnerability description information and real traffic PoC.

[0108] Existing technologies are mainly used for attack classification, while the embodiments of this application use an enterprise-level data warehouse as a data source. Through association retrieval and large language model analysis, scarce vulnerability exploitation details (PoC / EXP) are used as production targets, directly solving the industry pain point of missing exploitation details in existing vulnerability databases.

[0109] This application's embodiments introduce a manual review and feedback mechanism, enabling the large language model analysis module to dynamically optimize, overcoming the shortcomings of traditional methods that cannot iterate on their own, and realizing a complete value loop from vulnerability discovery to protection feature development.

[0110] For web vulnerabilities, most existing vulnerability databases only provide simple descriptions, making it almost impossible to deduce the details of the vulnerability. Security vendors need to know the exploit details to develop targeted protection solutions. The purpose of this application is to overcome the shortcomings of the prior art and provide an automated, highly efficient production system capable of automatically mining, analyzing, and associating vulnerability information from an enterprise security data warehouse, ultimately outputting a high-quality web vulnerability database containing accurate exploit details.

[0111] Based on the same inventive concept, this application also provides a vulnerability entry generation apparatus that includes details of vulnerability exploitation, such as... Figure 6 As shown, the vulnerability entry generation device 6000, which includes vulnerability exploitation details, comprises: The feature extraction unit 6001 is used to extract features from the original vulnerability information and obtain the core attack features contained in the original vulnerability information. The filtering unit 6002 is used to filter the Hypertext Transfer Protocol (HTTP) requests in the Web Application Firewall (WAF) traffic logs based on core attack characteristics to obtain at least one candidate vulnerability entry; the candidate vulnerability entry is an HTTP request containing triggering elements that conform to the core attack characteristics. The generation unit 6003 is used to, for each candidate vulnerability entry, determine, based on the original vulnerability information, that when the attack type of the candidate vulnerability entry is the same as that of the original vulnerability information, store the candidate vulnerability entry as a target vulnerability entry in the vulnerability database.

[0112] In some embodiments, the core attack characteristics include: a path entity and a parameter entity; the path entity represents a path that can be accessed by an HTTP request corresponding to the vulnerability description information; the parameter entity represents the name or field of the parameter being attacked. The feature extraction unit 6001 is specifically used for: Based on the security semantic model, non-technical noise and redundant descriptive information in the original vulnerability information are filtered out to obtain vulnerability description information; By using a predefined set of security semantic rules, feature extraction is performed on vulnerability description information to obtain path entities and parameter entities.

[0113] In some embodiments, the filtering unit 6002 is specifically used for: Filter HTTP requests in WAF traffic logs based on path entities to obtain HTTP requests in WAF traffic logs that contain path entities. If the parameter names in the HTTP request match the parameter entities, and the HTTP request also contains a preset special character sequence, then the HTTP request is considered a candidate vulnerability entry.

[0114] In some embodiments, the filtering unit 6002 is specifically used for: Using path entities as query conditions, an inverted index is created on the HTTP requests in the WAF traffic logs to obtain a set of candidate HTTP requests. Obtain the public disclosure time of the original vulnerability information, and trace back a preset time period from the public disclosure time to determine the attack time window; In the candidate HTTP request set, HTTP requests whose timestamp attributes fall within the attack time window are identified. The duration of the attack time window is a preset duration, and the end time is the time of public disclosure.

[0115] In some embodiments, before the generation unit determines, based on the original vulnerability information, that the candidate vulnerability entry has the same attack type as the original vulnerability information, and stores the candidate vulnerability entry as a target vulnerability entry in the vulnerability database, the apparatus further includes: Matching unit 6004 is used to obtain the request payload of candidate vulnerability entries; The request payload is matched with a predefined set of attack features to determine if the syntax structure of the request payload is consistent with the syntax structure of the malicious attack features contained in the attack feature set. The attack type corresponding to the malicious attack feature is then used as the attack type of the candidate vulnerability entry.

[0116] In some embodiments, the matching unit 6004 is further configured to: Extract the constraint entities from the original vulnerability information. The constraint entities represent the pre-triggering conditions required to exploit the vulnerability for an attack. Based on the language model, the session state information and request header features in the candidate vulnerability entries are obtained, and the candidate vulnerability entries are judged to contain pre-triggered conditions based on the session state information and request header features. When the attack type of a candidate vulnerability entry is the same as the attack type of the original vulnerability information, and the candidate vulnerability entry contains a prerequisite triggering condition, the candidate vulnerability entry is stored in the vulnerability database as a target vulnerability entry.

[0117] Based on the same inventive concept, this application also provides an electronic device. In this embodiment, the structure of the electronic device can be as follows: Figure 7 As shown, it includes a memory 701, a communication module 703, and one or more processors 702.

[0118] The memory 701 is used to store computer programs executed by the processor 702. The memory 701 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0119] Memory 701 may be volatile memory, such as random-access memory (RAM); memory 701 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 701 may be any other medium capable of carrying or storing a desired computer program having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 701 may be a combination of the above-mentioned memories.

[0120] Processor 702 may include one or more central processing units (CPUs) or digital processing units, etc. Processor 702 is used to implement the vulnerability entry generation method containing exploit details when calling a computer program stored in memory 701.

[0121] The communication module 703 is used to communicate with terminal devices and other servers.

[0122] This application embodiment does not limit the specific connection medium between the memory 701, communication module 703, and processor 702 described above. This application embodiment... Figure 7 The memory 701 and the processor 702 are connected via a bus 704, and the bus 704 is in Figure 7 The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. The 704 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 7 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0123] The memory 701 stores a computer storage medium containing computer-executable instructions for implementing the vulnerability entry generation method including exploit details according to embodiments of this application. The processor 702 executes the aforementioned vulnerability entry generation method including exploit details. Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium containing computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the vulnerability entry generation methods including exploit details described above. Since the principle by which the aforementioned computer-readable storage medium solves the problem is similar to that of the vulnerability entry generation method including exploit details, the implementation of the aforementioned computer-readable storage medium can refer to the implementation of the method; repeated details will not be elaborated further.

[0124] The following reference Figure 8 To describe a computing device 800 according to this embodiment of the present application. Figure 8 The computing device 800 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0125] like Figure 8 The computing device 800 is manifested in the form of a general-purpose computing device. The components of the computing device 800 may include, but are not limited to: at least one processing unit 801, at least one storage unit 802, and a bus 803 connecting different system components (including storage unit 802 and processing unit 801).

[0126] Bus 803 represents one or more of several bus structures, including a memory bus or memory controller, peripheral bus, processor, or a local bus using any of the various bus structures.

[0127] Storage unit 802 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 821 and / or cache memory 822, and may further include read-only memory (ROM) 823.

[0128] Storage unit 802 may also include a program / utility 825 having a set (at least one) of program modules 824, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0129] The computing device 800 can also communicate with one or more external devices 804 (e.g., keyboard, pointing device, etc.), and with one or more devices that enable a user to interact with the computing device 800, and / or with any device that enables the computing device 800 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 805. Furthermore, the computing device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 806. Figure 8 As shown, network adapter 806 communicates with other modules for computing device 800 via bus 803. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with computing device 800, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0130] This application also provides a computer program product. The methods in this application can be implemented, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in this application are executed, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, network equipment, user equipment, core network equipment, OAM, or other programmable devices.

[0131] A computer-readable storage medium can be an implementation of a computer program product. In other words, this application also provides a computer-readable storage medium that includes a computer program, which, when executed by a processor, implements any of the vulnerability entry generation methods described above that includes exploit details.

[0132] The computer program or instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions may be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; or an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0133] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0135] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0136] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0137] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for generating vulnerability entries including exploit details, characterized in that, The method includes: Feature extraction is performed on the original vulnerability information to obtain the core attack features contained in the original vulnerability information; Based on the core attack characteristics, the Hypertext Transfer Protocol (HTTP) requests in the Web Application Firewall (WAF) traffic logs are filtered to obtain at least one candidate vulnerability entry; the candidate vulnerability entry is an HTTP request containing triggering elements that conform to the core attack characteristics. For each candidate vulnerability entry, if the attack type of the candidate vulnerability entry is the same as that of the original vulnerability information, the candidate vulnerability entry is stored in the vulnerability database as a target vulnerability entry.

2. The method as described in claim 1, characterized in that, The core attack characteristics include: a path entity and a parameter entity; the path entity represents the path that can be accessed by an HTTP request corresponding to the vulnerability description information; the parameter entity represents the name or field of the parameter being attacked. The step of extracting features from the original vulnerability information to obtain the core attack features contained in the original vulnerability information includes: Based on the security semantic model, non-technical noise and redundant descriptive information in the original vulnerability information are filtered out to obtain vulnerability description information; By using a predefined set of security semantic rules, feature extraction is performed on the vulnerability description information to obtain the path entity and the parameter entity.

3. The method as described in claim 1, characterized in that, The process of filtering HTTP requests in WAF traffic logs based on the core attack characteristics yields at least one candidate vulnerability entry, including: Based on the path entity, the HTTP requests in the WAF traffic log are filtered to obtain the HTTP requests in the WAF traffic log that contain the path entity. If the parameter name contained in the HTTP request matches the parameter entity, and the HTTP request also contains a preset special character sequence, then the HTTP request is considered a candidate vulnerability entry.

4. The method as described in claim 3, characterized in that, The step of filtering WAF traffic logs based on the path entity to obtain HTTP requests in the WAF traffic logs that contain the path entity includes: Using the path entity as a query condition, an inverted index is created on the HTTP requests in the WAF traffic log to obtain a candidate HTTP request set. Obtain the public disclosure time of the original vulnerability information, and trace back a preset time period from the public disclosure time to determine the attack time window; In the candidate HTTP request set, HTTP requests whose timestamp attributes fall within the attack time window are identified, wherein the duration of the attack time window is the preset duration and the end time is the public disclosure time.

5. The method as described in claim 1, characterized in that, Before storing the candidate vulnerability entry as a target vulnerability entry in the vulnerability database when the candidate vulnerability entry is determined to have the same attack type as the original vulnerability information based on the original vulnerability information, the process includes: Obtain the request payload for the candidate vulnerability entries; The request payload is matched with a predefined set of attack features to determine if the syntax structure of the request payload is consistent with the syntax structure of the malicious attack features contained in the set of attack features. The attack type corresponding to the malicious attack feature is then used as the attack type of the candidate vulnerability entry.

6. The method as described in claim 5, characterized in that, The step of storing the candidate vulnerability entry as a target vulnerability entry in the vulnerability database when the attack type of the candidate vulnerability entry is the same as the attack type of the original vulnerability information further includes: Extract the constraint entity from the original vulnerability information. The constraint entity represents the pre-triggering conditions required to exploit the vulnerability for an attack. Based on the language model, the session state information and request header features in the candidate vulnerability entries are obtained, and based on the session state information and the request header features, it is determined whether the candidate vulnerability entries contain the pre-triggered conditions. When the attack type of the candidate vulnerability entry is the same as the attack type of the original vulnerability information, and the candidate vulnerability entry contains the pre-triggered condition, the candidate vulnerability entry is stored in the vulnerability database as a target vulnerability entry.

7. A vulnerability entry generation apparatus containing details of vulnerability exploitation, characterized in that, include: The feature extraction unit is used to extract features from the original vulnerability information and obtain the core attack features contained in the original vulnerability information. The filtering unit is used to filter the Hypertext Transfer Protocol (HTTP) requests in the Web Application Firewall (WAF) traffic logs based on the core attack characteristics to obtain at least one candidate vulnerability entry; the candidate vulnerability entry is an HTTP request containing triggering elements that conform to the core attack characteristics. The generation unit is configured to, for each candidate vulnerability entry, based on the original vulnerability information, determine that when the attack type of the candidate vulnerability entry is the same as that of the original vulnerability information, store the candidate vulnerability entry as a target vulnerability entry in the vulnerability database.

8. An electronic device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program stored in a computer-readable storage medium; characterized in that: When the processor of the electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the method of any one of claims 1 to 6.

Images