A PoC data completion method based on multi-source heterogeneous vulnerability data

Abstract

The present invention discloses a PoC data completion method based on multi-source heterogeneous vulnerability data, and relates to the field of computer technology. Compared with previous PoC data, the present invention solves the problems that public PoC often has low quality, large differences in expression forms, and serious information missing, defines and extracts the key information of PoC, forms a standardized expression method, and provides important guidance for the understanding, analysis and standardized use of PoC. Multi-source heterogeneous PoC information is deeply integrated, and a connection is established between vulnerability information and PoC information, which effectively supplements the missing information of PoC, enhances the understanding of key aspects of PoC information, and significantly improves the practical application value of PoC.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a PoC data completion method based on multi-source heterogeneous vulnerability data. Background Art

[0002] In the security field, vulnerability reports provide key information about security vulnerabilities, helping researchers and practitioners understand and respond to potential threats. In order to more effectively analyze the information in these reports, in recent years, researchers have begun to use natural language processing (NLP) and machine learning techniques to dig deeper into the content of vulnerability reports. These technologies make it possible to automatically extract key information from vulnerability reports, such as the characteristics of the vulnerability, the affected software versions, and the likelihood and time prediction of the vulnerability being exploited. This type of research not only deepens the understanding of the content of vulnerability reports, but also promotes the improvement of information consistency between security databases.

[0003] On this basis, the proof-of-concept (PoC) information of the vulnerability becomes an important supplement to the research. PoC provides actual vulnerability exploitation methods and code examples, which is a key part of verifying and reproducing security vulnerabilities. The existence of PoC greatly enriches the content of vulnerability reports, allowing researchers to understand and reproduce vulnerability scenarios more accurately.

[0004] Existing technologies explore the repeatability of vulnerabilities based on vulnerability reports, using vulnerability PoC to reproduce vulnerabilities and determining important information for reproducing vulnerabilities, such as how the PoC is triggered and the software version affected by the vulnerability. This has promoted the understanding of the information composition of PoC and the research on verifying PoC. PoC is also classified into four types: Shell commands, programs, malicious strings, and malicious files, which reflects the diversity of PoC forms and the difficulty of structured analysis of PoC, which is of great significance for understanding and analyzing the diversity of PoC. PoC is usually a deformed program that combines code with free-format text. The irregularity of the PoC form limits the applicability of existing program analysis technology, which helps to better understand the composition and characteristics of PoC and provides ideas for feature extraction of PoC.

[0005] However, there is relatively little in-depth research on PoC information, and the public PoCs often have problems such as low quality, large differences in presentation forms, and serious information missing. The lack and inconsistency of this information not only greatly reduces the usability and readability of PoC, but also may lead to the inability to accurately reproduce the vulnerability, and thus the inability to verify the effectiveness of PoC.

[0006] In order to solve the above problems, the present invention proposes a PoC data completion method based on multi-source heterogeneous vulnerability data. Summary of the invention

[0007] The purpose of the present invention is to propose a PoC data completion method based on multi-source heterogeneous vulnerability data to solve the problems raised in the background technology:

[0008] Public PoCs often have problems such as low quality, large differences in expression, and serious information missing.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] A PoC data completion method based on multi-source heterogeneous vulnerability data includes the following steps:

[0011] S1: Collect multi-source PoC data and build a dataset for manual analysis, and also collect CVE vulnerability information;

[0012] S2: Extract key aspects of PoC data and vulnerability information based on multi-source PoC data and CVE vulnerability information;

[0013] S3: Perform similarity analysis on PoC data and divide the fusion method of PoC data;

[0014] Before similarity analysis, multi-source PoC data are associated according to their corresponding CVE numbers, and PoC data sets are divided according to different CVE numbers;

[0015] A holistic information completion strategy is adopted for PoC data belonging to different CVE numbers: the complete data of a PoC is used to enrich the PoC information from a single source, so that the data of the PoC from a single source is comprehensive and diverse;

[0016] For PoCs with the same CVE number, select PoCs with the same programming language, compare the abstract syntax tree similarities of these PoC codes, and set a predetermined threshold; if the abstract syntax tree similarity of two code segments exceeds the threshold, compare and analyze the key aspects of their corresponding PoC data, and set a fusion method for the key aspects. As long as there is a difference in the information of a certain key aspect between the two PoCs, this key aspect will be completed; if the similarity does not reach the threshold, it will be included in the overall completion of the PoC information;

[0017] S4: Match and fuse the PoC data with the vulnerability information extracted in S2 according to their respective CVE numbers, formulate corresponding fusion rules, and determine whether to fuse specific field information in the vulnerability information into the PoC based on the fusion rules. If the key information of the PoC lacks a field, and the field exists in the corresponding vulnerability information and is a unique value, the key aspects of the PoC information are supplemented through the CVE information, and the field of this vulnerability information is supplemented into the PoC; for the PoC that meets the information completion rules, the vulnerability information is fused, and finally a PoC data set after multi-source information fusion is obtained.

[0018] Preferably, the manual analysis in S1 defines key aspects of PoC, each key aspect represents a specific information contained in PoC; the key aspects of PoC include trigger mode, test platform, software version, run record, title, author, reference link and time.

[0019] Preferably, the CVE vulnerability information includes vulnerability description and other defined field information, and defines seven key elements of the vulnerability information: product affected by the vulnerability, version, component, type of vulnerability, root cause, attack vector, and final impact.

[0020] Preferably, the BERT model is used in S2 to perform named entity recognition tasks and rule matching methods to extract key information, as follows:

[0021] S2.1: Manually annotate some entities and mark the named entities corresponding to the defined key aspects of PoC;

[0022] S2.2: Pre-training the BERT model using manually annotated data;

[0023] S2.3: Use the trained and optimized BERT model to extract key information such as test platform, software version, title, author, and time from the PoC data;

[0024] S2.4: Based on the observation, rule matching methods are defined for the trigger mode, operation record and reference link respectively, and keyword matching and fixed-format regular expressions are defined for the trigger mode and operation record respectively.

[0025] Preferably, in the vulnerability information extraction step in S2, vulnerability information of seven key elements is extracted from the vulnerability description; relevant information on four aspects of the product, version, component and vulnerability type affected by the vulnerability is extracted based on the named entity recognition model; the root cause, attack vector and final impact information are extracted based on the question-answer model; and the extracted information is merged with the field information defined in the CVE official database to obtain complete vulnerability information.

[0026] Compared with the prior art, the present invention provides a PoC data completion method based on multi-source heterogeneous vulnerability data, which has the following beneficial effects:

[0027] The present invention defines and extracts the key information of PoC, forms a standardized expression method, and provides important guidance for the understanding, analysis and standardized use of PoC. It deeply integrates multi-source heterogeneous PoC information and establishes a connection between vulnerability information and PoC information, effectively supplements the missing information of PoC, enhances the understanding of key aspects of PoC information, and significantly improves the practical application value of PoC. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flow chart of the method mentioned in Example 1 of the present invention. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0030] The present invention defines and extracts the key information of PoC, forms a standardized expression method, and provides important guidance for the understanding, analysis and standardized use of PoC. It deeply integrates multi-source heterogeneous PoC information, and establishes a connection between vulnerability information and PoC information, effectively supplements the missing information of PoC, enhances the understanding of key aspects of PoC information, and significantly improves the practical application value of PoC. Specifically, it includes the following contents.

[0031] Embodiment 1:

[0032] See also Figure 1 The present invention provides a PoC data completion method based on multi-source heterogeneous vulnerability data, comprising the following steps:

[0033] S1: Collect multi-source PoC data and build a dataset for manual analysis, and also collect CVE vulnerability information; the details are as follows:

[0034] We conducted extensive research on public PoC provision platforms, collected PoC data from them through crawling and downloading source code, and built a large-scale PoC dataset for subsequent multi-source fusion. We also obtained vulnerability information from the official CVE, including fields such as vulnerability description and vulnerability title, to facilitate the subsequent connection between vulnerability information and PoC information.

[0035] After collecting multi-source PoC data, the key aspects of PoC were defined by manually analyzing the PoC and combining it with previous research, including triggering method, test platform, software version, running record, title, author, reference link, and time. Each key aspect represents a specific information contained in the PoC.

[0036] We also accessed the official CVE database to obtain detailed vulnerability data, which mainly includes vulnerability descriptions and other defined field information, such as vulnerability type and affected software, etc. Based on this information, we further defined the key elements of vulnerability information, including the product, version, component, type of vulnerability, root cause, attack vector, and final impact affected by the vulnerability.

[0037] S2: Extract key aspects of PoC data and vulnerability information based on multi-source PoC data and CVE vulnerability information; the details are as follows:

[0038] Use a specific method to extract the key aspects of PoC. Specifically, the key aspects of PoC are extracted through the following two steps:

[0039] The BERT model is used to complete the named entity recognition (NER) task to extract key aspects. First, the PoC data is manually annotated, and the named entities corresponding to the defined PoC key aspects are marked in the PoC text. After the annotation is completed, we use these annotated data to train the BERT model so that the model can recognize and extract the key information of the PoC text. The trained and optimized model is then applied to the entire PoC dataset to extract key information about the test platform, software version, title, author, and time of PoC data from various sources.

[0040] In order to further improve the extraction and positioning accuracy of the three key aspects of PoC, namely trigger method, operation record and reference link, a series of keywords and regular expressions are specially designed. For example, the keywords and regular expressions such as "step", "steps", "trigger", "reproduce" and "complie with" are defined:

[0041] (\(\d+\)|\d+\)|\d+\.)' and '(\([a-zA-Z]\)|[a-zA-Z]\)|[a-zA-Z]\.)

[0042] Match the keyword indications and the list of numbers and letters that may appear in the trigger method in the PoC. Reference links are often in the form of URL links, so define link regular rules to extract them. Use these regular expressions to perform accurate pattern matching in all PoC data, thereby fully capturing all key information defined in the PoC.

[0043] Extraction of vulnerability information:

[0044] First, extract relevant information about the product, version, component, type of vulnerability, root cause, attack vector, and final impact from the vulnerability description. For specific information about the four aspects of the vulnerability product, version, component, and type, the named entity recognition (NER) model is used to extract relevant information. For information that appears in free-format phrases such as root cause, attack vector, and impact, the question-answering (QA) model is used to extract information. After that, the extracted information is merged with the field information defined in the CVE official database to obtain complete vulnerability information.

[0045] S3: Perform similarity analysis on PoC data and divide the fusion method of PoC data; the details are as follows:

[0046] CVE number linking. We associate PoC data from multiple sources according to their corresponding CVE numbers, and divide these PoCs into a set of PoCs with the same CVE number and a set of PoCs with different CVE numbers. For PoCs with different CVE numbers, we adopt a holistic information completion strategy, that is, using the complete data of a PoC using the same code language to enrich the PoC information from a single source, making the data of a single-source PoC comprehensive and diverse.

[0047] Define different code language recognition methods based on regular expressions and identify the code languages ​​used by different PoCs, for example, by:

[0048] \b(def|class)\s+\w+\s*\((?:\s*\w+\s*:\s*\w+\s*,?)*\)\s*->?\s*\w+?:|@[\w\.]+|\b(print|exec)\b|\bself\b

[0049] Determine whether a PoC is in Python.

[0050] Code similarity comparison. For PoCs with the same CVE number, we first screen out PoCs with the same programming language, such as Python, Ruby, C, etc. Then, we compare the abstract syntax tree (AST) similarity of these PoC codes and set a predetermined threshold. If the AST similarity of two pieces of code exceeds this threshold, it indicates that there is a possibility of information fusion between the two PoCs. Then, the key aspects of the corresponding PoC data are compared and analyzed, and a fusion method for the key aspects is set. It is believed that as long as there is a difference in the key aspect information of the two PoCs, this key aspect can be completed; for example, if the test platform of one PoC is "Linux" and the test platform of the other PoC is "Windows", it is believed that the two can complement each other; then, we compare the key aspects of the two PoCs extracted in step 1 to check whether there is complementary information that can be fused. For cases where the similarity does not reach the threshold, we include it in the overall completion of the PoC information.

[0051] S4: Match and merge the vulnerability information extracted from the PoC data according to their respective CVE numbers in S2. The details are as follows:

[0052] The PoCs obtained after the above steps are matched with the vulnerability information extracted in step 2 according to their respective CVE numbers. Then compare the key aspects of the PoC with the vulnerability information, and determine whether the specific field information in the vulnerability information can be integrated into the PoC according to the pre-set rules. Specifically, if the key information of the PoC lacks a field, but the field exists in the corresponding vulnerability information and is a unique value, it is considered that the CVE information can supplement the key aspects of the PoC, and the field of this vulnerability information is supplemented into the PoC. For example, if a PoC does not contain test platform information, but its corresponding CVE vulnerability information contains platform information, then the CVE vulnerability information can be used to complete the PoC information. Finally, determine whether to integrate the specific field information in the vulnerability information into the PoC according to the fusion rules. For those PoCs that meet the information completion rules, we fuse the vulnerability information and finally obtain a more comprehensive and accurate PoC data set after multi-source information fusion.

[0053] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A PoC data completion method based on multi-source heterogeneous vulnerability data, characterized in that: The steps include: S1: Collect multi-source PoC data and build a dataset for manual analysis, and also collect CVE vulnerability information; S2: Extract key aspects of PoC data and vulnerability information based on multi-source PoC data and CVE vulnerability information; S3: Perform similarity analysis on PoC data and divide the fusion method of PoC data; Before similarity analysis, multi-source PoC data are associated according to their corresponding CVE numbers, and PoC data sets are divided according to different CVE numbers; A holistic information completion strategy is adopted for PoC data belonging to different CVE numbers: the complete data of a PoC is used to enrich the PoC information from a single source, so that the data of the PoC from a single source is comprehensive and diverse; For PoCs with the same CVE number, select PoCs with the same programming language, compare the abstract syntax tree similarities of these PoC codes, and set a predetermined threshold; if the abstract syntax tree similarity of two code segments exceeds the threshold, compare and analyze the key aspects of their corresponding PoC data, and set a fusion method for the key aspects. As long as there is a difference in the information of a certain key aspect between the two PoCs, this key aspect will be completed; if the similarity does not reach the threshold, it will be included in the overall completion of the PoC information; S4: Match and fuse the PoC data with the vulnerability information extracted in S2 according to their respective CVE numbers, formulate corresponding fusion rules, and determine whether to fuse specific field information in the vulnerability information into the PoC based on the fusion rules. If the key information of the PoC lacks a field, and the field exists in the corresponding vulnerability information and is a unique value, the key aspects of the PoC information are supplemented through the CVE information, and the field of this vulnerability information is supplemented into the PoC; for the PoC that meets the information completion rules, the vulnerability information is fused, and finally a PoC data set after multi-source information fusion is obtained.

2. According to claim 1, a PoC data completion method based on multi-source heterogeneous vulnerability data is characterized in that: The manual analysis in S1 defines key aspects of PoC, each key aspect represents a specific information contained in PoC; the key aspects of PoC include trigger mode, test platform, software version, run record, title, author, reference link and time.

3. According to claim 1, a PoC data completion method based on multi-source heterogeneous vulnerability data is characterized in that: The CVE vulnerability information includes vulnerability description and other defined field information, and defines seven key elements of vulnerability information: product affected by the vulnerability, version, component, type of vulnerability, root cause, attack vector and final impact.

4. According to claim 3, a PoC data completion method based on multi-source heterogeneous vulnerability data is characterized in that: In S2, the BERT model is used to perform named entity recognition tasks and rule matching methods to extract key information, as follows: S2.1: Manually annotate some entities and mark the named entities corresponding to the defined key aspects of PoC; S2.2: Pre-training the BERT model using manually annotated data; S2.3: Use the trained and optimized BERT model to extract key information such as test platform, software version, title, author, and time from the PoC data; S2.4: Based on the observation, rule matching methods are defined for the trigger mode, operation record and reference link respectively, and keyword matching and fixed-format regular expressions are defined for the trigger mode and operation record respectively.

5. According to claim 3, a PoC data completion method based on multi-source heterogeneous vulnerability data is characterized in that: In the vulnerability information extraction step in S2, vulnerability information of seven key elements is extracted from the vulnerability description; relevant information of four aspects, namely, products, versions, components and vulnerability types affected by the vulnerability, is extracted based on the named entity recognition model; root causes, attack vectors and final impact information are extracted based on the question-answering model; and the extracted information is merged with the field information defined in the CVE official database to obtain complete vulnerability information.

Images