A vulnerability knowledge graph processing method, device, equipment and medium

Abstract

The application relates to the technical field of information security, in particular to a vulnerability knowledge graph processing method and device, equipment and a medium, which are used for improving the integrity and accuracy of a vulnerability knowledge graph. The method comprises the following steps: acquiring a data set related to a vulnerability; performing key data extraction on the data set; storing the extracted key data in a graph database according to a preset format to generate a vulnerability knowledge graph; the vulnerability knowledge graph is used for indicating the relationship between each first-class entity in the vulnerability field and the attribute value of each first-class entity; performing information completion and / or information correction on the vulnerability knowledge graph according to a pre-constructed vulnerability knowledge graph ontology structure to obtain a checked vulnerability knowledge graph; and the vulnerability knowledge graph ontology structure is used for indicating the relationship between each second-class entity in the vulnerability field and the attribute of each second-class entity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information security, in particular to a vulnerability knowledge graph processing method and device, equipment and medium. BACKGROUND

[0002] Network security problems have always been concerned by security researchers. With the rapid growth of network information, how to obtain high-quality and comprehensive vulnerability data from massive data and effectively manage them is a problem that needs to be solved. However, the existing multiple open source vulnerability databases are maintained by different organizations and stored in different forms, lacking a unified standard. Secondly, multiple open source vulnerability databases have inconsistent information update frequency, inconsistent data fields, and inconsistent data values, which will cause analysts to be unable to intuitively, comprehensively and accurately understand vulnerabilities. Therefore, as the core of network security risk analysis, it is of great significance to collect, integrate and verify vulnerability-related information from multiple sources of massive data and construct a vulnerability knowledge graph.

[0003] At present, vulnerability knowledge graphs are mainly constructed based on multi-source databases such as open source vulnerability libraries (NVD, CNNVD, CNVD), universal platform enumeration libraries (CPE), and universal defect enumeration libraries (CWE). However, there are differences in fields and values in multi-source databases, and the vulnerability knowledge graph constructed directly based on parsed data from multi-source databases may have inconsistent / ambiguous attribute values or relationships, which will mislead analysts. SUMMARY

[0004] The embodiments of the present application provide a vulnerability knowledge graph processing method, device, equipment and medium, which can improve the completeness and accuracy of the vulnerability knowledge graph.

[0005] In a first aspect, the present application provides a vulnerability knowledge graph processing method, comprising:

[0006] obtaining a data set related to vulnerabilities;

[0007] performing key data extraction on the data set, and storing the extracted key data in a graph database according to a preset format to generate a vulnerability knowledge graph; wherein the vulnerability knowledge graph is used to indicate the relationships between various first-class entities in the vulnerability field and the attribute values of the various first-class entities;

[0008] performing information completion and / or information correction on the vulnerability knowledge graph according to a pre-constructed vulnerability knowledge graph ontology structure to obtain a verified vulnerability knowledge graph; wherein the vulnerability knowledge graph ontology structure is used to indicate the relationships between various second-class entities in the vulnerability field and the attributes of the various second-class entities, and the second-class entities are entities including the features of the first-class entities.

[0009] In a possible embodiment, the data set includes structured data and unstructured data.

[0010] In a possible embodiment, key data extraction is performed on the data set, and the extracted key data is stored in the graph database in a preset format, including:

[0011] The unstructured data is input into the trained entity recognition model to obtain a first label and a second label of each word in the unstructured data; wherein the first label is used to indicate that the word is not an entity type, or the word is the beginning of an entity segment, or the word is the middle of an entity segment, or the word is the end of an entity segment, and the second label is used to indicate the entity type corresponding to the word when the word is an entity;

[0012] According to the first label and the second label of each word, a plurality of first words are extracted from the unstructured data and merged to obtain a plurality of second words; wherein the first word is a word with the same second label and the first label being the beginning or the middle or the end of an entity segment, and each second word corresponds to a first-class entity;

[0013] The relationships between the plurality of second words corresponding to the plurality of first-class entities and the attribute values of the plurality of second words corresponding to the plurality of first-class entities are stored in the graph database in a preset format; wherein the relationships between the plurality of second words corresponding to the plurality of first-class entities are determined according to the vulnerability knowledge graph ontology structure, and the attribute values of the plurality of second words corresponding to the plurality of first-class entities are determined according to the plurality of second words.

[0014] In a possible embodiment, according to a pre-constructed vulnerability knowledge graph ontology structure, information completion and / or information correction are performed on the vulnerability knowledge graph to obtain a verified vulnerability knowledge graph, including:

[0015] According to the vulnerability knowledge graph ontology structure, missing information and / or error information of a vulnerability entity in the vulnerability knowledge graph are determined;

[0016] If the missing information is multiple, the information completion is sequentially performed on the vulnerability knowledge graph according to a preset first priority, and / or if the error information is multiple, the information correction is sequentially performed on the vulnerability knowledge graph according to a preset second priority;

[0017] The verified vulnerability knowledge graph is obtained.

[0018] In a possible embodiment, after the verified vulnerability knowledge graph is obtained, the method further includes:

[0019] determine a plurality of similarities between the first vulnerability and the second vulnerability in a plurality of preset dimensions, wherein the first vulnerability and the second vulnerability are any two vulnerability entities in the verified vulnerability knowledge graph;

[0020] weight the plurality of similarities according to weights of the plurality of preset dimensions, and obtain a total similarity between the first vulnerability and the second vulnerability;

[0021] write the total similarity into the verified vulnerability knowledge graph, and obtain an updated vulnerability knowledge graph.

[0022] In a possible embodiment, determining a plurality of similarities between the first vulnerability and the second vulnerability in a plurality of preset dimensions comprises:

[0023] determine a first entity set related to the first vulnerability in any dimension and a second entity set related to the second vulnerability in the any dimension;

[0024] determine a number of entities in an intersection of the first entity set and the second entity set, and a number of entities in a union of the first entity set and the second entity set;

[0025] determine a similarity between the first vulnerability and the second vulnerability in the any dimension according to a ratio of the number of entities in the intersection to the number of entities in the union.

[0026] In a possible embodiment, after writing the total similarity into the verified vulnerability knowledge graph, and obtaining an updated vulnerability knowledge graph, the method further comprises:

[0027] receiving a request for similar vulnerabilities of a target vulnerability;

[0028] determining, according to the updated vulnerability knowledge graph, a plurality of similar vulnerabilities with a total similarity greater than a preset similarity to the target vulnerability;

[0029] displaying the plurality of similar vulnerabilities in an order from high to low according to the total similarity to the target vulnerability.

[0030] In a second aspect, the present application provides a vulnerability knowledge graph processing apparatus, comprising:

[0031] an acquisition module configured to acquire a data set related to vulnerabilities;

[0032] a generation module configured to perform key data extraction on the data set, store the extracted key data in a graph database according to a preset format, and generate a vulnerability knowledge graph; wherein the vulnerability knowledge graph is used to indicate relationships between various first-class entities in a vulnerability field and attribute values of the various first-class entities.

[0033] a verification module configured to perform information completion and / or information correction on the vulnerability knowledge graph according to a pre-constructed vulnerability knowledge graph ontology structure, to obtain a verified vulnerability knowledge graph; wherein the vulnerability knowledge graph ontology structure is used to indicate relationships between various second-class entities in the vulnerability field and attributes of the various second-class entities, and the second-class entities include features of the first-class entities.

[0034] In a possible embodiment, the data set includes structured data and unstructured data.

[0035] In a possible embodiment, the generation module is specifically configured to:

[0036] input the unstructured data into the trained entity recognition model, to obtain a first label and a second label of each word in the unstructured data; wherein the first label is used to indicate that the word is not an entity, or the word is the beginning of an entity fragment, or the word is the middle of an entity fragment, or the word is the end of an entity fragment, and the second label is used to indicate the entity type corresponding to the word when the word is an entity;

[0037] merge a plurality of first words extracted from the unstructured data according to the first label and the second label of each word, to obtain a plurality of second words; wherein the first word is a word with the same second label and the first label being the beginning or the middle or the end of an entity fragment, and each second word corresponds to a first-class entity;

[0038] store relationships between a plurality of first-class entities corresponding to the plurality of second words and attribute values of the plurality of first-class entities corresponding to the plurality of second words in a graph database in a preset format; wherein the relationships between the plurality of first-class entities corresponding to the plurality of second words are determined according to the vulnerability knowledge graph ontology structure, and the attribute values of the plurality of first-class entities corresponding to the plurality of second words are determined according to the plurality of second words.

[0039] In a possible embodiment, the verification module is specifically configured to:

[0040] determine missing information and / or incorrect information of a vulnerability entity in the vulnerability knowledge graph according to the vulnerability knowledge graph ontology structure;

[0041] if the missing information is multiple, perform information completion on the vulnerability knowledge graph in a preset first priority, and / or if the incorrect information is multiple, perform information correction on the vulnerability knowledge graph in a preset second priority;

[0042] obtain the verified vulnerability knowledge graph.

[0043] In a possible embodiment, the apparatus further includes a writing module, configured to:

[0044] After obtaining the checked vulnerability knowledge graph,

[0045] determine a plurality of similarities of the first vulnerability and the second vulnerability in a plurality of preset dimensions; wherein the first vulnerability and the second vulnerability are any two vulnerability entities in the checked vulnerability knowledge graph;

[0046] weight sum the plurality of similarities according to weights of the plurality of preset dimensions to obtain a total similarity of the first vulnerability and the second vulnerability;

[0047] write the total similarity into the checked vulnerability knowledge graph to obtain an updated vulnerability knowledge graph.

[0048] In a possible embodiment, the writing module is specifically configured to:

[0049] determine a first entity set related to the first vulnerability in any dimension and a second entity set related to the second vulnerability in the any dimension;

[0050] determine a number of entities in an intersection of the first entity set and the second entity set and a number of entities in a union of the first entity set and the second entity set;

[0051] determine the similarity of the first vulnerability and the second vulnerability in the any dimension according to a ratio of the number of entities in the intersection to the number of entities in the union.

[0052] In a possible embodiment, the apparatus further includes a querying module, configured to:

[0053] after writing the total similarity into the checked vulnerability knowledge graph to obtain an updated vulnerability knowledge graph, receive a request for similar vulnerabilities of a target vulnerability;

[0054] determine a plurality of similar vulnerabilities with a total similarity greater than a preset similarity to the target vulnerability according to the updated vulnerability knowledge graph;

[0055] display the plurality of similar vulnerabilities in an order from high to low of the total similarity to the target vulnerability.

[0056] In a third aspect, the present application provides an electronic device, including:

[0057] a memory configured to store program instructions;

[0058] A processor configured to invoke program instructions stored in the memory to perform the method of any one of the first aspect.

[0059] In a fourth aspect, the present application provides a computer readable storage medium storing a computer program, the computer program comprising program instructions which, when executed by a computer, cause the computer to perform the method of any one of the first aspect.

[0060] In the embodiments of the present application, key data extraction is performed on the vulnerability-related data set, the extracted key data is stored in the graph database according to a preset format, a vulnerability knowledge graph is generated, the vulnerability knowledge graph is used to represent vulnerability data, vulnerability-related data is standardized and uniformly stored, and an analyst can intuitively understand vulnerabilities, quickly mine deep-level associated information related to vulnerabilities, and comprehensively master multi-dimensional information of vulnerabilities according to the vulnerability knowledge graph. After the vulnerability knowledge graph is generated, information completion and / or information correction are performed on the vulnerability knowledge graph based on a pre-constructed vulnerability knowledge graph ontology structure, a verified vulnerability knowledge graph is obtained, the integrity and accuracy of the verified vulnerability knowledge graph are ensured, and errors in the vulnerability information obtained by the analyst due to incomplete or incorrect information of the vulnerability knowledge graph are avoided, thereby avoiding analysis difficulties. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0062] Figure 1 An application scenario diagram of a vulnerability knowledge graph processing method provided by the embodiments of the present application;

[0063] Figure 2 A flow diagram of a vulnerability knowledge graph processing method provided by the embodiments of the present application;

[0064] Figure 3 A diagram of a vulnerability knowledge graph ontology structure provided by the embodiments of the present application;

[0065] Figure 4 An example diagram of JSON format CVE data provided by the embodiments of the present application;

[0066] Figure 5 A data labeling example diagram provided by the embodiments of the present application;

[0067] Figure 6A correlation diagram of the first vulnerability and the second vulnerability provided for the embodiment of the present application is shown in FIG. 1.

[0068] Figure 7 A total similarity diagram between two vulnerabilities provided for the embodiment of the present application is shown in FIG. 2.

[0069] Figure 8 A structure diagram of a vulnerability knowledge graph processing device provided for the embodiment of the present application is shown in FIG. 3.

[0070] Figure 9 A structure diagram of an electronic device provided for the embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0071] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application. In the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other at will. And, although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0072] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the term "comprises" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0073] In the embodiments of the present application, "a plurality of" can mean at least two, for example, two, three or more, and the embodiments of the present application do not limit.

[0074] Before introducing the vulnerability knowledge graph processing method provided by the embodiments of the present application, in order to facilitate understanding, first, some English-Chinese translation relationships involved in the embodiments of the present application are described as shown in Table 1 below:

[0075] Table 1

[0076]

[0077]

[0078] The following explains some terms related to the embodiments of the present application:

[0079] 1. Named Entity Recognition: refers to identifying entities with specific meanings in text, including vulnerability numbers, vulnerability names, affected products and versions, causes of vulnerabilities, and impacts of vulnerabilities in the vulnerability field.

[0080] 2. Unstructured data: data that is not convenient to express with a two-dimensional logical table of a database is called unstructured data, including all formats of text, pictures, XML, HTML, various reports, images, and audio / video information, etc. A text file as a data form is unstructured data.

[0081] 3. Knowledge graph: a knowledge system formed by structuring human knowledge.

[0082] 4. Vulnerability knowledge graph: an application of knowledge graph in the vulnerability field, including vulnerability knowledge ontology structure based on ontology, and processing, processing, and integrating multi-source heterogeneous vulnerability-related information through natural language processing to transform into a structured vulnerability field knowledge base.

[0083] A complete and accurate vulnerability knowledge graph is not only beneficial for analysts to deeply understand vulnerability details and grasp the types and development trends of vulnerabilities as a whole, but also helps users to self-check their information asset environment and reduce threat risks. In order to improve the completeness and accuracy of the vulnerability knowledge graph, the embodiments of the present application provide a vulnerability knowledge graph processing method, which can be executed by an electronic device.

[0084] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the following application scenarios are only used to illustrate the embodiments of the present application and are not limiting. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0085] Please refer to Figure 1 The application scenario of the vulnerability knowledge graph processing method provided by the embodiments of the present application is shown in the figure, which includes a vulnerability-related data set 101 and an electronic device 102.

[0086] The electronic device 102 can be implemented by a terminal or a server. The terminal can be, for example, a mobile terminal, a fixed terminal, or a portable terminal such as a mobile phone, a multimedia computer, a multimedia tablet, a desktop computer, a notebook computer, a tablet computer, and the like. The server can be a stand-alone physical server or a virtual server, or a server cluster composed of multiple physical servers or a distributed system, or a cloud server providing cloud services, cloud databases, cloud computing, cloud storage, network services, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms, but is not limited thereto.

[0087] Specifically, after the electronic device 102 obtains the vulnerability-related data set 101, the electronic device 102 processes the vulnerability-related data set 101 to generate a vulnerability knowledge graph. Details of how the electronic device 102 generates the vulnerability knowledge graph will be described below.

[0088] As described above, the application scenario is described below with reference to the electronic device 102 in the vulnerability knowledge graph processing method in the vulnerability knowledge graph processing method. Figure 1 Figure 2 The vulnerability knowledge graph processing method provided in the embodiment of the present application is described below with reference to a flowchart of the vulnerability knowledge graph processing method.

[0089] S201, obtaining a vulnerability-related data set.

[0090] In one possible embodiment, the vulnerability-related data set is open source structured data. The open source structured data refers to a database related to vulnerabilities publicly available on the Internet, such as vulnerability databases (NVD, CNNVD, NVD), universal platform enumeration databases (CPE), universal vulnerability databases (CWE), and the like.

[0091] Considering that the data granularity of the structured data is coarse, it is difficult for analysts to understand the location of the vulnerability, the cause of the vulnerability, the impact of the vulnerability, and the like. Therefore, in one possible embodiment, the vulnerability-related data set includes structured data and unstructured data. In this embodiment, the unstructured data mainly represents text data related to vulnerabilities, such as vulnerability descriptions, vulnerability reports, and the like. Therefore, the unstructured text data will be used as an example to describe the embodiments of the present application in the subsequent description. The meaning of the structured data is described above and will not be repeated here.

[0092] In the embodiments of the present application, the coarse-grained open source structured data and the fine-grained unstructured text data are combined for data extraction to generate a vulnerability knowledge graph with more diverse data sources, and the vulnerability information of the vulnerability knowledge graph obtained finally is more complete.

[0093] ​As introduced above the specific content of the vulnerability-related dataset, the following describes several ways that the electronic device can perform when obtaining the vulnerability-related dataset.

[0094] The first way, after the other collection device collects the vulnerability-related dataset, it is sent to the electronic device.

[0095] Specifically, the other collection device collects various structured data and unstructured text data and sends it to the electronic device.

[0096] The second way, the electronic device downloads the vulnerability-related dataset from network resources.

[0097] Specifically, the electronic device downloads various structured data and unstructured text data from network resources.

[0098] The third way, the other collection device collects part of the vulnerability-related dataset and sends it to the electronic device, and the electronic device downloads another part of the vulnerability-related dataset from network resources.

[0099] Specifically, the other collection device collects unstructured text data and sends it to the electronic device, and the electronic device downloads structured data from network resources.

[0100] S202, key data extraction is performed on the dataset, and the extracted key data is stored in the graph database according to a preset format to generate a vulnerability knowledge graph.

[0101] After the electronic device obtains the vulnerability-related dataset, it can design a structured and applicable vulnerability knowledge graph ontology structure according to the actual business requirements such as scenario target, data storage granularity, etc., based on the pre-constructed vulnerability knowledge graph ontology structure, key data extraction is performed on the dataset, and the extracted key data is stored in the graph database according to a preset format to generate a vulnerability knowledge graph.

[0102] Among them, the graph database is a storage medium, which is the underlying storage of the vulnerability knowledge graph, such as Neo4j, Hugegraph, TigerGraph, etc. The embodiments of the present application do not limit it. The vulnerability knowledge graph is used to indicate the relationship between multiple first entities in the vulnerability field and the attribute values of the multiple first entities. The nodes in the vulnerability knowledge graph represent the first entities, including vulnerability entities and attribute values related to vulnerabilities, and the edges in the vulnerability knowledge graph represent the relationship between two entities.

[0103] The vulnerability knowledge graph ontology structure is used to indicate the relationship between each second-class entity in the vulnerability field and the attributes of each second-class entity. The second-class entity is an entity including the characteristics of the first-class entity, in other words, the second-class entity includes several attributes which are the same as the attributes of the first-class entity. Moreover, the second-class entity is an entity type to which at least one first-class entity belongs. For example, a certain second-class entity is CVE, and the corresponding at least one first-class entity includes vulnerability 1, vulnerability 2, vulnerability 3, etc., and the entity types of vulnerability 1, vulnerability 2, and vulnerability 3 are all CVE types.

[0104] In a possible embodiment, the vulnerability knowledge graph ontology structure takes the CVE entity as the center and is used to indicate the relationship between other second-class entities and the CVE entity. Please refer to Figure 3 , which is a schematic diagram of a vulnerability knowledge graph ontology structure provided by an embodiment of the present application. The following will be introduced in combination with Figure 3 to introduce the vulnerability knowledge graph ontology structure.

[0105] Among them, the circle node represents the second-class entity, and the vulnerability knowledge graph ontology structure takes the CVE entity as the core and expands outwardly, including coarse-grained entities and fine-grained entities. The coarse-grained entities include other vulnerability information entities (such as CNNVD, CNVD), vulnerability exploitation information entities (such as vulnerability verification code, vulnerability exploitation code, etc.), software platform information entities (such as CPE), vulnerability defect information entities (such as CWE), solution information entities (such as mitigation measures), etc. The fine-grained entities include fine-grained vulnerability impact product entities (such as products), vulnerability cause entities (such as reasons), vulnerability impact entities (such as impacts), user type entities of exploiting vulnerabilities (such as user types), and user exploiting vulnerability method entities (such as exploiting vulnerability methods), etc.

[0106] Figure 3 The edges between the circle nodes in the middle represent the relationship between each second-class entity, for example, the relationship between CNNVD, CNVD, CWE and CVE is “related”, the relationship between EDB and CVE is “POC”, the relationship between mitigation measures and CVE is “solution”, the relationship between vulnerability code and CVE is “code”, the relationship between vulnerability type and CVE is “type”, the relationship between impact and CVE is “cause”, the relationship between exploiting vulnerability method and CVE is “exploiting method”, the relationship between user type and CVE is “exploiter type”, the relationship between reason and CVE is “cause”, the relationship between product and CVE is “impact product”, and the relationship between CPE and CVE is “impact asset”.

[0107] It should be noted that each second-type entity can also have corresponding attributes, for example, the CVE entity has attributes such as vulnerability ID, vulnerability name, vulnerability description, vulnerability release time, vulnerability modification time, vulnerability risk level, etc., and the product entity has attributes such as product unique identifier, product manufacturer, product name, product version, etc. Figure 3 The attributes of each second-type entity are not shown.

[0108] Further, the electronic device extracts different types of data sets based on the vulnerability knowledge graph ontology structure, which will be introduced in the following examples.

[0109] The first case is for structured data in the data set.

[0110] The electronic device determines the target parser according to the format of the data file in the structured data, extracts the corresponding fields and field values from the structured data according to the vulnerability knowledge graph ontology structure using the target parser, and stores them in the form of <entity, entity type, attribute, attribute value> and <entity, entity type, relationship, entity, entity type> in the graph database.

[0111] Specifically, the electronic device first analyzes the structure of the data file in the data set to determine the format of the data file, such as JavaScript Object Notation (JSON), Extensible Markup Language (XML), etc. If it is a JSON file, the key in the JSON file is matched with the entity and attribute in the vulnerability knowledge graph ontology structure, and if it is an XML file, the element tag in the XML file is matched with the entity and attribute in the vulnerability knowledge graph ontology structure, and then a data extraction mapping table is developed. Then the electronic device extracts the corresponding entity and attribute value from the JSON, XML, etc. data file according to the data extraction mapping table, and represents it in the form of <entity, entity type, attribute, attribute value>. Finally, the electronic device determines the relationship between entities based on the obtained entity and the vulnerability knowledge graph ontology structure, and represents it in the form of <entity, entity type, relationship, entity, entity type>.

[0112] Please refer to Figure 4, which is an example diagram of CVE data in JSON format provided by the embodiments of this application. It can be seen that the field "data_type" corresponds to the entity type of CVE, the field "CVE_data_meta / ID" corresponds to the vulnerability ID of CVE, the field "description / value" corresponds to the vulnerability description of CVE, the field "publishedDate" corresponds to the vulnerability release time of CVE, the field "lastModifiedData" corresponds to the vulnerability modification time of CVE, the field "problemtype / value" corresponds to the ID of CWE, and the field "cpe_match / cpe23Uri" corresponds to the ID of CPE.

[0113] According to Figure 4 the CVE data shown in the table, the data extraction mapping table is obtained as shown in Table 2.

[0114] Table 2

[0115] Source file field Entity Entity attribute data_type CVE CVE entity type CVE_data_meta / ID CVE CVE vulnerability ID description / value CVE CVE vulnerability description publishedDate CVE CVE vulnerability publication time lastModifiedData CVE CVE vulnerability modification time problemtype / value CWE CWE ID cpe_match / cpe23Uri CPE CPE ID

[0116] Three entities (CVE, CWE, CPE) and their entity attributes can be obtained from Table 2. For example: the CVE entity instance <CVE-2022-29624, cve, release time, 2022-06-02T14:15Z>. Then according to Figure 3 the ontology structure of the vulnerability knowledge graph shown, there are relationships <CVE, cve, affects assets, CPE, cpe> and <CVE, cve, related to, CWE, cwe>. The relationship instances <CVE-2022-29624, cve, affects assets, cpe:2.3:a:tpcms_project:tpcms:3.2:*:*:*:*:*:*:*, cpe> and <CVE-2022-29624, cve, related to, CWE-434, cwe> can be obtained. Finally, the entities and relationship instances are stored in the graph database.

[0117] The second case is for unstructured data in the dataset.

[0118] In a possible embodiment, the electronic device can perform entity recognition and extraction on the unstructured text data that this application embodiment focuses on through an entity recognition model, and store the extracted data in the graph database in the form of <entity, entity type, attribute, attribute value> and <entity, entity type, relationship, entity, entity type>. The specific steps are as described in S1.1 - S1.3.

[0119] S1.1: Input the unstructured text data into the trained entity recognition model to obtain the first label and the second label of each word in the unstructured text data.

[0120] The trained entity recognition model is obtained by machine learning training based on a text data set with entity annotation, for example, CRF, BiLSTM+CRF, BERT+CRF, and the like, and the present application does not limit this. The text data set with entity annotation can be downloaded from the Internet or manually annotated. For example, unstructured text data is collected for data cleaning and sentence processing, and each word in the text data is annotated by a person according to the defined entity type using BIO or BIOE mode to obtain a text data set with entity annotation.

[0121] The electronic device inputs the unstructured text data into the trained entity recognition model to obtain a first label and a second label of each word in the unstructured text data. The first label is used to indicate that the word is not an entity, or the word is the beginning of an entity segment, or the word is the middle of an entity segment, or the word is the end of an entity segment, wherein the beginning of the entity segment indicates that the word is the first word of the entity, the middle of the entity segment indicates that the word is the middle character of the entity, and the end of the entity segment indicates that the word is the last character of the entity. For example, the first label O indicates that the word is not an entity, the first label B indicates that the word is the beginning of an entity segment, i.e., the first word of the entity, the first label I indicates that the word is the middle of an entity segment, i.e., the middle character of the entity, and the first label E indicates that the word is the end of an entity segment, i.e., the last character of the entity. The second label is used to indicate the entity type corresponding to the word when the word is an entity, for example, the second label VT indicates that the word is a vulnerability type entity, and the second label IM indicates that the word is an impact type entity.

[0122] Please refer to Figure 5 , which is a data annotation example provided by the present application. Wherein O represents non-entity, i.e., not an entity, B-X represents the beginning of an X type entity segment, i.e., the first word of the X type entity, and I-X represents the middle of an X type entity segment, i.e., the middle character of the X type entity. It should be noted that although the first label E is not embodied in the example of Figure 5 , in actual use, E-X can exist to indicate the end of an X type entity segment, i.e., the end character of the X type entity. X represents the entity type (for example, VT represents a vulnerability type entity, and IM represents an impact type entity).

[0123] For example, the first label and the second label of the word "CVE-2022-29624" are "B-CVE", indicating that "CVE-2022-29624" is the beginning of the CVE type entity fragment. The first label of "An", "in", "the", "to", "via" is "O", indicating that these words are not entities. The first label and the second label of "arbitrary" are "B-VT", indicating that "arbitrary" is the beginning of the vulnerability type entity fragment. The first label and the second label of "file", "upload", "vulnerability" are "I-VT", indicating that "file", "upload", "vulnerability" are the middle of the vulnerability type entity fragment. The first label and the second label of "execute" are "B-IM", indicating that "execute" is the beginning of the impact type entity fragment. The first label and the second label of "arbitrary", "code" are "I-IM", indicating that "arbitrary", "code" are the middle of the impact type entity fragment.

[0124] S1.2, according to the first label and the second label of each word, merging the plurality of first words extracted from the unstructured text data to obtain a plurality of second words.

[0125] Wherein, the first word is a word whose first label is the beginning of an entity fragment or the middle of an entity fragment or the end of an entity fragment, and the second label is the same, each second word corresponds to a first-class entity. For example, the words corresponding to B-X and I-X are extracted and merged to obtain a plurality of second words.

[0126] Continuing with the example of Figure 5 , the words corresponding to B-CVE are extracted and merged to obtain "CVE-2022-29624", according to the second label CVE, "CVE-2022-29624" is determined to be a CVE entity. The words corresponding to B-VT and I-VT are extracted and merged to obtain "arbitrary file upload vulnerability", according to the second label VT, "arbitrary file upload vulnerability" is determined to be a vulnerability type entity. The words corresponding to B-IM and I-IM are extracted and merged to obtain "execute arbitrary code", according to the second label IM, "execute arbitrary code" is determined to be an impact type entity.

[0127] S1.3, the relationship between the plurality of first-class entities corresponding to the plurality of second words, and the attribute values of the plurality of first-class entities are stored in the graph database according to a preset format.

[0128] The relationship between the plurality of first-class entities is determined according to a plurality of second words and a vulnerability knowledge graph ontology structure, and the attribute value of the plurality of first-class entities is determined according to the plurality of second words.

[0129] Continuing to take Figure 5 For example, the electronic device combines the words corresponding to B-X and I-X, and obtains 3 entities, the CVE entity "CVE-2022-29624", the vulnerability type entity "arbitrary file upload vulnerability", and the impact entity "execute arbitrary code", which are represented in the format of <entity, entity type, attribute, attribute value> as <CVE-2022-29624, cve, vulnerability ID, CVE-2022-29624>, <VT-001, vulnerability type, name, arbitrary file upload vulnerability>, and <IM-001, impact, name, execute arbitrary code>, respectively, where VT-001 is the unique ID of the vulnerability entity type instance, and IM-001 is the unique ID of the impact entity instance. According to the relationships <CVE, cve, type, vulnerability type, vulnerability type> and <CVE, cve, produce, impact, impact> existing in the vulnerability knowledge graph ontology structure, the relationship instances <CVE-2022-29624, cve, type, VT-001, vulnerability type> and <CVE-2022-29624, cve, produce, IM-001, impact> can be obtained. Finally, the entity instances and the relationship instances are stored in the graph database.

[0130] In the embodiments of the present application, the entity recognition model, i.e., the natural language processing technology, is used to identify and extract fine-grained vulnerability information from unstructured text data, so as to construct a vulnerability knowledge graph with richer data dimensions.

[0131] Considering that the entity recognition model may not be able to determine the type and impact of the vulnerability when extracting information such as the name of the software affected by the vulnerability, the version of the software, and the type of the vulnerability exploiter from unstructured text data, in a possible embodiment, the electronic device can pre-build a trained classification model, perform category prediction on the unstructured text data through the trained classification model, and store the predicted category results in the graph database in the form of <entity, entity type, attribute, attribute value> and <entity, entity type, relationship, entity, entity type>.

[0132] The trained classification model is obtained by machine learning training based on a text set with vulnerability type, impact, and other annotations, such as a Naive Bayes, support vector machine, eXtreme Gradient Boosting (XGBoost), and other classification models, and embodiments of the present application do not make specific limitations. The text set with category annotations can be downloaded from the Internet or manually annotated. For example, unstructured text data is collected for data cleaning, and the text set is manually annotated according to the classification target to obtain a text set with category annotations.

[0133] Specifically, the electronic device performs sentence processing on the unstructured text data to obtain multiple sentence text data, inputs each sentence text data into the trained classification model to obtain multiple category results of the multiple sentence text data, each category result corresponds to a first-class entity, and stores relationships between multiple first-class entities corresponding to the multiple category results and attribute values of the multiple first-class entities corresponding to the multiple category results in the graph database according to a preset format. Wherein, the relationships between the multiple first-class entities corresponding to the multiple category results are determined according to the multiple category results and the ontology structure of the vulnerability knowledge graph, and the attribute values of the multiple first-class entities corresponding to the multiple category results are determined according to the multiple category results.

[0134] It should be noted that the electronic device can define multiple trained classification models according to the demand, and input each sentence text data into the multiple trained classification models to obtain multiple category results of each sentence text data. Wherein, the multiple trained classification models are, for example, a vulnerability type classifier, a vulnerability impact classifier, and the like, a certain sentence text data is input into the vulnerability type classifier to obtain a vulnerability type, such as cross-site attack, injection, and the like, and the sentence text data is input into the vulnerability impact classifier to obtain an impact type caused by the vulnerability, such as code execution, data leakage, and the like.

[0135] For example, taking the description text of CVE-2022-29624 as an example, “An arbitrary file upload vulnerability in the Add File function of TPCMS v3.2 allows attackers to execute arbitrary code via a crafted PHP file”, inputting it into the vulnerability impact classifier, the output type is “execute arbitrary code”, and the impact entity instance is <IM-002, impact, name, execute arbitrary code>. According to the ontology structure, the CVE vulnerability and the vulnerability impact have a “produces” relationship, so there is a relationship instance <CVE-2022-29624, cve, produces, IM-002, impact>.

[0136] In the embodiments of the present application, the entity recognition model is used to extract the software name, software version, and vulnerability exploiter type information affected by the vulnerability in the unstructured text data, and the classification model is used to extract the type and impact of the vulnerability in the unstructured text data. Various vulnerability information is extracted from the unstructured text data.

[0137] Considering that in the data extraction tasks of structured data and unstructured text data, the extracted entities may have the same meaning but different representations, or the same meaning but different data formats, in order to avoid data redundancy and non-uniform data formats, in a possible embodiment, the electronic device can perform normalization processing on the extracted key data, convert the extracted key data into standard form or standard format data, and then store it in the graph database.

[0138] Specifically, entities with the same meaning but different representations are unified into a standard form, and entities with the same meaning but different data formats are unified into a standard format. For example, “execute arbitrary code” and “arbitrary code execution” both represent “execute arbitrary code”, and they are unified into the standard form “execute arbitrary code”. Or for example, there are three different date formats in the data: “2022-06-02T14:15Z”, “2022-06-02 14:15:00”, and “2022 / 06 / 02 14:15:00”. According to the standard format “yyyy-mm-ddThh:mm:ssZ”, the above three dates can be represented as “2022-06-02T14:15:00Z”.

[0139] In the embodiments of the present application, data value standardization and data format unification can not only alleviate the data redundancy problem in the construction process of the vulnerability knowledge graph, but also facilitate subsequent graph data verification work.

[0140] In S203, according to the pre-constructed vulnerability knowledge graph ontology structure, information completion and / or information correction are performed on the vulnerability knowledge graph to obtain a verified vulnerability knowledge graph.

[0141] In order to ensure the integrity and accuracy of the vulnerability knowledge graph, the electronic device can verify the generated vulnerability knowledge graph according to the pre-constructed vulnerability knowledge graph ontology structure. There are various verification methods, which will be introduced below.

[0142] Case one, information completion of the vulnerability knowledge graph.

[0143] According to the vulnerability knowledge graph ontology structure, the electronic device can determine the key information of the vulnerability. For each entity in the generated vulnerability knowledge graph, the associated information of each entity is obtained, including the attribute value and the relationship of each entity, and the associated information is compared with the corresponding attribute and relationship in the vulnerability knowledge graph ontology structure, so as to determine whether each entity has missing attribute value or relationship. If any entity has missing attribute value or relationship, the missing attribute value or relationship of the any entity is determined as the missing information of the vulnerability knowledge graph, and the information of the vulnerability knowledge graph is completed to perfect the graph data.

[0144] For example, Figure 3 The vulnerability knowledge graph ontology structure in the above table defines a relationship <CVE vulnerability, cve, cause, cause, cause>, but for CVE-2022-0839, this relationship data is not queried in the generated vulnerability knowledge graph, so the information of the vulnerability knowledge graph needs to be completed, and the relationship information <CVE-2022-0839, cve, cause, improper restriction of XML external entity reference, cause> is added to the vulnerability knowledge graph.

[0145] Further, the electronic device has various ways to complete the information of the vulnerability knowledge graph, which will be introduced below.

[0146] In the first way, after the electronic device determines the missing information in the vulnerability knowledge graph according to the vulnerability knowledge graph ontology structure, the missing information can be displayed, which is used to instruct the graph construction personnel to complete the information of the vulnerability knowledge graph. Then, the electronic device supplements the input information of the graph construction personnel to the vulnerability knowledge graph.

[0147] The second mode, the electronic device determines the missing information in the vulnerability knowledge graph according to the ontology structure of the vulnerability knowledge graph, and then uses the trained prediction model to predict the missing information and supplements the prediction result to the vulnerability knowledge graph.

[0148] Case two, information correction is performed on the vulnerability knowledge graph.

[0149] Since the collected multi-source data is maintained by different organizations, there may be inconsistent values of related information, and therefore inconsistent information needs to be corrected to ensure the accuracy of the graph.

[0150] The electronic device can set related attributes and relationships according to the ontology structure of the vulnerability knowledge graph, and check the vulnerability entities in the vulnerability knowledge graph one by one according to the related attributes and relationships. If the attribute values and relationship data are inconsistent, the inconsistent attribute values or relationships are determined as the error information of the vulnerability knowledge graph, and the information correction is performed on the vulnerability knowledge graph.

[0151] For example, the ontology structure of the vulnerability knowledge graph defines an attribute <CVE vulnerability, cve, risk level, string>, and string is the type of attribute value, indicating a string type. And the CNNVD vulnerability related to the CVE vulnerability defines an attribute <CNNVD vulnerability, cnnvd, harm level, string>, wherein “risk level” and “harm level” both represent the threat level of the vulnerability, and the values are high risk, medium risk and low risk. If the risk level of a CVE vulnerability is low risk, and the harm level of the CNNVD vulnerability related to the CVE vulnerability is high risk, low risk is not equal to high risk, that is, the attribute values of the related attributes are inconsistent, and therefore the information correction needs to be performed on the vulnerability knowledge graph.

[0152] Or for example, the ontology structure of the vulnerability knowledge graph includes a relationship <CVE vulnerability, cve, affected asset, CPE, cpe> and <CVE vulnerability, cve, affected product, product, product>, and the format of the CPE entity is cpe: / <part> : <vendor> : <product> : <version> : <update> : <edition> : <language>The attributes of the product entity include product manufacturer, product name, and product version. By parsing the CPE format, the manufacturer, product, and version fields can be extracted, which can correspond to the attributes of the product entity. If the CPE entity of the asset affected by a certain CVE vulnerability is cpe: / a:apache:log4j:2.13.1:~, the parsed fields are {manufacturer: apache, product: log4j, version: 2.13.1}, and the attributes of the product entity of the product affected by the CVE are {product manufacturer: apache, product name: log4j, product version: 2.12.3}. By comparing the fields one by one, it is found that the version of the affected product is inconsistent, and therefore information correction needs to be performed on the vulnerability knowledge graph.

[0153] Further, there are various ways for the electronic device to perform information correction on the vulnerability knowledge graph, which will be introduced below.

[0154] Method 1: After the electronic device determines the error information in the vulnerability knowledge graph according to the ontology structure of the vulnerability knowledge graph, the electronic device can display the error information, which is used to instruct the graph construction personnel to perform information correction on the vulnerability knowledge graph. Then, the electronic device performs information correction on the vulnerability knowledge graph according to the input information of the graph construction personnel.

[0155] Method 2: The electronic device pre-sets the credibility ranking of various data sources. After the electronic device determines the error information in the vulnerability knowledge graph according to the ontology structure of the vulnerability knowledge graph, the electronic device performs information correction on the vulnerability knowledge graph according to the data in the data source with higher credibility.

[0156] Case three: information completion and information correction of the vulnerability knowledge graph.

[0157] Specifically, the process of determining whether to perform information completion on the vulnerability knowledge graph and how to perform information completion is as described in the foregoing case 1, and the process of determining whether to perform information correction on the vulnerability knowledge graph and how to perform information correction is as described in the foregoing case 2, which will not be repeated here.

[0158] In a possible embodiment, the electronic device can set priorities for each attribute and relationship, and perform information completion and / or information correction on the vulnerability knowledge graph in the order of the priorities. The priority can be set by a security expert according to experience, for example, the higher the risk level of the vulnerability type, the higher the priority. The following cases are introduced.

[0159] For case one, information completion of the vulnerability knowledge graph.

[0160] The electronic device can determine missing information of the vulnerability entity in the vulnerability knowledge graph according to the ontology structure of the vulnerability knowledge graph. If there are multiple missing information, the information of the vulnerability knowledge graph is completed in turn according to the preset first priority, and the verified vulnerability knowledge graph is obtained.

[0161] For case two, the information of the vulnerability knowledge graph is modified.

[0162] The electronic device can determine the error information of the vulnerability entity in the vulnerability knowledge graph according to the ontology structure of the vulnerability knowledge graph. If there are multiple error information, the information of the vulnerability knowledge graph is modified in turn according to the preset second priority, and the verified vulnerability knowledge graph is obtained.

[0163] For case three, the information of the vulnerability knowledge graph is completed and modified.

[0164] The electronic device can determine the missing information and error information of the vulnerability entity in the vulnerability knowledge graph according to the ontology structure of the vulnerability knowledge graph. If there are multiple missing information, the information of the vulnerability knowledge graph is completed in turn according to the preset first priority, and if there are multiple error information, the information of the vulnerability knowledge graph is modified in turn according to the preset second priority, and the verified vulnerability knowledge graph is obtained. The first priority and the second priority can be the same or different.

[0165] In the embodiments of the present application, the information of the vulnerability knowledge graph is completed and / or modified according to the preset priority, so that the more important attribute values or relationships in the vulnerability knowledge graph can be completed or modified first.

[0166] In a possible embodiment, after obtaining the verified vulnerability knowledge graph, the electronic device can write the similarity between the vulnerabilities into the verified vulnerability knowledge graph, which facilitates subsequent searching for similar vulnerabilities. The specific steps are described in S3.1-S3.3.

[0167] S3.1, determine a plurality of similarities of the first vulnerability and the second vulnerability in a plurality of preset dimensions.

[0168] The electronic device can design a plurality of preset dimensions according to the ontology structure of the vulnerability graph and different analysis angles, for example: vulnerability impact asset / product dimension (<CVE vulnerability, cve, impact asset, CPE, cpe> and <CVE vulnerability, cve, impact product, product, product>), vulnerability type dimension (<CVE vulnerability, cve, related, CWE, cwe> and <CVE vulnerability, cve, type, vulnerability type, vulnerability type>), vulnerability exploiter type dimension (<CVE vulnerability, cve, exploiter type, user type, user type>), etc.

[0169] wherein the first vulnerability and the second vulnerability are any two vulnerability entities in the checked vulnerability knowledge graph. The electronic device can formulate a query path according to a plurality of preset dimensions, query an associated subgraph of the first vulnerability and the second vulnerability under any dimension in the checked vulnerability knowledge graph, and calculate the similarity of the first vulnerability and the second vulnerability.

[0170] The electronic device can determine the similarity of the two vulnerabilities under any dimension according to the number of shared nodes of the two vulnerabilities under the dimension. The specific calculation of the similarity of the first vulnerability and the second vulnerability under any dimension is introduced below.

[0171] Specifically, the electronic device can determine a first entity set related to the first vulnerability under any dimension, and a second entity set related to the second vulnerability under the dimension, determine the number of entities in the intersection of the first entity set and the second entity set, and the number of entities in the union of the first entity set and the second entity set, and determine the similarity of the first vulnerability and the second vulnerability under the dimension according to the ratio of the number of entities in the intersection to the number of entities in the union.

[0172] The higher the similarity, the more the shared nodes of the first vulnerability and the second vulnerability under the dimension account for. The calculation formula of the similarity is:

[0173]

[0174] wherein V1 represents the first vulnerability, V2 represents the second vulnerability, D represents any dimension, and sub_similar_score(V1, D, V2) represents the similarity of the first vulnerability V1 and the second vulnerability V2 under the dimension D. represents a first entity set related to the first vulnerability V1 under any dimension D, represents a second entity set related to the second vulnerability V2 under any dimension D, represents a union, and represents an intersection, represents the number of entities in the intersection, represents the number of entities in the union.

[0175] Please refer to Figure 6 , the associated diagram of the first vulnerability and the second vulnerability provided by the embodiment of the present application. Assuming that D represents the analysis dimension of "affecting product", then

[0176] Therefore, the similarity of the first vulnerability V1 and the second vulnerability V2 under the analysis dimension of "affecting product" is:

[0177] ​

[0178] Assuming D represents the analytical dimension of "vulnerability type", then achievable

[0179] The similarity between vulnerability V1 and vulnerability V2 in terms of "vulnerability type" is:

[0180]

[0181] Similarly, the number of nodes shared by the first vulnerability V1 and the second vulnerability V2 in the "impact" analysis dimension is 0, and their similarity is also 0.

[0182] S3.2. The similarity scores are weighted and summed according to the weights of multiple preset dimensions to obtain the total similarity between the first vulnerability and the second vulnerability.

[0183] Electronic devices obtain multiple similarities between a first vulnerability and a second vulnerability across multiple preset dimensions, for example... in, This indicates that the first vulnerability V1 and the second vulnerability V2 are in D n Similarity across dimensions. Electronic devices can have multiple preset dimensions weighted, for example, [ω1, ω2, ..., ω]. n ], where ω n D represents n Dimension weights. Multiple preset dimensions can have the same weight, for example, all of them can have 1, indicating that multiple dimensions are equally important in judging vulnerability similarity. Multiple preset dimensions can also have different weights. For example, "products affected by the vulnerability" is more important than "type of exploiters of the vulnerability" in judging vulnerability similarity, so the dimension of "vulnerability impact" can be given a higher weight.

[0184] The electronic device performs a weighted summation of multiple similarities according to the weights of multiple preset dimensions. The weighted summation formula is as follows:

[0185]

[0186] Where similar_score(V1,V2) represents the total similarity between the first vulnerability V1 and the second vulnerability V2, N represents the total number of multiple preset dimensions, and ω n and Please refer to the previous discussion for the meaning of , which will not be repeated here.

[0187] S3.3 Write the total similarity into the verified vulnerability knowledge graph to obtain the updated vulnerability knowledge graph.

[0188] After obtaining the total similarity between the first vulnerability and the second vulnerability, the electronic device writes the total similarity into the verified vulnerability knowledge graph, so that the updated vulnerability knowledge graph includes the total similarity between each pair of entities of vulnerabilities.

[0189] For example, an edge is established between the nodes corresponding to the first vulnerability V1 and the second vulnerability V2, and the attribute of the edge is the total similarity. Please refer to Figure 7 The figure shows the total similarity between two vulnerabilities provided by the embodiments of the present application. As can be seen, the two vulnerabilities are CVE-2021-44228 and CVE-2021-4104, and the total similarity between them is 0.96.

[0190] In the embodiments of the present application, the similarity of any pair of vulnerabilities, such as the first vulnerability and the second vulnerability, in multiple preset dimensions is calculated, and the total similarity of the pair of vulnerabilities is obtained by weighted summation, and the total similarity of the pair of vulnerabilities is extended to the vulnerability knowledge graph, so that the vulnerability information in the vulnerability knowledge graph is more abundant.

[0191] In a possible embodiment, the electronic device can perform similar vulnerability recommendation according to the updated vulnerability knowledge graph, to a certain extent, helping analysts to quickly obtain similar vulnerabilities for collaborative analysis and improving analysis efficiency.

[0192] Specifically, the electronic device receives a request for querying similar vulnerabilities of a target vulnerability, can determine multiple similar vulnerabilities with a total similarity greater than a preset similarity to the target vulnerability according to the updated vulnerability knowledge graph, and display the multiple similar vulnerabilities in descending order of the total similarity to the target vulnerability.

[0193] For example, the analyst inputs the vulnerability number of the target CVE vulnerability into the electronic device, and the vulnerability number is given by the official, such as CVE-2022-0168, CNNVD-2022-0168, etc. The electronic device responds to the input operation, queries the <CVE vulnerability, similarity, CVE vulnerability> relationship in the verified vulnerability knowledge graph, obtains similar CVE vulnerabilities to the target CVE vulnerability, and filters and sorts the total similarity according to the preset threshold, and displays the top-N similar CVE vulnerabilities to the target CVE vulnerability.

[0194] In the embodiments of the present application, according to the total similarity between each pair of vulnerabilities in the updated vulnerability knowledge graph, the similar vulnerabilities of the target vulnerability can be directly obtained, the similar vulnerabilities are filtered and sorted according to the preset similarity, and the top-N similar vulnerabilities of the target vulnerability are displayed to the analyst, which can help the analyst to quickly obtain similar vulnerabilities for collaborative analysis and improve analysis efficiency.

[0195] In a possible embodiment, an analyst can construct a query statement in the vulnerability knowledge graph by combining various attributes and relationships to obtain vulnerability information of interest. After the electronic device obtains the checked vulnerability knowledge graph, the electronic device can receive a query request, determine vulnerability information corresponding to input attribute information and / or relationship information according to the checked vulnerability knowledge graph, and display the vulnerability information.

[0196] For example, the electronic device queries products affected by a high-risk CVE vulnerability according to the input attribute "risk level of the CVE vulnerability = high risk" and the relationship "affected product". Or for example, the electronic device can query products affected by a vulnerability and causes according to the relationships "affected product" and "cause".

[0197] In the embodiments of the present application, an analyst can combine various attributes and relationships to design corresponding query modes for in-depth correlation query and analysis of vulnerabilities according to actual business scenarios, thereby improving analysis efficiency and expanding analysis angles.

[0198] To sum up, considering that vulnerability-related data is relatively scattered and not strongly correlated, which is not conducive to an analyst to quickly grasp the overall situation and deep correlation of vulnerabilities, the method provided in the embodiments of the present application uses a knowledge graph to standardize and uniformly store vulnerability-related data. Considering that the data granularity of an existing vulnerability knowledge graph is relatively coarse, which is not conducive to an analyst to understand multi-dimensional information of vulnerabilities, the embodiments of the present application extract key information of vulnerabilities from structured data and unstructured text data to fill the vulnerability knowledge graph. Considering that there are differences in fields and values in multi-source data fusion, directly constructing a vulnerability graph may lead to incorrect judgments by an analyst, the embodiments of the present application perform integrity and consistency checking on the vulnerability knowledge graph, thereby improving the integrity and accuracy of the vulnerability knowledge graph, writing similarity between vulnerabilities into the vulnerability knowledge graph, and enabling an analyst to perform in-depth correlation mining on vulnerabilities based on various analysis dimensions on the vulnerability knowledge graph, analyze the overall situation of vulnerabilities, and improve understanding depth and analysis efficiency.

[0199] Based on the same inventive concept, the present application further provides a vulnerability knowledge graph processing apparatus, which can be specifically arranged in the electronic device discussed above, please refer to Figure 8 The apparatus comprises:

[0200] The acquisition module 801 is configured to acquire a data set related to vulnerabilities.

[0201] The generation module 802 is configured to perform key data extraction on the data set, store the extracted key data in a graph database according to a preset format, and generate a vulnerability knowledge graph. The vulnerability knowledge graph is used to indicate relationships between various first-class entities in the vulnerability field and attribute values of the various first-class entities.

[0202] The checking module 803 is configured to perform information completion and / or information correction on the vulnerability knowledge graph according to a pre-constructed vulnerability knowledge graph ontology structure, to obtain a checked vulnerability knowledge graph; wherein the vulnerability knowledge graph ontology structure is used to indicate the relationship between each second entity in the vulnerability field and the attribute of each second entity, and the second entity is an entity including the features of the first entity.

[0203] In a possible embodiment, the data set includes structured data and unstructured text data.

[0204] In a possible embodiment, the generating module 802 is specifically configured to:

[0205] input the unstructured text data into the trained entity recognition model, to obtain a first label and a second label of each word in the unstructured text data; wherein the first label is used to indicate that the word is not an entity, or the word is the beginning of an entity fragment, or the word is the middle of an entity fragment, or the word is the end of an entity fragment, and the second label is used to indicate the entity type corresponding to the word when the word is an entity;

[0206] merge a plurality of first words extracted from the unstructured text data according to the first label and the second label of each word, to obtain a plurality of second words; wherein the first word is a word with the same second label and the first label being the beginning or the middle or the end of an entity fragment, and each second word corresponds to a first entity;

[0207] store the relationship between the plurality of first entities corresponding to the plurality of second words and the attribute values of the plurality of first entities in the graph database in a preset format; wherein the relationship between the plurality of first entities is determined according to the vulnerability knowledge graph ontology structure, and the attribute values of the plurality of first entities are determined according to the plurality of second words.

[0208] In a possible embodiment, the checking module 803 is specifically configured to:

[0209] determine missing information and / or incorrect information of the vulnerability ontology in the vulnerability knowledge graph according to the vulnerability knowledge graph ontology structure;

[0210] if the missing information is multiple, sequentially complete the information of the vulnerability knowledge graph according to a preset first priority, and / or if the incorrect information is multiple, sequentially correct the information of the vulnerability knowledge graph according to a preset second priority;

[0211] obtain the checked vulnerability knowledge graph.

[0212] In a possible embodiment, the apparatus further includes a writing module 804, which is configured to:

[0213] After obtaining the verified vulnerability knowledge graph

[0214] Determine the similarity between the first vulnerability and the second vulnerability across multiple preset dimensions; wherein the first vulnerability and the second vulnerability are any two vulnerability entities in the verified vulnerability knowledge graph;

[0215] The similarity scores are weighted and summed according to the weights of multiple preset dimensions to obtain the total similarity between the first vulnerability and the second vulnerability.

[0216] The total similarity is written into the verified vulnerability knowledge graph to obtain the updated vulnerability knowledge graph.

[0217] In one possible embodiment, the writing module 804 is specifically used for:

[0218] Identify the first set of entities that are related to the first vulnerability in any dimension, and the second set of entities that are related to the second vulnerability in any dimension;

[0219] Determine the number of entities in the intersection of the first entity set and the second entity set, and the number of entities in the union of the first entity set and the second entity set;

[0220] The similarity between the first vulnerability and the second vulnerability in any dimension is determined by the ratio of the number of entities in the intersection to the number of entities in the union.

[0221] In one possible embodiment, the device further includes a query module 805, which is used to:

[0222] After writing the total similarity into the verified vulnerability knowledge graph and obtaining the updated vulnerability knowledge graph, a request to query similar vulnerabilities of the target vulnerability is received.

[0223] Based on the updated vulnerability knowledge graph, identify multiple similar vulnerabilities whose total similarity to the target vulnerability is greater than the preset similarity.

[0224] Multiple similar vulnerabilities are displayed in descending order of their total similarity to the target vulnerability.

[0225] As one example, Figure 8 The device described can be used to perform Figure 2 The vulnerability knowledge graph processing method described in the illustrated embodiment can be used as a reference for understanding the functions that each functional module of the device can achieve. Figure 2 The description of the embodiments shown will not be repeated here.

[0226] It should be noted that although several modules or sub-modules of the apparatus are mentioned in the foregoing detailed description, such a division is merely exemplary and not mandatory. Indeed, according to embodiments of the application, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into modules embodied by several modules.

[0227] Based on the same inventive concept, an electronic device is also provided in the embodiments of the present application, please refer to Figure 9 , which comprises a processor 901 and a memory 902.

[0228] The memory 902 is configured to store program instructions.

[0229] The processor 901 is configured to invoke the program instructions stored in the memory 902, and execute the vulnerability knowledge graph processing method described above according to the obtained program instructions. The processor 901 can also implement the functions of various modules in the apparatus shown. Figure 8

[0230] The specific connection medium between the processor 901 and the memory 902 is not limited in the embodiments of the present application, Figure 9 for example, the connection between the processor 901 and the memory 902 through the bus 900. The bus 900 is represented by a thick line in Figure 9 , and the connection mode between other components is only schematically illustrated and is not limited. The bus 900 can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 9 in the foregoing description, only one thick line is used to represent the bus, but it does not mean that there is only one bus or only one type of bus. Alternatively, the processor 901 can also be referred to as a controller, and the name is not limited.

[0231] The processor 901 is the control center of the apparatus, and can connect various parts of the control device through various interfaces and lines, and monitor the whole apparatus by running or executing the instructions stored in the memory 902 and calling the data stored in the memory 902, thereby processing data and monitoring the whole apparatus.

[0232] In a possible design, the processor 901 can include one or more processing units, and the processor 901 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 901. In some embodiments, the processor 901 and the memory 902 can be implemented on the same chip, and in some embodiments, they can also be implemented on separate chips respectively. ​

[0233] The processor 901 can be a general processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor. The steps of the vulnerability knowledge graph processing method disclosed in the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0234] The memory 902 is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 902 can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. The memory 902 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 902 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used to store program instructions and / or data.

[0235] By designing and programming the processor 901, the code corresponding to the vehicle firmware upgrade method introduced in the foregoing embodiments can be fixed in the chip, so that the chip can execute the steps of the vulnerability knowledge graph processing method discussed above at runtime. How to design and program the processor 901 is a technology known to those skilled in the art, which will not be described here.

[0236] Based on the same inventive concept, the embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program includes program instructions, and the program instructions make the computer execute the vulnerability knowledge graph processing method as any one of the foregoing. Since the principle of solving problems of the above computer readable storage medium is similar to the vulnerability knowledge graph processing method, the implementation of the above computer readable storage medium can be referred to the implementation of the method, and the repeated parts will not be described herein.

[0237] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0238] The present application is described with reference to flowcharts and / or block diagrams according to the method, device (system), and computer program product of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0239] These computer program instructions can also be stored in a computer readable memory capable of guiding the computer or other programmable data processing devices to work in a specific way, so that the instructions stored in the computer readable memory produce a product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0240] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0241] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.< / language> < / edition> < / update> < / version> < / product> < / vendor> < / part>

Claims

1. A vulnerability knowledge graph processing method, characterized in that, The method comprises the following steps: obtaining a vulnerability-related data set; the data set comprises structured data and unstructured data; performing key data extraction on the data set, and storing the extracted key data in a graph database in a preset format to generate a vulnerability knowledge graph; wherein the vulnerability knowledge graph is used to indicate the relationship between each first-class entity in the vulnerability field and the attribute value of each first-class entity; According to the pre-constructed vulnerability knowledge graph ontology structure, the information of the vulnerability knowledge graph is completed and / or corrected, and the verified vulnerability knowledge graph is obtained; wherein the vulnerability knowledge graph ontology structure is used to indicate the relationship between each second-class entity in the vulnerability field and the attribute of each second-class entity, and the second-class entity is an entity including the characteristics of the first-class entity; wherein, the key data extraction on the data set and the storage of the extracted key data in the graph database in the preset format include: inputting the unstructured data into the trained entity recognition model to obtain the first label and the second label of each word in the unstructured data; wherein the first label is used to indicate that each word is not an entity, or the beginning of an entity fragment, or the middle of an entity fragment, or the end of an entity fragment, and the second label is used to indicate the entity type corresponding to the entity when the word is an entity; According to the first label and the second label of each word, a plurality of first words are extracted from the unstructured data and merged to obtain a plurality of second words; wherein the first word is a word with the same second label and the first label being the beginning or middle or end of an entity fragment, and each second word corresponds to a first-class entity; the relationship between the plurality of second words corresponding to the plurality of first-class entities, and the attribute value of the plurality of second words corresponding to the plurality of first-class entities are stored in the graph database in a preset format; wherein the relationship between the plurality of second words corresponding to the plurality of first-class entities is determined according to the vulnerability knowledge graph ontology structure, and the attribute value of the plurality of second words corresponding to the plurality of first-class entities is determined according to the plurality of second words.

2. The method of claim 1, wherein, According to the pre-constructed vulnerability knowledge graph ontology structure, the information of the vulnerability knowledge graph is completed and / or corrected, and the verified vulnerability knowledge graph is obtained; wherein the vulnerability knowledge graph ontology structure is used to indicate the relationship between each second-class entity in the vulnerability field and the attribute of each second-class entity, and the second-class entity is an entity including the characteristics of the first-class entity; wherein, the key data extraction on the data set and the storage of the extracted key data in the graph database in the preset format include: According to the vulnerability knowledge graph ontology structure, the missing information and / or incorrect information of the vulnerability entity in the vulnerability knowledge graph is determined; if the missing information is multiple, the information of the vulnerability knowledge graph is completed in turn according to the preset first priority, and / or if the incorrect information is multiple, the information of the vulnerability knowledge graph is corrected in turn according to the preset second priority; 3. The method of claim 1, wherein, obtaining the verified vulnerability knowledge graph. After obtaining the verified vulnerability knowledge graph, the method further comprises: determining a plurality of similarities between a first vulnerability and a second vulnerability in a plurality of preset dimensions; wherein the first vulnerability and the second vulnerability are any two vulnerability entities in the verified vulnerability knowledge graph; weighting and summing the plurality of similarities according to the weights of the plurality of preset dimensions, to obtain a total similarity between the first vulnerability and the second vulnerability; write the total similarity into the checked vulnerability knowledge graph to obtain an updated vulnerability knowledge graph.

4. The method of claim 3, wherein, determining the plurality of similarities between the first vulnerability and the second vulnerability in the plurality of preset dimensions comprises: determining a first entity set having a relationship with the first vulnerability in any dimension, and a second entity set having a relationship with the second vulnerability in the any dimension; determining the number of entities in the intersection of the first entity set and the second entity set, and the number of entities in the union of the first entity set and the second entity set; determining the similarity between the first vulnerability and the second vulnerability in the any dimension according to the ratio of the number of entities in the intersection to the number of entities in the union. After writing the total similarity into the checked vulnerability knowledge graph to obtain an updated vulnerability knowledge graph, the method further comprises:

5. The method of claim 4, wherein, receiving a request for querying similar vulnerabilities of a target vulnerability; determining a plurality of similar vulnerabilities having a total similarity greater than a preset similarity with the target vulnerability according to the updated vulnerability knowledge graph; displaying the plurality of similar vulnerabilities in order from high to low according to the total similarity with the target vulnerability. comprise:

6. A vulnerability knowledge graph processing apparatus, characterized by comprising: an acquisition module configured to acquire a data set related to vulnerabilities; the data set comprises structured data and unstructured data; a generation module configured to perform key data extraction on the data set, store the extracted key data in a graph database according to a preset format, and generate a vulnerability knowledge graph; wherein the vulnerability knowledge graph is used to indicate the relationships between various first entities in the vulnerability field and the attribute values of the various first entities; a verification module configured to perform information completion and / or information correction on the vulnerability knowledge graph according to a pre-constructed vulnerability knowledge graph ontology structure, to obtain a checked vulnerability knowledge graph; wherein the vulnerability knowledge graph ontology structure is used to indicate the relationships between various second entities in the vulnerability field and the attributes of the various second entities, and the second entities include features of the first entities; wherein, when the generation module is used to perform key data extraction on the data set, store the extracted key data in a graph database according to a preset format, and generate a vulnerability knowledge graph, it is specifically configured to: input the unstructured data into a trained entity recognition model to obtain a first label and a second label of each word in the unstructured data; wherein the first label is used to indicate that each word is not an entity, or that the word is the beginning of an entity fragment, or that the word is the middle of an entity fragment, or that the word is the end of an entity fragment, and the second label is used to indicate the entity type corresponding to the word when the word is an entity; merge a plurality of first words extracted from the unstructured data according to the first label and the second label of each word to obtain a plurality of second words; wherein the first word is a word with the same second label and the first label being the beginning or the middle or the end of an entity fragment, and each second word corresponds to a first entity; ​ The relationships between the plurality of first-class entities corresponding to the plurality of second words and the attribute values of the plurality of first-class entities corresponding to the plurality of second words are stored in a graph database in a preset format; wherein the relationships between the plurality of first-class entities corresponding to the plurality of second words are determined according to the vulnerability knowledge graph ontology structure, and the attribute values of the plurality of first-class entities corresponding to the plurality of second words are determined according to the plurality of second words.

7. An electronic device, comprising: Comprise: a memory for storing program instructions; a processor for calling the program instructions stored in the memory, and executing the method of any one of claims 1-5 according to the obtained program instructions.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program comprises program instructions, which, when executed by a computer, cause the computer to execute the method of any one of claims 1-5.

Images