A vulnerability type association analysis method based on knowledge graph

Abstract

This invention discloses a vulnerability type association analysis method based on knowledge graphs, comprising the following steps: Step 1, Knowledge Graph Database Construction: The Common Vulnerability and Disclosure Platform (CVE), the National Vulnerability Database (NVE), and the Japan Vulnerability Database (JVN) are pre-constructed into edges according to entity nodes and semantic relationships between entities, and aggregated into a comprehensive knowledge graph database. Through process coordination, the massive data in the CVE, NVE, and JVN are processed by first constructing the knowledge graph database, then classifying and screening vulnerability subsets, and finally performing first-level verification and second-level re-verification. This achieves the effect of comprehensive, detailed, and high-precision type association analysis of vulnerability data in massive data, reducing the limitations of vulnerability data relationship analysis and lowering the fault tolerance rate of vulnerability data relationship analysis.

Description

Technical Field

[0001] This invention relates to the field of vulnerability type association analysis technology based on knowledge graphs, specifically to a vulnerability type association analysis method based on knowledge graphs. Background Technology

[0002] Knowledge graphs, known in the library and information science field as knowledge domain visualization or knowledge domain mapping maps, are a series of different graphics that display the development process and structural relationships of knowledge. They use visualization technology to describe knowledge resources and their carriers, mine, analyze, construct, draw, and display knowledge and the interrelationships between them. By combining the theories and methods of applied mathematics, graphics, information visualization technology, and information science with methods such as bibliometric citation analysis and co-occurrence analysis, knowledge graphs vividly display the core structure, development history, cutting-edge fields, and overall knowledge architecture of a discipline, achieving the goal of multidisciplinary integration.

[0003] With the development of internet and big data technologies, massive amounts of data are generated across various industries. To better analyze and process vulnerability data within this massive dataset, it is necessary to develop vulnerability type association analysis methods based on knowledge graphs. However, most current knowledge graph vulnerability type association analysis methods are limited to vulnerability analysis of small data areas. They cannot perform comprehensive, detailed, and high-precision type association analysis of massive datasets by first building a knowledge graph database, then classifying and filtering vulnerability subsets, and finally conducting first-level verification and second-level re-verification. This approach is not only limited but also has a high fault tolerance rate, failing to meet the current development needs of internet and big data technologies. Therefore, this paper proposes a vulnerability type association analysis method based on knowledge graphs. Summary of the Invention

[0004] The purpose of this invention is to provide a vulnerability type association analysis method based on knowledge graphs, in order to solve the problems mentioned in the background art. Most of the currently used knowledge graph vulnerability type association analysis methods are limited to vulnerability analysis of small areas of data. They cannot perform comprehensive, detailed and high-precision type association analysis of massive amounts of data by first building a knowledge graph database, then classifying and screening vulnerability subsets, and then performing first-level verification and second-level re-verification. This not only has great limitations, but also has a high fault tolerance rate, which cannot meet the current development needs of Internet and big data technologies.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a vulnerability type association analysis method based on knowledge graphs, comprising the following steps:

[0006] Step 1: Construction of the Knowledge Graph Repository: The Common Vulnerability and Disclosure Platform (CVE), the National Vulnerability Database (NVE), and the Japan Vulnerability Database (JVN) are pre-constructed into edges based on entity nodes and semantic relationships between entities, and then aggregated into a comprehensive knowledge graph repository.

[0007] Step 2, Classification and Screening Module: The vulnerability data of various types in the knowledge graph database are then classified and screened using the SVM classification and screening method. The classification and screening module divides the vulnerability data into vulnerability set 1, vulnerability set 2 to vulnerability set N according to semantic feature relationships, keyword features and background corpus features, and N represents the total number of vulnerability data subsets.

[0008] Step 3, Conceptual Feature Level 1 Verification: Next, each vulnerability data subset after classification and screening is processed according to the instance conceptualization method. P(c / i), P(i / c), mutual information, point mutual information, standardized point mutual information, and conceptual feature algorithms are used to perform Level 1 verification on each vulnerability data subset after classification and screening according to the degree of association between instances and concepts. If any vulnerability data subset fails Level 1 verification, the corresponding vulnerability data subset that fails Level 1 verification is returned to Step 1 and Step 2 for re-construction of knowledge graph and classification and screening. This process is repeated three times. After three cycles, if there are still vulnerability data subsets that fail Level 1 verification, they are directly stored in the Level 1 verification failure database for manual review.

[0009] Step 4: Classification and Evaluation Second-Level Re-verification: If each vulnerability data subset directly passes the first-level verification and the problematic vulnerability data subset after three iterations passes the first-level verification, then each vulnerability data subset that passed the first-level verification is added to the original feature set according to the obtained concept words as extended features. This results in the semantic feature set of each vulnerability data subset. A classification and evaluation algorithm is then used to re-verify each vulnerability data subset that passed the first-level verification. The original vulnerability data of each vulnerability data subset that passed the first-level verification is set to K, the number of extensions for each re-verified vulnerability data subset is set to T, and the instance set of each re-verified vulnerability data subset is set to X. The re-verification set after the conceptual features of each vulnerability data subset that passed the first-level verification are extended is set to K*. The feature word set and related semantic set obtained by the classification evaluation algorithm for each vulnerability data subset that has passed the first-level verification are respectively set as (S1, S2, ..., Si) and topic (C1, C2, ..., Cj), then K*={S1C1, S2C2, ..., SiCj}, where i is the total number of feature word sets and j is the total number of related semantic sets. If each vulnerability data subset after the second-level verification fails the classification evaluation algorithm verification, the vulnerability data subset that fails the second-level verification is returned to steps one, two and three for re-construction of knowledge graph, classification and screening and first-level conceptual feature algorithm verification processing. This process is repeated three times. After three cycles, if there are still vulnerability data subsets that fail the second-level verification, they are directly stored in the second-level verification failure database for manual review.

[0010] Step 5: Data Vulnerability Decision Table Construction: If each subset of vulnerability data passes the second-level verification directly, and the subset of problematic vulnerability data after three cycles passes the second-level verification, they are directly summarized into the data vulnerability decision table.

[0011] Preferably, in the classification and screening module of step two, the various vulnerability data subsets in the knowledge graph database are further classified using WordXet for semantic disambiguation, keyword extraction, and background corpus in various environments. Then, a Bayesian classifier is used to further classify and screen the various vulnerability data subsets after the classification.

[0012] Preferably, in the first-level verification of conceptual features in step three, P(c / i) represents the probability that the concept corresponding to the current instance i is c, and P(i / c) represents the probability that the instance corresponding to the current concept c is i. Therefore, the formula for calculating P(c / i) is:

[0013] P(c / i)=n(c,i) / ∑ i∈c n(c j , i);

[0014] P(i / c)=n(c,i) / ∑ i∈c n(c,i jIn the formula, n(c, i) represents the number of times instance i and concept c appear simultaneously, ∑ i∈c n(c j ∑ii (ii) represents the number of times instance i and all concepts contained in instance i appear simultaneously. i∈c n(c,i j ) represents the number of times concept c and all instances contained in concept c appear simultaneously.

[0015] Preferably, in the first-level verification of conceptual features in step three, the mutual information W verifies the correlation between variables in each subset of vulnerability data, which is the amount of correlation information of another variable covered by each subset of vulnerability data variables, and the calculation formula is:

[0016] W(i / c)=∑P(i / c)log{(Pi / c) / P(i)P(c)}, where P(i / c) is the joint probability that instance i and concept c occur simultaneously.

[0017] Preferably, in the first-level verification of conceptual features in step three, the point mutual information verification verifies the correlation between instances and concepts in each vulnerability data subset, and the calculation formula is as follows:

[0018] PW(i / c) = log{(Pi / c) / P(i)P(c)} = logP(i / c) - logP(i), where logP(i) is a constant representing the association between instances and concepts in each vulnerability data subset, and the value of PW(i / c) depends on logP(i / c).

[0019] Preferably, in the first-level verification of conceptual features in step three, the standardized point mutual information verification verifies the correlation score level between instances and concepts in each vulnerability data subset, and the calculation formula is as follows:

[0020] NPW(i / c)=PW(i / c) / -logP(i / c)={logP(i / c)-logP(i)} / -logP(i / c);

[0021] Then, we use PW raised to the power of X for comparison and argumentation, and the formula for the argumentation is:

[0022] PW X (i / c) = logP(i / c) X / logP(i)P(c).

[0023] Preferably, in the conceptual feature level-one verification in step three, the conceptual feature algorithm verifies the feature correlation score level between instances and concepts in each vulnerability data subset, and the calculation formula is as follows:

[0024] SPW(i / c) = arg max max RPW(i / c), and the formula for calculating RPW(i / c) is:

[0025] RPW(i / c)=P(c / i)*P(i / c).

[0026] Preferably, in the fourth step of the classification assessment, the secondary verification classification assessment algorithm is as follows:

[0027] Q=(1+β 2 )*H*V / H+V*β 2 +t, where H is the ratio of the number of vulnerabilities that pass the second verification in each vulnerability data subset to the total number of vulnerability data, V is the ratio of the number of vulnerabilities that fail the second verification in each vulnerability data subset to the total number of vulnerability data, β is the weight of H and V, and the value of β is 1, and t is the weighting addition of β, and t is 0.1.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] In this invention, through the coordinated process of step one (construction of a knowledge graph database), step two (classification and screening module), step three (first-level verification of conceptual features), step four (second-level verification of classification and evaluation), and step five (construction of a data vulnerability decision table), massive data is processed by first building a knowledge graph database, then classifying and screening vulnerability subsets, and finally performing first-level verification and second-level verification. This achieves comprehensive, detailed, and high-precision type correlation analysis of vulnerability data, reduces the limitations of vulnerability data relationship analysis, lowers the fault tolerance rate of vulnerability data relationships, and meets the current development needs of Internet and big data technologies. Attached Figure Description

[0030] Figure 1 This is a knowledge graph construction diagram for the present invention;

[0031] Figure 2 This is a flowchart of the method of the present invention;

[0032] Figure 3 This is a flowchart of the method of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Example

[0035] Please see Figures 1-3 This invention provides a technical solution: a vulnerability type association analysis method based on knowledge graphs, comprising the following steps:

[0036] Step 1: Construction of the Knowledge Graph Repository: The Common Vulnerability and Disclosure Platform (CVE), the National Vulnerability Database (NVE), and the Japan Vulnerability Database (JVN) are pre-constructed into edges based on entity nodes and semantic relationships between entities, and then aggregated into a comprehensive knowledge graph repository.

[0037] Step 2, Classification and Screening Module: The vulnerability data in the constructed knowledge graph database is then classified and screened using the SVM method. The classification and screening module divides the vulnerability data into vulnerability sets 1, 2, and N according to semantic features, keyword features, and background corpus features, where N represents the total number of vulnerability data subsets. WordXet is used to perform semantic disambiguation, keyword extraction, and further refine the classification of each vulnerability data subset in various environments. A Bayesian classifier is then used to further classify and screen the refined vulnerability data subsets. This process strictly controls the classification and screening of each vulnerability data subset to ensure accuracy and provide accurate examples for subsequent correlation verification.

[0038] Step 3: First-level verification of conceptual features: Next, the various vulnerability data subsets after classification and screening are processed according to the instance conceptualization method. Then, P(c / i), P(i / c), mutual information, point mutual information, standardized point mutual information, and conceptual feature algorithms are used to perform first-level verification on each vulnerability data subset after classification and screening based on the correlation between instances and concepts. P(c / i) represents the probability that the concept corresponding to the current instance i is c, and P(i / c) represents the probability that the instance corresponding to the current concept c is i. The formula for calculating P(c / i) is:

[0039] P(c / i)=n(c,i) / ∑ i∈c n(c j , i);

[0040] P(i / c)=n(c,i) / ∑ i∈c n(c,i j In the formula, n(c, i) represents the number of times instance i and concept c appear simultaneously, ∑ i∈c n(c j ∑ii (ii) represents the number of times instance i and all concepts contained in instance i appear simultaneously. i∈c n(c,i jThe mutual information W represents the number of times concept c and all instances contained in concept c occur simultaneously. The mutual information W verifies the correlation between variables in each subset of vulnerability data, and represents the correlation information of another variable covered by each subset of vulnerability data. The calculation formula is as follows:

[0041] W(i / c)=∑P(i / c)log{(Pi / c) / P(i)P(c)}, where P(i / c) is the joint probability that instance i and concept c occur simultaneously. Point mutual information verification verifies the correlation between instances and concepts in each vulnerability data subset, and the calculation formula is:

[0042] PW(i / c) = log{(Pi / c) / P(i)P(c)} = logP(i / c) - logP(i), where logP(i) is a constant representing the relevance between instances and concepts in each vulnerability data subset, and the value of PW(i / c) depends on logP(i / c). The standardized point mutual information verification evaluates the relevance rating of instances and concepts in each vulnerability data subset, and the calculation formula is as follows:

[0043] NPW(i / c)=PW(i / c) / -logP(i / c)={logP(i / c)-logP(i)} / -logP(i / c);

[0044] Then, we use PW raised to the power of X for comparison and argumentation, and the formula for the argumentation is:

[0045] PW X (i / c) = logP(i / c) X / logP(i)P(c), the conceptual feature algorithm verifies the feature correlation score level between instances and concepts in each vulnerability data subset, and the calculation formula is:

[0046] SPW(i / c) = arg max max RPW(i / c), and the formula for calculating RPW(i / c) is:

[0047] RPW(i / c) = P(c / i) * P(i / c) performs a five-dimensional correlation level-one verification on each vulnerability data subset to prevent verification errors and omissions, and improve the verification accuracy of each vulnerability data subset. If any vulnerability data subset fails the level-one verification, the corresponding vulnerability data subset that fails the level-one verification is returned to steps one and two for re-construction of the knowledge graph and classification and screening. This process is repeated three times. After three iterations, if there are still vulnerability data subsets that fail the level-one verification, they are directly stored in the level-one verification failure database for manual review.

[0048] Step 4: Second-level verification and classification evaluation: If each vulnerability data subset directly passes the first-level verification, and the problematic vulnerability data subset after three iterations also passes the first-level verification, then the vulnerability data subsets that have passed the first-level verification are added to the original feature set according to the obtained concept words as extended features. This results in the semantic feature set of each vulnerability data subset. A classification evaluation algorithm is then used to re-verify each vulnerability data subset that has passed the first-level verification. The second-level verification classification evaluation algorithm is as follows:

[0049] Q=(1+β 2 )*H*V / H+V*β 2 +t, where H is the ratio of the number of vulnerabilities that passed the second verification in each vulnerability data subset to the total number of vulnerabilities, V is the ratio of the number of vulnerabilities that failed the second verification in each vulnerability data subset to the total number of vulnerabilities, β is the weight of H and V, and β is 1; t is the weighting addition of β, and t is 0.1. Precise verification is performed on each vulnerability data subset that passed the first-level verification to further ensure the accuracy of the verification. The original vulnerability data of each vulnerability data subset that passed the first-level verification is set as K, the expansion number of each vulnerability data subset verified by the classification evaluation algorithm is set as T, and the instance set of each vulnerability data subset verified is set as X. The verification set after the conceptual feature expansion of each vulnerability data subset that passed the first-level verification is set as... K*, and the feature word set and related semantic set obtained by the classification evaluation algorithm for each vulnerability data subset after passing the first-level verification are respectively set as (S1, S2, ..., Si) and topic (C1, C2, ..., Cj), then K* = {S1C1, S2C2, ..., SiCj}, where i is the total number of feature word sets and j is the total number of related semantic sets. If each vulnerability data subset after the second-level verification fails the classification evaluation algorithm verification, the vulnerability data subset that fails the second-level verification is returned to steps one, two and three for re-construction of knowledge graph, classification screening and first-level conceptual feature algorithm verification processing. This process is repeated three times. After three cycles, if there are still vulnerability data subsets that fail the second-level verification, they are directly stored in the second-level verification failure database for manual review.

[0050] Step 5: Data Vulnerability Decision Table Construction: If each subset of vulnerability data passes the second-level verification and the subset of problematic vulnerability data after three cycles passes the second-level verification, it is directly summarized into the data vulnerability decision table. Through the coordinated process of Step 1 (Knowledge Graph Library Construction), Step 2 (Classification and Screening Module), Step 3 (Conceptual Feature First-Level Verification), Step 4 (Classification Evaluation Second-Level Verification), and Step 5 (Data Vulnerability Decision Table Construction), this approach achieves comprehensive, detailed, and high-precision type correlation analysis of vulnerability data in massive datasets by first constructing a knowledge graph library, then classifying and screening vulnerability subsets, and finally performing first-level and second-level verification. This reduces the limitations of vulnerability data correlation analysis, lowers the fault tolerance rate of vulnerability data correlation, and meets the current development needs of Internet and big data technologies.

[0051] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A vulnerability type association analysis method based on knowledge graphs, characterized in that: Includes the following steps: Step 1: Construction of the Knowledge Graph Repository: The Common Vulnerability and Disclosure Platform (CVE), the National Vulnerability Database (NVE), and the Japan Vulnerability Database (JVN) are pre-constructed into edges based on entity nodes and semantic relationships between entities, and then aggregated into a comprehensive knowledge graph repository. Step 2, Classification and Screening Module: The vulnerability data of various types in the knowledge graph database are then classified and screened using the SVM classification and screening method. The classification and screening module divides the vulnerability data into vulnerability set 1, vulnerability set 2 to vulnerability set N according to semantic feature relationships, keyword features and background corpus features, and N represents the total number of vulnerability data subsets. Step 3, Conceptual Feature Level 1 Verification: Next, each vulnerability data subset after classification and screening is processed according to the instance conceptualization method. P(c / i), P(i / c), mutual information, point mutual information, standardized point mutual information, and conceptual feature algorithms are used to perform Level 1 verification on each vulnerability data subset after classification and screening based on the correlation between instances and concepts. P(c / i) represents the probability that the concept corresponding to the current instance i is c, and P(i / c) represents the probability that the instance corresponding to the current concept c is i. If any vulnerability data subset fails Level 1 verification, it is returned to Step 1 and Step 2 for re-construction of the knowledge graph and reclassification and screening. This process is repeated three times. After three iterations, any vulnerability data subsets that still fail Level 1 verification are directly stored in the Level 1 Verification Failure Database for manual review. Step 4: Classification and Evaluation Second-Level Re-verification: If each vulnerability data subset directly passes the first-level verification and the problematic vulnerability data subset after three iterations passes the first-level verification, then each vulnerability data subset that passed the first-level verification is added to the original feature set according to the obtained concept words as extended features. This results in the semantic feature set of each vulnerability data subset. A classification and evaluation algorithm is then used to re-verify each vulnerability data subset that passed the first-level verification. The original vulnerability data of each vulnerability data subset that passed the first-level verification is set to K, the number of extensions for each re-verified vulnerability data subset is set to T, and the instance set of each re-verified vulnerability data subset is set to X. The re-verification set after the conceptual features of each vulnerability data subset that passed the first-level verification are extended is set to K*. The feature word set and related semantic set obtained by the classification evaluation algorithm for each vulnerability data subset that has passed the first-level verification are respectively set as (S1, S2, ..., Si) and topic (C1, C2, ..., Cj), then K*={S1C1, S2C2, ..., SiCj}, where i is the total number of feature word sets and j is the total number of related semantic sets. If each vulnerability data subset after the second-level verification fails the classification evaluation algorithm verification, the vulnerability data subset that fails the second-level verification is returned to steps one, two and three for re-construction of knowledge graph, classification and screening and first-level conceptual feature algorithm verification processing. This process is repeated three times. After three cycles, if there are still vulnerability data subsets that fail the second-level verification, they are directly stored in the second-level verification failure database for manual review. Step 5: Data Vulnerability Decision Table Construction: If each subset of vulnerability data passes the second-level verification directly, and the subset of problematic vulnerability data after three cycles passes the second-level verification, they are directly summarized into the data vulnerability decision table.

2. The vulnerability type association analysis method based on knowledge graphs according to claim 1, characterized in that: In the classification and screening module of step two, the various vulnerability data subsets in the knowledge graph database are further classified using WordXet for semantic disambiguation, keyword extraction, and background corpus in various environments. Then, a Bayesian classifier is used to further classify and screen the various vulnerability data subsets after the classification.

3. The vulnerability type association analysis method based on knowledge graphs according to claim 2, characterized in that: In the conceptual feature first-level verification in step three, the formula for calculating P(c / i) is: P(c / i)=n(c,i) / ∑ i∈c n(c j ,i); P(i / c) = n(c, i) / ∑ i∈c n(c,i) j In the formula, n(c, i) is the number of times instance i and concept c appear simultaneously, ∑ i∈c n(c) j ∑ii (ii) represents the number of times instance i and all concepts contained in instance i appear simultaneously. i∈c n(c,i) j ) represents the number of times concept c and all instances contained in concept c appear simultaneously.

4. The vulnerability type association analysis method based on knowledge graphs according to claim 3, characterized in that: In the conceptual feature level-one verification in step three, the mutual information W verifies the correlation between variables in each subset of vulnerability data. It represents the correlation information of another variable encompassed within each subset of vulnerability data, and the calculation formula is as follows: W(i / c) = ∑P(i / c)log{(Pi / c) / P(i)P(c)}, where P(i / c) is the joint probability that instance i and concept c occur simultaneously.

5. The vulnerability type association analysis method based on knowledge graphs according to claim 4, characterized in that: In the conceptual feature level-one verification in step three, the point mutual information verification verifies the correlation between instances and concepts in each vulnerability data subset, and the calculation formula is as follows: PW(i / c) = log{(Pi / c) / P(i)P(c)} = logP(i / c) - logP(i), where logP(i) is a constant representing the association between instances and concepts in each vulnerability data subset, and the value of PW(i / c) depends on logP(i / c).

6. The vulnerability type association analysis method based on knowledge graphs according to claim 5, characterized in that: In the conceptual feature level-one verification in step three, the standardized point mutual information verification verifies the correlation score level between instances and concepts in each vulnerability data subset, and the calculation formula is as follows: NPW(i / c)=PW(i / c) / -logP(i / c)={logP(i / c)-logP(i)} / -logP(i / c); Then, we use PW raised to the power of X for comparison and argumentation, and the formula for the argumentation is: PW X (i / c)=logP(i / c) X / logP(i)P(c)。 7. The vulnerability type association analysis method based on knowledge graphs according to claim 6, characterized in that: In the conceptual feature level-one verification of step three, the conceptual feature algorithm verifies the feature correlation score level between instances and concepts in each vulnerability data subset, and the calculation formula is as follows: SPW(i / c) = arg max max RPW(i / c), and the formula for calculating RPW(i / c) is: RPW(i / c)=P(c / i)*P(i / c).

8. The vulnerability type association analysis method based on knowledge graphs according to claim 7, characterized in that: In the classification assessment in step four, the secondary verification classification assessment algorithm is as follows: Q = (1 + β) 2 )*H*V / H+V*β 2 +t, where H is the ratio of the number of vulnerabilities that pass the second verification in each vulnerability data subset to the total number of vulnerability data, V is the ratio of the number of vulnerabilities that fail the second verification in each vulnerability data subset to the total number of vulnerability data, β is the weight of H and V, and the value of β is 1, and t is the weighting addition of β, and t is 0.1.

Images