A vulnerability graph alignment method based on text and graph embedding interaction

Abstract

This invention relates to the field of network security technology and discloses a vulnerability graph alignment method based on text and graph embedding interaction, comprising the following steps: obtaining publicly available vulnerability data from the National Information Security Vulnerability Database and the National Information Security Vulnerability Sharing Platform; constructing a dataset for vulnerability entity alignment experiments; constructing a graph embedding representation model QuatAE based on the QuatE model; and introducing a new relation r. a Serving as a "connecting bridge" between two knowledge graphs, this vulnerability graph alignment method, based on text and graph embedding interaction, fully considers the characteristics of vulnerability graphs. It uses a unified model to comprehensively consider information at the entity, neighbor, and attribute levels, achieving good alignment results. Besides supporting vulnerability graphs, it is also compatible with entity alignment work for knowledge graphs in other domains because it incorporates information from various levels of the knowledge graph that affect knowledge alignment into the training process, demonstrating strong versatility.

Description

Technical Field

[0001] This invention relates to the field of network security technology, specifically to a vulnerability graph alignment method based on text and graph embedding interaction. Background Technology

[0002] Knowledge graph alignment technology aims to merge similar graphs from different sources to obtain a more comprehensive graph. Since the completeness of graph information greatly affects downstream applications related to knowledge graphs (such as question-answering systems and recommendation systems), entity alignment technology has received widespread attention from academia and industry. Traditional graph alignment techniques mainly rely on text matching or logical reasoning to align entities within the graph. With the development of artificial intelligence, cutting-edge achievements in knowledge representation learning have been introduced into entity alignment tasks. Utilizing graph structure features and text features learned by neural networks can enhance the effectiveness of entity alignment tasks. In entity alignment research, scholars mainly focus on general encyclopedic knowledge graph alignment, but encyclopedic knowledge graphs have rich semantic information, and there are still some differences compared to entity alignment in specialized fields. To address the rich semantic information brought by the graph structure in knowledge graphs, many entity alignment techniques utilize geometric models such as TransE and graph neural networks (GCN) to learn graph embeddings and use these embeddings to complete entity alignment. However, the interactions between entities in vulnerability knowledge graphs are not complex, and relying on graph structure information for entity alignment yields poor results. Therefore, we comprehensively considered the textual and structural information of entities and used the information of the entity and its neighboring entities to assist in completing the entity alignment task of the vulnerability graph.

[0003] Regarding existing patents, some entity alignment technologies have been proposed for threat intelligence, the financial industry, etc., but no relevant alignment technologies have been found in the field of vulnerability graphs. According to an entity alignment method in a knowledge graph of the "four insurances and one fund" domain published on the State Intellectual Property Office website (announcement number: CN112131404B), it uses TransE, LSTM, and a pre-trained BERT model to learn entity embedding representations, but it only considers entities and their attribute information.

[0004] Current methods primarily utilize graph structure features and textual information about entities and attributes for entity alignment. However, they only consider the information of the entity itself and do not fully leverage the contribution of its neighbors to entity alignment. Many techniques rely on TransE or GCN to learn graph embedding representations, but vulnerability graphs contain numerous one-to-many, many-to-one, and many-to-many relationship models. Using these models cannot effectively utilize graph structure information to distinguish between similar but different entity pairs. Furthermore, most existing techniques rely on graph structure features, but due to the unique nature of vulnerability graphs—a small number of relationships but a large number of entities—more information needs to be considered to achieve entity alignment. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the shortcomings of existing technologies, this invention provides a vulnerability graph alignment method based on the interaction of text and graph embedding. For vulnerability graphs with a large amount of similar vulnerability data, the proposed graph embedding model is first used to learn their graph structure features. Then, the graph structure features are concatenated with text embeddings learned based on the pre-trained language BERT, and fine-tuned using a small amount of labeled data. Subsequently, similarity scores from the perspectives of entity graph and text, entity neighbors, and entity attributes are combined, and these scores are interacted to obtain the final matching result. This technique is superior to methods that simply use text and graph structure features, and can not only achieve good results in vulnerability graph alignment, but can also be used for cross-language knowledge graph alignment.

[0007] (II) Technical Solution

[0008] To achieve the aforementioned excellent results in vulnerability graph alignment, this invention provides the following technical solution: a vulnerability graph alignment method based on text and graph embedding interaction, comprising the following steps:

[0009] S1, Vulnerability Knowledge Graph Construction Module

[0010] Publicly available vulnerability data was obtained from the National Information Security Vulnerability Database and the National Information Security Vulnerability Sharing Platform. A dataset was constructed for the vulnerability entity alignment experiment, which consisted of a set of relation triples and a set of attribute triples in the form of (entity h, relation r, entity t) and (entity h, attribute a, attribute value v), respectively.

[0011] S2, Graph Embedding Representation Module

[0012] Based on the QuatE model, a graph embedding representation model QuatAE is constructed, introducing a new relation r. a As a "connecting bridge" between two graphs, the alignment seed S is used. d and relation r a Add a set of training data The training objective is modified to make known aligned entity pairs closer to each other during training. To ensure that identical entities in the vulnerability graph are close together during training, the loss function for this module is designed as follows:

[0013]

[0014] S3, Entity Text-Graph Embedding Module

[0015] For the input entity text, the cross-lingual pre-trained model BERT is used to learn text information. The BERT CLS embedding is concatenated with the graph embedding learned by QuatAE, and then filtered through an MLP layer. Considering the differences between QuatAE embedding and BERT embedding, the QuatAE embedding is processed by an MLP layer for feature selection before concatenation.

[0016] G(e)=MLP(QuatAE(e))

[0017]

[0018] Using an alignment seed, randomly sample some negative samples as training data for fine-tuning BERT and QuatAE: D = {e, e} + e -}, where e + ∈S d , represents the same entity as e, while e - These are negative samples that are not identical to e. The sampling of negative sample pairs is calculated using the cosine similarity of the entity pairs. After obtaining the training data, fine-tuning is performed using pairwise edge loss, with the loss function as follows:

[0019]

[0020] S4, Multi-view Interactive Alignment Module

[0021] During alignment, considering neighbors and attributes makes it easier to distinguish entities that are similar in graph structure and name but are actually different. By incorporating neighbor and attribute perspectives and interacting with the graph-text perspective from the previous module, the final alignment result is obtained. First, from the neighbor perspective, for a given candidate matching entity pair (e... i e j The previous module can be used to obtain the embedding representations of their respective neighbor entities, which can then be used to calculate a similarity matrix Sr between neighbor entities. Any element s in the matrix... x，y Represents entity e i The xth neighbor and entity e j The similarity of the y-th neighbor;

[0022] Considering that the similarity of relationships between entities and their neighbors also affects the entity alignment effect—that is, entity pairs with similar neighbors and similar corresponding relationships are more likely to be truly aligned entity pairs—a neighbor relationship similarity matrix is ​​calculated using the text embedding of the relationship as a mask to improve the reliability of the neighbor perspective. The text embedding of the relationship is also obtained through an MLP layer using BERT's CLS embedding:

[0023] TC(r) = MLP(CLS(r))

[0024]

[0025] Where, m x，y Represents entity e i The x-th relation and entity e j The similarity of the y-th relation, these relation similarities can form a relation mask matrix M, and the neighbor similarity matrix s r Calculate the Hadamard product between the Hadamard matrix and the corresponding relation mask matrix M to obtain the final similarity matrix:

[0026] S′ r =S r ⊙M;

[0027] To more accurately utilize similar neighbor pairs, we take S′ r The maximum value of each row is used as the input to the RBF kernel function, meaning only e is of interest. i With e i How similar are the most likely pairs of entities that are the same entity among the neighbors? From this, we can obtain the similarity of entity e. i and entity e j S′ from the neighbor's perspective r Set of similarity scores for each row and each column:

[0028]

[0029] RBFr(·) aggregates the row elements of the similarity matrix, and RBFc(·) aggregates the column elements of the similarity matrix.

[0030] Preferably, in step S1, the National Information Security Vulnerability Database defines 13 entities, 16 relationships, and 4 attributes, while the National Information Security Vulnerability Sharing Platform defines 8 entities, 10 relationships, and 6 attributes. The vulnerability ontology of the National Information Security Vulnerability Database can basically cover all the ontology of the National Information Security Vulnerability Sharing Platform.

[0031] Preferably, in step S2, an r-value representing the alignment relationship is created. a And using the alignment seed S d and relation r a Add a set of training data

[0032] Preferably, in step S2, a graph embedding model QuatAE for entity alignment is proposed based on QuatE.

[0033] Preferably, in step S3, d(·) uses L1 distance to calculate the similarity between TG(e) and TG(e+) or TG(e-), and m is a marginal parameter that limits the distance between positive and negative sample pairs.

[0034] Preferably, in step S3, since the entity's description provides more information than its name, the description is prioritized as input. If the description is missing, the entity's name is used as input. After training by this module, for a given entity pair (e... i e j The similarity score can be calculated by comparing the cosine similarity between entities:

[0035]

[0036] By calculating the similarity between candidate entity pairs, for each entity e i K candidate alignment entities are initially selected for further refinement in the next module.

[0037] Preferably, in step S4, from the attribute perspective, the input relation triples from the neighbor perspective are replaced with attribute triples. We can obtain entity e using a similar method. i and entity e j A set of similarity scores from the perspective of neighbors The only difference is that the input data from the attribute perspective is the set of attribute values ​​of the target entity connected by attribute a, while the input data from the neighbor perspective is the set of entities of the target entity connected by relation r.

[0038] Preferably, in step S4, for a given entity pair (e) i e j By combining similarity score sets from the text / image perspective, the neighbor perspective, and the attribute perspective, these scores are concatenated and input into the MLP layer to obtain the final similarity score for the interaction of the three perspectives.

[0039]

[0040] (III) Beneficial Effects

[0041] Compared with existing technologies, this invention provides a vulnerability graph alignment method based on text and graph embedding interaction, which has the following beneficial effects:

[0042] 1. This vulnerability graph alignment method based on text and graph embedding interaction fully considers the characteristics of vulnerability graphs. It uses a unified model to comprehensively consider information at the entity, neighbor, and attribute levels, achieving good alignment results. In addition to supporting vulnerability graphs, it can also be compatible with entity alignment work of knowledge graphs in other domains because it incorporates information from all levels of knowledge graphs that affect knowledge alignment into the training, thus having strong versatility.

[0043] 2. This vulnerability graph alignment method based on text and graph embedding interaction proposes a graph embedding model designed for the characteristics of vulnerability graphs. The graph structure features learned by the model can assist entity alignment tasks, especially for some entity pairs that are similar in text but different in reality, it can provide deeper information. At the same time, graph embedding is introduced for auxiliary training, which achieves more effective alignment of entity pairs. The embedding representations output by the three modules can all be used for entity alignment tasks. In practical application scenarios, you can choose according to your own needs. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the process of the present invention;

[0045] Figure 2 This is a schematic diagram illustrating the application process of the model trained according to the present invention;

[0046] Figure 3 This is a schematic diagram of vulnerability map alignment for the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Please see Figure 1-3 This invention provides a technical solution: a vulnerability graph alignment method based on text and graph embedding interaction, comprising the following steps:

[0049] S1, Vulnerability Knowledge Graph Construction Module

[0050] Publicly available vulnerability data was obtained from the National Information Security Vulnerability Database and the National Information Security Vulnerability Sharing Platform. A dataset was constructed for the vulnerability entity alignment experiment, which consisted of a set of relation triples and a set of attribute triples in the form of (entity h, relation r, entity t) and (entity h, attribute a, attribute value v), respectively.

[0051] The National Information Security Vulnerability Database defines 13 entities, 16 relationships, and 4 attributes, while the National Information Security Vulnerability Sharing Platform defines 8 entities, 10 relationships, and 6 attributes. The vulnerability ontology of the National Information Security Vulnerability Database basically covers all the ontology of the National Information Security Vulnerability Sharing Platform.

[0052] The specific descriptions of entities, relationships, and attributes are shown below:

[0053]

[0054]

[0055] S2, Graph Embedding Representation Module

[0056] 1. Based on the QuatE model, a graph embedding representation model QuatAE is constructed, introducing a new relation ra as a "connection bridge" between two graphs, and utilizing the alignment seed S. d ={(e, e′)|e∈ε1, e′∈ε2, e≡e′} and relation r a Add a set of training data The training objective is modified to make known aligned entity pairs closer to each other during training. To ensure that identical entities in the vulnerability graph are close together during training, the loss function for this module is designed as follows:

[0057]

[0058] From The sampled negative sample triples, l∈{-1,1}, indicate whether the triple is a negative sample. The negative samples are sampled using adversarial sampling and Bernoulli sampling methods. λ1 and λ2 are hyperparameters.

[0059] S3, Entity Text-Graph Embedding Module

[0060] For the input entity text, the cross-lingual pre-trained model BERT is used to learn text information. The BERT CLS embedding is concatenated with the graph embedding learned by QuatAE, and then filtered through an MLP layer. Considering the differences between QuatAE embedding and BERT embedding, the QuatAE embedding is processed by an MLP layer for feature selection before concatenation.

[0061] G(e)=MLP(QuatAE(e))

[0062]

[0063] Using an alignment seed, randomly sample some negative samples as training data for fine-tuning BERT and QuatAE: D = {e, e} + e -}, where e + ∈S d , represents the same entity as e, while e - These are negative samples that are not identical to e. The sampling of negative sample pairs is calculated using the cosine similarity of the entity pairs. After obtaining the training data, fine-tuning is performed using pairwise edge loss, with the loss function as follows:

[0064]

[0065] d(·) uses L1 distance to calculate the similarity between TG(e) and TG(e+) or TG(e-), where m is a marginal parameter that limits the distance between positive and negative sample pairs.

[0066] Since the entity's description provides more information than its name, the description is prioritized as input. If the description is missing, the entity's name is used as input. After training with this module, for a given entity pair (e... i e j The similarity score can be calculated by comparing the cosine similarity between entities:

[0067]

[0068] By calculating the similarity between candidate entity pairs, for each entity e i K candidate alignment entities are initially selected for further refinement in the next module.

[0069] S4, Multi-view Interactive Alignment Module

[0070] During alignment, considering neighbors and attributes makes it easier to distinguish entities that are similar in graph structure and name but are actually different. By incorporating neighbor and attribute perspectives and interacting with the graph-text perspective from the previous module, the final alignment result is obtained. First, from the neighbor perspective, for a given candidate matching entity pair (e... i e j The previous module can be used to obtain the embedding representations of their respective neighbor entities, which can then be used to calculate a similarity matrix Sr between neighbor entities. Any element s in the matrix... x，y Represents entity e i The xth neighbor and entity e j The similarity of the y-th neighbor;

[0071] Considering that the similarity of relationships between entities and their neighbors also affects the entity alignment effect—that is, entity pairs with similar neighbors and similar corresponding relationships are more likely to be truly aligned entity pairs—a neighbor relationship similarity matrix is ​​calculated using the text embedding of the relationship as a mask to improve the reliability of the neighbor perspective. The text embedding of the relationship is also obtained through an MLP layer using BERT's CLS embedding:

[0072] TC(r) = MLP(CLS(r))

[0073]

[0074] Where, m x，y Represents entity e i The x-th relation and entity e jThe similarity of the y-th relation, these relation similarities can form a relation mask matrix M, and the neighbor similarity matrix S is... r Calculate the Hadamard product between the Hadamard matrix and the corresponding relation mask matrix M to obtain the final similarity matrix:

[0075] S′ r =S r ⊙M;

[0076] To more accurately utilize similar neighbor pairs, we take S′ r The maximum value of each row is used as the input to the RBF kernel function, meaning only e is of interest. i With e j How similar are the most likely pairs of entities that are the same entity among the neighbors? From this, we can obtain the similarity of entity e. i and entity e j S′ from the neighbor's perspective r Set of similarity scores for each row and each column:

[0077]

[0078] RBFr(·) aggregates the row elements of the similarity matrix, and RBFc(·) aggregates the column elements of the similarity matrix.

[0079] From an attribute perspective, by replacing the input relation triples from the neighbor perspective with attribute triples, we can obtain entity e using a similar method. i and entity e j A set of similarity scores from the perspective of neighbors The only difference is that the input data from the attribute perspective is the set of attribute values ​​of the target entity connected by attribute a, while the input data from the neighbor perspective is the set of entities of the target entity connected by relation r.

[0080] For a given pair of entities (e i e j By combining similarity score sets from the text / image perspective, the neighbor perspective, and the attribute perspective, these scores are concatenated and input into the MLP layer to obtain the final similarity score for the interaction of the three perspectives.

[0081]

[0082] This vulnerability graph alignment method, based on the interaction of text and graph embedding, first learns the graph structure features of a vulnerability graph with a large amount of similar vulnerability data using the proposed graph embedding model. Then, it concatenates the graph structure features with text embeddings learned based on the pre-trained language BERT, and fine-tunes them using a small amount of labeled data. Subsequently, it combines similarity scores from the perspectives of entity graph and text, entity neighbors, and entity attributes, and interacts these scores to obtain the final matching result. This technique outperforms methods that simply use text and graph structure features, and can not only achieve good results in vulnerability graph alignment, but also be used for cross-language knowledge graph alignment.

[0083] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A vulnerability graph alignment method based on text and graph embedding interaction, characterized in that, Includes the following steps: S1, Vulnerability Knowledge Graph Construction Module Publicly available vulnerability data were obtained from the National Information Security Vulnerability Database and the National Information Security Vulnerability Sharing Platform. A dataset for vulnerability entity alignment experiments was constructed, which consisted of a set of relation triples and a set of attribute triples in the form of (entity h, relation r, entity t) and (entity h, attribute a, attribute value v), respectively. S2, Graph Embedding Representation Module Based on the QuatE model, a graph embedding representation model QuatAE is constructed, introducing a new relation r. a As a "connecting bridge" between two graphs, the alignment seed S is used. d and relation r a Add a set of training data The training objective has been modified, and the loss function for this module is designed as follows: ； S3, Entity Text-Graph Embedding Module For the input entity text, the cross-lingual pre-trained model BERT is used to learn text information. The BERT CLS embedding is concatenated with the graph embedding learned by QuatAE, and then filtered through an MLP layer. Considering the differences between QuatAE embedding and BERT embedding, the QuatAE embedding is processed by an MLP layer for feature selection before concatenation. ， Using an alignment seed, some negative samples are randomly sampled as training data for fine-tuning BERT and QuatAE: ,in , indicating with The same entity, and To and For distinct negative samples, the sampling calculation of negative sample pairs is performed using the cosine similarity of entity pairs. After obtaining the training data, pairwise edge loss is used for fine-tuning, and the loss function is as follows: ； S4, Multi-view Interactive Alignment Module During alignment, considering neighbors and attributes makes it easier to distinguish entities that are similar in graph structure and name but are actually different. By incorporating neighbor and attribute perspectives and interacting with the graph-text perspective from the previous module, the final alignment result is obtained. First, from the neighbor perspective, for a given pair of candidate matching entities... The previous module obtains the embedding representations of their respective neighboring entities, which are then used to calculate a similarity matrix Sr between neighboring entities, where each element represents an entity. The xth neighbor and entity The similarity of the y-th neighbor; The text embedding of the relation is also obtained through an MLP layer using BERT's CLS embedding: in, Representing entities The x-th relation and entity The similarity of the y-th relation, these relation similarities can form a relation mask matrix M, and the neighbor similarity matrix is ​​used to... Calculate the Hadamard product between the Hadamard matrix and the corresponding relation mask matrix M to obtain the final similarity matrix: ； Pick The maximum value of each row is used as the input to the RBF kernel function, meaning only the maximum value of each row is considered. and How similar are the most likely pairs of entities that are the same entity among the neighbors? From this, we can obtain the entity... and entity From the neighbor's perspective Set of similarity scores for each row and each column: where RBF r (·) aggregating the row elements of the similarity matrix, RBF c (·) aggregating the column elements of the similarity matrix.

2. The vulnerability graph alignment method based on text and graph embedding interaction according to claim 1, characterized in that, In step S1, the National Information Security Vulnerability Database defines 13 entities, 16 relationships, and 4 attributes, while the National Information Security Vulnerability Sharing Platform defines 8 entities, 10 relationships, and 6 attributes. The vulnerability ontology of the National Information Security Vulnerability Database covers all the ontology of the National Information Security Vulnerability Sharing Platform.

3. The vulnerability graph alignment method based on text and graph embedding interaction according to claim 1, characterized in that, In step S3, d(·) uses L1 distance to calculate the similarity between TG(e) and TG(e+) or TG(e−), and m is a marginal parameter that limits the distance between positive and negative sample pairs.

4. The vulnerability graph alignment method based on text and graph embedding interaction according to claim 1, characterized in that, In step S3, since the entity's description provides more information than its name, the description is prioritized as input. If the description is missing, the entity's name is used as input. After training by this module, for a given pair of entities... The similarity score is calculated by comparing the cosine similarity between entities: By calculating the similarity between candidate entity pairs, for each entity... K candidate alignment entities are initially selected for further refinement in the next module.

5. The vulnerability graph alignment method based on text and graph embedding interaction according to claim 4, characterized in that, In step S4, from an attribute perspective, the input relation triples from the neighbor perspective are replaced with attribute triples. The input data from the attribute perspective is the set of attribute values ​​connected by attribute 'a' for the target entity, while the input data from the neighbor perspective is the set of entities connected by relation 'r' for the target entity, thus obtaining the entity... and entity A set of similarity scores from the perspective of neighbors .

6. The vulnerability graph alignment method based on text and graph embedding interaction according to claim 5, characterized in that, In step S4, for a given entity pair By combining similarity scores from text / image perspectives, neighbor perspectives, and attribute perspectives, and concatenating them before inputting them into an MLP layer, the final similarity score for the interaction of the three perspectives is obtained. 。

Images