Entity alignment method based on heterogeneous knowledge graph

Abstract

The invention discloses an entity alignment method based on heterogeneous knowledge maps, which belongs to the field of knowledge maps and comprises the following steps: acquiring and processing data of two heterogeneous knowledge maps; obtaining embedding of entity vectors in the knowledge graph by utilizing a pre-trained large language model; learning and modeling the features of the related entity relationship by using a graph neural network, and capturing potential correlation and structure in the knowledge graph; and fusing the two heterogeneous knowledge maps to obtain a knowledge map entity alignment result. According to the method, the entity and the relationship are embedded by using the large language model, and the semantics of the entity and the relationship is enhanced, so that the entity alignment effect can be better improved.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and in particular to an entity alignment method based on heterogeneous knowledge graphs. Background Art

[0002] In recent years, a large number of knowledge graphs (KGs) have emerged, such as YAGO, Dbpedia, and NELL. These large-scale knowledge graphs play an important role in intelligent services such as question-answering systems and personalized recommendations. In addition, in order to meet the needs of specific fields, more and more domain knowledge graphs have been derived, such as medical knowledge graphs. In the process of knowledge graph construction, it is inevitable to make a trade-off between coverage and accuracy. However, no knowledge graph can be complete or completely correct.

[0003] In order to improve the coverage and accuracy of knowledge graphs, a feasible method is to introduce relevant knowledge from other knowledge graphs, because there is knowledge redundancy and complementarity between knowledge graphs constructed in different ways. For example, a general knowledge graph extracted from web pages may only contain the names of drugs, while more information can be found in a medical knowledge graph built based on medical data. In order to integrate knowledge from external knowledge graphs into the target knowledge graph, the most important step is to align different knowledge graphs. To this end, the entity alignment (EA) task was proposed and received widespread attention. This task aims to find entity pairs that express the same meaning in different knowledge graphs. These entity pairs serve as hubs to link different knowledge graphs and serve subsequent tasks.

[0004] At present, mainstream entity alignment methods mainly use the structural features of knowledge graphs to determine whether two entities refer to the same thing. This type of method assumes that entities expressing the same meaning in different knowledge graphs have similar adjacency information. This type of method has achieved the best experimental results on artificially constructed datasets. However, the knowledge graphs in these artificially constructed datasets are denser than the knowledge graphs in the real world, and the entity alignment method based on structural features is much less effective on knowledge graphs with normal distribution.

[0005] In fact, by analyzing the entity distribution in the real-world knowledge graph, we can see that more than half of the entities are only connected to one or two other entities. These entities are called long-tail entities, which occupy most of the knowledge graph entities, making the graph as a whole highly sparse. This is also consistent with the understanding of the real-world knowledge graph: only a small number of entities are frequently used and have rich adjacency information; the vast majority of entities are rarely mentioned and contain little structural information. Therefore, the current entity alignment methods based on structural information do not perform well on real-world datasets. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides an entity alignment method based on heterogeneous knowledge graphs with simple operation and good alignment effect.

[0007] The technical solution of the present invention to solve the above technical problem is: an entity alignment method based on heterogeneous knowledge graph, comprising the following steps: Step S1: Obtain and process the data of two heterogeneous knowledge graphs; In step S1, two heterogeneous knowledge graphs and Expressed as and ,in represent Entity, represent Entity, represent The relationship between entities, represent The relationship between entities, represent The triples in represent The triples in ; the known entity pairs are represented as , , and Respectively and The Entity; In the step S1, polar coordinate space is selected as the knowledge representation space for entity alignment: for the data of the two heterogeneous knowledge graphs obtained, the relationship between the two heterogeneous knowledge graphs is processed by polar coordinate changes, and the corresponding data set is generated after processing; For two entities and , by introducing the point mutual information similarity space to calculate the similarity between the two: ; in, Representing Entities and entities By calculating the similarity of logarithm, the numerator Indicates that in a given entity Under the condition of entity The probability of occurrence, which is a conditional probability used to measure the probability of In the case The possibility of occurrence, Representing Entities The unconditional probability of occurrence, that is, in the absence of any other information The probability of occurrence, From , From , and is the knowledge graph of two entities to be aligned, sim represents the computational entity and entities Logarithmic similarity function; Multiple Combine to form a vector : ; As the attention parameter vector for the next step, and continuously fine-tune its value through learning; Step S2: Use the pre-trained large language model to obtain the embedding of entity vectors in the knowledge graph; Step S3: Use graph neural networks to learn and model the features of relevant entity relationships to capture the potential correlations and structures within the knowledge graph; Step S4: Fuse the two heterogeneous knowledge graphs to obtain the result of knowledge graph entity alignment.

[0008] In the above entity alignment method based on heterogeneous knowledge graph, in step S1, for entity alignment of knowledge graph, in addition to considering the similarity between entities, the physical structure of the entire knowledge graph must also be considered. The scores of similar entities are: ; ; in, yes exist The similarity score of the entire graph structure level is yes exist The similarity score of the entire graph structure is calculated; after calculation, a score on the corresponding graph is obtained, thereby realizing the score of the similarity of the calculation graph structure: ; in, express and The logarithmic similarity of Indicates that in a given entity Under the condition of entity The probability of occurrence, which is a conditional probability used to measure the probability of In the case of The possibility of occurrence, Representing Entities The unconditional probability of occurrence, that is, in the absence of any other information The probability of occurrence, In fact, it calculates the logarithmic similarity between nodes in the entire graph, that is, obtains a global feature.

[0009] In the above-mentioned entity alignment method based on heterogeneous knowledge graph, in step S2, in the HAKE method, the modeling of entities and relationships is achieved by mapping entities into a polar coordinate representation space under the action of relationships. Specifically, each entity is represented as a point in a polar coordinate system, and the relationship defines the transformation from one entity to another. In this way, the HAKE method captures the complex relationships between entities and performs reasoning and prediction in the knowledge graph; for each triple Calculates a score function : ; in , It represents two entities. By tensor decomposition The two components produced are By tensor decomposition The two components generated, r represents the , The relationship between and are the two components generated by tensor decomposition r; represents a hyperparameter; It means to calculate the p-ary norm, where It means that the calculated p-norm is 1, that is, the absolute value between entities is found, and L2 means that the calculated p-norm is 2, that is, the square root between entities; The calculation of entities is modeled in the above manner by calculating the polar coordinate changes between entities.

[0010] In the above entity alignment method based on heterogeneous knowledge graph, in step S2, a large language model is used to embed words between entities in the knowledge graph: ; in It is a word embedding matrix generated by the pre-trained large language model BERT, which represents the constituent units of words; It represents a set of words. Indicates that the current The word embedding result obtained by the entity is Represents a given prefix sequence hour The probability of occurrence; Represents the real entity. The formula describes a probability approximation, which is used as the attention weight matrix in the next step. ; The entity is activated by the LeakReLU activation function and entities The weight coefficient obtained after word embedding processing is called the attention weight coefficient. Through the entity and entities The word embedding is linearly combined and the LeakyReLU activation function is applied to measure the entity and entities The semantic similarity is yes The transpose of represents the activation function, and Respectively represent the entity and entities Perform word embedding operations. It means that and Perform splicing operation, splicing operation refers to and Combined into a matrix.

[0011] In the above entity alignment method based on heterogeneous knowledge graph, in step S3, a loss function based on log-likelihood is applied. As the optimization goal of the knowledge graph HAKE polar coordinate transformation model: ; in are two hyperparameters, is the number of samples of triples, >0, is a balancing hyperparameter, represents the set of all triples, represents the set generated by negative sampling of samples during training, It represents the error triples in the knowledge graph. represents the probability of the correct triple, represents the probability of a false triple.

[0012] In the above entity alignment method based on heterogeneous knowledge graph, in step S3, a graph neural network is used to learn and model the characteristics of the relevant entity relationships considering their topological results: ; in Is Entity and entities The normalized weight coefficient obtained after word embedding processing; It represents the node The set of neighbor nodes of is an adjustable hyperparameter whose value range is , to balance and The value of For all The sum of For the attention weights of graph neural networks, the purpose is to calculate the weighted sum from the nodes : ; in is the activation function.

[0013] In the above entity alignment method based on heterogeneous knowledge graph, in step S3, in a specific relationship triple The optimization goal is As shown below: ; Select the approximate loss function of the model To solve: ; That is, the calculated loss function is the Euclidean space of the two real components of the polar radius, which is used as the approximate loss function of the specific current L; ; The loss function of the model is div Find the divergence, and Indicates the partial derivative, which speeds up the calculation of the loss function. Indicates the polar diameter: ; The loss function is the sum of two linear parts.

[0014] The beneficial effects of the present invention are: 1. The present invention adopts advanced large language model technology, which not only enriches their semantic information by deeply embedding entities and relationships, but also significantly improves the accuracy and efficiency of entity alignment. Through the powerful semantic understanding ability of the large language model, the present invention can more accurately identify entities with the same name in different data sources, thereby achieving more accurate information integration and knowledge fusion.

[0015] 2. In addition, the present invention innovatively applies the graph neural network training method, which not only enhances the model's predictive ability in complex relational networks, but also significantly improves the model's relational completion performance in knowledge graphs. Graph neural networks can effectively capture the correlation between entities and the hierarchical structure between relationships, making the present invention more robust and generalizable when processing large-scale knowledge graphs. The application of this technology provides strong support for the construction and optimization of knowledge graphs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0017] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0018] like Figure 1 As shown, an entity alignment method based on heterogeneous knowledge graph includes the following steps: Step S1: Obtain and process the data of two heterogeneous knowledge graphs.

[0019] In step S1, two heterogeneous knowledge graphs and Expressed as and ,in represent Entity, represent Entity, represent The relationship between entities, represent The relationship between entities, represent The triples in represent The triples in ; the known entity pairs are represented as , , and Respectively and Entity alignment aims to use known entity pair information to find new entity pairs and generate the final alignment result. , where the equal sign represents that the two entities refer to the same real-world entity; given an entity, the process of finding its corresponding entity in another knowledge graph can be regarded as a ranking problem. That is, in a certain feature space, the similarity (distance) between a given entity and all entities in another knowledge graph is calculated and ranked, and the entity with the highest similarity (smallest distance) can be regarded as the alignment result.

[0020] Taking the medical knowledge graph as an example, in order to obtain more medical knowledge, multiple independent medical knowledge graphs can be integrated. In order to better integrate the medical knowledge graph, it is necessary to identify the entities therein, including the names of drugs, diseases, and symptoms. These three categories are the most basic entities of the medical knowledge graph. Aligning these three categories of entities is the most basic requirement of the medical knowledge graph. The extraction of other entities can be determined according to actual needs.

[0021] The polar coordinate space is selected as the knowledge representation space for entity alignment: for the data of the two heterogeneous knowledge graphs obtained, the relationship between the two heterogeneous knowledge graphs is processed through polar coordinate changes, and the corresponding data set is generated after processing.

[0022] For the data of the two heterogeneous knowledge graphs obtained, the relationship between the two heterogeneous knowledge graphs is processed, and the corresponding data sets are generated after processing, specifically: For two entities and , by introducing the point mutual information similarity space to calculate the similarity between the two: ; in, Representing Entities and entities The similarity calculated by logarithm is Indicates that in a given entity Under the condition of entity The probability of occurrence, which is a conditional probability used to measure the probability of In the case The possibility of occurrence, Representing Entities The unconditional probability of occurrence, that is, in the absence of any other information The probability of occurrence, From , From , and is the knowledge graph of two entities to be aligned, sim represents the computational entity and entities Logarithmic similarity function; multiple Combine to form a vector : ; As the attention parameter vector for the next step, and continuously fine-tune its value through learning.

[0023] For entity alignment of knowledge graph, in addition to considering the similarity between entities, the physical structure of the entire knowledge graph must also be considered. The scores of similar entities are: ; ; in, yes exist The similarity score of the entire graph structure is yes exist The similarity score of the entire graph structure is calculated; after calculation, a score on the corresponding graph is obtained, thereby realizing the score of the similarity of the calculation graph structure: ; in, express and The logarithmic similarity of Indicates that in a given entity Under the condition of entity The probability of occurrence, which is a conditional probability used to measure the probability of In the case of Possibility of occurrence. Representing Entities The unconditional probability of occurrence, that is, in the absence of any other information The probability of occurrence, In fact, it calculates the logarithmic similarity between nodes in the entire graph, that is, obtains a global feature.

[0024] Step S2: Use the pre-trained large language model to obtain the embedding of entity vectors in the knowledge graph.

[0025] In the HAKE method, the modeling of entities and relations is achieved by mapping entities into a polar coordinate representation space under the action of relations. Specifically, each entity is represented as a point in the polar coordinate system, and the relationship defines the transformation from one entity to another. In this way, the HAKE method captures the complex relationships between entities and performs reasoning and prediction in the knowledge graph; for each triple Calculates a score function : ; in , It represents two entities. By tensor decomposition The two components produced are By tensor decomposition The two components generated, r represents the , The relationship between and are the two components generated by tensor decomposition r; represents a hyperparameter; It means to calculate the p-ary norm, where Indicates that the p-norm of the calculation is 1, that is, the absolute value between entities is calculated, and L2 indicates that the p-norm of the calculation is 2, that is, the square root between entities. In the above way, the calculation of entities is modeled by calculating the polar coordinate changes between entities.

[0026] Use a large language model to embed words between entities in the knowledge graph: ; in It is a word embedding matrix generated by the pre-trained large language model BERT, which represents the constituent units of words; It represents a set of words. Indicates that the current The word embedding result obtained by the entity is Represents a given prefix sequence hour The probability of occurrence; Represents the real entity. The formula describes a probability approximation, which is used as the attention weight matrix in the next step. ; The entity is activated by the LeakReLU activation function and entities The weight coefficient obtained after word embedding processing is called the attention weight coefficient. Through the entity and entities The word embedding is linearly combined and the LeakyReLU activation function is applied to measure the entity and entities The semantic similarity is yes The transpose of represents the activation function, and Respectively represent the entity and entities Perform word embedding operations. It means that and Perform splicing operation, splicing operation refers to and Combined into a matrix.

[0027] Step S3: Use graph neural networks to learn and model the features of relevant entity relationships to capture the potential correlations and structures within the knowledge graph.

[0028] Apply a loss function based on log-likelihood As the optimization goal of the knowledge graph HAKE polar coordinate transformation model: ; in are two hyperparameters, is the number of samples of triples, >0, is a balancing hyperparameter, represents the set of all triples, represents the set generated by negative sampling of samples during training, It represents the error triples in the knowledge graph. represents the probability of the correct triple, represents the probability of a false triple.

[0029] Graph neural networks are used to learn and model the characteristics of relevant entity relationships considering their topological results: ; in Is Entity and entities The normalized weight coefficient obtained after word embedding processing; It represents the node The set of neighbor nodes of is an adjustable hyperparameter whose value range is , to balance and The value of For all The sum of For the attention weights of graph neural networks, the purpose is to calculate the weighted sum from the nodes : ; in is the activation function, where Refers to the normalization operation, which is a key step for neural network architectures with attention mechanisms, especially graph neural networks. Its function is to weightedly aggregate the features of neighbor nodes according to the attention weights, thereby updating the representation of the current node.

[0030] In a specific relation triple The optimization goal is As shown below: ; ,in It represents a complex number, which is a linear relationship between real and imaginary parts. When minimizing L, because L is a complex number, there is no analytical solution in most cases, so only approximate solutions can be used. This is because for complex numbers, they may not meet the condition of being analyzable, and even if they are analyzable, the specific minimization process is extremely complicated. For simplicity, the approximate loss function of the modulus is used. To solve: ; That is, the calculated loss function is the Euclidean space of the two real components of the polar radius, which is used as the approximate loss function of the specific current L; ; The loss function using the modulus is Find the divergence and speed up the calculation of the loss function: ; The loss function is the sum of two linear parts.

[0031] Step S4: Fuse the two heterogeneous knowledge graphs to obtain the result of knowledge graph entity alignment.

Claims

1. An entity alignment method based on heterogeneous knowledge graph, characterized in that: The following steps are involved: Step S1: Obtain and process the data of two heterogeneous knowledge graphs; In step S1, two heterogeneous knowledge graphs and Expressed as and ,in represent Entity, represent Entity, represent The relationship between entities, represent The relationship between entities, represent The triples in represent The triples in ; The known entity pairs are represented as , , and Respectively and The Entity; In the step S1, polar coordinate space is selected as the knowledge representation space for entity alignment: for the data of the two heterogeneous knowledge graphs obtained, the relationship between the two heterogeneous knowledge graphs is processed by polar coordinate changes, and the corresponding data set is generated after processing; For two entities and , by introducing the point mutual information similarity space to calculate the similarity between the two: ； in, Representing Entities and entities By calculating the similarity of logarithm, the numerator Indicates that in a given entity Under the condition of entity The probability of occurrence, which is a conditional probability used to measure the probability of In the case The possibility of occurrence, Representing Entities The unconditional probability of occurrence, that is, in the absence of any other information The probability of occurrence, From , From , and is the knowledge graph of two entities to be aligned, sim represents the computational entity and entities Logarithmic similarity function; Multiple Combine to form a vector : ； As the attention parameter vector for the next step, and continuously fine-tune its value through learning; Step S2: Use the pre-trained large language model to obtain the embedding of entity vectors in the knowledge graph; Step S3: Use graph neural networks to learn and model the features of relevant entity relationships to capture the potential correlations and structures within the knowledge graph; Step S4: Fuse the two heterogeneous knowledge graphs to obtain the result of knowledge graph entity alignment.

2. The entity alignment method based on heterogeneous knowledge graph according to claim 1 is characterized in that: In step S1, for entity alignment of the knowledge graph, in addition to considering the similarity between entities, the physical structure of the entire knowledge graph must also be considered. The scores of similar entities are: ； ； in, yes exist The similarity score of the entire graph structure is yes exist The similarity score of the entire graph structure is calculated; after calculation, a score on the corresponding graph is obtained, thereby realizing the score of the similarity of the calculation graph structure: ； in, express and The logarithmic similarity of Indicates that in a given entity Under the condition of entity The probability of occurrence, which is a conditional probability used to measure the probability of In the case of The possibility of occurrence, Representing Entities The unconditional probability of occurrence, that is, in the absence of any other information The probability of occurrence, In fact, it calculates the logarithmic similarity between nodes in the entire graph, that is, obtains a global feature.

3. The entity alignment method based on heterogeneous knowledge graph according to claim 2 is characterized in that: In step S2, in the HAKE method, the modeling of entities and relationships is achieved by mapping entities into a polar coordinate representation space under the action of relationships. Specifically, each entity is represented as a point in a polar coordinate system, and the relationship defines the transformation from one entity to another. In this way, the HAKE method captures the complex relationships between entities and performs reasoning and prediction in the knowledge graph; for each triple Calculates a score function : ； in , It represents two entities. By tensor decomposition The two components produced are By tensor decomposition The two components generated, r represents the , The relationship between and are the two components generated by tensor decomposition r; represents a hyperparameter; It means to calculate the p-ary norm, where It means that the calculated p-norm is 1, that is, the absolute value between entities is found, and L2 means that the calculated p-norm is 2, that is, the square root between entities; The calculation of entities is modeled in the above manner by calculating the polar coordinate changes between entities.

4. The entity alignment method based on heterogeneous knowledge graph according to claim 3 is characterized in that: In step S2, a large language model is used to embed words between entities in the knowledge graph: ； in It is a word embedding matrix generated by the pre-trained large language model BERT, which represents the constituent units of words; It represents a set of words. Indicates that the current The word embedding result obtained by the entity is Represents a given prefix sequence hour The probability of occurrence; Represents the real entity. The formula describes a probability approximation, which is used as the attention weight matrix in the next step. ； The entity is activated by the LeakReLU activation function and entities The weight coefficient obtained after word embedding processing is called the attention weight coefficient. Through the entity and entities The word embedding is linearly combined and the LeakyReLU activation function is applied to measure the entity and entities The semantic similarity is yes The transpose of represents the activation function, and Respectively represent the entity and entities Perform word embedding operations. It means that and Perform splicing operation, splicing operation refers to and Combined into a matrix.

5. The entity alignment method based on heterogeneous knowledge graph according to claim 4 is characterized in that: In step S3, a loss function based on log-likelihood is applied As the optimization goal of the knowledge graph HAKE polar coordinate transformation model: ； in are two hyperparameters, is the number of samples of triples, >0, is a balancing hyperparameter, represents the set of all triples, represents the set generated by negative sampling of samples during training, It represents the error triples in the knowledge graph. represents the probability of the correct triple, represents the probability of a false triple.

6. The entity alignment method based on heterogeneous knowledge graph according to claim 4 is characterized in that: In step S3, the graph neural network is used to learn and model the relevant entity relationships taking into account the characteristics of their topological results: ； in Is Entity and entities The normalized weight coefficient obtained after word embedding processing; It represents the node The set of neighbor nodes of is an adjustable hyperparameter whose value range is , to balance and The value of For all The sum of For the attention weights of graph neural networks, the purpose is to calculate the weighted sum from the nodes : ； in is the activation function.

7. The entity alignment method based on heterogeneous knowledge graph according to claim 5 is characterized in that: In step S3, in a specific relationship triple The optimization goal is As shown below: ； Select the approximate loss function of the model To solve: ； That is, the calculated loss function is the Euclidean space of the two real components of the polar radius, which is used as the approximate loss function of the specific current L; ； The loss function of the model is div Find the divergence, and Indicates the partial derivative, which speeds up the calculation of the loss function. Indicates the polar diameter: ； The loss function is the sum of two linear parts.

Images