Knowledge graph completion method based on conditional message propagation neural network

Abstract

The invention discloses a knowledge graph completion method based on a conditional message propagation neural network, and the conditional message propagation neural network is an excellent variant of a graph neural network in a multi-relation graph. The core of the invention is to enhance the neighborhood feature extraction capability of the conditional message propagation neural network by using propagation functions (including a message function, an aggregation function and an update function) with strong expression capability, help the knowledge graph completion model to learn embedding with stronger distinguishing capability, and improve the prediction performance of the model. The method follows an encoder-decoder architecture, an encoder consisting of an exploration module and a buffer module for accessing candidate answer entities and encoding the embedding of the candidate entities, and a decoder being a simple linear layer for predicting scores of the candidate answer entities. A query to be completed is given, firstly, an exploration module accesses a multi-hop neighbor of a query entity layer by layer, and the accessed neighbor entity is regarded as a candidate answer entity; secondly, an exploration module disseminates relation embedding information to a multi-hop neighbor of a query entity layer by layer, and learns embedding of candidate answer entities; then, the buffer module executes subgraph message propagation on the subgraph accessed by the exploration module, and continues to update embedding of the candidate answer entity; and finally, based on embedding of the candidate answer entities, scoring the candidate answer entities by using a linear layer of a decoder, and taking the candidate entity with the highest score as a predicted target entity.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and knowledge graphs, and in particular proposes a knowledge graph completion method based on a conditional message propagation neural network. Background Art

[0002] Knowledge graphs represent real-world knowledge in the form of entity-relationship-entity triples. Since it is impossible to collect all fact triples, knowledge graphs in the real world are usually incomplete and need to be completed. Knowledge graph completion methods predict missing links based on existing links in the knowledge graph. Knowledge graph completion has a wide range of applications in question-answering systems, personalized recommendations, and drug interaction prediction.

[0003] The present invention equips the conditional message propagation neural network with message functions, aggregation functions and update functions with strong expressive power, enhances the neighborhood feature extraction capability of the model, helps the model learn embeddings with stronger distinguishing capabilities, and improves the prediction performance of the model. Summary of the invention

[0004] In order to solve the problems mentioned in the above background, the present invention provides a knowledge graph completion method based on conditional message propagation neural network in combination with the propagation function design in graph neural network.

[0005] The method first visits the multi-hop neighbors of the query entity layer by layer by the exploration module, and regards the visited neighbor entities as candidate answer entities; secondly, the exploration module propagates the relationship embedding information to the multi-hop neighbors of the query entity layer by layer to learn the embedding of the candidate answer entity; then, the buffer module performs subgraph message propagation on the subgraph visited by the exploration module and continues to update the embedding of the candidate answer entity; finally, based on the embedding of the candidate answer entity, a simple linear layer is used to score the candidate answer entity, and the candidate entity with the highest score is the target entity.

[0006] 1. A knowledge graph completion method based on conditional message propagation neural network, characterized by:

[0007] (1) The exploration module visits the multi-hop neighbors of the query entity layer by layer to obtain candidate answer entities;

[0008] (2) The exploration module propagates the relation embedding information layer by layer to the multi-hop neighbors of the query entity and learns the embedding of the candidate answer entity;

[0009] (3) The buffer module performs subgraph message propagation on the subgraph accessed by the exploration module and updates the embedding of the candidate answer entity;

[0010] (4) The linear layer scores the candidate answer entities, and the candidate entity with the highest score is the target entity.

[0011] 2. According to the method of claim 1, the knowledge graph completion method based on conditional message propagation neural network is characterized in that: the exploration module in step (1) visits the multi-hop neighbors of the query entity layer by layer as follows:

[0012] Given a query (e q , r q ,? ), the exploration module visits the multi-hop neighbor entities and neighbor edges of the query entity layer by layer. After K steps, the entities and edges visited in each step are integrated to obtain the propagation path of the exploration module in, and are the sets of entities and edges visited in the kth step, respectively. Since self-loops are added to the knowledge graph data, the set of neighbor entities visited by each propagation layer includes the entities visited by the previous layer, that is, Therefore, the set of neighbor entities obtained after K steps is Contains the 1st to Kth order neighbors of the query entity, which are considered as the query entity (e q , r q ,? )’s candidate answer entities.

[0013] 3. According to the method of claim 1, the knowledge graph completion method based on conditional message propagation neural network is characterized in that: the message propagation process in step (2) is as follows:

[0014] The message propagation process mainly involves three types of propagation functions, including message function, aggregation function and update function. QRFGU is used as the message function and update function, and PNA is used as the aggregation function because they have powerful neighborhood feature extraction capabilities and good performance. The following takes the update of the k-th layer embedding of entity v as an example to explain the calculation process of the message function, aggregation function and update function.

[0015] QRFGU is the abbreviation of query-related fusion gate unit, which is a variant of GRU. It can model the sequence information in the relationship path and fuse the information of the query relationship. In general, given an edge (u, r, v), QRFGU calculates the message on the edge as follows:

[0016]

[0017] in, represents the embedding of entity u in the previous layer, represents the embedding of relation r at the current layer, Represents the query relation r q Embedding at the current layer.

[0018] Specifically, first of all, QRFGU is based on and Calculating the forget gate and update gate Secondly, based on and Calculate the latent vector Finally, by combining and Get the news from the side:

[0019]

[0020] Among them, σ represents the sigmoid activation function, [,] represents the concatenation operation, is the training parameter of the kth layer, d is the embedding dimension, and ⊙ represents the Hadamard product.

[0021] The PNA aggregation function first proposed a logarithmic degree scaler S(deg, α):

[0022]

[0023] Among them, deg represents the in-degree of an entity, that is, the number of messages received by the entity during the message propagation process, δ is a normalization parameter calculated on the entire training set train, and α is a variable parameter. α>0 is amplification, α<0 is reduction, and α=0 means no scaling. Finally, PNA combines four aggregators (i.e., mean, standard deviation, max, and min aggregators) and three degree scalers to obtain the final aggregator:

[0024]

[0025] in, represents the tensor product, μ, std, max and min represent the aggregation results of the above four aggregators respectively, It can be seen that due to the combination of multiple aggregators and degree scalers, PNA can capture neighborhood feature information from different aspects.

[0026] The QRFGU update function updates the embedding of entity v as follows:

[0027]

[0028] in, is the embedding of entity v in the previous layer, is the embedding of the current layer after aggregation and update operations. Represents the aggregation result, Represents the query relation embedding of the current layer.

[0029] 4. According to claim 1, a knowledge graph completion method based on conditional message propagation neural network is characterized in that: the buffer module in step (3) performs subgraph message propagation on the subgraph accessed by the exploration module, and the process of updating the embedding of the candidate answer entity is as follows:

[0030] The buffer module is executed after the exploration module. Specifically, the buffer module continues to perform message propagation in the subgraph composed of entities and edges visited in the last step of the exploration module. E After the exploration module propagation steps, continue to execute K B buffer module propagation steps, then the final propagation path is:

[0031]

[0032] in, and In general, without further increasing the number of propagation steps of the exploration module, the buffer module enables candidate answer entities to wait for important information retained in other entities, which effectively avoids excessive execution of the exploration module and accessing a large number of neighbor entities and neighbor edges.

[0033] The message function, aggregation function and update function used by the buffer module to update entity embedding are consistent with those in the exploration module, that is, QRFGU is used as the message function and update function, and PNA is used as the aggregation function.

[0034] 5. According to the method for completing a knowledge graph based on a conditional message propagation neural network in claim 1, it is characterized in that the process of the linear layer in step (4) scoring the candidate answer entity is as follows:

[0035]

[0036] in, represents the candidate answer entity set, e c represents a candidate entity in the set, Represents the candidate entity e c The final embedding, Represents the parameters to be trained for the decoder linear layer. After all candidate entities are scored, the candidate entity with the highest score is the target entity.

[0037] The beneficial effects of the above technical solution of the present invention are as follows:

[0038] In the above scheme, the method firstly visits the multi-hop neighbors of the query entity layer by layer by the exploration module, and regards the visited neighbor entities as candidate answer entities; secondly, the exploration module propagates the relationship embedding information to the multi-hop neighbors of the query entity layer by layer, and learns the embedding of the candidate answer entity; then, the buffer module performs subgraph message propagation on the subgraph visited by the exploration module, and continues to update the embedding of the candidate answer entity; finally, based on the embedding of the candidate answer entity, a simple linear layer is used to score the candidate answer entity, and the candidate entity with the highest score is the target entity. In the process of encoding the embedding of the candidate entity based on the conditional message propagation scheme, the present invention uses a message function, an aggregation function and an update function with strong expressive power to update the entity embedding, which enhances the model's neighborhood feature extraction capability, helps the model learn an embedding vector with stronger discrimination capability, and improves the model's ability to discover the target entity. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Flowchart of the present invention

[0040] Figure 2 The encoder-decoder architecture diagram of the model

[0041] Figure 3 Design a graph for the propagation function of the message propagation process DETAILED DESCRIPTION

[0042] 1. Data preparation:

[0043] use To represent a knowledge graph dataset, where and are entity sets and relationship sets, ε and They are fact triples and query triples respectively. Divided into and The present invention adopts a batch method to train, verify and test the model. In the training stage, the batch data is obtained from In the sampling, the model uses the existing links in ε as a basis to predict In the validation and testing phases, the model uses Links in the prediction and It should be noted that in order to enhance the connectivity of knowledge graph data, in practical applications, reverse relationship edges and self-loops are added to the original knowledge graph.

[0044] 2. The exploration module visits neighbor edges and neighbor entities and performs message propagation:

[0045] When processing batch queries, each query is actually processed separately and they do not affect each other. q , r q ,? ) as an example to illustrate.

[0046] The exploration module starts from the query entity eq and explores its multi-hop neighbor edges Visit multi-hop neighbor entities layer by layer Because of the existence of self-loops, the neighbor entity set accessed by the model at layer k will include the entity set accessed at layer k-1, that is, Therefore, after K E The set of neighbor entities obtained after the step Contains 1 to K of the query entities E order neighbors, these neighbor entities are considered as queries (e q , r q ,? )’s candidate answer entities.

[0047] After each step of neighbor visit in the exploration module is completed, the propagation function design component will be called to perform message propagation. The propagation function includes a message function, an aggregation function, and an update function. The present invention uses QRFGU as the message function and the update function, and the aggregation function uses PNA because they have powerful neighborhood feature extraction capabilities. The following takes the update of the k-th layer embedding of entity v as an example to specifically explain the calculation process of the message function, the aggregation function, and the update function.

[0048] For a neighbor edge (u, r, v), QRFGU calculates the message on the edge as follows:

[0049]

[0050] The detailed calculation process is as follows:

[0051]

[0052] in, represents the embedding of entity u in the previous layer, represents the embedding of relation r at the current layer, Represents the query relation r q The embedding of the current layer. σ represents the sigmoid activation function, [,] represents the concatenation operation, is the training parameter of the kth layer, d is the embedding dimension, and ⊙ represents the Hadamard product.

[0053] PNA combines four aggregators (i.e., mean, standard deviation, max, and min aggregators) and three degree scalers to obtain the final aggregator:

[0054]

[0055] in, represents the tensor product, μ, std, max and min represent the aggregation results of the above four aggregators respectively, Represents the final aggregation result.

[0056] The QRFGU update function updates the embedding of entity v as follows:

[0057]

[0058] in, is the embedding of entity v in the previous layer, is the embedding of the current layer after aggregation and update operations. Represents the aggregation result, Represents the query relation embedding of the current layer.

[0059] 3. The buffer module performs subgraph message propagation:

[0060] The cache module accesses the entity set in the last step of the exploration module and edge sets Message propagation continues in the subgraph formed. Assume that in K E After the exploration module propagation steps, continue to execute K B It should be noted that the message function, aggregation function and update function used by the buffer module to update entity embedding are consistent with those in the exploration module, that is, QRFGU is used as the message function and update function, and PNA is used as the aggregation function.

[0061] 4. Linear layer score prediction:

[0062] The decoder's linear layer scores the candidate answer entities as follows:

[0063]

[0064] in, represents the candidate answer entity set, e c represents a candidate entity in the set, Represents the candidate entity e c The final embedding, Represents the parameters to be trained for the decoder linear layer. After all candidate entities are scored, the candidate entity with the highest score is the target entity.

[0065] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A knowledge graph completion method based on conditional message propagation neural network, characterized by: (1) The exploration module visits the multi-hop neighbors of the query entity layer by layer to obtain candidate answer entities; (2) The exploration module propagates the relation embedding information layer by layer to the multi-hop neighbors of the query entity and learns the embedding of the candidate answer entity; (3) The buffer module performs subgraph message propagation on the subgraph accessed by the exploration module and updates the embedding of the candidate answer entity; (4) The linear layer scores the candidate answer entities, and the candidate entity with the highest score is the target entity.

2. According to claim 1, a knowledge graph completion method based on conditional message propagation neural network is characterized by: The exploration module in step (1) visits the multi-hop neighbors of the query entity layer by layer as follows: Given a query (e q ,r q ,? ), the exploration module visits the multi-hop neighbor entities and neighbor edges of the query entity layer by layer. After K steps, the entities and edges visited in each step are integrated to obtain the propagation path of the exploration module in, and are the sets of entities and edges visited in the kth step, respectively. Since self-loops are added to the knowledge graph data, the set of neighbor entities visited by each propagation layer includes the entities visited by the previous layer, that is, Therefore, the set of neighbor entities obtained after K steps is Contains the 1st to Kth order neighbors of the query entity, which are considered as the query entity (e q ,r q ,? )’s candidate answer entity.

3. The method for completing a knowledge graph based on a conditional message propagation neural network according to claim 1, characterized in that: The message propagation process in step (2) is as follows: The message propagation process mainly involves three types of propagation functions, including message function, aggregation function and update function. QRFGU is used as the message function and update function, and PNA is used as the aggregation function because they have powerful neighborhood feature extraction capabilities and good performance. The following takes the embedding of updating entity v as an example to explain the calculation process of message function, aggregation function and update function. QRFGU is the abbreviation of query-related fusion gate unit, which is a variant of GRU. It can model the sequence information in the relationship path and fuse the information of the query relationship. In general, given an edge (u, r, v), QRFGU calculates the message on the edge as follows: in, represents the embedding of entity u in the previous layer, represents the embedding of relation r at the current layer, Represents the query relation r q Embedding at the current layer. Specifically, first of all, QRFGU is based on and Calculating the forget gate and update gate Secondly, based on and Calculate the latent vector Finally, by combining and Get the news from the side: Among them, σ represents the sigmoid activation function, [,] represents the concatenation operation, is the training parameter of the kth layer, d is the embedding dimension, and ⊙ represents the Hadamard product. The PNA aggregation function first proposed a logarithmic degree scaler S(deg,α): Among them, deg represents the in-degree of an entity, that is, the number of messages received by the entity during the message propagation process, δ is a normalization parameter calculated on the entire training set train, and α is a variable parameter. α>0 is amplification, α<0 is reduction, and α=0 means no scaling. Finally, PNA combines four aggregators (i.e., mean, standard deviation, max, and min aggregators) and three degree scalers to obtain the final aggregator: in, represents the tensor product, μ, std, max and min represent the aggregation results of the above four aggregators respectively, It can be seen that due to the combination of multiple aggregators and degree scalers, PNA can capture neighborhood feature information from different aspects. The QRFGU update function updates the embedding of entity v as follows: in, is the embedding of entity v in the previous layer, is the embedding of the current layer after aggregation and update operations. Represents the aggregation result, Represents the query relation embedding of the current layer.

4. The method for completing a knowledge graph based on a conditional message propagation neural network according to claim 1, characterized in that: The buffer module in step (3) performs subgraph message propagation on the subgraph accessed by the exploration module, and the process of updating the embedding of the candidate answer entity is as follows: The buffer module is executed after the exploration module. Specifically, the buffer module continues to perform message propagation in the subgraph composed of entities and edges visited in the last step of the exploration module. E After the exploration module propagation steps, continue to execute K B buffer module propagation steps, then the final propagation path is: in, and In general, without further increasing the number of propagation steps of the exploration module, the buffer module enables candidate answer entities to wait for important information retained in other entities, which effectively avoids excessive execution of the exploration module and accessing a large number of neighbor entities and neighbor edges. The message function, aggregation function and update function used by the buffer module to update entity embedding are consistent with those in the exploration module, that is, QRFGU is used as the message function and update function, and PNA is used as the aggregation function.

5. The method for completing a knowledge graph based on a conditional message propagation neural network according to claim 1, characterized in that: The process of the linear layer in step (4) scoring the candidate answer entity is as follows: in, represents the set of candidate answer entities, e c represents a candidate entity in the set, Represents entity e c The final embedding, Represents the parameters to be trained for the decoder linear layer.

Images