A small sample knowledge graph completion method based on meta-learning

Abstract

The present invention discloses a small sample knowledge graph completion method based on meta-learning. The small sample knowledge graph completion method based on meta-learning includes two components: a neighborhood relationship meta-encoder and a matching processor. The neighborhood relationship meta-encoder transfers semantic similarity from the entity level to the relationship level, and uses the semantic similarity between entities and the information interaction within the neighborhood to describe the effect of the neighborhood relationship on the long-tail relationship. The acquired relationship embedding is transferred to the query set. After updating the relationship meta, the rationality score of the new triple corresponding to the query set is calculated, thereby solving the small sample knowledge graph completion task. For an incomplete triple with a missing tail entity in the knowledge graph, the head entity therein is called the target head entity, and the relationship is called the task relationship. A set of K head and tail entity pairs corresponding to a given task relationship is called a reference set, which can realize the prediction task of the missing tail entity.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graphs, and specifically to a small-sample knowledge graph completion method based on meta-learning. Background Art

[0002] Large knowledge graphs are usually sparse and require mining missing information or adding new entities to make them more complete. The task of predicting the missing parts in triples with long-tail relationships is called small-sample knowledge graph completion, which is divided into static knowledge graph completion and dynamic knowledge graph completion. Static knowledge graph completion is to infer implicit relationships in the knowledge graph, and dynamic knowledge graph completion is to supplement entities or relationships that have not appeared in the original knowledge graph.

[0003] Existing knowledge graph completion algorithms usually require a large number of triplets for training, which is not suitable for long-tail relationships with a small number of corresponding entity pairs and cannot be directly applied to small sample problems. Related models for small sample problems mainly focus on entity embedding rather than relationship embedding, ignoring the importance of relationship representation in the final prediction and failing to fully represent the characteristics of long-tail relationships. Summary of the Invention

[0004] The purpose of the present invention is to provide a small sample knowledge graph completion method based on meta-learning to solve the problems raised by the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A small sample knowledge graph completion method based on meta-learning includes the following steps:

[0007] 1) Neighborhood Relation Meta-Encoder: Utilizing the neighborhood information of the reference entity, we can obtain an effective representation of the relationship and model the information exchange between entities and relationships;

[0008] 2) Matching processor: The reference set information is used to update the relationship element, match the reference entity and the relationship element information, and update the relationship element.

[0009] On the basis of the above technical solutions, the present invention also provides the following optional technical solutions:

[0010] In an optional solution: in step 1), the similarity measure between the tail entity and the one-hop neighborhood entity of the head entity is calculated:

[0011] φ(e,e nbr )=e T W1e nbr +b (1)

[0012] where e and e nbrRepresent the original embedding of the head entity and tail entity neighborhood in the reference set, W1 and b are trainable parameters, the similarity between the head or tail entity and the neighborhood entity in the reference set is used as the attention of the neighborhood relationship to the task relationship, and the attention coefficient α between the neighborhood relationship numbered i in the neighborhood and the task relationship i for:

[0013]

[0014] in It is the set of neighborhood relations and neighborhood entities in the neighborhood corresponding to the reference head and tail entities.

[0015] In an alternative solution: the attention coefficient α i As the influence factor of the neighborhood relationship on the task relationship, the relation element representation corresponding to a reference entity with sequence number m in the reference set is obtained, which is called neighborhood-aware relation embedding:

[0016]

[0017] Among them It is the embedding of some randomly selected neighborhood relations. The original embedding of the task relation and its neighborhood-aware relation embedding are added and input into a fully connected layer to obtain the task relation embedding corresponding to the mth reference entity pair:

[0018]

[0019] where rel ori It is calculated by converting the score function h+r=t in the translation distance model to r=th, σ is the LeakyReLU activation function, W2 is a learnable parameter, and the embedding results of K reference entity pairs are averaged to obtain the relation element

[0020]

[0021] In one alternative, the reference entity embedding and the relation meta-embedding are concatenated to obtain triple embeddings, and then the triples are calculated using the scoring function in the ConvE model. Reasonableness score:

[0022]

[0023] in, represents the relation meta-embedding generated by the neighborhood relation meta-encoder, and They represent the two-dimensional reconstruction of entity and relation embeddings respectively, and then they are input into a two-dimensional convolutional layer with a filter of ω, f(·) is an activation function similar to ReLU, and vec(·) is a function that reconstructs the tensor into a two-dimensional vector.

[0024] In an alternative solution: use the weight matrix W3 to map it into a K-dimensional space, so that the tail entity t i The triple score is obtained by calculating the inner product, and the loss function of the support set is:

[0025]

[0026] Where [x] + = max{0, x} represents the standard hinge loss function, and is the negative reference set obtained by randomly replacing the tail entity of the reference entity pair, and γ is the interval used to distinguish between positive and negative examples.

[0027] In an alternative approach, the gradient information of the relation meta-data during the optimization of the support set loss is retained and transferred from the reference set to the query set as meta-information. The gradient meta-data is calculated using the following formula:

[0028]

[0029] Thus the task or The corresponding relation element is obtained by optimizing the corresponding gradient element:

[0030]

[0031] The updated relation element obtained can be input into the second component.

[0032] In an alternative approach: Apply the scoring function in the TransE model to calculate the matching score of the triples

[0033]

[0034] Loss function:

[0035]

[0036] in is the negative query set obtained by randomly replacing the tail entity in the query set entity pair, and γ is the interval used to distinguish positive and negative query set triplets.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The small-sample knowledge graph completion method based on meta-learning includes two components: a neighborhood relationship meta-encoder and a matching processor, which solves the task of completing the small-sample knowledge graph. For an incomplete triple with a missing tail entity in the knowledge graph, the head entity is called the target head entity, and the relationship is called the task relationship. The set of K head-tail entity pairs corresponding to the given task relationship is called the reference set. The knowledge graph contains neighborhood information in the reference entity, and a candidate entity set containing the real tail entity corresponding to a target head entity, which can realize the prediction task of the missing tail entity. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a method structure diagram of the small-sample knowledge graph completion method based on meta-learning. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] The specific implementation of the present invention is described in detail below with reference to specific embodiments.

[0042] like Figure 1 As shown, a small sample knowledge graph completion method based on meta-learning is provided as an embodiment of the present invention, called FNIF, which includes two parts: a neighborhood relationship meta-encoder and a matching processor.

[0043] 1) Neighborhood Relation Meta-Encoder

[0044] Since there is less head and tail entity information corresponding to the task relationship, we aggregate the one-hop neighborhood information of the entity and obtain the corresponding task relationship embedding. We use the semantic association between entities to represent the semantic similarity between relationships. First, we calculate the similarity measure between the tail entity and the one-hop neighborhood entity of the head entity:

[0045] φ(e,e nbr )=e T W1e nbr +b (1)

[0046] where e and e nbr Respectively represent the original embedding of the head entity (tail entity) and the neighborhood of the tail entity (head entity neighborhood) in the reference set. W1 and b are trainable parameters. The same method is used to calculate the relevance of the tail entity and the neighborhood of the head entity, but the weight matrix is ​​different. The similarity between the head or tail entity and the neighborhood entity in the reference set is used as the attention of the neighborhood relationship to the task relationship. The attention coefficient α between the neighborhood relationship numbered i in the neighborhood and the task relationship is i for:

[0047]

[0048] in is the set of neighborhood relations and neighborhood entities in the neighborhood corresponding to the reference head and tail entities. In order to integrate the information interaction between entities into the relationship interaction, the attention coefficient α obtained from the previous formula is i As the influence factor of the neighborhood relationship on the task relationship, the neighborhood relationship embedding is aggregated to aggregate the neighborhood information and obtain the relationship element representation corresponding to a reference entity with sequence number m in the reference set, which is called neighborhood-aware relationship embedding:

[0049]

[0050] Among them It is the embedding of some randomly selected neighborhood relations. Then the original embedding of the task relation and its neighborhood-aware relation embedding are added and input into a fully connected layer to obtain the task relation embedding corresponding to the mth reference entity pair:

[0051]

[0052] where rel ori It is calculated by converting the score function h+r=t in the translation distance model to r=th, σ is the LeakyReLU activation function, W2 is a learnable parameter, and the embedding results of K reference entity pairs are averaged to obtain the relation element

[0053]

[0054] 2) Matching processor

[0055] The matching processor consists of two parts. Its purpose is to update the relation element based on the reference set information. First, the reference entity embedding and the relation element embedding are concatenated to obtain the triple embedding. Then, the triple is calculated by the scoring function in the ConvE model. Reasonableness score:

[0056]

[0057] in, represents the relation meta-embedding generated by the neighborhood relation meta-encoder, and Represent the two-dimensional reconstruction of entity and relation embedding respectively, and then input them into the two-dimensional convolution layer with filter ω, f(·) is similar to the activation function of ReLU, vec(·) is the function of reconstructing the tensor into a two-dimensional vector, and uses the weight matrix W3 to map it into a K-dimensional space, so that it can be combined with the tail entity t iThe triple score is obtained by calculating the inner product. The following formula is the loss function of the support set:

[0058]

[0059] Where [x] + = max{0, x} represents the standard hinge loss function, and is the negative reference set obtained by randomly replacing the tail entity of the reference entity pair, and γ is the interval used to distinguish between positive and negative examples.

[0060] The gradient information of the relation meta-data during the optimization process of the support set loss is retained and transferred from the reference set to the query set as meta-information. The gradient meta-data is calculated using the following formula:

[0061]

[0062] Thus the task or The corresponding relation element is obtained by optimizing the corresponding gradient element:

[0063]

[0064] The updated relation element obtained can be input into the second component.

[0065] In the second part of the matching processor, the present invention concatenates the query head entity, the optimized relational meta-representation, and the real tail entity or the randomly replaced tail entity, and applies the score function in the TransE model to calculate the matching score s(h j , t j ):

[0066]

[0067] The loss function of the entire method is calculated as:

[0068]

[0069] in is the negative query set obtained by randomly replacing the tail entity in the query set entity pair, and γ is the interval used to distinguish positive and negative query set triplets.

[0070] The above embodiment of the present invention provides a small sample knowledge graph completion method based on meta-learning, called FNIF, which mainly applies an embedding-based method and a small sample relationship learning method. The embedding-based method is used to learn the feature representation of long-tail relationships. Small sample learning is the application of meta-learning in the field of supervised learning. Meta-learning is used to transfer the information of the reference set to the prediction task of the query set. The embedding-based model mainly focuses on the representation of entities and relationships in low-dimensional space. The meta-learning-based method includes a metric learning-based method, a model representation enhancement method, and a model optimization-based method. In the process of learning relationship embedding, a graph attention network is used to apply the attention mechanism to the graph structure in the knowledge graph.

[0071] The FNIF method consists of two components: a neighborhood relation meta-encoder and a matching processor. The neighborhood relation meta-encoder transfers semantic similarity from the entity level to the relation level. It uses semantic similarity between entities and information interaction within the neighborhood to describe the impact of neighborhood relations on long-tail relations. Combined with an attention mechanism, it achieves an efficient representation of relations. The obtained relation embeddings are transferred to the query set. After updating the relation meta-encoder, the rationality score of the new triples corresponding to the query set is calculated. It achieves good performance on both the NELL-One and Wiki-One datasets.

[0072] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A small sample knowledge graph completion method based on meta-learning, characterized by: The following steps are involved: 1) Neighborhood Relation Meta-Encoder: Utilizing the neighborhood information of a reference entity, we can obtain an effective representation of the relationship and model the information exchange between entities and relationships. 2) Matching processor: Updates the relational element based on the reference set information, matches the reference entity and relational element information, and updates the relational element; In step 1), the similarity measure between the tail entity and the one-hop neighborhood entities of the head entity is calculated: (1) in and denote the original embeddings of the neighborhood of the head entity and the tail entity in the reference set, respectively, and It is a trainable parameter, which uses the similarity between the head or tail entity and the neighborhood entity in the reference set as the attention of the neighborhood relationship to the task relationship, and the attention coefficient between the neighborhood relationship numbered i in the neighborhood and the task relationship for: (2) in It is the set of neighborhood relations and neighborhood entities in the neighborhood corresponding to the reference head and tail entities; The attention coefficient As the influence factor of the neighborhood relationship on the task relationship, the relation element representation corresponding to a reference entity with sequence number m in the reference set is obtained, which is called neighborhood-aware relation embedding: (3) Among them is the embedding of some randomly selected neighborhood relations. The original embedding of the task relation and its neighborhood-aware relation embedding are added and input into a fully connected layer to obtain the first The task relationship embedding corresponding to the reference entity pair: (4) in is the score function in the translation distance model , converted into Calculated, is the LeakyReLU activation function, is a learnable parameter, The embedding results of the reference entity pairs are averaged to obtain the relation element (5) 2. The small sample knowledge graph completion method based on meta-learning according to claim 1 is characterized in that: The reference entity embedding and the relation meta-embedding are concatenated to obtain triple embeddings, and then the triples are calculated using the scoring function in the ConvE model. Reasonableness score: (6) in, represents the relation meta-embedding generated by the neighborhood relation meta-encoder, and Represent the two-dimensional reconstruction of entity and relation embedding respectively, which are then input into the filter as The two-dimensional convolutional layer, similar The activation function, is a function that reshapes a tensor into a two-dimensional vector.

3. The small sample knowledge graph completion method based on meta-learning according to claim 2 is characterized in that: Using the weight matrix Map it into a K-dimensional space, and then The triple score is obtained by calculating the inner product, and the loss function of the support set is: (7) in, represents the standard hinge loss function, and is the negative reference set obtained by randomly replacing the tail entity of the reference entity pair, is the interval used to distinguish positive examples from negative examples.

4. The small sample knowledge graph completion method based on meta-learning according to claim 3 is characterized in that: The gradient information of the relation meta-data during the optimization process of the support set loss is retained and transferred from the reference set to the query set as meta-information. The gradient meta-data is calculated using the following formula: (8) Thus the task or The corresponding relation element is obtained by optimizing the corresponding gradient element: (9) The updated relation element obtained can be input into the second component.

5. The small sample knowledge graph completion method based on meta-learning according to claim 4 is characterized in that: Apply the scoring function in the TransE model to calculate the matching score of the triples : (10) Loss function: (11) in is the negative query set obtained by randomly replacing the tail entity in the query set entity pair, is the interval used to distinguish positive and negative query set triplets.

Images