A Knowledge Graph Completion Method Based on Graph Attention Network

Abstract

The present invention discloses a knowledge graph completion method based on a graph attention network, which is a completion method based on matching metrics. This method models the local neighborhood of elements of incomplete triples through an attention mechanism based on the graph attention network, assigns different weights to neighbors in the local neighborhood according to the importance of node features, generates a more accurate element representation, and then matches the incomplete triples with the reference set through a matching network, and selects the candidate element with the highest rank as the final completion value, so as to more accurately complete each incomplete triple in the knowledge graph, the completion result is more in line with objective facts, and the knowledge graph completion task can be efficiently and accurately completed.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graph technology, and specifically relates to a knowledge graph completion method based on graph attention network. Background Art

[0002] With the development of information technology, natural language processing technology has become increasingly important, and knowledge graphs have become a vital part of many downstream artificial intelligence applications such as question-answering systems, recommendation systems, and information retrieval. Many knowledge graphs represent each piece of information in the form of a triple (h, r, t), which includes: two entity elements, namely, element h as the head entity and element t as the tail entity, and a relationship element, namely, element r as the relationship.

[0003] The knowledge graph contains a large number of such triples, but some of these triples are often incomplete, missing one or some elements. At this time, the triples need to be completed. Completion is to predict the missing elements in the triples. Completing the knowledge graph is to complete all the incomplete triples in the knowledge graph.

[0004] Unfortunately, in the real world, long-tail relations are common in knowledge graphs, that is, a large portion of the data has only a few training instances. For sparse long-tail relations, the lack of sufficient training triplets greatly limits the effective implementation of knowledge graph completion.

[0005] In view of this, how to complete the knowledge graph efficiently and accurately has become a more urgent technical issue in the industry. Summary of the invention

[0006] To this end, the present invention intends to provide a knowledge graph completion method based on graph attention network, which can complete the knowledge graph efficiently and accurately.

[0007] According to the specific elements that need to be predicted and completed in the triples of the knowledge graph, the completion tasks can be divided into three categories: head entity completion (?, r, t), relationship completion (h, ?, t) and tail entity completion (h, r, ?).

[0008] To achieve the above purpose, the knowledge graph completion method based on graph attention network provided by the present invention specifically includes the following steps:

[0009] Step 1: extract an incomplete triple in a knowledge graph, where the incomplete triple has at least one element to be completed e and one existing entity element;

[0010] Step 2: Based on the existing entity meta-element, retrieve the neighbor triples (h, r, t) related to the existing entity meta-element in the knowledge graph, and group all the retrieved neighbor triples (h, r, t) into a neighbor triple set S;

[0011] Step 3: Calculate the normalized attention coefficient α of each neighbor triple (h, r, t) in the neighbor triple set S 归 , specifically as follows:

[0012] First, use the following formula to calculate the attention value α of each neighbor triple (h, r, t):

[0013] α = σ(U T W(h, r, t))

[0014] where the magnitude of the attention value α represents the importance degree of the neighbor triple to the existing entity element, σ is the non-linear activation function used in the graph attention network, U is the weight matrix of σ, W is the learnable linear transformation weight matrix, and W(h, r, t) is the vector representation of the neighbor triple (h, r, t);

[0015] Next, use the following formula to calculate the normalized attention coefficient α of each neighbor triple (h, r, t): 归 :

[0016] α 归 = Softmax(α)

[0017] where Softmax is the normalized exponential function used in the graph attention network;

[0018] Step 4: Based on the attention mechanism of the graph attention network, use the following formula to update the head entity element h and the tail entity element t of each neighbor triple (h, r, t) in the neighbor triple set S to obtain the updated neighbor triple (h', r, t'):

[0019]

[0020] where, is the vector representation of the updated neighbor triple (h', r, t'), σ is the non-linear activation function used in the graph attention network, W is the learnable linear transformation weight matrix, W(h, r, t) is the vector representation of the neighbor triple (h, r, t), W1 is the weight matrix, is the vector representation of the existing entity element, and b is the bias;

[0021] Step 5: Select N updated neighbor triples (h', r, t') from the neighbor triple set S and aggregate them into the reference set S N ;

[0022] Step 6: Use the following formula to calculate the reference normalized attention coefficient α between the incomplete triple and the reference set S N : N :

[0023]

[0024] Among them, is the vector representation of the incomplete triple, is the vector representation of the reference set S N ;

[0025] Step 7: Based on the attention mechanism, use the following formula to calculate the general representation suitable for different task relationships:

[0026]

[0027] Among them, is the general representation;

[0028] Step 8: Extract a meta e' in the knowledge graph as the completion value and fill it into the position of the meta e to be completed in the incomplete triple, and then use the following formula to calculate the semantic similarity β between the incomplete triple and the reference set S N :

[0029]

[0030] Among them, is the dot product metric function, is the vector representation of the incomplete triple after being completed with the meta e';

[0031] Step 9: Continuously repeat the above Step 8 until the semantic similarity β corresponding to all the meta e' in the knowledge graph is calculated, and use the meta e' corresponding to the maximum semantic similarity β as the final completion value to fill into the position of the meta e to be completed in the incomplete triple;

[0032] Step 10: Continuously repeat the above Steps 1-9 until all the incomplete triples in the knowledge graph are completed.

[0033] The Graph Attention Network (GAT) is a combination of a graph neural network and an attention layer. It introduces the attention mechanism into the graph neural network based on the spatial domain and has the ability to learn through graph data and provide more accurate results. The graph attention network does not require complex calculations using matrices such as Laplace, and only updates node features through the representations of first-order neighbor nodes.

[0034] The above knowledge graph completion method based on the graph attention network provided by the present invention models the local neighborhood of the meta of the incomplete triple through the attention mechanism based on the graph attention network, assigns different weights to the neighbors in the local neighborhood according to the importance of the node features, generates a more accurate meta representation, and thus can more accurately complete each triple in the knowledge graph, and its result is more in line with objective facts.

[0035] Essentially, the above-mentioned completion method provided by the present invention is a completion method based on matching metrics. It encodes the local neighborhood of the entity through a neighbor encoder, and then matches the incomplete triple and the reference set through a matching network, selecting the candidate entity with the highest rank as the final completion value.

[0036] Specifically, in steps 3-4 of this completion method, a neighbor encoder is constructed based on the graph attention network. By extracting the first-order neighbors of the target entity, a local neighborhood graph of the entity is obtained, thereby encapsulating more entity neighborhood information and relationship information in the neighbor encoder stage. Subsequently, in steps 5-8, a matching network is constructed based on the graph attention network. The entity information representation obtained from the neighbor encoder is input into this matching network to query the matching score, and the candidate entity with the highest rank is selected as the final completion value.

[0037] Compared with the prior art, the above-mentioned completion method provided by the present invention can obtain a more accurate entity representation when encoding the local neighborhood, can well reflect the objective facts, and improves the performance of knowledge graph completion. Especially for small-sample knowledge graphs with obvious long-tail relationships, it can also efficiently and accurately complete the completion task. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and / or additional aspects and advantages of the present invention will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings. In the drawings, unless otherwise specified, the same reference numerals throughout the several drawings denote the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed according to the present invention, used to provide a further understanding of the present invention, constitute a part of this application, and should not be regarded as a limitation on the scope of the present invention. Among them:

[0039] Figure 1 It is a principle block diagram of the completion method provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the following further describes the present invention in detail with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and should not be construed as a limitation on the present invention.

[0041] The above-mentioned knowledge graph completion method based on the graph attention network provided by the present invention models the local neighborhood of the entity in the incomplete triple through the attention mechanism based on the graph attention network, assigns different weights to the neighbors in the local neighborhood according to the importance of the node features, generates a more accurate entity representation, and thus can more accurately complete each triple in the knowledge graph, and the result is more in line with the objective facts.

[0042] As Figure 1 shown, this completion method is a completion method based on matching metrics. It encodes the local neighborhood of the entity through a neighbor encoder, and then matches the incomplete triple and the reference set through a matching network, selecting the candidate entity with the highest ranking as the final completion value.

[0043] In this embodiment, we specifically use the tail entity completion (h, r,?) task as an example for illustration. Suppose we want to complete an incomplete triple (User A, spouse,?) in a knowledge graph of human relationships. The knowledge graph completion method based on the graph attention network provided by the present invention includes the following steps:

[0044] Step 1: Extract an incomplete triple from the knowledge graph. Suppose in this embodiment we extract (User A, spouse,?). This incomplete triple has the tail entity e to be completed, as well as the existing head entity User A and the relationship spouse.

[0045] Step 2: Based on the existing head entity User A, retrieve the neighbor triples (h, r, t) related to the head entity User A in the knowledge graph, and gather all the retrieved neighbor triples (h, r, t) into the neighbor triple set S.

[0046] Step 3: Calculate the normalized attention coefficient α of each neighbor triple (h, r, t) in the neighbor triple set S 归 , specifically:

[0047] First, use the following formula to calculate the attention value α of each neighbor triple (h, r, t):

[0048] α = σ(U T W(h, r, t))

[0049] where the magnitude of the attention value α represents the importance degree of this neighbor triple to the head entity User A, σ is the non-linear activation function used by the graph attention network (such as the leakyRelu activation function), U is the weight matrix of σ, W is the learnable linear transformation weight matrix, and W(h, r, t) is the vector representation of the neighbor triple (h, r, t);

[0050] Next, use the following formula to calculate the normalized attention coefficient α of each neighbor triple (h, r, t) 归 :

[0051] α 归 = Softmax(α)

[0052] where Softmax is the normalized exponential function used by the graph attention network;

[0053] Step 4: Based on the attention mechanism of the graph attention network, update the head entity element h and the tail entity element t of each neighbor triple (h, r, t) in the neighbor triple set S using the following formula to obtain the updated neighbor triple (h', r, t'):

[0054]

[0055] Among them, is the vector representation of the updated neighbor triple (h', r, t'), σ is the non-linear activation function used by the graph attention network (such as the leakyRelu activation function), W is the learnable linear transformation weight matrix, W(h,r,t) is the vector representation of the neighbor triple (h, r, t), W1 is the weight matrix, is the vector representation of the existing head entity element User A, and b is the bias;

[0056] Step 5: Select 5 updated neighbor triples (h', r, t') from the neighbor triple set S, that is, N is taken as 5 in this embodiment, and gather them into the reference set S N ;

[0057] In this embodiment, the reference set S N is exemplarily selected as follows:

[0058] (User A, father_in_law, User B)

[0059] (User A, son_of, User C)

[0060] (User A, friend, User D)

[0061] (User A, education, Univ.of Pennsylvania)

[0062] (User A, occupation, Entrepreneur)

[0063] Step 6: Calculate the reference normalized attention coefficient α between the incomplete triple and the reference set S N using the following formula N :

[0064]

[0065] Among them, is the vector representation of the incomplete triple, is the vector representation of the reference set S N ;

[0066] Step 7: Based on the attention mechanism, calculate the general representation formula adapted to different task relationships using the following formula:

[0067]

[0068] Among them, is a general expression;

[0069] Step 8: Extract a meta e' from the knowledge graph and fill it into the position of the meta e of the tail entity to be completed in the incomplete triple, and then use the following formula to calculate the semantic similarity β between the incomplete triple and the reference set S N :

[0070]

[0071] Among them, is the dot product metric function, is the vector representation of the incomplete triple after being completed with the meta e';

[0072] Step 9: Continuously repeat the above Step 8 until the semantic similarity β corresponding to all the meta e' in the knowledge graph is calculated, and use the meta e' corresponding to the maximum semantic similarity β as the final completion value to fill into the position of the meta e of the tail entity to be completed in the incomplete triple;

[0073] In this embodiment, the knowledge graph includes, for example, these metas {User E, apple, User F, piano...}, and we fill them into the position of the meta e of the tail entity to be completed in the incomplete triple one by one, obtaining the following:

[0074] (User A, spouse, User E)

[0075] (User A, spouse, apple)

[0076] (User A, spouse, User F)

[0077] (User A, spouse, piano)

[0078] ...

[0079] Then, calculate their semantic similarities β respectively through Step 8, and finally it is obtained that the semantic similarity β of (User A, spouse, User F) is the largest, so we fill User F as the final completion value into the incomplete triple.

[0080] Thus, through the above Steps 1-9, an incomplete triple (User A, spouse,?) in the person relationship knowledge graph is completed.

[0081] Obviously, the above completion method is also applicable to the other two completion tasks.

[0082] It can be understood that the completion method of the present invention further includes: Step 10, continuously repeating the above Steps 1-9 until all incomplete triples in the person relationship knowledge graph are completed.

[0083] Furthermore, we separately extract a subset, namely the NELL-one dataset and the FB15k237-one dataset, from two well-known datasets in the field of knowledge graphs, the NELL dataset and the FB15k237 dataset, to verify the effect of the above completion method of the present invention.

[0084] Specifically, we select the relationships with more than 50 and less than 500 triples in the dataset as learning tasks. The number of tasks in the two datasets is 67 and 45 respectively. The task numbers are divided and set as 51 / 5 / 11 and 32 / 5 / 8 task relationships as the training set, the validation set, and the test set respectively.

[0085] The statistical information of the dataset is shown in Table 1 below. Among them, "#Ent" represents the number of all entities, "#Tri" represents the number of all relationship triples, "#Rel" represents the number of all relationships, and "Task" represents the number of relationships selected as small sample tasks.

[0086] Dataset #Ent #Tri #Rel Tasks NELL-one 68,545 181,109 358 67 FB15k237-one 14,478 309,621 237 45

[0087] Table 1 Statistical information of the NELL-one and FB15k237-one datasets

[0088] During the verification process, the network model is implemented using the Pytorch deep learning framework. The entity embeddings are initialized by TransE. The maximum number of neighbors in the dataset in this verification is set to 50. The embedding dimensions of the NELL-one dataset and the FB15k237-one dataset are set to 50, the N value is set to 5, the learning rate is set to 5e-5, the Adam optimizer is used to optimize the model parameters, and the L2 norm is further used to avoid overfitting. Two traditional evaluation metrics of link prediction are used to evaluate the effects of different methods on these two datasets, namely MRR and HITS@N. MRR represents the mean reciprocal rank, and HITS@N is the proportion of entities with correct rankings in the first N relationship prediction tasks. In this verification, the N value is set to 1, 5, and 10.

[0089] The model uses the same experimental environment on the two datasets. The method proposed in the present invention is experimentally compared with known baseline models such as the GMatching model, the FSRL model, and the MetaR model. The verification results are shown in Tables 2 and 3 below:

[0090]

[0091] Table 2 Verification Results of the NELL-one Dataset

[0092]

[0093] Table 3 Verification Results of the FB15k237-one Dataset

[0094] It can be seen that the MRR value of the completion method provided by the present invention is higher than that of other baseline models. This is because the GMatching model, the FSRL model, and the MetaR model all use R-GCN to encode the local graph structure, while the present invention uses a graph attention network to encode the local graph structure and adds an attention mechanism, which can capture more entity information. Therefore, it can effectively improve the performance of knowledge graph completion.

[0095] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention without departing from the principles and purposes of the present invention.

Claims

1. A method for completing a knowledge graph of character relationships based on a graph attention network, characterized in that, The method specifically includes the following steps: Step 1: Extract an incomplete triple from the person relationship knowledge graph. The incomplete triple has at least one element e to be completed and one existing entity element; Step 2: Based on the existing entity element, retrieve the neighbor triples (h, r, t) related to the existing entity element in the person relationship knowledge graph, and group all the retrieved neighbor triples (h, r, t) into a neighbor triple set S; Step 3: Calculate the normalized attention coefficient α of each neighbor triple (h, r, t) in the neighbor triple set S 归 , specifically as follows: First, calculate the attention value α of each neighbor triple (h, r, t) using the following formula: α = σ(U T W(h, r, t)) Among them, the magnitude of the attention value α represents the importance degree of the neighbor triple (h, r, t) to the existing entity element, σ is the non-linear activation function used by the graph attention network, U is the weight matrix of σ, W is the learnable linear transformation weight matrix, and W(h,r,t) is the vector representation of the neighbor triple (h, r, t); Next, calculate the normalized attention coefficient α for each neighbor triple (h, r, t) using the following formula 归 : α 归 = Softmax(α) Among them, Softmax is the normalized exponential function used by the graph attention network; Step 4: Based on the attention mechanism of the graph attention network, use the following formula to update the head entity element h and the tail entity element t of each neighbor triple (h, r, t) in the neighbor triple set S to obtain an updated neighbor triple (h', r, t'); Among them, To update the vector representation of the neighbor triple (h', r, t'), σ is the non-linear activation function used by the graph attention network, W is the weight matrix of the learnable linear transformation, W(h, r, t) is the vector representation of the neighbor triple (h, r, t), W1 is the weight matrix, Is the vector representation of the existing entity element, and b is the bias; Step 5: Select N updated neighbor triples (h', r, t') from the neighbor triple set S and aggregate them into a reference set S N ; Step 6. Calculate the reference normalized attention coefficient α between the incomplete triple and the reference set S N as follows N : Among them, is the vector representation of the incomplete triple, is the vector representation of the reference set S N , and Softmax is the normalized exponential function used by the graph attention network; Step 7: Based on the attention mechanism, calculate the general representation formula suitable for different task relationships using the following formula: wherein, is the general expression; Step 8: Extract a meta e' from the person relationship knowledge graph and fill it into the position of the to-be-complemented meta e in the incomplete triple, and then calculate the semantic similarity β between the incomplete triple and the reference set S using the following formula N between them: Among them, is the dot product metric function, is the vector representation after complementing the incomplete triple with the element e'. Step 9: Continuously repeat the above Step 8 until the semantic similarity β corresponding to all elements e' in the person relationship knowledge graph is calculated, and use the element e' corresponding to the maximum semantic similarity β as the final completion value to fill in the position of the element e to be completed in the incomplete triple; Step 10: Continuously repeat the above Steps 1-9 until all incomplete triples in the person relationship knowledge graph are completed.

2. The method for completing a knowledge graph of character relationships based on a graph attention network according to claim 1, wherein In the above Steps 3 and 4, the non-linear activation function σ is specifically the leakyRelu activation function.

3. A method for completing a knowledge graph of character relationships based on a graph attention network according to claim 1, characterized in that, In the above Step 5, the value of N is set to 5.

4. A method for completing a knowledge graph of character relationships based on a graph attention network according to any one of claims 1-3, characterized in that, The person relationship knowledge graph is a small-sample person relationship knowledge graph.

Images