Knowledge graph embedded vector generation method and application

Abstract

The invention provides a knowledge graph embedded vector generation method and application, and the method is characterized in that the method comprises the following steps: an embedded vector initialization step: initializing each node and relationship in a knowledge graph to obtain a node initialization embedded vector corresponding to each node and a relationship initialization embedded vector corresponding to each relationship; a relation adjacency matrix construction step: constructing a relation adjacency matrix according to graph structure information of the knowledge graph; and an embedded vector generation step: inputting all the node initialization embedded vectors, the relation initialization embedded vectors and the relation adjacency matrix into the fine-grained graph prediction model to obtain a relation embedded vector and a node embedded vector. In a word, the method can generate accurate node embedding vectors and relation embedding vectors.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graph completion, and specifically relates to a method for generating a knowledge graph embedding vector, as well as a knowledge graph prediction and completion method generated by the method and an application in a knowledge graph prediction and completion device. Background Art

[0002] In recent years, with the rapid development of big data and artificial intelligence technology, knowledge graphs, as one of the core of artificial intelligence, have received widespread attention. Knowledge graphs can effectively organize and express complex node relationships, so they are widely used in various fields, such as recommendation systems, search engines, and natural language processing. However, the construction and maintenance of knowledge graphs face many challenges, especially in efficiently predicting and completing missing relationships and nodes in graphs. Traditional graph prediction methods mostly rely on artificial rules or simple statistical methods to train node and relationship embedding vectors that meet their rules, so as to complete graph prediction and completion tasks based on the embedding vectors. Although more and more high-performance graph models have been proposed, their essence still depends on the embedding vectors. For example, from the proposal of the TransE model to a series of subsequent improvements, such as the TransH, TransR, and TransC models, they are all based on embedding vectors that meet the rules. Furthermore, based on the embedding vector, researchers also consider the structural information of the graph, and then propose a graph convolutional network, which is essentially to integrate the graph structure information by converting the embedding vector into the frequency domain.

[0003] With the development of the world and the further research of deep learning technology, models based on graph neural networks (GNNs) have gradually shown their advantages. They can take into account the local graph structure information of nodes and rely on the representation of node and relationship embedding vectors, making the model highly scalable and generalizable. Therefore, how to better utilize the embedding vectors of nodes and relationships is the main research direction of graphs at present.

[0004] However, the investigation found that few researchers consider two key issues at the same time: overfitting of embedding vector updates and the presence of nodes with high correlation but low influence in the neighbor set. First, the reason for the overfitting of embedding vectors is that the update of embedding vectors mainly depends on the loss function of the model, and the loss function of the model only considers the overall loss, ignoring the problem of excessive updating of embedding vectors of local nodes and relationships. Second, the traditional graph attention mechanism captures the correlation between neighbor nodes by learning the importance weights of target nodes. However, high correlation means that two nodes are closely connected in the graph structure, but this does not mean that the two nodes have a high influence on each other.

[0005] Therefore, there is an urgent need for a graph model method that can avoid overfitting of node and relationship embedding vectors and accurately identify high-influence neighbor information. Summary of the invention

[0006] The present invention is made to solve the above-mentioned problems, and its purpose is to provide a method and application for generating knowledge graph embedding vectors.

[0007] The present invention provides a method for generating a knowledge graph embedding vector, which is used to generate a relationship embedding vector and a node embedding vector corresponding to a knowledge graph, and has the following characteristics, including the following steps: an embedding vector initialization step, respectively initializing each node and relationship in the knowledge graph to obtain a node initialization embedding vector corresponding to each node and a relationship initialization embedding vector corresponding to each relationship; a relationship adjacency matrix construction step, constructing a relationship adjacency matrix according to the graph structure information of the knowledge graph; an embedding vector generation step, inputting all the node initialization embedding vectors, the relationship initialization embedding vectors and the relationship adjacency matrix into a fine-grained graph prediction model to obtain the relationship embedding vector and the node embedding vector, wherein the fine-grained graph prediction model includes: an initialization module, which is used to initialize each of the relationship initialization embedding vectors to obtain the corresponding initial data; a relationship update module, including a multi-layer relationship update network, which is used to obtain the relationship embedding vector corresponding to each of the relationships according to the relationship adjacency matrix and the initial data corresponding to each relationship; a node update module, including a multi-layer node update network, which is used to obtain the node embedding vector corresponding to each of the nodes according to all the relationship embedding vectors and the node initialization embedding vectors.

[0008] In the method for generating the knowledge graph embedding vector provided by the present invention, it may also have the following feature: wherein, in the initialization module, the calculation expression for generating the initial data is: Where r i is the i-th relation e in the knowledge graph i The corresponding embedding vector is For the relationship i Corresponding to the initial data, H is a trainable parameter matrix.

[0009] In the method for generating the knowledge graph embedding vector provided by the present invention, it can also have the following characteristics: wherein, in the relationship update module, the relationship update networks of each layer are connected in sequence, and the output of the relationship update network of the previous layer is used as the input of the relationship update network of the next layer, and the relationship embedding vector corresponding to each relationship is the corresponding output generated by the relationship update network of the last layer, and the relationship update network of the lth layer generates the i-th relationship e in the knowledge graph. i The corresponding output The calculation expression is: Where σ is the LeakyRelu activation function, For the relationship i The neighbor relationship set, e j Neighbor relationship set The jth relationship in A learnable parameter matrix corresponding to the l-th layer relationship update network, Update the network pair relationship e for the l-1th layer relationship j The generated output is, For the relationship i Relationship j Update the network weights at the lth layer. Update the learnable row weight vector of the network for the l-th layer relationship, d is the dimension of the initial embedding vector, Update the learnable relation parameter matrix of the network for the lth layer relation, Update the network pair relationship e for the l-1th layer relationship i The generated output, || is the concatenation operation, Update the relationship e in the network for the lth layer relationship i Relationship j The index of the importance between them, s(i,j) is the relationship e i Relationship j The importance between j′ Neighbor relationship set The j′th relationship in .

[0010] In the method for generating the knowledge graph embedding vector provided by the present invention, it can also have the following characteristics: wherein, the corresponding affinity values ​​are calculated for any relationship and each corresponding neighbor relationship respectively, and the importance of the relationship and the index of the relationship are obtained according to the hyperparameter B and all the affinity values. The specific method is as follows: when the total number of different affinity values ​​is less than the hyperparameter B, the different affinity values ​​are sorted from large to small, and filled with 0 to generate a sequence with B numerical values, the importance of the relationship and the relationship of any neighbor is the corresponding affinity value, and the index is generated according to the serial number corresponding to the affinity value in the sorting; the total number of different affinity values ​​is equal to the hyperparameter B, sort the different affinity values ​​from large to small to generate a sequence with B values, the importance of the relationship to any neighbor relationship is the corresponding affinity value, and the index is generated according to the sequence number corresponding to the affinity value in the sorting; when the total number of different affinity values ​​is greater than the hyperparameter B, the importance of the relationship to any neighbor relationship is calculated according to the relationship adjacency matrix, and the affinity values ​​are divided into B subsets, and the index is generated according to the subset sequence number corresponding to the importance in the sorting generated from large to small for each different affinity value. When the number of different affinity values ​​is greater than the hyperparameter B, the calculation expression of the importance s(i,j) of the relationship i and the neighbor relationship j is: Where A is the relationship adjacency matrix, rank(a ij ) is a non-zero element a ij The ranking in the sorted list of non-zero elements in the relational adjacency matrix A, num(A) is the total number of all non-zero elements in the relational adjacency matrix A, and the calculation expression of the affinity value is: Where A is the relationship adjacency matrix, rank(a ij ) is a non-zero element a ij The rank of the non-zero elements in the sorted list of the relation adjacency matrix A, num(A) is the total number of all non-zero elements in the relation adjacency matrix A, and s(i,j) is the affinity value between relations i and j.

[0011] The method for generating the knowledge graph embedding vector provided by the present invention may also have the following characteristics: wherein, in the node updating module, the node updating networks of each layer are connected in sequence, and the output of the node updating network of the previous layer is used as the input of the node updating network of the next layer, and the node embedding vector corresponding to each node is the corresponding output generated by the node updating network of the last layer, and the node updating network of the lth layer generates the kth node v in the knowledge graph. k The corresponding output The calculation expression is:

[0012] Where σ is the LeakyRelu activation function, Update the network to a learnable parameter matrix corresponding to the l-th layer node, For node v k The set of neighbor nodes, v j For collection The jth node in For node v j With node v k The set of neighbor nodes of the relationship between them, r t is the tth relation in the relation neighbor node set, For node v k The average embedding vector of all related entity embedding vectors, v j′ For collection The j'th node in Except for node v j′ Other neighbor nodes other than node v k The set of relationships between Update the node v in the network for the lth layer node k and node v k The fine-grained parameters of For node v k The weight vector at layer l, is the transformation matrix of the node updating the network at the lth layer, d is the dimension of the initialization embedding vector, z jk For node v k and node v j The relationship embedding vector between them, || is the concatenation operation, z j′k For node v k and node v j′ The relationship embedding vector between is the correlation between node i and node j at layer l, is the embedding vector of node i in layer l, is the embedding vector of node j in layer l.

[0013] In the method for generating a knowledge graph embedding vector provided by the present invention, it may also have the following characteristics: wherein, each triple in the existing knowledge graph is taken as a positive triple, and a triple that does not exist in the existing knowledge graph is generated as a negative triple based on each of the positive triples, the fine-grained graph prediction model calculates a boundary loss function based on all the positive triples and the negative triples, and optimizes the fine-grained graph prediction model based on the loss calculation result, and the calculation expression of the boundary loss function is:

[0014] In the formula is the loss calculation result, T is the set of all positive triples, T′ is the set of all negative triples, e h and e′ h are the head nodes in the corresponding triples, r ht is the relationship in the corresponding triple, e t and e′ t are the tail nodes in the corresponding triples, γ i is the adaptive threshold corresponding to the i-th positive and negative triplet pair, is the embedding vector of the head node when the final score value is calculated, is the embedding vector of the tail node when the final score value is calculated, is the embedding vector of the relationship between two nodes when the final score function is calculated. is a trainable transformation matrix, d is the dimension of the initialization embedding vector, is a diagonal matrix.

[0015] The method for generating a knowledge graph embedding vector provided by the present invention may also have the following characteristics: wherein, in the step of constructing the relational adjacency matrix, the expression for constructing the relational adjacency matrix according to the graph structure information is: A: =A h +A t , Where A is the relationship adjacency matrix, is the head node relationship frequency matrix, is the tail node relationship frequency matrix, is the head node degree matrix, is the tail node degree matrix, n is the number of nodes, and m is the number of relationships.

[0016] The method for generating the knowledge graph embedding vector provided by the present invention may also have the following characteristics: wherein, in the embedding vector initialization step, each of the nodes and the relationship is initialized by the Glorot initialization method, and its expression is: In the formula is the initialization embedding vector obtained by the Glorot initialization method, is a normal distribution, and d is the dimension of the initialization embedding vector.

[0017] The present invention also provides a knowledge graph prediction and completion method for predicting and completing a knowledge graph, having the following characteristics, comprising the following steps: step S1, generating a knowledge graph embedding vector for the knowledge graph to obtain a node embedding vector corresponding to each node of the knowledge graph and a relationship embedding vector corresponding to each relationship; step S2, predicting and completing the knowledge graph based on all the node embedding vectors and the relationship embedding vectors to obtain an updated knowledge graph, wherein the knowledge graph embedding vector is generated by the method for generating a knowledge graph embedding vector as described in any one of the above.

[0018] The present invention also provides a knowledge graph prediction and completion device for predicting and completing a knowledge graph, having the following characteristics, including: an embedding vector generation unit, used to generate a knowledge graph embedding vector for the knowledge graph, and obtain a node embedding vector corresponding to each node of the knowledge graph and a relationship embedding vector corresponding to each relationship, a prediction and completion unit, used to predict and complete the knowledge graph based on all the node embedding vectors and the relationship embedding vectors, and obtain an updated knowledge graph, wherein the knowledge graph embedding vector is generated as the knowledge graph embedding vector generation method described in any one of claims 1 to 8.

[0019] Functions and Effects of the Invention

[0020] According to the method and application of generating the knowledge graph embedding vector involved in the present invention, because in the node update module, the attention mechanism based on the comprehensive correlation factor is used to further screen the neighbor node information with high influence on the target node on the basis of the important neighbor node set, so as to obtain the neighbor nodes with high influence in the target node embedding vector update, and then obtain the node embedding vector corresponding to each node. Therefore, the method and application of generating the knowledge graph embedding vector of the present invention can generate accurate node embedding vectors and relationship embedding vectors. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the process of generating a knowledge graph embedding vector in an embodiment of the present invention;

[0022] Figure 2 is a block diagram of a fine-grained graph prediction model in an embodiment of the present invention;

[0023] Figure 3 is a block diagram of a knowledge graph prediction and completion device in an embodiment of the present invention;

[0024] Figure 4 It is a flowchart of the knowledge graph prediction and completion method in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following embodiments and the accompanying drawings specifically illustrate the generation method and application of the knowledge graph embedding vector of the present invention.

[0026] This embodiment provides a method for generating a knowledge graph embedding vector, which is used to generate a relationship embedding vector and a node embedding vector corresponding to the knowledge graph.

[0027] Figure 1 It is a flowchart of a method for generating a knowledge graph embedding vector in an embodiment of the present invention.

[0028] like Figure 1 As shown in Figure 1, the method for generating a knowledge graph embedding vector includes the following steps:

[0029] Step T1 is the embedding vector initialization step, which initializes each node and relationship in the knowledge spectrum graph respectively to obtain the node initialization embedding vector corresponding to each node and the relationship initialization embedding vector corresponding to each relationship.

[0030] Among them, step T1 initializes each node and relationship through the Glorot initialization method, and its expression is:

[0031]

[0032] In the formula is the initialization embedding vector obtained by the Glorot initialization method, is a normal distribution, and d is the dimension of the initialization embedding vector.

[0033] Step T2 is the step of constructing the relational adjacency matrix, which constructs the relational adjacency matrix based on the graph structure information of the knowledge graph.

[0034] Among them, the expression for constructing the relational adjacency matrix based on the graph structure information is:

[0035] A:=A h +A t ,

[0036]

[0037] Where A is the relationship adjacency matrix, is the head node relationship frequency matrix, is the tail node relationship frequency matrix, is the head node degree matrix, is the tail node degree matrix, n is the number of nodes, and m is the number of relationships.

[0038] In this embodiment, E h[i,j] represents the head node relationship frequency matrix E h The element in the i-th row and j-th column in , that is, the frequency of interaction between node i and relation j when node i is the head node. Similarly, E t [i,j] represents the frequency of interaction between node i and relationship j when node i is the tail node. Then, the head node degree matrix D h Middle D h [i,i]:=∑ j E h [i,j], and similarly we can get the degree matrix D of the tail node t How the elements in are calculated.

[0039] In this embodiment, step T1 is performed first and then step T2. In other embodiments, step T2 may be performed first and then step T1, or step T1 and step T2 may be performed simultaneously.

[0040] Step T3 is the embedding vector generation step, which inputs all node initialization embedding vectors, relationship initialization embedding vectors and relationship adjacency matrices into the fine-grained graph prediction model to obtain relationship embedding vectors and node embedding vectors.

[0041] Figure 2 is a block diagram of a fine-grained graph prediction model in an embodiment of the present invention.

[0042] like Figure 2 As shown, the fine-grained graph prediction model 100 includes an initialization module 11, a relationship update module 12 and a node update module 13.

[0043] The initialization module 11 is used to initialize each relationship initialization embedding vector to obtain corresponding initial data.

[0044] Among them, in the initialization module, the calculation expression for generating initial data is:

[0045]

[0046] In the formula is the i-th relation e in the knowledge graph i The corresponding embedding vector is For the relationship i The corresponding initial data, is a trainable parameter matrix.

[0047] The relationship updating module 12 includes a multi-layer relationship updating network, which is used to obtain the relationship embedding vector corresponding to each relationship according to the relationship adjacency matrix and the initial data corresponding to each relationship.

[0048] In the relation updating module 12, each layer of relation updating network is connected in sequence, and the output of the previous layer of relation updating network is used as the input of the next layer of relation updating network. Then, the relation embedding vector corresponding to each relation is the corresponding output generated by the last layer of relation updating network.

[0049] The l-th layer relation update network generates the i-th relation e in the knowledge graph i The corresponding output The calculation expression is:

[0050]

[0051] Where σ is the LeakyRelu activation function, For the relationship i The neighbor relationship set, e j Neighbor relationship set The jth relationship in A learnable parameter matrix corresponding to the l-th layer relationship update network, Update the network pair relationship e for the l-1th layer relationship j The generated output is the output embedding vector, For the relationship i Relationship j Update the network weights at the lth layer. Update the learnable row weight vector of the network for the l-th layer relationship, d is the dimension of the initial embedding vector, Update the learnable relation parameter matrix of the network for the lth layer relation, Update the network pair relationship e for the l-1th layer relationship i The generated output is the output embedding vector, || is the concatenation operation, Update the relationship e in the network for the lth layer relationship i Relationship j The index of the importance between them, s(i,j) is the relationship e i Relationship j The importance between j′ Neighbor relationship set The j′th relationship in .

[0052] The corresponding affinity values ​​are calculated for each of the arbitrary relationships and the corresponding neighbor relationships, and the calculation expression of the affinity value is:

[0053]

[0054] Where A is the relationship adjacency matrix, rank(a ij ) is a non-zero element a ijThe rank of the non-zero elements in the sorted list of the relation adjacency matrix A, num(A) is the total number of all non-zero elements in the relation adjacency matrix A, and s(i,j) is the affinity value between relations i and j.

[0055] Then, according to the hyperparameter B and all affinity values, the specific way to obtain the importance and index of the relationship with any neighbor is:

[0056] When the total number of different affinity values ​​is less than the hyperparameter B, the different affinity values ​​are sorted from large to small and filled with 0 to generate a sequence with B values. The importance of this relationship to any neighbor relationship is the corresponding affinity value, and the index is generated according to the sequence number corresponding to the affinity value in the sort.

[0057] When the total number of different affinity values ​​is equal to the hyperparameter B, the different affinity values ​​are sorted from large to small to generate a sequence with B values. The importance of this relationship with any neighbor relationship is the corresponding affinity value, and the index is generated according to the sequence number corresponding to the affinity value in the sort.

[0058] When the total number of different affinity values ​​is greater than the hyperparameter B, the importance of the relationship with any neighbor relationship is calculated according to the relationship adjacency matrix, and the affinity values ​​are divided into B subsets. The index is generated according to the subset number corresponding to the importance in the sorting generated from large to small according to the different importance.

[0059] Furthermore, when the number of different affinity values ​​is greater than the hyperparameter B, the calculation expression for the importance s(i,j) of relationship i and neighbor relationship j is:

[0060]

[0061] Where A is the relationship adjacency matrix, rank(a ij ) is a non-zero element a ij The rank in the sorted list of nonzero elements in the relational adjacency matrix A. num(A) is the total number of nonzero elements in the relational adjacency matrix A.

[0062] The node updating module 13 includes a multi-layer node updating network, which is used to obtain the node embedding vector corresponding to each node according to all relationship embedding vectors and node initialization embedding vectors.

[0063] In the node update module 13, each layer of node update network is connected in sequence, and the output of the node update network of the previous layer is used as the input of the node update network of the next layer. The node embedding vector corresponding to each node is the corresponding output generated by the node update network of the last layer.

[0064] The l-th layer node updates the network to generate the k-th node v in the knowledge graph kThe corresponding output The calculation expression is:

[0065]

[0066]

[0067] Where σ is the LeakyRelu activation function, Update the network to a learnable parameter matrix corresponding to the l-th layer node, For node v k The set of neighbor nodes, v j For collection The jth node in For node v j With node v k The set of neighbor nodes of the relationship between them, r t is the tth relation in the relation neighbor node set, For node v k The average embedding vector of all related entity embedding vectors, v j′ For collection The j′th node in Except for node v j′ Other neighbor nodes other than node v k The set of relationships between Update the node v in the network for the lth layer node k and node v k The fine-grained parameters of For node v k The weight vector at layer l, is the transformation matrix of the node updating the network at the lth layer, d is the dimension of the initialization embedding vector, z jk For node v k and node v j The relationship embedding vector between them, || is the concatenation operation, z j′k For node v k and node v j′ The relationship embedding vector between is the correlation between node i and node j at layer l, is the embedding vector of node i in layer l, is the embedding vector of node j in layer l.

[0068] In this embodiment, a comprehensive correlation factor is set to qualitatively evaluate the correlation between two nodes. For example, the comprehensive correlation factor corresponding to node i and node j is τ ij, thus solving the problem that the traditional attention weight only measures the importance of nodes without considering the relevance of these important nodes to the target node. When the comprehensive correlation factor is close to 1, it means that there is a strong correlation between the two nodes. When the comprehensive correlation factor tends to a value of -1, it means that there is a low correlation between the two nodes. When the comprehensive correlation factor is 0, it means that there is no obvious influence between the two nodes, that is, they are not related.

[0069] In this embodiment, the fine-grained graph prediction model 100 is trained using a training data set in combination with the loss calculation result of the boundary loss function, thereby optimizing the parameters of the fine-grained graph prediction model 100 .

[0070] The training data set includes positive triples and negative triples. Positive triples are triples in the existing knowledge graph. Negative triples are triples generated based on positive triples that do not exist in the existing knowledge graph. In this embodiment, negative triples are obtained by randomly replacing the head node or tail node of the positive triple.

[0071] The boundary loss function of this embodiment is used to improve the generalization and robustness of the fine-grained graph prediction model 100, and its calculation expression is:

[0072]

[0073] In the formula is the loss calculation result, T is the set of all positive triples, T′ is the set of all negative triples, e h and e′ h are the head nodes in the corresponding triples, r ht is the relationship in the corresponding triple, e t and e′ t are the tail nodes in the corresponding triples, γ i is the adaptive threshold corresponding to the i-th positive and negative triplet pair, is the embedding vector of the head node when the final score value is calculated, is the embedding vector of the tail node when the final score value is calculated, is the embedding vector of the relationship between two nodes when the final score function is calculated. is a trainable transformation matrix, d is the dimension of the initialization embedding vector, is a diagonal matrix.

[0074] γ i The calculation expression is:

[0075] γ i =ξ i +∈,

[0076] In the formula, ξ iis the relaxation factor of the ith positive triplet and the corresponding negative triplet, and ∈ is a preset minimum value. In this embodiment, the minimum value ∈ is 0.01.

[0077] In this embodiment, the fine-grained graph prediction model 100 is denoted as Our model, and its performance is compared with the existing GraIL, CoMPILE, SNRI, INDIGO, RMPI, CompGCN, NodePiece, NeuralLP, DRUM, BLP, QBLP, NBFNet, RED-GNN and IMGRAM models on multiple existing NELL-995 datasets. The specific results are shown in the following table:

[0078]

[0079] In the above table, the first column is each model, and the second to fourth columns, the fifth to seventh columns, the eighth to tenth columns, and the eleventh to thirteenth columns are the average reciprocal ranking MRR, top 1 hit rate H@1, and top 10 hit rate H@10 on the NL-100 subset, NL-75 subset, NL-50 subset, and NL-25 subset of the NELL-995 dataset, respectively. For example, the cell in the second column of the third row indicates that the MRR of the GraIL model on the NL-100 subset is 0.135. It can be seen that the fine-grained graph prediction model 100 has the best prediction performance compared to the above existing models.

[0080] In this embodiment, an ablation experiment is further performed on the Our model. The model that does not use the comprehensive correlation factor and the boundary loss function is used as the BM model, and the model that only uses the boundary loss function is used as the Model-1 model; the model that only uses the comprehensive correlation factor is used as the Model-2 model. The above models are tested on the NELL-995 dataset. The specific results are shown in the following table:

[0081]

[0082] In the above table, the first column is each model, and the second to fourth columns, the fifth to seventh columns, the eighth to tenth columns, and the eleventh to thirteenth columns are the average reciprocal ranking MRR, top 1 hit rate H@1, and top 10 hit rate H@10 on the NL-100 subset, NL-75 subset, NL-50 subset, and NL-25 subset of the NELL-995 dataset, respectively. For example, the cell in the second column of the third row indicates that the MRR of the BM model on the NL-100 subset is 0.351.

[0083] As can be seen from the above table, although the Model-1 model shows significant improvements in predicting tasks involving unknown nodes and relations, it does not fully address the fundamental limitations inherent in the baseline model. This shortcoming means that in cases where most of the data consists of known nodes and relations, using only the adaptive margin loss function does not fully improve the performance of the model. On the other hand, although the use of the comprehensive factor-based attention mechanism alone in the Model-2 model cannot fully make up for the shortcomings of the baseline model when dealing with a large amount of known data, it has a significant improvement in performance for datasets containing a large number of known nodes and relations. This observation emphasizes that the comprehensive factor-based attention mechanism can enhance the model's predictive ability in datasets where prior knowledge is prevalent.

[0084] The method for generating the knowledge graph embedding vector of this embodiment can generate a relationship embedding vector and a node embedding vector for knowledge graph prediction and completion. Therefore, in combination with the above-mentioned method for generating the knowledge graph embedding vector, this embodiment also provides a knowledge graph prediction and completion device.

[0085] Figure 3 It is a block diagram of a knowledge graph prediction and completion device in an embodiment of the present invention.

[0086] like Figure 3 As shown, the knowledge graph prediction and completion device 200 includes an embedding vector generation unit 21 and a prediction and completion unit 22.

[0087] The embedding vector generating unit 21 is used to generate a knowledge graph embedding vector for the knowledge graph, and obtain a node embedding vector corresponding to each node of the knowledge graph and a relationship embedding vector corresponding to each relationship. The embedding vector generating unit 21 uses the knowledge graph embedding vector generating method of this embodiment to generate the knowledge graph embedding vector.

[0088] The prediction and completion section 22 is used to predict and complete the knowledge graph based on all node embedding vectors and relationship embedding vectors to obtain an updated knowledge graph.

[0089] The following describes the process of using the knowledge graph prediction and completion device 200 to perform a knowledge graph prediction and completion method in conjunction with the accompanying drawings.

[0090] Figure 4 It is a flowchart of the knowledge graph prediction and completion method in an embodiment of the present invention.

[0091] like Figure 4 As shown, the knowledge graph prediction and completion method includes the following steps:

[0092] Step S1, use the embedding vector generation unit 21 to generate a knowledge graph embedding vector for the knowledge graph, and obtain a node embedding vector corresponding to each node of the knowledge graph and a relationship embedding vector corresponding to each relationship.

[0093] Step S2, using the prediction and completion unit 22 to predict and complete the knowledge graph according to all node embedding vectors and relationship embedding vectors to obtain an updated knowledge graph.

[0094] Functions and Effects of the Embodiments

[0095] According to the method and application of generating knowledge graph embedding vectors involved in this embodiment, in the node update module, the attention mechanism based on the comprehensive correlation factor is used to further screen the neighbor node information with high influence on the target node based on the set of important neighbor nodes, so as to obtain the neighbor nodes with high influence in updating the target node embedding vector, and then obtain the node embedding vector corresponding to each node. In short, this method can generate accurate node embedding vectors and relationship embedding vectors.

[0096] Furthermore, a specific threshold is provided for each positive triplet through a boundary loss function based on an adaptive boundary threshold, which avoids the overfitting problem of the embedding vectors of nodes and relations, thereby making the trained model highly generalizable and robust.

[0097] Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for generating a knowledge graph embedding vector, used to generate a relationship embedding vector and a node embedding vector corresponding to a knowledge graph, characterized in that: The following steps are involved: The embedding vector initialization step is to initialize each node and relationship in the knowledge spectrum graph respectively to obtain a node initialization embedding vector corresponding to each node and a relationship initialization embedding vector corresponding to each relationship; A relational adjacency matrix construction step, constructing a relational adjacency matrix according to the graph structure information of the knowledge graph; The embedding vector generation step inputs all the node initialization embedding vectors, the relationship initialization embedding vectors and the relationship adjacency matrix into the fine-grained graph prediction model to obtain the relationship embedding vector and the node embedding vector. Wherein, the fine-grained graph prediction model includes: An initialization module, used to initialize each of the relationship initialization embedding vectors to obtain corresponding initial data; A relationship updating module, comprising a multi-layer relationship updating network, for obtaining the relationship embedding vector corresponding to each relationship according to the relationship adjacency matrix and the initial data corresponding to each relationship; The node updating module includes a multi-layer node updating network, which is used to obtain the node embedding vector corresponding to each of the nodes according to all the relationship embedding vectors and the node initialization embedding vector.

2. The method for generating a knowledge graph embedding vector according to claim 1, characterized in that: in, In the initialization module, the calculation expression for generating the initial data is: Where r i is the i-th relation e in the knowledge graph i The corresponding embedding vector is For the relationship i Corresponding to the initial data, H is a trainable parameter matrix.

3. The method for generating a knowledge graph embedding vector according to claim 1, characterized in that: in, In the relationship update module, the relationship update networks of each layer are connected in sequence, and the output of the relationship update network of the previous layer is used as the input of the relationship update network of the next layer. The relation embedding vector corresponding to each of the relations is the corresponding output generated by the last layer of the relation update network. The l-th layer relation update network generates the i-th relation e in the knowledge graph i The corresponding output The calculation expression is: Where σ is the LeakyRelu activation function, For the relationship i The neighbor relationship set, e j Neighbor relationship set The jth relationship in A learnable parameter matrix corresponding to the l-th layer relationship update network, Update the network pair relationship e for the l-1th layer relationship j The generated output is, For the relationship i Relationship j Update the network weights at the lth layer. Update the learnable row weight vector of the network for the l-th layer relationship, d is the dimension of the initial embedding vector, Update the learnable relation parameter matrix of the network for the lth layer relation, Update the network pair relationship e for the l-1th layer relationship i The generated output, || is the concatenation operation, Update the relationship e in the network for the lth layer relationship i Relationship j The index of the importance between them, s(i,j) is the relationship e i Relationship j The importance between j′ Neighbor relationship set The j′th relationship in .

4. The method for generating a knowledge graph embedding vector according to claim 3, Features: The corresponding affinity values ​​are calculated for each of the arbitrary relations and the corresponding neighbor relations, and the specific method of obtaining the importance and the index of the relation and the arbitrary neighbor relations according to the hyperparameter B and all the affinity values ​​is as follows: When the total number of different affinity values ​​is less than the hyperparameter B, the different affinity values ​​are sorted from large to small and filled with 0 to generate a sequence with B values, the importance of the relationship with any neighbor relationship is the corresponding affinity value, and the index is generated according to the sequence number corresponding to the affinity value in the sorting; When the total number of different affinity values ​​is equal to the hyperparameter B, the different affinity values ​​are sorted from large to small to generate a sequence with B values, the importance of the relationship with any neighbor relationship is the corresponding affinity value, and the index is generated according to the sequence number corresponding to the affinity value in the sorting; When the total number of different affinity values ​​is greater than the hyperparameter B, the importance of the relationship with any neighbor relationship is calculated according to the relationship adjacency matrix, and the affinity values ​​are divided into B subsets. The index is generated according to the subset number corresponding to the importance in the order generated from large to small for each different importance. When the number of different affinity values ​​is greater than the hyperparameter B, the calculation expression of the importance s(i, j) of relationship i and neighbor relationship j is: Where A is the relationship adjacency matrix, rank(a ij ) is a non-zero element a ij The rank of the non-zero elements in the relational adjacency matrix A in the sorted list. num(A) is the total number of all non-zero elements in the relational adjacency matrix A. The calculation expression of the affinity value is: Where A is the relationship adjacency matrix, rank(a ij ) is a non-zero element a ij The rank of the non-zero elements in the sorted list of the relation adjacency matrix A, num(A) is the total number of all non-zero elements in the relation adjacency matrix A, and s(i,j) is the affinity value between relations i and j.

5. The method for generating a knowledge graph embedding vector according to claim 1, characterized in that: in, In the node update module, the node update networks of each layer are connected in sequence, and the output of the node update network of the previous layer is used as the input of the node update network of the next layer. The node embedding vector corresponding to each of the nodes is the corresponding output generated by the last layer of the node update network, The l-th layer node updates the network to generate the k-th node v in the knowledge graph k The corresponding output The calculation expression is: Where σ is the LeakyRelu activation function, Update the network to a learnable parameter matrix corresponding to the l-th layer node, For node v k The set of neighbor nodes, v j For collection The jth node in For node v j With node v k The set of neighbor nodes of the relationship between them, r t is the tth relation in the relation neighbor node set, For node v k The average embedding vector of all related entity embedding vectors, v j′ For collection The j′th node in Except for node v j′ Other neighbor nodes other than node v k The set of relationships between Update the node v in the network for the lth layer node k and node v k The fine-grained parameters of For node v k The weight vector at layer l, is the transformation matrix of the node updating the network at the lth layer, d is the dimension of the initialization embedding vector, z jk For node v k and node v j The relationship embedding vector between them, || is the concatenation operation, zj ′k For node v k and node v j′ The relationship embedding vector between is the correlation between node i and node j at layer l, is the embedding vector of node i in layer l, is the embedding vector of node j in layer l.

6. The method for generating a knowledge graph embedding vector according to claim 1, characterized in that: in, Each triple in the existing knowledge graph is regarded as a positive triple, and a triple that does not exist in the existing knowledge graph is generated according to each positive triple as a negative triple, The fine-grained graph prediction model calculates a boundary loss function according to all the positive triples and the negative triples, and optimizes the fine-grained graph prediction model according to the loss calculation result. The calculation expression of the boundary loss function is: Where L is the loss calculation result, T is the set of all positive triples, T′ is the set of all negative triples, e h and e′ h are the head nodes in the corresponding triples, r ht is the relationship in the corresponding triple, e t and e′ t are the tail nodes in the corresponding triples, γ i is the adaptive threshold corresponding to the i-th positive and negative triplet pair, is the embedding vector of the head node when the final score value is calculated, is the embedding vector of the tail node when the final score value is calculated, is the embedding vector of the relationship between two nodes when the final score function is calculated. is a trainable transformation matrix, d is the dimension of the initialization embedding vector, is a diagonal matrix.

7. The method for generating a knowledge graph embedding vector according to claim 1, characterized in that: in, In the step of constructing the relational adjacency matrix, the expression for constructing the relational adjacency matrix according to the graph structure information is: A:=A h +A t , Where A is the relationship adjacency matrix, is the head node relationship frequency matrix, is the tail node relationship frequency matrix, is the head node degree matrix, is the tail node degree matrix, n is the number of nodes, and m is the number of relationships.

8. The method for generating a knowledge graph embedding vector according to claim 1, characterized in that: in, In the embedding vector initialization step, each of the nodes and the relationship is initialized by the Glorot initialization method, and its expression is: In the formula is the initialization embedding vector obtained by the Glorot initialization method, is a normal distribution, and d is the dimension of the initialization embedding vector.

9. A knowledge graph prediction and completion method, used to predict and complete a knowledge graph, characterized in that: The following steps are involved: Step S1, generating a knowledge graph embedding vector for the knowledge graph to obtain a node embedding vector corresponding to each node of the knowledge graph and a relationship embedding vector corresponding to each relationship. Step S2: predicting and completing the knowledge graph according to all the node embedding vectors and the relationship embedding vectors to obtain an updated knowledge graph. Among them, the knowledge graph embedding vector is generated by the method for generating the knowledge graph embedding vector as described in any one of claims 1 to 8.

10. A knowledge graph prediction and completion device, used for predicting and completing a knowledge graph, characterized in that: include: An embedding vector generating unit is used to generate a knowledge graph embedding vector for the knowledge graph, and obtain a node embedding vector corresponding to each node of the knowledge graph and a relationship embedding vector corresponding to each relationship. Predict and complete all, used to predict and complete the knowledge graph according to all the node embedding vectors and the relationship embedding vectors to obtain an updated knowledge graph, Among them, the knowledge graph embedding vector is generated by the method for generating the knowledge graph embedding vector as described in any one of claims 1 to 8.

Images