A link prediction method based on strengthening entity structure feature representation learning

Abstract

The present invention discloses a link prediction method based on strengthening entity structure feature representation learning. First, the system generates an initialization vector according to entity data of a knowledge graph, and then inputs the entity initialization vector into an entity representation learning framework. In the representation learning framework, the entity initial vector is used to extract the entity's relational features through an RGCN model, and then a GCN model and a GAT model are combined to extract and learn the entity relationship structure features reflected by the entity structure features as a supplement to the relationship feature information, and multiple gating mechanisms are used to perform multi-feature fusion; finally, the entity vector is input into a link prediction model to generate a predicted relationship entity triple and verify the prediction result; the present invention effectively solves the problems of difficulty in multi-source information fusion and insufficient relational structure features faced by a link prediction system, and the relational feature information contained in the entity feature vector is sufficient to identify entities corresponding to complex relationships, thereby improving the effect of link prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of link prediction, and mainly to a link prediction method based on strengthening entity structure feature representation learning. Background Art

[0002] The knowledge graph is a knowledge base that presents entities in the real world and the relationships between entities in the form of a graph structure. In the knowledge graph, nodes represent entities in the real world, and the edges between nodes represent the relationships between entities. Although the knowledge graph can provide high-quality structured data, most of the knowledge graphs currently have sparse graph structures due to various factors, and a large number of implicit relationships between entities are not represented. Therefore, link prediction is an effective method to complete the knowledge graph, improve the knowledge representation in the knowledge graph, and improve the quality of the knowledge graph.

[0003] Link prediction refers to predicting the possible relationship between entities through the relationship entities in the existing knowledge graph. Many recent studies have focused on how to better represent the knowledge graph to improve the link prediction effect. The link prediction method based on entity representation learning is generally divided into an encoder part and a decoder part: in the encoder part, the entity data is input into the entity representation learning model for learning. During the learning process, the learning is carried out according to the existing positive triples in the knowledge graph. The purpose of setting the objective function is to make the generated entity representation vector, when used for the calculation of the score function, make the score of the correctly predicted relationship entity triple as high as possible and the score of the incorrectly predicted relationship entity triple as low as possible, and finally obtain the entity representation vector through learning; in the decoder part, the entity representation learning is obtained by using the score function to calculate the score of the predicted triple (head entity, relationship, tail entity), which is used as the basis for judging whether it is a correct triple.

[0004] Currently, the link prediction methods based on entity representation learning are mainly divided into methods based on the TransE model and methods based on graph convolutional networks. In the learning process of the link prediction method based on the TransE model, the main consideration is the connection between entities and relations. Usually, the entities are modeled and feature learned based on the head entity vector + relation vector = tail entity vector and the corresponding transformation equation.

[0005] In the process of entity representation learning, the link prediction method based on graph convolutional network models and learns the relationship between entities according to the feature representation of the adjacent nodes of the central entity. After obtaining the entity representation vector, the score of the corresponding predicted triple is calculated through the scoring function as the basis for judging whether it is the correct triple.

[0006] Currently, link prediction methods based on entity representation learning usually have the following problems:

[0007] (1) Difficulty in modeling complex relationships: In real graph data, the types of relationships between entities are very diverse and complex. In previous link prediction methods based on entity representation learning, only the simplest one-to-one relationship was generally considered, but the modeling capabilities of a large number of "one-to-many, many-to-many, and many-to-one" relationships in real graph data were insufficient.

[0008] (2) Difficulty in fusing multi-source feature information: The learning of entity feature information is usually insufficient. Due to the lack of feature information in entity representation, the distinction between entity feature vectors is insufficient, which can easily lead to the prediction of incorrect relationship entity triples between entities that should not have corresponding relationships when performing link prediction on sparse graph data.

[0009] (3) The sparseness of the knowledge graph leads to fewer positive examples: In real graph data, entities and relations are very sparse, and the number of different relations is also relatively large. This leads to a lack of sufficient positive example relation entity triples as learning samples in the process of entity representation learning. For relations that appear less frequently in relation triples, it is difficult to extract the difference features of different relations during the entity representation learning process due to the lack of positive examples for training, resulting in a decrease in link prediction effect.

[0010] Therefore, in response to the above three types of problems, new link prediction methods based on entity representation must make corresponding considerations and propose corresponding countermeasures in the process of entity representation learning. Summary of the invention

[0011] Purpose of the invention: In view of the problems existing in the above-mentioned background technology, the present invention provides a link prediction method based on strengthening the learning of entity structure feature representation. Based on the excellent ability of graph convolutional network to learn features of entity data, the relationship features of the entity are first extracted and learned, and then the adjacency structure features of the entity are processed accordingly as a supplement and enrichment of the entity relationship features, so as to improve the representation ability of the relationship features in the entity representation; finally, the score of the predicted triple is calculated by putting the entity representation and the relationship matrix into the scoring function to serve as the basis for whether the predicted triple is correct.

[0012] Technical solution: To achieve the above purpose, the technical solution adopted by the present invention is:

[0013] A link prediction method based on strengthening entity structure feature representation learning includes the following steps:

[0014] Step S1: Input the initialization vector generated by the knowledge graph entity data into the RGCN model, extract and learn the entity relationship features, and obtain the entity relationship feature representation vector;

[0015] Step S2, supplementing entity relationship features based on entity adjacency structure features;

[0016] The initialization vector generated by the knowledge graph entity data is input into the GCN model and the GAT model respectively; in each layer of the entity representation learning network, the GCN model extracts and learns the neighbor structure features of the central entity node, and the GAT model assigns weights according to the importance of the entity node to reflect the importance of the adjacent nodes, extracts features from the entity adjacent nodes, and uses the importance of the adjacent nodes to reflect the importance of the corresponding relationship; the importance features obtained by the GAT model are used as a supplement to the neighbor structure features obtained by the GCN model, and the two features are aggregated based on the average gating mechanism to obtain the entity's relationship structure feature representation vector;

[0017] Step S3, the entity relationship feature representation vector obtained in step S1 and the relationship structure feature representation vector in step S2 are aggregated again based on Res gate gating to obtain the entity representation vector; in the learning and training process, the sampling of negative samples is strengthened as a supplement to the lack of positive samples to improve the effect of learning and training the entity representation vector;

[0018] Step S4: performing link prediction based on the entity representation vector;

[0019] The scoring function based on DistMult decomposition is selected to calculate the score of the predicted relation entity triples to determine whether they can be regarded as correct triples.

[0020] Furthermore, the method for extracting entity relationship features in step S1 includes:

[0021] First, the adjacent nodes of the central entity node are classified according to the relationship between the central node and the corresponding adjacent nodes; then a learnable relationship transformation matrix is ​​set for the relationship between the central node and the corresponding adjacent nodes; finally, for the central node, a reflexive relationship is added to maintain the node's own characteristics during the forward propagation learning process, where the forward propagation formula for entity representation learning is as follows:

[0022]

[0023] in, The entity representation vector of the next layer node i in the network; σ represents the activation function, here relu() is used; R represents the set of all relations in the graph; N r represents the set of neighbor nodes corresponding to node i under relationship r; c i,r represents the regularization constant; Represents the relational feature conversion parameters at this layer; represents the feature representation of the adjacent node j; W l represents the learnable parameter matrix; It represents the feature representation of node i in the previous layer.

[0024] Furthermore, the formula for fusing feature information using the average gating mechanism in step S2 is expressed as follows:

[0025] h i =(1-α)h i1 +αh i2

[0026] where h i is the relational structural feature information of entity i; h i1 、h i2 are the relational structure feature vectors of entity i obtained by the GCN and GAT models respectively; α is the fusion coefficient, ranging from (0,1], which is used to aggregate relational distribution features and relational importance features.

[0027] Furthermore, the formula for fusing feature information using the Res gate gating mechanism in step S3 is as follows:

[0028] h i =W 2 (σ(W 1 h i1 +b))+h i2

[0029] where h i is the final feature representation vector of entity i; h i1 、h i2 are the relational structure feature vector and relational feature vector model of entity i; W 1 , W 2 is a learnable parameter matrix; σ is an activation function, and the relu() activation function is selected.

[0030] Furthermore, the score function based on DistMult decomposition is expressed as follows:

[0031]

[0032] Among them, h h The feature representation vector representing the head entity h in the triple; h t Represents the feature representation vector of the tail entity t in the triple; R r The diagonal matrix corresponding to the relation r is the relation transformation matrix. By using DistMult decomposition, the head and tail entity feature vectors and the relation r are projected to a specific value as the correct probability score of the link prediction triple.

[0033] Beneficial effects:

[0034] The link prediction method for strengthening the learning of entity structure feature representation provided by the present invention creatively uses entity adjacency structure features to supplement entity relationship features, learns entity representation on this basis, and fuses feature information through a multi-gating mechanism, so that the relationship feature information in the learned entity representation vector is sufficient, and the entity vector representation accuracy corresponding to the complex relationship is high. When calculating the score function for link prediction, the relationship feature information contained in the entity feature vector is sufficient to identify the entity corresponding to the complex relationship, which significantly improves the effect of link prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of a link prediction method based on enhanced entity structure feature representation learning provided by the present invention;

[0036] Figure 2 It is a flow chart of the method for extracting and learning relational structural features provided by the present invention. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] The present invention uses entity adjacency structure features to supplement entity relationship features, performs representation learning on entity data, and then uses the learned entity representation as the basis for calculating the triple score of link prediction. The main process is as follows: Figure 1As shown. The method proposed in the present invention is mainly divided into an encoding part and a decoding part. In the encoding process of the entity, the entity representation learning is performed by inputting the initialization representation of the entity into the improved RGCN model, the structural feature learning network composed of the GAT model and the GCN model respectively: the entity relationship feature representation is obtained through the RGCN model, and the entity structure feature information obtained in the structural feature learning network can extract the relationship situation reflected in the structural feature of the entity to obtain the relationship structure feature of the entity. In the learning process of relationship features and relationship structure features, a sampling method that strengthens the collection of negative samples is used to improve the learning effect of positive relationship entity triples. Finally, the learned multi-source feature representation is used to realize information fusion through a special gating mechanism, and the obtained entity representation can have sufficient distinction between complex relationships between entities. In the decoding part, the present invention uses a scoring function based on DistMult decomposition to score the predicted relationship entity triples to obtain the probability score of the predicted triple being the correct triple. The higher the score, the more likely the predicted triple is to be correct. The following Table 1 is the specific process of the link prediction method provided by the present invention:

[0039] Table 1 Flowchart of link prediction method

[0040]

[0041] The following is a specific example to further describe the link prediction method of the present invention:

[0042] Step S1: Input the initialization vector generated by the knowledge graph entity data into the RGCN model, extract and learn the entity relationship features, and obtain the entity relationship feature representation vector.

[0043] First, the adjacent nodes of the central entity node are classified according to the relationship between the central node and the adjacent nodes. Then, a learnable relationship transformation matrix is ​​set for the relationship between the central entity node and the adjacent nodes. Finally, for the central node, a reflexive relationship is added to maintain the node's own characteristics during the forward propagation learning process. The forward propagation formula for entity representation learning is as follows:

[0044]

[0045] in, The entity representation vector of the next layer node i in the network. σ represents the activation function, and relu() is used here. R represents the set of all relations in the graph. N r Represents the set of neighbor nodes corresponding to node i under relationship r. i,r represents the regularization constant. Represents the relation feature transformation parameters at this layer. W represents the feature representation of the adjacent node j. l represents the learnable parameter matrix. It represents the feature representation of node i in the previous layer.

[0046] Step S2: supplement entity relationship features based on entity adjacency structure features.

[0047] The initialization vectors generated by the knowledge graph entity data are input into the GCN model and the GAT model respectively. In each layer of the entity representation learning network, the GCN model extracts and learns the neighbor structure features of the central entity node. The GAT model assigns weights according to the importance of the entity node to reflect the importance of the adjacent nodes, extracts features from the entity adjacent nodes, and uses the importance of the adjacent nodes to reflect the importance of the corresponding relationship. The importance features obtained by the GAT model are used as a supplement to the neighbor structure features obtained by the GCN model. The two features are aggregated based on the average gating mechanism to obtain the entity's relational structure feature representation vector.

[0048] In this step, the basic GCN model is first used to learn and extract the neighbor structure features of the central node. At the same time, since the importance of each adjacent node to the central node is different, the importance of the adjacent node can also be used to represent the importance of the relationship. Therefore, this embodiment also uses the GAT model to capture structural features according to the different importance of adjacent nodes, so as to reflect the difference in importance between the relationships owned by the central entity. The entity representation learned by GAT is then used as a supplement to the structural features obtained by the GCN model, and the two are fused according to a certain weight through a gating mechanism, so that the neighbor structure feature information can reflect the distribution and quantity characteristics of the overall relationship, as well as the differences in the importance of the corresponding relationships of different nodes. Figure 2 As shown, in the learning module of relationship structure feature information, the present invention makes corresponding improvements and adjustments to the GCN model and the GAT model, and learns the adjacency structure features and relationship importance features that reflect the distribution characteristics of the central entity relationship in parallel in the entire structure feature learning framework, and then aggregates the relationship distribution features and relationship importance features through average gating.

[0049] The formula for fusion feature information of the average gating mechanism is as follows:

[0050] h i =(1-α)h i1 +αh i2

[0051] where h i It is the relational structural feature information of entity i. i1 、h i2are the relational structure feature vectors of entity i obtained by the GCN and GAT models, respectively. α is the fusion coefficient, ranging from (0,1], which is used to aggregate relational distribution features and relational importance features. Since the relational structure feature information of the central entity is related to the adjacent nodes, it is necessary to include sufficient adjacent structure feature information in the final relational structure feature representation vector. At the same time, due to the diversity of relations, it is necessary to learn the adjacent nodes according to their importance, so that the importance of the corresponding relationship can be reflected by the importance of the adjacent nodes, so that the difference between different relationships can be more obvious. By using average gating, the features learned by the GAT model based on the importance of the adjacent nodes can use the importance of the adjacent nodes as the importance of each relationship corresponding to the central entity, thereby reflecting the differentiated feature information on the relational structure. At the same time, the adjacent node features that are ignored in the GAT model due to the low attention coefficient can be retained in the node feature representation learned by GCN, so that the loss of feature information in the fusion process is minimized.

[0052] Step S3, the entity relationship feature representation vector obtained in step S1 and the relationship structure feature representation vector in step S2 are aggregated again based on Res gate gating to obtain the entity representation vector. In the learning and training process, the improved negative example sampling method is used to improve the effect of learning and training the entity representation vector.

[0053] The formula of the Res gate gating mechanism used to fuse feature information is as follows:

[0054] h i =W 2 (σ(W 1 h i1 +b))+h i2

[0055] where h i is the final feature representation vector of entity i. i1 、h i2 are the relational structure feature vector and relational feature vector model of entity i respectively. 1 , W 2 is a learnable parameter matrix. σ is an activation function, and relu() activation function is selected. Since relational features are the most important for link prediction, they should be retained as much as possible during the fusion process. At the same time, in order to ensure that the relational structural feature information can be used as a supplement to the relational features without weakening the entity relation features contained in the overall vector due to the introduction of relational structural features, the adjusted Res gate gating is used as the fusion method of relational feature information and relational structural feature information.

[0056] A new negative example sampling method is used in the data sampling process when the model is trained. Due to the sparsity of real graph data, the number of correct relationship entity triplets as positive examples is small. Therefore, the link prediction method proposed in this embodiment strengthens the sampling of negative example samples as a supplement to the lack of positive example samples during learning. In previous studies, negative example sampling methods mostly randomly replace the head entity or tail entity in each correct relationship triple with other entities, so that the original correct relationship triple becomes an incorrect triple sample. The negative example sampling method used in this embodiment will randomly replace the relationship while replacing the head and tail entities, so as to try to avoid generating correct relationship entity triples when randomly replacing the negative example triplets.

[0057] Step S4: perform link prediction based on the entity representation vector.

[0058] After obtaining the entity feature representation vector, the entity representation vector can be decoded to complete the link prediction. In this embodiment, a score function based on DistMult decomposition is selected to calculate the score of the predicted relationship entity triple to determine whether it can be regarded as a correct triple.

[0059] The score function based on DistMult decomposition is expressed as follows:

[0060]

[0061] Among them, h h The feature representation vector representing the head entity h in the triple. t Represents the feature representation vector of the tail entity t in the triple. R r The diagonal matrix corresponding to the relation r is set as a learnable parameter matrix, namely the relation transformation matrix. By using DistMult decomposition, the head and tail entity feature vectors and the relation r are projected to a specific value as the correct probability score of the link prediction triple.

[0062] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A link prediction method based on enhanced entity structure feature representation learning, characterized in that: The following steps are involved: Step S1: Input the initialization vector generated by the knowledge graph entity data into the RGCN model, extract and learn the entity relationship features, and obtain the entity relationship feature representation vector; Step S2, supplementing entity relationship features based on entity adjacency structure features; The initialization vector generated by the knowledge graph entity data is input into the GCN model and the GAT model respectively; in each layer of the entity representation learning network, the GCN model extracts and learns the neighbor structure features of the central entity node, and the GAT model assigns weights according to the importance of the entity node to reflect the importance of the adjacent nodes, extracts features from the entity adjacent nodes, and uses the importance of the adjacent nodes to reflect the importance of the corresponding relationship; the importance features obtained by the GAT model are used as a supplement to the neighbor structure features obtained by the GCN model, and the two features are aggregated based on the average gating mechanism to obtain the entity's relationship structure feature representation vector; Step S3, the entity relationship feature representation vector obtained in step S1 and the relationship structure feature representation vector in step S2 are aggregated again based on Res gate gating to obtain the entity representation vector; in the learning and training process, the sampling of negative samples is strengthened as a supplement to the lack of positive samples to improve the effect of learning and training the entity representation vector; Step S4: performing link prediction based on the entity representation vector; Select the scoring function based on DistMult decomposition to calculate the score of the predicted relation entity triples to determine whether they can be regarded as correct triples; The specific method for extracting entity relationship features in step S1 includes: First, the adjacent nodes of the central entity node are classified according to the relationship between the central node and the corresponding adjacent nodes; then a learnable relationship transformation matrix is ​​set for the relationship between the central node and the corresponding adjacent nodes; finally, for the central node, a reflexive relationship is added to maintain the node's own characteristics during the forward propagation learning process, where the forward propagation formula for entity representation learning is as follows: in, The entity representation vector of the next layer node i in the network; σ represents the activation function; R represents the set of all relationships in the graph; N r represents the set of neighbor nodes corresponding to node i under relationship r; c i,r represents the regularization constant; Represents the relational feature conversion parameters at this layer; represents the feature representation of the adjacent node j; W l represents the learnable parameter matrix; It represents the feature representation of node i in the previous layer; The formula for fusion feature information of the average gating mechanism used in step S2 is as follows: h i =(1-a)h i1 +ah i2 where h i is the relational structural feature information of entity i; h i1 、h i2 are the relational structure feature vectors of entity i obtained by the GCN and GAT models respectively; α is the fusion coefficient, ranging from (0,1], which is used for the aggregation of relational distribution features and relational importance features; The formula for fusing feature information using the Res gate gating mechanism in step S3 is as follows: h i =W2(σ(W1h i1 +b))+h i2 where h i is the final feature representation vector of entity i; h i1 、h i2 are the relational structure feature vector and relational feature vector model of entity i respectively; W1 and W2 are learnable parameter matrices; σ is the activation function, and the relu() activation function is selected.

2. A link prediction method based on enhanced entity structure feature representation learning according to claim 1, characterized in that: The score function based on DistMult decomposition is expressed as follows: Among them, h h The feature representation vector representing the head entity h in the triple; h t Represents the feature representation vector of the tail entity t in the triple; R r The diagonal matrix corresponding to the relation r is the relation transformation matrix. By using DistMult decomposition, the head and tail entity feature vectors and the relation r are projected to a specific value as the correct probability score of the link prediction triple.

Images