Knowledge graph link prediction method based on relation path modeling

Abstract

The invention discloses a knowledge graph link prediction method based on relation path modeling. The method comprises the following steps: firstly, generating embedded representation of an entity by utilizing structure, attribute and type information of the entity in a knowledge graph; and then vectorizing the paths to obtain a path vector set, and integrating modes of different paths by using an attention mechanism so as to accurately predict whether a potential candidate relationship exists between entities. According to the knowledge graph link prediction method based on relation path modeling, the accuracy and robustness of knowledge graph completion are improved, and the prediction accuracy and interpretability are improved.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graphs and relates to a knowledge graph link prediction method based on relationship path modeling. Background Art

[0002] The knowledge graph is based on the relationship between entities and entities, and describes rich real-world knowledge. However, due to the continuous updating of knowledge, there are many unknown entity relationships in the knowledge graph, which limits the integrity and application effect of the knowledge graph. Therefore, knowledge graph completion has become an important research direction, aiming to expand and improve the knowledge graph by analyzing existing entity relationships and predicting unknown entity relationships.

[0003] The embedding-based completion method maps entities and relations to vector space and predicts the relationship between entities by the similarity between vectors. This method makes entities and relations with similar semantics closer in the embedding space. By measuring the vector similarity between entities, unknown entity relations can be predicted. Embedding-based methods have good scalability and interpretability, and have made significant progress in the field of knowledge graph completion. Among them, TransE is one of the earliest proposed embedding-based methods, which measures the relationship between entities by simple Euclidean distance. However, TransE has a limitation that it cannot handle relations with ambiguity. In order to overcome the limitations of TransE, an improved embedding-based method, TransH, is proposed. TransH introduces a relation-specific hyperplane to represent the relation vector, thereby capturing the semantic information of the relation. Each relation has a corresponding hyperplane, and the entity vector is projected on this hyperplane to obtain the semantic representation of the relation. This method can better handle relations with ambiguity and improve the accuracy and robustness of knowledge graph completion. However, despite its advantages in handling ambiguous relations, TransH also has some challenges and limitations. First, it requires the definition of a hyperplane, which increases the complexity and computational cost of the model. Second, the TransH method may face the problem of information loss when dealing with long-distance relations because the projection of the entity vector only considers the local information of the relation. In addition, the TransH method may have a poor modeling effect on sparse relations because the representation of the relation vector is limited by the entity vector.

[0004] At the same time, the core of path-based completion methods is to deeply analyze various relationship paths between entities and predict unknown entity relationships. These paths describe the complex associations between entities and can help understand the semantic relationships between entities. Path-based methods extract important relationship paths and predict unknown entity relationships through path extraction and path scoring. This method performs well on small-scale knowledge graphs, but may face efficiency and scalability challenges on large-scale knowledge graphs. PRA, as a classic method, has attracted widespread attention. The PRA method predicts unknown entity relationships through path reasoning and path scoring. First, it extracts relationship paths between entities from an existing knowledge graph, and then scores these paths to determine the importance of the paths. By selecting the path with the highest path score or setting a threshold to screen high-quality prediction results, the PRA method can provide accurate entity relationship predictions. However, although the PRA method has achieved good performance on small-scale knowledge graphs, it may face efficiency and scalability challenges when processing large-scale knowledge graphs. In addition, the path scoring method of the PRA method may be limited by the path length and the relationship weight in the path, resulting in the accuracy of the prediction results being affected. Summary of the invention

[0005] In order to overcome the shortcomings of the existing technology, the present invention proposes a knowledge graph link prediction method based on relational path modeling to improve the accuracy and robustness of knowledge graph completion, and enhance the accuracy and interpretability of prediction. First, the structure, attributes, and type information of entities in the knowledge graph are used to generate an embedded representation of the entity. Then the path is vectorized to obtain a path vector set, and the attention mechanism is used to synthesize the patterns of different paths, thereby accurately predicting whether there are potential candidate relationships between entities.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A knowledge graph link prediction method based on relationship path modeling, the method comprising the following steps:

[0008] Step 1: Representation of knowledge graph structural information: Knowledge graph is a structured knowledge representation form, which consists of a series of head entities h, relations r and tail entities t, forming a network of knowledge system, in which entities in the knowledge graph are connected through relations. Based on a selected entity as the center of the circle, other entities are regarded as points on the circle, thus forming a hierarchical structure on the knowledge graph. Then, two types of entities are divided in the knowledge graph, and the structural information representation of the entities is obtained by embedding the two types of entities.

[0009] Step 2: Knowledge graph attribute information representation: In the knowledge graph, each entity has some attribute information. These attribute information are directly connected to the first-order entities of the entity itself. The attribute information of the entity is encoded into a vector representation;

[0010] Step 3: Knowledge graph type information representation: The type of an entity is considered to be a series of words, and each entity has a different abstract type;

[0011] Step 4: Unified embedding representation: Given a knowledge graph, learn a unified embedding representation of entities and relations;

[0012] Step 5: Path embedding representation;

[0013] Step 6. The loss function is defined as follows:

[0014]

[0015] Where T represents the set of positive triples, F represents the set of negative triples, and P represents the probability that the head entity and the tail entity can be connected by the relation r. The training goal of this loss function is to optimize the model so that it produces higher scores on correct triples and lower scores on incorrect or missing triples, thereby minimizing the overall error.

[0016] Further, the process of the first step is as follows:

[0017] Step (1.1), the first type of entities are entities at different levels in the hierarchy, that is, entities whose distances from the point to the center of the circle are not equal; the second type of entities are entities at the same level in the hierarchy, that is, entities whose distances from the point to the center of the circle are equal;

[0018] Step (1.2), use the modulus part to represent the entity embeddings on different hierarchical structures, and use the phase part to represent the entity embeddings on the same hierarchical structure;

[0019] Step (1.3) embeds entities at different hierarchical levels. Each entity has certain hierarchical properties. The modulus information is used to represent entities at different levels, because the modulus can reflect the depth of the hierarchy. The modulus of each head entity h is represented as h m The modulus of the tail entity t is t m , and denote the modulus of each relation r as r m , the modulus part is expressed as follows:

[0020]

[0021] in Represents the Hadamard product operation;

[0022] Step (1.4), the distance formula is:

[0023]

[0024] Where ||||2 represents the L2 norm, D r,m (h m ,t m ) means h m and t m The distance between

[0025] In step (1.5), entities at higher levels are set to have smaller moduli because they are closer to the representation of the candidate entities. The entity embeddings on the same hierarchy are represented using the phase part. The phase of each head entity h is represented as h p , the phase of the tail entity t is denoted by t p , and denote the phase of each relation r as r p , the phase part is expressed as follows:

[0026] (h p +r p )mod2π=t p

[0027] Where mod represents the remainder operator;

[0028] Step (1.6), the distance formula is:

[0029] D r,p (h p ,t p )=||sin(h p +r p -t p ) / 2||1

[0030] Where sin() is an operation that applies a sine function to each element of the input, ||||1 represents the L1 norm, and a sine function is used to measure the distance between phases because the phase has a periodic characteristic, D r,p (h p ,t p ) means h p and t p The distance between

[0031] Step (1.7), the modulus part and the phase part are combined, and the entity is mapped in the polar coordinate system, where the radial coordinate represents the modulus part and the angular coordinate corresponds to the phase part. An entity h is represented as [h m ||h p ], where h m and h pThe relationship r is expressed as [r m ||r p ],|| represents the connection of two vectors;

[0032] Step (1.8) obtains the hierarchical information embedding vector e of entities and relations hs and e rs , which is expressed as follows:

[0033] e hs =[h m ||h p ]

[0034] e rs =[r m ||r p ].

[0035] Going further, the process of the second step is as follows:

[0036] Step (2.1), the representation of attribute vectors, uses GloVE to pre-train attribute word embeddings. The input of GloVE is the attribute set D l =[d1,d2...,d m ],l∈1,2...,m, the output is a vector set of attributes d m represents the mth attribute of the entity, e m Represents the embedding of the mth attribute of an entity;

[0037] Step (2.2), each attribute contributes differently to the characteristics of the entity, so the attribute information is embedded by using the attribute-level attention mechanism, for entity h and its attribute d l , the attention mechanism first learns the attention weights α between them l ;

[0038] Step (2.3), based on the attention weight set [α1, α2..., α m ] to attribute vector set Weighted aggregation to obtain the embedded representation of attribute information Defined as:

[0039]

[0040] Among them, exp() represents the exponential function, LeakyReLU is the activation function, which assigns a non-zero slope to all negative values, and the parameters A1 and B1 are the weight vector and weight matrix respectively. h and e l Represents entity h and its attribute d respectively l The embedding Indicates that except e l Outside Other vectors in are embedded, T represents matrix transpose;

[0041] Step (2.4) obtains the attribute information vector representation of entity h

[0042]

[0043] Furthermore, the process of the third step is as follows:

[0044] Step (3.1), given an entity h, merge its type set T t =[t1,t2...,t n ],t∈1,2...,n,t n Represents the nth type of entity h. GloVE is used to pre-train the vector embedding of abstract types to obtain an entity type vector sequence E T =[e ht1 ,e ht2 ...,e htn ],e htn The embedding representation of the nth type of entity h;

[0045] Step (3.2), input type vector sequence E T =[e ht1 ,e ht2 ...,e htn ] and its reverse vector sequence E T_re =[e htn ,e ht(n-1) ...,e ht1 ], enter the forward layer and backward layer of the Bi-LSTM model respectively. The core idea of ​​Bi-LSTM is to run two LSTM networks at the same time at each time step, one to process the input sequence from front to back, and the other to process the input sequence from back to front; then, combine the hidden state h of the forward layer ft and the hidden state h of the backward layer bt , get Type vector representation of step size The calculation is as follows:

[0046]

[0047] Get the hidden state set of the Bi-LSTM model output at all step sizes

[0048] Step (3.3) uses the self-attention mechanism to summarize the attention weights, giving due attention to each vector in the set. The input of the attention mechanism is the hidden LSTM state. The output is the weights of the vector Given by the equation:

[0049]

[0050] in, is a nonlinear mapping function, and is the projection parameter, T represents the matrix transpose;

[0051] Step (3.4), the embedding of entity type information is calculated by the formula:

[0052]

[0053] Step (3.5), in the same way as above, obtain the vector representation of the relationship type

[0054] Step (3.6), given a triple, design a constraint on the entity type based on the type of relationship:

[0055]

[0056] in and They are the vector representation of entity type and the vector representation of relationship type, W e and W r are the weight parameter matrices of type, entity and relationship respectively, Represents the addition operation;

[0057] Step (3.7), the type information of the entity should be assigned different weights in different scenarios to improve the relationship prediction between entities in a given path. Therefore, a type-level attention mechanism is adopted to merge the relationship type information into the entity type information representation. The type attention weight Is made by:

[0058]

[0059] Where exp represents the exponential function, c j In addition to Other entity types besides

[0060] Step (3.8), Multiply each element of and its corresponding attention weight to get the entity type information embedding

[0061]

[0062] The process of the fourth step is as follows:

[0063] Step (4.1), entity representation:

[0064] The entity hierarchy information hs , attribute information and type information Mapped to the same representation space through three projection matrices:

[0065] es=M s e hs

[0066]

[0067]

[0068] Among them, M s , M d and M t They are the projection matrices of structure information, attribute information, and type information respectively;

[0069] Using the attention mechanism, we adjust the attention weights of the hierarchical structure embedding, attribute information embedding, and type information embedding, and then merge these embeddings to learn the representation of the entity v e :

[0070]

[0071] Where W f , w f is the weight parameter matrix, represents the merged entity embedding information, β e Represents the normalized entity embedding information;

[0072] Step (4.2), Representation of Relations

[0073] Hierarchical information of relationships rs and relationship type information The relationship v is expressed in the same way as in step (4.1) r :

[0074] rs=M s e rs

[0075]

[0076] Where W r , w ris the weight parameter matrix, represents the merged relation embedding information, β r Represents the normalized relation embedding information.

[0077] The process of the fifth step is as follows:

[0078] Step (5.1), path set representation P = [p1, p2..., p n ], where p n represents the nth path representation between a given entity pair, p n =[v e1 ,v r1 ,v e2 ,...v rk ,v ek ], where v rk represents the kth relationship in this path, v ek Represents the kth entity representation in this path; then, GRU is used to encode the path into a vector representation. At each time step t, GRU first calculates the update gate and reset gate, then uses the output of these gated units to calculate the candidate state, and finally updates the hidden state h at the current moment according to the update gate and candidate state t , forming a path vector representation set in The vector representation of the nth path;

[0079] Step (5.2), use the attention mechanism to assign different weights to different paths, and the final matrix of the path It is expressed as:

[0080]

[0081] Among them, M p is the mapping matrix of the path matrix, γ x represents the attention weight of the x-th path, W p and w γ are projection parameters;

[0082] Step (5.3), the relationship prediction y is given by:

[0083]

[0084] Among them, M r is the representation matrix of the relationship type, and b is the bias vector.

[0085] The beneficial effects of the present invention are mainly manifested in: using the combination of structure, attribute, and type information to embed rich entities and relationships to improve the performance of knowledge graph link prediction; combining embedding and path methods to capture complex relationships between entities and predict unknown entity relationships; using the semantic information of relationship paths to improve the accuracy and robustness of knowledge graph completion. The present invention uses the structure, attribute, and type information of entities to generate entity embedding vectors, so that the model has strong generalization capabilities, while retaining the neighborhood information of entities, so that the model can capture the semantic structure of entities and relationships, thereby improving the accuracy and interpretability of predictions. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 It is a schematic diagram of structural information embedding of the present invention.

[0087] Figure 2 It is a schematic diagram of embedding attribute information of the present invention.

[0088] Figure 3 It is a schematic diagram of embedding type information of the present invention.

[0089] Figure 4 It is a unified embedding schematic diagram of the present invention. DETAILED DESCRIPTION

[0090] The present invention will be further described below in conjunction with the accompanying drawings.

[0091] Reference Figure 1 to Figure 4 , a knowledge graph link prediction method based on relationship path modeling, comprising the following steps:

[0092] The first step is to represent the knowledge graph structure information. The process is as follows:

[0093] Knowledge graph is a structured knowledge representation form, which consists of a series of head entities h, relations r and tail entities t, forming a network of knowledge system, in which entities in the knowledge graph are connected through relations, based on a selected entity as the center of the circle, other entities are regarded as a point on the circle, thus forming a hierarchical structure on the knowledge graph, and then two types of entities are divided in the knowledge graph, and the structural information representation of the entities is obtained by embedding the two types of entities;

[0094] Step (1.1), the first type of entities are entities at different levels in the hierarchy, that is, entities whose distances from the point to the center of the circle are not equal, and the second type of entities are entities at the same level in the hierarchy, that is, entities whose distances from the point to the center of the circle are equal;

[0095] Step (1.2), a point in the circle is divided into two parts: modulus and phase to represent a point. The modulus is the distance from the point to the center of the circle, and the phase is the angle between the line connecting the point to the center of the circle and the horizontal line. Therefore, the modulus part is used to represent the embedding of entities on different hierarchies, and the phase part is used to represent the embedding of entities on the same hierarchy.

[0096] Step (1.3) embeds entities at different hierarchical levels. Each entity has certain hierarchical properties. Use modulus information to represent entities at different levels because the modulus can reflect the depth of the hierarchy. The modulus of each head entity h is represented as h m The modulus of the tail entity t is t m , and denote the modulus of each relation r as r m , the modulus part is expressed as follows:

[0097]

[0098] in Represents the Hadamard product operation;

[0099] Step (1.4), the distance formula is:

[0100]

[0101] Where ||||2 represents the L2 norm, D r,m (h m ,t m ) means h m and t m The distance between

[0102] In step (1.5), entities at higher levels are set to have smaller moduli because they are closer to the representation of the candidate entities. The entity embeddings on the same hierarchy are represented using the phase part. The phase of each head entity h is represented as h p , the phase of the tail entity t is denoted by t p , and denote the phase of each relation r as r p , the phase part is expressed as follows:

[0103] (h p +r p )mod2π=t p

[0104] Where mod represents the remainder operator;

[0105] Step (1.6), the distance formula is:

[0106] D r,p (h p ,t p)=||sin(h p +r p -t p ) / 2||1

[0107] Where sin() is an operation that applies a sine function to each element of the input, ||||1 represents the L1 norm, and a sine function is used to measure the distance between phases because the phase has a periodic characteristic, D r,p (h p ,t p ) means h p and t p The distance between

[0108] Step (1.7), the modulus part and the phase part are combined to map the entity in the polar coordinate system (a two-dimensional coordinate system that defines the position on a plane by the distance between a point and a fixed point and the angle between the point and a fixed ray), where the radial coordinate represents the modulus part and the angular coordinate corresponds to the phase part. An entity h is represented as [h m ||h p ], where h m and h p The relationship r is expressed as [r m ||r p ],|| represents the connection of two vectors;

[0109] Step (1.8), such as Figure 1 As shown, the hierarchical information embedding vector e of entities and relations is obtained hs and e rs , which is expressed as follows:

[0110] e hs =[h m ||h p ]

[0111] e rs =[r m ||r p ]

[0112] The second step is to represent the attribute information of the knowledge graph. The process is as follows:

[0113] like Figure 2 As shown in the figure, in the knowledge graph, each entity has some attribute information, which are often first-order entities directly connected to the entity itself. The attribute information of these entities can also reflect some characteristics of the entity to a certain extent, so the attribute information of the entity is encoded as a vector representation;

[0114] Step (2.1), the representation of attribute vectors, uses GloVE to pre-train attribute word embedding. GloVE is an unsupervised learning algorithm for distributed word representation. The input of GloVE is the attribute set D l =[d1,d2...,d m ],l∈1,2...,m, the output is a vector set of attributes d m Represents the mth attribute of an entity. m Represents the embedding of the mth attribute of an entity;

[0115] Step (2.2), each attribute contributes differently to the characteristics of the entity, so the attribute information is embedded by using the attribute-level attention mechanism, for entity h and its attribute d l , the attention mechanism first learns the attention weights α between them l ;

[0116] Step (2.3), based on the attention weight set [α1, α2..., α m ] to attribute vector set Weighted aggregation to obtain the embedded representation of attribute information Defined as:

[0117]

[0118] Among them, exp() represents the exponential function, LeakyReLU is the activation function, which assigns a non-zero slope to all negative values, and the parameters A1 and B1 are the weight vector and weight matrix respectively. h and e l Represents entity h and its attribute d respectively l The embedding Indicates that except e l Outside Other vectors in are embedded, T represents matrix transpose;

[0119] Step (2.4) obtains the attribute information vector representation of entity h

[0120]

[0121] The third step is to represent the knowledge graph type information. The process is as follows:

[0122] like Figure 3 As shown, the type of an entity is considered as a sequence of words, each of which has a different abstract type.

[0123] Step (3.1), given an entity h, merge its type set Tt =[t1,t2...,t n ],t∈1,2...,n,t n Represents the nth type of entity h. In the same way, GloVE is used to pre-train the vector embedding of abstract types to obtain an entity type vector sequence E T =[e ht1 ,e ht2 ...,e htn ],e htn The embedding representation of the nth type of entity h;

[0124] Step (3.2), input type vector sequence E T =[e ht1 ,e ht2 ...,e htn ] and its reverse vector sequence E T_re =[e htn ,e ht(n-1) ...,e ht1 ], enter the forward layer and backward layer of the Bi-LSTM model respectively. The core idea of ​​Bi-LSTM is to run two LSTM networks (a special recurrent neural network that can learn and memorize long-term dependency information. They are particularly effective in processing and predicting time series data) at each time step, one processing the input sequence from front to back, and the other processing the input sequence from back to front; then, combine the hidden state h of the forward layer ft and the hidden state h of the backward layer bt , can be obtained in Type vector representation of step size The calculation is as follows:

[0125]

[0126] Get the hidden state set of the Bi-LSTM model output at all step sizes

[0127] Step (3.3), use the self-attention mechanism (through different weight matrices to convert each element of the input sequence into a corresponding vector, then calculate the dot product of the query vector and the key vector to get the attention score, and then use the softmax function to normalize it to get the attention weight of each element to other elements) to summarize the attention weights, give due attention to each vector in the set, and the input of the attention mechanism is the hidden LSTM state The output is the weights of the vector Given by the equation:

[0128]

[0129] in, is a nonlinear mapping function, and is the projection parameter, T represents the matrix transpose;

[0130] Step (3.4), the embedding of entity type information is calculated by the formula:

[0131]

[0132] Step (3.5), in the same way as above, obtain the vector representation of the relationship type

[0133] Step (3.6), given a triple, design a constraint on the entity type based on the type of relationship:

[0134]

[0135] in and They are the vector representation of entity type and the vector representation of relationship type, W e and W r are the weight parameter matrices of type, entity and relationship respectively, Represents the addition operation;

[0136] Step (3.7), the type information of the entity should be assigned different weights in different scenarios to improve the relationship prediction between entities in a given path. Therefore, a type-level attention mechanism is adopted to merge the relationship type information into the entity type information representation. The type attention weight Is made by:

[0137]

[0138] Where exp represents the exponential function. j In addition to Other entity types besides

[0139] Step (3.8), Multiply each element of and its corresponding attention weight to get the entity type information embedding

[0140]

[0141] Step 4: Unified embedding representation: Given a knowledge graph, learn the unified embedding representation of entities and relations. The process is as follows:

[0142] Step (4.1), entity representation:

[0143] like Figure 4 As shown, the entity hierarchy information e hs , attribute information and type information Mapped to the same representation space through three projection matrices:

[0144] es=M s e hs

[0145]

[0146]

[0147] Among them, M s , M d and M t They are the projection matrices of structure information, attribute information, and type information respectively;

[0148] Using the attention mechanism, we adjust the attention weights of the hierarchical structure embedding, attribute information embedding, and type information embedding, and then merge these embeddings to learn the representation of the entity v e :

[0149]

[0150] Where W f , w f is the weight parameter matrix, represents the merged entity embedding information, β e Represents the normalized entity embedding information;

[0151] Step (4.2), Representation of Relations

[0152] Hierarchical information of relationships rs and relationship type information The relationship v is expressed in the same way as in step (4.1) r :

[0153] rs=M s e rs

[0154]

[0155]

[0156] Where W r , w r is the weight parameter matrix, represents the merged relation embedding information, β r Represents the normalized relation embedding information;

[0157] Step 5: Path embedding representation. The process is as follows:

[0158] Step (5.1), path set representation P = [p1, p2..., p n ], where p n represents the nth path representation between a given entity pair, p n =[v e1 ,v r1 ,v e2 ,...v rk ,v ek ], where v rk represents the kth relationship in this path, v ek Represents the kth entity representation in this path; then, GRU is used to encode the path into a vector representation. GRU is a recurrent neural network used to process sequence data. It controls the flow of information and the update of the state through a series of gating units. At each time step t, GRU first calculates the update gate and reset gate, and then uses the output of these gating units to calculate the candidate state. Finally, the hidden state h at the current moment is updated according to the update gate and the candidate state. t , forming a path vector representation set in The vector representation of the nth path;

[0159] Step (5.2), use the attention mechanism to assign different weights to different paths, and the final matrix of the path It is expressed as:

[0160]

[0161] Among them, M p is the mapping matrix of the path matrix, γ x represents the attention weight of the x-th path, W p and w γ are projection parameters;

[0162] Step (5.3), the relationship prediction y is given by:

[0163]

[0164] Among them, M r is the representation matrix of the relationship type, and b is the bias vector;

[0165] Step 6. The loss function is defined as follows:

[0166]

[0167] Where T represents the set of positive triples, F represents the set of negative triples, and P represents the probability that the head entity and the tail entity can be connected by the relation r. The training goal of this loss function is to optimize the model so that it produces higher scores on correct triples and lower scores on incorrect or missing triples, thereby minimizing the overall error.

[0168] This embodiment uses three commonly used knowledge graph datasets for experiments, namely NELL995, FB15K-237 and WN18RR. NELL995 is a knowledge graph dataset created by a research team at Carnegie Mellon University. It is built based on structured knowledge automatically extracted from the Internet. The present invention uses the NELL dataset generated by Xiong et al. (2017) in the 995th iteration. The dataset covers knowledge in multiple fields, such as people, places, organizations, events, etc., and contains numerous entities and relationships. The FB15K-237 dataset is a subset extracted from the Freebase knowledge graph. Freebase is a huge knowledge graph that includes a large number of entities and relationships. FB15K-237 reduces the scale by removing some high-frequency relationships. The dataset includes 14,541 entities and 237 relationships. The relationship types include "father-child relationship", "brother relationship", "nationality", etc. WN18RR is a subset of WN18, consisting of 18 relationships and 40,943 entities. The statistics of the three datasets are shown in Table 1.

[0169]

[0170] Table 1

[0171] This invention introduces four commonly used indicators for evaluating the effectiveness of the method. The following are the specific calculation formulas for these four indicators:

[0172]

[0173] MAP is an average measure of the precision scores of all correct triples in their respective ranking positions, and |ET| represents the number of elements contained in the set ET. AP specifically refers to the average precision score of the position occupied by entity e in the correct triple arrangement.

[0174]

[0175] MR refers to the average ranking position of the first correct triple in the test sample. MRR evaluates performance by calculating the average of the reciprocal ranking positions of the first correct triple in the test sample.

[0176]

[0177] The present invention uses n=1, 3, and 10 as the test standard to check the proportion of correct triples ranked in the top n in all test samples. The higher the value of Hits@n, the better the link prediction ability of the method.

[0178] This paper selects 8 commonly used link prediction methods, including the classic link prediction methods TransE, PRA, RWR, DistMult, ComplEx, RotatE, PTransE and TAPR. The experimental results show that the present invention is superior to the benchmark method in all performance indicators, as shown in Table 2 and Table 3. Table 2 shows the link prediction results on FB15K-237 and WN18RR;

[0179]

[0180]

[0181] Table 2

[0182] From the results in Table 2, it can be seen that the present invention outperforms other baseline models in all indicators, which shows the effectiveness of the present invention. The present invention not only utilizes the structural information in the path, but also introduces the semantic knowledge of the path information to enhance the expressive power of the model. Moreover, all triple information in the path is used instead of just a single triple, which is very helpful for improving the performance of link prediction.

[0183] Table 3 shows the link prediction results on FB15K-237;

[0184]

[0185] Table 3

[0186] As can be seen from Table 3, the present invention conducted experiments on the FB15K-237 dataset. The results show that PRA is significantly better than TransE in experimental indicators. This is because PRA can handle multiple types of relationships, while TransE can only handle one-to-one relationships. At the same time, PRA can utilize remote associations between entities, while TransE may ignore this information. RWR is slightly better than PRA because RWR considers the propagation effect between entities, while PRA does not. In RWR, each entity will assign a part of the probability to its adjacent entities, and these probabilities will decrease with the depth of the walk. The present invention is significantly better than RWR, PRA and PTransE in all indicators, which is attributed to the denoising of path information in the method of the present invention, and the influence of those paths with weaker associations is screened or reduced through the attention mechanism. Compared with TAPR, the present invention also shows advantages. TAPR also uses structure, attribute and type information, but the present invention embeds entities at different levels by distinguishing two types of entities in the structural information, giving full play to the role of structural information, indicating that distinguishing different levels of structures has a certain improvement on the model performance.

[0187] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A knowledge graph link prediction method based on relationship path modeling, characterized in that: The method comprises the following steps: Step 1: Representation of knowledge graph structural information: Knowledge graph is a structured knowledge representation form, which consists of a series of head entities h, relations r and tail entities t, forming a network of knowledge system, in which entities in the knowledge graph are connected through relations. Based on a selected entity as the center of the circle, other entities are regarded as points on the circle, thus forming a hierarchical structure on the knowledge graph. Then, two types of entities are divided in the knowledge graph, and the structural information representation of the entities is obtained by embedding the two types of entities. Step 2: Knowledge graph attribute information representation: In the knowledge graph, each entity has some attribute information. These attribute information are directly connected to the first-order entities of the entity itself. The attribute information of the entity is encoded into a vector representation; Step 3: Knowledge graph type information representation: The type of an entity is considered to be a series of words, and each entity has a different abstract type; Step 4: Unified embedding representation: Given a knowledge graph, learn a unified embedding representation of entities and relations; Step 5: Path embedding representation; Step 6. The loss function is defined as follows: Where T represents the set of positive triples, F represents the set of negative triples, and P represents the probability that the head entity and the tail entity can be connected by the relation r. The training goal of this loss function is to optimize the model so that it produces higher scores on correct triples and lower scores on incorrect or missing triples, thereby minimizing the overall error.

2. The knowledge graph link prediction method based on relationship path modeling according to claim 1, characterized in that: The process of the first step is as follows: Step (1.1), the first type of entities are entities at different levels in the hierarchy, that is, entities whose distances from the point to the center of the circle are not equal; the second type of entities are entities at the same level in the hierarchy, that is, entities whose distances from the point to the center of the circle are equal; Step (1.2), use the modulus part to represent the entity embeddings on different hierarchical structures, and use the phase part to represent the entity embeddings on the same hierarchical structure; Step (1.3) embeds entities at different hierarchical levels. Each entity has certain hierarchical properties. The modulus information is used to represent entities at different levels, because the modulus can reflect the depth of the hierarchy. The modulus of each head entity h is represented as h m The modulus of the tail entity t is t m , and denote the modulus of each relation r as r m , the modulus part is expressed as follows: in Represents the Hadamard product operation; Step (1.4), the distance formula is: Where || ||2 represents the L2 norm, D r,m (h m ,t m ) means h m and t m The distance between In step (1.5), entities at higher levels are set to have smaller moduli because they are closer to the representation of the candidate entities. The entity embeddings on the same hierarchy are represented using the phase part. The phase of each head entity h is represented as h p , the phase of the tail entity t is denoted by t p , and denote the phase of each relation r as r p , the phase part is expressed as follows: (h p +r p )mod2π=t p Where mod represents the remainder operator; Step (1.6), the distance formula is: D r,p (h p ,t p )=||sin(h p +r p -t p ) / 2||1 Where sin() is an operation that applies a sine function to each element of the input, || ||1 represents the L1 norm, and a sine function is used to measure the distance between phases because the phase has a periodic characteristic, D r,p (h p ,t p ) means h p and t p The distance between Step (1.7), the modulus part and the phase part are combined, and the entity is mapped in the polar coordinate system, where the radial coordinate represents the modulus part and the angular coordinate corresponds to the phase part. An entity h is represented as [h m ||h p ], where h m and h p The relationship r is expressed as [r m ||r p ],|| represents the connection of two vectors; Step (1.8) obtains the hierarchical information embedding vector e of entities and relations hs and e rs , which is expressed as follows: e hs =[h m ||h p ] e rs ＝[r m ||r p ]。 3. The knowledge graph link prediction method based on relationship path modeling according to claim 1 or 2, characterized in that: The process of the second step is as follows: Step (2.1), the representation of attribute vectors, uses GloVE to pre-train attribute word embeddings. The input of GloVE is the attribute set D l =[d1,d2...,d m ],l∈1,2...,m, the output is a vector set of attributes d m represents the mth attribute of the entity, e m Represents the embedding of the mth attribute of an entity; Step (2.2), each attribute contributes differently to the characteristics of the entity, so the attribute information is embedded by using the attribute-level attention mechanism, for entity h and its attribute d l , the attention mechanism first learns the attention weights α between them l ; Step (2.3), based on the attention weight set [α1, α2..., α m ] to attribute vector set Weighted aggregation to obtain the embedded representation of attribute information Defined as: Among them, exp() represents the exponential function, LeakyReLU is the activation function, which assigns a non-zero slope to all negative values, and the parameters A1 and B1 are the weight vector and weight matrix respectively. h and e l Represents entity h and its attribute d respectively l The embedding Indicates that except e l Outside Other vectors in are embedded, T represents matrix transpose; Step (2.4) obtains the attribute information vector representation of entity h 4. The knowledge graph link prediction method based on relationship path modeling according to claim 1 or 2, characterized in that: The process of the third step is as follows: Step (3.1), given an entity h, merge its type set T t =[t1,t2...,t n ],t∈1,2...,n,t n Represents the nth type of entity h. GloVE is used to pre-train the vector embedding of abstract types to obtain an entity type vector sequence E T =[e ht1 ,e ht2 ...,e htn ],e htn The embedding representation of the nth type of entity h; Step (3.2), input type vector sequence E T =[e ht1 ,e ht2 ...,e htn ] and its reverse vector sequence E T_re =[e htn ,e ht(n-1) ...,e ht1 ], enter the forward layer and backward layer of the Bi-LSTM model respectively. The core idea of ​​Bi-LSTM is to run two LSTM networks at the same time at each time step, one to process the input sequence from front to back, and the other to process the input sequence from back to front; then, combine the hidden state h of the forward layer ft and the hidden state h of the backward layer bt , get Type vector representation of step size The calculation is as follows: Get the hidden state set of the Bi-LSTM model output at all step sizes Step (3.3) uses the self-attention mechanism to summarize the attention weights, giving due attention to each vector in the set. The input of the attention mechanism is the hidden LSTM state. The output is the weights of the vector Given by the equation: in, is a nonlinear mapping function, and is the projection parameter, T represents the matrix transpose; Step (3.4), the embedding of entity type information is calculated by the formula: Step (3.5), in the same way as above, obtain the vector representation of the relationship type Step (3.6), given a triple, design a constraint on the entity type based on the type of relationship: in and They are the vector representation of entity type and the vector representation of relationship type, W e and W r are the weight parameter matrices of type, entity and relationship respectively, Represents the addition operation; Step (3.7), the type information of the entity should be assigned different weights in different scenarios to improve the relationship prediction between entities in a given path. Therefore, a type-level attention mechanism is adopted to merge the relationship type information into the entity type information representation. The type attention weight Is made by: Where exp represents the exponential function, c j In addition to Other entity types besides Step (3.8), Multiply each element of and its corresponding attention weight to get the entity type information embedding 5. The knowledge graph link prediction method based on relationship path modeling according to claim 1 or 2, characterized in that: The process of the fourth step is as follows: Step (4.1), entity representation: The entity hierarchy information hs , attribute information and type information Mapped to the same representation space through three projection matrices: is=M s and hs Among them, M s , M d and M t They are the projection matrices of structure information, attribute information, and type information respectively; Using the attention mechanism, we adjust the attention weights of the hierarchical structure embedding, attribute information embedding, and type information embedding, and then merge these embeddings to learn the representation of the entity v e : Where W f , w f is the weight parameter matrix, represents the merged entity embedding information, β e Represents the normalized entity embedding information; Step (4.2), Representation of Relations Hierarchical information of relationships rs and relationship type information The relationship v is expressed in the same way as in step (4.1) r : rs=M s e rs Where W r , w r is the weight parameter matrix, represents the merged relation embedding information, β r Represents the normalized relation embedding information.

6. The knowledge graph link prediction method based on relationship path modeling according to claim 1 or 2, characterized in that: The process of the fifth step is as follows: Step (5.1), path set representation P = [p1, p2..., p n ], where p n represents the nth path representation between a given entity pair, p n =[v e1 ,v r1 ,v e2 ,...v rk ,v ek ], where v rk represents the kth relationship in this path, v ek Represents the kth entity representation in this path; then, GRU is used to encode the path into a vector representation. At each time step t, GRU first calculates the update gate and reset gate, then uses the output of these gated units to calculate the candidate state, and finally updates the hidden state h at the current moment according to the update gate and candidate state t , forming a path vector representation set in The vector representation of the nth path; Step (5.2), use the attention mechanism to assign different weights to different paths, and the final matrix of the path It is expressed as: Among them, M p is the mapping matrix of the path matrix, γ x represents the attention weight of the x-th path, W p and w γ are projection parameters; Step (5.3), the relationship prediction y is given by: Among them, M r is the representation matrix of the relationship type, and b is the bias vector.

Images