Subgraph-based interpretable tense knowledge graph reasoning method

Abstract

The invention relates to a subgraph-based interpretable tense knowledge graph reasoning method, and belongs to the field of artificial intelligence. The method comprises the following steps: S1, initializing embedded representation of entities, relationships and timestamps; s2, updating the entity and relation embedding representation based on a tense knowledge graph encoder; s3, based on a given query, constructing and initializing an inference graph; s4, sampling neighbor edges and nodes, and iteratively expanding the reasoning graph; s5, updating sub-graph embedding of the sampling nodes in the inference graph; s6, spreading attention to the sampling nodes; s7, trimming the reasoning graph based on attention scores of edges and nodes; s8, obtaining a query result according to the reasoning graph obtained through expansion; and S9, extracting a relation path, mining a logic rule, and further providing interpretability for a query result.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and relates to an interpretable temporal knowledge graph reasoning method based on a subgraph. Background Art

[0002] A knowledge graph is a graph data structure used to represent facts in the real world. It consists of nodes and edges, where nodes represent entities that exist objectively in the real world and edges represent the relationships between entities. A temporal knowledge graph is an extended form that introduces time information based on a knowledge graph to describe the relationships and evolution processes between entities at different time points. Temporal knowledge graphs can be used to describe the historical and future relationships between entities, reflecting the evolution process and dynamic changes between entities, and have a wide range of application value.

[0003] Reasoning tasks are one of the current hot issues for temporal knowledge graphs, which aims to use existing facts with time information to predict missing facts in temporal knowledge graphs. Most existing reasoning methods are designed for traditional static knowledge graph tasks, and are unable to consider and utilize the additional time information in temporal knowledge graphs, resulting in the inability to solve the predictive reasoning problem of temporal knowledge graphs. Currently, academia and industry have proposed methods including but not limited to recurrent neural network-based methods and graph neural network-based methods for reasoning tasks on temporal knowledge graphs to improve prediction accuracy. However, these methods operate in a black box manner and cannot explain the prediction results. Some methods provide explainability for prediction results by generating logical rules, but cannot guarantee high prediction accuracy. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide an interpretable temporal knowledge graph reasoning method based on subgraphs, initialize the embedded representations of entities, relationships, and timestamps, update the embedded representations of entities and relationships based on the temporal knowledge graph encoder, build and initialize the reasoning graph for a given query, and expand it by iteratively sampling neighbor edges and nodes, update the subgraph embedding of the sampled nodes, propagate attention to the sampled nodes, prune the reasoning graph based on the attention scores of edges and nodes, and realize the interpretability of temporal knowledge graph reasoning and reasoning results based on the expanded reasoning graph.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] S1: Initialize the embedding representation of entities, relations, and timestamps;

[0007] S2: Update entity and relationship embedding representations based on the temporal knowledge graph encoder;

[0008] S3: Based on the given query, build and initialize the reasoning graph;

[0009] S4: Sample neighbor edges and nodes and iteratively expand the reasoning graph;

[0010] S5: Update the subgraph embedding of the sampled node in the reasoning graph;

[0011] S6: propagate attention to sampling nodes;

[0012] S7: Pruning the inference graph based on the attention scores of edges and nodes;

[0013] S8: Obtain query results according to the expanded reasoning graph;

[0014] S9: Extract relationship paths, mine logical rules, and further provide explainability for query results.

[0015] Furthermore, in step S1, the embedding representation of entities, relations, and timestamps is initialized. Specifically, given a temporal knowledge graph G = {V, R, T, F}, where V represents the entity set, R represents the relationship set, T represents the timestamp set, and F = {(s, r, o, t) | s, o ∈ V, r ∈ R, t ∈ T} represents the fact set; let h v represents the embedding of entity v∈V, g r Represents the embedding of relation r, using standard normal distribution for h v and r Initialize; let e t Represents the embedding of timestamp t, initialized to where w i ,b i , i=1,...,d are learnable parameters, and d represents the dimension of time embedding.

[0016] Furthermore, in step 2, a temporal knowledge graph encoder is constructed to update the entity and relationship embedding representations. Specifically, a temporal graph convolutional network GCN is constructed. represents the embedding of entity o at the kth layer of the temporal GCN, and uses the attention mechanism to aggregate neighbor information to update the entity embedding, namely: in represents the entity transformation matrix of the kth layer of the temporal GCN, N o = {s|(s,r,o,t)∈F} represents the incoming neighbor node set of entity o, P s,o = {(r, t)|(s, r, o, t) ∈ F} represents the set of temporal relationship edges between entity o and incoming neighbor node s, The transformation matrix representing the k-th layer of the temporal GCN generated by the embedding of relation r and timestamp t is modeled as: in and represents the learnable parameters, represents the embedding of the k-th layer relation r of the temporal GCN, R represents the function that converts the vector into a matrix, The normalized attention score of the temporal relation edge (r, t) between entity o and the incoming neighbor node s is modeled as: Where exp represents the exponential function; relation embedding The update formula is: in is the relation transformation matrix of the kth layer of the temporal GCN corresponding to relation r; let h′ v , g′ r Represents the embeddings of entity v and relation r output by the temporal GCN.

[0017] Further, in step S3, based on the given query, a reasoning graph is constructed and initialized, specifically: given query q = (s q ,r q ,? ,o q ), construct the inference graph, and expand the inference graph by iteratively searching the outgoing neighbor nodes of the nodes in the inference graph. represents the reasoning graph obtained by the lth reasoning step, let represents the subgraph embedding of entity v added to the reasoning graph at the lth reasoning step, let represents the attention score of entity v; using the query head entity s q Initialize the inference graph to get Initialize query header entity s q Subgraph embedding of for Attention score is 1.

[0018] Further, in step S4, the neighboring edges and nodes of the entities in the reasoning graph are sampled, and the reasoning graph is iteratively expanded. Specifically, let represents the outgoing neighbor node set of entity s, let Represents the set of temporal relationship edges between entity s and its outgoing neighbors; in the lth reasoning step, for the entity s added in the l-1th reasoning step, its outgoing neighbor nodes and edges are sampled, and the sampling probability of the neighbor edge with modeling timestamp t is: Add the sampled edges and corresponding nodes to the inference graph in; order Represents the outgoing neighbor edge set of entity s obtained after sampling, let represents the set of outgoing neighbor nodes of entity s obtained after sampling. If the edge between entity s and its outgoing neighbor node o is sampled, o is added to In the above example, let N o ′ indicates adding to the reasoning graph The set of incoming neighbor nodes of entity o in .

[0019] Furthermore, in step S5, the subgraph embedding of the sampled node in the reasoning graph is updated. Specifically, for the entity o added to the reasoning graph in the lth reasoning step, the embedding of entity o and the subgraph embedding of the passed neighboring nodes are aggregated using the attention mechanism to update the subgraph embedding of entity o. The update formula is: in represents the entity transformation matrix in the lth reasoning step, represents the transformation matrix generated by the embedding of relation r and timestamp t in the lth reasoning step, which is modeled as: in and represents the learnable parameters, The normalized attention score of the temporal relationship edge (r, t) between entity o and the incoming neighbor node s can be calculated by the semantic relevance of the query to the current edge, that is: Where q represents the query embedding, which is modeled as: Where W q Represents the query transformation matrix.

[0020] Further, in step S6, the attention score of the sampling node is determined based on the attention scores of the neighboring nodes. Specifically, for the entity o added to the reasoning graph in the lth reasoning step, the attention score of the entity o is obtained by aggregating the attention scores of the incoming neighboring nodes. Modeled as:

[0021]

[0022] Further, in step S7, the inference graph is pruned based on the attention scores of the edges and nodes, specifically: represents the contribution score of the temporal relation edge (r, t) between entity o and its incoming neighbor node s added to the reasoning graph at the lth reasoning step, modeled as: The edges newly added to the reasoning graph in the lth reasoning step are sorted from large to small according to their contribution scores, and only the first M edges with the largest contribution scores are retained.

[0023] Further, in step S8, the query result is obtained according to the expanded reasoning graph, specifically: the reasoning graph is expanded L times, and o a Represents the query result, using the entity with the highest attention score in the inference graph obtained by the last expansion as o a ,The reasoning graph obtained by iterative expansion can be used as a graph level explanation of the query results.

[0024] Furthermore, in step S9, the relationship path is extracted and the logical rules are mined to further provide explainability for the query results. Specifically, Represents entity s in the reasoning graph q and entity o a All the relationship paths between Represents entity s q and entity o a The i-th relationship path between i Indicates u i The corresponding path embedding uses the gated recurrent unit GRU for the path u i Each relationship embedding on is modeled; the path u i As a reason, the relationship r q As a result, we get rule u i = > r q , let g(u i ,r q ) represents the confidence of the rule, which is modeled as: g(u i ,r q )=cos(u i ,r q ), where cos represents cosine similarity, and the path with the highest confidence can be used as the rule-level explanation of the query result.

[0025] The beneficial effects of the present invention are as follows: the present invention is oriented to the task of temporal knowledge graph reasoning, and uses existing facts with time information to predict the missing fact information of the temporal knowledge graph, and proposes a subgraph-based interpretable temporal knowledge graph reasoning method. Initialize the embedding representation of entities, relationships, and timestamps, update the embedding representation of entities and relationships based on the temporal knowledge graph encoder, and can capture the global features of entities and relationships. For a given query, build and initialize the reasoning graph, and expand it by iteratively sampling neighbor edges and nodes, update the subgraph embedding of the sampled nodes, propagate attention to the sampled nodes, prune the reasoning graph based on the attention scores of edges and nodes, and realize temporal knowledge graph reasoning based on the expanded reasoning graph, and provide graph-level and rule-level explanations for the reasoning results.

[0026] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0028] Figure 1 A schematic diagram of temporal knowledge graph reasoning of the method of the present invention;

[0029] Figure 2 The figure is a schematic diagram of the process of the method of the present invention. DETAILED DESCRIPTION

[0030] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0031] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0032] Figure 1 This is a schematic diagram of temporal knowledge graph reasoning of the method described in the present invention. As shown in the figure, the embedded representations of entities, relationships, and timestamps are initialized, the embedded representations of entities and relationships are updated based on the temporal knowledge graph encoder, and for a given query, a reasoning graph is constructed and initialized, and expanded by iteratively sampling neighbor edges and nodes, the subgraph embedding of the sampled nodes is updated, attention is propagated to the sampled nodes, the reasoning graph is pruned based on the attention scores of the edges and nodes, and based on the expanded reasoning graph, the temporal knowledge graph reasoning and the interpretability of the reasoning results are achieved.

[0033] Figure 2 The flowchart of the method of the present invention is shown in the figure. As shown in the figure, the method of the present invention specifically includes the following steps:

[0034] 1) Initialize the embedding representation of entities, relations, and timestamps

[0035] Given a temporal knowledge graph G = {V, R, T, F}, where V represents the entity set, R represents the relationship set, T represents the timestamp set, and F = {(s, r, o, t) | s, o ∈ V, r ∈ R, t ∈ T} represents the fact set; let h v represents the embedding of entity v∈V, g r Represents the embedding of relation r, using standard normal distribution for h vand r Initialize; let e t Represents the embedding of timestamp t, initialized to where w i ,b i , i=1,...,d are learnable parameters, and d represents the dimension of time embedding.

[0036] 2) Update entity and relationship embedding representation based on temporal knowledge graph encoder

[0037] Construct a temporal graph convolutional network GCN, let represents the embedding of entity o at the kth layer of the temporal GCN, and uses the attention mechanism to aggregate neighbor information to update the entity embedding, namely: in represents the entity transformation matrix of the kth layer of the temporal GCN, N o = {s|(s,r,o,t)∈F} represents the incoming neighbor node set of entity o, P s,o = {(r, t)|(s, r, o, t) ∈ F} represents the set of temporal relationship edges between entity o and incoming neighbor node s, The transformation matrix representing the k-th layer of the temporal GCN generated by the embedding of relation r and timestamp t is modeled as: in and represents the learnable parameters, represents the embedding of the k-th layer relation r of the temporal GCN, R represents the function that converts the vector into a matrix, The normalized attention score of the temporal relation edge (r, t) between entity o and the incoming neighbor node s is modeled as: Where exp represents the exponential function; relation embedding The update formula is: in is the relation transformation matrix of the kth layer of the temporal GCN corresponding to relation r; let h′ v , g′ r Represents the embeddings of entity v and relation r output by the temporal GCN.

[0038] 3) Based on the given query, build and initialize the reasoning graph

[0039] Given a query q = (s q ,r q ,? ,o q ), construct the inference graph, and expand the inference graph by iteratively searching the outgoing neighbor nodes of the nodes in the inference graph. represents the reasoning graph obtained by the lth reasoning step, let represents the subgraph embedding of entity v added to the reasoning graph at the lth reasoning step, let represents the attention score of entity v; using the query head entity s q Initialize the inference graph to get Initialize query header entity s q Subgraph embedding of for Attention score is 1.

[0040] 4) Sample neighbor edges and nodes, and iteratively expand the inference graph

[0041] make represents the outgoing neighbor node set of entity s, let Represents the set of temporal relationship edges between entity s and its outgoing neighbors; in the lth reasoning step, for the entity s added in the l-1th reasoning step, its outgoing neighbor nodes and edges are sampled, and the sampling probability of the neighbor edge with modeling timestamp t is: Add the sampled edges and corresponding nodes to the inference graph in; order Represents the outgoing neighbor edge set of entity s obtained after sampling, let represents the set of outgoing neighbor nodes of entity s obtained after sampling. If the edge between entity s and its outgoing neighbor node o is sampled, o is added to In the above example, let N o ′ indicates adding to the reasoning graph The set of incoming neighbor nodes of entity o in .

[0042] 5) Update the subgraph embedding of the sampled node in the inference graph

[0043] Update the subgraph embedding of the sampled node in the reasoning graph. Specifically, for the entity o added to the reasoning graph in the lth reasoning step, use the attention mechanism to aggregate the embedding of entity o and the subgraph embedding of the incoming neighbor nodes, and update the subgraph embedding of entity o. The update formula is: in represents the entity transformation matrix in the lth reasoning step, represents the transformation matrix generated by the embedding of relation r and timestamp t in the lth reasoning step, which is modeled as: in and represents the learnable parameters, The normalized attention score of the temporal relationship edge (r, t) between entity o and the incoming neighbor node s can be calculated by the semantic relevance of the query to the current edge, that is: Where q represents the query embedding, which is modeled as: Where Wq Represents the query transformation matrix.

[0044] 6) Propagate attention to sampling nodes

[0045] For the entity o added to the reasoning graph in the lth reasoning step, the attention score of entity o is obtained by aggregating the attention scores of the incoming neighbor nodes Modeled as:

[0046] 7) Pruning the inference graph based on the attention scores of edges and nodes

[0047] make represents the contribution score of the temporal relation edge (r, t) between entity o and its incoming neighbor node s added to the reasoning graph at the lth reasoning step, modeled as: The edges newly added to the reasoning graph in the lth reasoning step are sorted from large to small according to their contribution scores, and only the first M edges with the largest contribution scores are retained.

[0048] 8) Obtain query results based on the expanded reasoning graph

[0049] Expand the reasoning graph L times, let o a Represents the query result, using the entity with the highest attention score in the inference graph obtained by the last expansion as o a ,The reasoning graph obtained by iterative expansion can be used as a graph level explanation of the query results.

[0050] 9) Extract relationship paths, mine logical rules, and provide further explainability for query results

[0051] make Represents entity s in the reasoning graph q and entity o a All the relationship paths between Represents entity s q and entity o a The i-th relationship path between i Indicates u i The corresponding path embedding uses the gated recurrent unit GRU for the path u i Each relationship embedding on is modeled; the path u i As a reason, the relationship r q As a result, we get rule u i = > r q , let g(u i ,r q ) represents the confidence of the rule, which is modeled as: g(u i ,r q )=cos(u i ,rq ), where cos represents cosine similarity, and the path with the highest confidence can be used as the rule-level explanation of the query result.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A subgraph-based interpretable temporal knowledge graph reasoning method, characterized by: The method comprises the following steps: S1: Initialize the embedding representation of entities, relations, and timestamps; S2: Update entity and relationship embedding representations based on the temporal knowledge graph encoder; S3: Based on the given query, build and initialize the reasoning graph; S4: Sample neighbor edges and nodes and iteratively expand the reasoning graph; S5: Update the subgraph embedding of the sampled node in the reasoning graph; S6: propagate attention to sampling nodes; S7: Pruning the inference graph based on the attention scores of edges and nodes; S8: Obtain query results according to the expanded reasoning graph; S9: Extract relationship paths, mine logical rules, and further provide explainability for query results.

2. According to claim 1, a subgraph-based interpretable temporal knowledge graph reasoning method is characterized by: In step S1, the embedding representation of entities, relations, and timestamps is initialized. Specifically, given a temporal knowledge graph G = {V, R, T, F}, where V represents the entity set, R represents the relationship set, T represents the timestamp set, and F = {(s, r, o, t) | s, o ∈ V, r ∈ R, t ∈ T} represents the fact set; let h v represents the embedding of entity v∈V, g r Represents the embedding of relation r, using standard normal distribution for h v and r Initialize; let e t Represents the embedding of timestamp t, initialized to where w i ,b i , i=1,...,d are learnable parameters, and d represents the dimension of time embedding.

3. The method according to claim 1, characterized in that: In step 2, a temporal knowledge graph encoder is constructed to update the entity and relationship embedding representations. Specifically, a temporal graph convolutional network GCN is constructed. represents the embedding of entity o at the kth layer of the temporal GCN, and uses the attention mechanism to aggregate neighbor information to update the entity embedding, namely: in represents the entity transformation matrix of the kth layer of the temporal GCN, N o = {s|(s,r,o,t)∈F} represents the incoming neighbor node set of entity o, P s,o = {(r, t)|(s, r, o, t) ∈ F} represents the set of temporal relationship edges between entity o and incoming neighbor node s, The transformation matrix representing the k-th layer of the temporal GCN generated by the embedding of relation r and timestamp t is modeled as: in and represents the learnable parameters, represents the embedding of the k-th layer relation r of the temporal GCN, R represents the function that converts the vector into a matrix, The normalized attention score representing the temporal relation edge (r, t) between entity o and incoming neighbor node s is modeled as: Where exp represents the exponential function; relation embedding The update formula is: in is the relation transformation matrix of the kth layer of the temporal GCN corresponding to relation r; let h′ v , g′ r Represents the embeddings of entity v and relation r output by the temporal GCN.

4. The method according to claim 1, characterized in that: In step S3, based on the given query, a reasoning graph is constructed and initialized. Specifically, given query q = (s q ,r q ,? ,t q ), construct the inference graph, and expand the inference graph by iteratively searching the outgoing neighbor nodes of the nodes in the inference graph. represents the reasoning graph obtained by the lth reasoning step, let represents the subgraph embedding of entity v added to the reasoning graph at the lth reasoning step, let represents the attention score of entity v; Use query header entities q Initialize the inference graph to get Initialize query header entity s q Subgraph embedding of for Attention score is 1.

5. The method according to claim 1, characterized in that: In step S4, the neighboring edges and nodes of the entities in the reasoning graph are sampled, and the reasoning graph is iteratively expanded. Specifically, let represents the outgoing neighbor node set of entity s, let Represents the set of temporal relationship edges between entity s and its outgoing neighbors; in the lth reasoning step, for the entity s added in the l-1th reasoning step, its outgoing neighbor nodes and edges are sampled, and the sampling probability of the neighbor edge with modeling timestamp t is: Add the sampled edges and corresponding nodes to the inference graph in; order Represents the outgoing neighbor edge set of entity s obtained after sampling, let represents the set of outgoing neighbor nodes of entity s obtained after sampling. If the edge between entity s and its outgoing neighbor node o is sampled, o is added to In the above example, let N o ′ indicates adding to the reasoning graph The set of incoming neighbor nodes of entity o in .

6. The method according to claim 1, characterized in that: In step S5, the subgraph embedding of the sampled node in the reasoning graph is updated. Specifically, for the entity o added to the reasoning graph in the lth reasoning step, the embedding of entity o and the subgraph embedding of the incoming neighbor nodes are aggregated using the attention mechanism to update the subgraph embedding of entity o. The update formula is: in represents the entity transformation matrix in the lth reasoning step, represents the transformation matrix generated by the embedding of relation r and timestamp t in the lth reasoning step, which is modeled as: in and represents the learnable parameters, The normalized attention score of the temporal relationship edge (r, t) between entity o and the incoming neighbor node s can be calculated by the semantic relevance of the query to the current edge, that is: Where q represents the query embedding, which is modeled as: Where W q Represents the query transformation matrix.

7. The method according to claim 1, characterized in that: In step S6, the attention score of the sampling node is determined based on the attention scores of the neighboring nodes. Specifically, for the entity o added to the reasoning graph in the lth reasoning step, the attention score of the entity o is obtained by aggregating the attention scores of the incoming neighboring nodes. Modeled as:

8. The method according to claim 1, characterized in that: In step S7, the inference graph is pruned based on the attention scores of the edges and nodes, specifically: represents the contribution score of the temporal relation edge (r, t) between entity o and its incoming neighbor node s added to the reasoning graph at the lth reasoning step, modeled as: The edges newly added to the reasoning graph in the lth reasoning step are sorted from large to small according to their contribution scores, and only the first M edges with the largest contribution scores are retained.

9. The method according to claim 1, characterized in that: In step S8, the query result is obtained according to the expanded reasoning graph, specifically: the reasoning graph is expanded L times, and o a Represents the query result, using the entity with the highest attention score in the inference graph obtained by the last expansion as o a ,The reasoning graph obtained by iterative expansion can be used as a graph level explanation of the query results.

10. The method according to claim 1, characterized in that: In step S9, the relationship path is extracted and the logical rules are mined to further provide explainability for the query results. Specifically, Represents entity s in the reasoning graph q and entity o a All the relationship paths between Represents entity s q and entity o a The i-th relationship path between i Indicates u i The corresponding path embedding uses the gated recurrent unit GRU for the path u i Each relationship embedding on is modeled; the path u i As a reason, the relationship r q As a result, we get rule u i = > r q , let g(u i ,r q ) represents the confidence of the rule, which is modeled as: g(u i ,r q )=cos(u i ,r q ), where cos represents cosine similarity, and the path with the highest confidence can be used as the rule-level explanation of the query result.

Images