Temporal Knowledge Graph Reasoning with Distributed Attention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for temporal knowledge graph reasoning fail to accurately capture the time-varying nature of temporal knowledge graphs, leading to incomplete and inaccurate predictions due to the use of constant entity embedding representations and insufficient modeling of temporal evolution.
Innovation Solution
The method employs distributed attention mechanisms to recombine temporal knowledge graphs into sparse matrices, constructing initial and second-layer attention based on historical frequency information, and using a flexible parameter training strategy to adjust attention scores, allowing for more accurate representation and prediction of temporal relationships.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If constant entity embedding representations are used for temporal knowledge graph reasoning, then the model structure is simple, but the accuracy of capturing time-varying features deteriorates
Solution Approach 1:
The patent segments the temporal knowledge graph into multiple timestamp subgraphs, where each subgraph represents the knowledge state at a specific timestamp. This segmentation allows the model to capture temporal evolution by processing each timestamp's subgraph separately while maintaining relationships across time, thus improving the accuracy of capturing time-varying features without requiring overly complex model structures.
Solution Approach 2:
The patent introduces a temporal dimension to the knowledge graph representation by creating timestamp-specific subgraphs. This dimensional transformation from a static graph to a sequence of temporal subgraphs enables the model to capture time-varying characteristics effectively, as each subgraph preserves the structural information at its corresponding timestamp while the sequence captures evolution over time.
2Ease of manufacture
If traditional reasoning methods (embedding, reinforcement learning, graph convolutional networks) are used, then the implementation is straightforward, but the modeling of temporal evolution is insufficient
Solution Approach 1:
The patent employs dynamic graph convolutional networks that adapt to temporal changes. The model dynamically adjusts the convolution operations based on the temporal characteristics of each timestamp subgraph, allowing it to capture evolving patterns in the knowledge graph. This dynamic approach significantly improves temporal evolution modeling compared to static traditional methods, while maintaining reasonable implementation complexity through modular architecture design.
3Measurement precision
If distributed attention mechanisms are applied to capture temporal features, then the accuracy of temporal reasoning is improved, but the computational complexity increases
Solution Approach 1:
The patent applies attention mechanisms separately to each timestamp subgraph rather than processing the entire temporal sequence at once. This segmentation of the attention computation reduces the overall computational complexity while still capturing temporal dependencies, as the attention operations are performed on smaller, manageable subgraphs at each timestamp independently.
Solution Approach 2:
The patent extracts and processes only the relevant information from each timestamp subgraph using attention mechanisms, rather than processing all information uniformly. This selective extraction of important temporal features improves reasoning accuracy by focusing computational resources on significant patterns while reducing unnecessary computations.
Data Source
AI summary
The present invention relates to a method for temporal knowledge graph reasoning based on distributed attention, comprising: recombining a temporal knowledge graph in a temporal serialization manner, accurately expressing the structural dependencies between time-evolution features and temporal subgraphs, and then extracting historical repetition facts and historical frequency information based on the sparse matrix storing historical subgraph information; assigning, by the query fact, initial first-layer attention to the facts that are historically repeated using an attention mechanism, and then by capturing the latest changes in the historical frequency information, assigning attention reward and punishment of the second-layer attention to the scores of the first-layer attention, respectively, to make attention more adaptable to time-varying features; finally, using the scores of the two layers of attention to make reasoning-based prediction about future events. Compared with traditional prediction methods, the present invention endows learnable distributed attention on different historical timestamps instead of obtaining a fixed embedding representation through an encoder, so that the model has better ability to solve time-varying problems.