A relationship-oriented attention network-based time knowledge graph completion method
By reconstructing the temporal knowledge graph using a relation-oriented attention network, the temporal evolution trajectory of relations is explicitly depicted, solving the problem of neglecting dynamic information of relations in existing technologies and achieving higher completion accuracy and interpretability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF ELECTRONICS SCI & TECH OF CHINA
- Filing Date
- 2026-05-11
- Publication Date
- 2026-06-09
AI Technical Summary
Existing time-series knowledge graph completion methods ignore the dynamic information of relation evolution over time, resulting in limited prediction performance and an inability to effectively explore the intrinsic dependencies and evolutionary patterns of relation sequences.
By employing a relation-oriented attention network, attention is directly calculated on relation sequences through reconstruction of the graph structure, generating time-aware relation embeddings, explicitly characterizing the temporal evolution trajectory of relations, and using an entity-time mapping matrix for dynamic projection to enhance the context-awareness of relation representation.
It significantly improves the accuracy of time-series knowledge graph completion, effectively captures the dynamic evolution of relationships, and enhances the prediction accuracy of missing entities, especially when dealing with complex relationship sequences.
Smart Images
Figure CN122174966A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of temporal knowledge graph technology, specifically relating to a temporal knowledge graph completion method based on a relation-oriented attention network. Background Technology
[0002] Temporal Knowledge Graphs (TKGs) represent dynamic facts as quadruples in the form of (subject, relation, object, time), which are essentially sequences of knowledge graph snapshots with their own timestamps. Temporal Knowledge Graph Completion (TKGC) aims to predict unknown facts by modeling historical knowledge graph snapshots, with interpolation settings focusing on completing missing historical facts, which is the core task of this invention.
[0003] Furthermore, accurate prediction of missing facts requires a comprehensive understanding of the evolution of historical facts. Existing research on TKGC, whether extending static knowledge graph models or employing graph neural networks, largely follows a common paradigm: focusing on modeling entity features and aiming to learn entity embedding representations with time-aware graph structures. In terms of data structure, these methods typically represent TKG as a multi-edge form, where multiple relationships between a pair of entities at different points in time are treated as a set of independent edges. However, these studies have a fundamental limitation: they treat the relationships themselves as static embedding vectors, ignoring the dynamic information of relationship evolution over time and severing the potential temporal continuity between events at different points in time. This entity-centric modeling bias and set-based data representation cannot effectively explore the intrinsic dependencies and evolutionary patterns of relationship sequences. Summary of the Invention
[0004] To address the issues of insufficient modeling of temporal features of relationships and limited prediction performance due to defects in data representation structure in existing temporal knowledge graph completion methods, this invention provides a temporal knowledge graph completion method based on relation-oriented attention networks. This method shifts the modeling focus from entities to relationships, reconstructs the graph structure, and directly performs attention calculations on the relation sequences to provide more accurate temporal knowledge graph completion results.
[0005] The technical solution adopted in this invention is as follows:
[0006] A temporal knowledge graph completion method using a relation-oriented attention network includes the following steps:
[0007] Step 1: Obtain the known entities in the query to be completed Traverse the temporal knowledge graph and organize the known entities in ascending order of time. The relationships between relevant facts yield multiple relational chains;
[0008] Step 2: Obtain the target relation in the query to be completed. and timestamp Based on known entities and timestamp Generate entity-time mapping matrix Based on entity-time mapping matrix , for target relationship In the multi-chain relationship, the first Relationship Projecting the embedding vector yields the corresponding target mapping vector. and historical mapping relationship vector Where R is the total number of relations contained in a multi-chain relation;
[0009] Step 3: Map the target relationship vector For query vector, historical mapping relationship vector Given key vectors and value vectors, calculate the target relation. Relations in a multi-chain relation The relevance of the elements is weighted and aggregated to obtain a relation representation vector that incorporates contextual information. ;
[0010] Step 4: Represent the relation vector Perform deep nonlinear transformation and residual normalization processing, and correlate with the target. The embedding vectors are weighted and fused to obtain the time-aware relation embedding. ;
[0011] Step 5: Treat all entities in the time-series knowledge graph as unknown entities in the query to be completed. This generates multiple candidate facts to be evaluated; an entity temporal embedding method is used to embed facts based on timestamps. Generate known entities Embedded representation and the unknown entities of each candidate fact Embedded representation Embedded relationship with time perception All inputs are fed into the preset scoring function. In this process, the credibility score of each candidate fact is calculated;
[0012] Step 6: Obtain the training dataset processed by negative sampling technique, and perform iterative training with the goal of maximizing the credibility score of positive facts and minimizing the credibility score of negative facts. After training, input the time-series knowledge graph to be completed, and output the entity with the highest credibility score for completion.
[0013] Furthermore, the specific process of step 1 is as follows:
[0014] Step 1.1: Convert the temporal knowledge graph into a quadruple form, including subject, object, relation, and timestamp;
[0015] Step 1.2: Obtain the known entities in the query to be completed. Traverse the fact set in the time-series knowledge graph to obtain the known entities A subset of facts with the subject as the subject, and with known entities as the subject as the subject. A subset of facts of the object;
[0016] Step 1.3: Using known entities A subset of facts as the main body and known entities Sort the relations in the factual subset of the object in ascending order of timestamps to obtain the known entities. Corresponding subject relationship chain and object relation chain Based on known entities Select the subject relation chain in the query to be completed. or object relation chain , as a relational multi-chain.
[0017] Furthermore, the specific process of step 2 is as follows:
[0018] Step 2.1: Obtain the target relation in the query to be completed. and timestamp ; timestamp It is decomposed into three time components: year, month, and day, and a low-dimensional year vector representation is generated. Monthly vector representation Daily vector representation Adding them together yields a timestamp vector. Simultaneously, it learns to generate known entities. Target Relationship Relationships in multi-chain relationships The low-dimensional vector representations are denoted as entity vectors. Target relation vector and historical relationship vector ;
[0019] Step 2.2: Calculate entity vectors and timestamp vector The outer product yields a dynamic, context-specific entity-time mapping matrix. In the formula, the superscript T indicates transpose;
[0020] Step 2.3: Based on the entity-time mapping matrix For the target relation vector and historical relationship vector Perform a linear transformation to obtain the target mapping relationship vector in the entity-time subspace. and historical mapping relationship vector .
[0021] Furthermore, in step 2.1, a discrete feature embedding method is used to learn and generate a low-dimensional vector representation.
[0022] Furthermore, the specific process of step 3 is as follows:
[0023] Step 3.1: Map the target relationship vector For query vector, historical mapping relationship vector These are key vectors and value vectors;
[0024] Step 3.2: Calculate the query vector using a scaled dot product attention mechanism. With key vector Attention scores between ;
[0025] Step 3.3: Use the Softmax function to score all attention levels. Normalization is performed to obtain the relation. Attention weights ;
[0026] Step 3.4: Based on attention weights For all value vectors By performing a weighted summation, a relational representation vector that incorporates contextual information is obtained. .
[0027] Furthermore, the attention score in step 3.2 The calculation formula is:
[0028]
[0029] In the formula, is the dimension of the low-dimensional vector representation.
[0030] Furthermore, following step 3.4, the method also includes processing the relation representation vector. The multi-head attention mechanism processing process.
[0031] Furthermore, the specific process of step 4 is as follows:
[0032] Step 4.1: Represent the relation vector Perform a deep nonlinear transformation to obtain the output vector. ;
[0033] Step 4.2: Using residual joins and layer normalization, the relation representation vector is transformed. With output vector After addition, normalization is performed to obtain a vector. ;
[0034] Step 4.3: Transfer the vector Relationship vector with target Weighted fusion is performed to obtain the final time-aware relation embedding. In the formula, These are adjustable fusion weight hyperparameters.
[0035] Furthermore, in step 4.1, a deep nonlinear transformation is performed based on a feedforward neural network consisting of two linear transformation layers and a nonlinear activation function.
[0036] Furthermore, the preset scoring function in step 5 Specifically:
[0037]
[0038] In the formula, This represents the Hadama product.
[0039] Furthermore, the training dataset mentioned in step 6 is specifically ICEWS14, ICEWS05-15, or GDELT.
[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0041] 1. This invention proposes a relation-oriented attention network-based temporal knowledge graph completion method, fundamentally changing the focus of structural representation and modeling of temporal knowledge graphs from entity-centric to relation-centric. It restructures the traditional multi-edge graph representation of temporal knowledge graphs, linking a series of relational events related to the same entity and occurring in chronological order to form relational multi-chains, explicitly depicting the evolution trajectory of relations over time. An entity-time mapping matrix is designed, dynamically embedding and projecting relations from the relational multi-chains into a specific entity-time subspace based on the entities and timestamps in the query to be completed, endowing each relation with context-aware capabilities. A relation-oriented attention mechanism is applied to the relation sequence, automatically identifying and focusing on the most important historical relations for the current prediction task, generating a dynamic time-aware relation embedding that integrates rich temporal information, and then using high-quality relation representations to complete fact scoring and entity prediction.
[0042] 2. This invention can effectively capture the temporal dynamic information of relationships that is ignored by existing technologies, and significantly improves the accuracy of temporal knowledge graph completion on multiple standard datasets, especially suitable for processing complex relation sequences. Attached Figure Description
[0043] Figure 1 This is a flowchart of the temporal knowledge graph completion method for the relation-oriented attention network proposed in Example 1. Detailed Implementation
[0044] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.
[0045] Example 1
[0046] This embodiment proposes a temporal knowledge graph completion method using a relation-oriented attention network, the process of which is as follows: Figure 1 As shown, it includes the following steps:
[0047] Step 1: Traverse the input time-series knowledge graph and identify the known entities in the query to be completed. By organizing the relationships between related facts in ascending chronological order, a multi-chain relationship is reconstructed, specifically as follows:
[0048] Step 1.1: Convert the input time-series knowledge graph into a quadruple form, including subject, object, relation, and timestamp;
[0049] Step 1.2: Obtain the known entities in the query to be completed. Traverse the fact set in the time-series knowledge graph to obtain the known entities A subset of facts with the subject as the subject, and with known entities as the subject as the subject. A subset of facts of the object;
[0050] Step 1.3: Using known entities A subset of facts as the main body and known entities Sort the relations in the factual subset of the object in ascending order of timestamps to obtain the known entities. Corresponding subject relationship chain and object relation chain Based on known entities Select the subject relation chain in the query to be completed. or object relation chain As a relational multi-chain, it contains a total of R relations;
[0051] Step 2: Generate a context-dependent subspace based on the entities and timestamps in the query to be completed, for time mapping, specifically:
[0052] Step 2.1: Obtain the target relation in the query to be completed. and timestamp ; timestamp The time component is decomposed into three time components: year, month, and day. A discrete feature embedding method is used to learn and generate a low-dimensional year vector representation. Monthly vector representation Daily vector representation Adding them together yields a timestamp vector. Simultaneously, a discrete feature embedding method is used to learn and generate known entities. Target Relationship In the multi-chain relationship, the first Relationship The low-dimensional vector representations are denoted as entity vectors. Target relation vector and historical relationship vector ;
[0053] Step 2.2: Calculate entity vectors and timestamp vector The outer product yields a dynamic, context-specific entity-time mapping matrix. In the formula, the superscript T indicates transpose;
[0054] Step 2.3: Based on the entity-time mapping matrix For the target relation vector and historical relationship vector Perform a linear transformation to obtain the target mapping relationship vector in the entity-time subspace. and historical mapping relationship vector .
[0055] Step 3: Relation-oriented attention computation generates a relation representation that incorporates temporal context information, specifically as follows:
[0056] Step 3.1: Map the target relationship vector For query vector, historical mapping relationship vector These are key vectors and value vectors;
[0057] Step 3.2: Calculate the query vector using a scaled dot product attention mechanism. With key vector Attention scores between ;
[0058] Step 3.3: Use the Softmax function to score all attention levels. Normalization is performed to obtain the relation. Attention weights ;
[0059] Step 3.4: Based on attention weights For all value vectors By performing a weighted summation, a relational representation vector that incorporates contextual information is obtained. ;
[0060] Step 3.5: To enhance the model's representational capabilities, a multi-head attention mechanism is employed to process the relation representation vectors. Specifically, H independent attention heads are set up in parallel, and the output vectors of the H heads are concatenated to form the final multi-head attention output. ;
[0061] Step 4: Perform deep transformation and fusion on the multi-head attention output to obtain the final time-aware relationship embedding, specifically as follows:
[0062] Step 4.1: Output multi-head attention The input is fed into a feedforward neural network consisting of two linear transformation layers and a nonlinear activation function, where a deep nonlinear transformation is performed to obtain the output vector. ;
[0063] Step 4.2: Using residual joins and layer normalization, the relation representation vector is transformed. With output vector After addition, normalization is performed to obtain a vector. ;
[0064] Step 4.3: Transfer the vector Relationship vector with target Weighted fusion is performed to obtain the final time-aware relation embedding. In the formula, These are adjustable fusion weight hyperparameters;
[0065] Step 5: Utilize time-aware relation embedding and entity time-aware embedding to perform entity prediction using a scoring function, specifically:
[0066] Step 5.1: Employ the entity time-series embedding method, combining it with the timestamp. Known entities are generated at the same time. Embedded representation ;
[0067] Step 5.2: Treat all entities in the time-series knowledge graph as unknown entities in the query to be completed. This generates multiple candidate facts to be evaluated;
[0068] Step 5.3: Employ the entity time-series embedding method to integrate the timestamps in the query to be completed. At the same point in time, unknown entities are generated for each candidate fact. Embedded representation ;
[0069] Step 5.4: Embedded representation , Embedding with time perception All inputs are fed into the preset scoring function. In this process, the credibility score of each candidate fact is calculated;
[0070] Among them, the preset scoring function Specifically:
[0071]
[0072] In the formula, This represents the Hadamard product, which is element-wise multiplication.
[0073] Step 6: Obtain the training dataset after processing with negative sampling techniques (i.e., constructing several negative examples for each real fact), and perform iterative training with the goal of maximizing the credibility score of positive examples while minimizing the credibility score of negative examples, and optimize based on the cross-entropy loss function; after training, input the time-series knowledge graph to be completed, and output the entity with the highest credibility score for completion.
[0074] Below, we compare the temporal knowledge graph completion method of the relation-oriented attention network proposed in this embodiment with common knowledge graph related methods in this field on common temporal knowledge graph datasets, using a general evaluation index in the field of temporal knowledge graph completion.
[0075] Common temporal knowledge graph datasets in this field include ICEWS14 (Integrated Early Warning System for Crisis 2014 dataset), ICEWS05-15 (Integrated Early Warning System for Crisis 2005-2015 dataset), and GDELT (Global Events, Languages and Tone Database).
[0076] Common evaluation metrics include MRR (mean reciprocal ranking), Hits@1 (the proportion of correct entities ranked 1st), and Hits@10 (the proportion of correct entities ranked within the top 10). The higher the value of the metric, the better the model's predictive performance.
[0077] Common knowledge graph-related methods in this field include traditional static knowledge graph embedding methods, basic temporal extension models, and DE series models based on entity temporal evolution. Among them, traditional static knowledge graph embedding methods include TransE (translation embedding model), DistMult (distributed multilinear embedding model), and SimplE (simple embedding model); basic temporal extension models include TTransE (temporal translation embedding model), HyTE (hyperplane-based temporal knowledge graph embedding model), TA-TransE (time-aware translation embedding model), and TA-DistMult (time-aware distributed multilinear embedding model); and DE (diachronic evolution embedding) series models based on entity temporal evolution include DE-TransE (diachronic evolution embedding-translation embedding model), DE-DistMult (diachronic evolution embedding-distributed multilinear embedding model), and DE-SimplE (diachronic evolution embedding-simple embedding model).
[0078] Table 1 shows a comparison of the temporal knowledge graph completion method (TRGAN) proposed in this embodiment with the general evaluation metrics of common knowledge graph-related methods in this field.
[0079] Table 1
[0080]
[0081] As can be seen, compared with common knowledge graph methods in this field, the temporal knowledge graph completion method based on relation-oriented attention networks proposed in this embodiment has significantly improved the MRR, Hits@1 and Hits@10 evaluation metrics on three datasets.
[0082] The above results demonstrate that the method of the present invention can effectively capture the dynamic evolution of relationships in time-series knowledge graphs, significantly improve the prediction accuracy of missing entities, and verify the effectiveness and superiority of the method.
[0083] The temporal knowledge graph completion method based on relation-oriented attention networks proposed in this invention solves key problems in temporal knowledge graph completion through the following technical means:
[0084] 1. Direct Modeling of Relationship Temporal Dynamics: This invention creates a "relational multi-chain" data structure, linking a series of relational events related to the same entity and occurring in chronological order, fundamentally changing the existing "set-based" multi-edge graph structure representation. This "sequence-based" representation explicitly characterizes the evolutionary trajectory of relationships over time, enabling the model to directly model the inherent dependencies and evolutionary patterns of relational sequences, thereby uncovering key temporal information overlooked by existing technologies.
[0085] 2. Context-Aware Dynamic Relation Representation: Through time mapping, this invention can dynamically generate a mapping matrix based on the entities and timestamps in the query to be completed. This matrix projects all static relation embedding vectors in a multi-chain relation into a specific entity-time subspace, enabling each relation representation to be aware of its specific entity and time context, transforming it from a static vector into a dynamic vector closely related to the current query, greatly enhancing the accuracy and flexibility of the representation.
[0086] 3. Enhanced Interpretability and Accuracy: This invention utilizes a relationship-oriented attention mechanism to distinguish the importance of different historical relationships within a relationship chain to the current prediction task. By visualizing the attention weights, it becomes clear which historical relationships the model considers more important when making predictions. This transparent decision-making process not only enhances the model's interpretability but also allows it to focus on the most critical information, thereby significantly improving prediction accuracy.
[0087] Compared to existing technologies, this invention performs better in handling the temporal dynamics of relationships. By shifting the modeling focus from entities to relationships, it effectively captures overlooked relationship evolution patterns. It maintains high completion accuracy, especially when dealing with complex relationship sequences. Furthermore, this method offers significant advantages in model interpretability and framework compatibility.
[0088] It should be noted that this is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A temporal knowledge graph completion method using a relation-oriented attention network, characterized in that, Includes the following steps: Step 1: Obtain the known entities to be completed in the query. Traverse the temporal knowledge graph and organize the known entities in ascending order of time. The relationships between relevant facts yield multiple relational chains; Step 2: Obtain the target relation to be completed in the query. and timestamp Based on known entities and timestamp Generate entity-time mapping matrix ; Based on entity-time mapping matrix , for target relationship In the multi-chain relationship, the first Relationship Projecting the embedding vector yields the corresponding target mapping vector. and historical mapping relationship vector Where R is the total number of relations contained in a multi-chain relation; Step 3: Map the target relationship vector For query vector, historical mapping relationship vector Given key vectors and value vectors, calculate the target relation. With each relationship The correlations are weighted and aggregated to obtain a relation representation vector. ; Step 4: Represent the relation vector Perform deep nonlinear transformation and residual normalization processing, and correlate with the target. The embedding vectors are weighted and fused to obtain the time-aware relation embedding. ; Step 5: Treat all entities in the time-series knowledge graph as unknown entities in the query to be completed. This generates multiple candidate facts to be evaluated; an entity temporal embedding method is used to embed facts based on timestamps. Generate known entities Embedded representation and the unknown entities of each candidate fact Embedded representation Embedded relationship with time perception All inputs are fed into the preset scoring function. In this process, the credibility score of each candidate fact is calculated; Step 6: Obtain the training dataset processed by negative sampling technique, and perform iterative training with the goal of maximizing the credibility score of positive facts and minimizing the credibility score of negative facts. After training, input the time-series knowledge graph to be completed, and output the entity with the highest credibility score for completion.
2. The temporal knowledge graph completion method of the relation-oriented attention network according to claim 1, characterized in that, The specific process of step 1 is as follows: Step 1.1: Convert the temporal knowledge graph into a quadruple form, including subject, object, relation, and timestamp; Step 1.2: Obtain the known entities to be completed in the query. Traverse the fact set in the time-series knowledge graph to obtain the known entities A subset of facts with the subject as the subject, and with known entities as the subject as the subject. A subset of facts of the object; Step 1.3: Using known entities A subset of facts as the main body and known entities The relations within the factual subset of the object are sorted in ascending order by timestamp to obtain the subject relation chain. and object relation chain Based on known entities Select the subject relation chain in the query to be completed. or object relation chain , as a relational multi-chain.
3. The temporal knowledge graph completion method of the relation-oriented attention network according to claim 2, characterized in that, The specific process of step 2 is as follows: Step 2.1: Obtain the target relation to be completed in the query. and timestamp ; timestamp It is decomposed into three time components: year, month, and day, and a low-dimensional year vector representation is generated. Monthly vector representation Daily vector representation Adding them together yields a timestamp vector. Simultaneously, it learns to generate known entities. Target Relationship Relationships in multi-chain relationships The low-dimensional vector representations are denoted as entity vectors. Target relation vector and historical relationship vector ; Step 2.2: Calculate entity vectors and timestamp vector The outer product yields the entity-time mapping matrix. In the formula, the superscript T indicates transpose; Step 2.3: Based on the entity-time mapping matrix For the target relation vector and historical relationship vector Perform a linear transformation to obtain the target mapping vector. and historical mapping relationship vector .
4. The temporal knowledge graph completion method of the relation-oriented attention network according to claim 3, characterized in that, In step 2.1, a discrete feature embedding method is used to learn and generate a low-dimensional vector representation.
5. The temporal knowledge graph completion method for relation-oriented attention networks according to claim 4, characterized in that, The specific process of step 3 is as follows: Step 3.1: Map the target relationship vector For query vector, historical mapping relationship vector These are key vectors and value vectors; Step 3.2: Calculate the query vector using a scaled dot product attention mechanism. With key vector Attention scores between ; Step 3.3: Use the Softmax function to score all attention levels. Normalization is performed to obtain the relation. Attention weights ; Step 3.4: Based on attention weights For all value vectors Perform a weighted summation to obtain the relation representation vector. .
6. The temporal knowledge graph completion method of the relation-oriented attention network according to claim 5, characterized in that, Attention score in step 3.2 The calculation formula is: ； In the formula, is the dimension of the low-dimensional vector representation.
7. The temporal knowledge graph completion method for relation-oriented attention networks according to claim 5, characterized in that, Following step 3.4, the process also includes processing the relation representation vector. The multi-head attention mechanism processing process.
8. The temporal knowledge graph completion method of the relation-oriented attention network according to claim 7, characterized in that, The specific process of step 4 is as follows: Step 4.1: Represent the relation vector Perform a deep nonlinear transformation to obtain the output vector. ; Step 4.2: Using residual joins and layer normalization, the relation representation vector is transformed. With output vector After addition, normalization is performed to obtain a vector. ; Step 4.3: Transfer the vector Relationship vector with target Weighted fusion is performed to obtain time-aware relation embeddings. In the formula, These are adjustable fusion weight hyperparameters.
9. The temporal knowledge graph completion method for relation-oriented attention networks according to claim 8, characterized in that, In step 4.1, a deep nonlinear transformation is performed based on a feedforward neural network consisting of two linear transformation layers and a nonlinear activation function.
10. The temporal knowledge graph completion method of the relation-oriented attention network according to claim 1, characterized in that, The preset scoring function in step 5 Specifically: ； In the formula, This represents the Hadama product.