Time sequence knowledge graph reasoning method based on evolution retention mechanism and cross-dimension interaction

Abstract

The invention discloses a time sequence knowledge graph reasoning method based on an evolution retention mechanism and cross-dimension interaction, and belongs to the technical field of artificial intelligence and time sequence knowledge graphs. According to the method, a knowledge forgetting threshold value is set through a dynamic evolution capture network, historical snapshots in an effective time window are reserved, information weights are controlled based on time decay factors, knowledge importance is gradually attenuated until outdated information is completely forgotten, and a knowledge forgetting threshold value is obtained. Historical information of continuous time points is reserved in each time snapshot to simulate continuous dynamic evolution of knowledge; and cross-dimensional feature interaction of entities, relationships and time is modeled by using multi-layer expansion convolution and a residual network. According to the method, the modeling capability for discontinuous information is reserved and enhanced through an evolution mechanism, nonlinear dependence is captured through cross-dimension deep interaction, the prediction flexibility and robustness are remarkably improved, and the method is widely applied to dynamic knowledge reasoning scenes such as recommendation systems and intelligent medical treatment.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and knowledge graph technology, specifically to a temporal knowledge graph reasoning method based on evolutionary preservation mechanisms and cross-dimensional interaction, used to predict missing facts in future event stamps. Background Technology

[0002] Temporal knowledge graph (TKG) reasoning enhances traditional triples by incorporating time-based information into quadruples (subject, relation, object, timestamp) to better capture the dynamic nature of the real world. A common approach is to represent the temporal knowledge graph as a series of snapshots across multiple time steps, using graph structure encoders and temporal encoders to derive entity and relation representations. However, this snapshot-based approach faces two significant problems. First, the lack of a dynamic retention strategy for effective time windows means that important connections between adjacent snapshots are not modeled as continuous evolutionary information, and much outdated and worthless information is retained due to processing long-term historical snapshots, thus reducing the accuracy and efficiency of reasoning. Second, cross-dimensional interactions and potential semantic sub-feature dependencies of entities and relations are often ignored, resulting in insufficient dynamic evolutionary representation and poor model reasoning capabilities. Summary of the Invention

[0003] Current methods typically divide the TKG into isolated static snapshots at each time point, using graph structure encoders and temporal encoders to derive entity and relation representations. However, this snapshot-based approach faces two significant problems: it fails to capture valuable continuous evolutionary processes, and cross-dimensional interactions between entities and relations, as well as potential semantic sub-feature dependencies, are often overlooked, leading to insufficient dynamic evolutionary representation and poor model inference capabilities. (See...) Figure 1 (a) To address the above issues, a temporal knowledge graph reasoning framework that integrates evolution preservation mechanisms and cross-dimensional interactions is proposed. This framework captures the continuous dynamic evolution process and cross-dimensional interactions between entity and relation sub-features in the temporal knowledge graph. (See...) Figure 1 (b) An evolutionary retention mechanism is designed, which can aggregate snapshot information of the current timestamp while retaining historical information from previous times. A knowledge forgetting threshold is set to lock the timestamp range of aggregated historical snapshots, forgetting outdated information and retaining important historical information. Information is updated step by step by combining graph convolutional networks and recurrent neural networks to effectively capture valuable continuous dynamic evolution processes. A cross-dimensional interaction module is designed to capture cross-dimensional interactions between different semantic sub-features of entities and relations using multi-layer extended convolutional operations. This enables the network to capture the implicit correlations and dependencies of entities and relations at the same level, enhancing the modeling ability of continuous dynamic evolution processes.

[0004] The network proposed in this invention sets a knowledge forgetting threshold Δt. forgetA dynamic time window is constructed to retain valid historical snapshots and gradually decay outdated information. The weight of historical information is dynamically adjusted through a time decay factor to ensure that recent information dominates the prediction and long-term information is gradually weakened. A graph convolutional network is used to model the local structure of each time snapshot, embedding and splicing the neighbor nodes and their relationships of the target entity. Joint features are generated through a learnable parameter matrix, and the aggregation weight is dynamically calculated based on the difference between the current and historical timestamps to weightedly fuse neighbor information. The hidden state of entities and relationships is updated through a gated recurrent unit, and the historical state is fused with the current features and passed to the next timestamp to ensure the collaborative modeling of long-term temporal dependencies and short-term dynamic changes.

[0005] The network proposed in this invention follows the following modeling principles: (1) Forgetting outdated knowledge: Within a time window, knowledge gradually loses importance through a decay factor until it exceeds the knowledge forgetting threshold and is completely forgotten. For fact f t =(s, r, o, t), tt f ≤Δt forget When f t timestamp t f Exceeding the knowledge forgetting threshold Δt forget At that time, f t (2) Preserve historical knowledge: for each time t, It not only preserves valuable historical facts from the current timestamp, but also preserves previous timestamps. This is beneficial for characterizing continuous evolutionary processes, even if the current time t does not contain the fact. (3) Updating new knowledge: As new facts emerge, the knowledge graph is updated to reflect these changes. When a new fact f appears... t+1 =(s, r, o, t+1), which will update the snapshot. Existing relationships This ensures that the knowledge graph maintains its current state and accurately represents the latest state of entity relationships. (4) Timestamp annotation: In each snapshot, each edge is marked with the actual time when the relationship occurred. Snapshot The facts in f t =(s, r, o, t) will be represented as an edge This annotation is crucial because it provides temporal context, enabling the model to distinguish relationships occurring at different times and helping to characterize continuous temporal evolution.

[0006] The proposed network employs multi-layer dilated convolutions to extract cross-dimensional features layer by layer, capturing the non-linear dependencies of entities, relationships, and time. Shallow convolutions capture local fine-grained interactions, while deep convolutions cover global dependencies through exponential dilation. The output of each convolutional layer is superimposed on the input through residual connections. A 1×1 convolutional kernel is used to align dimensions, and a ReLU activation function is introduced to enhance non-linear expressive power. The binary cross-entropy loss function is optimized by maximizing the probability of positive samples and minimizing the probability of negative samples, thus distinguishing the real interactions between entities and relationships and accurately predicting the inference results.

[0007] Compared with the prior art, the present invention has the following obvious advantages:

[0008] (1) This invention introduces a knowledge forgetting threshold and a time decay factor through a dynamic evolution capture network, constructs an adaptive time window, retains the effective information in the current and recent historical snapshots, and gradually weakens outdated information through linear decay, ensuring that the model simulates the dynamic evolution process while avoiding redundant interference.

[0009] (2) This invention designs multi-layer dilated convolution and residual connections through a cross-dimensional interactive network. The dilation rate increases exponentially, covering local fine-grained interactions layer by layer to global cross-time dependencies. The residual connections align dimensions through 1×1 convolution and superimpose non-linear activations to avoid gradient vanishing and enhance feature representation capabilities. Attached Figure Description

[0010] Figure 1 A comparison diagram of existing methods for temporal knowledge graph reasoning and the present invention is provided to illustrate the differences in capturing continuous evolutionary reasoning.

[0011] Figure 2 This is an architecture diagram of the cross-dimensional temporal knowledge graph reasoning method (CDTENet) based on the evolution preservation mechanism proposed in this invention, which includes the Dynamic Evolution Capture Network (DECN) and the Cross-Dimensional Interaction Network (CDIN). Detailed Implementation

[0012] To facilitate understanding and use of the present invention by those skilled in the art, the specific processes and implementation methods of the present invention will be described in detail with reference to the accompanying drawings.

[0013] Figure 2 This paper demonstrates the flowchart of the cross-dimensional temporal knowledge graph reasoning method based on the evolution preservation mechanism proposed in this invention. Input: historical snapshot sequence. Each snapshot contains a quadruple (s, r, o, t), extracting the entity set ε and the relation set. For each time snapshot Aggregate neighbor information and merge embedded x through entity-relationship combination operations such as simple join, element addition, and element-wise multiplication. o Annotate the actual time when each relationship occurred. Provide temporal context and weight the aggregated features. Utilize the current timestamp t, historical timestamp t', and combined embedding x. o The weights α are calculated, and the current time, historical time, and their combination are embedded and mapped into a feature space. The temporal context information is modeled using the non-linear activation functions LeakyReLU and Sigmoid, and the weights are dynamically adjusted based on the relationship between the current time and historical time. A time decay factor β is introduced. t-t′ This reduces the impact of earlier interactions, ensuring the model focuses more on more recent interactions. It combines weights α and a decay factor β. t-t′ The neighbors of the aggregation node o obtain the final representation X. o Set a knowledge forgetting threshold Δt. forget Entity e and relation r establish a valid time window sequence at t timestamps. Starting from time t, the importance of knowledge is gradually reduced until it exceeds the knowledge forgetting threshold, at which point outdated information is completely forgotten. This operation allows the model to retain the latest and most relevant knowledge, reflecting continuous dynamic features that change over time. A time snapshot sequence is input into the RNN to capture continuous temporal features, progressively updating the hidden states of entity e and relation r. and and The historical representation of entities and relations at time t: and

[0014] The entities, relations, and embeddings at time t are stacked into a joint embedding matrix of dimension D. Forming the input sequence For X t The model employs extended convolutions, introducing expansion between the convolutional kernels to allow the convolutional filters to cover a wider range of input data. This model consists of four layers of extended convolutions, ultimately outputting a sequence. Each dimension captures combined features from other dimensions. Combining extended convolutions with residual blocks addresses the vanishing gradient and overfitting issues in deep network training, ultimately yielding a D-dimensional predicted embedding O. The residual block comprises extended convolutional layers, weight normalization layers, ReLU layers, and dropout layers. To reconcile the difference in input and output widths, additional 1×1 convolutions are applied to maintain element-wise addition compatibility. The predicted embedding O is transformed into a d-dimensional vector form vec(O), associated with each tail entity e. o By performing the inner product, we obtain the matching score ψ. r (e s e o The inference results are predicted by minimizing the binary cross-entropy loss function.

[0015] The experimental results on the ICEWS14, ICEWS18, WIKI, and GDELT datasets are shown in Tables 1 and 2, comparing the performance of different models in temporal knowledge graph reasoning. Baseline models were divided into three categories: static methods, interpolation methods, and extrapolation methods. The experimental results reveal the following: (1) Static methods (such as DistMult and ComplEx) performed poorly because they failed to effectively utilize dynamic temporal information for modeling. (2) Interpolation methods (such as TTransE and TA-DistMult) showed improvement over static methods, but their performance was still insufficient compared to extrapolation methods, highlighting the importance of considering future knowledge trends when making accurate predictions. (3) Among extrapolation methods, CDTENet performed exceptionally well on the mean reciprocal rank (MRR) evaluation metric, improving by 1.5%, 8.8%, 6.5%, and 2.2%, respectively, demonstrating its significant advantage in temporal reasoning tasks. Through these experiments, we can more clearly understand the effectiveness and limitations of different methods in temporal knowledge graph reasoning.

[0016] Table 1

[0017] Table 2

[0018]

[0019] This invention addresses key shortcomings of existing methods, such as redundant and fragmented historical information and insufficient cross-dimensional dependencies, through two core technologies: dynamic evolution preservation mechanism and cross-dimensional deep interaction modeling. Experimental data and theoretical design jointly demonstrate that CDTENet has significant advantages in dynamic knowledge graph reasoning tasks, providing a more flexible and reliable solution for studying continuous dynamic evolution and complex interaction scenarios.

Claims

1. A temporal knowledge graph reasoning method based on evolutionary preservation mechanisms and cross-dimensional interaction, characterized in that, Includes the following steps: (1) Construct a continuous evolution snapshot of the temporal knowledge graph through the Dynamic Evolutionary Capture Network (DECN), set a knowledge forgetting threshold to lock the effective time window, and retain information of the current moment and the historical moment; Graph convolutional networks and recurrent neural networks are used to perform structural modeling and temporal updates of snapshots, and the weights of historical information are dynamically adjusted by combining time decay factors. (2) The cross-dimensional semantic sub-feature dependencies of entities and relationships are captured by the cross-dimensional interaction network (CDIN), and multi-layer dilated convolution and residual network are used to extract multi-dimensional interaction features. The model parameters are optimized based on inner product matching score and binary cross-entropy loss function to predict entity relationships of future timestamps.

2. The method according to claim 1, characterized in that, The specific implementation of the DECN includes: (a) Using graph convolutional networks to realize the interaction between entities and relations, entity x s Relationship x r The embedding is generated by linear transformation and combination operations to form a joint embedding x. o The formula is: x o =W e φ(x s ,x r ), in, It is a combination operator in the form of simple join, element-wise addition, element-wise multiplication, etc. The learnable parameter matrix; (b) Annotate the actual time of occurrence for each relationship. This provides temporal context and weights the aggregated features. Combined embeddings x o The weight α is calculated using the current timestamp t and the original timestamp t' as follows: α=Sigmoid(LeakyReLU(W t t+W t′ t'+W x x o )), (c) A weighted mechanism is used to capture the temporal relevance of each interaction. Considering historical context, more recent interactions are given higher importance. The core function is to reflect the continuous dynamic characteristics of nodes. This is achieved through a time decay factor β. t-t′ The neighbor information is aggregated with weight α to generate entity representation Xo, as shown in the formula: (d) Set the knowledge forgetting threshold Δt forget Entity e and relation r establish a valid time window sequence at t timestamps: Δt forget Within a valid time window, starting from time t, the importance of knowledge is gradually reduced until the knowledge forgetting threshold is exceeded, at which point outdated information is completely forgotten. This operation allows the model to retain the latest and most relevant knowledge, reflecting continuous dynamic features that change over time. (e) By modeling the hidden states of entities and relationships through recurrent neural networks, the hidden states of historical timestamps are fused with current features to capture continuous temporal features and generate a time-series dynamic representation for future predictions: Where σ is the activation function, b h ,b h′ This is a bias term.

3. The method according to claim 1, characterized in that, The specific implementation of CDIN includes: (a) Cross-dimensional features are extracted using convolutional kernels with an inflation rate of d. The joint features of entities, relationships, and timestamps are input into a multi-layer dilated convolutional network. The inflation rate of each convolutional kernel increases exponentially to cover feature interactions across different dimensions. The formula is as follows: Final output sequence Each dimension captures composite features from the other dimensions. (b) Combining extended convolution with residual blocks addresses the gradient vanishing and overfitting problems in deep network training, enhancing the model's ability to express cross-dimensional nonlinear relationships. The computation process for residual blocks is as follows: O=f(Z t +F(Z t )), Z t Indicates the input, F(Z) t The feature extraction from the residual blocks yields the D-dimensional predicted embedding o. The residual blocks consist of extended convolutional layers, weight normalization layers, ReLU layers, and dropout layers. To reconcile the difference between input and output widths, additional 1×1 convolutions are applied to maintain element-wise addition compatibility.

4. The method according to claim 1, characterized in that, The training objective of the method is multi-task joint optimization, and the total loss function is: (a) Transform the predicted embedding O into a vector form vec(O) with d dimensions, and associate it with each tail entity e. o We calculate the inner product to obtain the matching score, and then select the candidate entity with the highest score as the prediction result: ψ r (and s ,and o )=f(old(O)W)e。 (b) By minimizing the binary cross-entropy loss function, the true facts and negative samples can be distinguished, enabling more accurate prediction of inference results: Where q j =σ(ψ) r ), n j ∈{0,1} represents the label. By maximizing the probability of positive samples and minimizing the probability of negative samples, the model is able to distinguish the real interactions between entities and relationships.

Images