Time sequence knowledge graph reasoning method based on autoregressive conditional diffusion generation

Abstract

The invention discloses a time sequence knowledge graph reasoning method based on autoregression condition diffusion generation, and belongs to the technical field of artificial intelligence and dynamic knowledge graphs. According to the method, entity structure dependency in a single time snapshot is aggregated through a relation-aware graph neural network, time-gated loop unit autoregression modeling is combined to capture cross-snapshot time sequence evolution characteristics, and dynamic embedding is output; simulating future uncertainty through a forward diffusion process under the condition of the embedding, and generating future representation aligned with real distribution through reverse denoising iteration to cover a multivariate potential evolution path; and designing an entity and relation fusion gate, adaptively balancing historical experience and generating a prediction weight, and combining a ConvTransE decoder to realize high-precision link prediction. According to the method, adaptive capacity to uncertainty is enhanced through generative modeling, modeling comprehensiveness is improved through multi-granularity feature fusion, history and generated information are balanced through a dynamic gating mechanism, prediction flexibility and robustness are remarkably improved, and the method is suitable for sequential reasoning tasks in multiple fields such as event prediction and a dynamic recommendation system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and knowledge graph technology, in particular to a self-regressive conditional diffusion generated temporal knowledge graph reasoning method, which is suitable for predicting missing facts in dynamic knowledge graphs, such as event prediction, intelligent recommendation and dynamic network analysis. BACKGROUND

[0002] Temporal Knowledge Graph (TKG) marks the dynamic evolution of entity relationships through timestamps, but its high uncertainty and complexity lead to the following problems for traditional methods: static embedding methods such as TransE and DistMult complete missing link prediction by modeling the semantic association of entities and relationships, but completely ignore the time dimension and cannot capture temporal dependencies; existing methods (such as RE-NET, CyGNet) directly use historical information to predict the future, ignoring the generation process of future distribution, resulting in insufficient prediction flexibility; future events in the real world are influenced by multiple factors (such as policy changes, unexpected events), and need to model potential distribution rather than a single deterministic path. SUMMARY

[0003] The present application proposes a dynamic reasoning framework that combines self-regressive modeling and diffusion generation to address the three core problems of static modeling, simple temporal extrapolation, and uncertainty modeling in the prior art, generating multiple possible future distributions and providing a more flexible and robust evolution path for future prediction of temporal knowledge graphs. Through a relationship-aware graph neural network and a time-gated recurrent unit, the local structural dependencies of entities and relationships and the global temporal evolution are captured; based on the aforementioned dynamic embedding, the generation process of the future distribution is simulated through noise disturbance and iterative denoising, enhancing the adaptability to uncertainty; a gating mechanism is designed to adaptively fuse historical and generated representations, and a convolutional translation decoder is combined to output high-confidence predictions, which can be extended to event prediction, dynamic recommendation systems and other multi-domain temporal reasoning tasks.

[0004] The network proposed in the present application jointly computes and aggregates neighbor entity and relationship embeddings to the target entity, learns the semantic contribution of different relationships through linear transformation of weight matrices, aggregates neighbor information at each time, and captures the local interaction pattern of entities within a single time snapshot; an autoregressive mechanism is used to update the entity and relationship representations of multiple historical snapshots using update gates, reset gates, hidden states and other gating units, dynamically integrating new information and preserving historical information at each time step, thereby capturing the evolution of knowledge over time.

[0005] The network provided by the application approaches the real data distribution as much as possible through the distribution obtained by optimization learning, minimizes the cross entropy between the prediction distribution and the actual historical distribution, and learns the time evolution process of entity and relationship representation;In the forward diffusion process, the representation of knowledge is gradually added with Gaussian noise to simulate future uncertainty, and in the reverse denoising process, the historical embedding is used as a condition, combined with the static knowledge representation, the noise is iteratively removed through the noise prediction network, and the representation aligned with the real future is gradually generated.

[0006] The network provided by the application uses a convolution kernel to extract local interaction features by designing a gating mechanism, measures the matching degree of the target entity and the joint representation by inner product, dynamically adjusts the representation according to the relative importance of historical information and future prediction, adaptively balances historical experience and generated prediction, and avoids overfitting history or blindly relying on generated results;The prediction conditional probability is used to realize the temporal knowledge graph reasoning task, the more accurate entity and relationship representation is learned by minimizing the link prediction loss, and the prediction accuracy is improved.

[0007] Compared with the prior art, the application has the following obvious advantages:

[0008] (1) The application adds noise to simulate uncertainty by forward diffusion, iteratively generates multi-element future representation by reverse denoising, covers potential evolution paths, and ensures that the generated result conforms to logical evolution rather than single deterministic prediction, thereby enhancing the robustness to uncertainty and noise.

[0009] (2) The application aggregates neighbor information by using a graph neural network with relationship perception to accurately model complex interactions within a single time snapshot;The time gating cycle unit distinguishes long-term trends from short-term mutations and avoids noise interference.

[0010] The local structure dependence and global time evolution are captured by autoregressive modeling, avoiding the limitations of static models and simple time extrapolation.

[0011] (3) The contribution of historical patterns and generated representations is balanced by the gating mechanism to flexibly cope with mixed scenarios of stable periodicity and sudden changes, avoid overfitting history or blindly relying on generated results, and enhance the adaptability to noise and unseen events. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 The comparison chart of the existing method for temporal knowledge graph reasoning and the application shows the difference between direct prediction and generative prediction.

[0013] Figure 2This is an architecture diagram of the time-series knowledge graph reasoning method (ACDPM) based on autoregressive conditional diffusion generation proposed in this invention, including the interaction flow of the recurrent evolutionary network (REN), the diffusion probability generation network (DPGN), and the decoder module. Detailed Implementation

[0014] To facilitate understanding and use of the present invention by those skilled in the art, the specific processes and implementation methods of the present invention will be described in detail with reference to the accompanying drawings.

[0015] Figure 1 This paper demonstrates the differences between existing methods for temporal knowledge graph reasoning and the present invention. Temporal knowledge graphs consist of a series of static snapshots. Most existing reasoning methods typically utilize historical snapshot information to learn dynamic representations of entities and predict future facts, such as... Figure 1 As shown in (a). However, the inherent complexity of the real world means that past historical patterns cannot fully determine future states, and relying solely on historical semantic reasoning for the future has limitations. This invention aims to introduce historical structural information and temporal information from autoregressive modeling as conditions into the generative model to simulate the entity distribution at future time t+1 and achieve superior reasoning performance, such as... Figure 1 As shown in (b).

[0016] Figure 2 This paper demonstrates the flowchart of the time-series knowledge graph reasoning method based on autoregressive conditional diffusion generation proposed in this invention. It mainly consists of three parts: REN, DPGN, and a decoder. Input is a historical snapshot sequence. Each snapshot contains a quadruple (s, r, o, t), extracting the entity set ε and the relation set. For each time snapshot By aggregating neighbor information through a relation-aware graph neural network, an entity representation x is generated. o，t The generated entity and relation representations from m consecutive historical snapshots are input into a gating unit. An update gate controls the weighting of the current input and historical information, while a reset gate controls the influence of the previous hidden state on the current candidate activation. The final output is the embedding x of the entity and relation at time t. t and r t During the forward diffusion process, samples are drawn from the distribution of historical entity representations. By gradually adding noise, a pure noise representation that has been diffused over N steps is generated. In the reverse denoising process, from pure noise Initially, x is represented using entities. t Static knowledge representation x, based on a noise prediction network ∈ θ Gradually denoise to generate future representation Synchronous embedding of relations Perform the diffusion process to generate a relational representation of future timestamps. By designing a gating mechanism, the model dynamically adjusts its representation based on the relative importance of historical information and future predictions, effectively capturing the impact of dynamic evolution on the states of entities and relations, and generating the final representation x of entities and relations at the next time step. t+1 and r t+1 The ConvTransE decoder is used to predict the conditional probability p(o|s,r,t+1) and output the prediction result (s,r,query,t+1).

[0017] p(o|s,r,t+1)=σ(f(υec(M(x s,t+1 ,r t+1 ,t))W comv )x o,t+1 )

[0018] Where M(x) s,t+1 ,r t+1 ) represents the main representation x obtained through nonlinear convolution operation. s，t+1 The relation r is represented by t+1 The combination of W. conv This represents the learnable convolutional weight matrix. f is a non-linear activation function. vec(·) is the operation that transforms the feature map into a vector. Joint optimization link prediction loss. Entity generation loss and relation generation loss Ensuring that the prediction results align with the true labels and that the future representation approximates the true distribution effectively improves the adaptability of temporal knowledge graph reasoning to noise and unseen events.

[0019] Tables 1 and 2 show the results of experiments conducted on the ICEWS14, ICEWS18, WIKI, and GDELT datasets, comparing the performance of existing models in temporal knowledge graph inference. The baselines in the tables are divided into three categories: static methods, interpolation methods, and extrapolation methods. The experimental results are as follows: (1) Static methods (DistMult, ComplEx) perform poorly because they lack dynamic temporal information in the modeling. (2) Interpolation methods (TTransE, TA-DistMult) show some improvement over static methods, but still show lower performance compared to extrapolation inference methods, highlighting the importance of considering future knowledge trends when making accurate predictions. (3) Compared with other extrapolation methods, ACDPM has superior performance in the evaluation metric mean reciprocal rank (MRR), improving by 3%, 3.8%, 0.5%, and 4.3%, respectively.

[0020] Table 1

[0021]

[0022] Table 2

[0023]

[0024] This invention addresses key shortcomings of existing methods in areas such as uncertainty modeling, noise robustness, and scene adaptability through three core technologies: generative modeling of future distributions, multi-granularity spatiotemporal feature fusion, and dynamic adaptive decoding. Experimental data and theoretical design jointly demonstrate that ACDPM has significant advantages in dynamic knowledge graph reasoning tasks, providing a more flexible and reliable solution for complex time series prediction.

Claims

1. A time-series knowledge graph reasoning method based on autoregressive conditional diffusion generation, characterized in that, The method comprises the following steps: (1) Autoregressive dynamic modeling of multiple time snapshots of historical time series knowledge graph through recurrent evolution network (REN), extracting structural dependencies of entities and relationships within each time snapshot, and capturing time series evolution characteristics between adjacent time snapshots; (2) Based on the diffusion probability generation network (DPGN), the dynamic embedding output by the REN is used as the condition, the forward diffusion process is used to gradually add noise to simulate the potential distribution of the future time stamp, and the backward denoising process is used to iteratively generate entity and relationship representations aligned with the real future distribution; (3) Dynamically fuse historical representation and generated future representation through the decoder module, calculate the conditional probability of entities and relationships using the ConvTransE decoder, and complete the link prediction task of the future time stamp.

2. The method of claim 1, wherein, The REN comprises: (a) a relationship-aware graph neural network layer for aggregating neighbor information of entities within each time snapshot, specifically implemented by the following formula: wherein, and is an entity representation of the l-1 layer, and is a learnable weight matrix, and f(·) is a non-linear activation function. (b) a time-gated recurrent unit layer for modeling the time series evolution of multiple time snapshots, which includes updating gate ut, reset gate zt and candidate state ht', and finally outputs dynamic entity and relationship representations: x t = (1 - u t )h t-1 + u t h' t 3. The method of claim 1, wherein, The DPGN comprises: (a) Forward diffusion process: On historical entity and relation representations Gaussian noise is added step by step to generate a sequence of noisy representations The noise intensity is controlled by a hyperparameter βn, satisfying: (b) Reverse denoising process: based on historical condition information x t-1 and static knowledge representation x, iteratively remove noise through noise prediction network ∈θ(·) to gradually recover the representation of future timestamps which is calculated as: wherein z is a Gaussian noise; (c) Optimization objective: by minimizing the noise prediction error and the cross-entropy loss between the generated distribution and the true distribution, specifically including entity generation loss L gen and relation generation loss L rel , respectively defined as:

4. The method of claim 1, wherein, The decoder module fuses historical representation and generated representation through a dynamic gating mechanism, specifically including: (a) entity fusion gate: (b) relationship fusion gate: Wherein, σ(·) is the Sigmoid activation function, and is the element-wise product.

5. The method of claim 1, wherein, The training target of the method is multi-task joint optimization, and the total loss function is: where L task is the negative log-likelihood loss of the link prediction task, and λ1and λ2are hyperparameters used to balance the weights of the generation loss and the task loss.

Images