Nonholonomic spatiotemporal data completion system based on diffusion model and training method

Abstract

The application provides a non-complete space-time data completion system and a training method based on a diffusion model, which comprises a bidirectional S4 network, a space-time dependence extraction network, a noise prediction network and a diffusion model; non-complete data is input into the non-complete space-time data completion system based on the diffusion model, pre-completion is performed through the bidirectional S4 network, and a pre-completion result is obtained; a space-time dependence relationship is extracted through the space-time dependence extraction network and input into the noise prediction network; the pre-completion result and the value of the missing position data after adding noise are combined and input into the noise prediction network, prediction noise is obtained, and a completion result is output after diffusion model processing. The application considers the space-time dependence relationship that is unchanged in the data itself during the diffusion model generation process and the dependence relationship that changes with the denoising step, better learns the distribution of the non-complete space-time data, has good generalization performance under different data missing rates, and effectively completes space-time data such as traffic and environment.

Description

Technical Field

[0001] This invention belongs to the field of data mining technology, specifically, it relates to a non-complete spatiotemporal data completion system and training method based on a diffusion model. Background Technology

[0002] Spatiotemporal data encompasses the complex and dynamic changes of the spatial distribution of objects over time. This type of data has wide applications in numerous fields, such as traffic management and environmental monitoring. However, in real-world applications, spatiotemporal data often contains missing values ​​due to various reasons, including sensor malfunctions and data transmission errors. These missing values ​​not only affect the completeness and accuracy of the data but may also mislead the analysis and decision-making based on it. Therefore, effectively handling missing values ​​in spatiotemporal data to ensure data quality and usability is an important issue in current research and practice.

[0003] In recent years, numerous methods have been proposed to complete missing spatiotemporal data. Among these methods, deep learning-based generative models, namely Generative Adversarial Networks (GANs) and Diffusion Models, have achieved excellent results. Although Diffusion Models have achieved comparable results to GANs when applied to spatiotemporal data completion, existing methods still have limitations.

[0004] Firstly, a key step in applying the diffusion model to spatiotemporal data completion is extracting spatiotemporal dependencies and using them as conditions to guide the generation of missing data by the diffusion model. Many existing works use linear interpolation methods along the time dimension to pre-complete the missing parts and then extract spatiotemporal dependencies from the pre-completed data. However, linear interpolation suffers from significant pre-completeness errors, causing a shift in the distribution of spatiotemporal data and resulting in inaccurate extracted spatiotemporal dependencies. Furthermore, the errors introduced by linear interpolation propagate continuously during the computation of the deep network, and the resulting error accumulation negatively impacts the overall model performance.

[0005] The patent document "A self-supervised trajectory completion method in general scenarios" (CN202410661443) discloses a method that combines deterministic models and conditional diffusion models and enhances the representation of location information through gridded encoding. Although the error caused by linear interpolation pre-completion is avoided through self-supervised training and gridded processing, it does not take into account the changes in the data separation and dependency during the denoising generation process. At the same time, the pre-completion information only includes missing location data, and the accuracy of the model learning data is low.

[0006] Existing methods treat the extracted spatiotemporal dependencies as invariant global prior information to guide diffusion models in learning the distribution of spatiotemporal data. However, in reality, for each step of the denoising generation process in the diffusion model, the distribution of the input data changes with the added noise, and the dependencies in the data also change. Existing methods ignore these changing dependencies.

[0007] The patent document "Training, Processing Method, Apparatus and Medium for Spatiotemporal Data Processing Model" (CN202211267605) discloses the use of a spatiotemporal completion network to obtain spatial and temporal correlations to estimate spatiotemporal data, and the use of a spatiotemporal hybrid completion network to jointly capture the spatiotemporal correlation information of urban spatiotemporal data. However, extracting the pre-completed results together with noisy data does not reflect changes caused by noise, and the accuracy of the output is still lacking.

[0008] Therefore, a new method for incomplete spatiotemporal data completion is needed to accurately complete the missing parts of the data and dynamically associate spatiotemporal dependencies with noise changes, thereby obtaining more accurate model results. Summary of the Invention

[0009] To address the shortcomings of existing technologies, the purpose of this invention is to provide a non-complete spatiotemporal data completion system and training method based on a diffusion model.

[0010] A training method for a non-complete spatiotemporal data completion system based on a diffusion model, provided by the present invention, includes:

[0011] The bidirectional S4 network steps are as follows: perform completion on the sequences of incomplete data input in both forward and reverse order to obtain pre-complete results, and monitor the difference between the two pre-complete results;

[0012] The spatiotemporal dependency extraction network steps are as follows: extract spatiotemporal dependencies from the pre-complete results;

[0013] The noise prediction network steps are as follows: After adding noise to the incomplete data and the missing data, the values ​​of the data at the missing positions are denoised and the missing data is output. The outputs of each residual layer are added together and output through a convolutional layer to predict the noise.

[0014] Diffusion model steps: Based on incomplete data, construct a diffusion model considering data missing information, pre-complete results, and spatiotemporal dependencies;

[0015] Loss function steps: Jointly train the loss function of the diffusion model and the bidirectional S4 network based on the error of the learning distribution and the error of pre-completion, and optimize the training.

[0016] Preferably, the diffusion modeling step involves constructing a diffusion model with a forward noise addition process and a reverse noise reduction process;

[0017] The positive noise addition process is represented as:

[0018]

[0019] The reverse denoising process is represented as:

[0020]

[0021] Where, X∈R l×d Represents complete data;

[0022] l represents the length of the sequence's time dimension;

[0023] d represents the number of features in the data at each time step;

[0024] M∈{0,1} l×d This indicates whether the data is missing; 1 indicates observable, and 0 indicates missing.

[0025] X M This represents the observable portion of the data;

[0026] X c This indicates the pre-completion result;

[0027] U represents a spatiotemporal dependency;

[0028] Indicates the missing part;

[0029] t represents the number of noise addition steps in the diffusion model;

[0030] T represents the total number of steps;

[0031] β represents the hyperparameter of the diffusion model;

[0032] This represents the value of the missing data after adding noise when the number of noise-adding steps is t;

[0033] This represents the value of the missing data after adding noise, corresponding to steps 1 to T.

[0034] Preferably, the diffusion model learns the conditional distribution. The distribution of the predicted noise and the data from the previous denoising step is used to obtain the distribution of the data for the current denoising step, and then sampling is performed based on the distribution to obtain the current denoised data.

[0035] The noise prediction network step is based on a residual neural network, and the output of the noise prediction network is used to... Denoising is performed on the input. The output is obtained after a noise reduction step. Missing data is generated after T steps. Generate the final prediction noise.

[0036] Preferably, in the noise prediction network step, the globally invariant spatiotemporal dependencies are handled using cross-attention, calculated as follows:

[0037] K = XW k Q = XW Q V = XW V ,

[0038] K and Q are computed using the extracted spatiotemporal dependencies U:

[0039] K = UW k Q = UW Q V = XW V ,

[0040] Different weights are assigned to the outputs of the two Transformers, with the weighting determined based on the actual input data. Let R1 represent the output of the self-attention layer and R2 represent the output of the cross-attention layer. These weights are input into a multilayer perceptron to obtain the gating weights G. The two weighted sums are then used to obtain the final output R of the attention layer.

[0041] G=σ(W(R1+R2)+b)R=G·R1+(1-G)·R2

[0042] Where K, Q, and V are the key, query, and value matrices of the Transformer, respectively;

[0043] W k W Q W V These represent the parameter matrices used to calculate K, Q, and V, respectively.

[0044] W represents the parameter matrix;

[0045] b represents the bias term of the parameter;

[0046] • The symbol represents element-wise multiplication of matrices.

[0047] Preferably, the output y of time step t in the bidirectional S4 network step is... t Represented as:

[0048] y t =CA t Bu0+CA t-1 Bu1+...+CBu t

[0049] Where u represents the input sequence;

[0050] Matrices A, B, and C all represent parameter matrices initialized using Hippo.

[0051] For the forward and backward networks, incomplete data are sequentially input into the S4 model, the feature dimension Transformer model, and the second S4 model, and the reconstruction error corresponding to the output of each model and the observable part of the data is calculated.

[0052] The missing locations are filled in by analyzing the data changes over time in both directions before and after the missing locations. The error is calculated and supervised. The mean of the pre-filling results from the forward and backward networks is the pre-filling result X. c .

[0053] Preferably, the spatiotemporal dependency extraction network step uses a combination of two attention mechanisms to fully utilize the spatiotemporal dependencies that are unchanging in the data itself and the spatiotemporal dependencies that change with the data distribution as the diffusion model denoising process changes. The extracted spatiotemporal dependencies are input into the residual network of the noise prediction network as global spatiotemporal conditions for different denoising steps of the conditional diffusion model.

[0054] The loss function step calculates the consistency error l between the outputs of the forward and backward models of the bidirectional S4 network corresponding to missing input data. c , Supervision c ;

[0055] The loss function of the diffusion model itself is compared with l f , l b , l c Summing these values ​​yields the loss function for the entire framework:

[0056]

[0057] Optimize the loss function and train the model;

[0058] Where M∈{0,1} l×d This indicates whether the data is missing; 1 indicates observable, and 0 indicates missing.

[0059] θ represents the model parameters;

[0060] ∈ represents the noise added to the diffusion model at the denoising step number t;

[0061] This represents the output of the noise prediction model;

[0062] X c This indicates the pre-completion result;

[0063] U represents a spatiotemporal dependency;

[0064] G represents the graph corresponding to the spatiotemporal data;

[0065] t represents the number of noise addition steps in the diffusion model;

[0066] λ represents the hyperparameter;

[0067] l f This represents the reconstruction error loss function of the forward model;

[0068] l b This represents the reconstruction error of the inverse pre-completion network.

[0069] The present invention provides a non-complete spatiotemporal data completion system based on a diffusion model, which is trained using the aforementioned training method for non-complete spatiotemporal data completion systems based on a diffusion model, and includes a bidirectional S4 network, a spatiotemporal dependency extraction network, a noise prediction network, and a diffusion model.

[0070] Incomplete data is processed by a bidirectional S4 network, and then spatiotemporal dependencies are extracted by a spatiotemporal dependency extraction network and noise is added by a diffusion model before being input into a noise prediction network.

[0071] The noise prediction network outputs the completed result after the predicted noise is processed by the diffusion model.

[0072] Preferably, the spatiotemporal dependent network uses a time-dimensional Transformer to extract temporal variation patterns and a graph convolutional network and a feature-dimensional Transformer to extract spatial relationships.

[0073] The bidirectional S4 network adopts a time-bidirectional structure and uses an S4 model initialized with a Hippo matrix.

[0074] The reconstruction error loss function of the forward model is calculated from the difference between the output of the supervised forward model and the observable portion of the input data. Reconstruction error of inverse pre-complete network

[0075] in, This indicates a positive pre-complete network output;

[0076] This indicates the output of the reverse pre-complete network;

[0077] X∈R l×d Represents complete data;

[0078] l represents the length of the sequence's time dimension;

[0079] d represents the number of features in the data at each time step;

[0080] M∈{0,1}l×d This indicates whether the data is missing; 1 indicates that it is observable, and 0 indicates that it is missing.

[0081] Preferably, the noise prediction network includes residual layers and convolutional layers.

[0082] The residual layer includes a graph convolutional network and a feature dimension Transformer, a structured state space S4 model, a time dimension Transformer, and a GCGRU.

[0083] The output is the sum of the input and output of the network layer.

[0084] According to the present invention, a method for incomplete spatiotemporal data completion based on a diffusion model is provided, which uses the aforementioned incomplete spatiotemporal data completion system based on a diffusion model for data completion, including:

[0085] Incomplete data is input into an incomplete spatiotemporal data completion system based on a diffusion model, and then pre-completed using a bidirectional S4 network to obtain the pre-completed result.

[0086] Spatiotemporal dependencies are extracted using a spatiotemporal dependency extraction network and then input into a noise prediction network.

[0087] The pre-complete result is combined with the noisy values ​​of the missing position data and input into the noise prediction network to obtain the predicted noise;

[0088] The predicted noise is processed by a diffusion model and then the output is completed.

[0089] Compared with the prior art, the present invention has the following beneficial effects:

[0090] 1. This invention adopts a spatiotemporal data completion framework based on a conditional diffusion model, which can accurately and effectively utilize the spatiotemporal dependencies in observable data as conditions to better learn the distribution of incomplete spatiotemporal data.

[0091] 2. This invention does not require linear interpolation pre-completion and can be directly applied to datasets with incomplete features. It considers the error of the learning distribution and the error of pre-completion to jointly train the diffusion model and the pre-completion network, and has good generalization performance under different data missing rates.

[0092] 3. The system structure of the present invention is reasonable, taking into account the spatiotemporal dependencies of the data itself that remain unchanged during the diffusion model generation process and the dependencies that change with the denoising step. It can effectively complete spatiotemporal data such as traffic and environment, is easy to use, and can achieve accurate completion results. Attached Figure Description

[0093] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0094] Figure 1 A schematic diagram of a non-complete spatiotemporal data completion system based on a diffusion model. Detailed Implementation

[0095] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0096] This invention provides a system for completing incomplete spatiotemporal data based on a diffusion model. For example... Figure 1 As shown, the training method for the incomplete spatiotemporal data completion system based on the diffusion model includes the following steps:

[0097] Step A: Using the missing spatiotemporal data as input, construct an architecture based on the diffusion model, learn the conditional distribution of the missing data based on the observable data, and generate the missing data according to the conditional distribution learned by the model.

[0098] Step A includes the following:

[0099] Step A1: Define incomplete data, that is, spatiotemporal data with missing information, and the symbolic representation of the missing information.

[0100] Specifically, X∈R l×d This represents the complete data, where l represents the length of the sequence's time dimension, and d represents the number of features at each time step. M∈{0,1} l×d This indicates whether the data is missing, with 1 representing observables and 0 representing missing data. X represents the missing data. M To represent the observable portion of the data, we use [a specific symbol]; otherwise, we use [another symbol] to represent the incomplete or missing portion. express.

[0101] Step A2: Define the conditional variables for the diffusion model to generate the conditional distribution: the missing data condition M, the pre-complete result, and the global spatiotemporal dependencies extracted using a spatiotemporal network.

[0102] Specifically, using X c This represents the pre-completion result, with U representing the spatiotemporal dependency.

[0103] Step A3: Construct a diffusion model with forward noise addition and reverse noise reduction processes.

[0104] The positive noise addition process is represented as:

[0105]

[0106] Where t represents the number of noise-adding steps in the diffusion model, T represents the total number of steps, and β is the hyperparameter of the diffusion model. This represents the value of the missing data after adding noise when the number of noise-adding steps is t. This represents the value of the missing data after adding noise, corresponding to steps 1 to T.

[0107] The reverse denoising process is represented as:

[0108]

[0109] Step A4: Use a diffusion model to learn the conditional distribution The missing data is generated by sampling based on this conditional distribution.

[0110] Specifically, the distribution of the data for the current denoising step is obtained based on the predicted noise and the distribution of the data from the previous denoising step, and then the current denoised data is obtained by sampling based on the distribution.

[0111] Step B: Construct the noise prediction network, i.e. the denoising network, required for the diffusion model implementation based on the residual neural network. Denoise the missing data according to the output of the noise prediction network to generate the final missing data.

[0112] Step B includes the following:

[0113] Construct a denoising network, also known as a noise prediction network, to denoise the input... The output is obtained after a noise reduction step. After T steps, the missing data can be generated. The purpose.

[0114] The denoising network employs a structure of multiple residual layers connected layer by layer. Specifically, the output of each residual layer is the sum of the input and output of that layer. Denoising is achieved based on the noise prediction network's prediction of the noise added at step t.

[0115]

[0116] Each residual layer contains a Graph Convolutional Network (GCN) and a Transformer for handling spatial information, a Structured State-Space Sequence Model (S4) and a Transformer for handling temporal information, and a GCGRU to progressively process both temporal and spatial information. Finally, the outputs of each residual layer are summed and passed through a convolutional layer to obtain the output of the noise prediction network.

[0117] In both the spatiotemporal dependency extraction network and the noise prediction network, graph convolutional networks (GCNs) are used to extract and process spatiotemporal dependencies based on graph structures. Compared with learning spatiotemporal dependencies based on Transformer structures, GCNs can better utilize prior information about known graph structures.

[0118] Step C: Considering the distributional shift between missing and complete data, to reduce its impact on the learning distribution of the diffusion model, a temporal bidirectional network is constructed as a pre-completion network to pre-complete the missing data. The output of the temporal bidirectional pre-completion network extracts spatiotemporal dependencies. Compared with directly extracting spatiotemporal dependencies from the missing data, the pre-completion result better reflects the spatiotemporal dependencies of the true data and can also reduce the impact of missing data on the model's learning distribution and the generation of error accumulation.

[0119] If this pre-completion is not performed and the data containing the missing parts is directly input into the diffusion model, it will affect the model's learning of the data distribution. It will also cause the error between the missing position and its original true value to accumulate as the model calculation continues, affecting the model's performance.

[0120] Step C includes the following:

[0121] Filling missing data points with zeros or using linear interpolation for pre-completion introduces significant errors, shifting the data distribution and impacting the model's learning of the distribution. Therefore, using neural networks for pre-completion of missing data allows the input to include more information about the missing locations, resulting in a more accurate model learning of the data distribution and more precise noise prediction.

[0122] The pre-completion network adopts a temporally bidirectional structure, performing completion based on both forward and reverse input sequences, and supervising to minimize the difference between the completion results. The network structure primarily uses the S4 model initialized with the Hippo matrix. This is because this model assigns equal weight to all past time steps, rather than prioritizing time steps closer to the current time. This minimizes the impact of missing recent time steps on the current time step's state, resulting in less information loss for the S4 model. The output y of the S4 model at time step t is... t Represented as:

[0123] y t =CA t Bu0+CA t-1 Bu1+...+CBu t

[0124] Where u is the input sequence, and matrices A, B, and C are parameter matrices initialized with Hippo.

[0125] For the forward network, incomplete data is sequentially input into the S4 model, the feature dimension Transformer model, and the second S4 model, and the reconstruction error corresponding to the output of each model and the observable part of the data is calculated; the same structure is used for the reverse network and the reconstruction error is calculated.

[0126] Since there are usually unmissable observable data before and after the missing data step, to utilize data before and after the missing position for pre-completion, calculating the reconstruction error of the observable portion allows the pre-completion network to learn the temporal changes of the entire sequence, thus performing more accurate pre-completion. For the outputs of the forward and backward networks corresponding to the missing parts, the missing position is filled in by observing the temporal changes of data before and after the missing position. The error between them is calculated and monitored to ensure the consistency of pre-completion between the forward and backward networks. Finally, the pre-completion result X of the entire model is obtained. c This is the average of the pre-completion results from the forward and reverse networks.

[0127] Step D: Construct a spatiotemporal dependency extraction network containing a spatiotemporal module to extract spatiotemporal dependencies in the data as conditions for the diffusion model to generate conditional distributions. This spatiotemporal network, using a combination of two attention mechanisms, fully leverages both the inherently invariant spatiotemporal dependencies of the data and the dependencies that change with the data distribution as the diffusion model denoises, providing effective guidance for the model to learn the spatiotemporal distribution.

[0128] Step D includes the following steps:

[0129] Step D1: Spatiotemporal network, or spatiotemporal dependency extraction network, from the pre-complete result X c The spatiotemporal dependency U is extracted from the network. Specifically, the spatiotemporal network uses a time-dimensional Transformer to extract temporal variation patterns, and a graph convolutional network (GCN) and a feature-dimensional Transformer to extract spatial relationships.

[0130] Step D2: Input the extracted spatiotemporal dependencies into the residual network of the noise prediction network as global spatiotemporal conditions for different denoising steps of the conditional diffusion model.

[0131] Step E: Within each residual layer, simultaneously consider the invariant spatiotemporal dependencies of the data itself and the dependencies of the data that change in each denoising step of the diffusion model.

[0132] Step E includes the following:

[0133] Within each residual layer, a Transformer is used as the primary structure for extracting and utilizing dependencies. These dependencies include both spatiotemporal dependencies that are inherently invariant to the spatiotemporal data, and intra-data dependencies that vary depending on the distribution of the residual layer input data as different noises are added in each denoising step.

[0134] Specifically, the noise-aware spatiotemporal network simultaneously simulates two types of dependencies, which are extracted using two different Transformer structures. The first Transformer calculates its attention score using the spatiotemporal prior information output by the spatiotemporal dependency extraction network; that is, its attention score represents the global spatiotemporal dependency. The second Transformer calculates its attention score using noisy data as input, therefore its attention score itself reflects the data distribution that can extract the variations caused by noise. The noise-free pre-complete result X is extracted separately. c The global spatiotemporal dependencies in the data, as well as the dependencies of the data distribution containing noise.

[0135] For globally invariant spatiotemporal dependencies, cross-attention is used. Typically, attention within a Transformer is calculated as follows:

[0136] K = XW k Q = XW Q V = XW V ,

[0137] Where K, Q, and V are the key, query, and value matrices of the Transformer, respectively, and W... k W Q W V These represent the parameter matrices used to calculate K, Q, and V, respectively.

[0138] For cross-attention, the extracted spatiotemporal dependencies U are used to compute K and Q:

[0139] K = UW k Q = UW Q V = XW V ,

[0140] The cross-attention output obtained by this calculation method can better reflect the spatiotemporal dependencies in the noisy data input to the residual layer.

[0141] Different weights are assigned to the outputs of the two Transformers. The proportion of the two dependencies (global spatiotemporal dependency and dependency in data that takes into account the changes in data distribution caused by noise) is determined according to the actual input data. The final extracted dependency is then used to predict and denoise noise in the diffusion model, which is more in line with the spatiotemporal data scenario of the diffusion model.

[0142] Let R1 represent the output of the self-attention layer and R2 represent the output of the cross-attention layer. Inputting both into a multilayer perceptron yields the gating weights G. Then, G is used to weight and sum the two outputs to obtain the final output R of the attention layer.

[0143] G=σ(W(R1+R2)+b)R=G·R1+(1-G)·R2

[0144] W is the parameter matrix, and b is the bias term of the parameters. The · symbol indicates element-wise matrix multiplication. The resulting R takes into account both global spatiotemporal dependencies and data dependencies that change with the denoising process of the diffusion model.

[0145] Step F: Reconstruct the loss function of the diffusion model, and jointly train the diffusion model and the pre-complete network by taking into account the error of the learning distribution and the error of pre-completeness.

[0146] Step F includes the following:

[0147] Step F1: Reconstruct the loss function of the entire framework, including the loss function of the diffusion model itself, the reconstruction error loss function of the bidirectional pre-complete network, and the consistency loss function of the pre-complete results in both directions.

[0148] The reconstruction error loss function of the forward model of the bidirectional pre-complete network is calculated by the difference between the output of the supervised forward model and the observable part of the input data, denoted by the symbol l. f express, This represents the output of the forward pre-completion network. The reconstruction error of the backward pre-completion network is represented by l. b express, This indicates the output of the reverse pre-complete network.

[0149] Step F2: Calculate the consistency error between the outputs of the forward and backward models corresponding to the missing input data, denoted by l. c express, Supervising this part of the error can ensure that the pre-complete results of the forward and backward models are consistent.

[0150] Step F3: Combine the loss function of the diffusion model itself with l f , l b , l cSumming these values ​​yields the loss function for the entire framework:

[0151]

[0152] Where θ represents the model parameters, and ∈ represents the noise added to the diffusion model at the denoising step number t. The output of the noise prediction model is G, which represents the graph corresponding to the spatiotemporal data, and λ is an adjustable hyperparameter that controls the weight of the loss function.

[0153] The model was trained by optimizing the loss function. After convergence, the model can complete incomplete spatiotemporal data inputs and has excellent generalization ability under different types of spatiotemporal data such as traffic and environment and different missing rates.

[0154] In further optimized examples, experiments were conducted on datasets with varying missing rates for both artificially and naturally missing data, including air quality, subway passenger flow, and electricity usage datasets. The results showed that the performance was significantly better than existing spatiotemporal data completion methods based on incomplete data, as indicated by the root mean square error and mean absolute error.

[0155] The incomplete spatiotemporal data completion system based on the diffusion model is trained using the training method for the incomplete spatiotemporal data completion system based on the diffusion model, and includes a bidirectional S4 network, a spatiotemporal dependency extraction network, a noise prediction network, and a diffusion model.

[0156] Incomplete data is processed by a bidirectional S4 network, and then spatiotemporal dependencies are extracted by a spatiotemporal dependency extraction network and noise is added by a diffusion model before being input into a noise prediction network.

[0157] The noise prediction network outputs the completed result after the predicted noise is processed by the diffusion model.

[0158] According to the present invention, a method for incomplete spatiotemporal data completion based on a diffusion model is provided, which uses the aforementioned incomplete spatiotemporal data completion system based on a diffusion model for data completion, including:

[0159] Incomplete data is input into an incomplete spatiotemporal data completion system based on a diffusion model, and then pre-completed using a bidirectional S4 network to obtain the pre-completed result.

[0160] Spatiotemporal dependencies are extracted using a spatiotemporal dependency extraction network and then input into a noise prediction network.

[0161] The pre-complete result is combined with the noisy values ​​of the missing position data and input into the noise prediction network to obtain the predicted noise;

[0162] The predicted noise is processed by a diffusion model and then the output is completed.

[0163] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A training method for an incomplete spatiotemporal data completion system based on a diffusion model, characterized in that, include: The bidirectional S4 network steps are as follows: perform completion on the sequences of incomplete data input in both forward and reverse order to obtain pre-complete results, and monitor the difference between the two pre-complete results; The spatiotemporal dependency extraction network steps are as follows: extract spatiotemporal dependencies from the pre-complete results; The noise prediction network steps are as follows: After adding noise to the incomplete data and the missing data, the values ​​of the data at the missing positions are denoised and the missing data is output. The outputs of each residual layer are added together and output through a convolutional layer to predict the noise. Diffusion model steps: Based on incomplete data, construct a diffusion model considering data missing information, pre-complete results, and spatiotemporal dependencies; Loss function steps: Jointly train the loss function of the diffusion model and the bidirectional S4 network based on the error of the learning distribution and the error of pre-completion, and optimize the training.

2. The training method for an incomplete spatiotemporal data completion system based on a diffusion model according to claim 1, characterized in that, The diffusion modeling step involves constructing a diffusion model with forward noise addition and reverse noise reduction processes. The positive noise addition process is represented as: The reverse denoising process is represented as: Where, X∈R l×d Represents complete data; l represents the length of the sequence's time dimension; d represents the number of features in the data at each time step; M∈{0,1} l×d This indicates whether the data is missing; 1 indicates observable, and 0 indicates missing. X M This represents the observable portion of the data; X c This indicates the pre-completion result; U represents a spatiotemporal dependency; Indicates the missing part; t represents the number of noise addition steps in the diffusion model; T represents the total number of steps; β represents the hyperparameter of the diffusion model; This represents the value of the missing data after adding noise when the number of noise-adding steps is t; This represents the value of the missing data after adding noise, corresponding to steps 1 to T.

3. The training method for an incomplete spatiotemporal data completion system based on a diffusion model according to claim 2, characterized in that, The diffusion model learns the conditional distribution. The distribution of the predicted noise and the data from the previous denoising step is used to obtain the distribution of the data for the current denoising step, and then sampling is performed based on the distribution to obtain the current denoised data; The noise prediction network step is based on a residual neural network, and the output of the noise prediction network is used to... Denoising is performed on the input. The output is obtained after a noise reduction step. Missing data is generated after T steps. Generate the final prediction noise.

4. The training method for an incomplete spatiotemporal data completion system based on a diffusion model according to claim 1, characterized in that, In the noise prediction network steps, the globally invariant spatiotemporal dependencies are handled using cross-attention, calculated as follows: K and Q are computed using the extracted spatiotemporal dependencies U: Different weights are assigned to the outputs of the two Transformers, with the weighting determined based on the actual input data. Let R1 represent the output of the self-attention layer and R2 represent the output of the cross-attention layer. These weights are input into a multilayer perceptron to obtain the gating weights G. The two weighted sums are then used to obtain the final output R of the attention layer. G=σ(W(R1+R2)+b)R=G·R1+(1-G)·R2 Where K, Q, and V are the key, query, and value matrices of the Transformer, respectively; W k W Q W V These represent the parameter matrices used to calculate K, Q, and V, respectively. W represents the parameter matrix; b represents the bias term of the parameter; • The symbol represents element-wise multiplication of matrices.

5. The training method for a non-complete spatiotemporal data completion system based on a diffusion model according to claim 1, characterized in that, The output y of time step t in the bidirectional S4 network step t Represented as: y t =CA t Bu0+CA t-1 Bu1+...+CBu t Where u represents the input sequence; Matrices A, B, and C all represent parameter matrices initialized with Hippo; For the forward and backward networks, incomplete data are sequentially input into the S4 model, the feature dimension Transformer model, and the second S4 model, and the reconstruction error corresponding to the output of each model and the observable part of the data is calculated. The missing locations are filled in by analyzing the data changes over time in both directions before and after the missing locations. The error is calculated and supervised. The mean of the pre-filling results from the forward and backward networks is the pre-filling result X. c .

6. The training method for an incomplete spatiotemporal data completion system based on a diffusion model according to claim 1, characterized in that, The spatiotemporal dependency extraction network step uses a combination of two attention mechanisms to fully utilize the spatiotemporal dependencies that are invariant in the data itself and the spatiotemporal dependencies that change with the data distribution as the diffusion model denoising process changes. The extracted spatiotemporal dependencies are input into the residual network of the noise prediction network as global spatiotemporal conditions for different denoising steps of the conditional diffusion model. The loss function step calculates the consistency error l between the outputs of the forward and backward models of the bidirectional S4 network corresponding to missing input data. c , Supervision c ; The loss function of the diffusion model itself is compared with l f , l b , l c Summing these values ​​yields the loss function for the entire framework: Optimize the loss function and train the model; Where M∈{0,1} l×d This indicates whether the data is missing; 1 indicates observable, and 0 indicates missing. θ represents the model parameters; ∈ represents the noise added to the diffusion model at the denoising step number t; This represents the output of the noise prediction model; X c This indicates the pre-completion result; U represents a spatiotemporal dependency; G represents the graph corresponding to the spatiotemporal data; t represents the number of noise addition steps in the diffusion model; λ represents the hyperparameter; l f This represents the reconstruction error loss function of the forward model; l b This represents the reconstruction error of the inverse pre-completion network.

7. A non-complete spatiotemporal data completion system based on a diffusion model, characterized in that, The data is trained using the training method for the incomplete spatiotemporal data completion system based on the diffusion model as described in any one of claims 1-6, and includes a bidirectional S4 network, a spatiotemporal dependency extraction network, a noise prediction network, and a diffusion model. Incomplete data is processed by a bidirectional S4 network, and then spatiotemporal dependencies are extracted by a spatiotemporal dependency extraction network and noise is added by a diffusion model before being input into a noise prediction network. The noise prediction network outputs the completed result after the predicted noise is processed by the diffusion model.

8. The incomplete spatiotemporal data completion system based on the diffusion model according to claim 7, characterized in that, The spatiotemporal dependent network uses a time-dimensional Transformer to extract temporal variation patterns and a graph convolutional network and a feature-dimensional Transformer to extract spatial relationships. The bidirectional S4 network adopts a time-bidirectional structure and uses an S4 model initialized with a Hippo matrix. The reconstruction error loss function of the forward model is calculated from the difference between the output of the supervised forward model and the observable portion of the input data. Reconstruction error of inverse pre-complete network in, This indicates a positive pre-complete network output; This indicates the output of the reverse pre-complete network; X∈R l×d Represents complete data; l represents the length of the sequence's time dimension; d represents the number of features in the data at each time step; M∈{0,1} l×d This indicates whether the data is missing; 1 indicates that it is observable, and 0 indicates that it is missing.

9. The incomplete spatiotemporal data completion system based on the diffusion model according to claim 7, characterized in that, The noise prediction network includes residual layers and convolutional layers; The residual layer includes a graph convolutional network and a feature dimension Transformer, a structured state space S4 model, a time dimension Transformer, and a GCGRU. The output is the sum of the input and output of the network layer.

10. A method for completing incomplete spatiotemporal data based on a diffusion model, characterized in that, Data completion is performed using the diffusion model-based incomplete spatiotemporal data completion system according to any one of claims 8-9, including: Incomplete data is input into an incomplete spatiotemporal data completion system based on a diffusion model, and then pre-completed using a bidirectional S4 network to obtain the pre-completed result. Spatiotemporal dependencies are extracted using a spatiotemporal dependency extraction network and then input into a noise prediction network. The pre-complete result is combined with the noisy values ​​of the missing position data and input into the noise prediction network to obtain the predicted noise; The predicted noise is processed by a diffusion model and then the output is completed.

Images