Denoising diffusion model-based multivariate time sequence missing variable prediction method, program, system and storage medium

Abstract

The invention discloses a multivariate time sequence missing variable prediction method, program and system based on a denoising diffusion model, and a storage medium, and belongs to the field of deep learning. The invention provides a novel model named DiffMissing based on DDPM. The novel model comprises a forward diffusion module, a context condition encoder module, a denoising network module and a prediction module. According to the model, two view angles of variables and time are fused, and the consistency of the interiors of the variables and the time can be more accurately ensured, so that information in data is more comprehensively understood, and the capability of predicting missing variables is remarkably improved. The DiffMissing prediction model provided by the invention has a remarkable technical effect, and can effectively improve the function of reconstructing the missing variable and the function of predicting the variable of the time sequence data.

Description

Technical Field

[0001] The present invention belongs to the field of computers, in particular to the field of deep learning in computers, and specifically relates to a method, program, system and storage medium for predicting missing variables in multivariate time series based on a denoising diffusion model. Background Art

[0002] Multivariate time series forecasting is widely used in various fields. Despite significant progress in this area in recent years, most existing methods assume that time series data are complete. However, due to sensor failures, data transmission interruptions, or human error, multivariate time series data collected in practice often suffer from unavoidable missing values. Even worse, under certain conditions, data for some variables may be completely missing throughout the observation period. Compared with randomly missing observations, the missing value pattern of completely missing variables is more complex and challenging, mainly reflected in the following two aspects: First, due to the complete missingness of some variables, the time series characteristics of these variables are completely unobservable, and traditional time series modeling methods cannot directly act on these data; second, the missingness of some variables breaks the internal consistency between variables in the multivariate time series data, which also interferes with the modeling and forecasting of other observed variables.

[0003] Currently, existing research on solving the problem of completely missing variables can be divided into the following two categories:

[0004] One approach is a two-stage approach that combines imputation and prediction. However, the effectiveness of the imputation stage depends heavily on the existing data distribution. When the proportion of missing data is large or the variables are missing for a long time, the imputation results may deviate significantly from the true distribution. This discrepancy directly affects the performance of the subsequent forecasting model and may even introduce considerable errors, further exacerbating forecast inaccuracies. Furthermore, the separation of the imputation and prediction processes leads to error accumulation between stages, hindering model performance and resulting in suboptimal solutions.

[0005] One approach is an end-to-end approach, which jointly models missing data with the prediction task. This approach uses the spatial relationship between observed and missing variables to complete the missing variables while modeling spatiotemporal dependencies. However, existing research ignores the conditional dependencies between observed and missing variables, which can compromise the authenticity and consistency of the completion results. Summary of the Invention

[0006] In order to solve the problem of distribution consistency between missing variables and observed variables in multivariate time series prediction with missing variables, the present invention proposes an end-to-end framework such as DiffMissing based on the denoising diffusion probability model DDPM. DiffMissing uses a denoising network to generate missing variables while optimizing the prediction module. The framework contains a contextual conditional encoder, which fuses the local and global contextual condition information of the observed variables through a cross-attention mechanism, and uses this to guide the denoising network to generate missing variables with consistent distribution. In the denoising network, adaptive sparse attention is designed to explicitly model the dynamic interaction pattern between the observed variables and the missing variables, so as to ensure the internal consistency between the variables.

[0007] The present invention provides a method for predicting missing variables in multivariate time series based on a denoising diffusion model:

[0008] A missing variable method for multivariate time series prediction based on denoising diffusion model:

[0009] Get the original multivariate time series with missing variables Add random sampling noise ∈ to obtain noisy data

[0010] The conditional information representation at the diffusion step t is obtained through the context conditional encoder The processed condition information is represented as Diffusion step t, noisy data Jointly generate intermediate hidden layer states

[0011] Through the intermediate hidden layer state Noise the data Restore to the original state, through the denoising network ∈ θ (·) Get the prediction noise ∈ θ ;

[0012] By predicting the noise ∈ θ Reverse speculation of intermediate states Predicting future values

[0013] Furthermore, we can obtain the original multivariate time series with missing variables Add random sampling noise ∈ to obtain noisy data Specifically: In the forward diffusion process, a noise variance scheduling parameter β is first set for each diffusion step t t , then define α t , extract a diffusion step t from the uniform distribution (1, T), randomly sample noise ∈ to simulate the noise in the data, and then use the noise ∈ to generate the noisy data

[0014] α t =1-β t ,β t ∈(0,1), t~(1,T)

[0015]

[0016] Where: T is the diffusion step length, is the original signal at diffusion step t In the noisy data The proportion of .

[0017] Furthermore, the conditional information representation on the diffusion step t is obtained through the context conditional encoder Specifically:

[0018] Step 1.1: Use one-dimensional convolution operation and bidirectional LSTM to obtain local features of observation variables and global context information

[0019]

[0020] in: is the mask matrix, is the observed variable,

[0021] Step 1.2: After the cross attention mechanism is fully integrated, the conditional information representation is output through the linear layer

[0022]

[0023] in: W Q ,W k ,W k is the learnable weight matrix, d is the dimension of the weight matrix, Represents the intermediate result of cross-attention fusion of local features and global context information.

[0024] Furthermore, the processed condition information is represented as Diffusion step t, noisy data Jointly generate intermediate hidden layer states Specifically:

[0025] Step 2.1: Through the diffusion step t, noise data Processing to generate hidden representation

[0026]

[0027] Step 2.2: Hidden Representation and the conditional information processed by one-dimensional convolution Add up to get the middle hidden layer state

[0028]

[0029] in: is the conditional information at diffusion step t form of expression.

[0030] Furthermore, through the intermediate hidden layer state Noise the data Restore to the original state and predict the noise ∈ θ , specifically:

[0031] Step 3.1: Use attention weights to capture the dynamic interaction patterns between observed variables and missing variables. For each missing variable and its associated observed variables Calculate the attention weight α between them ij , and adaptively update the missing variable generation value in a weighted summation manner

[0032]

[0033] in: denote observed variables and missing variables, respectively. LR(·) represents the LeakyReLU function, W a ,W b ,W c is the learnable weight matrix, represents the set of key observed variables with top-k importance for each specific missing variable.

[0034] Step 3.2: Generate values ​​for each updated missing variable Concatenate them to get the hidden output of sparse attention

[0035]

[0036] Step 3.3: Use GRU layers to embed temporal dependency constraints and inverse Transformer layers to capture variable dependencies, and Add the residuals to get Subsequently, two gating functions, tanh(·) and sigmoid(·), are used to filter out the feature information of each denoising step and embed it into the linear layer. The output layer combines the outputs of multiple residual blocks to obtain

[0037]

[0038] in: Represents the output of the GRU layer, Represents the output of the reverse Transformer layer, Represents the output of the l-th layer residual block in the diffusion step t.

[0039] Step 3.4: Generate the noisy data Diffusion step t and Gradually restore the original state of the data and predict the noise ∈ θ :

[0040]

[0041] Furthermore, by predicting the noise ∈ θ Reverse speculation of intermediate states Predicting future values Specifically: by predicting the noise ∈ θ Reverse calculation of the intermediate state The intermediate state After reconstructing the complete time series input, the prediction model TimeMixer based on the MLP structure predicts the future time steps

[0042]

[0043] in: It is a prediction network based on TimeMixer.

[0044] The present invention also provides a computer device / equipment / system, comprising a memory, a processor, and a computer program stored in the memory, wherein when the processor executes the computer program, the steps of the method for predicting missing variables in a multivariate time series based on a denoising diffusion model as described above are implemented.

[0045] The present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of any of the above-mentioned methods for predicting missing variables in a multivariate time series based on a denoising diffusion model.

[0046] The present invention also provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of any of the above-mentioned methods for predicting missing variables in a multivariate time series based on a denoising diffusion model.

[0047] Beneficial effects

[0048] The present invention proposes a missing variable model for multivariate time series prediction based on a denoising diffusion model. It introduces a conditional diffusion model approach and integrates local and global temporal information through temporal convolution. At the same time, it obtains observed variable information through a cross-attention mechanism, which can be used to assist the denoising network in generating missing variables with consistent distribution. In the denoising network, adaptive sparse attention is proposed to explicitly model the dynamic interaction pattern between observed variables and missing variables to ensure internal consistency between the variables. The DiffMissing prediction model proposed in the present invention has significant technical effects and can effectively improve the function of reconstructing missing variables and predicting variables of time series data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Conditional denoising diffusion probability model;

[0050] Figure 2 Context-conditional encoder module and denoising network module;

[0051] Figure 3 Ablation experiment results;

[0052] Figure 4 Hyperparameter sensitivity analysis results. DETAILED DESCRIPTION

[0053] In the field of deep learning models for time series analysis, this paper proposes a novel model called DiffMissing based on DDPM. This model comprises a forward diffusion module, a context-conditional encoder module, a denoising network module, and a prediction module. This model integrates both inter-variable and temporal perspectives, more accurately ensuring consistency across variables and across time, thereby providing a more comprehensive understanding of the information in the data and significantly improving the ability to predict missing variables.

[0054] Given a multivariate time series The entire historical data of some variables may be missing in . To simulate these missing variables, M variables are randomly masked from the N variables of historical observations, which means that the values ​​of these M variables are all 0 at the historical time step. On this basis, the mask matrix is ​​defined Indicates missing conditions, where m (n) =1 indicates a normal variable, m (n)=0 indicates missing variables. The future time step data that you want to predict is represented as The goal of multivariate time series forecasting with missing variables is to Get the output

[0055] The denoising diffusion probability model DDPM consists of a forward diffusion and a backward denoising process, the latter of which uses a learnable denoising network to gradually transform the data from a standard normal distribution to a target distribution, such as Figure 1 In the forward diffusion stage, the data samples Gaussian noise is added through the Markov chain. In the reverse denoising stage, the data recovery process is also modeled by the Markov chain. In multivariate time series forecasting with missing variables, the goal is to use the observed variables Conditional information to reconstruct missing variable values Therefore, the reverse denoising process of the conditional diffusion model can be defined as follows: where c represents the local and global contextual condition information extracted based on the observed variables.

[0056]

[0057] The technical details of the four modules of the DiffMissing model are as follows:

[0058] Forward diffusion module: The noise data sample at any given time step can be obtained by Obtain, among which α t =1-β t and In this process, missing variables are treated the same way as normal variables, and noise is injected step by step.

[0059] Contextual Condition Encoder Module: A cross-attention mechanism is introduced to fuse the local and global contextual information of the observed variables, ensuring that the generated conditional representation is instructive for the reconstruction of missing variables. Figure 2 As shown in Figure 2, the short-term patterns and local features of the observed variables in the time dimension are captured through one-dimensional convolution operations, while the global context is processed by introducing a bidirectional LSTM to capture the global temporal dependencies in both the forward and backward directions. Then, the cross-attention mechanism is used to fully integrate the local and global context information, and the conditional information representation is output through the linear layer. The conditional information from the diffusion step is then used in the denoising network to guide the network to capture the dependencies between observed and missing variables while maintaining internal consistency.

[0060] Denoising network module: This module accepts noise data Diffusion step t and contextual condition information To estimate the noise of missing variables, thus generating missing values, such as Figure 2 shown.

[0061] The specific process is as follows:

[0062] 1) The denoising network consists of multiple stacked residual blocks. For each residual block, the noise data Fuse with the embedding of diffusion step t to obtain the hidden representation Then combined with the conditional information processed by one-dimensional convolution Add together to get

[0063]

[0064] 2) In order to model the dependency between observed variables and missing variables, an adaptive sparse attention aggregation function is designed to capture the dynamic interaction patterns between observed variables and missing variables using attention weights. and missing variables You can use the mask matrix Respectively expressed as

[0065] 3) In order to ensure structural sparsity and reduce redundant interactions, we first obtain the top-k key observation variable set by simply calculating the Pearson correlation coefficient between variables. For each missing variable and its associated observed variables The attention weight α between them ij It is expressed as follows, where LR(·) represents the LeakyReLU function, W a ,W b is a learnable weight matrix.

[0066]

[0067] 4) The attention output is used to adaptively update the missing variable generation value in a weighted summation manner in each denoising step. This means that during the denoising process, the missing variable relies on the information of the key observed variables to generate its representation to ensure internal consistency between variables.

[0068]

[0069] 5) Obtaining hidden output of sparse attention Afterwards, the GRU layer is used to embed the time dependency constraints and the inverse Transformer layer is used to capture the variable dependencies and combine them with the contextual condition information. Add the residuals to obtain the hidden representation Subsequently, two gating functions, tanh(·) and sigmoid(·), are used to filter out the feature information of each denoising step and embed it into the linear layer.

[0070] 6) The output layer combines the outputs of multiple residual blocks to obtain Finally, the prediction noise ∈ is generated by a feed-forward network implemented with two fully connected layers with ReLU activation function. θ .

[0071] Prediction module: In order to achieve joint modeling of missing variable reconstruction and future value estimation, we first use the intermediate state of the diffusion step t Reconstruct the complete time series input and then introduce a simple but effective prediction model TimeMixer based on pure MLP structure to predict future time steps in Represents a prediction network based on TimeMixer.

[0072]

[0073] Evaluation indicators: root mean square error RMSE, mean absolute error MAE, mean absolute percentage error MAPE

[0074] To fully evaluate the effectiveness of our proposed method, we conducted the following experimental validation. The datasets were selected from four commonly used real-world datasets from different fields: the traffic speed datasets METRLA and PEMSBAY, the climate dataset Weather, and the traffic flow dataset PEMS08. The main characteristics of these datasets are shown in Table 1.

[0075] Table 1 Main characteristics of the dataset

[0076]

[0077] For comprehensive evaluation, we selected the following popular baseline methods for comparison. They include a two-stage model (diffusion model-based interpolation + MLP-based prediction): GRUI (Multivariate time series imputation with generative adversarial networks. https: / / papers.nips.cc / paper_files / paper / 2018 / file / 96b9bff013acedfb1d140579e2fbeb63-Paper.pdf) + STID (Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecasting. https: / / arxiv.org / pdf / 2208.05233), CSDI (Csdi: Conditional score-based diffusion models for probabilistic time series imputation. https: / / arxiv.org / pdf / 2107.03502) + TSMixer (TSMixer: An All-MLP Architecture for Time Series Forecasting.》https: / / arxiv.org / abs / 2303.06053), MTSCI(《Mtsci:A conditional diffusion model for multivariate time seriesconsistent imputation.》https: / / arxiv.org / pdf / 2408.05740)+HDMixer(《HDMixer:Hierarchical Dependency with Extendable Patch for Multivariate Time SeriesForecasting.》https: / / ojs.aaai.org / index.php / AAAI / article / view / 29155); and end-to-end model: GRU-D (《Recurrent neural networks for multivariate time series with missing values.》https: / / arxiv.org / pdf / 1606.01865), GCN-M (《Graph convolutionalnetworks for traffic forecasting with missing values.》https: / / arxiv.org / pdf / 2212.06419), TriD-MAE (《Trid-mae: A generic pretrained model for multivariate time series with missing values.》https: / / dl.acm.org / doi / pdf / 10.1145 / 3583780.3615097), BiTGraph(《Biased temporal convolution graph network for timeseries forecastingwith missing Values.》https: / / openreview.net / pdf?id=O9nZCwdGcG), GinAR(《Ginar: An end-to-end multivariate time series forecastingmodel suitable for variable variable missing.》https: / / arxiv.org / pdf / 2405.11333). .

[0078] All models are implemented in PyTorch and trained using an NVIDIA GeForce RTX 4090 GPU.

[0079] These baseline methods cover a variety of different time series forecasting techniques, providing a rich comparative perspective for the experiment. By comparing with these baseline methods, the performance of the present invention on different data sets can be more comprehensively evaluated. This experiment uses root mean square error (RMSE), mean absolute error (MAE) and mean absolute percentage error (MAPE) as metrics, and the results are shown in Table 2. The lower the values ​​of these indicators, the better the results. All data sets are uniformly divided into training set, validation set and test set in a ratio of 6:2:2. The historical review window of all methods is P=12, and the prediction step is Q=12. We randomly set missing variables at ratios of 25%, 50% and 90%, and the values ​​of missing variables are uniformly regarded as 0. For each missing rate, the experiment was repeated using 5 different random seeds, and the final result is the average of the repeated experiments.

[0080] Table 2 summarizes the performance comparison results of DiffMissing and other baselines on four datasets. The DiffMissing model performs well on multiple datasets, with average RMSE, MSE, and MAPE performance improvements of 8.40%, 9.03%, and 9.12%, respectively, compared to the best baseline. This is because it is able to leverage the contextual information of the observed variables to help generate missing variables with a consistent distribution, while the end-to-end joint optimization strategy alleviates the error accumulation problem. Notably, even at a high missing rate of 90%, the DiffMissing model is still able to predict future values ​​and shows greater superiority, highlighting the effectiveness of explicitly modeling internal consistency between variables.

[0081] Table 2 Performance of different methods on different datasets

[0082]

[0083]

[0084] In order to verify the effectiveness of the key components in DiffMissing, ablation experiments are conducted on two datasets, METRLA and Weather, under three different missing rates. Figure 3 As shown. A total of four groups of ablation experiments were designed: (i) w / o CCE: removing the context conditional encoder, (ii) w / o ASA: removing the adaptive sparse attention, (iii) w / o GiTL: removing the GRU and iTransformer layers, (iv) w / o TMixer: replacing the TimeMixer network with a linear layer as the prediction module. Analysis Figure 3 The results show that the most promising component is the adaptive sparse attention module. Particularly when the missing rate is high, removing the adaptive sparse attention significantly reduces prediction accuracy, indicating that attention weights facilitate the reconstruction of missing variables, providing important support for maintaining internal consistency between variables, thereby reducing prediction bias. Furthermore, using observed variables to provide supplementary conditional information significantly enhances the generation of missing variables. This suggests that the local and global temporal dependencies of observed variables can impose contextual consistency constraints on the reconstruction of missing variables, thereby mitigating estimation errors in the completion results. Furthermore, the lightweight prediction module based on the MLP is able to estimate future values ​​more accurately than a standard linear layer. In summary, the combination of all components produces the best results.

[0085] In addition, the sensitivity of DiffMissing to the following key hyperparameters is analyzed on the datasets PEMSBAY and PEMS08: the maximum noise level β t , the number of residual blocks L in the denoising network and the prediction loss weight coefficient λ. The results are as follows Figure 3As shown in the figure, it can be observed that for the diffusion model sampling noise level, selecting a moderate value is more beneficial for the noise prediction of the denoising network. Clearly, the best results are achieved when the number of layers is set to 3. This is because too many layers may lead to overfitting, while too few layers cannot adequately update the missing variable generation values ​​and capture temporal and variable dependencies. In particular, to balance the interpolation and prediction optimization learning objectives of DiffMissing during training, it is necessary to adjust the prediction loss weight coefficient within a small range to achieve better performance.

[0086] Through the above technical solutions, the DiffMissing model is able to integrate local and global contextual information of observed variables, thereby facilitating the generation of missing variables. Furthermore, adaptive sparse attention is used to update the missing variable reconstruction results, ensuring the inherent distribution consistency between the observed and missing variables. Furthermore, by directly using the recovered complete time series for forecasting, the joint modeling of missing variable reconstruction and future value estimation effectively reduces cumulative error.

[0087] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solution of the present invention will be supplemented below with reference to the accompanying drawings, so as to provide a clearer and more complete description of the solution.

[0088] The present invention proposes a DiffMissing model to reconstruct missing variables and predict future values, including a training process and an inference process.

[0089] Training algorithm process:

[0090] Input: Time series with missing variables Mask Matrix Diffusion step length T

[0091] Output: trained denoising network ∈ θ (·) and the prediction function

[0092] Step S1 forward diffusion to obtain noisy data

[0093] Draw a diffusion step t (current diffusion step timestamp) from the uniform distribution (1, T), and then randomly sample noise To simulate the noise in the data and generate noisy data

[0094]

[0095] Step S2: Obtain context information

[0096] One-dimensional convolution operation and bidirectional LSTM are used to obtain local and global context information in time, and after being fully integrated by the cross-attention mechanism, the conditional information representation is output through the linear layer. This process is expressed by the following formula. W Q ,W k ,W k is a learnable weight matrix.

[0097]

[0098] Step S3 obtains the hidden layer representation

[0099] Calculate the diffusion step t embedding and combine it with the noise data to get the hidden representation Hide the representation and the conditional information processed by one-dimensional convolution Add up to get the middle hidden layer state is the conditional information at diffusion step t form of expression.

[0100]

[0101] Step S4 uses the denoising network to predict the noise ∈ θ :

[0102] Step S4.1 Get the intermediate hidden state

[0103] Hidden layer representation Obtain hidden output after sparse attention After that, it passes through the GRU layer and the inverse Transformer layer in sequence, and is combined with the contextual condition information Add the residuals to get the intermediate hidden state in

[0104]

[0105]

[0106] Step S4.2 obtains the output of each residual block

[0107] Hidden intermediate state The output of a single residual block is obtained by embedding it into the linear layer through two gating functions tanh(·) and sigmoid(·);

[0108]

[0109] Step S4.3 generates prediction noise ∈ θ :

[0110] The output layer combines the outputs of multiple residual blocks to obtain After passing through the feedforward network, the prediction noise ∈ θ ;

[0111]

[0112] Step S5: Calculate the intermediate reconstruction state and predict future values

[0113] Using prediction noise ∈ θ , and reversely calculate the intermediate reconstruction state in is random noise that follows a standard normal distribution to further predict future values

[0114]

[0115] Step S6 calculates the total loss and updates the gradient

[0116] For each prediction noise ∈ θ and future time steps The denoising loss is calculated by the following formula and predicted losses And add them together using the weighting coefficient λ to get the total loss

[0117]

[0118] Repeat the above steps S1 to S6 until the model converges, thus obtaining the denoising network ∈ θ (·) and the prediction function

[0119] Reasoning algorithm flow:

[0120] The inference process uses the trained denoising function ∈ θ (·)right Perform stepwise sampling without updating the gradient of the prediction module. Instead, the prediction is made directly using the restored complete time series.

[0121] Input: time series with missing variables Mask Matrix Diffusion step size T, trained denoising network ∈ θ (·) and the prediction function

[0122] Output: predicted value for future time steps

[0123] Step S1: Obtain noise data Same as step S1 of the training algorithm process.

[0124] Step S2 calculates steps S2.1 to S2.3 for each diffusion step t=T,…,1 cycle:

[0125] Step S2.1: Obtain context information

[0126] Step S2.2 calculates the residual block joint output

[0127] Step S2.3 Sampling And order

[0128] Step S3 outputs the predicted value of the future time step

[0129] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A missing variable prediction method for multivariate time series based on a denoising diffusion model, characterized by: Get the original multivariate time series with missing variables Add random sampling noise ∈ to obtain noisy data The conditional information representation at the diffusion step t is obtained through the context conditional encoder The processed condition information is represented as Diffusion step t, noisy data Jointly generate intermediate hidden layer states Through the intermediate hidden layer state Noise the data Restore to the original state, through the denoising network ∈ θ (·) Get the prediction noise ∈ θ ; By predicting the noise ∈ θ Reverse speculation of intermediate states Predicting future values 2. The missing variable prediction method for multivariate time series based on a denoising diffusion model according to claim 1, characterized in that: Get the original multivariate time series with missing variables Add random sampling noise ∈ to obtain noisy data Specifically: In the forward diffusion process, a noise variance scheduling parameter β is first set for each diffusion step t t , then define α t , extract a diffusion step t from the uniform distribution (1, T), randomly sample noise ∈ to simulate the noise in the data, and then use the noise ∈ to generate the noisy data a t =1-b t ,b t ∈(0,1)、t~(1,T) Where: T is the diffusion step length, is the original signal at diffusion step t In the noisy data The proportion of .

3. The missing variable prediction method for multivariate time series based on a denoising diffusion model according to claim 2, characterized in that: The conditional information representation at the diffusion step t is obtained through the context conditional encoder Specifically: Step 1.1: Use one-dimensional convolution operation and bidirectional LSTM to obtain local features of observation variables and global context information in: is the mask matrix, is the observed variable, Step 1.2: After the cross attention mechanism is fully integrated, the conditional information representation is output through the linear layer in: W Q ,W k ,W k is the learnable weight matrix, d is the dimension of the weight matrix, Represents the intermediate result of cross-attention fusion of local features and global context information.

4. The missing variable prediction method for multivariate time series based on a denoising diffusion model according to claim 3, characterized in that: The processed condition information is represented as Diffusion step t, noisy data Jointly generate intermediate hidden layer states Specifically: Step 2.1: Through the diffusion step t, noise data Processing to generate hidden representation Step 2.2: Hidden Representation and the conditional information processed by one-dimensional convolution Add up to get the middle hidden layer state in: is the conditional information at diffusion step t form of expression.

5. The missing variable prediction method for multivariate time series based on a denoising diffusion model according to claim 4, characterized in that: Through the intermediate hidden layer state Noise the data Restore to the original state and predict the noise ∈ θ , specifically: Step 3.1: Use attention weights to capture the dynamic interaction patterns between observed variables and missing variables. For each missing variable and its associated observed variables Calculate the attention weight α between them ij , and adaptively update the missing variable generation value in a weighted summation manner in: denote observed variables and missing variables, respectively. LR(·) represents the LeakyReLU function, W a ,W b ,W c is the learnable weight matrix, represents the set of key observed variables with top-k importance for each specific missing variable; Step 3.2: Generate values ​​for each updated missing variable Concatenate them to get the hidden output of sparse attention Step 3.3: Use GRU layers to embed temporal dependency constraints and inverse Transformer layers to capture variable dependencies, and Add the residuals to get Subsequently, two gating functions, tanh(·) and sigmoid(·), are used to filter out the feature information of each denoising step and embed it into the linear layer. The output layer combines the outputs of multiple residual blocks to obtain in: Represents the output of the GRU layer, Represents the output of the reverse Transformer layer, represents the output result of the l-th layer residual block in the diffusion step t; Step 3.4: Generate the noisy data Diffusion step t and Gradually restore the original state of the data and predict the noise ∈ θ :

6. The missing variable prediction method for multivariate time series based on a denoising diffusion model according to claim 5, characterized in that: By predicting the noise ∈ θ Reverse speculation of intermediate states Predicting future values Specifically: by predicting the noise ∈ θ Reverse calculation of the intermediate state The intermediate state After reconstructing the complete time series input, the prediction model TimeMixer based on the MLP structure predicts the future time steps in: It is a prediction network based on TimeMixer.

7. A computer device / apparatus / system comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images