Time series data accelerated generation method based on diffusion model

Abstract

The invention relates to a time series data accelerated generation method based on a diffusion model, and aims to improve the calculation efficiency of time series data interpolation and maintain a high-quality generation effect at the same time. According to the method, accurate interpolation of missing values is realized through learning conditional probability distribution, and advanced features in adjacent denoising steps are cached and reused for time redundancy in a diffusion model denoising process, so that repeated calculation is reduced, and a generation process is accelerated. Specifically, in an up-and-down sampling structure of time series data, high-level features are selectively cached and reused in subsequent steps by analyzing correlation of adjacent time step features, while only low-level features are updated to maintain model precision. In addition, a 1: N cache updating strategy is adopted, and the calculation efficiency is further optimized. According to the method, the model can be dynamically compressed during operation without additional training, experiments show that the time sequence data generation speed is greatly increased on the premise that the interpolation quality is guaranteed, and an efficient and practical solution is provided for time sequence data processing.

Description

TECHNICAL FIELD

[0001] The application relates to a diffusion model-based time series data accelerated generation method and belongs to the technical field of pattern recognition in computer intelligence science. BACKGROUND

[0002] Time series data has wide application in the fields of financial market analysis, medical health monitoring, industrial process control, and Internet of Things device management, and is usually represented as a sequence of multi-dimensional features changing over time. However, missing values exist widely due to device failure, incomplete data collection, etc., and have become one of the major obstacles to data analysis. The commonly used traditional interpolation methods, such as linear interpolation, spline interpolation, autoregressive model and recurrent neural network, although perform well in simple time series data scenarios, are often difficult to effectively model the potential dependency structure when facing high-dimensional, long-sequence or highly nonlinear complex data, resulting in the interpolation results deviating from the true data distribution, and the accuracy and robustness are limited.

[0003] In recent years, the rapid development of generative models has provided new possibilities for time series data interpolation, among which the conditional diffusion model has attracted much attention due to its strong distribution modeling capability. The conditional diffusion model can generate the probability distribution of missing values based on observed data through a forward process to gradually add noise to the data and a backward process to recover the data from the noise, thereby generating high-quality interpolation results consistent with the original data distribution. However, there are several difficulties in applying diffusion models to time series data interpolation. First, time series data usually contains complex short-term fluctuations and long-term trends, and has highly nonlinear feature interaction relationships, while traditional diffusion models are mainly designed for image data and are difficult to directly model the dynamic evolution characteristics of time series data, so it is necessary to build a special time series coding architecture to capture multi-scale dependency relationships. Second, the conditional information is unevenly distributed in time and feature dimensions, and how to efficiently integrate these sparse conditions to improve the consistency of interpolation constitutes a key technical difficulty. In addition, time series data in different industries differ significantly in distribution characteristics, noise level and missing mechanism, and the model often needs to be adapted or retrained for specific tasks, which significantly limits its generalization ability and flexibility of practical application.

[0004] In terms of inference speed, the conditional diffusion model has a significant bottleneck, and its iterative generation process usually requires hundreds to thousands of steps for denoising, each of which relies on complex neural network operations. Especially when dealing with high-dimensional or long-sequence time series data, the inference time is very long, which is difficult to meet the real-time or large-scale application requirements. Although the intermediate features between adjacent time steps are highly correlated, existing methods do not fully exploit this redundancy, resulting in a lot of repeated calculations. In addition, existing acceleration strategies such as DDIM reduce the denoising steps, or reduce the complexity through model distillation, which can alleviate the inference burden to some extent, but often requires additional training cost, and may bring down the interpolation quality, and there is still a significant deficiency in the utilization of time series feature correlation.

[0005] Therefore, there is an urgent need for a time series interpolation method that balances generation quality and inference efficiency. The present application proposes an efficient time series data generation framework based on diffusion model, which innovatively introduces a feature caching mechanism to exploit the redundancy between adjacent time steps, significantly reducing the computational overhead. Without additional training, the method achieves dynamic acceleration of the conditional diffusion process, providing a practical and generalizable solution for efficient interpolation of complex time series data. SUMMARY

[0006] To solve the above problems, the present application proposes a time series data acceleration generation method based on diffusion model, which can use powerful diffusion model to generate high-quality data in the field of time series analysis, and significantly reduce the computational overhead through the feature caching mechanism.

[0007] To achieve the above purpose, the present application provides the following technical solutions:

[0008] A time series data acceleration generation method based on diffusion model, the method comprising the following steps:

[0009] Step 1, given a sample x0 containing missing values, the method aims to generate the interpolation target by using the observed values where and are different subsets of the sample space , the missing values are denoted as and the observed values are denoted as Therefore, the goal of probabilistic interpolation is to use the model distribution to approximate the true conditional distribution First, the method defines the basic forward diffusion and reverse diffusion process, which includes the following sub-steps:

[0010] Step 1-1, the forward diffusion process of the diffusion model is defined by the following Markov process, specifically as follows:

[0011]

[0012] where β t represents the noise level, x t the sample can be represented as: where x t can be represented as where

[0013] Step 1-2, the inverse diffusion process denoises x t to recover the original data x0, and is defined by the following Markov chain:

[0014]

[0015] where μ θ (x t ,t) and σ θ (x t ,t) can be specifically represented as:

[0016]

[0017] where ∈ θ represents a trainable denoising function network, and subsequently, μ θ (x t ,t) and σ θ (x t ,t) are denoted as μ DDPM (x t ,t, ∈ θ (x t ,t)) and σ DDPM (x t ,t).

[0018] In step 2, the conditional-based time series diffusion process in the field of time series analysis is defined as follows, specifically including:

[0019] Step 2-1, based on formula 2, the observed value x Then the conditional inverse diffusion based on time series data can be defined as:

[0020]

[0021] Step 2-2, based on formula (4) (5), the parameterization method therein is extended to the conditional case:

[0022]

[0023] Given the function ∈ θ and the data x0, through the inverse process in formula (8) and formula (9), x Sampling is performed. At the time of sampling, all observation values in x0 are set as conditional observations All missing values are set as the interpolation target When there is no conditional observation, the conditional model degenerates into an unconditional model and can also be used for data generation tasks.

[0024] Step 3, through upsampling and downsampling, the high-order features of the time series are saved, and the acceleration process of the deep cache is defined (as shown in Figure 2 ), which specifically includes:

[0025] Step 3-1, for time series data, use 1D convolution downsampling for downsampling process to obtain its low-dimensional features, which is defined as follows:

[0026]

[0027] Where represents the features after m times of downsampling on , K represents the size of the convolution kernel, and S represents the step length.

[0028] Step 3-2, for time series data, use 1D deconvolution for upsampling process to obtain its high-dimensional features, which is defined as follows:

[0029]

[0030] Step 3-3, cache the last sampled feature for time t, denoted as:

[0031]

[0032] Where represents the last sampled feature cache for time t.

[0033] Step 3-4, reuse the cached features with low-order features, thereby reducing repeated calculations while accelerating the generation process, the specific process is defined as:

[0034]

[0035] Step 4, the non-uniform acceleration sampling process of 1:N can be defined as:

[0036]

[0037]

[0038] Wherein the space linear function linear_space(s, n, e) is used to uniformly distribute n numbers between s and e (not including e), unique_int() converts the number to an integer and ensures the uniqueness of the time step in the sequence, c is the hyperparameter of the selected center time step, p represents the frequency domain, c e {10, 20, 50, 80, 100, 120, 150, 200}, p e {1.05, 1.1, 1.2, 1.3, 1.4, 1.5}.

[0039] Step 5, the overall training target of the time series condition diffusion model can be defined as:

[0040]

[0041] Through the above process, the training process can be completed, and the model can be dynamically compressed at runtime without additional training, while generating high-quality time series data.

[0042] A computer readable storage medium having stored thereon computer instructions, which, when executed by a processor, implement the steps of the diffusion model-based time series data accelerated generation method.

[0043] An electronic device comprising:

[0044] A processor;

[0045] A memory for storing processor-executable instructions;

[0046] Wherein the processor is configured to execute the instructions to implement the diffusion model-based time series data accelerated generation method.

[0047] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0048] (1) The scheme significantly improves the time series data interpolation effect. As can be seen from Table 1, on different missing rate settings of two real world data sets, the method has achieved performance improvement compared with other existing methods. This performance improvement is mainly because the method considers the conditional generation of observed data as a condition while using the diffusion model. This conditional generation method can consider the coupling relationship between observed values and missing values, and can better capture the time dependence in the observed values compared to existing diffusion model methods, thereby forming a more reasonable time series data generation result as shown in Table 1.

[0049] (2) greatly reduce redundant calculation, shorten the inference time, suitable for real-time application and long sequence high-dimensional data processing. Through the downsampling and upsampling method, the method can compress the redundant information in the data and retain the key information, while considering the time series data from a multi-scale perspective, and using the high-dimensional features and low-dimensional features in the time series data. By combining high-dimensional features and low-dimensional features, the method can focus on time series data from a global perspective and a local perspective, respectively, compared to other methods, so that the model can learn more rich time series dependencies.

[0050] (3) without additional training, dynamic acceleration can be realized, which is better than model distillation or DDIM method which needs to be retrained. Compared with other existing methods, the method combines 1:N non-uniform acceleration sampling, which is a pure mathematical transformation method and can be directly applied to any model without adjusting the model structure. By adjusting the parameters in the sampling, the position and range of dense sampling region can be flexibly controlled to adapt to different task requirements, providing efficient and practical solutions for time series data processing in financial, medical, industrial control and other fields. In addition, the low computational complexity of this method makes it suitable for resource-constrained environments such as edge devices. Compared with other existing methods without acceleration sampling module, the method can realize intelligent sampling through nonlinear mapping, balancing between computational efficiency and information retention, achieving good data generation effect while accelerating calculation. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 The method flowchart of the present application.

[0052] Figure 2 The time series data acceleration generation method provided by the present application is shown in the schematic diagram.

[0053] Figure 3 The time series generation effect visualization provided by the present application Figure 1 .

[0054] Figure 4 The time series generation effect visualization provided by the present application Figure 2 . DETAILED DESCRIPTION

[0055] The technical solutions provided by the present application will be described in detail below in combination with specific embodiments, and it should be understood that the following specific embodiments are only used to illustrate the present application and not to limit the scope of the present application.

[0056] Embodiment:

[0057] Experimental conditions: all experiments were implemented on NVIDIA RTX A6000 48G GPU using PyTorch.

[0058] Experiments are conducted on two datasets: (1) The healthcare dataset from the 2012 PhysioNet challenge, which contains 4000 clinical time series from intensive care units (ICUs) with 35 variables and 48 hours time span. The dataset is processed into time series with 48 time steps in the experiment. The processed dataset has about 80% missing values, and 10% / 50% / 90% of observations are randomly selected as ground truth on the test data. (2) The second one is the air quality dataset. 36 stations in Beijing are used in the experiment, and 36 consecutive time steps are set as a time series. The dataset has 13% missing values, and the missing pattern is not random. The overall evaluation metric uses root mean square error (RMSE) to measure.

[0059] As shown in Table 1, the generation effect of the model in this paper on the two datasets is better than that of other sota time series data generation models.

[0060] Table 1 Comparison of model generation performance results (RMSE)

[0061]

[0062] As Figure 3 shown, on the time steps that need to be interpolated, the data generated by the method is more stable and closer to the true value than other methods, while other models are prone to unstable jump phenomenon

[0063] As Figure 4 shown, on the time steps with large data changes, the interpolation results of the method are more in line with the change situation and change trend of the true value, while the interpolation of other models cannot capture reasonable change time steps.

[0064] It should be noted that the above embodiments are not intended to limit the scope of protection of the present application, and any equivalent transformations or substitutions made on the basis of the above technical solutions fall within the scope of protection of the claims of the present application.

Claims

1. A method for accelerating the generation of time-series data based on a diffusion model, characterized in that, The method includes the following steps: Step 1: Given a sample x0 containing missing values, define the ordinary forward diffusion process and the backward diffusion process. Step 2: Based on Step 1, add the observed values. Define conditional forward diffusion and backward diffusion processes. Step 3: By upsampling and downsampling, high-order features of the time series are saved, and the acceleration process of deep caching is defined. Step 4, define the sampling method for non-uniform sampling in the 1:N diffusion step. Step 5: Train the model using the training objective.

2. The method for accelerating the generation of time-series data based on a diffusion model according to claim 1, characterized in that, In step 1, the data processing procedure is as follows: Step 1, given a sample x0 containing missing values, use the observed values... To generate interpolation targets in and It is the sample space Different subsets of , the missing values ​​are denoted as The observed value is denoted as The goal of probabilistic interpolation is to use model distributions To approximate the true conditional distribution First, the method defines the basic forward and backward diffusion processes, which specifically include the following sub-steps: Step 1-1: The forward diffusion process of the diffusion model is defined by the following Markov process, as follows: Where β t x is a variable representing the noise level. t The sampling is represented as: in x t Represented as in Steps 1-2, the reverse diffusion process on x t Denoising is performed to recover the original data x0, and is defined by the following Markov chain: Where μ θ (x t ,t) and σ θ (x t Specifically, t) is represented as: Where ∈ θ This represents a trainable denoising function network, denoised subsequently by μ. θ (x t ,t) and σ θ (x t ,t) is μ DDPM (x t ,t,∈ θ (x t ,t)) and σ DDPM (x t ,t).

3. The method for accelerating the generation of time-series data based on a diffusion model according to claim 2, characterized in that, In step 2, the conditional temporal diffusion process is updated as follows: Step 2-1: Based on Formula 2, add the observed values. Then, conditional backdivergence based on time series data is defined as: Step 2-2, based on formulas (4) and (5), extend the parameterization method to the conditional case: Given function ∈ θ And the data x0, through the reverse process in formulas (8) and (9) During sampling, all observations in x0 are set as conditional observations. Set all missing values ​​as the imputation target When there are no conditional observations, the conditional model degenerates into an unconditional model, which is then used for the same data generation task.

4. The method for accelerating the generation of time-series data based on a diffusion model according to claim 1, characterized in that, In step 3, the acceleration process based on deep caching is defined as follows: Step 3-1: For time-series data, use 1D convolutional downsampling to perform a downsampling process to obtain its low-dimensional features, which are defined as follows: in Represents the The features are obtained after m downsampling operations, where K represents the kernel size and S represents the stride. Step 3-2: For time-series data, use 1D deconvolution for upsampling to obtain its high-dimensional features, defined as follows: Step 3-3: Cache the previous sampled feature for feature arrival time t, denoted as: in The buffer representing the previous sampled feature at time t. Steps 3-4 involve reusing cached features and low-order features to reduce redundant computations and accelerate the generation process. The specific process is defined as follows:

5. The method for accelerating the generation of time-series data based on a diffusion model according to claim 1, characterized in that, In step 4, the 1:N non-uniform accelerated sampling process is defined as follows: The linear function `linear_space(s,n,e)` is used to uniformly distribute n numbers between s and e (excluding e). `unique_int()` converts the numbers to integers and ensures the uniqueness of time steps in the sequence. `c` is the hyperparameter of the selected center time step, and `p` represents the frequency domain. `c∈{10,20,50,80,100,120,150,200}`, `p∈{1.05,1.1,1.2,1.3,1.4,1.5}`.

6. The method for accelerating the generation of time-series data based on a diffusion model according to claim 1, characterized in that, In step 5, the overall training objective is defined as: The training process can be completed through the above steps, and the model can be dynamically compressed at runtime without additional training, while generating high-quality time series data.

7. A computer-readable storage medium storing computer instructions thereon, characterized in that: When executed by a processor, the computer instructions implement the steps of the time-series data acceleration generation method based on the diffusion model as described in any one of claims 1-6.

8. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the time-series data acceleration generation method based on the diffusion model as described in any one of claims 1 to 6.

Images