Industrial time sequence generation method based on time sequence decomposition conditional diffusion model

Abstract

The invention provides an industrial time sequence generation method based on a time sequence decomposition conditional diffusion model. The method comprises the steps that industrial time sequence data are collected and preprocessed to obtain a data set; introducing conditional variables to construct a conditional diffusion model, and injecting noise through forward diffusion; designing a time sequence decomposition and reconstruction UNet module, performing feature extraction on noisy data and conditional variables to obtain an intermediate state, performing time sequence decomposition on the intermediate state, and performing feature reconstruction by using a decoder; the Sinkhorn distance is used as a regularization term to be fused into conditional noise prediction loss to construct a loss function for training; and randomly generating pure Gaussian noise, inputting the pure Gaussian noise into the trained TDA-CDM model, obtaining predicted noise of the current time step, calculating noisy data of the next time step, performing cyclic operation until a time sequence without noise is obtained, and accelerating sampling by using a denoising diffusion implicit model in circulation. According to the method, coexisting multi-scale dynamic features in the complex industrial MTS can be carefully and effectively captured and restored, and the comprehensive quality of generated data is remarkably improved.

Description

Technical Field

[0001] This invention relates to the technical field of industrial time series, and in particular to a method for generating industrial time series based on a diffusion model. Background Technology

[0002] Industrial rotary kilns, as core heat treatment equipment in industries such as cement, metallurgy, and chemicals, involve highly complex internal processes, encompassing multidimensional, nonlinear, and dynamic physicochemical reactions. Traditional mechanism-based modeling methods struggle to achieve precise monitoring and control. Therefore, data-driven soft measurement, process monitoring, and optimization control technologies have become crucial for improving rotary kiln operating efficiency and product quality.

[0003] However, data acquisition and application in complex industrial processes generally face challenges. First, data samples under critical operating conditions (such as failures or anomalies) are often scarce. Second, harsh operating conditions and sensor limitations lead to data quality issues such as noise and missing data. More importantly, data generated by processes such as rotary kilns are typically high-dimensional multivariate time series (MTS) with strong coupling between variables and containing complex time-series dynamics of multiple scales and diverse variations (such as the superposition of trends, periods, transient changes, and random noise). Traditional data-driven methods struggle to effectively capture and model this complexity.

[0004] To address data scarcity, generative models have been applied to data augmentation. Early Generative Adversarial Networks (GANs) and their variants have been used in industrial applications, but their inherent adversarial training mechanisms often lead to unstable training, difficulty in convergence, and a tendency for pattern collapse, making it difficult to generate sufficiently diverse and high-fidelity complex industrial MTS samples.

[0005] In recent years, Denoising Diffusion Probability Models (DDPMs) have emerged as a new generative paradigm, demonstrating superior training stability, sample quality, and diversity compared to GANs, showing great potential in fields such as industrial fault diagnosis, remaining service life prediction, and soft measurement. However, directly applying diffusion models to the generation of complex industrial MTSs such as rotary kilns still faces challenges: existing models (including some time-series diffusion models) often employ general network structures, making it difficult to effectively capture and decouple the coexisting multi-scale and complex dynamic features in MTSs; fine matching of high-dimensional data distributions still needs optimization; and the iterative sampling process results in slow generation speeds, making it difficult to meet the needs of real-time or near-real-time industrial applications.

[0006] Patent application number 202311315844.0 discloses a time series data prediction method and system based on a conditional diffusion model. The method includes: processing 10 days of dynamic time series data in a training set through a feature encoder to obtain high-dimensional dynamic time series data; constructing a conditional diffusion model with a conditional input Transformer as its backbone, using the 10 days of high-dimensional dynamic time series data in the training set as noisy data, and inputting the conditional features obtained by fusing the 10 days of high-dimensional dynamic time series data with static data into the conditional diffusion model for training; inputting the conditional features obtained by fusing the high-dimensional dynamic time series data (upgraded from the first dynamic time series data to be predicted) with the first static data into the conditional diffusion model, performing denoising operations, predicting the high-dimensional dynamic time series features, and inputting them into a feature decoder to obtain the predicted second dynamic time series data. This invention supports long-term dimensional modeling and constructs a yield prediction method with high recognition accuracy and good generalization. However, the conditional diffusion model in the aforementioned patent is difficult to effectively capture multi-scale dynamic features, cannot accurately match high-dimensional distributions, and has a slow sampling speed. Summary of the Invention

[0007] To overcome the challenges of existing technologies in generating complex industrial multivariate time series, and addressing the technical problems of existing diffusion models' inability to effectively capture multi-scale dynamic features, their inability to accurately match high-dimensional distributions, and their slow sampling speed, this invention proposes an industrial time series generation method based on a time-series decomposition conditional diffusion model. This method can generate high-fidelity, high-efficiency industrial multivariate time series, alleviate data scarcity problems, reduce data acquisition costs, improve the performance of downstream data-driven models (soft measurement, process monitoring, optimization control, fault diagnosis and prediction), and enhance model robustness and generalization ability.

[0008] To achieve the above objectives, the technical solution of this invention is as follows: an industrial time series generation method based on a time-series decomposition conditional diffusion model, comprising the following steps:

[0009] Step 1: Collect industrial time-series data and preprocess the industrial time-series data to obtain a dataset;

[0010] Step 2: Introduce conditional variables into the standard diffusion model to construct a conditional diffusion model. Inject noise into the industrial time series data in the training set through forward diffusion to obtain noisy data with different levels of noise injection.

[0011] Step 3: Design a temporal decomposition and reconstruction UNet module in the conditional denoising network. Use the encoder to extract features from the noisy data and conditional variables to obtain intermediate states. Perform temporal decomposition on the intermediate states to obtain trend terms and peak terms. Use a multilayer perceptron to process the trend terms and a convolutional self-attention mechanism to process the peak terms. Then, perform fusion to generate fused features. Use the decoder to reconstruct the features of the fused features to obtain the predicted noise at different time steps.

[0012] Step 4: Incorporate the Sinkhorn distance as a regularization term into the conditional noise prediction loss to construct a loss function. Use the loss function and the dataset to train the conditional denoising network to obtain the trained TDA-CDM model.

[0013] Step 5: Randomly generate pure Gaussian noise as input to the trained TDA-CDM model to obtain the noise of the current time step. Subtract the noise of the current time step from the pure Gaussian noise to obtain the noisy data of the next time step. Repeat the conditional diffusion operation until a noise-free time series is obtained. During the loop, the accelerated sampling strategy of the denoising diffusion implicit model is used.

[0014] Preferably, the mathematical form of the forward diffusion is:

[0015]

[0016] Where, x t and x t-1 These are the data states at time steps t and t-1, respectively; q(x t |x t-1 ) represents the given data state x t-1 At that time, data state x t The conditional probability distribution of β; t It is a predefined noise figure; Indicates data state x t Follow the mean The variance is β t I is a Gaussian distribution; I is the identity matrix;

[0017] The reverse generation process of the conditional diffusion model is as follows:

[0018] Where, p θ (x t-1 |x t c) represents the given data state x t When the condition variable c is used, the data state x t-1 The conditional probability distribution; μ θ (x t ,t,c) represents the mean predicted by the conditional denoising network; It is a predefined variance, derived from the noise figure β. t Decide.

[0019] Preferably, the method for generating the noisy data is as follows:

[0020] At each time step t, a Gaussian noise with the same dimension as the time series data is randomly generated. Where I is the identity matrix;

[0021] The calculated noise figure β t Multiply with the generated Gaussian noise ∈ and add to the given data state x. t-1 In the middle, the data status of the current time step is obtained.

[0022] The noise figure The parameter s is the offset, and T is the total time step;

[0023] The industrial time-series data includes seven process variables that are closely related to temperature, namely coal feed rate, kiln head temperature, kiln tail temperature, main motor load, and cooler load.

[0024] The preprocessing includes using sliding window filtering and outlier detection;

[0025] Use sintering temperature as the condition variable c.

[0026] Preferably, the temporal decomposition and reconstruction UNet module includes three encoders, three decoders, and a temporal decomposition module. The three encoders are connected sequentially and then connected to the temporal decomposition module. The temporal decomposition module is connected to the first decoder. The three decoders are connected sequentially. The first encoder and the third decoder are residually connected, and the second encoder and the first decoder are residually connected. Each encoder contains two sequentially arranged 1-D convolutional blocks, followed by a downsampling operation. Each decoder also contains two sequentially arranged 1-D convolutional blocks, followed by an upsampling operation.

[0027] The predicted noise is the input noisy sequence minus the noise reconstructed by the decoder.

[0028] Preferably, the processing flow of each 1-D convolutional block is as follows: the input signal is processed by feature extraction through the first 1-D convolutional layer and then subjected to group normalization transformation; the sinusoidal embedding condition variable c and the information of the current time step are used; then, the SiLU activation function is applied to introduce nonlinearity; the processed features are then passed through the second 1-D convolutional layer, and group normalization transformation and SiLU activation are performed again; the input signal is added to the group normalized output of the second 1-D convolutional layer through a residual connection.

[0029] Preferably, the method by which the timing decomposition module performs timing decomposition is as follows:

[0030] The intermediate state of the input is decomposed into two different sets of features using average pooling with padding and max pooling operations respectively: trend term X. trend and peak term X peak ;

[0031] Using a multilayer perceptron to analyze the trend term X trendBy performing nonlinear transformation and modeling, the trend characteristics are obtained: X′ trend =MLP(X trend ); where MLP() is the operation function of the multilayer perceptron;

[0032] A convolutional self-attention mechanism is used to focus on the peak term X. peak Modeling is performed to capture its fine-grained information: peak term X peak The process involves three independent 1-D convolutional layers, each generating a query matrix Q for the attention mechanism. conv Key matrix K conv Sum matrix V conv The characteristics of the peak features are obtained by applying a self-attention mechanism: Where, d k It is the scaling factor;

[0033] The trend feature X′ after decoupling modeling trend and peak feature X′ peak The concatenation is performed and integrated through a 1-D convolutional layer to generate fused features: F = Conv1d(Concat(X′) trend ,X′ peak ));

[0034] The fused feature F is fed into the decoder, where the time dimension is gradually recovered through upsampling and convolution operations. The feature is then reconstructed into the original time series size and number of channels through a convolutional layer at the end, thus completing the time series generation process.

[0035] Preferably, the loss function of the conditional diffusion model is L. diff =(1-ω)L c +ωL Sinkhorn ;

[0036] Among them, L c For conditional noise prediction loss, L Sinkhorn For Sinkhorn distance loss, ω∈[0,1] is a weight parameter.

[0037] Preferably, the conditional noise prediction loss L c =||∈-∈ θ (x t ,t,c)|| 2 Where ∈ represents real noise, ∈ θ (x t (t,c) represents the conditional denoising network passing through data state x t Noise predicted by time step t and condition vector c; 2 Indicates L2 loss;

[0038] The Sinkhorn distance loss L Sinkhorn =Sinkhorn(P gen ,P real ); where Sinkhorn() represents the Sinkhorn distance function, P gen To generate the sample distribution, P real This represents the true sample distribution.

[0039] Preferably, the denoising diffusion implicit model establishes a non-Markovian implicit generation process, enabling sampling to be completed with far fewer iterations than the original diffusion process by selecting a sparse subsequence of the original diffusion time step without significantly losing the quality of the generated samples; directly from the sample x at time step t... t and the noise predicted by the trained TDA-CDM model ∈ θ (x t Calculate the sample data x at time step s < t using the formula t,c). s ,and:

[0040]

[0041] Where c is the condition variable, x t This is a noisy sample at the current time step t, which is then back-diffused to gradually restore the sample from pure Gaussian noise. s The sample data to be calculated for the next time step s, ∈ θ (x t (t,c) represents the conditional denoising network passing through data state x t The noise predicted by time step t and condition vector c; and β is the parameter for variance accumulation during forward diffusion. i It is the i-th predefined noise figure.

[0042] Preferably, a sparse subsequence T containing K time steps is selected. K <T K-1 <...<T1<T0, where T0=T, T K =0, and K << T, where T represents the total time steps, and in each sampling step, the data is obtained from the sample data. Calculate sample data In each step of the calculation, the conditional denoising network uses the noisy sample data from the previous step. Current time step T k And the condition variable c is used as input to predict the corresponding noise.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. Enhanced multi-scale dynamic capture capability: By introducing a time series decomposition mechanism, the time series is decomposed into trend and peak terms, and different network structures (MLP and convolutional self-attention mechanism) are used for decoupled modeling, which can capture and restore the multi-scale dynamic features (such as long-term trends and transient changes) coexisting in complex industrial MTS more precisely and effectively, significantly improving the overall quality of the generated data.

[0045] 2. Optimize the matching degree of high-dimensional data distribution: The Sinkhorn distance is incorporated into the loss function as an additional regularization term to measure the difference between the generated data and the distribution of real high-dimensional industrial data. By optimizing this loss function, the generated samples are made to more closely match the distribution of real data in terms of overall statistical properties, thereby improving the fidelity and realism of the generated samples.

[0046] 3. Significantly improve sampling efficiency: The integrated DDIM (Denoising Diffusion Implicit Models) accelerated sampling strategy utilizes its non-Markov implicit generation process to significantly reduce the number of iterations required for generation without significantly losing the quality of generated samples. This greatly improves the model's sampling speed, making it more suitable for the real-time requirements of practical industrial applications.

[0047] 4. Enhance the model's adaptability to complex operating conditions: By introducing conditional variables, the model can generate highly consistent data samples based on specific industrial operating condition information, overcoming the problem of insufficient adaptability of traditional generative models when dealing with the scarcity of data for key operating conditions (such as faults or anomalies), and providing strong support for data augmentation, fault diagnosis, etc.

[0048] 5. High training stability: It inherits the training stability advantage of diffusion models compared to GANs, and avoids the problems of training instability, difficulty in convergence and mode collapse that are often caused by adversarial training mechanisms, making the model easier to train and obtain high-quality generation results. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart of the present invention.

[0051] Figure 2The frequency histogram of the data generated in this invention is shown in the following diagram: (a) is the distribution of feature 1, (b) is the distribution of feature 2, (c) is the distribution of feature 3, (d) is the distribution of feature 4, (e) is the distribution of feature 5, (f) is the distribution of feature 6, and (g) is the distribution of feature 7.

[0052] Figure 3 The normal distribution curves for the data generated in this invention are shown, where (a) is the standard distribution of feature 1, (b) is the standard distribution of feature 2, (c) is the standard distribution of feature 3, (d) is the standard distribution of feature 4, (e) is the standard distribution of feature 5, (f) is the standard distribution of feature 6, and (g) is the standard distribution of feature 7.

[0053] Figure 4 This is a visualization of the t-sne results of the data generated in this invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] like Figure 1 As shown, this invention proposes an industrial time series generation method based on Temporal Decomposition Conditional Diffusion Model (TDA-CDM). The TDA-CDM model introduces a temporal decomposition mechanism and utilizes different network structures (MLP and convolutional self-attention mechanism) to decouple and model the trend term and peak / complex components of the time series, so as to capture and restore the multi-scale dynamic features coexisting in complex industrial MTS more precisely and effectively. The Sinkhorn distance is incorporated into the loss function to optimize the matching degree between the generated data and the real high-dimensional industrial data distribution, thereby improving the fidelity of the generated samples. The DDIM accelerated sampling strategy is integrated to significantly reduce the number of iteration steps required for generation, greatly improve the sampling efficiency of the model, and make it more suitable for practical industrial application needs.

[0056] The core of this invention's framework lies in gradually injecting noise into real industrial multivariate time series data through a forward diffusion process until its distribution approximates a standard normal distribution. Subsequently, a conditional denoising network is constructed, taking noisy data, the current diffusion time step, and conditional information as input, with the goal of accurately predicting and removing the added noise. Within the conditional denoising network, a temporal decomposition and reconstruction UNet module is designed. This module performs temporal decomposition on intermediate time series features, separating trend terms from peak terms, and employs differentiated network structures for modeling and reconstruction: using a multilayer perceptron (MLP) to handle trend terms and a convolutional self-attention mechanism to handle peak terms. This design effectively captures and restores the complex multi-scale dynamic features in industrial time series. This invention introduces a composite loss function, combining traditional noise prediction loss and Sinkhorn distance to optimize the denoising network, thereby significantly improving the matching degree between the generated data and real high-dimensional industrial data in terms of overall distribution. Furthermore, in the sampling stage, this invention integrates an accelerated sampling strategy using a Denoising Diffusion Implicit Model (DDIM), enabling rapid data generation to meet the high-efficiency requirements of industrial scenarios. The specific implementation steps of this invention are as follows:

[0057] Step 1: Collect industrial time-series data and preprocess the industrial time-series data to obtain a dataset.

[0058] Seven process variables closely related to temperature were selected, including coal feed rate, kiln head temperature, kiln tail temperature, main motor load, and cooler load.

[0059] Industrial data is often affected by random noise and outliers caused by harsh process environments. Therefore, sliding window filtering and outlier detection are used for preprocessing.

[0060] The total length of the data is 7000 time points, and the data from the first 5600 time points is selected as the dataset.

[0061] Step 2: Introduce conditional variables into the standard diffusion model to construct a conditional diffusion model. Inject noise into the industrial multivariate time series data in the training set through forward diffusion to obtain a series of time series data at different times (degrees of noise injection).

[0062] In the standard diffusion model, the forward diffusion process involves gradually adding noise to the real data until it eventually approximates a standard normal distribution. The mathematical form is:

[0063]

[0064] Where, x t and x t-1 These are the data states at time steps t and t-1, respectively; q(x t |x t-1) represents the given data state x t-1 At that time, data state x t The conditional probability distribution of β. t It is a predefined noise figure, usually a small positive number, used to control the amount of noise added at each step; Indicates data state x t Follow the mean The variance is β t I is a Gaussian distribution; I is the identity matrix.

[0065] The reverse generation process involves starting with the noise distribution and gradually removing noise through the inverse learning process to recover the true data. The core formula for reverse generation is:

[0066]

[0067] Where, p θ (x t-1 |x t ) represents the given data state x t At that time, data state x t-1 The conditional probability distribution of μ. θ (x t (t) is the mean predicted by the conditional denoising network (whose input is x). t and time step t), It is a predefined variance. It is determined by the noise figure β t Decision made, and

[0068]

[0069] Among them, the intermediate variable α t =1-β t ,

[0070] To adapt to industrial MTS generation tasks, a conditional variable c is introduced to control the generation process. The inverse generation process of the conditional diffusion model is further extended as follows:

[0071]

[0072] Where, p θ (x t-1 |x t c) represents the given data state x t When c, the data state x t-1 The conditional probability distribution of μ. θ (x t (t,c) represents the mean predicted by the conditional denoising network (whose input is x). t , t and c).

[0073] Noise is added using a cosine modulation method, with a noise figure of [missing information]. The parameter s is an offset, set to s = 0.008. T is the total time step, set to T = 1000.

[0074] At each time step t, a Gaussian noise with the same dimension as the time series data is randomly generated. Where I is the identity matrix.

[0075] The calculated noise figure β t Multiply with the generated Gaussian noise ∈ and add to the original data x. t-1 In order to obtain the data at the current time step. The condition variable c has no effect on forward diffusion.

[0076] Objective function during training of the conditional diffusion model:

[0077]

[0078] Where ∈ is the real noise, ∈ θ This represents the noise predicted by the conditional diffusion model, and x0 is the original data. β i Let || represent the noise figure at time step i, || || 2 Indicates L2 loss. Data state. ∈ θ (x t (t,c) represents the noise predicted by the conditional denoising network (whose input is x). t , t and c).

[0079] By employing an objective function, the conditional diffusion model can generate data samples that are highly consistent with the actual scenario based on conditional information, thereby overcoming the problem of insufficient adaptability of traditional generative models to complex working conditions.

[0080] Step 3: Design a conditional denoising network, specifically the UNet module for time series decomposition and reconstruction, using an encoder to process the noisy data state x in the time series data. t The intermediate state is obtained by extracting features from the conditional information c. The intermediate state is then decomposed temporally to obtain the trend term and the peak term. The trend term is processed by a multilayer perceptron, and the peak term is processed by a convolutional self-attention mechanism. The two components are then fused to generate a fused feature. The fused feature is then reconstructed using a decoder to obtain the prediction noise at different time steps.

[0081] This invention proposes a novel network architecture based on the classic U-Net framework and optimized for time series generation tasks, aiming to effectively capture and reconstruct complex time series patterns. The noise is obtained by subtracting the reconstructed sequence from the noisy sequence. Since there are noisy time series at different time steps (noise intensities), the resulting data represents noise at different time steps.

[0082] The core of the temporal decomposition and reconstruction UNet module consists of three encoders and three decoders. The three encoders are connected sequentially and then connected to the temporal decomposition module, which in turn is connected to the first decoder. The three decoders are connected sequentially, with residual connections between the first and third encoders, and between the second and first encoders. Each encoder contains two sequentially arranged 1-D convolutional blocks followed by a downsampling operation to progressively reduce the temporal dimension and expand the receptive field, thereby capturing multi-scale features of the time series. Correspondingly, each decoder also contains two sequentially arranged 1-D convolutional blocks followed by an upsampling operation, designed to progressively recover the temporal dimension and perform refined feature reconstruction by combining high-level features from the encoder. The processing flow within each 1-D convolutional block is as follows: The input signal (pure Gaussian noise) first undergoes feature extraction through the first 1-D convolutional layer, followed by Group Normalization to stabilize the training process. After this, conditional information c (using sintering temperature as conditional information) and current time step information (using sinusoidal embedding) are introduced to provide the model with additional context-aware capabilities. Next, the SiLU activation function is applied to introduce nonlinearity. The processed features are then passed through a second 1-D convolutional layer, and subjected to GroupNorm (group normalization) transformation and SiLU activation again. Notably, information from the input signal is directly added to the GroupNorm output of the second 1-D convolutional layer through a residual connection. This helps to promote information flow and gradient propagation, mitigating the degradation problem in deep network training.

[0083] Following the encoder, to better learn complex temporal patterns within the context of time series generation, a time series decomposition technique was applied to the multivariate time series (MTS) data X output by the (third) encoder. Specifically, using padded average pooling and max pooling operations, the MTS data X was decomposed into two distinct sets of features: the trend term X... trend and peak term X peak Trend Item X trend The average pooling operation extracts the trend or smoothing component inherent in the input MTS data X, aiming to capture the data's inherent average trend.

[0084] X trend =AvgPool(X)

[0085] AvgPool() is the function for the flat pooling operation.

[0086] Peak term X peak Then, max pooling is used to extract peak terms from the input MTS data X to highlight the largest changes or significant local extrema in the data.

[0087] X peak =MaxPool(X)

[0088] MaxPool() is the function for the max pooling operation.

[0089] This decomposition strategy helps to break down complex time series patterns into potential pattern categories that are easier to model.

[0090] After time series decomposition, different structures are used to process these two types of features. For the trend term X... trend The nonlinear transformation and modeling were performed using a multilayer perceptron (MLP) to obtain the processed trend features:

[0091] X′ trend =MLP(X trend )

[0092] MLP() is the operation function of the multilayer perceptron. MLP can effectively learn complex nonlinear relationships in the trend term.

[0093] For the peak term X peak A convolutional self-attention mechanism is employed for modeling to capture its temporal dependency information. First, the peak feature X... peak The process involves three independent 1-D convolutional layers, each generating a query matrix Q for the attention mechanism. conv Key matrix K conv Sum matrix V conv The characteristics are as follows:

[0094] Q conv =Conv1d(X peak )

[0095] K conv =Conv1d(X peak )

[0096] V conv =Conv1d(X peak )

[0097] Here, Conv1d() represents the operation function of a 1-D convolutional layer.

[0098] Subsequently, a self-attention mechanism is applied, and its output is the modeled peak feature representation:

[0099]

[0100] Among them, Q conv ,K conv V conv The feature representation obtained through 1-D convolutional layers, d k It is a scaling factor. This attention mechanism can highlight important regions that are beneficial to feature representation by calculating the similarity between different time steps, thereby helping to learn fine-grained information and local dependencies of peak features.

[0101] Subsequently, the trend feature X′ after decoupling modeling will be... trend and peak feature X′ peak The concatenation is performed and then integrated through a 1-D convolutional layer to generate a fused feature F:

[0102] F = Conv1d(Concat(X′) trend ,X′ peak ))

[0103] This step aims to effectively combine the separately modeled trend and peak information. Finally, the fused features F are fed into the U-Net decoder layer, where the temporal dimension is gradually recovered through upsampling and convolution operations. Finally, a final convolutional layer reconstructs the features to the original time series size and number of channels, completing the time series generation process. This U-Net architecture, combining temporal decomposition and decoupled modeling, enhances the model's ability to capture and reconstruct complex temporal patterns by selectively processing different components of the time series.

[0104] Step 4: Incorporate the Sinkhorn distance as a regularization term into the conditional noise prediction loss to construct a loss function. Use the loss function and the dataset to train the conditional denoising network to obtain the trained TDA-CDM model.

[0105] In the task of generating high-dimensional, complex, multivariate time series data, the traditional conditional diffusion model loss, which relies solely on local noise reconstruction, has limitations and struggles to comprehensively capture the overall distribution and global structure of the generated data. To overcome this deficiency, this invention proposes a novel conditional diffusion loss function, which, compared to the traditional conditional noise prediction loss L... c Based on this, Sinkhorn distance is introduced as an additional regularization term L. SinkhornThe Sinkhorn distance is an efficient approximation of Earth Mover's Distance (Wasserstein distance), particularly suitable for measuring the differences between high-dimensional probability distributions, and can more accurately capture the overall distribution characteristics of the data.

[0106] Specifically, the regularization term L Sinkhorn Used to quantize and generate sample distribution P gen and the true sample distribution P real The difference between them can be expressed mathematically as follows:

[0107] L Sinkhorn =Sinkhorn(P gen ,P real )

[0108] Where Sinkhorn() represents the Sinkhorn distance function, generating the sample distribution P. gen The true sample distribution P is obtained by reverse generation or sampling of the data state. real It is obtained through the input data status.

[0109] By using the traditional conditional noise prediction loss L c Distance loss L from Sinkhorn Sinkhorn By performing weighted fusion, this invention constructs a novel conditional diffusion loss function L. diff Its definition is as follows:

[0110] L diff =(1-ω)L c +ωL Sinkhorn

[0111] Here, ω∈[0,1] is a weight parameter, experimentally determined to be 0.5, used to adjust the relative importance of the two losses.

[0112] Conditional diffusion loss function L diff By weighted combination of local noise reconstruction loss L c Matching loss L with global distribution Sinkhorn This provides multi-dimensional optimization objectives for model training. The introduction of the weight parameter ω allows the loss function to flexibly adjust the relative importance of standard noise prediction loss and distribution matching loss according to the needs of specific tasks, thereby achieving a balance between local accuracy and global realism, significantly enhancing the model's adaptability and generalization ability. By minimizing the Sinkhorn distance, the model enables the generated samples to more closely match the distribution of real data in terms of overall statistical properties, rather than just matching noise, thus significantly improving the realism and usability of the generated time series.

[0113] Step 5: Randomly generate pure Gaussian noise and input it into the trained TDA-CDM model to obtain the noise of the current time step. Subtract the noise of the current time step from the pure Gaussian noise to obtain the noisy data of the next time step. Repeat the operation until a noise-free time series is obtained. During the loop, the accelerated sampling strategy of the denoising diffusion implicit model is used.

[0114] In the industrial sector, the ability to generate high-quality, high-efficiency multivariate time-series data is urgently needed, which is crucial for data augmentation, anomaly detection, predictive model training, and system simulation. Existing conditional diffusion models have shown the potential to generate high-fidelity data consistent with given conditions. However, their standard sampling process based on the denoised diffusion probability model (DDPM) typically requires hundreds or even thousands of iterative denoising steps, resulting in slow sampling speeds, high computational resource consumption, and difficulty in meeting the stringent real-time or high-throughput requirements of industrial scenarios.

[0115] To overcome this computational bottleneck and significantly improve the sampling efficiency of the conditional diffusion model, making it more suitable for practical industrial applications, this invention proposes a method for generating conditional diffusion models that incorporates accelerated sampling techniques from Denoising Diffusion Implicit Models (DDIM).

[0116] Unlike the Markov inverse process of DDPM, DDIM establishes a non-Markov implicit generation process. This allows sampling to be completed with far fewer iterations than the original diffusion process, by selecting a sparse subsequence of the original diffusion time step, without significantly sacrificing the quality of the generated samples. The core of DDIM lies in its deterministic inverse sampling (the preceding inverse generation) update formula, which allows direct generation from sample x at time step t. t and the trained model ∈ θ Predicted noise ∈ θ (x t The sample x at time step s < t is deterministically calculated. s ,Right now:

[0117]

[0118] Where c represents the given condition information, i.e., the condition variable, and x t This is a noisy sample at the current time step t, which is then back-diffused to gradually restore the sample from pure Gaussian noise. s This is the sample to be computed at the next time step s (in the subsequence). ∈ θ (x t ,t,c) is the noise predicted at time step t by the trained U-Net model, and this prediction depends on the input condition variable c. and β is the parameter for variance accumulation during forward diffusion. i It is the predetermined variance scheduling, i.e., the i-th predefined noise figure.

[0119] By selecting a sparse subsequence T containing K time steps K <T K-1 <...<T1<T0, where T0=T, T K =0, and K << T, where T represents the total diffusion time step of 1000. This invention can use the above formula to extract samples in each sampling step. Calculate samples In each step of the calculation, the conditional denoising network All of them are noisy samples from the previous step. Current time step T k Given a condition variable c as input, the invention predicts the corresponding noise. In this way, the invention successfully skips many intermediate steps in the original diffusion process while ensuring the consistency between the generated samples and the condition information c.

[0120] To comprehensively evaluate the quality of the generated samples, this invention employs indicators including Kullback-Leibler (KL) divergence, Jensen-Shannon (JS) divergence, Maximum Mean Discrepancy (MMD), and average Pearson correlation coefficient to conduct a comprehensive evaluation from both distribution characteristics and correlation perspectives.

[0121] KL divergence is used to measure the difference between the generated data distribution Q(x) (i.e., the generated sample distribution P) and the generated data distribution Q(x). gen Approximate true data distribution P(x) (i.e., true sample distribution P) real The "information loss" at that time is defined as:

[0122]

[0123] The KL divergence reflects the relative entropy between two distributions; the smaller the value, the closer the generated distribution is to the actual data distribution.

[0124] JS divergence is a symmetric variant of KL divergence, exhibiting better robustness and interpretability. It is defined as:

[0125]

[0126] JS divergence is achieved by introducing an intermediate distribution. It avoids the numerical instability of KL divergence in some cases and is more suitable for measuring the difference between two distributions.

[0127] MMD is a non-parametric metric that compares generated data with real data in a reproducing kernel Hilbert space. The mean embeddings in the data are used to estimate the distributional differences between them, which are defined as:

[0128]

[0129] Where, μ P and μ Q These are the mean embeddings of real and generated data in the reproducing kernel Hilbert space, respectively. MMD can effectively capture the distributional differences in high-dimensional data and is suitable for comparing complex data distributions.

[0130] The average Pearson correlation coefficient is used to calculate the average correlation between the original data and the generated data, and it is defined as follows:

[0131]

[0132] Where, x ij and y ij These are the j-th feature points of the i-th sample in the original data and the generated data, respectively. and These are their means. This indicator reflects the overall correlation between the generated samples and the original samples in terms of feature dimensions; the closer the value is to 1, the higher the correlation.

[0133] Table 1. Quality Assessment of Generated Samples

[0134]

[0135] This invention comprehensively evaluates the similarity between the generated data and the original data using various charts and quantitative indicators. Figure 2 The frequency histogram shows the comparison of the numerical distribution between the generated data and the original data. Through the shape and height of the histogram, it can be intuitively observed that the generated data can basically reproduce the distribution characteristics of the original data. Figure 3 The normal distribution curve is used to assess whether the generated data conforms to the statistical characteristics of the original data. The shape and fit of the curve reflect the similarity of the generated data and the original data in terms of probability distribution. Figure 4 The t-SNE visualization results use dimensionality reduction techniques to map high-dimensional data to a two-dimensional space, intuitively demonstrating the clustering and similarity between the generated and original data in the feature space. It can be determined that the generated data retains the structure and patterns of the original data.

[0136] Table 1 provides a comprehensive sample quality assessment of the method of this invention and the traditional Conditional Diffusion Model (CDM), covering key indicators such as KL divergence, JS divergence, MMD (maximum mean difference), and average Pearson correlation. These indicators quantify and compare the distribution characteristics and correlations of the generated data and the original data. The assessment results show that this invention, through its time-series decomposition module, excels in multi-scale feature capture, significantly improving the overall quality of the generated data, especially in handling the long-term trends and short-term fluctuations of complex industrial time-series data. Furthermore, the loss function integrating Sinkhorn distance and conditional constraints further optimizes the generation quality, providing strong guarantees for high-quality generated data in multiple aspects, including distribution consistency, structural similarity, and operational adaptability.

[0137] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating industrial time series based on a time-series decomposition conditional diffusion model, characterized in that, The steps are as follows: Step 1: Collect industrial time-series data and preprocess the industrial time-series data to obtain a dataset; Step 2: Introduce conditional variables into the standard diffusion model to construct a conditional diffusion model. Inject noise into the industrial time series data in the training set through forward diffusion to obtain noisy data with different levels of noise injection. Step 3: Design a temporal decomposition and reconstruction UNet module in the conditional denoising network. Use the encoder to extract features from the noisy data and conditional variables to obtain intermediate states. Perform temporal decomposition on the intermediate states to obtain trend terms and peak terms. Use a multilayer perceptron to process the trend terms and a convolutional self-attention mechanism to process the peak terms. Then, perform fusion to generate fused features. Use the decoder to reconstruct the features of the fused features to obtain the predicted noise at different time steps. Step 4: Incorporate the Sinkhorn distance as a regularization term into the conditional noise prediction loss to construct a loss function. Use the loss function and the dataset to train the conditional denoising network to obtain the trained TDA-CDM model. Step 5: Randomly generate pure Gaussian noise as input to the trained TDA-CDM model to obtain the noise of the current time step. Subtract the noise of the current time step from the pure Gaussian noise to obtain the noisy data of the next time step. Repeat the conditional diffusion operation until a noise-free time series is obtained. During the loop, the accelerated sampling strategy of the denoising diffusion implicit model is used.

2. The industrial time series generation method based on the time-series decomposition conditional diffusion model according to claim 1, characterized in that, The mathematical form of the forward diffusion is: Where, x t and x t-1 These are the data states at time steps t and t-1, respectively; q(x t |x t-1 ) represents the given data state x t-1 At that time, data state x t The conditional probability distribution of β; t It is a predefined noise figure; Indicates data state x t Follow the mean The variance is β t I is a Gaussian distribution; I is the identity matrix; The reverse generation process of the conditional diffusion model is as follows: Where, p θ (x t-1 |x t c) represents the given data state x t When the condition variable c is used, the data state x t-1 The conditional probability distribution; μ θ (x t ,t,c) represents the mean predicted by the conditional denoising network; It is a predefined variance, derived from the noise figure β. t Decide.

3. The industrial time series generation method based on the time-series decomposition conditional diffusion model according to claim 2, characterized in that, The method for generating the noisy data is as follows: At each time step t, a Gaussian noise with the same dimension as the time series data is randomly generated. Where I is the identity matrix; The calculated noise figure β t Multiply with the generated Gaussian noise ∈ and add to the given data state x. t-1 In the middle, the data status of the current time step is obtained. The noise figure The parameter s is the offset, and T is the total time step; The industrial time-series data includes seven process variables that are closely related to temperature, namely coal feed rate, kiln head temperature, kiln tail temperature, main motor load, and cooler load. The preprocessing includes using sliding window filtering and outlier detection; Use sintering temperature as the condition variable c.

4. The industrial time series generation method based on the time-series decomposition conditional diffusion model according to any one of claims 1-3, characterized in that, The temporal decomposition and reconstruction UNet module includes three encoders, three decoders, and a temporal decomposition module. The three encoders are connected sequentially and then connected to the temporal decomposition module. The temporal decomposition module is connected to the first decoder. The three decoders are connected sequentially. The first encoder and the third decoder are residually connected, and the second encoder and the first decoder are residually connected. Each encoder contains two sequentially arranged 1-D convolutional blocks, followed by a downsampling operation. Each decoder also contains two sequentially arranged 1-D convolutional blocks, followed by an upsampling operation. The predicted noise is the input noisy sequence minus the noise reconstructed by the decoder.

5. The industrial time series generation method based on the time-series decomposition conditional diffusion model according to claim 4, characterized in that, The processing flow for each 1-D convolutional block is as follows: the input signal is processed by feature extraction through the first 1-D convolutional layer and then undergoes group normalization transformation; the sinusoidal embedding condition variable c and the information of the current time step are used; then, the SiLU activation function is applied to introduce nonlinearity; the processed features are then processed by the second 1-D convolutional layer, and group normalization transformation and SiLU activation are performed again; the input signal is then added to the group normalized output of the second 1-D convolutional layer through a residual connection.

6. The industrial time series generation method based on a time-series decomposition conditional diffusion model according to any one of claims 1-3 or 5, characterized in that, The method for time series decomposition by the time series decomposition module is as follows: The intermediate state of the input is decomposed into two different sets of features using average pooling with padding and max pooling operations respectively: trend term X. trend and peak term X peak ; Using a multilayer perceptron to analyze the trend term X trend By performing nonlinear transformation and modeling, the trend characteristics are obtained: X′ trend =MLP(X trend ); where MLP() is the operation function of the multilayer perceptron; A convolutional self-attention mechanism is used to focus on the peak term X. peak Modeling is performed to capture its fine-grained information: peak term X peak The process involves three independent 1-D convolutional layers, each generating a query matrix Q for the attention mechanism. conv Key matrix K conv Sum matrix V conv The characteristics of the peak features are obtained by applying a self-attention mechanism: Where, d k It is the scaling factor; The trend feature X′ after decoupling modeling trend and peak feature X′ peak The concatenation is performed and integrated through a 1-D convolutional layer to generate fused features: F = Conv1d(Concat(X′) trend ,X′ peak )); The fused feature F is fed into the decoder, where the time dimension is gradually recovered through upsampling and convolution operations. The feature is then reconstructed into the original time series size and number of channels through a convolutional layer at the end, thus completing the time series generation process.

7. The industrial time series generation method based on the time-series decomposition conditional diffusion model according to claim 6, characterized in that, The loss function of the conditional diffusion model is L diff =(1-ω)L c +ωL Sinkhorn ; Among them, L c For conditional noise prediction loss, L Sinkhorn For Sinkhorn distance loss, ω∈[0,1] is a weight parameter.

8. The industrial time series generation method based on the time-series decomposition conditional diffusion model according to claim 7, characterized in that, The conditional noise prediction loss L c =||∈-∈ θ (x t ,t,c)|| 2 Where ∈ represents real noise, ∈ θ (x t (t,c) represents the conditional denoising network passing through data state x t Noise predicted by time step t and condition vector c; 2 Indicates L2 loss; The Sinkhorn distance loss L Sinkhorn =Sinkhorn(P gen ,P real ); where Sinkhorn() represents the Sinkhorn distance function, P gen To generate the sample distribution, P real This represents the true sample distribution.

9. The method for generating industrial time series based on a time-series decomposition conditional diffusion model according to claim 7 or 8, characterized in that, The denoising diffusion implicit model establishes a non-Markovian implicit generation process, enabling sampling to be completed with far fewer iterations than the original diffusion process, without significantly sacrificing the quality of the generated samples, by selecting a sparse subsequence of the original diffusion time step; directly from the sample x at time step t... t and the noise predicted by the trained TDA-CDM model ∈ θ (x t Calculate the sample data x at time step s < t using the formula t,c). s ,and: Where c is the condition variable, x t This is a noisy sample at the current time step t, which is then back-diffused to gradually restore the sample from pure Gaussian noise. s The sample data to be calculated for the next time step s, ∈ θ (x t (t,c) represents the conditional denoising network passing through data state x t The noise predicted by time step t and condition vector c; and β is the parameter for variance accumulation during forward diffusion. i It is the i-th predefined noise figure.

10. The industrial time series generation method based on the time-series decomposition conditional diffusion model according to claim 9, characterized in that, Choose a sparse subsequence T containing K time steps. K <T K-1 <...<T1<T0, where T0=T, T K =0, and K << T, where T represents the total time steps, and in each sampling step, the data is obtained from the sample data. Calculate sample data In each step of the calculation, the conditional denoising network uses the noisy sample data from the previous step. Current time step T k And the condition variable c is used as input to predict the corresponding noise.

Images