An adaptive dilated convolution industrial time series interpolation method and system

Abstract

The application provides an adaptive expansion convolution industrial time series interpolation method and system, and relates to industrial data processing technology. The dynamic adaptive expansion rate is calculated for each time step to realize dynamic matching of the convolution receptive field and the actual change period of the industrial time series, and the Huber loss function is used to switch to the absolute loss when the prediction error is large to suppress the gradient explosion, effectively solving the feature extraction mismatch problem caused by the fixed expansion rate and the training instability problem caused by the mean square error loss, and having the advantages that the actual change period of the industrial time series can be dynamically matched to improve the interpolation precision, and the gradient amplification effect of large errors is suppressed by the Huber loss function to ensure the training stability.

Description

Technical Field

[0001] This application relates to industrial data processing technology, and in particular to an adaptive dilated convolution industrial time series interpolation method and system. Background Technology

[0002] In the process of modern industrial systems transforming towards intelligence and digitalization, the refined management and control of industrial production highly relies on real-time and accurate operational data provided by underlying sensing networks. Industrial sensor networks continuously collect key process parameters such as temperature, pressure, and flow rate, providing a data foundation for process optimization, fault early warning, and predictive maintenance. However, the high-temperature, high-pressure, and strong electromagnetic interference environments of industrial sites easily lead to frequent sensor failures and network link interruptions in data transmission, resulting in varying degrees of data loss in industrial time series data. This severely restricts the effectiveness of data-driven methods in industrial scenarios. The problem of missing industrial time series data not only affects the reliability of real-time monitoring but may also lead to biases in subsequent analysis models, thereby causing a decline in production efficiency or safety hazards.

[0003] In the methodology for addressing missing data in industrial time series data, the implicit-explicit diffusion model-driven data imputation and restoration method demonstrates significant performance advantages. The implicit feature extraction module of this model employs a stacked dilated causal convolutional network, using a pre-defined fixed dilation rate set {1,2,4,8,16,32} to extract multi-scale periodic features of industrial process variables. The explicit feature extraction module, based on a structured state-space sequence model, designs the system's initial state matrix using high-order polynomial projection operator theory. This enables the model to effectively capture long-period patterns in industrial time series, compensating for the limitation of the implicit feature extraction module in extracting long-range dependencies in industrial data due to its limited receptive field. After concatenation of implicit and explicit features, a weighted fusion is performed via a gated fusion mechanism. Then, residual convolution and skip convolution are used to progressively aggregate the feature information. The aggregated data features are then used as conditional information to guide the gradual denoising process of the conditional diffusion model. After denoising, the final imputation and restoration result is obtained.

[0004] While implicit-explicit diffusion model-driven industrial data imputation and restoration methods have achieved some success in various industrial scenarios, they still have significant technical limitations. First, the implicit feature extraction module uses a fixed dilation rate set, limiting feature extraction to a preset discrete time scale. However, the actual cycles of industrial processes are often complex and variable, not perfectly matching the preset "minute-hour-day" time scales. When the periodicity of actual data fluctuations does not align with the fixed dilation rate, convolutional operations cannot effectively cover relevant time points, leading to the omission of key feature information. This, in turn, affects the model's accurate perception of data change patterns near missing locations, ultimately reducing the accuracy of data imputation and restoration. Second, existing methods only use the mean squared error loss function as the training objective. In cases of sudden changes in operating conditions or severe data loss, the prediction error may increase significantly. Due to the squared effect of the mean squared error on large errors, the loss value is amplified dramatically. This can easily trigger gradient explosion during backpropagation, causing instability in the training process, difficulty in converging the loss function, and preventing the model parameters from reaching their optimal state, thus impairing the overall performance of imputation and restoration.

[0005] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0006] This application provides an adaptive dilated convolutional industrial time series interpolation method and system. It has the advantages of dynamically matching the actual variation period of the industrial time series to improve interpolation accuracy, while simultaneously suppressing the gradient amplification effect of large errors through the Huber loss function to ensure training stability.

[0007] Firstly, the adaptive dilated convolution industrial time series interpolation method provided in this application adopts the following technical solution: An adaptive dilated convolutional industrial time series interpolation method includes: Acquire industrial time series data to be interpolated, which includes observations collected by multivariable industrial sensors at multiple sampling times, where some sampling times have missing data; Implicit feature extraction is performed on the industrial time series data. An adaptive dilation rate is dynamically calculated for each time step based on a temporal self-attention mechanism, and dilated causal convolution is performed according to the adaptive dilation rate to generate implicit feature representations. Explicit feature extraction is performed on the industrial time series data, and the long-range temporal dependence of the industrial time series data is modeled based on the structured state-space model to generate explicit feature representations. Based on the gating fusion mechanism, the implicit feature representation and the explicit feature representation are weighted and fused to generate a fused feature representation; The fused feature representation is input as conditional information into the conditional diffusion model. Data estimates for missing locations are generated through iterative denoising, and the complete industrial time series after interpolation and repair is output. The conditional diffusion model is trained using the Huber loss function. With a preset threshold τ as the boundary, MSE is used to calculate the loss when the prediction error is less than τ, and MAE is used to calculate the loss when the prediction error is greater than τ.

[0008] Optionally, dynamically calculating the adaptive expansion rate for each time step based on the temporal self-attention mechanism includes: The input feature matrix is ​​linearly transformed by the learnable projection matrix to generate the query vector and the key vector. The attention score is obtained by scaling the dot product of the query vector and the key vector, and the attention weight is generated by softmax normalization. The proposed expansion rate is obtained by weighting the candidate offset distances with the attention weights. A sigmoid gating module is introduced to perform bounded modulation on the proposed expansion rate using the original input features at the current time step, thereby generating the adaptive expansion rate.

[0009] Optionally, performing dilated causal convolution according to the adaptive dilation rate includes: When the adaptive dilation rate is non-integer, linear interpolation is used for the convolution sampling position, and adjacent integer time steps are taken and weighted by the fractional part to obtain the feature value of the non-integer position. The adaptive expansion rate is calculated independently at each time step, so that each time step has an independent expansion rate, thereby achieving dynamic matching between the convolutional receptive field and the actual change cycle of the industrial time series.

[0010] Optionally, modeling the long-range temporal dependencies of the industrial time series data based on a structured state-space model includes: The state transition and output response of an industrial time series are described by a state-space equation, which includes state transition parameters, input parameters, and output parameters. The state transition parameters are discretized to generate discrete state transition parameters, and a convolution kernel is constructed. Based on the direct convolution operation between the convolution kernel and the input sequence, the explicit feature representation is output without the need to iterate through the hidden state step by step. The state matrix of the state space equation is initialized based on a high-order polynomial projection operator, which makes the computational cost of the model increase linearly with the sequence length when processing long input sequences.

[0011] Optionally, the weighted fusion of the implicit feature representation and the explicit feature representation based on the gating fusion mechanism includes: The implicit feature representation is concatenated with the explicit feature representation; The concatenated features are subjected to nonlinear transformation by the tanh function and then gating weights are generated by the sigmoid function. The nonlinear transformation result is multiplied element-wise by the gating weights to generate the fused feature representation.

[0012] Optionally, the fused feature representation is input as conditional information into the conditional diffusion model, and the generation of data estimates for missing locations through iterative denoising includes: During the forward process, Gaussian noise is gradually added to the industrial time series data according to the preset noise variance scheduling, so that the data distribution approaches the standard Gaussian distribution; In the reverse process, based on the fused feature representation as conditional information, the noise component added in the forward process is approximately estimated through a neural network, and denoising is performed step by step; The denoised result is used as the data estimate for the missing position, and merged with the observations for the non-missing positions to output the complete industrial time series after interpolation and repair.

[0013] Optionally, the Huber loss function calculates the loss only at the locations where data is missing, and the specific form of the Huber loss function is as follows: When the prediction error is less than the preset threshold τ, the loss value is calculated using squared loss. When the prediction error is greater than or equal to the preset threshold τ, the loss value is calculated using absolute loss. At this time, the backpropagation gradient is always constant and no longer increases linearly with the increase of error. The preset threshold τ is set to the 75% to 90% percentile of the error distribution of missing locations in the training set.

[0014] Optionally, the step of inputting the fused feature representation as conditional information into the conditional diffusion model further includes: The fused feature representation is concatenated with the gated fused feature and then refined layer by layer through multiple residual convolution modules. The intermediate features output by each residual convolution module are transformed to the same dimension by skip convolution and then accumulated to obtain the skip features; The jump feature is used as conditional information, concatenated with the position encoding feature of the current diffusion step, and then input into the conditional diffusion model to generate differentiated noise predictions.

[0015] Secondly, this application provides an adaptive dilated convolutional industrial time series interpolation system that performs the method described above, including: The data acquisition module is used to acquire industrial time series data to be interpolated. The industrial time series data includes observations collected by multivariable industrial sensors at multiple sampling times, where some sampling times have missing data. The implicit extraction module is used to perform implicit feature extraction on the industrial time series data. It dynamically calculates the adaptive dilation rate for each time step based on the temporal self-attention mechanism, and performs dilated causal convolution according to the adaptive dilation rate to generate implicit feature representations. The explicit extraction module is used to perform explicit feature extraction on the industrial time series data, and to model the long-range temporal dependence of the industrial time series data based on the structured state space model to generate explicit feature representations. The fusion module is used to perform weighted fusion of the implicit feature representation and the explicit feature representation based on a gated fusion mechanism to generate a fused feature representation; The output module is used to input the fused feature representation as conditional information into the conditional diffusion model, generate data estimates for missing positions through iterative denoising, and output the complete industrial time series after interpolation and repair.

[0016] In summary, this application achieves dynamic matching between the convolutional receptive field and the actual change cycle of the industrial time series by dynamically calculating the adaptive dilation rate for each time step. It also employs the Huber loss function to switch to absolute loss when the prediction error is large in order to suppress gradient explosion. This effectively solves the problem of feature extraction mismatch caused by fixed dilation rate and the training instability caused by mean square error loss. It has the advantages of being able to dynamically match the actual change cycle of the industrial time series to improve interpolation accuracy, while suppressing the gradient amplification effect of large errors through the Huber loss function to ensure training stability. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the first embodiment of the adaptive dilated convolution industrial time series interpolation method of this application; Figure 2 This is a structural block diagram of the first embodiment of the adaptive dilated convolution industrial time series interpolation system of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] In the process of modern industrial systems transforming towards intelligence and digitalization, the problem of missing industrial time series data is widespread, severely restricting the application effectiveness of data-driven methods in industrial scenarios. Traditional implicit-explicit diffusion model-driven interpolation methods use a fixed expansion rate in their implicit feature extraction modules, making it difficult for the model to adapt to the variable periodicity of industrial data, thus limiting interpolation accuracy. Furthermore, this method uses only a single MSE loss function, which is prone to gradient explosion when operating conditions change abruptly or data is severely missing, affecting the model's stable convergence.

[0020] This application provides an adaptive dilated convolution industrial time series interpolation method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the adaptive dilated convolution industrial time series interpolation method of this application.

[0021] In this embodiment, the adaptive dilated convolution industrial time series interpolation method includes the following steps: Step S10: Obtain the industrial time series data to be interpolated. The industrial time series data includes observations collected by multivariable industrial sensors at multiple sampling times, where some sampling times have missing data.

[0022] For ease of understanding, the following explains some key terms in this embodiment: Industrial time series data refers to multivariate observations that are continuously collected by industrial sensors at different sampling times and arranged in chronological order. These data are often used to monitor industrial processes, but data gaps may exist.

[0023] Implicit feature extraction aims to capture local patterns and periodic features from industrial time series data that are not easily observed directly.

[0024] Temporal self-attention is a neural network component that dynamically generates attention weights by calculating the correlation between different time steps in a sequence, thereby effectively capturing the dependencies within the time series.

[0025] Adaptive dilation rate refers to the sampling interval of the convolutional kernel in the time dimension. This interval can be dynamically adjusted according to the characteristics of the input data, rather than using a fixed value.

[0026] Dilated causal convolution is a special type of convolution operation that expands the receptive field by inserting holes between the elements of the convolution kernel, while ensuring that the output of the current time step depends only on the input of the current and past time steps, thus preserving the causality of the time series.

[0027] Implicit feature representation is a numerical vector that reflects the local and periodic patterns of data after being processed by the implicit feature extraction module.

[0028] Explicit feature extraction aims to capture explicit structured features such as long-term trends and global dependencies from industrial time series data.

[0029] The structured state-space model is a model that can efficiently model long-term temporal dependencies. It describes the system dynamics through state-space equations, and its computational complexity increases linearly with the sequence length when dealing with long sequences.

[0030] Long-range time series dependency refers to the mutual influence or correlation between time steps that are far apart in a time series.

[0031] Explicit feature representation is a numerical vector that reflects the global and long-range dependency patterns of data after being processed by the explicit feature extraction module.

[0032] Gated fusion is a fusion method that dynamically controls the flow of information through gating units (such as the sigmoid function), and can perform weighted combinations based on the importance of input features.

[0033] The fusion feature representation is a comprehensive feature vector obtained by processing the implicit feature representation and the explicit feature representation through a gating fusion mechanism.

[0034] The conditional diffusion model is a generative model that generates data through a forward process of progressively adding noise and a reverse process of progressively denoising, wherein the reverse denoising process is guided by additional conditional information.

[0035] Iterative denoising refers to the process by which a conditional diffusion model gradually recovers the original data from noise by repeatedly performing noise estimation and data update operations during the reverse process.

[0036] The Huber loss function is a loss function that combines the advantages of mean squared error (MSE) and mean absolute error (MAE). It uses squared loss when the prediction error is small and absolute loss when the prediction error is large, thus making it robust to outliers.

[0037] The preset threshold τ is the boundary value used in the Huber loss function to distinguish between squared loss and absolute loss.

[0038] Mean Squared Error (MSE) is a commonly used loss function that calculates the average of the squared differences between the predicted and actual values.

[0039] MAE (Mean Absolute Error) is a commonly used loss function that calculates the average of the absolute values ​​of the differences between the predicted and actual values.

[0040] In practical implementation, in industrial processes, for multivariate time series containing missing values, let the complete data matrix be... (L is the number of time steps, K is the number of sensors), the observation mask matrix is: ( =1 indicates that position (t,k) is missing. The imputation objective is to reconstruct the true value at the missing position using the observable portion. Output: in The complete prediction matrix generated for the diffusion model. To remove missing values ​​from the observation matrix, ⊙ represents element-wise multiplication. The interpolation and repair process is performed simultaneously across the entire sequence, making full use of both local and global temporal context.

[0041] Step S20: Perform implicit feature extraction on the industrial time series data, dynamically calculate the adaptive dilation rate for each time step based on the temporal self-attention mechanism, and perform dilated causal convolution according to the adaptive dilation rate to generate implicit feature representation.

[0042] It should be noted that the adaptive expansion rate dynamically calculated for each time step based on the temporal self-attention mechanism includes: performing a linear transformation on the input feature matrix through a learnable projection matrix to generate a query vector and a key vector; performing a scaled dot product on the query vector and the key vector to obtain an attention score, and generating attention weights through softmax normalization; using the attention weights to perform a weighted average of the candidate offset distances to obtain the proposal expansion rate; and introducing a sigmoid gating module to perform bounded modulation on the proposal expansion rate using the original input features of the current time step to generate the adaptive expansion rate.

[0043] Through the above technical solution, this embodiment can dynamically calculate a more accurate and stable adaptive dilation rate for each time step. Specifically, by converting the input feature matrix into query vectors and key vectors, and generating attention weights based on a scaled dot product attention mechanism, the model can identify the temporal dependency strength between different time steps. Subsequently, by using these attention weights to perform a weighted average of preset candidate offset distances, a preliminary proposed dilation rate can be obtained, which has initially considered the global temporal context. Furthermore, a sigmoid gating module is introduced, and the proposed dilation rate is boundedly modulated in conjunction with the original input features of the current time step. This not only makes the adjustment of the dilation rate more refined and localized, but also ensures the stability of the dilation rate, avoiding fluctuations caused by over-reliance on global information. This dynamic and controlled dilation rate calculation method allows the dilated causal convolution to adjust its receptive field more flexibly, thereby effectively capturing the complex and variable temporal dependencies in industrial time series, improving the accuracy and robustness of implicit feature extraction, and thus enhancing the performance of the overall interpolation method.

[0044] It should be noted that performing dilated causal convolution according to the adaptive dilation rate includes: when the adaptive dilation rate is not an integer, linear interpolation is used for the convolution sampling position, adjacent integer time steps are taken and weighted by the fractional part to obtain the feature value of the non-integer position; the adaptive dilation rate is calculated independently at each time step, so that each time step has an independent dilation rate, thereby realizing the dynamic matching between the convolution receptive field and the actual change cycle of the industrial time series.

[0045] Through the above technical solution, this embodiment effectively solves the problem of sampling non-integer adaptive dilation rates in discrete time series, ensuring that dilated causal convolution can flexibly utilize any calculated dilation rate. Simultaneously, by independently calculating and applying the dilation rate at each time step, the model can capture local and global temporal dependencies in industrial time series more precisely and accurately, enabling the convolutional receptive field to dynamically adapt to the complexity and non-stationarity of the data itself. This significantly enhances the accuracy and robustness of implicit feature representation, providing high-quality conditional information for subsequent fusion and imputation processes, thereby improving the accuracy of missing data estimation, and is particularly suitable for the complex and variable temporal patterns commonly found in industrial time series data.

[0046] In practical implementation, the temporal self-attention adaptive dilated convolution mechanism (implicit feature extraction module) includes: The original implicit feature extraction module uses an L-layer stack of dilated causal convolutions, with a fixed dilation rate of {1, 2, 4, 8, 16, 32} for each layer, extracting periodic features at three scales: "minute-hour-day". However, the actual industrial data cycle is not limited to these three time scales. When the actual data cycle does not match the preset scale, key periodic patterns will be missed, causing the model to fail to accurately perceive the data change patterns near the missing locations, thus reducing the interpolation accuracy. To address this, this embodiment innovatively introduces a temporal self-attention mechanism, dynamically generating an adaptive dilation rate for each time step before each convolutional layer is executed. This ensures that the receptive field of the convolution matches the actual change cycle of each industrial process variable in real time. The specific mechanism is as follows: Dilated causal convolution pairs input sequences At time step The calculation formula at the location is: in For expansion rate, The kernel size is [size]. This is a convolutional filter. By stacking convolutional layers with different dilation rates, the effective receptive field (ERF) of each layer is: The attention score and weight calculation includes: For the Layer input feature matrix Each through a learnable projection matrix and Perform a linear transformation to generate a query vector for each time step t. Generate a key vector for each historical step t-δ. The two scores are scaled and dot-productd to obtain the original attention score: in Divide by the query / key vector dimension. To prevent the softmax function from saturating due to excessively large inner product values ​​of high-dimensional vectors, thus avoiding the gradient minima problem, the attention weights are obtained by softmax normalization of all candidate offsets δ. Weight satisfy , ∈(0,1). A high-weighted δ indicates that the historical features at that offset distance are most valuable for predicting the current location.

[0047] The proposed expansion rate calculation includes: The proposed expansion rate at position t is obtained by weighting all candidate offset distances with attention weights. When attention is focused on small deviations Smaller (narrower receptive field), concentrated during large offsets Too large (wide receptive field). This weighted averaging operation affects the projection parameters. , Differentiable, supporting end-to-end training.

[0048] Among them, sigmoid-gated modulation includes: Introducing a gating module to use the original input features at position t Bounded modulation of the proposed expansion rate: in , For learnable gating parameters, σ(·) is the sigmoid function, whose output is in the range (0,1). The coefficient obtained by multiplying by 2 falls in the range (0,2). The relationship between the obtained coefficient and 1 determines whether to amplify, shrink, or directly use the suggested expansion rate. This allows the dynamic adjustment of the expansion rate to take into account both the real periodicity of the sequence change and the local characteristics of the current position, effectively preventing the receptive field of convolution from expanding or shrinking without constraint.

[0049] Among them, linear interpolation with non-integer expansion includes: because Typically, these are continuous real numbers, and the convolution sampling positions often do not fall on integer time steps. For non-integer positions, linear interpolation is used: adjacent integer steps are taken, and a weighted average is calculated using the fractional part as the weight. The specific formula is as follows: This operation is for Segmented differentiability ensures that the entire network can be jointly optimized end-to-end.

[0050] The implementation of the adaptive receptive field includes: The above adaptive expansion rate calculation is performed once at each time step t in the sequence, for a total of T times. Each time, it is performed from the same input matrix. Different rows are selected to generate different query vectors, resulting in different attention weights and expansion rates. Compared to the original implicit feature extraction module's strategy of using a fixed expansion rate, each time step now has an independent adaptive expansion rate. This truly achieves dynamic matching between the receptive field of the expanded causal convolution and the actual change cycle of each industrial process variable, thus giving the model a stronger ability to adapt to situations such as operation mode switching and sudden load changes in industrial processes. Furthermore, in scenarios with severely missing data, the model can proactively seek more distant effective observation points as references, effectively improving the interpolation and repair accuracy under sparse observation conditions.

[0051] Step S30: Perform explicit feature extraction on the industrial time series data, model the long-range temporal dependence of the industrial time series data based on the structured state-space model, and generate explicit feature representations; It should be noted that modeling the long-range temporal dependence of the industrial time series data based on the structured state-space model includes: describing the state transitions and output responses of the industrial time series based on state-space equations, wherein the state-space equations include state transition parameters, input parameters, and output parameters; discretizing the state transition parameters to generate discrete state transition parameters, and constructing convolution kernels; and outputting the explicit feature representation based on the direct convolution operation between the convolution kernels and the input sequence, without the need for progressively recursively deriving the hidden states; wherein, the state matrix of the state-space equations is initialized based on a high-order polynomial projection operator, so that the computational cost of the model increases linearly with the sequence length when processing long input sequences.

[0052] Through the above technical solution, this embodiment effectively solves the problem of high computational complexity in traditional methods when processing long sequences for explicit feature extraction of industrial time series data. By discretizing the state space equation and constructing it as a convolution kernel, and combining it with a high-order polynomial projection operator to initialize the state matrix, explicit feature representations can be directly extracted based on convolution operations, avoiding the computational overhead of progressively iterating the hidden states. This allows the computational load to increase linearly with the sequence length when processing long input sequences, significantly improving the efficiency and real-time performance of processing large-scale industrial time series data. This enables faster and more accurate generation of explicit feature representations with long-range temporal dependencies, providing high-quality conditional information for subsequent fusion and interpolation.

[0053] In practical implementation, the structured state-space model (explicit feature extraction module) includes: In scenarios with high missing rates (60%–80% data missing), implicit feature extraction modules that rely on dilated causal convolution to capture multi-scale periodic features struggle to find effective imputation references due to the sparseness of observation points near the missing locations. Therefore, the explicit feature extraction module employs S4 (a structured state-space sequence model for effectively capturing long-range temporal dependencies in industrial time series), whose core mathematical description is as follows: in For one-dimensional input, For one-dimensional output, Given N-dimensional hidden states, the model adjusts the state transition parameters during training. Continuous optimization is performed. Discrete state transition parameters are obtained by discretizing the time step Δ. Then construct the convolution kernel. , Each element records the system's historical response at different times. The final output response can be based on... = Direct computation eliminates the need for iterative derivation of hidden states, significantly improving computational efficiency. The state matrix A of S4 is initialized based on HiPPO (High-order Polynomial Projection Operators), ensuring that the computational cost of the model increases only linearly with the sequence length (O(L)) when processing long input sequences. This approach preserves historical input information while maintaining computational efficiency, making it particularly suitable for situations with severely missing data, where effective observations from earlier moments in the input sequence can be used to provide data imputation at missing locations.

[0054] Specifically, the S4 module consists of multiple stacked S4 Blocks. Each Block sequentially passes through an S4 layer, GELU activation, Dropout regularization, residual connections, and layer normalization (Layer Normalization) before outputting. The residual connections between Blocks ensure stable gradient propagation during deep network training. The S4 module ultimately outputs explicit features. This provides global feature information covering the complete industrial time series for subsequent gating fusion mechanisms.

[0055] Step S40: Based on the gating fusion mechanism, the implicit feature representation and the explicit feature representation are weighted and fused to generate a fused feature representation.

[0056] It should be noted that the weighted fusion of the implicit feature representation and the explicit feature representation based on the gating fusion mechanism includes: concatenating the implicit feature representation and the explicit feature representation; performing a nonlinear transformation on the concatenated features using the tanh function and generating gating weights using the sigmoid function; and multiplying the nonlinear transformation result element-wise with the gating weights to generate the fused feature representation.

[0057] Through the aforementioned gating fusion mechanism, this embodiment effectively integrates implicit and explicit feature representations. First, a concatenation operation initially combines the two types of features, providing a unified input for subsequent deep interaction. Next, the tanh function is used to perform a nonlinear transformation on the concatenated features, enhancing their expressive power and complexity, enabling them to capture deeper patterns. Simultaneously, gating weights are dynamically generated using the sigmoid function. These weights adaptively evaluate and adjust the importance of different feature components based on the characteristics of the current data. Finally, by multiplying the nonlinear transformation result element-wise with the gating weights, refined filtering and weighting of feature information are achieved. This mechanism avoids information redundancy or conflicts that may result from simple concatenation or summation, allowing the model to intelligently focus on the most critical features for the imputation task, thereby generating a more representative and discriminative fused feature representation. This fused feature representation can more comprehensively and accurately reflect the inherent patterns of industrial time series data, providing high-quality conditional information for missing data estimation in subsequent conditional diffusion models, thus significantly improving the imputation accuracy and robustness of industrial time series data.

[0058] In its implementation, the gating fusion mechanism includes: implicit and explicit feature extraction modules output local multi-scale features and global long-range dependencies, respectively. The two feature properties are complementary. To achieve effective fusion, a parallel gating mechanism using tanh and sigmoid is adopted. tanh(·) performs a nonlinear transformation on the fused features, constraining the feature values ​​within a bounded interval; σ(·) generates the corresponding gating weights, dynamically controlling the selection and rejection of fused feature information; the results from the two calculations are multiplied element-wise to obtain the final result. .

[0059] In practical implementation, the residual convolution module and the skip convolution output layer also include: fusion features of the gated fusion output. With diffusion steps After unifying the feature dimensions and concatenating them, the features are refined layer by layer through six residual convolutional modules (Res Conv). The intermediate features output by each residual module are transformed to the same dimension by skip convolution (Skip Conv) and then accumulated to obtain six feature information at different depth levels for noise prediction. This drives the conditional diffusion model to gradually denoise and generate corresponding interpolation and repair results.

[0060] The residual convolution module includes: The design philosophy of the residual convolution module originates from the residual learning framework. Its core principle is to allow the network layers to fit the residual mapping between the input and output, rather than directly fitting the target function. The classic mathematical expression of this idea is: Where x is the input feature of the residual module, y is the output feature, and F(x) is the residual mapping learned by combining several convolutional layers and nonlinear activation functions. By directly superimposing the input x (i.e., the identity mapping) at the module output, the gradient can be directly backpropagated to the shallow layer via the identity branch during backpropagation, fundamentally alleviating the gradient vanishing problem in deep network training and ensuring the training stability of the 6-layer cascaded architecture.

[0061] Specifically, the information processing flow within each residual convolutional module is as follows: The input feature h is first subjected to Conv1D convolution and ReLU activation to complete the nonlinear transformation, and then the current diffusion step is embedded. Injected into the module, it becomes aware of the current denoising step of the model. After a second Conv1D and Dropout regularization process, the features are fused with the gated features. Perform element-wise multiplication to make The observation condition information carried in the input h is used in the calculation of the current feature, and finally superimposed with the input h to obtain the output of this module. This process can be represented as: Where h is the input feature of this module, and h' is the input feature h after double convolution, diffusion step embedding and injection, and... The intermediate features obtained after element-wise multiplication are denoted as hout, which is the output feature of this module. ⊙ represents element-wise multiplication, and σ(·) is the sigmoid function. The residual connection operation (h'+h) ensures that the parameters of each layer can obtain effective gradient signals, and the six modules can still be trained stably after cascading.

[0062] The skip convolution output layer includes: After completing the above operations, each residual convolutional module will output an intermediate feature sr (the result of a linear projection of h') to the skip convolutional output layer. The intermediate features output by each of the six residual modules will be accumulated element-wise to obtain the skip feature S: The skip feature S is processed by ReLU activation and the final convolutional layer to obtain the noise prediction value at the current diffusion step t: The core function of the skip convolution output layer is to integrate the feature information captured by the six residual modules at different depths (shallow layers capture fine-grained local fluctuations, and deep layers capture abstract global structures) into the final noise prediction, effectively avoiding the problem of shallow features being forgotten due to relying solely on the output of the last layer in deep networks.

[0063] Step S50: Input the fused feature representation as conditional information into the conditional diffusion model, generate data estimates for missing positions through iterative denoising, and output the complete industrial time series after interpolation and repair.

[0064] The conditional diffusion model is trained using the Huber loss function. With a preset threshold τ as the boundary, MSE is used to calculate the loss when the prediction error is less than τ, and MAE is used to calculate the loss when the prediction error is greater than τ.

[0065] It should be noted that the process of inputting the fused feature representation as conditional information into the conditional diffusion model and generating data estimates for missing locations through iterative denoising includes: in the forward process, gradually adding Gaussian noise to the industrial time series data according to a preset noise variance schedule to make the data distribution approximate a standard Gaussian distribution; in the reverse process, based on the fused feature representation as conditional information, approximating the noise components added in the forward process through a neural network, and gradually performing denoising; using the denoised result as the data estimate for the missing location, merging it with the observations of the non-missing locations, and outputting the complete industrial time series after interpolation and repair.

[0066] Through the above technical solution, this embodiment clarifies the specific iterative denoising mechanism of the conditional diffusion model in industrial time series interpolation. In the forward process, Gaussian noise is gradually added, enabling the model to learn the transformation process of data from ordered to disordered. In the reverse process, based on the fused feature representation as conditional information, a neural network accurately estimates and removes noise components, effectively solving the challenge of efficiently and accurately recovering high-quality missing data from noise in complex industrial time series. Guided by the fused feature representation, this iterative denoising process fully utilizes the temporal dependence and structured information of the data, ensuring that the generated missing data estimates are highly consistent with existing observations, thereby significantly improving the accuracy and rationality of the interpolation results. The final output complete industrial time series has higher reliability.

[0067] It should be noted that the Huber loss function only calculates the loss at the location of missing data. The specific form of the Huber loss function is as follows: when the prediction error is less than the preset threshold τ, the squared loss is used to calculate the loss value; when the prediction error is greater than or equal to the preset threshold τ, the absolute loss is used to calculate the loss value. At this time, the backpropagation gradient is always constant and no longer increases linearly with the increase of error; and the preset threshold τ is taken as the 75% to 90% quantile of the error distribution at the location of missing data in the training set.

[0068] Through the above technical solution, this embodiment effectively solves the problems of inaccurate loss function calculation range and unreasonable Huber loss function threshold setting in industrial time series interpolation tasks. Specifically, by applying the Huber loss function only to the calculation of missing data locations, the model can concentrate all optimization resources on the accurate estimation of missing data during training, avoiding repeated learning of known observations, thereby significantly improving the efficiency of model training and the specificity of interpolation. Simultaneously, by adaptively setting the preset threshold τ to the 75% to 90% quantile of the error distribution of missing locations in the training set, it is ensured that the Huber loss function can dynamically adjust its error penalty mechanism according to the actual data characteristics. When the prediction error is small, squared loss is used to encourage the model to make refined predictions; when the prediction error is large, absolute loss is switched, effectively suppressing the excessive influence of outliers on gradient updates, ensuring the stability of training and the robustness of the model. This Huber loss function, which combines precise loss calculation range and adaptive threshold setting, with the conditional diffusion model, enables the model to learn the potential distribution of industrial time series data more accurately and stably, generate high-quality missing data estimates, and finally output the complete industrial time series after imputation and repair. It is especially suitable for complex situations in industrial scenarios where the data missing rate is high and noise and outliers are common.

[0069] It should be noted that the step of inputting the fused feature representation as conditional information into the conditional diffusion model further includes: concatenating the fused feature representation with the gated fused feature, and refining it layer by layer through multiple residual convolution modules; the intermediate features output by each residual convolution module are transformed to the same dimension by jump convolution and then accumulated to obtain jump features; the jump features are used as conditional information, concatenated with the position encoding features of the current diffusion step, and then input into the conditional diffusion model to generate differentiated noise predictions.

[0070] In practical implementation, the conditional diffusion probability model includes: the diffusion model guides the recovery of missing data locations by progressively adding noise in the forward direction and iteratively denoising in the reverse direction, using observed data as a condition. The forward noise addition process progressively adds Gaussian noise to the data according to variance scheduling. in ∈(0,1) represents the preset noise variance scheduling. After T steps of positive noise addition, the overall data distribution approaches the standard Gaussian distribution.

[0071] The reverse denoising process uses feature fusion For conditional information: in This is an estimation of the noise component added to the forward process, approximated by a neural network. This is based on fusion features The obtained conditional information.

[0072] To enable the denoising network to perceive the current diffusion step t, sinusoidal positional encoding is used to map t into a 128-dimensional feature vector. : After transformation by MLP (Multilayer Perceptron), the fused features are along the feature channel direction. By concatenating the input tensors into a unified input tensor, the denoising network can simultaneously acquire information about the current step and the observation conditions at each step, thereby generating differentiated noise predictions.

[0073] In practical implementation, the Huber loss function improvement includes: The original method uses MSE as the training loss function. When there are sudden changes in operating conditions or severe data loss, the large error values ​​are further amplified by the square operation in MSE, easily leading to gradient explosion during backpropagation and parameter updates, causing the training loss function to fail to converge stably. This patent introduces the Huber loss function to replace the MSE loss function, defined as follows: in To interpolate the difference between predicted and true values, This is the segmentation threshold hyperparameter. When... At that time, the loss was Consistent with MSE, this means that fine-grained penalties are still maintained for small errors; when At that time, the loss is switched to With the same order as MAE, the gradient of this loss function remains constant during backpropagation. This prevents the gradient from amplifying linearly with increasing error, thus fundamentally curbing the gradient explosion problem during backpropagation. Piecewise functions in... The gradient is continuous and differentiable, which ensures the smoothness of gradient updates.

[0074] Training only at missing locations ( The loss is calculated in the following form: in For the missing location index set, denoted as the total number of missing elements. Compared to the MSE loss function, the Huber loss function can effectively avoid the gradient explosion problem during backpropagation in scenarios with high missing rates in the Debutanizer and SRU datasets, while also improving the RMSE (Root Mean Squared Error) metric.

[0075] threshold The value is taken as the 75th to 90th percentile of the missing location error distribution in the training set, ensuring that the vast majority of normal samples in the training set fall within this range. The interval is used to calculate the loss based on MSE, and its gradient magnitude is... The parameter update step size will decrease synchronously as the error decreases, so that the parameter update step size will automatically decrease when approaching the optimal solution, avoiding continuous oscillation of parameters near the optimal solution that could cause the model to fail to converge; only a few outlier samples with large errors fall into the range of 0. In the interval, the loss is calculated based on MAE, and the gradient magnitude is constant. It no longer changes with the error, thus effectively avoiding the gradient explosion problem.

[0076] Reference Figure 2 , Figure 2 This is a structural block diagram of the first embodiment of the adaptive dilated convolution industrial time series interpolation system of this application.

[0077] like Figure 2 As shown, the adaptive dilated convolution industrial time series interpolation system proposed in this application includes: Data acquisition module 10 is used to acquire industrial time series data to be interpolated. The industrial time series data includes observations collected by multivariable industrial sensors at multiple sampling times, where some sampling times have missing data. The implicit extraction module 20 is used to perform implicit feature extraction on the industrial time series data. It dynamically calculates the adaptive dilation rate for each time step based on the temporal self-attention mechanism, and performs dilated causal convolution according to the adaptive dilation rate to generate implicit feature representation. The explicit extraction module 30 is used to perform explicit feature extraction on the industrial time series data, model the long-range temporal dependence of the industrial time series data based on the structured state space model, and generate explicit feature representations. The fusion module 40 is used to perform weighted fusion of the implicit feature representation and the explicit feature representation based on a gated fusion mechanism to generate a fused feature representation; The output module 50 is used to input the fused feature representation as conditional information into the conditional diffusion model, generate data estimates for missing positions through iterative denoising, and output the complete industrial time series after interpolation and repair.

[0078] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solution of this application. In specific applications, those skilled in the art can make settings as needed, and this application does not impose any restrictions on this.

[0079] This embodiment achieves dynamic matching between the convolutional receptive field and the actual change cycle of the industrial time series by dynamically calculating the adaptive dilation rate for each time step. It also uses the Huber loss function to switch to absolute loss when the prediction error is large in order to suppress gradient explosion. This effectively solves the problem of feature extraction mismatch caused by fixed dilation rate and training instability caused by mean square error loss. It has the advantages of being able to dynamically match the actual change cycle of the industrial time series to improve interpolation accuracy, and suppressing the gradient amplification effect of large errors through the Huber loss function to ensure training stability.

[0080] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this application. In practical applications, those skilled in the art can select some or all of it to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0081] In addition, for technical details not described in detail in this embodiment, please refer to the adaptive dilated convolution industrial time series interpolation method provided in any embodiment of this application, which will not be repeated here.

[0082] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0083] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0084] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application. The above are only preferred embodiments of this application and do not limit the patent scope of this application. All equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An adaptive dilated convolution industrial time series interpolation method, characterized in that, include: Acquire industrial time series data to be interpolated, which includes observations collected by multivariable industrial sensors at multiple sampling times, where some sampling times have missing data; Implicit feature extraction is performed on the industrial time series data. An adaptive dilation rate is dynamically calculated for each time step based on a temporal self-attention mechanism, and dilated causal convolution is performed according to the adaptive dilation rate to generate implicit feature representations. Explicit feature extraction is performed on the industrial time series data, and the long-range temporal dependence of the industrial time series data is modeled based on the structured state-space model to generate explicit feature representations. Based on the gating fusion mechanism, the implicit feature representation and the explicit feature representation are weighted and fused to generate a fused feature representation; The fused feature representation is input as conditional information into the conditional diffusion model. Data estimates for missing locations are generated through iterative denoising, and the complete industrial time series after interpolation and repair is output. The conditional diffusion model is trained using the Huber loss function. With a preset threshold τ as the boundary, MSE is used to calculate the loss when the prediction error is less than τ, and MAE is used to calculate the loss when the prediction error is greater than τ.

2. The method according to claim 1, characterized in that, The adaptive expansion rate is dynamically calculated for each time step based on the temporal self-attention mechanism, including: The input feature matrix is ​​linearly transformed by the learnable projection matrix to generate the query vector and the key vector. The attention score is obtained by scaling the dot product of the query vector and the key vector, and the attention weight is generated by softmax normalization. The proposed expansion rate is obtained by weighting the candidate offset distances with the attention weights. A sigmoid gating module is introduced to perform bounded modulation on the proposed expansion rate using the original input features at the current time step, thereby generating the adaptive expansion rate.

3. The method according to claim 1, characterized in that, Performing dilated causal convolution according to the adaptive dilation rate includes: When the adaptive dilation rate is non-integer, linear interpolation is used for the convolution sampling position, and adjacent integer time steps are taken and weighted by the fractional part to obtain the feature value of the non-integer position. The adaptive expansion rate is calculated independently at each time step, so that each time step has an independent expansion rate, thereby achieving dynamic matching between the convolutional receptive field and the actual change cycle of the industrial time series.

4. The method according to claim 1, characterized in that, Modeling the long-range temporal dependencies of the industrial time series data based on a structured state-space model includes: The state transition and output response of an industrial time series are described by a state-space equation, which includes state transition parameters, input parameters, and output parameters. The state transition parameters are discretized to generate discrete state transition parameters, and a convolution kernel is constructed. Based on the direct convolution operation between the convolution kernel and the input sequence, the explicit feature representation is output without the need to iterate through the hidden state step by step. The state matrix of the state space equation is initialized based on a high-order polynomial projection operator, which makes the computational cost of the model increase linearly with the sequence length when processing long input sequences.

5. The method according to claim 1, characterized in that, The weighted fusion of the implicit feature representation and the explicit feature representation based on the gating fusion mechanism includes: The implicit feature representation is concatenated with the explicit feature representation; The concatenated features are subjected to nonlinear transformation by the tanh function and then gating weights are generated by the sigmoid function. The nonlinear transformation result is multiplied element-wise with the gating weights to generate the fused feature representation.

6. The method according to claim 1, characterized in that, The fused feature representation is input as conditional information into the conditional diffusion model, and the generation of data estimates for missing locations through iterative denoising includes: During the forward process, Gaussian noise is gradually added to the industrial time series data according to the preset noise variance scheduling, so that the data distribution approaches the standard Gaussian distribution; In the reverse process, based on the fused feature representation as conditional information, the noise component added in the forward process is approximately estimated through a neural network, and denoising is performed step by step; The denoised result is used as the data estimate for the missing position, and merged with the observations for the non-missing positions to output the complete industrial time series after interpolation and repair.

7. The method according to claim 1, characterized in that, The Huber loss function calculates the loss only at locations where data is missing. The specific form of the Huber loss function is as follows: When the prediction error is less than the preset threshold τ, the loss value is calculated using squared loss. When the prediction error is greater than or equal to the preset threshold τ, the loss value is calculated using absolute loss. At this time, the backpropagation gradient is always constant and no longer increases linearly with the increase of error. The preset threshold τ is set to the 75% to 90% percentile of the error distribution of missing locations in the training set.

8. The method according to claim 1, characterized in that, The step of inputting the fused feature representation as conditional information into the conditional diffusion model further includes: The fused feature representation is concatenated with the gated fused feature and then refined layer by layer through multiple residual convolution modules. The intermediate features output by each residual convolution module are transformed to the same dimension by skip convolution and then accumulated to obtain the skip features; The jump feature is used as conditional information, concatenated with the position encoding feature of the current diffusion step, and then input into the conditional diffusion model to generate differentiated noise predictions.

9. An adaptive dilated convolution industrial time series interpolation system, characterized in that, Performing the method as described in claim 1 includes: The data acquisition module is used to acquire industrial time series data to be interpolated. The industrial time series data includes observations collected by multivariable industrial sensors at multiple sampling times, where some sampling times have missing data. The implicit extraction module is used to perform implicit feature extraction on the industrial time series data. It dynamically calculates the adaptive dilation rate for each time step based on the temporal self-attention mechanism, and performs dilated causal convolution according to the adaptive dilation rate to generate implicit feature representations. The explicit extraction module is used to perform explicit feature extraction on the industrial time series data, and to model the long-range temporal dependence of the industrial time series data based on the structured state space model to generate explicit feature representations. The fusion module is used to perform weighted fusion of the implicit feature representation and the explicit feature representation based on a gated fusion mechanism to generate a fused feature representation; The output module is used to input the fused feature representation as conditional information into the conditional diffusion model, generate data estimates for missing positions through iterative denoising, and output the complete industrial time series after interpolation and repair.

Images