Fusion of temporal convolutional network and self-attention mechanism for data imputation of smelting furnace

Abstract

This invention relates to the field of industrial data processing technology, specifically to a data interpolation method for casting furnaces that integrates temporal convolutional networks and a self-attention mechanism, to address the problems of high missing rates and strong noise interference in current side-blown casting furnace scenarios. The method includes a temporal feature extraction method based on integrated temporal convolutional networks, capturing parameter transformations through dilated convolutions and residual connections, and capturing the spatiotemporal dependencies between multiple variables based on a multi-dimensional time series input self-attention mechanism algorithm, improving adaptability to complex missing patterns. An error loss module is constructed by combining temporal convolutional network feature extraction with dynamic time warping assistance, achieving accurate interpolation of cross-modal and frequency data in industrial scenarios with high missing rates and strong noise. Furthermore, based on the self-attention mechanism framework, by extracting temporal features through integrated temporal convolutional networks and introducing a dynamic time warping loss function, the high missing rate and strong noise interference in side-blown casting furnace scenarios are effectively avoided.

Description

Technical Field

[0001] This invention relates to the field of industrial data processing technology, specifically to a data interpolation method for casting furnaces that integrates temporal convolutional networks and self-attention mechanisms. Background Technology

[0002] A side-blown lance injects oxygen-rich air and fuel into the molten pool, while continuous feeding from the top of the furnace causes the material to undergo an oxidation reaction in the slag layer, releasing heat and maintaining a high-temperature reaction atmosphere inside the furnace. In actual production, the sampling frequency of key smelting indicators varies significantly. Matte temperature is collected hourly using thermocouples, while the slag-iron-silicon ratio and nickel-mate grade require manual sampling and fluorescence analysis. Due to sensor malfunctions and manual recording errors, the actual recorded data has some random gaps. The input data is minute-level sensor sampling data, with virtually no gaps. When measured on an hourly basis, the gap rate for mate temperature is around 3%-4%, while the gaps for slag-iron-silicon ratio and nickel-mate grade are 60%-70%. In industrial settings, traditional approaches to addressing missing time-series data primarily include directly deleting samples with missing values ​​or simple imputation based on statistical assumptions. Direct deletion is convenient but leads to the loss of valuable information. In continuous production scenarios like metallurgy, direct deletion disrupts the continuity of the process sequence and affects the accuracy of modeling. While statistical imputation methods can preserve data integrity, they assume a stable data distribution and linear correlation between variables, making them ill-suited to the strong nonlinearity and dynamic correlations of industrial data. In recent years, deep learning technology has provided better solutions for industrial data governance through nonlinear modeling and adaptive feature extraction capabilities. However, existing imputation methods still have bottlenecks in scenario adaptability. Time series missing value imputation methods based on bidirectional recurrent neural networks optimize historical and future information together to reduce error accumulation, but the gradient vanishing problem of its RNN architecture limits its ability to model long sequences. Improved gated recurrent unit models model the impact of missing values ​​through time decay mechanisms, but lack robustness in high missing value scenarios. SAITS uses diagonal mask self-attention to dynamically model global dependencies, but performs poorly in local feature extraction. Extended recurrent neural networks capture temporal dependencies through multi-directional propagation, but the computational complexity increases dramatically with the sequence length. Summary of the Invention

[0003] This invention provides a data interpolation method for casting furnaces that integrates temporal convolutional networks and self-attention mechanisms to solve the problems mentioned above.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A data interpolation method for melting furnaces that integrates temporal convolutional networks and self-attention mechanisms includes:

[0006] (1) A temporal feature extraction method based on fused temporal convolutional networks uses techniques such as causal convolution, dilated convolution, and residual connections in the process of constructing the time series model. Causal convolution ensures that the output at the current moment depends only on the current and historical inputs, thus maintaining the causality of the time series data. Dilated convolution effectively captures the long-term dependencies of the filtered variables by exponentially expanding the receptive field. Residual connections directly transmit shallow features to deep networks and work in conjunction with dilated convolution to more accurately model the dynamic transformation process of parameters. Then, the joint probability distribution of input parameters and output indicators is calculated through mutual information to determine the key influencing factors. The formula for calculating mutual information is as follows:

[0007] ,

[0008] In the formula X As input features, Y For the output index, p( x,y )yes X and Y The joint probability density function, and p( x ) and p( y They are respectively X and Y The marginal probability density function.

[0009] (2) The input self-attention mechanism algorithm based on multidimensional time series captures the spatiotemporal dependency relationship between multiple variables. It includes a self-attention DMSA block with two diagonal masks and a weighted combination. The DMSA block captures the temporal dependency and feature correlation by shielding its own influence and only calculating the correlation relationship with other time steps, thereby improving the adaptability to complex missing patterns.

[0010] (3) Through a bidirectional masked self-attention architecture, the masked imputation task (MIT) randomly masks 10%-20% of the observations in the input data to generate double-missing samples, while the observation reconstruction task (ORT) reconstructs the corresponding output of the model. A dynamic time warping loss function is designed to suppress outlier interference, and the maximum absolute error is reduced compared with traditional methods. It has strong adaptability to sudden changes in the operating conditions. The loss function formula is:

[0011] ,

[0012] in, For interpolated values, For the true value, For missing data, MAE is the mean absolute error, and DTW is the dynamic time warp. αThe weight parameters are adjusted according to the actual situation, and the loss function is minimized through joint optimization methods to update the model parameters. The interpolation effect of the model is evaluated using evaluation indicators such as mean absolute error (MAE), mean absolute percentage error (MAPE), and coefficient of determination (R²). The performance of TCN-SAITS is compared with other benchmark models to verify the effectiveness and robustness of the model, thereby achieving high-precision interpolation of industrial time series data with high missing rate and strong noise in melting and casting furnaces.

[0013] Furthermore, in the process of extracting spatiotemporal dependencies, the SAITS algorithm is used, which uses two diagonal masked self-attention blocks and a weighted combination. The DMSA block, by shielding its own influence, only calculates the correlation with other time steps, thus capturing temporal dependencies and feature correlations.

[0014] Furthermore, during model training, MIT is used to randomly mask 10%-20% of the observations in the input data to generate double missing samples, and ORT then reconstructs the corresponding output of the model.

[0015] Furthermore, a joint loss function consisting of the mean absolute error (MAE) and the dynamic time warping (DTW) distance is designed to suppress outlier interference, reduce the maximum absolute error compared to traditional methods, and demonstrate strong adaptability to sudden change conditions.

[0016] Furthermore, during model training, a joint optimization method is used to minimize the loss function and update the model parameters. The interpolation performance of the model is evaluated using metrics such as MAE, MAPE, and R², and the performance of TCN-SAITS is compared with other benchmark models to verify the model's effectiveness and robustness.

[0017] Furthermore, during the model optimization process, methods such as linear interpolation, KNN, BRITS, and simple SAITS were compared. The interpolation effect of TCN-SAITS on different indicators (maton temperature, slag-iron-silicon ratio, nickel-maton grade) was analyzed, and the performance of different models in handling high missing rate and complex noise environments was compared.

[0018] The present invention has the following beneficial effects:

[0019] This invention significantly improves the accuracy and robustness of data interpolation through feature selection, local feature extraction, and global dependency modeling. In scenarios with high missing rates, the interpolation performance of the TCN-SAITS model is superior to traditional methods, verifying its effectiveness in complex industrial environments. It overcomes challenges such as collaborative missing data from multiple sources and heterogeneous data, nonlinear fluctuations, and high noise interference, and achieves high-precision interpolation of core smelting indicators such as matte temperature, slag-iron-silicon ratio, and nickel-matte grade.

[0020] This invention employs a Temporal Convolutional Network (TCN) to extract local spatiotemporal features, which can effectively capture short-term dependencies in the data and improve the accuracy of interpolation. It uses a Self-Attention System (SAITS) model to extract global features and achieves joint training for interpolation and reconstruction, which enhances the model's ability to understand long-term dependencies. It uses a joint loss function of Dynamic Time Warping (DTW) and Mean Absolute Error (MAE) to further improve the accuracy of temporal interpolation, especially performing well in scenarios with high missing rates.

[0021] This invention includes a temporal convolutional network, a self-attention mechanism, and a DTW-MAE joint loss function, which can be used on side-blown casting furnace data. When the algorithm performs the above functions, it can effectively solve the data interpolation problem under high missing rate and strong noise interference in the industrial production scenario of side-blown casting furnace. Attached Figure Description

[0022] Figure 1 This is an overall block diagram of the TCN-SAITS of the present invention.

[0023] Figure 2 This is a structural diagram of the SAITS of the present invention.

[0024] Figure 3 This is a structural diagram of the joint optimization training method for reconstructing the corresponding output based on random mask MIT and ORT according to an embodiment of the present invention.

[0025] Figure 4 This is a graph showing the top ten important variables for pheromone screening in this embodiment of the invention.

[0026] Figure 5 This is a diagram showing the effect of matte interpolation of the KNN model on the test set in an embodiment of the present invention.

[0027] Figure 6 This is a diagram showing the effect of matte interpolation of the linear model on the test set in an embodiment of the present invention.

[0028] Figure 7 This is a diagram showing the matte interpolation effect of the BRITS model on the test set in an embodiment of the present invention.

[0029] Figure 8 This is a diagram showing the effect of matte interpolation of the SAITS model on the test set in an embodiment of the present invention.

[0030] Figure 9 This is a diagram showing the matte interpolation effect of another SAITS model on the test set in an embodiment of the present invention. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0032] The furnace data interpolation method proposed in this embodiment, which integrates temporal convolutional networks and self-attention mechanisms, has the following TCN-SAITS model flow: Figure 1 As shown, it includes the following three parts:

[0033] 1. Feature selection and extraction:

[0034] First, feature selection is performed. Considering the multi-source, heterogeneous data characteristics of industrial scenarios, minute-level sensor data and low-frequency laboratory data are aligned using timestamps to construct a unified spatiotemporal matrix. Nonlinear correlations are quantified using Mutual Information (MI), and features highly correlated with output indicators are selected. The formula for calculating mutual pheromone is as follows. By calculating the joint probability distribution of input parameters and output indicators, key influencing factors are determined.

[0035]

[0036] Then, a temporal convolutional network (TCN) is used to effectively capture long-range dependencies in industrial time-series data through the local perception capability and hierarchical feature extraction mechanism of convolutional neural networks, while strictly maintaining temporal causality. Compared with traditional recurrent neural networks (RNNs), TCN, with its advantages in parallel computing and gradient stability, is more suitable for dynamic modeling of high-frequency sensor data in industrial production processes. Its core components and design principles are as follows:

[0037] Causal convolutions use right-side zero padding to ensure the kernel operates only on historical data, with the output at the current time step depending solely on past and current inputs, thus preventing the leakage of future information. Dilated convolutions expand the receptive field of the kernel through interval sampling, covering a longer historical range exponentially. Residual connections directly pass shallow features to deep networks via skip paths, mitigating gradient vanishing and feature degradation issues. Each residual block contains two layers of dilated causal convolutions, weight normalization, and ReLU activation, and aligns the input and output dimensions using a 1×1 convolution.

[0038] 2. Self-attention mechanism: such as Figure 2 As shown, SAITS is a multivariate temporal data interpolation model based on a self-attention mechanism. It consists of two diagonally masked self-attention (DMSA) blocks and a weighted combination. Through the DMSA blocks, the estimate of the current time step masks its own influence, calculating only the correlation with other time steps. This mechanism allows DMSA to capture the temporal dependencies and feature correlations between time steps in a high-dimensional space with only a single attention operation. SAITS employs a dual-task joint training framework, as shown... Figure 3As shown, this includes the Masked Imputation (MIT) task and the Observation Reconstruction (ORT) task. MIT involves randomly masking 10%-20% of the observations in the time-series data input to the model, generating training samples with double missing values ​​(original missing values ​​+ artificial mask); ORT reconstructs the model's output based on the remaining observations. Joint Loss Function It is determined by the following formula, where For the losses of the reconstruction mission, To compensate for task losses, weights Adjustments will be made based on actual circumstances.

[0039]

[0040] 3. Error loss construction:

[0041] DTW (Time Distance Measurement) is a method for measuring the similarity between two time series, particularly suitable for handling series of different lengths or with time shifts. Its core idea is to find the path with the minimum cumulative distance by aligning the time axes through dynamic programming, thereby quantifying the similarity between the series. Given two time series P={p1,p2,...,pm} and Q={q1,q2,...,qn}, where the lengths of series P and Q are m and n respectively, pi represents the observation of series P at time step i, and qj represents the observation of series Q at time step j, the local distance matrix D of series P and Q is first calculated as m rows and n columns, denoted as D∈Rm×n. Each element d(i,j) in the matrix represents the local distance between the i-th point pi of series P and the j-th point qj of series Q, defined as the Euclidean distance, i.e.:

[0042]

[0043] The goal of DTW is to find an optimal path from the starting point (1,1) to the ending point (n,m) of the matrix, such that the cumulative distance along the path is minimized. The cumulative distance matrix γ is recursively calculated using the state transition equation:

[0044]

[0045] The boundary conditions are set as follows: γ(0,0)=0, γ(i,0)=+∞ (i>0), γ(0,j)=+∞ (j>0) to ensure that the alignment path starts from (1,1) and to avoid invalid alignment.

[0046] Ultimately, the DTW distance is γ(n,m), which is the minimum cumulative distance at the end of the path.

[0047] In this embodiment, a side-blown casting furnace is used as an example. The production data of a certain factory's side-blown casting furnace is used for simulation. The production data is as follows:

[0048] Input parameters (minute-level input, no missing values) include:

[0049] Feed parameters: feed rates of concentrate, lump coal, and cold materials; smelting parameters: main pipe flow rate and temperature (oxygen, compressed air, and natural gas); lance flow rate adjustment value and pressure.

[0050] Output metrics (hourly output, some are missing), including:

[0051] Laboratory data: Slag-iron-silicon ratio (Fe / SiO2, dimensionless, missing rate 60%); Nickel-matte grade (Ni, unit: %, missing rate 60%); Matte temperature (T, unit: ℃, missing rate 3.7%).

[0052] High missing rates in slag-iron-silicon ratios and nickel-matte grades require strong temporal feature interpolation, while low missing rates in matte temperature can be achieved using traditional methods. However, the experiment uniformly adopted the SAITS model for processing.

[0053] In time series interpolation tasks, commonly used evaluation metrics include MAE (mean absolute error), MAPE (mean absolute percentage error), R² (coefficient of determination), and MAX (maximum absolute error), as shown in Table 1.

[0054] Table 1. Evaluation Indicators

[0055]

[0056] The goal of time-series interpolation is to achieve interpolated values ​​with low average deviation and similar data trends, without exhibiting extreme interpolation effects.

[0057] The top ten important variables for sulfonium temperature, as determined by mutual information filtering, are as follows: Figure 4As shown, comparative experiments are introduced, mainly comparing linear interpolation, KNN interpolation, BRITS, and simple SAITS models. Linear interpolation fills missing values ​​by fitting local trends through linear relationships, but it has poor adaptability to complex fluctuation patterns. KNN interpolation fills missing values ​​based on the weighted average of nearest neighbor samples, suitable for data with high local similarity, but sensitive to high-dimensional time-series features; the k value is set to 2. The BRITS bidirectional recurrent neural network model optimizes interpolated values ​​through forward and backward time propagation, reducing the error accumulation effect; the RNN hidden layers are set to 128 according to experiments. Simple SAITS is a baseline model trained using only MAE loss, used to verify the optimization effect of the joint loss. To ensure the reliability of the experimental results, all comparative models and the proposed TCN-SAITS model were tested multiple times under the same data partitioning and preprocessing conditions, and the final performance index was the average of multiple runs. The evaluation metrics mentioned above are used to evaluate the interpolation performance of the models. Interpolation models were established for three indicators: matte temperature, slag-iron-silicon ratio, and nickel-matte grade. The interpolation effect was reflected by manually introducing the missing parts. A comparison chart of matte temperature interpolation for different interpolation models is shown below. Figure 5 , Figure 6 , Figure 7 , Figure 8 The interpolation indices are shown in Table 2-4 below.

[0058] Table 2. Matte temperature interpolation parameters

[0059]

[0060] Table 3. Slag-Iron-Silicon Ratio Interpolation Indices

[0061]

[0062] Table 4. Nickel matte grade interpolation parameters

[0063]

[0064] A comparison of the interpolation results for three types of indicators—matte temperature, slag-iron-silicon ratio, and nickel-matte grade—shows that the TCN-SAITS model demonstrates significant advantages across all data types.

[0065] In the sulfonium temperature interpolation task, TCN-SAITS achieved a MAE of 4.36℃, significantly outperforming linear interpolation (6.87℃), KNN (7.90℃), BRITS (7.11℃), and SAITS (6.94℃), with a reduction of approximately 36.4% compared to the second-best result. Its R² (0.7895) was the highest among all methods, and its MAPE (0.3383) was also significantly lower than other models, more accurately capturing the nonlinear fluctuations of the temperature sequence. At the same time, its maximum absolute error (20.86℃) was nearly 15℃ lower than the worst linear interpolation. In the slag-iron-silicon ratio interpolation task, TCN-SAITS outperforms linear interpolation (0.0212), KNN (0.0322), BRITS (0.0454), and SAITS (0.0282) in terms of MAE (0.0184), and its R² (0.9236) is close to the theoretical optimum. Compared with the abnormal performance of the BRITS model on this metric (R² only 0.5362), it shows that the synergistic effect of the temporal convolutional network (TCN) and the self-attention mechanism (SAITS) can effectively overcome the problem of the traditional RNN architecture being sensitive to numerical scale. The maximum absolute error (0.0711) is reduced by 51.8% compared with SAITS, which confirms the robustness of the two-stage feature extraction mechanism in handling outliers. In the nickel matte grade interpolation task, TCN-SAITS' MAE (0.3181) is significantly better than linear interpolation (0.5737), KNN (0.6625), BRITS (0.7138) and SAITS (0.6087), reducing the efficiency by 47.8% compared to the second-best method; its R² (0.7786) is also the highest among all methods.

[0066] A horizontal comparison of the interpolation performance of three indicators—maton temperature, slag-iron-silicon ratio, and nickel-maton grade—reveals that traditional methods have significant shortcomings. While linear interpolation achieves R² values ​​above 0.5 for maton temperature and slag-iron-silicon ratio, its maximum absolute error (MAE) remains high, indicating poor adaptability to data abrupt changes. KNN outperforms linear interpolation in terms of MAE for slag-iron-silicon ratio and nickel-maton grade, but its overall MAE is lower. Among deep learning methods, BRITS suffers from significant performance degradation due to the gradient decay problem of RNNs when interpolating slag-iron-silicon ratio and nickel-maton grade. Although SAITS improves temporal modeling capabilities through self-attention mechanisms, its single attention structure struggles to balance local abrupt changes with global trends, resulting in a high maximum error (MAX).

[0067] TCN-SAITS achieves systemic advantages through collaborative modeling of local and global data: the TCN module employs causal convolution and dilated convolution, efficiently covering long time spans with multi-scale receptive fields while strictly maintaining temporal causality, accurately extracting local dynamics and short-term mutations in minute-level sensor data, effectively overcoming the gradient decay problem of RNNs; the SAITS module introduces diagonal masked self-attention (DMSA) to mask autocorrelation redundancy, explicitly modeling long-range dependencies between multiple variables across time steps, improving robustness to high-missing patterns; the DTW-MAE joint loss function optimizes numerical error (MAE) while aligning the imputed sequence with the true trajectory through dynamic time warping constraints, effectively suppressing extreme errors in highly volatile indicators such as nickel matte grade. In contrast, traditional methods lack dynamic modeling capabilities, BRITS struggles with long sequences, and SAITS neglects local details. TCN-SAITS, through the triple synergy of convolutional local perception, attention-based global modeling, and morphological constraint loss, achieves significant reductions in MAE and MAX in interpolation tasks for three key indicators: matte temperature, slag-iron-silicon ratio, and nickel-matte grade. 2 Its comprehensive leadership validates its advantages in high-deficit, high-noise industrial scenarios.

[0068] This invention addresses the problem of missing industrial time-series data in side-blown casting smelting scenarios. Through feature selection, spatiotemporal feature extraction, and joint optimization strategies, the model effectively overcomes the limitations of traditional methods in nonlinear, high-noise industrial environments. Experimental results show that:

[0069] Significantly improved accuracy: In the interpolation tasks of three key indicators, namely matte temperature, slag-iron-silicon ratio (Fe / SiO2) and nickel-matte grade (Ni%), the MAE of TCN-SAITS was reduced by 37.2%, 34.7% and 47.8% respectively compared with the original SAITS model, and the highest R² value reached 0.9236, which verified the advantages of multi-scale spatiotemporal feature extraction in modeling complex industrial data.

[0070] Enhanced robustness: For data with a high missing rate of 60%-70% (such as slag-iron-silicon ratio), the model captures long-range fluctuation patterns through the dilated convolution and residual connection of TCN, and suppresses outlier interference by combining the DTW-MAE joint loss function. The maximum absolute error is reduced by 51.8% compared with the traditional method, demonstrating strong adaptability to sudden operating conditions.

Claims

1. A method for data imputation of a smelting furnace by fusing a convolutional neural network with a self-attention mechanism, characterized in that, include: (1) A temporal feature extraction method based on fused temporal convolutional networks uses causal convolution, dilated convolution, and residual connection techniques in the process of constructing the time series model. Causal convolution ensures that the output at the current moment depends only on the current and historical inputs, thus maintaining the causality of the time series data. Dilated convolution effectively captures the long-term dependencies of the filtered variables by exponentially expanding the receptive field. Residual connection directly transmits shallow features to the deep network and works in conjunction with dilated convolution to more accurately model the dynamic transformation process of parameters. Then, the joint probability distribution of input parameters and output indicators is calculated through mutual information to determine the key influencing factors. The formula for calculating mutual information is as follows: ， where X is the input feature, Y is the output indicator, p( x,y ) is X and Y the joint probability density function, while p( x ) and p( y ) are the marginal probability density functions of X and Y respectively; (2) The input self-attention mechanism algorithm based on multidimensional time series captures the spatiotemporal dependency relationship between multiple variables. It includes a self-attention DMSA block with two diagonal masks and a weighted combination. The DMSA block captures the temporal dependency and feature correlation by shielding its own influence and only calculating the correlation relationship with other time steps, thereby improving the adaptability to complex missing patterns. (3) Through a bidirectional masked self-attention architecture, the masked imputation task (MIT) randomly masks 10%-20% of the observations in the input data to generate double-missing samples, while the observation reconstruction task (ORT) reconstructs the corresponding output of the model. A dynamic time warping loss function is designed to suppress outlier interference, and the maximum absolute error is reduced compared with traditional methods. It has strong adaptability to sudden changes in the operating conditions. The loss function formula is: ， in, For interpolated values, For the true value, The missing mask is used, MAE is the mean absolute error, and DTW is the dynamic time warp. α The weight parameters are adjusted according to the actual situation, and the loss function is minimized through joint optimization methods to update the model parameters. The interpolation effect of the model is evaluated using the mean absolute error (MAE), mean absolute percentage error (MAPE), and coefficient of determination (R²) evaluation indicators. The performance of TCN-SAITS is compared with other benchmark models to verify the effectiveness and robustness of the model, thereby achieving high-precision interpolation of industrial time series data with high missing rate and strong noise in melting and casting furnaces.

Images