Industrial process soft measurement method based on de-noising attention network
By stacking supervised reconstruction inputs with noise reduction autoencoder (SSRDAE) and interval attention LSTM (IALSTM) networks, the problem of insufficient information loss and noise robustness in industrial data is solved, and better denoising effect is achieved and quality prediction accuracy.
Patent Information
- Application Number
- CN202411482574.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-22
- Publication Date
- 2025-02-14
AI Technical Summary
The prior art has problems of information loss and insufficient noise robustness when processing industrial data, especially when sampling intervals are uneven, it is difficult to extract long-term historical features.
Denoising processing is performed using a noise reduction autoencoder (SSRDAE) with supervised reconstruction inputs, and combining the interval attention mechanism and LSTM network, the impact of different sampling intervals is adjusted to capture long-term historical features.
By reducing information loss and improving noise robustness, the SSRDAE-IALSTM model can more effectively extract deep features of industrial processes, improve the accuracy of quality prediction, and adapt to data with uneven sampling intervals.
Smart Images
Figure CN119441806A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an industrial process soft measurement method, in particular to an industrial process soft measurement method with uneven sampling intervals based on a denoising attention network. Background Art
[0002] In modern industrial process control and automation systems, measurement technology plays an important role. However, in practical applications, many key physical or chemical quantities are difficult to measure directly in real time through hardware sensors due to technical or economic reasons. In order to solve this problem, soft measurement technology came into being. In the feature modeling of early industrial processes, learning methods represented by principal component analysis PCA, independent principal component analysis ICA and partial least squares PLS were used to extract data features. With the continuous advancement of technology, these traditional methods face increasing challenges in processing large data. Shallow neural networks have limited ability to express complex features between data, while deep learning algorithms with powerful feature representation learning capabilities have gradually emerged and become an important modeling tool in the field of soft measurement technology. However, existing methods still face some challenges in practical applications.
[0003] 1) In the data collection and transmission process of actual industrial systems, various factors such as machine equipment or environmental interference, signal attenuation, and transmission delay may cause industrial data to contain a lot of noise. Stacked denoising autoencoders (SDAE) are widely introduced and applied in the field of soft measurement because they can restore the original data from damaged data and learn feature representations that are highly robust to noise. Although the existing SDAE can learn a certain robust feature representation through denoising when processing industrial data, the minimum reconstruction error of each layer in the existing SDAE is usually greater than zero, which means that there will be information loss in each layer. Since SDAE is composed of multiple layers of denoising autoencoders stacked together, the information loss of the original data will accumulate from the low layer to the high layer. Therefore, the learned denoising features may not be able to represent the original industrial data well. In addition, for quality prediction tasks, the extracted features may contain information irrelevant to quality prediction, which is not conducive to quality prediction.
[0004] 2) On the other hand, due to sampling conditions and technical limitations, the sampling frequency of industrial process time series is irregular, resulting in uneven sampling intervals. There are two main methods to solve this problem. The first is to generate uniformly sampled time series data in the form of interpolation or data supplementation, such as RNN-based interpolation, bidirectional RNN interpolation, etc. However, the data filled by the interpolation method may introduce additional noise, thereby affecting the quality of feature extraction. The second method is to directly incorporate the unevenly sampled sequence data into the modeling process and improve the processing of unevenly sampled sequence samples. However, the current method can only capture the influence of neighboring samples, and cannot focus on samples at different time steps at the same time, making it difficult to extract the long-term historical characteristics of the samples.
[0005] In order to solve the problems existing in the above-mentioned background technology 1), a stacked supervised reconstruction input denoising autoencoder (SSRDAE) is used to extract features from noisy data, because each output layer always seeks to reconstruct the original input data and quality data. In this way, the information loss of the original industrial data can be reduced, so that the learned deep features can represent the original industrial input data to a large extent, which can achieve better denoising effect than the traditional SDAE. In addition, in each layer, the output quality data is reconstructed. Based on this constraint, the learned features are conducive to predicting quality variables; for the problems existing in background technology 2), attention weights are introduced to calculate the time correlation between historical samples and current samples in the sliding window according to the sampling interval. The deep denoising features extracted by SSRDAE are adaptively calculated according to the interval with the current moment to adjust the influence of features at different moments on the current moment, and the intermediate vector is obtained by fusion. Finally, this intermediate vector is used as the input of LSTM to expand the receptive field of LSTM to capture long-term historical features, so that time series data can be better described. Summary of the invention
[0006] 1. A soft measurement method SSRDAE-IALSTM for industrial processes with uneven sampling intervals based on a denoising attention network, characterized in that: the process data with uneven sampling intervals and containing noise are reconstructed to obtain denoised static features, and weights are assigned to static features with different sampling intervals through an interval attention function; the denoised static features are weightedly fused to perform model training for soft measurement tasks on quality variables, and the data with uneven sampling intervals collected online are input into the trained and tested soft measurement model for online prediction, ultimately obtaining a real-time soft measurement effect.
[0007] 2. The method of claim 1, wherein the method is characterized in that:
[0008] The process of obtaining the denoising static feature is as follows: Assuming that the original data of the process data variable is Where N and T represent the number of process variables and sample length, respectively. The denoising network is applied to industrial process data containing a lot of noise to extract denoised static data features. By decoding the original data layer by layer, the problem of reconstruction error accumulation in the traditional stacked denoising autoencoder is prevented while extracting denoising features. In addition, due to the involvement of quality variables, the extracted features are more suitable for soft measurement modeling. Through the above operations, the quality-related denoising static features H can be obtained. l The sample dimension of the feature is consistent with the sample dimension of the input process data, and the extracted denoised static representation is l represents the number of network layers, and d represents the feature dimension.
[0009] 3. The method of soft sensing of industrial processes with uneven sampling intervals based on denoising attention network SSRDAE-IALSTM according to claim 2, characterized in that:
[0010] The process of quality prediction is: the above features after assigning weights to static features of different sampling intervals through the interval attention function are fused, and the obtained features are used as the input of LSTM. It effectively controls the flow of input information and the retention of memory by introducing three "gate" structures (forget gate, input gate and output gate) and a cell state, so that it performs well when processing long time series data. Based on its ability to capture long-term dependencies of time series data. By continuously updating cell states and hidden states, LSTM can learn the time patterns and trends in the data and predict future values. After the above fused feature input, the receptive field of LSTM is expanded so that long-term historical features can be captured to better describe time series data and realize the prediction of quality variables.
[0011] 4. The method of claim 3 for soft sensing of industrial processes with uneven sampling intervals based on a denoising attention network, SSRDAE-IALSTM, is characterized by:
[0012] The network mainly consists of two parts, namely a stacked supervised reconstruction input denoising autoencoder (SSRDAE) and an LSTM with interval attention (IALSTM). First, the original data is denoised using SSRDAE. Unlike the traditional DAE, each output layer of SRDAE always seeks to reconstruct the original input data and quality data. In this way, the information loss of the original industrial data can be reduced, so that the learned deep features can represent the original industrial input data to a large extent, which can achieve better denoising effect than the traditional SDAE. Then, the contribution of historical samples to the current samples is adjusted by adaptively learning the attention weights at the sampling interval, which better adapts to the time series data with uneven sampling moments, improves the defect that the traditional method can only capture the influence of neighboring samples, and achieves the purpose of capturing long-term historical features.
[0013] 5. The method of claim 3 for soft sensing of industrial processes with uneven sampling intervals based on a denoising attention network, SSRDAE-IALSTM, is characterized by:
[0014] The SSRDAE-IALSTM model built based on the denoising attention network includes an input layer, a hidden layer, a feature fusion layer, and an output layer. The input layer includes the original industrial process data, and the hidden layers use sigmoid as the activation function. The calculation process of the first hidden layer is: in It is obtained by adding noise to the original input data X. W1 and b1 are the weight matrix and bias vector of the first layer encoder; the original input data reconstructed by the output layer of SRDAE1 and quality data for: in and is the weight matrix and bias vector of the first layer decoder.
[0015] The calculation process of the second hidden layer is: in is obtained by adding noise to H1, W2 and b2 are the weight matrix and bias vector of the second layer encoder; the original input data reconstructed by the output layer of SRDAE1 and quality data for: in and is the weight matrix and bias vector of the second layer decoder.
[0016] The calculation process of the lth hidden layer is: all l SIDAE modules are trained in sequence and stacked one by one until the lth SIDAE is obtained. Therefore, the features of the original input data can be gradually learned from the lowest hidden layer H1 to the highest hidden layer H l Then the hidden features after adding noise through the lth layer Continue to seek to reconstruct the original input data and quality data in It is to H l-1 After adding noise, W l and b l is the weight matrix and bias vector of the l-th layer encoder; the original input data reconstructed by the output layer of SRDAE1 and quality data for: in and is the weight matrix and bias vector of the l-th layer decoder.
[0017] The reconstruction error of each layer above is:
[0018] The calculation process of the feature fusion layer is: in is the attention weight of the jth hidden state. V j and U j is the adaptive weight matrix, b j is the bias vector. j Yes t-k+j and h k The interval between samples x t-k+j and x k The interval between them is fused to obtain the intermediate vector C t ,in Once the intermediate vector C at time t is obtained t , and compare it with the given target sequence {y0,y1,…,y t-1}, the hidden state S of LSTM can be updated through the states of input gate, forget gate, output gate, intermediate state and memory state (t) .
[0019] The update method of the forget gate is f (t) =σ(W f [y t-1 ,S t-1 ,C t ]+b f ), the input gate is updated as i (t) =σ(Wi [y t-1 ,S t-1 ,C t ]+b i ), the output gate is updated as o (t) =σ(W o [y t-1 ,S t-1 ,C t ]+b o ), the intermediate state is The memory status is After getting these, we can update the hidden state of LSTM: (t) =o (t) ⊙tanh(c(t)), where W f , W i , W o , W c 、b f 、b i 、b o 、b c is a relevant parameter.
[0020] The calculation process of the output layer is: W is the connection weight matrix and b is the bias vector.
[0021] 6. The method of claim 5, wherein the method is characterized in that:
[0022] The model is verified by using the penicillin fermentation process simulation data set and the debutanizer process data set. The training set and test set are set to train and test the soft measurement effect of the model respectively. The test data is used for soft measurement in the trained model. Support vector regression SVR, artificial neural network ANN, stacked autoencoder SAE, stacked denoising autoencoder SDAE, and sample interval perception long short-term memory neural network SIA-LSTM are selected as comparative methods for verification. The effectiveness of the proposed model and the effect of soft measurement are verified by R 2 , MAE, and RMSE evaluation. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 .The overall framework diagram of the present invention.
[0024] Figure 2 .Schematic diagram of the denoising autoencoder structure with stacked supervised reconstruction input.
[0025] Figure 3 .Schematic diagram of industrial process soft measurement of the present invention. DETAILED DESCRIPTION
[0026] The present invention is further explained in detail below in conjunction with specific implementations. Figure 1-Figure 3 This paper includes the overall framework diagram of the present invention, the schematic diagram of the denoising autoencoder structure with stacked supervised reconstruction input, and the schematic diagram of the industrial process soft measurement of the present invention. This method inherits the advantages of the denoising autoencoder in denoising, while making up for its information loss and the shortcomings of extracting quality-independent features. In view of the problem of uneven sampling intervals of industrial process data, the interval attention mechanism is used to adjust the influence of different intervals, while capturing long-term historical features, so as to improve the accuracy of industrial process soft measurement. Figure 1 As shown in the figure, it includes three parts: data acquisition, denoising feature extraction (SSRDAE) and model prediction (IALSTM). The original data of the industrial process contains noise and the sampling interval is uneven. First, SSRDAE denoises the process data to obtain quality-related denoised static features. Secondly, the denoised features are input into the interval attention LSTM (IALSTM) to adjust the impact of different historical samples in the irregular sampling data on the current sample, thereby capturing long-term time series features to improve the accuracy of quality prediction.
[0027] The detailed steps of the present invention are as follows:
[0028] Step 1: Obtain industrial process data and normalize it.
[0029] In the actual industrial process, multi-channel sensors are used to randomly collect parameters of process variables and quality variables such as equipment parameters and material parameters in the industrial production process, thereby obtaining time series data with uneven sampling intervals, where the original process variable is X and the quality variable is Y. Noise is added to the original process variables in the training set.
[0030] Step 2: De-noise the data obtained in step 1. By decoding the original data layer by layer, the problem of reconstruction error accumulation in traditional SDAE is prevented while extracting denoising features. In addition, due to the participation of quality variables, the extracted features are more suitable for soft measurement modeling. The calculation process of the denoising autoencoder based on stacked supervised reconstruction input (SSRDAE) method is as follows:
[0031] Step 2.1: SRDAE1 obtains the feature H1 of the first hidden layer by minimizing the reconstruction error, where the minimum reconstruction error is The hidden features are
[0032] Step 2.2: Use H1 as the input of SRDAE2 after adding noise, and obtain the feature H2 of the second hidden layer by minimizing the reconstruction error.
[0033] Step 2.3: Train all SRDAEs in a progressive manner, such as Figure 3 As shown, the final quality-related denoising feature H is obtained l ,in
[0034] Step 3: The denoised features are sent to the interval attention LSTM (IALSTM) to adjust the impact of different historical samples in the sliding window on the current sample, thereby solving the problem of irregular sampling and capturing long-term time series features at the same time, and then predicting the quality.
[0035] Step 3.1: Through the operation in step 2, the input data with uneven sampling intervals can be used to extract denoised data features through the denoising network. Then, a sliding window of size k is used to perform moving interception with a step size of 1, and T-k+1 groups of N×k feature sets F={F1,F2,…,F T-k+1},in
[0036] Step 3.2: Then introduce the attention weight to distinguish the temporal correlation between historical samples and current samples according to the sampling interval. For time t, the attention weight is adaptively calculated based on the interval between the deep denoising features extracted by the denoising network and the current time.
[0037] Step 3.3: Calculate the weighted sum of all hidden states to obtain the fusion feature C t ,in Then, the fused features are used as the input data of IALSTM and compared with the given target sequence {y0,y1,…,y t-1}, we can update the hidden state S of LSTM (t) :The calculation process is:
[0038] f (t) =σ(W f [y t-1 ,S t-1 ,C t ]+b f )
[0039] i (t) =σ(W i [y t-1 ,S t-1 ,C t ]+b i )
[0040] o (t) =σ(W o [y t-1 ,St-1 ,C t ]+b o )
[0041]
[0042] After obtaining the forget gate, input gate and previous cell state, the current cell state and hidden state can be updated. S (t) =o (t) ⊙tanh(c(t)). Where W f , W i , W o , W c 、b f 、b i 、b o 、b c is a relevant parameter.
[0043] Step 3.4: Finally, the prediction output can be calculated based on a fully connected layer Where W is the connection weight matrix and b is the bias vector.
[0044] Step 4: During the training process, the quality prediction results are compared with the true values of the quality variables in the training set. The root mean square error is selected as the loss function to continuously optimize the model parameters until the model converges.
[0045] Step 5: The model is verified by the penicillin fermentation process simulation data set and the debutanizer process data set. The training set and test set are set to train and test the soft measurement effect of the model respectively. The test data is used for soft measurement in the trained model. Support vector regression SVR, artificial neural network ANN, stacked autoencoder SAE, stacked denoising autoencoder SDAE, and sample interval perception long short-term memory neural network SIA-LSTM are selected as the control group of the experiment to verify the effectiveness of the proposed model. The experimental hardware parameters are as follows:
[0046] Table 1 Experimental hardware parameters
[0047]
[0048] Experiment 1: Use the penicillin fermentation process data set as the simulation experimental data for this experiment. The reaction time is set to 400h. Randomly select the sampling interval of 0.02h-0.8h in chronological order, and obtain a total of 937 samples with uneven sampling intervals. Select the concentration of penicillin as the quality variable. The remaining 11 variables are auxiliary variables for experimental verification. Select the first 600 samples as the training set and the last 337 samples as the test set. For the quality measurement results of the model, this experiment uses three evaluation indicators:
[0049] (1) Goodness of fit
[0050] (2) Mean absolute error
[0051] (3) Root mean square error
[0052] The soft sensor experimental results of penicillin fermentation process simulation are shown in Table 2.
[0053] Table 2 Evaluation indexes of soft measurement results of quality variables in penicillin fermentation process by various methods
[0054]
[0055] Experiment 2: The debutanizer dataset is used as the real experimental data for this experiment. Due to sampling restrictions, the process is sampled at intervals between 12 and 60 minutes, resulting in a total of 675 samples with irregular sampling intervals. Seven typical process variables measured by hardware sensors were selected, and butane content was used as the quality variable for soft measurement. The first 450 samples were selected as the training set, and the last 225 samples were selected as the test set. Table 3 shows the soft measurement experimental results of the debutanizer process.
[0056] Table 3 Evaluation indexes of soft measurement results of quality variables in the debutanizer process by each method
[0057]
[0058] From the results in Tables 2 and 3, it can be seen that traditional machine learning models such as SVR and ANN have relatively poor prediction effects on complex data due to their shallow network structures. Deep learning models such as SAE and SDAE improve prediction accuracy by extracting deep abstract features of raw data, surpassing traditional shallow network models. However, as static models, these methods ignore the dynamic nature of the industrial process itself. In contrast, the SIALSTM model can effectively process data with uneven sampling intervals and capture the temporal correlation by introducing interval-aware functions. However, in real data with noise, the prediction value of SIALSTM still fluctuates greatly. The SSIDAE-IASLTM model proposed in this paper recursively reconstructs the original input and output data, and introduces an attention mechanism to integrate a wide range of historical influences, so that the model has achieved significant improvements in prediction performance when processing dynamic data and noisy data.
Claims
1. A SSRDAE-IALSTM soft-sensing method for industrial processes with uneven sampling intervals based on a denoising attention network, characterized by: The process data with uneven sampling intervals and noise is reconstructed to obtain denoised static features, and weights are assigned to static features with different sampling intervals through the interval attention function; After weighted fusion of denoised static features, the model of soft measurement task is trained for quality variables. The data with uneven sampling intervals collected online are input into the trained and tested soft measurement model for online prediction, and finally a real-time soft measurement effect is obtained.
2. The method of claim 1, wherein the method is characterized in that: The process of obtaining the denoising static feature is as follows: Assuming that the original data of the process data variable is Where N and T represent the number of process variables and sample length, respectively. The denoising network is applied to industrial process data containing a lot of noise to extract denoised static data features. By decoding the original data layer by layer, the problem of reconstruction error accumulation in the traditional stacked denoising autoencoder is prevented while extracting denoising features. In addition, due to the involvement of quality variables, the extracted features are more suitable for soft measurement modeling. Through the above operations, the quality-related denoising static features H can be obtained. l The sample dimension of the feature is consistent with the sample dimension of the input process data, and the extracted denoised static representation is l represents the number of network layers, and d represents the feature dimension.
3. The method of soft sensing of industrial processes with uneven sampling intervals based on denoising attention network SSRDAE-IALSTM according to claim 2, characterized in that: The process of quality prediction is: the above features after assigning weights to static features of different sampling intervals through the interval attention function are fused, and the obtained features are used as the input of LSTM. It effectively controls the flow of input information and the retention of memory by introducing three "gate" structures (forget gate, input gate and output gate) and a cell state, so that it performs well when processing long time series data. Based on its ability to capture long-term dependencies of time series data. By continuously updating cell states and hidden states, LSTM can learn the time patterns and trends in the data and predict future values. After the above fused feature input, the receptive field of LSTM is expanded so that long-term historical features can be captured to better describe time series data and realize the prediction of quality variables.
4. The method of claim 3 for soft sensing of industrial processes with uneven sampling intervals based on a denoising attention network, SSRDAE-IALSTM, is characterized by: The network mainly consists of two parts, namely a stacked supervised reconstruction input denoising autoencoder (SSRDAE) and an LSTM with interval attention (IALSTM). First, the original data is denoised using SSRDAE. Unlike the traditional DAE, each output layer of SRDAE always seeks to reconstruct the original input data and quality data. In this way, the information loss of the original industrial data can be reduced, so that the learned deep features can represent the original industrial input data to a large extent, which can achieve better denoising effect than the traditional SDAE. Then, the contribution of historical samples to the current samples is adjusted by adaptively learning the attention weights at the sampling interval, which better adapts to the time series data with uneven sampling moments, improves the defect that the traditional method can only capture the influence of neighboring samples, and achieves the purpose of capturing long-term historical features.
5. The method of claim 3 for soft sensing of industrial processes with uneven sampling intervals based on a denoising attention network, SSRDAE-IALSTM, is characterized by: The SSRDAE-IALSTM model built based on the denoising attention network includes an input layer, a hidden layer, a feature fusion layer, and an output layer. The input layer includes the original industrial process data, and the hidden layers use sigmoid as the activation function. The calculation process of the first hidden layer is: in It is obtained by adding noise to the original input data X. W1 and b1 are the weight matrix and bias vector of the first layer encoder; the original input data reconstructed by the output layer of SRDAE1 and quality data for: in and is the weight matrix and bias vector of the first layer decoder. The calculation process of the second hidden layer is: in is obtained by adding noise to H1, W2 and b2 are the weight matrix and bias vector of the second layer encoder; the original input data reconstructed by the output layer of SRDAE1 and quality data for: in and is the weight matrix and bias vector of the second layer decoder. The calculation process of the lth hidden layer is: all l SIDAE modules are trained in sequence and stacked one by one until the lth SIDAE is obtained. Therefore, the features of the original input data can be gradually learned from the lowest hidden layer H1 to the highest hidden layer H l Then the hidden features after adding noise through the lth layer Continue to seek to reconstruct the original input data and quality data in It is to H l-1 After adding noise, W l and b l is the weight matrix and bias vector of the l-th layer encoder; the original input data reconstructed by the output layer of SRDAE1 and quality data for: in and is the weight matrix and bias vector of the l-th layer decoder. The reconstruction error of each layer above is: The calculation process of the feature fusion layer is: in is the attention weight of the jth hidden state. V j and U j is the adaptive weight matrix, b j is the bias vector. j Yes t-k+j and h k The interval between samples x t-k+j and x k The interval between them is fused to obtain the intermediate vector C t ,in Once the intermediate vector C at time t is obtained t , and compare it with the given target sequence {y0,y1,…,y t-1 }, the hidden state S of LSTM can be updated through the states of input gate, forget gate, output gate, intermediate state and memory state (t) . The update method of the forget gate is f (t) =σ(W f [y t-1 ,S t-1 ,C t ]+b f ), the input gate is updated as i (t) =σ(W i [y t-1 ,S t-1 ,C t ]+b i ), the output gate is updated as o (t) =σ(W o [y t-1 ,S t-1 ,C t ]+b o ), the intermediate state is The memory status is After getting these, we can update the hidden state of LSTM: (t) =o (t) ⊙tanh(c(t)), where W f , W i , W o , W c 、b f 、b i 、b o 、b c is a relevant parameter. The calculation process of the output layer is: W is the connection weight matrix and b is the bias vector.
6. The method of claim 5, wherein the method is characterized in that: The model is verified by using the penicillin fermentation process simulation data set and the debutanizer process data set. The training set and test set are set to train and test the soft measurement effect of the model respectively. The test data is used for soft measurement in the trained model. Support vector regression SVR, artificial neural network ANN, stacked autoencoder SAE, stacked denoising autoencoder SDAE, and sample interval perception long short-term memory neural network SIA-LSTM are selected as comparative methods for verification. The effectiveness of the proposed model and the effect of soft measurement are verified by R 2 , MAE, and RMSE evaluation.