Long-time prediction method for key parameters of industrial processes

Abstract

The application relates to a long-time prediction method for key parameters of an industrial process, which takes the original variable data at multiple continuous time points as the input of a subsequent prediction model after preprocessing and standardization. Through a multi-attention module and a time convolution network (TCN) module, the prediction model effectively mines the correlation between the potential output and the input variables, the correlation between the output and the input time points, and the dynamic nonlinear characteristics of the output variables. Finally, real industrial process data is collected for experiments, sample data sets are obtained by selecting variable data at corresponding time points, and the sample data sets are proportionally divided for training, verification and testing. The application, in view of the strong coupling and dynamic nonlinearity of key parameters in an industrial process, combines a double-attention module and a TCN module, overcomes the limitations of existing methods that cannot effectively solve feature coupling and cannot sufficiently mine time dynamic characteristics, and has theoretical and practical significance for the prediction of key parameters of an industrial process.

Description

Technical Field

[0001] This invention belongs to the field of industrial process time series prediction technology, specifically a long-term prediction method for key parameters of industrial processes. Background Technology

[0002] With the rapid development of global industry and information technology, industrial production processes are becoming increasingly sophisticated, and production scale is constantly expanding. Simultaneously, time-series data on key parameters of industrial production processes, such as flame temperature in kiln sintering, molten iron quality in blast furnace ironmaking, and heating temperature in coal-fired boilers, reveal to some extent the characteristics and patterns of changes in actual production conditions. Therefore, time-series modeling and prediction of key parameters in industrial processes can effectively uncover the patterns of changes in industrial production conditions. This is crucial for achieving high-quality, high-efficiency, and high-yield stable operation of industrial processes and helps enterprises transform and upgrade towards green and intelligent directions.

[0003] In the field of time series forecasting, if the time series is complex, a single model often cannot capture different time series patterns simultaneously. To overcome the limitations of a single model, experts have proposed various combination techniques. The basic idea of ​​model combination in forecasting tasks is to utilize the unique features of each model to capture different patterns in the data. Therefore, to capture different distribution patterns in the data, hybrid models combining deep learning have become a development trend. Especially for complex industrial processes, hybrid models combining deep learning can better uncover dynamic features over time and resolve coupling issues between multiple variables. Summary of the Invention

[0004] This invention provides a long-term prediction method for key parameters of industrial processes. It uses a multi-attention mechanism and a deep hybrid neural network TCN to automatically extract spatiotemporal coupling features and dynamic time series features from multivariate thermal time series of industrial processes, and accurately predict the future values ​​of key parameters.

[0005] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0006] A long-term prediction method for key parameters in industrial processes includes the following steps:

[0007] Step 1: Collect key parameter data of the industrial process and perform preprocessing;

[0008] Step 2: Input processing: Combine the variable values ​​at m historical time points and the predicted target value t seconds later to form a sample data; where m is the number of input historical time points and t is the prediction time point;

[0009] Step 3: Divide the sample dataset into training dataset, validation dataset, and prediction dataset according to the proportions, and then perform standardization.

[0010] Step 4: Establish a deep hybrid network prediction model based on multi-attention and TCN. This model uses the multi-attention mechanism to mine coupling features and temporal dynamic features, uses TCN to extract the temporal dynamic features and nonlinear change features of the variable to be tested, and finally combines the two features to complete the prediction.

[0011] Step 5: Train a deep neural network using the training dataset to obtain the corresponding weights and biases. Input the obtained weights and biases into the validation dataset, calculate the prediction error of the validation dataset, and save the weights and biases that minimize the prediction error of the validation dataset. Select the AdamW optimizer, use the mean squared error as the loss function, and backpropagate to adjust the parameters of the prediction model for key parameters of the industrial process, thereby improving the prediction accuracy of the prediction model on the test set.

[0012] Step 6: Substitute the weights and biases that minimize the prediction error on the validation set from the previous step into the prediction model, and then apply them to the prediction dataset to calculate the prediction value of the prediction model on the prediction dataset.

[0013] The preprocessing includes the following steps:

[0014] Step 1-1: Remove outliers;

[0015] Steps 1-2: Filter the variables to be predicted.

[0016] The establishment of a hybrid network prediction model based on multi-attention and TCN includes the following steps:

[0017] Step 4-1: Determine the model input dimension: batchsize × m × n fea tures， `batchsize` is the batch size set during model batch training, `m` is the number of input historical time points, and `n` is the number of input historical time points. fea tures Let n be the number of features at each time step of the input. fea tures It is 24;

[0018] Step 4-2: Determine the structural parameters of the prediction model based on the model input dimensions obtained in Step 4-1.

[0019] The multi-attention mechanism includes: utilizing the SpatialAttention Module and Channel Attention Module from the field of computer vision to mine time-series features.

[0020] Multi-attention mechanisms specifically include:

[0021] Step 4-3: Generate a channel attention map by utilizing the inter-channel relationships of features;

[0022] First, spatial information from the feature map is aggregated using average pooling and max pooling operations to generate two distinct spatial features: and They represent the average pooling feature and the max pooling feature, respectively;

[0023] These two features are then fed into a shared network to generate a channel attention map. The shared network consists of a multilayer perceptron (MLP) and a hidden layer; to reduce parameter overhead, the hidden layer size is set to... Where r is the reduction rate;

[0024] After applying the shared network to each feature, the output feature vector is merged using element-wise summation; the channel attention is calculated as follows:

[0025] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0026] Step 4-4: Generate a spatial attention map by utilizing the spatial relationships of features;

[0027] First, average pooling and max pooling operations are applied along the channel axis. By using these two pooling operations, the channel information of the feature map is aggregated, generating two two-dimensional maps: and Each two-dimensional graph represents the average set feature and the maximum set feature of the entire channel;

[0028] Then, these are connected and convolved using standard convolutional layers to generate a two-dimensional spatial attention map; the spatial attention calculation is as follows:

[0029] M s (F)=σ(f([AvgPool(F);MaxPool(F)]))

[0030] f represents the convolution operation, and σ is the activation function.

[0031] The Temporal Convolutional Module (TCN) effectively mines the temporal dynamics and nonlinear features of the variable to be predicted by utilizing causal convolution.

[0032] The training and computation process for hybrid network prediction models includes the following steps:

[0033] Step 5-1: Determine the hyperparameters of the neural network: learning rate lr, maximum number of iterations I. MAX Randomly initialize the weight matrix w and bias β of each network layer, and set the initial iteration number I = 0;

[0034] Step 5-2: Perform a reshape operation on the input two-dimensional matrix Xy = [[X(t), ..., X(tm)], [y(t), ..., y(tm)]]:

[0035] The input time points are divided into multiple time segments in chronological order into a three-dimensional matrix Xy′=[[[X(t),...,X(tm / sec)],[y(t),...,y(tm / sec)]],…,[[X(tm(sec-1) / sec),...,X(tm)],[y(tm(sec-1) / sec),...,y(tm)]]], where X is the input variable excluding the variable to be predicted, y is the variable to be predicted, m is the number of historical time points, and sec is the number of time segments;

[0036] Meanwhile, the variable to be predicted, Y = [y(t), ..., y(ym], is used as the input to the TCN layer to mine the time dynamics and nonlinear variation characteristics of the variable to be predicted in the industrial process. Its calculation process can be written as follows:

[0037] F tcn =TCN(Y)

[0038] Step 5-3: Calculate the channel attention of the above input 3D matrix:

[0039]

[0040] The final feature maps for each time segment are calculated using the channel attention:

[0041]

[0042] Wherein, F has the same shape as Xy′,

[0043] Step 5-4: Use a one-dimensional convolution to extract the overall features of each feature at each time point within the time period:

[0044]

[0045] in, This represents the time-time coupled features obtained from a one-dimensional convolution operation, where * denotes the convolution operation, and... This is the weight of the Nth filter, with a step size of 1 and a size of sec×1, where sec is the number of time intervals, and β. N This refers to the bias of the convolutional layer. ReLU is the ReLU activation function, expressed as ReLU(x) = max(0, x). Ignoring boundary padding, the output size of this one-dimensional convolutional layer is 1×i, where i is the number of input features.

[0046] Step 5-5: Obtain time-time coupling features Subsequently, to uncover the long-term dependencies and spatiotemporal coupling features of the time series, the obtained time-segment coupling features are input into a parallel dual-attention module. This module consists of a parallel channel attention module and a spatial attention module, thereby better uncovering the spatiotemporal coupling features. The calculation process is as follows:

[0047]

[0048]

[0049]

[0050]

[0051] F attention =MLP(concat(F c F s ))

[0052] f 3×3 This indicates a convolution operation with a 3×3 kernel. `concat` indicates a tensor connection operation. `MLP` stands for fully connected layer, primarily used because the obtained feature M has a very large dimension, and a fully connected layer maps it to a low-dimensional tensor.

[0053] Steps 5-6: Obtain the spatiotemporal coupling features F obtained through the above steps for multi-attention mechanism mining. attention and time dynamic features F tcn Finally, the weighted fusion features of the fully connected layer are used to obtain the final output;

[0054] y = f(w) a F attention +w t F tcn +β)

[0055] y represents the model's final predicted value; w a w t denoted as the transformation matrix; β represents the bias value; and f is the nonlinear activation function.

[0056] The present invention has the following beneficial effects and advantages:

[0057] A long-term prediction method for key parameters in industrial processes is proposed. This invention utilizes a deep hybrid neural network combining a multi-attention mechanism and a Transient Neural Network (TCN) to effectively mine the potential temporal dynamics and spatiotemporal coupling features in industrial process time series. Furthermore, to uncover long-term dependencies in the time series and improve the model's long-term prediction performance, a time-segmented input modeling approach is adopted. A channel attention mechanism is used to mine the correlation between the output and time periods, and one-dimensional convolution is used to mine the overall features within each segment. This is equivalent to using the weighted overall features within the time segment to replace the features of individual time points, thereby better establishing long-term dependencies. Attached Figure Description

[0058] Figure 1 This is a schematic diagram illustrating an industrial example of the method of the present invention.

[0059] Figure 2 This is a flowchart of the method of the present invention.

[0060] Figure 3 This is a flowchart of the network training process in this invention.

[0061] Figure 4 This is a comparison chart of the prediction errors of different models on the test set.

[0062] Figure 5 This is a comparison chart of the prediction results of the method of the present invention with those of other models. Detailed Implementation

[0063] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0065] like Figure 1-2 The diagram shown is a schematic diagram of an industrial example and a flowchart of the method of the present invention.

[0066] A long-term prediction method for key parameters of industrial processes is proposed. The method involves preprocessing and standardizing industrial process data to form three-dimensional model input data. The model combines a multi-attention mechanism and a temporal convolutional network (TCN) to address the multivariate, strongly coupled, and dynamically nonlinear characteristics of the industrial process data. The multi-attention mechanism effectively mines temporal coupling features and long-term dependencies, while the TCN extracts the dynamic nonlinear features of the variables to be predicted. Finally, a weighted fusion method is used to complete the long-term prediction of key parameters of the industrial process. The programming language used in the execution steps of this invention is not limited to MATLAB or Python.

[0067] The specific steps of this invention are as follows:

[0068] Step 1: Collect key parameters of the industrial process. Taking an actual loose rehydration process as an example, the material is conveyed to the drum, where humidifying water is atomized by high-temperature steam, allowing the material to fully absorb moisture. The operator adjusts the amount of humidifying water based on the real-time readings from various sensors during this process, thereby controlling the outlet moisture content to meet the target. Key parameters in this process mainly include the inlet moisture content of the material at the drum inlet, the outlet moisture content of the material at the drum outlet, and 24 variables involved in the process, such as the actual water addition value, water valve opening, actual outlet temperature, water addition setpoint, actual temperature of material bucket 1, actual temperature of material bucket 2, and water addition coefficient (where material buckets are used to hold auxiliary materials, and the number of material buckets is set according to actual conditions). The key parameter to be predicted is the outlet moisture content. The collected variable data is then preprocessed.

[0069] Step 1-1: Remove outliers according to the set error range;

[0070] Steps 1-2: Filter the variables to be predicted; Example of filtering method: wavelet thresholding denoising.

[0071] Step 2: Input processing is performed on the preprocessed data, which involves combining the variable values ​​at m historical time points and the predicted target value t seconds later to form a sample data point. Here, m is the number of input historical time points, and t is the prediction time point.

[0072] Step 3: Divide the sample data set collected in the previous step into a 70% training dataset, a 10% validation dataset, and a 20% prediction dataset. Then perform a standardization operation, as shown in the following formula:

[0073]

[0074] Where, mean(x) i ) and std(x i ) represent variables x respectively i The mean and variance of.

[0075] Step 4: Establish a hybrid network prediction model based on multi-attention and TCN. Use the multi-attention mechanism to mine coupling features and temporal dynamic features, use TCN to extract the temporal dynamic features and nonlinear change features of the key parameters to be predicted, and finally combine the two effectively to complete the prediction task.

[0076] Step 4-1: Determine the model input dimension: batchsize × m × n fea tures， `batchsize` is the batch size set during model batch training, `m` is the number of input historical time points, and `n` is the number of input historical time points. features Let n be the number of features at each time step of the input. fea tures It is 24.

[0077] Step 4-2: Determine the structural parameters of the prediction model based on the model input dimensions obtained in Step 4-1.

[0078] Step 4-3: Generate channel attention maps by utilizing the inter-channel relationships of features. First, spatial information from the feature maps is aggregated using average pooling and max pooling operations to generate two distinct spatial features: and These represent the average pooling feature and the max pooling feature, respectively. These two features are then fed into a shared network to generate a channel attention map. The shared network consists of a multilayer perceptron (MLP) and a hidden layer. To reduce parameter overhead, the hidden layer size is set to... Where r is the reduction ratio. After applying the shared network to each feature, we use element-wise summation to merge the output feature vectors. In short, channel attention is calculated as follows:

[0079] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))) (2)

[0080] Where F is the input to channel attention; M c (F) represents the output of the channel attention;

[0081] Step 4-4: Generate a spatial attention map by utilizing the spatial relationships of features. To compute the spatial attention map, first apply average pooling and max pooling operations along the channel axis. By using these two pooling operations to aggregate the channel information of the feature map, two two-dimensional maps are generated: and Each 2D map represents the average and maximum ensemble features of the entire channel. These are then concatenated and convolved using a standard convolutional layer to generate our 2D spatial attention map. In short, spatial attention is calculated as follows:

[0082] Ms (F)=σ(f([AvgPool(F);MaxPool(F)])) (3)

[0083] f represents the convolution operation, and σ is the activation function.

[0084] Step 5: Train a deep neural network using the training dataset to obtain the corresponding weights and biases. Input these weights and biases into the validation dataset, calculate the prediction error of the validation dataset, and save the weights and biases that minimize the prediction error. Use the AdamW optimizer with mean squared error as the loss function, and backpropagate to adjust the parameters of the industrial process key parameter prediction model, thereby improving the prediction accuracy of the model on the test set. Figure 3 The diagram shown is a flowchart of the network training process in this invention.

[0085] Step 5-1: Determine the hyperparameters for neural network training: learning rate lr, maximum number of iterations I. MAX Randomly initialize the weight matrix w and bias β of each network layer, and set the initial iteration number I = 0.

[0086] Step 5-2: Perform a reshape operation on the input two-dimensional matrix Xy = [[X(t), ..., X(tm)], [y(t), ..., y(tm)]] to divide the input time points into multiple time segments in chronological order. Specifically, the input two-dimensional matrix is ​​converted into a three-dimensional matrix Xy′ = [[[X(t), ..., X(tm / sec)], [y(t), ..., y(tm / sec)]], ..., [[X(tm(sec-1) / sec), ..., X(tm)], [y(tm(sec-1) / sec), ..., y(tm)]]], where X is the input variable (except for the outlet moisture content), y is the outlet moisture content, m is the number of input historical time points, and sec is the number of time segments.

[0087] Meanwhile, the variable to be predicted, Y = [y(t), ..., y(ym], is used as the input to the TCN layer to mine the time dynamics and nonlinear variation characteristics of the key parameters to be predicted in the industrial process. Its calculation process can be written as follows:

[0088] F tcn =TCN(Y) (4)

[0089] Step 5-3: Then calculate the channel attention of the above input 3D matrix:

[0090]

[0091] The channel attention obtained above This allows us to uncover the correlation between the output and the segmented time periods, and determine which part of the input needs to be focused on. Then, we use this channel attention to calculate the final feature maps for each time segment.

[0092]

[0093] Wherein, F has the same shape as Xy′,

[0094] Step 5-4: To effectively mine the long-term dependencies of key parameters in industrial processes, it is necessary to pay more attention to the correlations between each time period, while the specific time points within each time period do not require much attention. Therefore, after obtaining the time period feature map F, it is not directly input into the subsequent structure, but instead, a one-dimensional convolution is used to mine the overall features of each feature at each time point within the time period.

[0095]

[0096] in, This represents the time-time coupled features obtained from a one-dimensional convolution operation, where * denotes the convolution operation, and... This is the weight of the Nth filter, with a step size of 1 and a size of sec×1, where sec is the number of time intervals, and β. N This refers to the bias of the convolutional layer. ReLU is the ReLU activation function, expressed as ReLU(x) = max(0, x). Ignoring boundary padding, the output size of this one-dimensional convolutional layer is 1×i, where i is the number of input features, i.e., n. fea tures .

[0097] Step 5-5: Obtain time-time coupling features Subsequently, to uncover the long-term dependencies and spatiotemporal coupling features of the time series, the obtained time-segment coupling features are input into a parallel dual-attention module. This module consists of a parallel channel attention module and a spatial attention module, thereby better uncovering the spatiotemporal coupling features. The calculation process is as follows:

[0098]

[0099]

[0100]

[0101]

[0102] F attention =MLP(concat(F c F s (12)

[0103] f 3×3This indicates a convolution operation with a 3×3 kernel. `concat` represents a tensor connection operation. `MLP` stands for fully connected layer, primarily used because the obtained feature M has a very large dimension, and a fully connected layer maps it to a low-dimensional tensor.

[0104] Steps 5-6: Through steps 7-2 and 7-5 above, the spatiotemporal coupling feature F of the multi-attention mechanism mining is obtained. attention and time dynamic features F tcn Therefore, the final output is obtained by using the weighted fusion features of the fully connected layer.

[0105] y = f(w) a F attentipn +w t F tcn +β) (13)

[0106] y represents the model's final predicted value; w a wt represents the transformation matrix; β represents the bias value; and f is the nonlinear activation function.

[0107] Step 6: Substitute the weights and biases that minimize the prediction error on the validation set from the previous step into the prediction model, and then apply them to the prediction dataset to calculate the prediction value of the prediction model on the prediction dataset.

[0108] The results of the above methods are as follows Figure 4 , 5 As shown. Figure 4 A comparison chart showing the average absolute error of predicting export moisture content in the next 60, 90, and 120 seconds using historical data of the 120s industrial loose rehydration process. Figure 5 The chart shows a comparison between the predicted and actual 60s outlet moisture content of each model in this process. It is evident that this method can effectively predict key parameters in the loosening and rehydration process, improving the accuracy of these predictions. Furthermore, the future values ​​of key parameters in the loosening and rehydration process can be directly reflected on the operator's control interface, generating adjustment suggestions (using the comparison between predicted and target values ​​as a reference to adjust input parameters of other industrial processes related to the predicted parameters, thereby adjusting the future values ​​of the predicted parameters). This guides operator adjustments, improving the stability of the production process and product quality.

[0109] In summary, this invention establishes a deep hybrid model, MACnet, based on a multi-attention mechanism and TCN. The multi-attention mechanism mines long-term dependencies and handles coupling features present in industrial processes, while TCN mines the dynamic temporal features of the variables to be predicted. Finally, weighted fusion is used to complete the long-term prediction of key parameters in industrial processes. This invention effectively solves the problems of strong coupling features and dynamic nonlinear feature extraction in the long-term prediction of complex time series of industrial processes, and has theoretical and practical significance for the long-term prediction of key parameters in industrial processes.

[0110] The embodiments described above will help those skilled in the art to further understand the present invention, but do not limit the present invention in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

Claims

1. A long-term prediction method for key parameters of an industrial process, characterized in that, The steps include the following: Step 1: Collect key parameter data of the industrial process and perform preprocessing; Step 2: Input processing: Combine the variable values ​​at m historical time points and the predicted target value t seconds later to form a sample data; where m is the number of input historical time points and t is the prediction time point; Step 3: Divide the sample dataset into training dataset, validation dataset, and prediction dataset according to the proportions, and then perform standardization. Step 4: Establish a deep hybrid network prediction model based on multi-attention and TCN. This model uses the multi-attention mechanism to mine coupling features and temporal dynamic features, uses TCN to extract the temporal dynamic features and nonlinear change features of the variable to be tested, and finally combines the two features to complete the prediction. Step 5: Train a deep neural network using the training dataset to obtain the corresponding weights and biases. Input the obtained weights and biases into the validation dataset, calculate the prediction error of the validation dataset, and save the weights and biases that minimize the prediction error of the validation dataset. Select the AdamW optimizer, use the mean squared error as the loss function, and backpropagate to adjust the parameters of the prediction model for key parameters of the industrial process, thereby improving the prediction accuracy of the prediction model on the test set. Step 6: Substitute the weights and biases that minimize the prediction error on the validation set from the previous step into the prediction model, and then apply them to the prediction dataset to calculate the prediction value of the prediction model on the prediction dataset.

2. The long-term prediction method for key parameters of industrial processes according to claim 1, characterized in that, The preprocessing includes the following steps: Step 1-1: Remove outliers; Steps 1-2: Filter the variables to be predicted.

3. The long-term prediction method for key parameters of industrial processes according to claim 1, characterized in that, The establishment of a hybrid network prediction model based on multi-attention and TCN includes the following steps: Step 4-1: Determine the model input dimension: batchsize × m × n features batchsize is the batch size set during model batch training, m is the number of input historical time points, and n features Let n be the number of features at each time step of the input. features It is 24; Step 4-2: Determine the structural parameters of the prediction model based on the model input dimensions obtained in Step 4-1.

4. The method for long-term prediction of key parameters in an industrial process according to claim 1, characterized in that, The multi-attention mechanism includes: utilizing the Spatial Attention Module and the Channel Attention Module from the field of computer vision to mine time-series features.

5. The method for long-term prediction of key parameters in an industrial process according to claim 4, characterized in that, Multi-attention mechanisms specifically include: Step 4-3: Generate a channel attention map by utilizing the inter-channel relationships of features; First, spatial information from the feature map is aggregated using average pooling and max pooling operations to generate two distinct spatial features: and They represent the average pooling feature and the max pooling feature, respectively; These two features are then fed into a shared network to generate a channel attention map. The shared network consists of a multilayer perceptron (MLP) and a hidden layer; to reduce parameter overhead, the hidden layer size is set to... Where r is the reduction rate; After applying the shared network to each feature, the output feature vector is merged using element-wise summation; the channel attention is calculated as follows: M c (F)<σ(MLP(AvgPool(F))+MLP(MaxPool(F))) Step 4-4: Generate a spatial attention map by utilizing the spatial relationships of features; First, average pooling and max pooling operations are applied along the channel axis. By using these two pooling operations, the channel information of the feature map is aggregated, generating two two-dimensional maps: and Each two-dimensional graph represents the average set feature and the maximum set feature of the entire channel; Then, these are connected and convolved using standard convolutional layers to generate a two-dimensional spatial attention map; the spatial attention calculation is as follows: M s (F)=σ(f([AvgPool(F);MaxPool(F)])) f represents the convolution operation, and σ is the activation function.

6. The method for long-term prediction of key parameters in an industrial process according to claim 1, characterized in that, The Temporal Convolutional Module (TCN) effectively mines the temporal dynamics and nonlinear features of the variable to be predicted by utilizing causal convolution.

7. The method for long-term prediction of key parameters in an industrial process according to claim 1, characterized in that, The training and computation process for hybrid network prediction models includes the following steps: Step 5-1: Determine the hyperparameters of the neural network: learning rate lr, maximum number of iterations I. MAX Randomly initialize the weight matrix w and bias β of each network layer, and set the initial iteration number I = 0; Step 5-2: Perform a reshape operation on the input two-dimensional matrix Xy = [[X(t),…,X(tm)],[y(t),…,y(tm)]]: The input time points are divided into a three-dimensional matrix Xy in chronological order. ′ = [[[X(t),…,X(tm / sec)],[y(t),…,y(tm / sec)]],…,[[X(tm(sec-1) / sec),…,X(tm)],[y(tm(sec-1) / sec),…,y(tm)]]], where X is the input variable excluding the variable to be predicted, y is the variable to be predicted, m is the number of historical time points, and sec is the number of time periods; Meanwhile, the variable to be predicted, Y = [y(t), ..., y(ym], is used as the input to the TCN layer to mine the time dynamics and nonlinear variation characteristics of the variable to be predicted in the industrial process. Its calculation process can be written as follows: F tcn =TCN(Y) Step 5-3: Calculate the channel attention of the above input 3D matrix: The final feature maps for each time segment are calculated using the channel attention: Wherein, F has the same shape as Xy′, Step 5-4: Use a one-dimensional convolution to extract the overall features of each feature at each time point within the time period: in, This represents the time-time coupled features obtained from a one-dimensional convolution operation, where * denotes the convolution operation, and... This is the weight of the Nth filter, with a step size of 1 and a size of sec×1, where sec is the number of time intervals, and β. N This refers to the bias of the convolutional layer. ReLU is the ReLU activation function, expressed as ReLU(x) = max(0,x). Ignoring boundary padding, the output size of a one-dimensional convolutional layer is 1×i, where i is the number of input features, i.e., n. features ; Step 5-5: Obtain time-time coupling features Subsequently, to uncover the long-term dependencies and spatiotemporal coupling features of the time series, the obtained time-segment coupling features are input into a parallel dual-attention module. This module consists of a parallel channel attention module and a spatial attention module, thereby better uncovering the spatiotemporal coupling features. The calculation process is as follows: F attention =MLP(concat(F c ,F s )) f 3×3 This indicates a convolution operation with a 3×3 kernel; concat indicates a tensor connection operation; MLP indicates a fully connected layer, mainly because the obtained feature M has a large dimension, so a fully connected layer is used to map it onto a low-dimensional tensor. Steps 5-6: Obtain the spatiotemporal coupling features F obtained through the above steps for multi-attention mechanism mining. attention and time dynamic features F tcn Finally, the weighted fusion features of the fully connected layer are used to obtain the final output; y=f(w a F attention +w t F tcn +β) y represents the model's final predicted value; w a w t denoted as the transformation matrix; β represents the bias value; and f is the nonlinear activation function.

Images