Industrial concentration parameter prediction method based on caudformer model

Abstract

This invention discloses a method for predicting industrial concentration parameters based on the CauDformer model, relating to the field of industrial concentration parameter prediction. This method effectively captures complex dependencies in multivariate time series by integrating inverted embedding and causal attention mechanisms into a CauDformer-based prediction model. Furthermore, by combining the frequency domain enhancement characteristics of the CauDformer model, it further enhances the model's ability to capture periodic and volatile features. Simultaneously, the dynamically adjusted combined loss function and the Bayesian optimization strategy implemented using Optuna ensure efficient model training and optimal final performance. This achieves high-precision prediction of industrial concentration parameters, providing key state monitoring capabilities for intelligent management of production processes, significantly improving the monitoring and management level of production processes, and meeting the high-precision requirements for industrial concentration parameter prediction.

Description

Technical Field

[0001] This invention relates to the field of industrial concentration parameter prediction, specifically a method for predicting industrial concentration parameters based on the CauDformer model. Background Technology

[0002] Industrial concentration is a crucial step in modern production. It involves using physical or chemical methods to concentrate certain components of a substance, increasing their concentration or extracting valuable components. The stability and precise control of its process parameters directly impact product quality and production efficiency. This process generates a large number of raw parameters (time-series parameters, equipment status parameters, and process quality standards). Cleaning, polymerizing, compressing, or feature-extracting these raw parameters to form high-value, low-redundancy, and rapidly analyzable parameters are known as industrial concentration parameters. Accurate prediction of industrial concentration parameters can improve industrial production efficiency, safety, and sustainability, which is of great significance in industries such as pharmaceuticals, food processing, papermaking, and chemicals.

[0003] Traditional methods for monitoring and predicting process parameters, such as predictions based on empirical formulas or simple statistical models, rely on summarizing existing experimental parameters or empirical formulas and using mathematical relationships for prediction; or they predict future parameter changes by fitting historical parameter trends. However, methods based on empirical formulas are usually based on simplified assumptions and limited parameters, which leads to limited prediction accuracy. The prediction effect of simple statistical models is highly dependent on the quality and quantity of parameters. If there is noise, missing values, or outliers in the parameters, the prediction results of the model will be affected. In addition, the applicability of these two methods is limited. When the system's operating mode changes or exceeds the original parameter range, it is often difficult to capture complex nonlinear relationships and long-term time-series dependencies, resulting in insufficient prediction accuracy and failing to meet the high-precision requirements for condition monitoring and fine management in modern production processes.

[0004] Existing time series forecasting models, such as traditional machine learning methods, learn relationships between variables from historical parameters through feature engineering and simulation training to predict the future, but their ability to handle complex multivariate inputs is limited. Self-attention models, such as Transformers, capture long-distance dependencies in sequence parameters through attention mechanisms, thus performing well in time series modeling, but they may face challenges in computational complexity and memory consumption when processing long series. Furthermore, most models focus on time-domain information, neglecting the potential predictive value of frequency-domain features in the periodicity and volatility of industrial processes, and lack efficient and systematic optimization strategies for model hyperparameters. Summary of the Invention

[0005] To achieve high-precision prediction of industrial concentration parameters and provide more accurate prediction parameters for intelligent manufacturing, this invention proposes a high-precision prediction method for industrial concentration parameters based on the CauDformer model. This method collects and preprocesses industrial concentration parameters from specific industrial processes, and then constructs an industrial concentration parameter prediction model based on CauDformer. This model integrates a causal attention mechanism and incorporates a combined loss function that dynamically adjusts frequency domain loss and time domain loss. Furthermore, the model's hyperparameters are optimized using the Optuna library. The optimized prediction model is then used to predict industrial concentration parameters.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0007] This invention is a method for predicting industrial concentration parameters based on the CauDformer model, comprising the following operations:

[0008] Step 1: Collect historical industrial condensed parameters from specific industrial processes to construct a dataset;

[0009] Step 2: Construct a prediction model based on CauDformer;

[0010] Step 3: Train the prediction model built in Step 2 using the dataset from Step 1;

[0011] Step 4: Optimize the hyperparameters of the model trained in Step 3;

[0012] Step 5: Predict the industrial concentration parameters based on the prediction model obtained in Step 4.

[0013] The specific industrial processes in Step 1 include wastewater treatment, chemical production, and food processing; the industrial concentration parameters include the influent flow rate, steam pressure, cumulative effluent from the large circulation, effluent temperature, effluent flow rate, effluent density, and the temperature and pressure of the first, second, and third effects during the T-period before prediction; the dataset is divided into training set, validation set, and test set, with a ratio of 7:1:2.

[0014] The industrial enrichment parameter prediction model built in step two includes a normalization layer, an inverted parameter embedding module, an L-layer stacked encoder module, an inverse normalization layer, a frequency domain enhancement module, and a Bayesian optimization module.

[0015] The normalization layer involves standardizing the parameters, specifically by subtracting the mean from each variable for each sample and dividing by the standard deviation. The expression is as follows:

[0016] X norm =(X raw -mean) / stdev

[0017] Among them, X raw Here, represents the input parameters, 'mean' represents the sample mean, and 'stdev' represents the standard deviation. Its expression is:

[0018]

[0019] Where n is the number of data points in the training set, x i It is the i-th data point in the training set.

[0020] The inverted data embedding module embeds the input parameters from R by inverting the normalized parameters. B ×T×C Convert to model hidden dimension D m The deep features are expressed as follows:

[0021] Among them, R B×T×C The representation is a three-dimensional tensor with shape B×T×C, where B represents the number of parameter samples, T represents the number of parameter time points, and C represents the number of channels.

[0022] The L-layer stacked encoder module contains a causal multi-head self-attention layer (C-MHSA) and a feedforward network layer (FFN) within each encoder. Each causal multi-head self-attention layer and feedforward network layer is followed by residual connections and layer normalization (Add & Norm) to promote information flow and model training stability. Through this layer-by-layer progression, it enables multi-level and multi-dimensional deep abstraction and information integration of features in the input information; specifically including:

[0023] The input to layer l is H. (l-1) The output is H (l) Their shapes are all Where l∈{1,2,…,L} represents the number of layers in the stacked encoder, H 0 It is the output value of the inverted data embedding module;

[0024] The C-MHSA layer consists of multiple parallel single-head attention layers, and its expression is:

[0025] C-MHSA(H (l-1) =MultiHeadAttention(H) (l-1) )

[0026] Among them, H (l-1) Represents the input from the previous layer;

[0027] The formula for calculating single-head attention is:

[0028]

[0029] Where Q, K, and V are the query, key, and value matrices, respectively, T represents the transpose, and d k M represents the dimension of the matrix; temporal The lower triangle is used as the time causality mask, with the upper triangle set to negative infinity to ensure that each variable only focuses on itself and the variables preceding it, thus strictly following the causal relationship of the time series;

[0030] The FFN layer consists of two linear layers, with a nonlinear transformation performed between them using the GELU activation function, the expression of which is:

[0031] FFN(Y)=Linear2[GELU(Linear1(Y))]

[0032] Where Y represents the output of the previous attention sublayer, and its formula is:

[0033] Y = LayerNorm(H (l-1) +Dropout(C-MHSA(H (l-1) )))

[0034] The output H of the layer l encoder l The expression is:

[0035] H l =LayerNorm(Y+Dropout(FFN(Y)))

[0036] Dropout represents a random deactivation layer used to prevent overfitting, and LayerNorm represents layer normalization.

[0037] After passing through the encoder at layer L, multivariate information representation is obtained. This multivariate information is then used as the input for the subsequent denormalization layer.

[0038] The denormalization layer performs denormalization on the received data; it also reduces the hidden dimension D. m Mapping to the prediction length F yields the normalized representation Y of the prediction result. pred_norm ∈R B×F×C ; For Y pred_norm Destandardization is performed, and the original scale is recovered using the mean and standard deviation of the original parameters to obtain the final prediction result Y. pred ∈R B×F×C .

[0039] After the denormalization layer processes the output, the result is transmitted to the frequency domain enhancement module, where a frequency domain auxiliary loss function L is introduced. auxi Enhancing predictive capabilities through frequency domain learning; Lauxi Predicting sequences using comparison models The final L is obtained by comparing the difference between the real sequence Y and the real sequence Y after the real Fast Fourier Transform (RFFT). auxi ;

[0040] The adaptive time-frequency fusion loss function L of the prediction model total (t) includes the temporal reconstruction loss function l rec and frequency domain auxiliary loss function L auxi Furthermore, it is dynamically adjusted at training time step t, and its expression is:

[0041] L total (t)=λ rec (t)·L rec +λ auxi (t)·L auxi

[0042] Where, λ rec (t) represents the temporal reconstruction loss function L rec The weight, λ auxi (t) represents the frequency domain auxiliary loss function L auxi The weights;

[0043] λ rec The expression for (t) is:

[0044] λ rec (t)=λ rec_start +(λ rec_end -λ rec_start P(t)

[0045] Where, λ rec_start λ represents the initial weight. rec_end P(t) represents the end weight, and P(t) represents the training progress ratio, which is expressed as:

[0046]

[0047] T max Represents the total training time and number of steps;

[0048] λ auxi The expression for (t) is:

[0049] λ auxi (t)=1-λ rec (t)

[0050] λ rec and λ auxi The total weight of (t) is 1 to maintain the uniformity of the loss scale.

[0051] In step three, the internal parameters and weights of the prediction model are optimized by comparing the training results with the true parameters, and early stopping is used to avoid overfitting. Early stopping refers to a regularization technique widely used in machine learning and deep learning to prevent the model from overfitting.

[0052] The training of the prediction model ends when the adaptive time-frequency fusion loss function of the prediction model no longer decreases as the training process continues.

[0053] In step four, the Bayesian optimization module uses the Optuna library to implement the Bayesian optimization method to tune the hyperparameters of the model, constructs a surrogate model to fit the relationship between the hyperparameter space and the model performance, and uses the sampling function to guide the next hyperparameter sampling in order to efficiently explore the hyperparameter space, minimize the validation loss, and find the optimal combination of hyperparameters.

[0054] The experiment mainly focused on the learning rate and the adaptive time-frequency fusion loss weight parameter (λ). rec_start , λ rec_end The system optimizes six types of hyperparameters: batch size, model dimension, number of encoder layers, and dropout rate.

[0055] Specifically, the core strategy of the Optuna library is implemented by TPESampler (Tree-structured ParzenEstimator sampler). TPESampler divides historical experimental results into two categories: a set of hyperparameters with better performance and a set with worse performance, represented by probability density functions l(x) and g(x), respectively. The formula for calculating the sampling function EI(x) is as follows:

[0056]

[0057] This module performs the next sampling in the region with the highest EI(X) value to intelligently balance the exploration of the hyperparameter space and the utilization of known high-performance regions. The parameters that minimize the validation loss of the prediction model on the validation set are the optimal parameter combinations; this method enables the optimization process to converge quickly, thereby finding the optimal hyperparameter combination with minimal validation loss.

[0058] In step five, the parameters are input into the optimized CauDformer-based industrial concentration parameter prediction model to predict the industrial concentration parameters.

[0059] The beneficial effects of this invention are:

[0060] This invention discloses a method for predicting industrial concentration parameters based on the CauDformer model. By integrating inverted embedding and causal attention mechanisms into the CauDformer-based prediction model, it effectively captures complex dependencies in multivariate time series. Furthermore, by combining the frequency domain enhancement characteristics of the CauDformer model, it further enhances the model's ability to capture periodic and volatile features. Simultaneously, the dynamically adjusted combined loss function and the Bayesian optimization strategy implemented using Optuna ensure efficient model training and optimal final performance. These combined technical advantages achieve high-precision prediction of industrial concentration parameters, providing key state monitoring capabilities for intelligent management of production processes, significantly improving the monitoring and management level of production processes, and meeting the high-precision requirements for industrial concentration parameter prediction. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart of an industrial concentration parameter prediction method based on the CauDformer model provided in Embodiment 1 of the present invention;

[0063] Figure 2 This is a schematic diagram of the prediction model structure based on CauDformer in the industrial concentration parameter prediction method based on the CauDformer model provided in Embodiment 1 of the present invention.

[0064] Figure 3 This is a schematic diagram of the cumulative prediction results of the large circulation liquid output in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention;

[0065] Figure 4 This is a schematic diagram of the predicted flow rate of the large circulation liquid in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention;

[0066] Figure 5 This is a schematic diagram of the predicted liquid density of the large circulation outlet in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention;

[0067] Figure 6 This is a schematic diagram of the predicted temperature of the large circulation outlet liquid in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention;

[0068] Figure 7This is a schematic diagram of the predicted temperature of the second-effect reactor in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention;

[0069] Figure 8 This is a schematic diagram of the predicted results of the double-effect pressure in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention;

[0070] Figure 9 This is a schematic diagram of the inlet flow rate prediction result in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention;

[0071] Figure 10 This is a schematic diagram of the predicted temperature of the triple-effect reactor in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention;

[0072] Figure 11 This is a schematic diagram of the three-effect pressure prediction results in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention;

[0073] Figure 12 This is a schematic diagram of the first-effect temperature prediction result in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention;

[0074] Figure 13 This is a schematic diagram of the first-effect pressure prediction result in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention;

[0075] Figure 14 This is a schematic diagram of the steam pressure prediction result in an industrial concentration parameter prediction method provided in Embodiment 2 of the present invention. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0077] Example 1

[0078] This embodiment provides a method for predicting industrial concentration parameters based on the CauDformer model. The flowchart of this method is as follows: Figure 1 As shown, firstly, industrial condensed parameters are collected to construct a dataset, and the constructed dataset is standardized in sequence; then, a prediction model based on CauDformer is constructed, and the prediction model is trained and evaluated based on the constructed dataset. Bayesian optimization is used to optimize the hyperparameters of the prediction model, and finally, the optimized prediction model is used to predict industrial condensed parameters.

[0079] The method includes:

[0080] Step 1: Collect historical industrial condensed parameters from specific industrial processes to construct a dataset;

[0081] Step 2: Construct a prediction model based on CauDformer;

[0082] Step 3: Train the prediction model built in Step 2 using the dataset from Step 1;

[0083] Step 4: Optimize the hyperparameters of the model trained in Step 3;

[0084] Step 5: Predict the industrial concentration parameters based on the prediction model obtained in Step 4.

[0085] The specific industrial processes in Step 1 include wastewater treatment, chemical production, and food processing; the industrial concentration parameters include the influent flow rate, steam pressure, cumulative effluent from the large circulation, effluent temperature, effluent flow rate, effluent density, and the temperature and pressure of the first, second, and third effects during the T-period before prediction; the dataset is divided into training set, validation set, and test set, with a ratio of 7:1:2.

[0086] The industrial concentration parameter prediction model built in step two is as follows: Figure 2 As shown, it includes a normalization layer, an inverted data embedding module, an L-layer stacked encoder module, an inverse normalization layer, a frequency domain enhancement module, and a Bayesian optimization module;

[0087] The normalization layer involves standardizing the data, specifically by subtracting the mean from each variable in each sample and dividing by the standard deviation. The expression for this is:

[0088] X norm =(X raw -mean) / stdev

[0089] Among them, X raw Here, represents the input data, 'mean' represents the sample mean, and 'stdev' represents the standard deviation. Its expression is:

[0090]

[0091] Where n is the number of data points in the training set, x i It is the i-th data point in the training set.

[0092] The inverted data embedding module performs inverted embedding by reversing or flipping the standardized parameters, thus embedding the input parameters from R... B×T×C Convert to model hidden dimension D m The deep features are expressed as follows:

[0093] Among them, R B×T×CThe representation is a three-dimensional tensor with shape B×T×C, where B represents the number of parameter samples, T represents the number of parameter time points, and C represents the number of channels.

[0094] In the L-layer stacked encoder module, each encoder contains a causal multi-head self-attention layer (C-MHSA) and a feedforward network layer (FFN). Each sub-layer (C-MHSA and FFN) is connected using residual connections and layer normalization (Add & Norm) to promote information flow and model training stability; specifically including:

[0095] The input to layer l is H. (l-1) The output is H (l) Their shapes are all Where l∈{1,2,…,L} represents the number of layers in the stacked encoder, H 0 It is the output value of the inverted data embedding module;

[0096] The C-MHSA layer consists of multiple parallel single-head attention layers, and its expression is:

[0097] C-MHSA(H (l-1) =MultiHeadAttention(H) (l-1) )

[0098] Among them, H (l-1) Represents the input from the previous layer;

[0099] The formula for calculating single-head attention is:

[0100]

[0101] Where Q, K, and V are the query, key, and value matrices, respectively, T represents the transpose, and d k M represents the dimension of the matrix; temporal The lower triangle is used as the time causality mask, with the upper triangle set to negative infinity to ensure that each variable only focuses on itself and the variables preceding it, thus strictly following the causal relationship of the time series;

[0102] The FFN layer consists of two linear layers, with a nonlinear transformation performed between them using the GELU activation function, the expression of which is:

[0103] FFN(Y)=Linear2[GELU(Lijnear1(Y))]

[0104] Where Y represents the output of the previous attention sublayer, and its formula is:

[0105] Y = LayerNorm(H (l-1)+Dropout(C-MHSA(H (l-1) )))

[0106] The output H of the layer l encoder l The expression is:

[0107] H l =LayerNorm(Y+Dropout(FFN(Y)))

[0108] Dropout represents a random deactivation layer used to prevent overfitting, and LayerNorm represents layer normalization.

[0109] After passing through the encoder at layer L, multivariate information representation is obtained. This multivariate information is then used as the input for the subsequent denormalization layer.

[0110] The denormalization layer performs denormalization on the received input; it also reduces the hidden dimension D. m Mapping to the prediction length F yields the normalized representation Y of the prediction result. pred_norm ∈R B×F×C ; For Y pred_norm Destandardization is performed, and the original scale is recovered using the mean and standard deviation of the original parameters to obtain the final prediction result Y. pred ∈R B×F×C .

[0111] After the denormalization layer processes the output, the result is transmitted to the frequency domain enhancement module, where a frequency domain auxiliary loss function L is introduced. auxi Enhancing predictive capabilities through frequency domain learning; L auxi Predicting sequences using comparison models The final L is obtained by comparing the difference between the real sequence Y and the real sequence Y after the real Fast Fourier Transform (RFFT). auxi ;

[0112] The adaptive time-frequency fusion loss function L of the prediction model total (t) includes the temporal reconstruction loss function l rec and frequency domain auxiliary loss function L auxi Furthermore, it is dynamically adjusted at training time step t, and its expression is:

[0113] L total (t)=λ rec (t)·L rec +λ auxi (t)·L auxi

[0114] Where, λ rec (t) represents the temporal reconstruction loss function L recThe weight, λ auxi (t) represents the frequency domain auxiliary loss function L auxi The weights;

[0115] λ rec The expression for (t) is:

[0116] λ rec (t)=λ rec_start +(λ rec_end -λ rec_start P(t)

[0117] Where, λ rec_start λ represents the initial weight. rec_end P(t) represents the end weight, and P(t) represents the training progress ratio, which is expressed as:

[0118]

[0119] T max Represents the total training time and number of steps;

[0120] λ auxi The expression for (t) is:

[0121] λ auxi (t)=1-λ rec (t)

[0122] λ rec and λ auxi The total weight of (t) is 1 to maintain the uniformity of the loss scale.

[0123] In step three, the internal parameters and weights of the prediction model are optimized by comparing the training results with the true parameters, and early stopping is used to avoid overfitting.

[0124] The training of the prediction model ends when the adaptive time-frequency fusion loss function of the prediction model no longer decreases as the training process continues.

[0125] In step four, the Bayesian optimization module uses the Optuna library to implement the Bayesian optimization method to tune the hyperparameters of the model, constructs a surrogate model to fit the relationship between the hyperparameter space and the model performance, and uses the sampling function to guide the next hyperparameter sampling in order to efficiently explore the hyperparameter space, minimize the validation loss, and find the optimal combination of hyperparameters.

[0126] The experiment mainly focused on the learning rate and the adaptive time-frequency fusion loss weight parameter (λ). rec_start , λ rec_endThe system optimizes six types of hyperparameters: batch size, model dimension, number of encoder layers, and dropout rate.

[0127] Specifically, the core strategy of the Optuna library is implemented by TPESampler (Tree-structured ParzenEstimator sampler). TPESampler divides historical experimental results into two categories: a set of hyperparameters with better performance and a set with worse performance, represented by probability density functions l(x) and g(x), respectively. The formula for calculating the sampling function EI(x) is as follows:

[0128]

[0129] This module performs the next sampling in the region with the highest EI(X) value to intelligently balance the exploration of the hyperparameter space and the utilization of known high-performance regions. The optimal parameter combination is the parameter whose validation loss is minimized on the validation set when the adaptive time-frequency fusion loss function of the prediction model is minimized. This method enables the optimization process to converge quickly, thereby finding the optimal hyperparameter combination with the minimum validation loss.

[0130] In step five, the parameters to be predicted are input into the optimized CauDformer-based prediction model to predict the industrial concentration parameters.

[0131] Example 2

[0132] This embodiment provides a method for predicting industrial concentration parameters. The method is based on the prediction model described in Embodiment 1. The method includes: inputting the parameters to be predicted into the prediction model based on CauDformer to predict the industrial concentration parameters.

[0133] Figure 3 , Figure 4 , Figure 5 , Figure 6 , Figure 7 , Figure 8 , Figure 9 , Figure 10 , Figure 11 , Figure 12 , Figure 13 as well as Figure 14The figures show the predicted results for the following parameters: cumulative output liquid from the main circulation system, output liquid velocity from the main circulation system, output liquid density from the main circulation system, output liquid temperature from the main circulation system, temperature of the second effect, pressure of the second effect, inlet flow rate, temperature of the third effect, pressure of the third effect, temperature of the first effect, pressure of the first effect, and steam pressure. As can be seen from the figures, the predicted curves almost completely overlap with the actual curves, proving that the prediction model based on CauDformer proposed in this invention can achieve accurate prediction of industrial concentration parameters.

[0134] To verify the performance of the CauDformer-based prediction model proposed in this invention, its prediction performance at 30 minutes and 60 minutes in the enrichment process is compared with that of the TiDE prediction model, DLinear prediction model, TimeXer prediction model, and PatchTST prediction model. For the TiDE prediction model, please refer to Das A, Kong W, Leach A, et al. Long-term forecasting with tide: Time-series dense encoder[J]. arXiv preprint arXiv:2304.08424,2023.; for the DLinear prediction model, please refer to Zeng A, Chen M, Zhang L, et al. Are transformers effective for time series forecasting? [C] / / Proceedings of the AAAI conference on artificial intelligence. 2023, 37(9): 11121-11128.;The TimeXer prediction model can be found in Wang Y, Wu H, Dong J, et al. Timexer: Empowering transformers for time series forecasting with exogenous variables[J]. Advances in Neural Information Processing Systems, 2024, 37: 469-498.;The PatchTST prediction model can be found in Nie Y, Nguyen NH, Sinthong P, et al. A time series is worth 64 words: Long-term forecasting with transformers[J]. arXiv preprint arXiv: 2211.14730, 2022.;

[0135] Mean absolute error (MAE) and mean square error (MSE) are used as evaluation criteria;

[0136] Mean Absolute Error (MAE) measures the accuracy of a forecast by calculating the average of the absolute differences between the predicted and actual values. Its expression is:

[0137]

[0138] Mean squared error (MSE) highlights larger errors by calculating the average of the squared differences. Its expression is:

[0139]

[0140] Where, in the formula, y i Represents the true value. Represents the predicted value. represents the true mean, and N represents the sample size.

[0141] The results of the comparison are shown in Table 1;

[0142] Table 1 Comparison of Evaluation Results of Different Prediction Models

[0143]

[0144] As shown in Table 1, with a 60-minute prediction lead time, CauDformer's MAE is 0.0957 and MSE is 1.1048. These three indicators are the best among all models, indicating that it has the smallest average deviation between the predicted results and the actual values, the highest accuracy, and the strongest explanatory power for data variation. With a 30-minute prediction lead time, CauDformer's MAE is 0.0579 and MSE is 0.6381, again maintaining its leading position in all indicators. This demonstrates that the CauDformer-based prediction model proposed in this invention has a more accurate prediction effect compared to other prediction models.

[0145] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0146] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An industrial concentration parameter prediction method based on a CauDformer model, characterized in that, The method includes: Step 1: Collect historical parameters from specific industrial processes to construct a condensed industrial dataset; Step 2: Construct a prediction model based on CauDformer; Step 3: Train the prediction model built in Step 2 using the industrial concentration dataset from Step 1; Step 4: Optimize the hyperparameters of the model trained in Step 3; Step 5: Predict the industrial concentration parameters based on the prediction model obtained in Step 4; The prediction model based on CauDformer in step two includes a normalization layer, an inverted data embedding module, and... The module consists of a stacked encoder module, an inverse normalization layer, a frequency domain enhancement module, and a Bayesian optimization module. The data input to the prediction model is first standardized by a normalization layer, then transmitted to an inverted data embedding module to convert the parameters into features of the model's hidden dimensions, and then input to... Deep feature extraction of mixed multivariate time series is performed in a stacked encoder module. The output of the stacked encoder module is transmitted to the denormalization layer for denormalization. The denormalized data is then transmitted to the frequency domain enhancement module to update the loss function of the prediction model. Finally, it is transmitted to the Bayesian optimization module to optimize the hyperparameters of the prediction model. The CauDformer-based prediction model The stacked encoder module consists of The layers are stacked, and each encoder contains a causal multi-head self-attention layer and a feedforward network layer. Each sub-layer is connected by residual connections and layer normalization. The causal multi-head self-attention layer is the first l The input of the layer is The output is ,in , representing the number of layers in the stacked encoder. It is the output value of the inverted data embedding module; The causal multi-head self-attention layer consists of several parallel single-head attention layers, and its expression is: wherein, represents the input of the previous layer; The formula for calculating single-head attention is: in, These are query, key, and value matrices, respectively. Represents transpose. Represents the dimension of the matrix; The lower triangle is used as a time causality mask, with the upper triangle set to negative infinity to ensure that each variable only focuses on itself and the variables preceding it, thus following the causal relationship of the time series; The feedforward network layer consists of two linear layers, with a nonlinear transformation performed in between via the GELU activation function, the expression of which is: Where Y represents the output of the previous attention sublayer, its expression is: in, Representative level normalization, This represents a random deactivation layer used to prevent overfitting.

2. The method according to claim 1, characterized in that, The output of the layer encoder The output of the layer encoder The expression for the output of the layer encoder is go through After the encoder of the layer, multivariate information representation is obtained. This multivariate information is then used as the input for the subsequent denormalization layer.

3. The method of claim 2, wherein, The loss function of the prediction model is an adaptive time-frequency fusion loss function. Including the temporal reconstruction loss function frequency domain auxiliary loss function Furthermore, it is dynamically adjusted at training time step t, and its expression is: in, Represents the temporal reconstruction loss function The weight, Represents the frequency domain auxiliary loss function The weights; The expression is: wherein, represents the starting weight, represents the ending weight, represents the training progress ratio, which is expressed as: represents the total number of training time steps; The expression is: and The total weight of 1 keeps the loss scale uniform.

4. The method according to claim 3, characterized in that, The normalization layer includes standardizing the data by subtracting the mean from each variable in each sample and dividing by the standard deviation, as expressed by: in, Represents input data, Represents the mean of the sample. The standard deviation of the sample is expressed as: wherein, is the number of data points in the training set, is the i-th data point in the training set; The inverted data embedding module embeds the input data by inverting or reversing the standardized data. Convert to model hidden dimensions The deep features are expressed as follows: ; in, The representative shape is The three-dimensional tensor Represents the number of data samples. Represents the number of data points in time. Represents the number of channels; The denormalization layer includes denormalization processing to hide dimensions. Mapping to the prediction length F yields a normalized representation of the prediction result. ;right Destandardization is performed, using the mean and standard deviation of the original data to recover the original scale, thus obtaining the final prediction result. .

5. The method of claim 4, wherein, The specific industrial processes in step one include wastewater treatment, chemical production, and food processing. The industrial concentration parameters include the inlet flow rate, steam pressure, cumulative outflow of the large circulation system, outflow temperature, outflow rate, outflow density, and the temperature and pressure of the first, second, and third effects during the time period before prediction. The industrial concentration parameter set is divided into a training set, a validation set, and a test set, with a division ratio of 7:1:

2.

6. The method according to claim 5, characterized in that, In step three, the training results are compared with the true parameters to optimize the internal parameters and weights of the prediction model, and early stopping is used to avoid overfitting. The training of the prediction model ends when the adaptive time-frequency fusion loss function of the prediction model no longer decreases as the training process continues.

7. The method according to claim 6, characterized in that, In step four, the Bayesian optimization module uses the Optuna library to implement the Bayesian optimization method to tune the hyperparameters of the model, constructs a surrogate model to fit the relationship between the hyperparameter space and the model performance, uses the acquisition function to guide the next hyperparameter sampling, minimizes the validation loss, and finds the optimal combination of hyperparameters.

8. A method for predicting industrial concentration parameters of honeysuckle extract, characterized in that, The method is implemented based on the method described in any one of claims 1-7.

Images