A multi-time scale dynamic soft-sensing modeling method for multi-sampling rate data

Abstract

The application relates to a multi-time scale dynamic soft measurement modeling method for multi-sampling rate data. In the training stage, the original data is divided into data blocks corresponding to different time scales in the order of high to low sampling rate. Unsupervised training and migration of a multi-sampling rate dynamic migration variational autoencoder (MDTVAE) model are gradually carried out between unlabeled data blocks. A regression module is added to the MDTVAE model, and the labeled data block is trained in a supervised training manner. In the test stage, the model is used to predict the quality variable, and the prediction accuracy is systematically evaluated through a quantitative index. The application can effectively capture the dynamic characteristics of each data block at a specific time scale, improve the learning ability of the model for multi-scale dynamics, share and migrate the learned time sequence knowledge in the training of different time scale data blocks, and show stronger robustness and accuracy in the prediction of the quality variable.

Description

A Multi-Time-Scale Dynamic Soft Measurement Modeling Method for Multi-Sampling Rate Data Technical Field

[0001] This invention belongs to the field of soft measurement technology, specifically relating to a multi-timescale dynamic soft measurement modeling method for multi-sampling rate data. Background Technology

[0002] With the rapid development of industrial automation and intelligent technologies, the types of variables collected in industrial processes are increasing daily. Due to significant differences in sensor characteristics, data sources, and process requirements among different variables, the sampling rates vary considerably. Soft sensing modeling methods for multi-sampling-rate data can be broadly categorized into two types: the first is based on data preprocessing strategies, which unify the sampling rate by upsampling or downsampling data at different sampling rates; the second is based on model design strategies, which directly process multi-sampling-rate data by constructing models with specially structured architectures. The former method is simple, but the data preprocessing process often destroys the multi-scale structure and correlations between variables in the original data, leading to model inaccuracies and affecting prediction accuracy. In contrast, the latter can directly achieve cross-scale feature extraction, preserving the original characteristics and inherent correlations of data at different sampling rates. Furthermore, the complex feedback mechanisms prevalent in industrial processes cause systems to exhibit significant dynamic characteristics, with correlations between variables spanning both time and space dimensions. Therefore, accurately capturing the dynamic characteristics hidden in multi-sampling-rate industrial data and constructing an ideal soft sensing model based on this is a critical problem that urgently needs to be solved in the industrial field. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a multi-timescale dynamic soft measurement modeling method for multi-sampling rate data. This method constructs multi-timescale data blocks based on sampling frequency, then extracts the dynamic characteristics of each timescale sequentially according to the sampling rate from high to low, and gradually performs transfer learning. This enables the model to systematically grasp the dynamic patterns at different timescales, thereby solving the challenges of multi-sampling rate data modeling and quality variable prediction in dynamic processes.

[0004] The technical solution adopted in this invention is as follows:

[0005] A method for multi-timescale dynamic soft measurement modeling of multi-sampling rate data, the method comprising the following steps:

[0006] (1) Collect process variables and quality variables from the industrial site, construct the original dataset, and eliminate the difference in dimensions through standardization;

[0007] (2) Divide the original data into data blocks of different time scales in order of sampling rate from high to low;

[0008] (3) Construct a multi-sampling-rate dynamic transfer variational autoencoder (MDTVAE) model and complete the hyperparameter settings and parameter initialization;

[0009] (4) Optimize the MDTVAE model parameters using the first data block;

[0010] (5) Transfer the model parameters obtained in step (4) to the second data block and use the second data block to train the model, and so on, until the transfer learning between all data blocks is completed;

[0011] (6) Collect test data online, and after standardization, select samples of full sampling of process variables. Use the MDTVAE model obtained in step (5) to predict quality variables, and evaluate the prediction effect through performance indicators.

[0012] Furthermore, the specific process of step (1) is as follows:

[0013] The Z-score standardization method was used to normalize the process variables and quality variables in the original dataset, converting them into a standard normal distribution with a mean of 0 and a standard deviation of 1, thus eliminating the dimensional differences between different variables.

[0014] Furthermore, the specific process of step (2) is as follows:

[0015] Suppose the process variable contains K sampling rates, namely , ,…, The sampling frequency is The number of variables is And so on, the sampling frequency is The number of variables is The quality variable includes one sampling rate. .in The first data block is The sampling time is In integer multiples of the sample, the first The process variables constitute the second data block. The sampling time is In integer multiples of the sample, the first The process variables constitute the Kth data block; and so on. The sampling time is In integer multiples of the sample, the first It consists of process variables. Additionally, the data block containing quality variables is... .

[0016] Furthermore, the specific process of step (3) is as follows:

[0017] The MDTVAE model employs a supervised variational autoencoder framework, comprising three key components: an encoder, a decoder, and a regression module. To effectively capture the temporal dynamics of industrial process data, both the encoder and decoder are composed of self-attention modules.

[0018] Before training the MDTVAE model, set its hyperparameters, including network structure parameters (number of layers, hidden layer dimension), optimization parameters (learning rate, training epochs), and self-attention mechanism parameters (number of heads, key dimension, value dimension). At the same time, randomly initialize all weights and biases in the model and select the corresponding activation functions.

[0019] Furthermore, the specific process of step (4) is as follows:

[0020] During the training phase of the MDTVAE model, the first data block is used first. Model training is performed in an unsupervised manner. Assume... It is a data block Given a time series, where m is the series length, Query matrices are generated through linear projection transformation. Key matrix Sum matrix The specific transformation relationships are as follows:

[0021]

[0022]

[0023]

[0024] in, , and The weight parameters for the Q, K, and V transforms are respectively. , and These are the bias parameters for the Q, K, and V transforms, respectively. Then, using... , and Calculate attention weights:

[0025]

[0026] in, This is the dimension of the key vector. By summing the attention weights over the value vector, we obtain the output sequence of the self-attention module.

[0027]

[0028] Then, Perform a linear mapping to obtain the hidden layer vector of MDTVAE. mean and variance The above calculation process completely constructs the encoder part of MDTVAE, that is... The decoder of MDTVAE, similar to the encoder, is also composed of self-attention modules, represented as... The prior distribution of the hidden layer vectors is Therefore, based on the first data block When training the model, the loss function is:

[0029]

[0030] Furthermore, the specific process of step (5) is as follows:

[0031] As shown in Figure 1 (taking four sampling rates as examples), the following will be used: The trained model is represented as MDTVAE 1 and pass its parameters to MDTVAE 2 As initialization parameters, the second data block is then used. Train the model. Perform transfer experiments sequentially across different unlabeled data blocks in the same manner until the MDTVAE is obtained. K Through the transfer learning method between the aforementioned data blocks, the model can capture and understand the dynamics across multiple time scales, thereby improving its ability to model complex time-series data.

[0032] Then, MDTVAE K The model parameters are passed to MDTVAE K+1 Using the (K+1)th labeled data block Supervised training was performed. Compared to the first K MDTVAE models, MDTVAE... K+1 In the decoder, the last hidden layer connects to the regression module for predicting quality variables. Assume... It is a data block In a certain time series, This corresponds to its quality variable. At this stage, the encoder is represented as... The decoder containing the regressor is represented as The loss function is:

[0033]

[0034] Furthermore, the specific process of step (6) is as follows:

[0035] During the testing phase, the data was standardized, and samples in which all variables were fully sampled were selected as test samples. The test samples were then constructed into a time-series data sequence and input into the MDTVAE model trained in step (5) to obtain the corresponding quality variable prediction results. To evaluate the prediction performance of the MDTVAE model, root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R²) were used. 2 As a quantitative indicator, its calculation method is as follows:

[0036]

[0037]

[0038]

[0039] in, The number of samples in the test set. and Let be the true value and predicted value of the quality variable at time t.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] (1) This invention divides data into blocks according to the sampling rate, with each data block corresponding to a specific time scale. For each data block, a corresponding MDTVAE model is trained. The self-attention module in the model can automatically identify and effectively capture the dynamic features of each data block at a specific time scale, thereby improving the model's ability to learn from multi-scale dynamic changes. Furthermore, the division of data blocks fully utilizes all the information from the multi-sampling rate data, avoiding the problem of dynamics being destroyed by upsampling or downsampling.

[0042] (2) This invention utilizes a cross-scale transfer learning mechanism between data blocks, enabling the model to share and transfer learned temporal knowledge during training with data blocks at different time scales. The model retains key temporal information from previous data blocks while adaptively integrating the unique dynamic characteristics of new data blocks. In this way, the model can effectively improve its perception and adaptation to different dynamic changes when dealing with data at multiple time scales, thus exhibiting stronger robustness and accuracy in the prediction of quality variables. Attached Figure Description

[0043] Figure 1 is a schematic diagram of multi-sampling rate data segmentation and MDTVAE modeling;

[0044] Figure 2 is a flowchart of the multi-timescale dynamic soft measurement modeling method of the present invention;

[0045] Figure 3 is a schematic diagram of the fitting effect between the predicted value and the true value provided in the embodiment of the present invention;

[0046] Figure 4 is an error box plot provided in an embodiment of the present invention. Detailed Implementation

[0047] This invention addresses the problem of insufficient dynamic correlation extraction from multi-sampling rate data by proposing a multi-timescale dynamic soft-sensor modeling method for multi-sampling rate data. The invention is described in detail below with a practical industrial case study. Taking a pre-decarbonization unit in an ammonia synthesis process as an example, the MDTVAE model is applied to predict the residual CO2 gas concentration.

[0048] Referring to Figure 2, a multi-timescale dynamic soft sensor modeling method for multi-sampling rate data includes the following steps:

[0049] (1) Training set construction and data preprocessing: Multi-sampling rate data were collected from the real-time database of the distributed control system as the training set. The Z-score standardization method was used to normalize the process variables and quality variables in the original dataset, converting them into a standard normal distribution with a mean of 0 and a standard deviation of 1, thus eliminating the dimensional differences between different variables.

[0050] (2) Data block partitioning: Assume that the process variable contains K sampling rates, namely , ,…, The sampling frequency is The number of variables is And so on, the sampling frequency is The number of variables is The quality variable includes one sampling rate. .in The first data block is The sampling time is In integer multiples of the sample, the first The process variables constitute the second data block. The sampling time is In integer multiples of the sample, the first The process variables constitute the Kth data block; and so on. The sampling time is In integer multiples of the sample, the first It consists of process variables. Additionally, the data block containing quality variables is... .

[0051] (3) MDTVAE model structure design and hyperparameter setting, specifically including the following steps:

[0052] The MDTVAE model employs a supervised variational autoencoder framework, comprising three key components: an encoder, a decoder, and a regression module. To effectively capture the temporal dynamics of industrial process data, both the encoder and decoder are composed of self-attention modules.

[0053] Before training the MDTVAE model, set its hyperparameters, including network structure parameters (number of layers, hidden layer dimension), optimization parameters (learning rate, training epochs), and self-attention mechanism parameters (number of heads, key dimension, value dimension). At the same time, randomly initialize all weights and biases in the model and select the corresponding activation functions.

[0054] (4) Based on the first data block Perform MDTVAE 1 Model training specifically includes the following steps:

[0055] Assumption It is a data block Given a time series, where m is the series length, Query matrices are generated through linear projection transformation. Key matrix Sum matrix The specific transformation relationships are as follows:

[0056]

[0057]

[0058]

[0059] in, , and The weight parameters for the Q, K, and V transforms are respectively. , and These are the bias parameters for the Q, K, and V transforms, respectively. Then, using... , and Calculate attention weights:

[0060]

[0061] in, This is the dimension of the key vector. By summing the attention weights over the value vector, we obtain the output sequence of the self-attention module.

[0062]

[0063] Then, Perform a linear mapping to obtain the hidden layer vector of MDTVAE. mean and variance The above calculation process completely constructs the encoder part of MDTVAE, that is... The decoder of MDTVAE, similar to the encoder, is also composed of self-attention modules, represented as... The prior distribution of the hidden layer vectors is Therefore, based on the first data block When training the model, the loss function is:

[0064]

[0065] (5) Model migration between data blocks, specifically including the following steps:

[0066] MDTVAE 1 The model parameters are passed to MDTVAE 2 As initialization parameters, the second data block is then used. Train the model. Perform the transfer between different data blocks sequentially in the same manner until the MDTVAE is obtained. K Through the transfer learning method between the aforementioned data blocks, the model can capture and understand the dynamics across multiple time scales, thereby improving its ability to model complex time-series data.

[0067] Then, MDTVAE K The model parameters are passed to MDTVAE K+1 Using the (K+1)th labeled data block Supervised training was performed. Compared to the first K MDTVAE models, MDTVAE... K+1 In the decoder, the last hidden layer connects to the regression module for predicting quality variables. Assume... It is a data block In a certain time series, This corresponds to its quality variable. At this stage, the encoder is represented as... The decoder containing the regressor is represented as The loss function is:

[0068]

[0069] (6) Model performance testing, specifically including the following steps:

[0070] During the testing phase, the test data is standardized, and samples where all variables are fully sampled are selected as test samples. The test samples are then constructed into a time-series data sequence and input into the MDTVAE model trained in step (5) to obtain the corresponding quality variable prediction results. Finally, the root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R²) are used to predict the results. 2 () is used as a quantitative indicator to evaluate the predictive performance of the MDTVAE model.

[0071] This invention is applied to the pre-decarbonization unit of the ammonia synthesis process in a chemical plant. The goal is to predict the residual CO2 concentration in the gas after decarbonization in real time, so as to optimize the operating parameters of the absorption tower and ensure that the CO2 removal efficiency meets the process requirements.

[0072] First, 9000 sample points were collected from the database of the distributed control system to form a multi-rate time series. Process variables U1-U8 were sampled at 1-minute intervals, process variables U9-U13 at 3-minute intervals, process variables U14-U20 at 5-minute intervals, and the quality variable U21 at 6-minute intervals. Based on this multi-rate time series data, data blocks were partitioned and the MDTVAE model was trained.

[0073] Then, time series data of 300 samples were collected as a test dataset. The MDTVAE model was used to predict the residual CO2 concentration in the gas after pre-decarbonization, and the results were compared with the actual values. The comparison results are shown in Figure 3. As can be seen from Figure 3, the prediction results of the model proposed in this invention have a high degree of fit with the actual values, indicating that this invention can effectively address the problem of multiple sampling rates in time series data in actual industrial applications and significantly improve the prediction accuracy of quality variables.

[0074] To verify the crucial role of multi-timescale modeling and transfer mechanisms in the MDTVAE model, the MDTVAE model was compared with the MTVAE and MDVAE models. The MTVAE model retained the data block partitioning and transfer process of MDTVAE, but adopted a static VAE model (without self-attention mechanism); the MDVAE model maintained a similar network structure to MDTVAE, but eliminated the data block partitioning strategy, using only a small number of fully sampled variables for training. The prediction performance comparison results are shown in Table 1 and Figure 4. Experimental data show that the proposed MDTVAE model achieves better performance in RMSE, MAE, and R... 2The MDTVAE model significantly outperforms the comparison model in all key metrics. This comparison result fully verifies that: (1) the multi-timescale dynamic modeling with the introduction of self-attention mechanism can effectively capture the temporal characteristics of industrial process data; (2) the transfer learning strategy based on data block can significantly alleviate the problem of insufficient full-sample data. Through the synergistic optimization of the two, the MDTVAE model achieves more accurate soft measurement modeling of time-series data in multi-sample-rate industrial processes.

[0075] Table 1: Comparison of Predictive Performance of Different Soft Sensing Models

[0076] Soft measurement model MTVAEMDVAEMDTVAERMSE 0.1674 0.1625 0.1114MAE 0.1451 0.1420 0.0945R 2 0.7208 0.7371 0.8765 surface.

Claims

1. A multi-timescale dynamic soft sensor modeling method for multi-sampling rate data, characterized in that... The process includes the following steps: (1) Collect process variables and quality variables from the industrial site, construct the original dataset, and eliminate dimensional differences through standardization. Apply the dataset to the pre-decarbonization unit of the ammonia synthesis process, with the goal of real-time prediction of the residual CO2 concentration in the decarbonized gas; (2) Divide the original data into data blocks of different time scales according to the sampling rate from high to low. The specific process is as follows: Assume that the process variables contain K sampling rates, which are respectively , ,…, The sampling frequency is The number of variables is And so on, the sampling frequency is The number of variables is ； The quality variable includes one sampling rate. ,in The first data block is The sampling time is In integer multiples of the sample, the first The process variables constitute the second data block. The sampling time is In integer multiples of the sample, the first The process variables constitute the Kth data block; and so on. The sampling time is In integer multiples of the sample, the first It consists of process variables, and in addition, the data block containing quality variables is... (3) Construct a multi-sampling rate dynamic transfer variational autoencoder (MDTVAE) model. The MDTVAE model adopts a supervised variational autoencoder framework, which includes three key components: encoder, decoder and regression module. In order to effectively capture the temporal dynamic characteristics of industrial process data, both the encoder and decoder are composed of self-attention modules. Before training the MDTVAE model, set its hyperparameters, including network structure parameters, optimization parameters and self-attention mechanism parameters. The network structure parameters include the number of layers and the hidden layer dimension. The optimization parameters include the learning rate and the number of training rounds. The self-attention mechanism parameters include the number of heads, the key dimension and the value dimension. At the same time, randomly initialize all weights and biases in the model and select the corresponding activation functions. Complete the hyperparameter setting and parameter initialization. (4) Optimize the MDTVAE model parameters using the first data block. (5) Transfer the model parameters obtained in step (4) to the second data block and use the second data block for model training. Repeat this process until the transfer learning between all data blocks is completed. The specific process is as follows: use the first data block to optimize the MDTVAE model parameters. The trained model is represented as MDTVAE 1 and pass its parameters to MDTVAE 2 As initialization parameters, the second data block is then used. The model is trained by sequentially transferring data between different unlabeled data blocks in the same manner until the MDTVAE is obtained. K Through transfer learning between the aforementioned data blocks, the model captures and understands the dynamics across multiple time scales, thereby improving its ability to model complex time-series data; then, MDTVAE K The model parameters are passed to MDTVAE K+1 Using the (K+1)th labeled data block Supervised training was performed, and compared to the first K MDTVAE models, MDTVAE... K+1 In the decoder, the last hidden layer is connected to the regression module for predicting quality variables, assuming... It is a data block In a certain time series, For its corresponding quality variable, at this stage, the encoder is represented as... The decoder containing the regressor is represented as The loss function is: (6) Collect test data online, and after standardization, select samples of full sampling of process variables. Use the MDTVAE model obtained in step (5) to predict quality variables, and evaluate the prediction effect through performance indicators.

2. The multi-timescale dynamic soft sensor modeling method for multi-sampling rate data according to claim 1, characterized in that, The specific process of step (1) is as follows: the process variables and quality variables in the original dataset are normalized by using the Z-score standardization method, and converted into a standard normal distribution with a mean of 0 and a standard deviation of 1, thereby eliminating the dimensional differences between different variables.

3. The multi-timescale dynamic soft sensor modeling method for multi-sampling rate data according to claim 2, characterized in that, The specific process of step (4) is as follows: In the MDTVAE model training phase, the first data block is first used. Model training is performed in an unsupervised manner, assuming It is a data block Given a time series, where m is the series length, Query matrices are generated through linear projection transformation. Key matrix Sum matrix The specific transformation relationships are as follows: ， ， ,in, ， and The weight parameters for the Q, K, and V transforms are respectively. ， and These are the bias parameters for the Q, K, and V transforms, respectively. Then, using... ， and Calculate attention weights: ,in, The dimension of the key vector is denoted by . By summing the attention weights over the value vector, we can obtain the output sequence of the self-attention module. Then, Perform a linear mapping to obtain the hidden layer vector of MDTVAE. mean and variance Construct the encoder part of MDTVAE, i.e. The decoder of MDTVAE consists of self-attention modules, represented as follows: The prior distribution of the hidden layer vectors is Therefore, based on the first data block When training the model, the loss function is: 。 4. The multi-timescale dynamic soft sensor modeling method for multi-sampling rate data according to claim 3, characterized in that, The specific process of step (6) is as follows: In the testing phase, the data is standardized and samples in which all variables are fully sampled are selected as test samples. The test samples are constructed into a time series data sequence in chronological order and input into the MDTVAE model trained in step (5) to obtain the corresponding quality variable prediction results. In order to evaluate the prediction effect of the MDTVAE model, the root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R²) are used. 2 As a quantitative indicator, its calculation method is as follows: ， ， ,in, The number of samples in the test set. and Let be the true value and predicted value of the quality variable at time t.

Images