Industrial process soft measurement method of multi-module self-attention gating network

Abstract

The invention relates to an industrial process soft measurement method, in particular to an industrial process soft measurement method of a multi-module self-attention gating network, which is familiar with an actual industrial process and determines a process variable and a quality variable; industrial data are collected and divided; performing data preprocessing; the processed data are transmitted to an input and output isomorphic auto-encoder network (IOSIAE) to learn effective information, and deep quality related features are extracted; and taking the depth features extracted by the IOSIAE as the input of a self-attention mechanism, taking the features after the self-attention extraction as the input of a gating mechanism, carrying out dynamic weighting on the gating mechanism, constructing a hybrid model to realize quality variable prediction, and verifying the model prediction performance according to an evaluation index. On-line prediction is carried out through a soft measurement model, an industrial product quality variable prediction value is obtained, a production operation process is monitored and controlled in real time, and a production strategy is adjusted to ensure production efficiency, improve product quality and realize green production.

Description

Technical Field

[0001] The invention relates to an industrial process soft measurement method, in particular to an industrial process soft measurement method of a multi-module self-attention gated network. Background Art

[0002] Faced with large-scale and highly complex modern industrial processes, monitoring, evaluating and optimizing the production process are of great significance for ensuring safe production and improving product quality. Generally speaking, key indicators or quality variables in industrial processes are the primary way to reflect the production status, and people always expect to obtain these indicators as quickly and accurately as possible. In the past, due to the relatively simple scale of the industry, the key variables of the industrial process could be directly obtained through online hard equipment or offline analytical instruments. However, with the continuous growth of the industry scale and the increasing complexity between process variables, it has become very difficult to directly obtain key variables due to technical or economic limitations. The traditional strategy of using online hard equipment is slowly being abandoned, and data-driven soft transmission measurement technology is becoming more and more popular. Different types of industrial data are obtained in different industrial processes, so as to more accurately extract the features and structural information hidden in the data. Operators can manipulate production process information in real time and effectively adjust production strategies. It is of great significance to achieve green production.

[0003] Industrial data often have strong nonlinearity, dynamic characteristics and redundant information. Using these data to directly establish a regression model for quality variables may lead to inaccurate model prediction and poor robustness. Therefore, it is necessary to represent the input data and extract useful information for soft measurement modeling. Autoencoders have been widely used in industrial processes due to their excellent feature extraction and dimensionality reduction capabilities. The original autoencoder lacks the constraint of label information in the feature extraction process, resulting in the existence of features that are irrelevant to the expected target variable, affecting the prediction results. At the same time, as the number of network layers increases, the mapping relationship between the features extracted by the hidden layer and the original input variables gradually weakens, affecting the modeling accuracy. Considering that the dynamic nature of process data will affect the model results, the traditional autoencoder lacks the problem of dynamic extraction and analysis of data. To this end, the original input loss and the quality prediction loss are added to the autoencoder, and an input-output isomorphic autoencoder network is established to extract deep quality-related features and reduce the loss of related information. Considering the multi-condition nature of the data, the features extracted by multiple IOSIAE modules are used as the input of the self-attention mechanism to focus on the differences and extract features, which are used as the input of the gating mechanism. The gating mechanism dynamically fuses weights to deeply extract quality-related features under multiple conditions. Summary of the invention

[0004] The purpose of the present invention is to provide an industrial process soft measurement method of a multi-module self-attention gating network. The present invention introduces the original input error and the error of quality prediction in the stacked autoencoder network, establishes an input-output isomorphic autoencoder network to extract quality-related features, and uses the output of the input-output isomorphic autoencoder network (IOSIAE) as the input of the self-attention mechanism, uses the features extracted by multiple IOSIAE modules as the input of the self-attention mechanism, focuses on the difference, extracts the features, and uses them as the input of the gating mechanism. The gating mechanism dynamically fuses the weights to construct a multi-module self-attention gating network. Accurate online prediction of product quality is achieved, and product quality is improved.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A method for industrial process soft sensing of a multi-module self-attention gated network, the method comprising the following steps:

[0007] S1: Master the industrial process theory and basic operation procedures, and understand the relationship between process variables and quality variables;

[0008] S2: Collect and organize process data collected by industrial field instruments or samples obtained by industrial system simulation platforms;

[0009] S3: Divide the data into training set and test set according to 8:2;

[0010] S4: preprocessing the divided data sets;

[0011] S5: Use the input-output isomorphic autoencoder network to extract deep quality-related features, use the deep quality-related features extracted by IOSIAE as the input data of the autoencoder, and the features extracted by self-attention as the input of the gating mechanism. The gating mechanism dynamically assigns weights to build a hybrid model.

[0012] S6: Minimize the loss function to train the model and fine-tune the optimization model parameters;

[0013] S7: Calculate RMSE, MAE and R 2 The evaluation indicators are used to input the test data into the trained model for prediction to obtain the predicted values ​​of quality variables and improve product quality.

[0014] The industrial process soft measurement method of a multi-module self-attention gating network introduces label information and enhancement layers into the original input error and the error of quality prediction, establishes an input-output isomorphic autoencoder network to extract quality-related features, and uses the output of the input-output isomorphic autoencoder network (IOSIAE) as the input of the self-attention mechanism. The features extracted by multiple IOSIAE modules are used as the input of the self-attention mechanism, focusing on the differences, extracting features, and using them as the input of the gating mechanism. The gating mechanism dynamically fuses weights to construct a multi-module self-attention gating network to extract features.

[0015] The industrial process soft sensing method of a multi-module self-attention gated network, the training process is:

[0016] (1) Preprocess the data;

[0017] (2) Divide the training set and test set;

[0018] (3) Add the loss of reconstructing the original input and the loss of quality prediction to the autoencoder, establish an input-output homogeneous autoencoder network, and extract deep quality-related features;

[0019] (4) Construct a hybrid model, take the output of the input-output isomorphic autoencoder network (IOSIAE) as the input of the self-attention mechanism, take the features extracted by multiple IOSIAE modules as the input of the self-attention mechanism, focus on the differences, extract features, and use them as the input of the gating mechanism. The gating mechanism dynamically fuses weights and trains network parameters.

[0020] (5) Minimize the loss function to obtain the optimal parameters and obtain the soft sensor model;

[0021] The industrial process soft measurement method of a multi-module self-attention gated network, the test process is:

[0022] (1) Preprocess the test data;

[0023] (2) The processed data is input into the trained model to obtain the predicted values ​​of the key quality variables of industrial products and calculate the evaluation indicators RMSE, MAE and R 2 To evaluate the prediction effect of the model, finally draw conclusions and output the model.

[0024] The advantages and effects of the present invention are:

[0025] 1. Based on the autoencoder network, the present invention proposes an industrial process soft measurement method of a multi-module self-attention gating network (MEAGN). First, the loss of reconstructing the original input and the loss of quality prediction are added to the autoencoder, and the quality information is used to supervise and guide the process of feature learning. An input-output isomorphic autoencoder network is established. Considering the dynamic nature of the data, the output of the input-output isomorphic autoencoder network (IOSIAE) is used as the input of the self-attention mechanism, and the features extracted by multiple IOSIAE modules are used as the input of the self-attention mechanism, focusing on the differences, extracting features, and using them as the input of the gating mechanism, and the gating mechanism dynamically fuses weights. Real-time prediction of product quality is achieved while improving the stability of the prediction.

[0026] 2. The multi-module self-attention gating network of the present invention deeply extracts quality-related information and reduces information loss, ensuring the integrity of effective information. Compared with other models, the multi-module self-attention gating network has better prediction performance under nonlinearity and dynamics.

[0027] 3. The present invention meets the needs of real-time monitoring of actual industrial production operation and control of industrial product quality, and improves the stability and accuracy of online measurement. Real-time monitoring of process status to adjust abnormal conditions in the production process reduces production costs and ensures production safety and efficiency.

[0028] 4. In terms of quality prediction, the present invention can accurately predict the key quality variables of the actual industrial production process; in terms of online detection, it can monitor the operating status of the production process in real time, and realize the evaluation, control and detection of the process equipment; in terms of product output, it plays an important role in the output of high-quality products, and is of great significance in improving production efficiency and ensuring production safety, and ultimately achieves the goal of green production. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the technical route of the present invention;

[0030] Figure 2 is a prediction result diagram of quality variables of the sulfur recovery process of the present invention;

[0031] Figure 3 is a prediction error result diagram of quality variables of the sulfur recovery process of the present invention;

[0032] Figure 4 It is a prediction result diagram of quality variables of thermal power generation process of the present invention;

[0033] Figure 5 It is a schematic diagram of the overall structure of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be described in detail below with reference to the embodiments shown in the accompanying drawings.

[0035] Glossary:

[0036] IOSIAE: Input-Output Homomorphic Autoencoder;

[0037] MEAGN: Multi-module Self-Attention Gated Network;

[0038] The present invention proposes an industrial process soft sensing method based on a multi-module self-attention gated network (MEAGN), and the process is as follows:

[0039] S1: Master the theoretical knowledge of industrial processes, be familiar with basic operating procedures, and select process variables and quality variables;

[0040] S2: Collect and organize process data collected by industrial field instruments or samples obtained by industrial system simulation platforms

[0041] S3: Divide the data into training set and test set;

[0042] S4: preprocess the data;

[0043] S5: Add the loss of reconstructing the original input and the loss of quality prediction to the autoencoder, and use the quality information to supervise and guide the feature learning process. Establish an input-output isomorphic autoencoder network. Considering the dynamic nature of the data, the output of the input-output isomorphic autoencoder network (IOSIAE) is used as the input of the self-attention mechanism, and the features extracted by multiple IOSIAE modules are used as the input of the self-attention mechanism, focusing on the differences, extracting features, and using them as the input of the gating mechanism, which dynamically fuses weights;

[0044] S6: Update the objective function and calculate the evaluation indicators of RMSE, MAE and R2 to optimize the model parameters;

[0045] S7: Input the test data into the trained model for prediction, obtain the predicted value of quality variables, and improve product quality.

[0046] As a further improvement, the present invention uses the input-output isomorphic autoencoder described in step S5 to extract features, reduce the cumulative loss of original input information, and use quality information to guide learning. Finally, the output of the input-output isomorphic autoencoder network is used as the input of the self-attention mechanism to deeply extract quality-related features and efficiently predict product quality. The specific steps are as follows:

[0047] S51: Input and output homogeneous autoencoders are used to extract features, which can eliminate irrelevant information in the features and retain the effective reconstruction information of the model.

[0048] S52: The formula for reconstructing the original input of the input-output autoencoder is as follows:

[0049]

[0050] S53: The formula for introducing the hidden layer of the input and output autoencoder is as follows:

[0051] h=f(W e x+b e ) (2)

[0052] S54: The calculation formula of the difference focusing autoencoder is as follows:

[0053]

[0054] Among them, d k is the characteristic dimension of Q and K, is the scaling factor, D i is the difference matrix, a i is the difference weight of the i-th module.

[0055] S61: The loss function formula is as follows:

[0056]

[0057] The input-output homogeneous autoencoder network is constructed. Multiple input-output homogeneous autoencoders are stacked together to extract deep quality-related features. The output of the input-output homogeneous autoencoder is used as the input of the autoencoder, and the gating mechanism is used to predict the output. Due to the introduction of the original input reconstruction loss and the quality information prediction loss, the proposed features reflect the mapping relationship between input and output well.

[0058] As the instruction manual Figure 5 The method is a multi-module self-attention gating network industrial process soft measurement method, which is based on input-output isomorphic autoencoders, self-attention mechanisms and gating mechanisms to achieve real-time monitoring of industrial processes and effective prediction of product quality. The process of predicting quality variables in the present invention is as follows:

[0059] Training phase:

[0060] (1) Data preprocessing;

[0061] (2) Initialize network structure and parameters;

[0062] (3) Reconstruct the original input data and the prediction error of the target value to train the first layer of the network, thereby obtaining the first layer features. The features of the previous layer are used as the input of the subsequent layers, and this process is repeated until all layers are trained, and the weight and bias parameters {W eL ,b eL},L=1,2...,L;

[0063] (4) The output layer of the input-output isomorphic autoencoder network is used as the input of the self-attention mechanism, and the pre-trained parameters {W eL ,b eL}, L = 1, 2..., L is used as the initial value of the IOSIAE network parameters and the output layer is initialized;

[0064] (5) Input the processing results of IOSIAE into the self-attention mechanism to obtain the feature information after differential aggregation, and use it as the input of the gating mechanism for prediction;

[0065] (6) Fine-tune network parameters using the Adam optimizer and prediction error loss function;

[0066] (7) Save the network structure and parameters.

[0067] Testing phase:

[0068] The same method is used for data preprocessing in the testing phase and the training phase, and the predicted value is obtained using the trained model. The root mean square error (RMSE), mean absolute error (MAE) and coefficient of determination (R 2 ) evaluation index to verify the prediction effect of the model. As shown in equations (6), (7) and (8):

[0069]

[0070] Where N is the number of test samples; is the i-th predicted value, y i is the i-th true value. is the average value of the output value in the test data set. The closer RMSE and MAE are to 0, the higher the R 2 The closer to 1, the better the effect.

[0071] Example 1

[0072] Sulfur recovery is an important part of the chemical process, which is used for chemical conversion and environmental protection. In the industrial process, in order to predict the concentration of sulfur dioxide. The present invention collects 5 process variables as model inputs, including gas flow MEA gas, air flow AIR, MEA, secondary air flow AIRMEA2, SWS zone air flow, SWS zone air flow, and sulfur dioxide concentration in tail gas. 10071 samples were selected, of which the first 8000 samples were used as training sets to train model parameters, and the remaining samples were used as test sets to evaluate the model prediction ability.

[0073] Table 1 Performance evaluation indicators of different models of debutanizer process

[0074]

[0075] From Table 1 and the instruction manual Figure 2 And the instruction manual Figure 3 It can be seen that the evaluation performance indicators of the MEAGN model are better than those of other models, and its predicted values ​​can track the changes of the true values ​​well. This is because MEAGN plays a supervisory and guiding role in quality information, while ensuring the integrity of feature information extraction, as well as good nonlinear and dynamic learning capabilities, which greatly improves the prediction performance.

[0076] Example 2

[0077] Thermal power generation is a commonly used industrial power generation method. Its core lies in the energy generated during the combustion process of converting water into steam, which is used to drive the generator to generate electricity. Among them, steam flow rate (SF) is a key indicator affecting power generation efficiency. Therefore, developing a soft sensor model to predict SF is crucial to improving power generation efficiency. There are 38 factors that affect SF, including the oxygen intake of the boiler, the feed rate of combustibles, and the amount of water intake. The quality variable is steam flow rate (SF). A total of 2888 samples were collected, the first 2200 samples were used as training sets, and the rest were used for model testing.

[0078] Table 2 Performance evaluation indicators of different models in thermal power generation process

[0079]

[0080] Table 2 and the instruction manual Figure 4 The results show that the trained MEAGN model has the smallest RMSE and MAE values ​​compared with the other four models. 2 The value is the largest, and the predicted result matches the actual curve better in terms of value and trend, and the predicted result is closest to the true value. This is mainly because the present invention can deeply extract quality-related features under multiple working conditions and ensure the integrity of effective information. Therefore, the online prediction ability of the MEAGN soft measurement model in thermal power generation is better than that of the traditional model, and it is a soft measurement model more suitable for the thermal power generation process.

[0081] The specific embodiments of the present invention have been disclosed above. The present invention is applicable to various fields mentioned in the specification and other suitable fields. Without departing from the general concept defined by the claims and their equivalent scope, the present invention is not limited to the description illustrations and specific related details given in the specification.

Claims

1. A method for industrial process soft sensing based on a multi-module self-attention gated network, characterized in that: The steps of the method are as follows: S1: Familiar with the specific operation process of actual industrial production and understand the relationship between process variables and quality variables; S2: Acquire samples based on on-site collection of process data from industrial devices or industrial system simulation platforms; S3: Divide the data into training set and test set according to 8:2, and preprocess the data; S4: Use the input-output isomorphic autoencoder network to extract deep quality-related features, use the deep quality-related features extracted by IOSIAE as the input data of the autoencoder, and the features extracted by self-attention as the input of the gating mechanism. The gating mechanism dynamically assigns weights to build a hybrid model. S5: Minimize the loss function to train the model and optimize the model parameters; S6: Input the test data into the trained soft measurement model to obtain the predicted values ​​of the key quality variables of the product, monitor the production process in real time, and adjust the operation plan to ensure normal production operation.

2. The industrial process soft sensing method of a multi-module self-attention gated network according to claim 1 is characterized in that: The method is familiar with the operation flow of industrial processes and the processing of data.

3. The industrial process soft sensing method of a multi-module self-attention gated network according to claim 3 is characterized in that: The method uses an input-output isomorphic autoencoder network to extract deep quality-related features, uses the deep quality-related features extracted by IOSIAE as the input data of the autoencoder, and uses the features extracted by self-attention as the input of the gating mechanism. The gating mechanism performs dynamic weighting to construct a hybrid model.

4. The industrial process soft sensing method of a multi-module self-attention gated network according to claim 1 is characterized in that: The training part is: (1) Preprocess the data; (2) Divide the training set and test set; (3) Add the original input loss and the quality prediction loss to the autoencoder to establish an input-output homogeneous autoencoder network and extract deep quality-related features; (4) Construct a hybrid model and use the input-output isomorphic autoencoder network to extract deep quality-related features. The deep quality-related features extracted by IOSIAE are used as the input data of the autoencoder, and the features extracted by self-attention are used as the input of the gating mechanism. The gating mechanism dynamically integrates the weighted prediction and trains the network parameters. (5) Minimize the loss function to obtain the optimal parameters and obtain the soft sensor model.

5. The industrial process soft sensing method of a multi-module self-attention gated network according to claim 1 is characterized in that: The test parts are: (1) Preprocess the test data; (2) The processed data is input into the trained model to obtain the predicted values ​​of the key quality variables of industrial products and calculate the evaluation indicators RMSE, MAE and R 2 To evaluate the prediction effect of the model, finally draw conclusions and output the model.

Images