Process industry soft measurement method based on feature extraction and incremental learning

Abstract

The invention provides a process industry soft measurement method based on feature extraction and incremental learning, and the method comprises the steps: introducing an end-to-end neural network soft measurement model which is formed by connecting a plurality of parallel time sequence feature extraction auto-encoder networks and a regression device network in a front-and-back manner; and extracting time sequence characteristics from the multi-process variable time sequence data, and applying the time sequence characteristics to performance index soft measurement modeling. In addition, the method also provides an incremental learning method based on time sequence characteristics based on the time sequence characteristics extracted by the model. The soft measurement method provided by the invention can effectively and accurately carry out soft measurement on some performance indexes which are difficult to directly measure or high in measurement cost in the process industry, and ensures that the soft measurement model keeps better soft measurement performance for a long time in an industrial data stream with stronger real-time performance through an incremental learning method. And reference and guidance are provided for monitoring, measurement, optimization and control of various parameters in the industrial production process.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology and artificial intelligence, and in particular to a process industry soft measurement method based on feature extraction and incremental learning. The method is mainly used to establish a soft measurement model for a series of important performance indicators in the process of process industry production, and to carry out real-time soft measurement using industrial data streams. Background Art

[0002] Some terms involved in the present invention are introduced below.

[0003] (1) Process Industry: Process Industry refers to the industrial process of converting raw materials into high value-added products by using physical, chemical and other processes. This type of industry usually involves continuous material processing and complex physical changes, chemical reactions, etc. Chemical industry, petroleum, cement, metallurgy, etc. all fall within the scope of process industry. The biggest feature of process industry is that its production process is usually divided into multiple different processes, and the process in each process is usually more complicated.

[0004] (2) Process variables: Process variables refer to key parameters used to describe and control the process in the process industry. These variables usually include various physical and chemical properties, such as temperature, pressure, flow, liquid level, concentration, etc. The real-time data of these process variables can reflect the real-time status of the process industry production process.

[0005] (3) Performance Indicator: Performance indicators refer to key parameters used to evaluate and measure product quality, production efficiency and process stability in process industries. These indicators usually include the chemical and physical properties of the product and some summary parameters of the production process, such as energy consumption and product conversion rate. By monitoring and analyzing these indicators, the efficiency of the process industry production process and the quality of the output products can be ensured.

[0006] (4) Soft Sensing: Soft sensing refers to a technical method in process industry that uses some process variables that are easy to measure or measure frequently to indirectly estimate performance indicators that are difficult to measure directly or frequently through mathematical models and algorithms. The mathematical model used for soft sensing modeling can be a mechanism model based on the mechanism of process industry, or a data-driven model based on data statistics and machine learning methods. Soft sensing integrates the data of multiple process variables and establishes a prediction model through regression algorithms to achieve real-time fitting and estimation of key performance indicators.

[0007] (5) Feature extraction: Feature extraction refers to the process of extracting representative and significant features (i.e., data attributes or variables) from raw data by analyzing and processing them in data processing. These features are used to describe and characterize key information of the data to facilitate subsequent modeling, analysis, and decision-making.

[0008] (6) Incremental Learning: Incremental learning refers to a method of continuously improving model performance by gradually updating and learning new data during the machine learning process. Incremental learning can adapt to the ever-changing data environment and keep the model up-to-date and effective.

[0009] The following is an introduction to the background technology of the present invention.

[0010] In the process industry, due to the mutual coupling of physical and chemical reactions and the complex thermal state in the production process, there is often a complex nonlinear relationship between process variables and the required soft measurement performance indicators. Such a relationship is difficult to describe through precise expressions or mechanism models, so it is often necessary to establish a data-driven soft measurement model to obtain real-time estimation results of the required soft measurement performance indicators.

[0011] Early soft sensor modeling methods mainly rely on classical statistical learning methods, such as partial least squares (PLS) regression and support vector regression (SVR) to fit the relationship between process variables and soft sensor targets. These methods have fast modeling speed and strong interpretability, but they do not take into account the nonlinear relationship between performance indicators and process variables in process industries, so their modeling capabilities have an obvious upper limit. With the proposal and development of artificial neural networks, technicians in related fields have developed a series of modeling methods based on neural networks, such as back propagation neural networks (BPNN), random vector functional link networks (RVFL), convolutional neural networks (CNN) and recurrent neural networks (RNN), etc. These networks have certain processing capabilities for the above nonlinear relationships. Based on these networks, some more complex soft sensor modeling frameworks based on deep networks have been proposed in recent years, such as autoencoders and twin neural networks. However, the current methods for constructing soft sensor models based on deep networks usually rely on using a single network structure to extract a set of features, and hope that these features can fully represent the knowledge in the original data. This method has very obvious limitations for process industries with numerous processes and complex reactions. How to combine the characteristics of process industry to construct a specific network structure and feature extraction method is the key to further improving the performance of soft sensor models based on deep networks.

[0012] Another problem faced by soft sensor modeling in process industry is that when the soft sensor model trained on offline data is deployed on the corresponding production line, its performance may decline over time. This is caused by the "concept drift" phenomenon in industrial data streams. There are many factors that cause "concept drift", including changes in process conditions, changes in production materials, the existence of abnormal conditions, etc. In order to achieve the continuous update of soft sensor models in industrial data streams, technicians have introduced a series of continuous update strategies, such as recursive modeling methods, sliding window methods, just-in-time learning methods, and incremental learning methods. Recursive modeling methods are usually limited to combining with some statistical learning methods and are not applicable to complex neural networks; sliding window methods can be used in combination with any data-driven methods, but each model update may lead to the loss of a large amount of historical knowledge; just-in-time learning methods retrain the model when new situations occur in industrial data streams, but ignore the persistence and accumulation of knowledge required for soft sensor models. Incremental learning is a continuous and adaptive method. This method uses the concept of "knowledge base" to continuously update the knowledge base and use the knowledge base to adjust the model parameters in real time, so that the model can maintain a continuous learning state to adapt to the drift of industrial data streams. However, most neural network-based incremental learning methods usually retrain the entire model when updating, which may lead to catastrophic forgetting of important previously acquired knowledge and cause unnecessary waste of computing resources. How to develop a stable and efficient incremental learning method is the current technical focus of incremental learning algorithm development. Summary of the invention

[0013] In view of the above technical background, the present invention proposes a process industry soft sensing method based on feature extraction and incremental learning.

[0014] The technical solution of the present invention is as follows:

[0015] A process industry soft sensing method based on feature extraction and incremental learning includes the following steps:

[0016] (1) Read data from the DCS database, classify data variables according to the process, and construct a multi-process process variable time series dataset for pre-training the end-to-end neural network soft sensor model;

[0017] The end-to-end neural network soft measurement model is composed of multiple parallel time series feature extraction autoencoder networks and a regressor network connected front and back, wherein the time series feature extraction autoencoder network is used to extract a set of one-dimensional time series features from the two-dimensional process variable time series of the corresponding process; the regressor network is used to integrate the high-dimensional time series features extracted by multiple time series feature extraction autoencoder networks, and fit the soft measurement results of the performance indicators; the multi-process process variable time series data set is composed of multiple groups of process variable time series data and performance indicator soft measurement label data, and its construction method is: first, the process variables available in the DCS (distributed control system) database are divided into multiple groups, and appropriate time series lengths are set for each of them, and then the timestamp of the soft measurement performance indicator label value collection in the DCS database is used as the starting point, and the process variable time series of corresponding lengths are respectively intercepted forward according to the group of the process variable, so that each performance indicator label value within the data set range can correspond to the corresponding multiple process variable time series;

[0018] (2) setting pre-training parameters and using the constructed process variable time series dataset and the designed loss function to pre-train the end-to-end neural network soft sensor model;

[0019] (3) saving the pre-trained end-to-end neural network soft measurement model, and extracting corresponding features from the process variable time series data set through the soft measurement model to build a process knowledge base, wherein the process knowledge base is a collection of time series features extracted from the historical data process variable time series;

[0020] (4) The pre-trained soft measurement model, process knowledge base, and incremental learning algorithm are deployed on the online platform. While the model is performing the soft measurement task, the corresponding incremental learning program is run to enable the parameters of the regression network part of the model to be continuously updated in the industrial data stream.

[0021] (5) Uploading the soft measurement results of the model and the incremental learning process to the DCS database for storage; the incremental learning process includes the size and composition of the process knowledge base, the time when the incremental learning of the soft measurement model occurs, the length of time consumed by the incremental learning process, etc.

[0022] The temporal feature extraction self-encoder network of the end-to-end neural network soft measurement model described in step (1) is a hierarchical temporal feature extraction self-encoder network consisting of an encoder network and a decoder network; the encoder network of the hierarchical temporal feature extraction self-encoder network is composed of a self-attention mechanism layer, a long short-term memory (Long Short Term Memory, LSTM) network layer, a self-attention mechanism layer and a temporal convolution network (Temporal Convolution Network, TCN) encoder layer connected in sequence; the decoder network of the hierarchical temporal feature extraction self-encoder network is composed of a TCN decoder layer, a point-by-point convolution layer and an LSTM layer connected in sequence;

[0023] The encoder network of the hierarchical time series feature extraction self-encoder network first uses a layer of self-attention mechanism to weight the important variables and time steps in the process variable time series X to obtain the weighted process variable time series X att ; for X att Take the average value along the time dimension to get the global feature Z g On this basis, the LSTM network layer and a layer of attention mechanism are used to analyze X att Processing is performed to obtain the dynamic information sequence H; taking the result of the last time step of H, the dynamic feature Z is obtained v ; Further, the TCN autoencoder is used to process H, and its feature vector of the last time step is retained to obtain the local feature Z l ; In the extracted global feature Z g , dynamic feature Z v and local feature Z l A residual connection is established between them, and the final output is Z g , Z v and Z l The concatenated feature set Z;

[0024] The hierarchical temporal features are extracted from the decoder network of the encoder network. The decoder network first passes through the TCN decoder layer, and reconstructs the two-dimensional reconstructed dynamic sequence using the one-dimensional feature set Z output by the encoder network. Furthermore, the point-by-point convolution layer is introduced The variable dimension is adjusted; finally, the process variable time series is reconstructed through the LSTM layer

[0025] The pre-training parameters in step (2) include the number of training rounds, the training batch size, the learning rate, and the drop rate; the loss function is a semi-supervised loss function, expressed as:

[0026]

[0027] in, are the supervised loss function term and the unsupervised loss function term respectively; λ1 and λ2 are the regularization coefficients of the loss function; The expression is:

[0028]

[0029] Among them, B is the number of training batches, Y b is the true value of the performance indicator label in the bth batch, and is the prediction result of the model for the performance index in the bth batch, The expression is:

[0030]

[0031] Among them, M is the number of processes, L m is the length of the process variable time series of the mth process, N m is the variable dimension of the process variable time series of the mth process, is the value of the input process variable time series at the position (b, l, n), and is the value of the reconstructed process variable time series at the position (b, l, n).

[0032] The online platform in step (4) refers to a system platform for deploying and running soft measurement models and their incremental learning programs, including cloud computing platforms, industrial Internet of Things platforms, and digital twin platforms; the incremental learning program refers to a corresponding program and algorithm for updating the process knowledge base in real time based on the constructed process knowledge base, using the time series features extracted from the real-time industrial data stream received by the online platform by the deployed soft measurement model, and instantly training and iterating the parameters of the regressor network part of the soft measurement model to ensure that the soft measurement model maintains soft measurement performance in the real-time industrial data stream; the industrial data stream refers to the process data stream read in real time by the online platform from the process industry data source including various sensors, equipment, and control systems.

[0033] The incremental learning procedure described in step (4) includes the following steps:

[0034] (4.1) Through the time series feature extraction of the neural network soft measurement model, the new process variable time series X is input into the soft measurement model from the current moment of the industrial data stream. in In the above example, the corresponding new time series feature Z is extracted. in ;

[0035] (4.2) is the temporal feature Z in the constructed process knowledge base K. i , calculate Z in and Z i The similarity index Sim i (Z i ,Z in ), the similarity index Sim i (Z i ,Z in ) is calculated as:

[0036]

[0037] Among them, ||Z i || and ||Z in || represents Z i and Z in The norm of , ∑ represents the covariance matrix corresponding to the two;

[0038] (4.3) Determine each similarity index Sim calculated i (Z i ,Z in ) are all less than a set threshold s min If yes, add Z to the current knowledge base K in ; If not, terminate this round of incremental learning process;

[0039] (4.4) Assume that the result of removing the oldest set of time series features from K is K′, calculate the information loss InfoLoss(K,K) caused by removing this set of time series features. ′ ); when InfoLoss(K,K ′ ) is less than a set threshold l min When , the oldest set of time series features is removed from K; the information loss InfoLoss(K,K ′ ) is calculated as:

[0040]

[0041] InfoLoss(K,K ′ )=H(K)-H(K ′ )

[0042] Among them, P(X i,j ) represents the probability that the elements of matrix X appear in the i-th box and the j-th feature position after the value range of the j-th feature is binned;

[0043] (4.5) Determine whether the length of K after the update is divisible by a set integer bs. If so, use the current K as the training data set and use the supervised loss function to fine-tune the regressor network part to update the parameters of the regressor network part of the soft sensor model; if not, directly terminate this round of incremental learning process; the expression of the supervised loss function is:

[0044]

[0045] The beneficial effects of the present invention are:

[0046] This incremental learning method based on time series features can directly use the current set of time series features to update the regressor network part of the proposed end-to-end neural network soft measurement model, thereby avoiding the repeated training of the encoder network part for extracting time series features with strong versatility. While significantly saving computing resources, it also ensures the soft measurement performance of the soft measurement model in real-time industrial data streams, thereby improving the soft measurement performance and service life of the proposed process industry performance indicator soft measurement model. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 The present invention proposes a flow chart of a process industry soft sensing method based on feature extraction and incremental learning.

[0048] Figure 2 It is a schematic diagram of the workflow of a hierarchical temporal feature extraction autoencoder network proposed in the present invention.

[0049] Figure 3 It is a flow chart of an incremental learning method based on time series features proposed by the present invention. DETAILED DESCRIPTION

[0050] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings. It should be noted that the various symbols in the expressions involved in the present invention are only for simplification and clear expression, and different examples may have different expressions. Therefore, the key to the present invention lies in the method of extracting process variable data features in the process industry, and the incremental learning ideas based on the extracted features; in addition, in the subsequent examples of the specific implementation of the present invention, the numbers "A", "B", "C" and so on are only for distinguishing the contents of different objects, and have no other special meanings.

[0051] The purpose of the present invention is to provide a simple, efficient and versatile process industry soft sensing method based on feature extraction and incremental learning. For example, for a soft sensing task of a process industry process divided into M steps, the implementation method of the present invention is as follows: Figure 1As shown, it specifically includes the following 5 steps:

[0052] Step 1.1: Read data from the DCS database, classify the data variables according to the process, and construct a multi-process process variable time series dataset for pre-training the end-to-end neural network soft sensing model.

[0053] The first step of the present invention is to prepare a data set for pre-training a soft measurement model. In the present invention, the input of the soft measurement model is a time series with a fixed length composed of process variable data arranged in chronological order, rather than a single process variable data point. The expansion of the input dimension of the soft measurement model is conducive to the model extracting more comprehensive and extensive time series features from the process variable data. The method for constructing a multi-process process variable time series data set is: first, the process variables available in the DCS (Distributed Control System) database are divided into (A, B, C, ... M) M groups, and appropriate time series lengths (L) are set for each of them. A , L B , L C , ...L M ), then take the timestamp of the soft measurement performance index label value acquisition in the DCS database as the starting point, and intercept the length of (L A , L B , L C , ...L M ) of the process variable time series, so that each performance indicator label value within the data set can correspond to the corresponding M process variable time series.

[0054] In the present invention, the soft measurement model adopted is an end-to-end neural network soft measurement model, which is mainly composed of M parallel time series feature extraction self-encoder networks and a regressor network connected front and back. Among them, the function of the time series feature extraction self-encoder network is to extract a set of one-dimensional time series features from the two-dimensional process variable time series of the corresponding process, and the function of the regressor network is to integrate the high-dimensional time series features extracted by the M time series feature extraction self-encoder networks to fit the soft measurement results of the performance indicators.

[0055] Step 1.2: Set pre-training parameters and use the constructed process variable time series dataset and the designed loss function to pre-train the end-to-end neural network soft sensor model.

[0056] In the present invention, it is necessary to use the constructed process variable time series data set to pre-train the adopted end-to-end neural network soft measurement model, so that the soft measurement model obtains a set of preliminary parameters. For the pre-training process, it is first necessary to set a series of pre-training parameters, such as the number of training rounds, training batch size, learning rate, discard rate, etc. On this basis, it is necessary to construct a loss function to guide the training of the model. In the present invention, a semi-supervised loss function composed of a supervised loss function of prediction accuracy and an unsupervised loss function of time series reconstruction accuracy is used to guide the training of the model. Among them, the prediction accuracy loss, that is, the expression of the supervised loss function term is:

[0057]

[0058] Among them, B is the number of training batches, Y b is the true value of the performance indicator label in the bth batch, and is the prediction result of the model for the performance index in the bth batch. In addition, the time series reconstruction accuracy loss, that is, the expression of the unsupervised loss function term is:

[0059]

[0060] Among them, M is the number of processes, L m is the length of the process variable time series of the mth process, N m is the variable dimension of the process variable time series of the mth process, is the value of the input process variable time series at the position (b, l, n), and is the value of the reconstructed process variable time series at the position (b, l, n). Finally, the overall semi-supervised learning loss function is expressed as:

[0061]

[0062] Among them, λ1 and λ2 are regularization coefficients of the loss function, which are used to adjust the influence of each part of the loss function. Using the above loss function, the soft measurement model is pre-trained, so that the soft measurement model can further improve the prediction accuracy of the model by using unlabeled data during the parameter update process.

[0063] Step 1.3: Save the pre-trained end-to-end neural network soft measurement model, and extract corresponding features from the process variable time series data set through the soft measurement model to build a process knowledge base.

[0064] In the present invention, in order to carry out the incremental learning of the subsequent soft measurement model, it is necessary to construct a process knowledge base to store the knowledge learned by the soft measurement model from the historical data. The process knowledge base is a collection of time series features extracted from the time series of the process variables in the historical data, which contains relatively rich process knowledge that can be used by the soft measurement model. Each group of time series in the process variable time series data set is input into the pre-trained process variable soft measurement model to obtain the corresponding time series features, and all the time series features are summarized to obtain the corresponding process knowledge base.

[0065] Step 1.4: Deploy the pre-trained soft measurement model, process knowledge base and incremental learning algorithm on the online platform. While the model is performing the soft measurement task, run the corresponding incremental learning program to enable the parameters of the regression network part of the model to be continuously updated in the industrial data stream.

[0066] In the present invention, the pre-trained soft measurement model needs to be run in an online environment to achieve the effect of real-time soft measurement. This process needs to be completed with the help of an online platform. The online platform refers to a system platform for deploying and running soft measurement models and their incremental learning programs, such as cloud computing platforms, industrial Internet of Things platforms, digital twin platforms, etc.

[0067] In the present invention, while deploying the pre-trained soft measurement model on the online platform to perform the soft measurement task, the constructed process knowledge base and the supporting incremental learning algorithm for real-time updating of the soft measurement model parameters are also deployed. This incremental learning algorithm is based on the constructed process knowledge base, uses the deployed soft measurement model to extract the time series features from the real-time industrial data stream received by the online platform to update the process knowledge base in real time, and trains and iterates the parameters of the regressor network part of the soft measurement model in real time to ensure that the soft measurement model maintains good soft measurement performance in the real-time industrial data stream for a long time. In the present invention, the industrial data stream refers to the process data stream read by the online platform from various sensors, equipment, control systems and other data sources in the process industry, which has high real-time performance.

[0068] Step 1.5: Upload the model's soft measurement results and incremental learning process to the DCS database for storage.

[0069] In the present invention, it is necessary to upload the soft measurement results of the deployed soft measurement model to the DCS database to achieve long-term storage of the soft measurement results, thereby providing immediate reference for downstream tasks such as optimization and control. In addition, the process of incremental learning of the deployed soft measurement model in the industrial data stream will also be uploaded and stored in the DCS database. In the present invention, the incremental learning process includes the size and composition of the process knowledge base, the occurrence time of the incremental learning of the soft measurement model, the length of time consumed by the incremental learning process, etc. This information has a high reference value for the optimization of the incremental learning algorithm, as well as the monitoring and inference of changes in process conditions in the process industry.

[0070] The present invention proposes a process industry soft measurement method based on feature extraction and incremental learning, which can effectively and accurately perform soft measurement on some performance indicators in the process industry that are difficult to measure directly or have high measurement costs. In addition, the incremental learning method is used to ensure that the soft measurement model can maintain good soft measurement performance in the real-time industrial data stream for a long time, thereby providing reference and guidance for the monitoring, measurement, optimization and control of various parameters in the production process of the process industry.

[0071] The above is an introduction to a process industry soft measurement method based on feature extraction and incremental learning. Among them, the structural design of the temporal feature extraction autoencoder network not only determines the performance upper limit of the soft measurement model, but also determines the composition of the process knowledge base extracted from historical data and the effect of incremental learning based on the process knowledge base. Therefore, designing a reliable and efficient temporal feature extraction autoencoder network plays a key role in the practical application effect of the present invention. The following is an introduction to a hierarchical temporal feature extraction autoencoder network proposed by the present invention.

[0072] Figure 2 Schematic diagram of the workflow of a hierarchical temporal feature extraction autoencoder network proposed in the present invention. Figure 2 ,The designed hierarchical temporal feature extraction self-encoder network consists of two parts: the encoder network and the decoder network.

[0073] The encoder network of the designed autoencoder network is composed of a self-attention mechanism layer, an LSTM network layer, a self-attention mechanism layer, and a TCN encoder layer connected in sequence. In the designed encoder network, after the process variable time series X is input into the autoencoder part of the network, it will first be processed by a layer of self-attention mechanism to weight the important variables and time steps in the input time series. att By averaging along the time dimension, we can obtain a set of one-dimensional feature vectors Z g, this set of feature vectors represents the global average level of the input process variable time series, so it is named global feature. On this basis, a set of LSTM networks are used to process X att , and then through a layer of attention mechanism, the processing results of the LSTM network are weighted twice to obtain a dynamic information sequence H representing the dynamic changes of the input process variable time series. According to the principle of the LSTM network, taking the result of the last time step of H, another set of one-dimensional feature vectors Z can be obtained v , this set of feature vectors represents the dynamic level of the input process variable time series, so it is named dynamic feature. Finally, a TCN autoencoder is introduced to process the extracted H through the convolutional network:

[0074]

[0075] In the above expression, k and d represent the size and expansion rate of the TCN convolution kernel, respectively, and w represents the weight of the convolution kernel. Due to the characteristics of causal convolution adopted by TCN, the output of TCN at each time step is only affected by the previous information. Finally, combined with the above characteristics of TCN, the feature vector Z of the last time step output by TCN is saved l This set of feature vectors contains the fine-grained spatiotemporal variation information of the input process variable time series, so it is named local feature. The designed encoder network extracts the global feature Z g , dynamic feature Z v and local feature Z l A residual connection is established between them, and the final output is Z g , Z v and Z l The concatenated feature set Z.

[0076] The decoder network of the designed autoencoder network is composed of a TCN decoder layer, a point-by-point convolution layer and an LSTM layer connected in sequence. In the designed decoder network, a TCN decoder is first introduced to reconstruct the two-dimensional reconstructed dynamic sequence using the one-dimensional feature set Z output by the encoder network. For the reconstructed A point-by-point convolution layer is introduced, that is, a convolution layer with a convolution kernel of size 1 in each dimension to adjust its variable dimension, and finally reconstruct the process variable time series through a layer of LSTM network.

[0077] The hierarchical time series feature extraction autoencoder network proposed in the present invention can reliably and efficiently extract various time series features from the input process variable time series. The design of this autoencoder network structure can not only improve the overall performance of the soft measurement model, but also improve the diversity of the extracted time series features and the richness of the information contained, thereby providing support for a series of downstream tasks based on time series features.

[0078] The above is an introduction to a hierarchical temporal feature extraction autoencoder network proposed in the present invention. This autoencoder network constitutes the temporal feature extraction autoencoder network part of the end-to-end neural network soft measurement model proposed in the present invention. In addition, the present invention does not limit the form and structure of the regressor network part of the proposed neural network soft measurement model. Any network that can obtain performance index prediction results by fitting high-dimensional temporal features can be used as the regressor network part of the proposed neural network soft measurement model.

[0079] Based on the overall technical solution proposed by the present invention and the hierarchical temporal feature extraction autoencoder network proposed by the present invention, an efficient and fast incremental learning method based on the temporal features extracted by the end-to-end neural network soft measurement model can be constructed, thereby guiding the soft measurement model to maintain good soft measurement performance in the real-time industrial data stream for a long time. The following is an introduction to an incremental learning method based on temporal features proposed by the present invention.

[0080] Figure 3 It is a flow chart of an incremental learning method based on time series features proposed in the present invention. It should be noted that the proposed incremental learning method is carried out in parallel with the soft measurement process of the model. Figure 3 ,The proposed incremental learning method specifically includes the following five steps:

[0081] Step 2.1: Extract the time series features of the neural network soft measurement model from the encoder network and input the new process variable time series X of the soft measurement model from the current moment of the industrial data stream in In the above example, the corresponding new time series feature Z is extracted. in .

[0082] Step 2.2: For each temporal feature Z in the constructed process knowledge base K i , calculate Z in and Z i The similarity index Sim i (Z i ,Z in ). i (Z i ,Z in ) is calculated as:

[0083]

[0084] Among them, ||Z i || and ||Z in || represents Z i and Z in The norm of , ∑ represents the covariance matrix corresponding to the two. For each Z in K i , all calculated to get a Sim i (Z i ,Z in ), thus achieving Z in With each Z in K i Comprehensive comparison of knowledge similarity.

[0085] Step 2.3: Determine each similarity index Sim calculated i (Z i ,Z in ) are all less than a set threshold s min If yes, add Z to the current knowledge base K in ; If not, terminate this round of incremental learning process.

[0086] Specifically, if each similarity index Sim calculated i (Z i ,Z in ) are both smaller than s min , then it means that the X input into the soft sensor model at the current moment in The Z extracted from in The similarity with each time series feature vector in K is not high, so it can be considered that Z in is a new knowledge independent of K, so it should be added to K; on the contrary, when part Z i and Z in Sim i (Z i ,Z in ) is greater than s min When Z in And this part Z in K i The similarity is large, so it is considered that Z in It is not independent of K, so there is no need to perform the subsequent incremental learning process.

[0087] Step 2.4: Assume that the result of removing the oldest set of time series features from K is K′, calculate the information loss InfoLoss(K,K) caused by removing this set of time series features. ′ ). When InfoLoss(K,K ′ ) is less than a set threshold lmin When , the oldest set of time series features is removed from K.

[0088] In the process of executing most incremental learning algorithms, as industrial data streams continue to be input, the size of the knowledge base will continue to expand, which may lead to two problems: first, the training time and computing resources required for subsequent regression network updates will continue to increase; second, the old knowledge in the knowledge base may not provide effective information for soft sensor modeling under the latest working conditions, and may even have a negative impact. Therefore, each time a new time series feature Z is incorporated into K, in Later, a logic to determine whether to remove the oldest set of time series features in K will be executed, and based on the result of the logic determination, it will be determined whether to remove the oldest set of time series features from K. ′ ) is calculated as:

[0089]

[0090] InfoLoss(K,K ′ )=H(k)-H(k ′ )

[0091] Among them, P(X i,j ) represents the probability that the elements of matrix X appear in the i-th bin and the j-th feature position after the value range of the j-th feature is binned. Through the above knowledge base discrimination and operation mechanism, the scale of k can be controlled while ensuring the richness of its overall information.

[0092] Step 2.5: Determine whether the length of the updated k is divisible by a set integer bs. If so, use the current K as the training data set and use the supervised loss function to fine-tune the regressor network part to update the parameters of the regressor network part of the soft measurement model; if not, terminate this round of incremental learning process directly.

[0093] In the designed incremental learning algorithm, if fine-tuning training of the regressor network is performed after each update of K, it may consume a lot of training time and computing resources, and have a negative impact on the real-time performance of the soft measurement model. Therefore, a more feasible solution is to wait until K accumulates to a certain scale before performing fine-tuning training of the regressor network, so as to achieve a balance between the real-time performance and training resources of the soft measurement model. The integer bs set should be consistent with the training batch size used in the fine-tuning training process. In the fine-tuning training process of the regressor network, the calculation expression of the supervised loss function used is:

[0094]

[0095] An incremental learning method based on time series features proposed in the present invention can directly use the current set K of time series features to update the regressor network part of the proposed end-to-end neural network soft measurement model, thereby avoiding repeated training of the time series feature extraction autoencoder network part with strong versatility, while significantly saving computing resources, it also ensures the soft measurement performance of the soft measurement model in real-time industrial data streams, thereby improving the soft measurement performance and service life of the proposed process industry performance indicator soft measurement model.

[0096] It should also be noted that the terms "such as", "including" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentences "such as...", "including..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0097] For those skilled in the art, the present invention may have various changes and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A process industry soft sensing method based on feature extraction and incremental learning, characterized in that: The steps include: (1) Read data from the DCS database, classify data variables according to the process, and construct a multi-process process variable time series dataset for pre-training the end-to-end neural network soft sensor model; The end-to-end neural network soft measurement model is composed of a plurality of parallel time series feature extraction autoencoder networks connected front and back to a regressor network, wherein the time series feature extraction autoencoder network is used to extract a set of one-dimensional time series features from a two-dimensional process variable time series of a corresponding process; The regressor network is used to extract high-dimensional time series features extracted by the encoder network by synthesizing multiple time series features, and fitting to obtain soft measurement results of performance indicators; The multi-process process variable time series data set is composed of multiple groups of process variable time series data and performance indicator soft measurement label data. The construction method is: first, the process variables available in the DCS (distributed control system) database are divided into multiple groups, and appropriate time series lengths are set for each of them. Then, the timestamp of the soft measurement performance indicator label value acquisition in the DCS database is used as the starting point, and the process variable time series of corresponding lengths are intercepted forward according to the group of the process variable, so that each performance indicator label value within the data set can correspond to the corresponding multiple process variable time series; (2) setting pre-training parameters and using the constructed process variable time series dataset and the designed loss function to pre-train the end-to-end neural network soft sensor model; (3) saving the pre-trained end-to-end neural network soft measurement model, and extracting corresponding features from the process variable time series data set through the soft measurement model to build a process knowledge base, wherein the process knowledge base is a collection of time series features extracted from the historical data process variable time series; (4) The pre-trained soft measurement model, process knowledge base, and incremental learning algorithm are deployed on the online platform. While the model is performing the soft measurement task, the corresponding incremental learning program is run to enable the parameters of the regression network part of the model to be continuously updated in the industrial data stream. (5) Uploading the soft measurement results of the model and the incremental learning process to the DCS database for storage; the incremental learning process includes the size and composition of the process knowledge base, the time when the incremental learning of the soft measurement model occurs, the length of time consumed by the incremental learning process, etc.

2. The method according to claim 1, characterized in that The temporal feature extraction self-encoder network of the end-to-end neural network soft measurement model described in step (1) is a hierarchical temporal feature extraction self-encoder network consisting of an encoder network and a decoder network; the encoder network of the hierarchical temporal feature extraction self-encoder network is composed of a self-attention mechanism layer, a long short-term memory (Long Short Term Memory, LSTM) network layer, a self-attention mechanism layer and a temporal convolution network (Temporal Convolution Network, TCN) encoder layer connected in sequence; the decoder network of the hierarchical temporal feature extraction self-encoder network is composed of a TCN decoder layer, a point-by-point convolution layer and an LSTM layer connected in sequence; The encoder network of the hierarchical time series feature extraction self-encoder network first uses a layer of self-attention mechanism to weight the important variables and time steps in the process variable time series X to obtain the weighted process variable time series X att ; for X att Take the average value along the time dimension to get the global feature Z g On this basis, the LSTM network layer and a layer of attention mechanism are used to analyze X att Processing is performed to obtain the dynamic information sequence H; taking the result of the last time step of H, the dynamic feature Z is obtained v ; Further, the TCN autoencoder is used to process H, and its feature vector of the last time step is retained to obtain the local feature Z l ; In the extracted global feature Z g , dynamic feature Z v and local feature Z l A residual connection is established between them, and the final output is Z g , Z v The feature set Z concatenated with Z1; The hierarchical temporal features are extracted from the decoder network of the encoder network. The decoder network first passes through the TCN decoder layer, and reconstructs the two-dimensional reconstructed dynamic sequence using the one-dimensional feature set Z output by the encoder network. Furthermore, the point-by-point convolution layer is introduced The variable dimension is adjusted; finally, the process variable time series is reconstructed through the LSTM layer 3. The method according to claim 1, characterized in that The pre-training parameters in step (2) include the number of training rounds, the training batch size, the learning rate, and the drop rate; the loss function is a semi-supervised loss function, expressed as: in, and are the supervised loss function term and the unsupervised loss function term respectively; λ1 and λ2 are the regularization coefficients of the loss function; The expression is: Among them, B is the number of training batches, Y b is the true value of the performance indicator label in the bth batch, and is the prediction result of the model for the performance index in the bth batch, The expression is: Among them, M is the number of processes, L m is the length of the process variable time series of the mth process, N m is the variable dimension of the process variable time series of the mth process, is the value of the input process variable time series at the position (b, l, n), and is the value of the reconstructed process variable time series at the position (b, l, n).

4. The method according to claim 1, characterized in that: The online platform in step (4) refers to a system platform for deploying and running soft measurement models and their incremental learning programs, including cloud computing platforms, industrial Internet of Things platforms, and digital twin platforms; the incremental learning program refers to a corresponding program and algorithm for updating the process knowledge base in real time based on the constructed process knowledge base, using the time series features extracted from the real-time industrial data stream received by the online platform by the deployed soft measurement model, and instantly training and iterating the parameters of the regressor network part of the soft measurement model to ensure that the soft measurement model maintains soft measurement performance in the real-time industrial data stream; the industrial data stream refers to the process data stream read in real time by the online platform from the process industry data source including various sensors, equipment, and control systems.

5. The method according to claim 1 or 4, characterized in that: The incremental learning procedure described in step (4) includes the following steps: (4.1) Through the time series feature extraction of the neural network soft measurement model, the new process variable time series X is input into the soft measurement model from the current moment of the industrial data stream. in In the above example, the corresponding new time series feature Z is extracted. in ; (4.2) is the temporal feature Z in the constructed process knowledge base K. i , calculate Z in and Z i The similarity index Sim i (Z i ,Z in ), the similarity index Sim i (Z i ,Z in ) is calculated as: Among them, ||Z i || and ||Z in || represents Z i and Z in The norm of , ∑ represents the covariance matrix corresponding to the two; (4.3) Determine each similarity index Sim calculated i (Z i ,Z in ) are all less than a set threshold s min If yes, add Z to the current knowledge base K in ; If not, terminate this round of incremental learning process; (4.4) Assume that the result of removing the oldest set of time series features from K is K′, calculate the information loss InfoLoss(K, K′) caused by removing this set of time series features to the knowledge base; when the value of InfoLoss(K, K′) is less than a set threshold l min When , the oldest set of time series features is removed from K; the calculation expression of the information loss InfoLoss(K,K′) is: InfoLoss(K,K′)=H(K)-H(K′) Among them, P(X i , j ) represents the probability that the elements of matrix X appear in the i-th box and the j-th feature position after the value range of the j-th feature is binned; (4.5) Determine whether the length of K after the update is divisible by a set integer bs. If so, use the current K as the training data set and use the supervised loss function to fine-tune the regressor network part to update the parameters of the regressor network part of the soft sensor model; if not, directly terminate this round of incremental learning process; the expression of the supervised loss function is:

Images