A soft sensing method for process industry based on feature extraction and incremental learning

Abstract

The application provides a process industry soft measurement method based on feature extraction and incremental learning. The method introduces an end-to-end neural network soft measurement model composed of multiple parallel time series feature extraction self-encoder networks and a regressor network connected in front and back, extracts time series features from multi-process variable time series data, and uses the time series features for performance index soft measurement modeling. In addition, based on the time series features extracted by the model, the method proposes an incremental learning method based on time series features. The soft measurement method provided by the application can effectively and accurately perform soft measurement on some performance indexes that are difficult to directly measure or have high measurement cost in the process industry, and through the incremental learning method, the soft measurement model can long-term maintain good soft measurement performance in the industrial data stream with strong real-time performance, thereby providing reference and guidance for monitoring, measurement, optimization and control of various parameters in the process industry production process.

Description

Technical Field

[0001] This invention belongs to the fields of computer technology and artificial intelligence, and more particularly relates to a soft-sensing method for process industries based on feature extraction and incremental learning. This method is primarily used to establish soft-sensing models for a series of important performance indicators in process industry production, and to perform real-time soft-sensing using industrial data streams. Background Art

[0002] Some noun terms involved in the present invention are introduced below.

[0003] (1) Process Industry: Process industry refers to the industrial process of converting raw materials into high value-added products through physical, chemical and other processes. This type of industry usually involves continuous material processing and complex physical changes, chemical reactions, etc. Chemical industry, petroleum, cement, metallurgy, etc. all fall within the scope of process industry. The biggest feature of process industry is that its production process is usually divided into multiple different steps, and the process in each step is usually relatively complex.

[0004] (2) Process variables: Process variables are key parameters used to describe and control the process in the process industry. These variables typically include various physical and chemical properties, such as temperature, pressure, flow rate, liquid level, concentration, etc. The real-time data of these process variables can reflect the real-time status of the process industry production process.

[0005] (3) Performance indicators: Performance indicators refer to key parameters used in process industries to evaluate and measure product quality, production efficiency, and process stability. These indicators typically include the chemical and physical properties of the product, as well as some summary parameters of the production process, such as energy consumption and product conversion rate. By monitoring and analyzing these indicators, the efficiency of the process industry production process and the quality of the output products can be ensured.

[0006] (4) Soft Sensing: Soft sensing refers to a technical method in process industries that uses mathematical models and algorithms to indirectly estimate performance indicators that are difficult to measure directly or frequently using process variables that are easy to measure or measure frequently. The mathematical model used for soft sensing modeling can be a mechanistic model based on the mechanism of the process industry or a data-driven model based on data statistics and machine learning methods. Soft sensing integrates data from multiple process variables and establishes a prediction model through regression algorithms to achieve real-time fitting and estimation of key performance indicators.

[0007] (5) Feature extraction: Feature extraction refers to the process of analyzing and processing raw data to extract representative and significant features (i.e., data attributes or variables) from it in data processing. These features are used to describe and represent the key information of the data to facilitate subsequent modeling, analysis, and decision-making.

[0008] (6) Incremental learning: Incremental learning refers to a method of continuously improving model performance by updating and learning new data step by step in the process of machine learning. Incremental learning can adapt to changing data environments, maintain the latest state and effectiveness of the model.

[0009] The background of the present application is introduced as follows.

[0010] In the field of process industry, due to the mutual coupling of physical and chemical reactions in the production process and the complexity of thermal state, there is often a complex nonlinear relationship between process variables and the performance indicators of the required soft measurement. Such a relationship is difficult to describe through an accurate expression or mechanism model, so it is often necessary to establish a data-driven soft measurement model to obtain real-time estimation results of the performance indicators of the required soft measurement.

[0011] Early soft measurement modeling methods mainly rely on classical statistical learning methods, such as using partial least squares (PLS) regression and support vector regression (SVR) to fit the relationship between process variables and soft measurement targets. These methods have fast modeling speed and strong interpretability, but they do not adequately consider the nonlinear relationship between performance indicators and process variables in the process industry, so there is an obvious upper limit to their modeling ability. With the development of artificial neural networks, related technical personnel have developed a series of modeling methods based on neural networks, such as back propagation neural network (BPNN), random vector functional link network (RVFL), convolutional neural network (CNN), and recurrent neural network (RNN), etc. These networks have certain processing capabilities for the above-mentioned nonlinear relationship. Based on these networks, some more complex deep network-based soft measurement modeling frameworks have been proposed in recent years, such as autoencoders and twin neural networks, etc. However, the current method of constructing deep network-based soft measurement models usually relies on using a single network structure to extract a set of features, and hopes that these features can fully represent the knowledge in the original data. This method has a very obvious limitation for process industries with many processes and complex reactions. How to combine the characteristics of process industry processes to construct specific network structures and feature extraction methods is the key to further improving the performance of deep network-based soft measurement models.

[0012] Another challenge facing soft sensor modeling in process industries is that once a soft sensor model trained on offline data is deployed on a production line, its performance may degrade over time. This is due to the phenomenon of "concept drift" in industrial data streams. Concept drift can be caused by numerous factors, including changes in process conditions, raw materials, and anomalies. To achieve continuous updating of soft sensor models in industrial data streams, researchers have introduced a range of continuous updating strategies, such as recursive modeling, sliding window methods, just-in-time learning, and incremental learning. Recursive modeling is typically limited to combining with statistical learning methods and is not suitable for complex neural networks. Sliding window methods can be combined with any data-driven approach, but each model update can result in the loss of significant historical knowledge. Just-in-time learning methods retrain the model as new data streams emerge, but they ignore the persistence and accumulation of knowledge required for soft sensor models. Incremental learning is a continuous and adaptive approach that leverages the concept of a "knowledge base" to continuously update the knowledge base and use it to adjust model parameters in real time, thereby maintaining a continuous learning state to adapt to the drift of industrial data streams. However, most neural network-based incremental learning methods typically retrain the entire model when updating, which can lead to catastrophic forgetting of important previously acquired knowledge and unnecessary waste of computing resources. Developing a stable and efficient incremental learning method is currently a key technical focus in the development of incremental learning algorithms. Summary of the Invention

[0013] In view of the above technical background, the present invention proposes a process industry soft sensing method based on feature extraction and incremental learning.

[0014] The technical solutions of the present invention are as follows:

[0015] A soft sensing method for process industry based on feature extraction and incremental learning includes the following steps:

[0016] (1) Read data from the DCS database, classify the data variables according to the process, and construct a multi-process variable time series dataset for pre-training the end-to-end neural network soft sensing model;

[0017] The end-to-end neural network soft measurement model is composed of multiple parallel time series feature extraction autoencoder networks and a regressor network connected in front and back. The time series feature extraction autoencoder network is used to extract a set of one-dimensional time series features from the two-dimensional process variable time series of the corresponding process; the regressor network is used to integrate the high-dimensional time series features extracted by the multiple time series feature extraction autoencoder networks and fit the soft measurement results of the performance indicators; the multi-process process variable time series dataset is composed of multiple groups of process variable time series data and performance indicator soft measurement label data, and its construction method is: first, the process variables available in the DCS (distributed control system) database are divided into multiple groups, and appropriate time series lengths are set for each of them. Then, starting from the timestamp of the soft measurement performance indicator label value collection in the DCS database, the process variable time series of corresponding length is intercepted forward according to the group of the process variable, so that each performance indicator label value within the dataset can correspond to the corresponding multiple process variable time series;

[0018] (2) Setting pre-training parameters, using the constructed process variable time series dataset and the designed loss function to pre-train the end-to-end neural network soft sensing model;

[0019] (3) saving the pre-trained end-to-end neural network soft measurement model, and extracting corresponding features from the process variable time series data set through the soft measurement model to build a process knowledge base, wherein the process knowledge base is a collection of time series features extracted from the historical data process variable time series;

[0020] (4) The pre-trained soft measurement model, process knowledge base, and incremental learning algorithm are deployed on an online platform. While the model is performing the soft measurement task, the corresponding incremental learning program is run to enable the parameters of the regression network part of the model to be continuously updated in the industrial data stream.

[0021] (5) Uploading the soft measurement results of the model and the incremental learning process to the DCS database for storage; the incremental learning process includes the size and composition of the process knowledge base, the time when the incremental learning of the soft measurement model occurs, the length of time consumed by the incremental learning process, etc.

[0022] The temporal feature extraction self-encoder network of the end-to-end neural network soft measurement model described in step (1) is a hierarchical temporal feature extraction self-encoder network consisting of an encoder network and a decoder network; the encoder network of the hierarchical temporal feature extraction self-encoder network is composed of a self-attention mechanism layer, a long short-term memory (Long Short Term Memory, LSTM) network layer, a self-attention mechanism layer and a temporal convolution network (Temporal Convolution Network, TCN) encoder layer connected in sequence; the decoder network of the hierarchical temporal feature extraction self-encoder network is composed of a TCN decoder layer, a point-by-point convolution layer and an LSTM layer connected in sequence;

[0023] The hierarchical time series feature extraction self-encoder network first uses a layer of self-attention mechanism to weight the important variables and time steps in the process variable time series X, and obtains the weighted process variable time series X. att ; for X att Take the average value along the time dimension to obtain the global feature Z g On this basis, the LSTM network layer and the attention mechanism are used to analyze X att Processing is performed to obtain the dynamic information sequence H; taking the result of the last time step of H, the dynamic feature Z is obtained v ; Further, the TCN autoencoder is used to process H and retain its feature vector of the last time step to obtain the local feature Z l ; In the extracted global feature Z g , dynamic feature Z v and local features Z l Establish a residual connection between them and finally output Z g , Z v and Z l The concatenated feature set Z;

[0024] The hierarchical temporal features are extracted from the decoder network of the encoder network. The decoder network first passes through the TCN decoder layer and reconstructs the two-dimensional reconstructed dynamic sequence using the one-dimensional feature set Z output by the encoder network. Furthermore, the point-by-point convolution layer is introduced Finally, the process variable time series is reconstructed through the LSTM layer.

[0025] The pre-training parameters in step (2) include the number of training rounds, training batch size, learning rate, and dropout rate; the loss function is a semi-supervised loss function, expressed as:

[0026]

[0027] in, are the supervised loss function term and the unsupervised loss function term respectively; λ1 and λ2 are the regularization coefficients of the loss function; The expression is:

[0028]

[0029] Among them, B is the number of training batches, Y b is the true value of the performance indicator label in the bth batch, and is the prediction result of the model for the performance index in the bth batch, The expression is:

[0030]

[0031] Among them, M is the number of processes, L m is the length of the process variable time series of the mth step, N m is the variable dimension of the process variable time series of the mth step, is the value of the input process variable time series at the position (b, l, n), and is the value of the reconstructed process variable time series at the position (b, l, n).

[0032] The online platform in step (4) refers to a system platform for deploying and running soft measurement models and their incremental learning programs, including cloud computing platforms, industrial Internet of Things platforms, and digital twin platforms; the incremental learning program refers to a corresponding program and algorithm that updates the process knowledge base in real time based on the constructed process knowledge base, using the time series features extracted from the real-time industrial data stream received by the online platform by the deployed soft measurement model, and instantly trains and iterates the parameters of the regressor network part of the soft measurement model to ensure that the soft measurement model maintains soft measurement performance in the real-time industrial data stream; the industrial data stream refers to the process data stream read in real time by the online platform from process industry data sources including various sensors, equipment, and control systems.

[0033] The incremental learning procedure described in step (4) includes the following steps:

[0034] (4.1) Through the time series feature extraction of the neural network soft measurement model, the new process variable time series X is input into the soft measurement model from the current moment of the industrial data stream. in In the above example, the corresponding new time series feature Z is extracted. in ;

[0035] (4.2) is the temporal feature Z of each process knowledge base K constructed i , calculate Z in and Z i The similarity index Sim between i (Z i ,Z in ), the similarity index Sim i (Z i ,Z in ) is calculated as:

[0036]

[0037] Among them, ||Z i || and ||Z in || represents Z i and Z in The norm of , ∑ represents the covariance matrix corresponding to the two;

[0038] (4.3) Determine each similarity index Sim calculated i (Z i ,Z in ) are all less than a set threshold s min If yes, add Z to the current knowledge base K in ; If not, terminate this round of incremental learning process;

[0039] (4.4) Assume that the result of removing the oldest set of time series features from K is K′, calculate the information loss InfoLoss(K,K) caused by removing this set of time series features. ′ ); When InfoLoss(K,K ′ ) is less than a set threshold l min When , the oldest set of time series features is removed from K; the information loss InfoLoss(K,K ′ ) is calculated as:

[0040]

[0041] InfoLoss(K,K ′ )=H(K)-H(K ′ )

[0042] Among them, P(X i,j ) represents the probability that the elements of matrix X appear in the i-th bin and the j-th feature position after the value range of the j-th feature is binned;

[0043] (4.5) Determine whether the updated length of K is divisible by a set integer bs. If so, use the current K as the training data set and use the supervised loss function to fine-tune the regressor network part to update the parameters of the regressor network part of the soft sensor model. If not, terminate this round of incremental learning process directly. The expression of the supervised loss function is:

[0044]

[0045] The beneficial effects of the present invention are:

[0046] This incremental learning method based on time series features can directly use the current set of time series features to update the regressor network part of the proposed end-to-end neural network soft measurement model, thereby avoiding the repeated training of the encoder network part for extracting time series features with strong versatility. While significantly saving computing resources, it also ensures the soft measurement performance of the soft measurement model in real-time industrial data streams, thereby improving the soft measurement performance and service life of the proposed process industry performance indicator soft measurement model. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of a process industry soft sensing method based on feature extraction and incremental learning proposed by the present invention.

[0048] Figure 2 This is a schematic diagram of the workflow of a hierarchical temporal feature extraction autoencoder network proposed in the present invention.

[0049] Figure 3 It is a flowchart of an incremental learning method based on time series features proposed by the present invention. DETAILED DESCRIPTION

[0050] The specific embodiments of the present invention are further described below with reference to the accompanying drawings. It should be noted that the symbols in the expressions involved in the present invention are only for simplification and clarity of expression, and different expressions can be used in different examples. Therefore, the key to the present invention lies in the method for extracting process variable data features in the process industry and the incremental learning ideas based on the extracted features; in addition, in the subsequent examples of the specific embodiments of the present invention, the numbers "A", "B", "C" and so on are only used to distinguish the contents of different objects and have no other special meanings.

[0051] The purpose of the present invention is to provide a simple, efficient and versatile process industry soft sensing method based on feature extraction and incremental learning. For example, for a soft sensing task of a process industry process divided into M steps, the implementation method of the present invention is as follows Figure 1As shown, it specifically includes the following 5 steps:

[0052] Step 1.1: Read data from the DCS database, classify the data variables according to the process, and construct a multi-process variable time series dataset for pre-training the end-to-end neural network soft sensing model.

[0053] The first step of the present invention is to prepare a data set for pre-training a soft measurement model. In the present invention, the input of the soft measurement model is a time series with a fixed length composed of process variable data arranged in chronological order, rather than a single process variable data point. The expansion of the input dimension of the soft measurement model is conducive to the model extracting more comprehensive and extensive time series features from the process variable data. The method for constructing a multi-process process variable time series data set is: first, the process variables available in the DCS (Distributed Control System) database are divided into M groups (A, B, C, ... M), and appropriate time series lengths (L) are set for each of them. A , L B , L C 、……L M ), then take the timestamp of the soft measurement performance index tag value collected in the DCS database as the starting point, and intercept the data with a length of (L A , L B , L C 、……L M ) process variable time series, so that each performance indicator label value within the data set can be matched with the corresponding M process variable time series.

[0054] The soft measurement model used in this invention is an end-to-end neural network soft measurement model. This network model mainly consists of M parallel time series feature extraction autoencoder networks connected in a front-to-back manner with a regressor network. The time series feature extraction autoencoder network extracts a set of one-dimensional time series features from the two-dimensional process variable time series of the corresponding process, while the regressor network integrates the high-dimensional time series features extracted by the M time series feature extraction autoencoder networks to fit the soft measurement results of the performance indicators.

[0055] Step 1.2: Set the pre-training parameters and use the constructed process variable time series dataset and the designed loss function to pre-train the end-to-end neural network soft sensing model.

[0056] In the present invention, it is necessary to use the constructed process variable time series data set to pre-train the adopted end-to-end neural network soft measurement model, so that the soft measurement model obtains a set of preliminary parameters. For the pre-training process, it is first necessary to set a series of pre-training parameters, such as the number of training rounds, training batch size, learning rate, drop rate, etc. On this basis, it is necessary to construct a loss function to guide the training of the model. In the present invention, a semi-supervised loss function composed of a supervised loss function of prediction accuracy and an unsupervised loss function of time series reconstruction accuracy is used to guide the training of the model. Among them, the prediction accuracy loss, that is, the expression of the supervised loss function term is:

[0057]

[0058] Among them, B is the number of training batches, Y b is the true value of the performance indicator label in the bth batch, and is the prediction result of the model for the performance index in the bth batch. In addition, the time series reconstruction accuracy loss, that is, the expression of the unsupervised loss function term is:

[0059]

[0060] Among them, M is the number of processes, L m is the length of the process variable time series of the mth step, N m is the variable dimension of the process variable time series of the mth step, is the value of the input process variable time series at the position (b, l, n), and is the value of the reconstructed process variable time series at the position (b, l, n). Finally, the overall semi-supervised learning loss function is expressed as:

[0061]

[0062] Where λ1 and λ2 are regularization coefficients of the loss function, which are used to adjust the influence of each part of the loss function. Using the above loss function, the soft sensor model is pre-trained, so that the soft sensor model can further improve the prediction accuracy of the model by using unlabeled data during the parameter update process.

[0063] Step 1.3: Save the pre-trained end-to-end neural network soft measurement model and use the soft measurement model to extract corresponding features from the process variable time series dataset to build a process knowledge base.

[0064] In order to carry out incremental learning of the subsequent soft-sensing model, a process knowledge base is constructed to store the knowledge learned by the soft-sensing model from historical data. The process knowledge base is a collection of time series features extracted from the historical process variable time series data, containing a relatively rich set of process knowledge that can be utilized by the soft-sensing model. By inputting each time series in the process variable time series dataset into the pre-trained process variable soft-sensing model, the corresponding time series features are obtained. By summarizing all the time series features, the corresponding process knowledge base is obtained.

[0065] Step 1.4: Deploy the pre-trained soft measurement model, process knowledge base, and incremental learning algorithm on the online platform. While the model is performing the soft measurement task, run the corresponding incremental learning program to enable the parameters of the regression network part of the model to be continuously updated in the industrial data stream.

[0066] In this paper, the pre-trained soft-sensing model needs to be run in an online environment to achieve real-time soft-sensing. This process requires the use of an online platform. An online platform refers to a system platform used to deploy and run the soft-sensing model and its incremental learning program, such as a cloud computing platform, an industrial Internet of Things platform, a digital twin platform, and so on.

[0067] In the present invention, a pre-trained soft measurement model is deployed on an online platform to perform soft measurement tasks. A constructed process knowledge base and a supporting incremental learning algorithm for updating the soft measurement model parameters in real time are also deployed. Based on the constructed process knowledge base, this incremental learning algorithm uses the time series features extracted by the deployed soft measurement model from the real-time industrial data stream received by the online platform to update the process knowledge base in real time. The parameters of the regressor network of the soft measurement model are trained and iterated immediately to ensure that the soft measurement model maintains good soft measurement performance in the real-time industrial data stream over the long term. In the present invention, the industrial data stream refers to the process data stream read by the online platform from various sensors, equipment, control systems, and other data sources in the process industry, and has high real-time performance.

[0068] Step 1.5: Upload the model's soft measurement results and incremental learning process to the DCS database for storage.

[0069] In the present application, it is necessary to upload the soft measurement results of the deployed soft measurement model to the DCS database to realize long-term storage of the soft measurement results, thereby providing instant reference for downstream tasks such as optimization and control. In addition, the process of incremental learning of the deployed soft measurement model in the industrial data stream is also uploaded and stored in the DCS database. In the present application, the process of incremental learning includes the size and composition of the process knowledge base, the time of occurrence of the incremental learning of the soft measurement model, the length of time consumed by the incremental learning process, and the like. These information has high reference value for the optimization of the incremental learning algorithm, and for the monitoring and inference of the process condition changes of the process industry.

[0070] The process industry soft measurement method based on feature extraction and incremental learning provided by the present application can effectively and accurately perform soft measurement on some performance indicators in the process industry that are difficult to directly measure or have high measurement cost, and through the method of incremental learning, the soft measurement model can long-term maintain good soft measurement performance in the industrial data stream with strong real-time performance, thereby providing reference and guidance for monitoring, measurement, optimization and control of various parameters in the process industry production process.

[0071] The above is an introduction to a process industry soft measurement method based on feature extraction and incremental learning. Among them, the structure design of the time sequence feature extraction autoencoder network not only determines the performance upper limit of the soft measurement model, but also determines the composition of the process knowledge base extracted from the historical data and the effect of the incremental learning based on the process knowledge base. Therefore, designing a reliable and efficient time sequence feature extraction autoencoder network plays a key role in the practical application effect of the present application. A hierarchical time sequence feature extraction autoencoder network proposed by the present application is introduced below.

[0072] Figure 2 is a working process schematic diagram of a hierarchical time sequence feature extraction autoencoder network proposed by the present application. Referring to Figure 2 , the designed hierarchical time sequence feature extraction autoencoder network is composed of an encoder network and a decoder network.

[0073] The encoder network of the designed autoencoder network is connected in order by a self-attention mechanism layer, an LSTM network layer, a self-attention mechanism layer, and a TCN encoder layer. In the designed encoder network, after the process variable time sequence X input into the autoencoder part of the network, it will first be processed by a layer of self-attention mechanism, so as to weight the important variables and time steps in the input time sequence. After weighting, the process variable time sequence X att The average value along the time dimension is obtained, that is, a group of one-dimensional feature vectors Z g, which represents the global average level of the input process variable time series, and is named as global feature. On this basis, a set of LSTM networks are used to process X att , and a set of dynamic information sequences H representing the dynamic changes of the input process variable time series are obtained by further weighting the processing results of the LSTM networks through an attention mechanism. According to the principle of the LSTM network, the last time step of H is taken, and another one-dimensional feature vector Z v is obtained, which represents the dynamic level of the input process variable time series, and is named as dynamic feature. Finally, a TCN autoencoder is introduced to process H extracted by the convolutional network:

[0074]

[0075] In the above expression, k and d represent the size and dilation rate of the TCN convolution kernel respectively, and w represents the weight of the convolution kernel. Due to the causal convolution characteristics of TCN, the output of TCN at each time step is only affected by the previous information. Finally, combined with the above characteristics of TCN, the feature vector Z l of the last time step of the TCN output is saved, which contains the fine-grained spatio-temporal change information of the input process variable time series, and is named as local feature. The designed encoder network establishes a residual connection between the extracted global feature Z g , dynamic feature Z v and local feature Z l , and finally outputs the feature set Z g , Z v and Z l after splicing.

[0076] The decoder network of the designed autoencoder network is composed of a TCN decoder layer, a pointwise convolution layer and an LSTM layer connected in order. In the designed decoder network, a TCN decoder is first introduced to reconstruct a two-dimensional reconstructed dynamic sequence from the one-dimensional feature set Z output by the encoder network. A pointwise convolution layer, that is, a convolution layer with each dimension size of the convolution kernel being 1, is introduced to adjust the variable dimension, and an LSTM network is further introduced to finally reconstruct the process variable time series

[0077] This paper proposes a hierarchical temporal feature extraction autoencoder network that can reliably and efficiently extract a wide range of temporal features from the input process variable time series. This autoencoder network architecture not only improves the overall performance of the soft sensing model but also enhances the diversity and information richness of the extracted temporal features, thereby supporting a range of downstream tasks based on temporal features.

[0078] The above is an introduction to the hierarchical temporal feature extraction autoencoder network proposed in this invention. This autoencoder network constitutes the temporal feature extraction autoencoder network portion of the proposed end-to-end neural network soft-sensing model. Furthermore, this invention does not restrict the form or structure of the regressor network portion of the proposed neural network soft-sensing model. Any network capable of deriving performance indicator prediction results by fitting high-dimensional temporal features can be used as the regressor network portion of the proposed neural network soft-sensing model.

[0079] Based on the overall technical solution proposed in this invention and the hierarchical temporal feature extraction autoencoder network proposed in this invention, an efficient and rapid incremental learning method based on the temporal features extracted by an end-to-end neural network soft sensing model can be constructed, thereby guiding the soft sensing model to maintain good soft sensing performance in real-time industrial data streams over the long term. The following introduces the incremental learning method based on temporal features proposed in this invention.

[0080] Figure 3 This is a flow chart of an incremental learning method based on time series features proposed in this invention. It should be noted that the proposed incremental learning method is carried out in parallel with the soft measurement process of the model. Figure 3 ,The proposed incremental learning method specifically includes the following five steps:

[0081] Step 2.1: Extract the time series features of the neural network soft measurement model from the encoder network and input the new process variable time series X of the soft measurement model from the current moment of the industrial data stream. in In the above example, the corresponding new time series feature Z is extracted. in .

[0082] Step 2.2: For each temporal feature Z in the constructed process knowledge base K i , calculate Z in and Z i The similarity index Sim between i (Z i ,Z in ). Sim i (Z i ,Z in ) is calculated as:

[0083]

[0084] Among them, ||Z i || and ||Z in || represents Z i and Z in The norm of , ∑ represents the covariance matrix corresponding to the two. For each Z in K i , all calculated to get a Sim i (Z i ,Z in ), thus achieving Z in and Z of each of K i Comprehensive comparison of knowledge similarity.

[0085] Step 2.3: Determine each similarity index Sim calculated i (Z i ,Z in ) are all less than a set threshold s min If yes, add Z to the current knowledge base K in If not, terminate this round of incremental learning process.

[0086] Specifically, if each similarity index Sim i (Z i ,Z in ) are all less than s min , then it means that the X input to the soft sensor model at the current moment in The Z extracted from in The similarity between Z and each time series feature vector in K is not high. in is a new knowledge independent of K, so it should be added to K; on the contrary, when part Z i and Z in Sim i (Z i ,Z in ) is greater than s min When Z in And this part Z in K i The similarity is large, then it is considered that Z in It is not independent of K, so there is no need to perform the subsequent incremental learning process.

[0087] Step 2.4: Assume that the result of removing the oldest set of time series features from K is K′, and calculate the information loss InfoLoss(K,K) caused by removing this set of time series features. ′ ). When InfoLoss(K,K ′ ) is less than a set threshold lmin When , the oldest set of time series features is removed from K.

[0088] In the process of executing most incremental learning algorithms, as industrial data streams are continuously input, the size of the knowledge base will continue to expand, which may lead to two problems: first, the training time and computing resources required for subsequent regression network updates will continue to increase; second, the old knowledge in the knowledge base may not provide effective information for soft measurement modeling under the latest working conditions, and may even have a negative impact. Therefore, each time a new time series feature Z is incorporated into K, in Later, a logic will be executed to determine whether the oldest set of time series features in K needs to be removed, and based on the result of the logic judgment, it will be decided whether the oldest set of time series features needs to be removed from K. ′ ) is calculated as:

[0089]

[0090] InfoLoss(K,K ′ )=H(k)-H(k ′ )

[0091] Among them, P(X i,j ) represents the probability that an element of matrix X appears in the i-th bin and the j-th feature position after the value range of the j-th feature is binned. Through the above knowledge base discrimination and operation mechanism, the scale of k can be controlled while ensuring the richness of its overall information.

[0092] Step 2.5: Determine whether the length of the updated k is divisible by a set integer bs. If so, use the current K as the training data set and use the supervised loss function to fine-tune the regressor network part to update the parameters of the regressor network part of the soft measurement model; if not, terminate this round of incremental learning process directly.

[0093] In the designed incremental learning algorithm, if fine-tuning training of the regressor network is performed after each update of K, it may consume a lot of training time and computing resources, and have a negative impact on the real-time performance of the soft measurement model. Therefore, a more feasible solution is to wait until K accumulates to a certain size before performing fine-tuning training of the regressor network, so as to achieve a balance between the real-time performance and training resources of the soft measurement model. The integer bs set should be consistent with the training batch size used in the fine-tuning training process. During the fine-tuning training of the regressor network, the calculation expression of the supervised loss function used is:

[0094]

[0095] The present invention proposes an incremental learning method based on time series features, which can directly use the set K of currently available time series features to update the regressor network part of the proposed end-to-end neural network soft measurement model, thereby avoiding the repeated training of the time series feature extraction autoencoder network part with strong versatility. While significantly saving computing resources, it also ensures the soft measurement performance of the soft measurement model in real-time industrial data streams, thereby improving the soft measurement performance and service life of the proposed process industry performance indicator soft measurement model.

[0096] It should also be noted that the terms "such as," "including," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, elements defined by the phrases "such as," "including," or "including" do not exclude the presence of other identical elements in the process, method, or apparatus comprising the elements.

[0097] It will be apparent to those skilled in the art that various modifications and variations may be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A soft sensing method for process industry based on feature extraction and incremental learning, characterized in that: The steps include: (1) Read data from the DCS database, classify the data variables according to the process, and construct a multi-process variable time series dataset for pre-training the end-to-end neural network soft sensing model; The end-to-end neural network soft measurement model consists of multiple parallel time series feature extraction autoencoder networks connected in front and back to a regressor network. The time series feature extraction autoencoder network is used to extract a set of one-dimensional time series features from the two-dimensional process variable time series of the corresponding process. The regressor network is used to integrate multiple time series features to extract high-dimensional time series features extracted by the encoder network, and fit the soft measurement results of the performance indicators; The multi-process variable time series dataset consists of multiple groups of process variable time series data and performance indicator soft measurement label data. The construction method is as follows: first, the process variables obtained in the DCS database are divided into multiple groups, and appropriate time series lengths are set for each group. Then, starting from the timestamp of the acquisition of the soft measurement performance indicator label value in the DCS database, the process variable time series of corresponding lengths are intercepted forward according to the process variable group, so that each performance indicator label value in the dataset can be matched with the corresponding multiple process variable time series; (2) Setting pre-training parameters, using the constructed process variable time series dataset and the designed loss function to pre-train the end-to-end neural network soft sensing model; (3) saving the pre-trained end-to-end neural network soft measurement model, and extracting corresponding features from the process variable time series data set through the soft measurement model to build a process knowledge base, wherein the process knowledge base is a collection of time series features extracted from the historical data process variable time series; (4) The pre-trained soft measurement model, process knowledge base, and incremental learning algorithm are deployed on an online platform. While the model is performing the soft measurement task, the corresponding incremental learning program is run to enable the parameters of the regression network part of the model to be continuously updated in the industrial data stream. (5) Uploading the soft measurement results of the model and the incremental learning process to the DCS database for storage; the incremental learning process includes the size and composition of the process knowledge base, the time when the incremental learning of the soft measurement model occurs, and the length of time consumed by the incremental learning process.

2. The method according to claim 1, characterized in that The temporal feature extraction self-encoder network of the end-to-end neural network soft measurement model described in step (1) is a hierarchical temporal feature extraction self-encoder network consisting of an encoder network and a decoder network; the encoder network of the hierarchical temporal feature extraction self-encoder network is composed of a self-attention mechanism layer, a long short-term memory, an LSTM network layer, a self-attention mechanism layer and a temporal convolution network, and a TCN encoder layer connected in sequence; the decoder network of the hierarchical temporal feature extraction self-encoder network is composed of a TCN decoder layer, a point-by-point convolution layer and an LSTM layer connected in sequence; The hierarchical time series feature extraction self-encoder network first uses a layer of self-attention mechanism to weight the important variables and time steps in the process variable time series X, and obtains the weighted process variable time series X. att ; for X att Take the average value along the time dimension to obtain the global feature Z g On this basis, the LSTM network layer and the attention mechanism are used to analyze X att Processing is performed to obtain the dynamic information sequence H; taking the result of the last time step of H, the dynamic feature Z is obtained v ; Further, the TCN autoencoder is used to process H and retain its feature vector of the last time step to obtain the local feature Z l ; In the extracted global feature Z g , dynamic feature Z v and local features Z l Establish a residual connection between them and finally output Z g 、Z v and Z l The concatenated feature set Z; The hierarchical temporal features are extracted from the decoder network of the encoder network. The decoder network first passes through the TCN decoder layer and reconstructs the two-dimensional reconstructed dynamic sequence using the one-dimensional feature set Z output by the encoder network. Furthermore, the point-by-point convolution layer is introduced Finally, the process variable time series is reconstructed through the LSTM layer.

3. The method according to claim 1, characterized in that The pre-training parameters in step (2) include the number of training rounds, training batch size, learning rate, and dropout rate; the loss function is a semi-supervised loss function, expressed as: in, and are the supervised loss function term and the unsupervised loss function term respectively; λ1 and λ2 are the regularization coefficients of the loss function; The expression is: Among them, B is the number of training batches, Y b is the true value of the performance indicator label in the bth batch, and is the prediction result of the model for the performance index in the bth batch, The expression is: Among them, M is the number of processes, L m is the length of the process variable time series of the mth step, N m is the variable dimension of the process variable time series of the mth step, is the value of the input process variable time series at the position (b, l, n), and is the value of the reconstructed process variable time series at the position (b, l, n).

4. The method according to claim 1, wherein The online platform in step (4) refers to a system platform for deploying and running soft measurement models and their incremental learning programs, including cloud computing platforms, industrial Internet of Things platforms, and digital twin platforms; the incremental learning program refers to a corresponding program and algorithm that updates the process knowledge base in real time based on the constructed process knowledge base, using the time series features extracted from the real-time industrial data stream received by the online platform by the deployed soft measurement model, and instantly trains and iterates the parameters of the regressor network part of the soft measurement model to ensure that the soft measurement model maintains soft measurement performance in the real-time industrial data stream; the industrial data stream refers to the process data stream read in real time by the online platform from process industry data sources including various sensors, equipment, and control systems.

5. The method according to claim 1 or 4, characterized in that: The incremental learning procedure described in step (4) includes the following steps: (4.1) Through the time series feature extraction of the neural network soft measurement model, the new process variable time series X is input into the soft measurement model from the current moment of the industrial data stream. in In the above example, the corresponding new time series feature Z is extracted. in ; (4.2) is the temporal feature Z of each process knowledge base K constructed i , calculate Z in and Z i The similarity index Sim between i (Z i ,Z in ), the similarity index Sim i (Z i ,Z in ) is calculated as: Among them, ||Z i || and ||Z in || represents Z i and Z in The norm of , ∑ represents the covariance matrix corresponding to the two; (4.3) Determine each similarity index Sim calculated i (Z i ,Z in ) are all less than a set threshold s min If yes, add Z to the current knowledge base K in ; If not, terminate this round of incremental learning process; (4.4) Assume that the result of removing the oldest set of time series features from K is K′, and calculate the information loss InfoLoss(K, K′) caused by removing this set of time series features to the knowledge base; when the value of InfoLoss(K, K′) is less than a set threshold l min When , the oldest set of time series features is removed from K; the calculation expression of the information loss InfoLoss(K,K′) is: InfoLoss(K,K′)=H(K)-H(K′) Among them, P(X i,j ) represents the probability that the elements of matrix X appear in the i-th bin and the j-th feature position after the value range of the j-th feature is binned; (4.5) Determine whether the updated length of K is divisible by a set integer bs. If so, use the current K as the training data set and use the supervised loss function to fine-tune the regressor network part to update the parameters of the regressor network part of the soft sensor model. If not, terminate this round of incremental learning process directly. The expression of the supervised loss function is:

Images