Industrial process soft measurement method and device for latent predictability embedded deep learning

Abstract

The invention provides a potential predictability embedded deep learning industrial process soft measurement method and device, and relates to the technical field of deep learning industrial process soft measurement. The method comprises the following steps: designing a predictability regularization item according to a probability dynamic model of potential features constructed by a variational cycle network, a time graph constructed by a training data set and a time lag constant, constructing a target-related auto-encoder embedded by a probability potential predictability model, constructing a supervision depth network embedded by the probability potential predictability model, and constructing a target-related auto-encoder embedded by the probability potential predictability model. And obtaining a quality variable prediction value of the industrial production process. According to the method, firstly, a potential feature prediction design regularization item of probability distribution is designed, and network parameters are optimized through a joint training model to ensure overall convergence of the network. Secondly, an improved variational graph recurrent neural network is provided, randomness is brought into an advanced potential space, modeling is carried out on potential predictability of a supervised deep network, and additional graph structure information is added to help to analyze time dependence;

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning industrial process soft measurement, and in particular to an industrial process soft measurement method and device with potential predictability embedded in deep learning. Background Art

[0002] With the increasing requirements for product quality and economic benefits, modern industrial processes have become very complex and large-scale. In such processes, shallow learning models have difficulty extracting strong nonlinear features from high-dimensional process data. To overcome these inherent limitations, people have begun to use deep architectures to model complex industrial data. Therefore, the use of deep learning for soft sensor development has become increasingly important.

[0003] Among various deep learning models, SAE (Stacked Auto-encoders) is one of the most widely used neural networks. In order to enhance the functionality of the original encoder, STAE (Stacked Auto-encoders) was introduced. STAE aims to extract deep features that are particularly relevant to the prediction of the desired output. However, STAE is mainly used for self-reconstruction of process variables and quality variables, so it is difficult to describe dynamic industrial processes because the trend information hidden behind the dynamic process will be lost.

[0004] Traditional data-driven modeling methods mainly include various multivariate statistical analysis methods and machine learning models. With the continuous improvement of people's requirements for product quality and environmental protection, modern industrial processes have become very complex and large-scale. Shallow learning models are difficult to handle these strongly nonlinear, high-dimensional, strongly coupled, and dynamic process data. Therefore, it is necessary to be able to input process data into an end-to-end soft measurement model. The deep learning method is different from traditional shallow technology. It simulates the information transmission mode of human brain neurons and extracts deep features hidden in the data by deploying multi-layer, nonlinear perception and mapping. It reduces the dependence on signal processing and artificial feature extraction, allows end-to-end learning, and is applied to the field of industrial process soft measurement.

[0005] In recent years, most of the models used in the work are based on static assumptions. In the application of industrial process soft measurement, there are drawbacks of ignoring dynamic autocorrelation in process data and losing trend information. How to capture dynamic autocorrelation in process data and mining process trend information for soft measurement modeling is a problem worth studying. At the same time, the potential feature layer of the deep learning model has complex distribution and time dependence, so a specific method is needed to establish a potential predictability model to design regularization terms to extract smooth and predictable potential features. Dynamic model methods are mainly divided into dynamic matrix expansion, time series models and dynamic neural networks, such as dynamic principal component analysis models, autoregressive sliding average models, recurrent neural networks and long short-term memory. Although most of the current work can achieve dynamic feature extraction to a certain extent, the above dynamic models focus on dynamic feature extraction, and the dynamic characteristics constraints of the potential features should be introduced into soft measurement based on stacked autoencoders.

[0006] Nowadays, the ARAE (Autoregressive Autoencoder) model is inspired by the method involving an explicit dynamic model and an external model compatible with the internal dynamic model structure, which is designed for nonlinear dynamic latent variable extraction in long-term dependent process systems. However, since the iterative training will not stop until the latent space is determined, the network may fall into a local minimum through the cross-training of the reconstruction loss and the autoregressive loss. Therefore, compared with the explicit introduction of the autoregressive model in the AE (Autoencoder), the AE related to manifold learning has better results in dynamic feature extraction. In order to determine the dynamic neighborhood relationship between adjacent samples, an adjacency graph is constructed using temporal distance weights. However, since the temporal dependency is only roughly estimated, this method may produce redundant or false neighbor nodes. Similarly, the dynamic relationship caused by periodicity or lag is not considered. These problems limit the application of autoencoders in soft sensors for dynamic processes. Summary of the invention

[0007] In order to solve the existing technical problems of 1. The deep learning model in the industrial process soft measurement model is based on static assumptions, ignoring the dynamic autocorrelation of process data and losing the trend information in the process; 2. The complex time dependency and distribution of potential features; 3. The traditional alternating training predictability loss and supervised deep network loss lead to the inability to guarantee the overall network convergence and easy to fall into the local minimum, the embodiment of the present invention provides an industrial process soft measurement method and device with potential predictability embedded in deep learning. The technical solution is as follows:

[0008] On the one hand, a method for industrial process soft measurement with potential predictability embedded in deep learning is provided, the method is implemented by an industrial process soft measurement device, and the method includes:

[0009] S1. Obtain industrial production process data, preprocess the industrial production process data, and obtain a training data set.

[0010] S2. Analyze the time correlation of the training data set and select a time lag constant.

[0011] S3. According to the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the time graph constructed by the training dataset, and the time lag constant, a predictability regularization term is designed, and a target-dependent autoencoder for embedding the probabilistic latent predictability model is constructed according to the predictability regularization term.

[0012] S4. Stack multiple target-related autoencoders for probabilistic potential predictability model embedding, build a supervised deep network for probabilistic potential predictability model embedding, train the supervised deep network for probabilistic potential predictability model embedding layer by layer, and obtain a verified supervised deep network for probabilistic potential predictability model embedding.

[0013] S5. Obtain process variables in the industrial production process and input them into the supervised deep network embedded with the verified probabilistic potential predictability model to obtain the predicted values ​​of the quality variables in the industrial production process.

[0014] Optionally, the probabilistic dynamic model of the latent features constructed according to the variational recurrent network, the time graph constructed from the training data set, and the time lag constant in S3 are used to design a predictability regularization term, including:

[0015] The training data set is input into the encoding layer of the target-related autoencoder, and a latent feature space is obtained after transformation. Minimize the projected random variable in the latent feature space The sum of the variances of .

[0016] Among them, according to the time graph constructed by the training data set and the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the autocorrelation of the latent features across time is constructed, and the predictability regularization term is designed using the negative log-likelihood.

[0017] The predictability regularization term is added to the loss function of the target-dependent autoencoder, resulting in the loss function of the supervised deep network embedded with the probabilistic latent predictability model.

[0018] Optionally, the autocorrelation of the latent features across time is as shown in the following formula (1):

[0019]

[0020] In the formula, p(·|·) represents the conditional probability, represents the potential feature at the i-th moment, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is The normal distribution of .

[0021] Optionally, a predictability regularization term is given by equation (2):

[0022]

[0023] In the formula, represents the predictability regularization term, N represents the number of samples, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is The normal distribution of .

[0024] Optionally, the loss function of the supervised deep network embedded with the probabilistic latent predictability model is as shown in equation (3):

[0025]

[0026] In the formula, The loss function of the supervised deep network representing the embedding of the probabilistic latent predictability model, represents the loss function of the supervised deep network, represents the predictability regularization term, λ p Represents the adjustable coefficient of the predictability regularization term.

[0027] Optionally, a probabilistic dynamics model of the latent features to describe the temporal dependencies between the latent features in the latent space of the stacked target-dependent autoencoders.

[0028] The probabilistic dynamic model of latent features constructed based on the variational recurrent network in S3 includes:

[0029] A probabilistic dynamic model of latent features is constructed according to the variational recurrent network. Based on the evidence lower bound of the variational autoencoder, the time factor is introduced into the probabilistic dynamic model of latent features. According to the factorization formula, the variational lower bound is obtained. The goal of the probabilistic dynamic model of latent features is set to maximize the evidence lower bound, and the empirical loss function is obtained.

[0030] Optionally, the temporal graph is used to guide the regularization term so that the latent features reconstructed in the probabilistic dynamic model of the latent features can maintain the temporal graph structure in the original data as much as possible.

[0031] The training dataset in S3 constructs a time graph, including:

[0032] Augment the training data set to obtain the augmented data According to the original data vector in the augmented data The original data vector with τ time steps The probability distribution difference of τ is used to measure the distance between the original data vectors of τ time steps, and then the time graph is constructed; where τ represents the time lag constant.

[0033] On the other hand, a potential predictability embedded deep learning industrial process soft measurement device is provided, and the device is applied to the potential predictability embedded deep learning industrial process soft measurement method, and the device includes:

[0034] The acquisition module is used to acquire industrial production process data, pre-process the industrial production process data, and obtain a training data set.

[0035] The selection module is used to analyze the time correlation of the training data set and select the time lag constant.

[0036] A module is designed to design a predictability regularization term based on the probabilistic dynamic model of latent features constructed by a variational recurrent network, a time graph constructed from a training dataset, and a time lag constant, and a target-dependent autoencoder for constructing a probabilistic latent predictability model embedding based on the predictability regularization term.

[0037] The training module is used to stack target-related autoencoders embedded with multiple probabilistic potential predictability models, build a supervised deep network embedded with probabilistic potential predictability models, train the supervised deep network embedded with probabilistic potential predictability models layer by layer, and obtain a verified supervised deep network embedded with probabilistic potential predictability models.

[0038] The output module is used to obtain process variables in the industrial production process and input them into the supervised deep network embedded in the verified probabilistic potential predictability model to obtain the predicted values ​​of the quality variables in the industrial production process.

[0039] Optionally, the design module is further configured to:

[0040] The training data set is input into the encoding layer of the target-related autoencoder, and a latent feature space is obtained after transformation. Minimize the projected random variable in the latent feature space The sum of the variances of .

[0041] Among them, according to the time graph constructed by the training data set and the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the autocorrelation of the latent features across time is constructed, and the predictability regularization term is designed using the negative log-likelihood.

[0042] The predictability regularization term is added to the loss function of the target-dependent autoencoder, resulting in the loss function of the supervised deep network embedded with the probabilistic latent predictability model.

[0043] Optionally, the autocorrelation of the latent features across time is as shown in the following formula (1):

[0044]

[0045] In the formula, p(·|·) represents the conditional probability, represents the potential feature at the i-th moment, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is The normal distribution of .

[0046] Optionally, a predictability regularization term is given by equation (2):

[0047]

[0048] Where, L p represents the predictability regularization term, N represents the number of samples, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is The normal distribution of .

[0049] Optionally, the loss function of the supervised deep network embedded with the probabilistic latent predictability model is as shown in equation (3):

[0050]

[0051] In the formula, The loss function of the supervised deep network representing the embedding of the probabilistic latent predictability model, represents the loss function of the supervised deep network, represents the predictability regularization term, λ p Represents the adjustable coefficient of the predictability regularization term.

[0052] Optionally, a probabilistic dynamics model of the latent features to describe the temporal dependencies between the latent features in the latent space of the stacked target-dependent autoencoders.

[0053] Design modules are further used to:

[0054] A probabilistic dynamic model of latent features is constructed according to the variational recurrent network. Based on the evidence lower bound of the variational autoencoder, the time factor is introduced into the probabilistic dynamic model of latent features. According to the factorization formula, the variational lower bound is obtained. The goal of the probabilistic dynamic model of latent features is set to maximize the evidence lower bound, and the empirical loss function is obtained.

[0055] Optionally, the temporal graph is used to guide the regularization term so that the latent features reconstructed in the probabilistic dynamic model of the latent features can maintain the temporal graph structure in the original data as much as possible.

[0056] Design modules are further used to:

[0057] Augment the training data set to obtain the augmented data According to the original data vector in the augmented data The original data vector with τ time steps The probability distribution difference of τ is used to measure the distance between the original data vectors of τ time steps, and then the time graph is constructed; where τ represents the time lag constant.

[0058] On the other hand, an industrial process soft measurement device is provided, which includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned industrial process soft measurement methods for embedding potential predictability into deep learning is implemented.

[0059] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned industrial process soft measurement methods with potential predictability embedded in deep learning.

[0060] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0061] In the present invention, the influence of dynamic autocorrelation and trend information in process data on quality variables can be effectively mined, the time dependency of potential features can be modeled using a graph-guided potential probability dynamic model, the extraction of potential features at the current time point can be constrained using potential features from previous multi-step time points, and all optimizers can be merged to train the parameters of the entire network, thereby improving the smoothness and predictability of potential features, ensuring the convergence of the network, and improving the accuracy of soft measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0063] Figure 1 It is a flow chart of an industrial process soft sensing method with potential predictability embedded in deep learning provided by an embodiment of the present invention;

[0064] Figure 2 It is a structural schematic diagram of an industrial process soft-sensing method with potential predictability embedded in deep learning provided by an embodiment of the present invention;

[0065] Figure 3 is a schematic diagram of a debutanizer process provided by an embodiment of the present invention;

[0066] Figure 4 is a prediction result diagram provided by an embodiment of the present invention;

[0067] Figure 5 It is a block diagram of an industrial process soft measurement device with potential predictability embedded in deep learning provided by an embodiment of the present invention;

[0068] Figure 6 The present invention is a schematic diagram of the structure of an industrial process soft measurement device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0070] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0071] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0072] In the embodiments of the present invention, sometimes the subscripts such as W 1It may be written in non-subscript form such as W1. When the difference is not emphasized, the meaning is the same.

[0073] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0074] The embodiment of the present invention provides an industrial process soft measurement method with potential predictability embedded in deep learning, which can be implemented by an industrial process soft measurement device, which can be a terminal or a server. Figure 1 The flowchart of the industrial process soft sensing method with potential predictability embedded in deep learning is shown in the figure. The processing flow of the method may include the following steps:

[0075] S1. Obtain industrial production process data, preprocess the industrial production process data, and obtain a training data set.

[0076] In a feasible implementation, Figure 2 As shown in the figure, the easily measurable process variables in the industrial production process are determined as the input of soft measurement, the relevant data for soft measurement modeling is read from the industrial process database, and the data is serialized and preprocessed into a training set. Assuming that there are M easily measurable process variables, N samples, and one is the quality variable that needs to be predicted in the data set, the original data set can be expressed as where x i,j represents the jth sample of the i-th process variable; y j Represents the jth sample of the quality variable to be measured.

[0077] S2. Analyze the time correlation of the training data set and select the time lag constant τ.

[0078] S3. According to the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the time graph constructed by the training dataset, and the time lag constant, a predictability regularization term is designed, and a target-dependent autoencoder for embedding the probabilistic latent predictability model is constructed according to the predictability regularization term.

[0079] In one feasible implementation, the original data is input into a stacked autoencoder, a probabilistic model is applied in the latent space, and a predictability regularization term is designed using the latent features of the past τ time steps and the probabilistic prediction loss and added to the decoding loss to learn smooth and predictable features.

[0080] Specifically, in step S3, a potential predictability loss is designed based on the distribution difference between the current potential features and the past neighborhood points.

[0081] Furthermore, according to the probabilistic dynamic model of the potential features constructed by the variational recurrent network, the time graph constructed by the training dataset, and the time lag constant, a predictability regularization term is designed, including:

[0082] The training data set is input into the encoding layer of the target-related autoencoder, and a latent feature space is obtained after transformation. Minimize the projected random variable in the latent feature space The sum of the variances of .

[0083] In one possible implementation, each feature in the latent feature layer of the autoencoder should have a low variance with the features of the previous τ time steps. As its variance decreases, predictability and smoothness will increase. The original data is input into the encoding layer of the autoencoder, and after transformation f θ Get a latent feature space Assume that x in the process data t There is a strong correlation with the data of the previous τ time steps, and the parameters are learned in the latent space by designing constraints to maintain this autocorrelation across time. and In other words, the proposed method aims to minimize the projected random variable in this space. The trace of the expected matrix of the covariance is:

[0084]

[0085] In the formula, Represents the expectation operator of the covariance matrix between the current latent feature and the combined output of the latent features at time τ in the past.

[0086] Among them, the past process data x is measured by mutual information t-k For the current data x t The contribution of mRMR (Minimal Redundancy Maximal Relevance) criterion is used to determine the time lag constant τ. After determining the time lag constant, for potential features with complex time dependencies, a probability model is introduced to construct the autocorrelation of potential features across time:

[0087]

[0088] In the formula, p(·|·) represents the conditional probability, represents the potential feature at the i-th moment, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is The normal distribution of .

[0089] Furthermore, according to the above probability model, the predictability regularization term is designed using negative log-likelihood:

[0090]

[0091] In the formula, represents the predictability regularization term, N represents the number of samples, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is The normal distribution of .

[0092] Furthermore, the designed predictability regularization term is added to the loss function of the target-dependent autoencoder, and the loss function of the supervised deep network embedded with predictability is for:

[0094]

[0095] In the formula, The loss function of the supervised deep network representing the embedding of the probabilistic latent predictability model, represents the loss function of the supervised deep network, represents the predictability regularization term, λ p Represents the adjustable coefficient of the predictability regularization term.

[0096] Furthermore, the present invention designs a potential probability dynamic model and designs a regularization term guided by the original data graph structure based on the probability distribution difference.

[0097] Specifically, a probabilistic dynamic model is designed for the latent space of the stacked target-related autoencoder to describe the complex temporal dependencies between latent features. For latent features, the construction of the latent probabilistic dynamic model is divided into four modules, including prior, generation, recursion, and reasoning. Based on the ELBO (Evidence Lower Bound) of VAE (Variational Autoencoder), the time factor is introduced into the latent probabilistic dynamics based on VRNN (Variational recurrent neural network). According to the factorization formula, the variational lower bound can be obtained, and the following loss function is designed by minimizing the negative variational lower bound:

[0098]

[0099] Where the first term is the inference term, which is the approximate posterior distribution q(z t |H ≤t ,z ＜t ) and the conditional prior distribution p(z t |H ＜t ,z ＜t ), where D KL (·||·) represents the Kullback-Leibler divergence between the two distributions, represents the loss function of VRNN, φ represents the mapping function of VRNN network, θ represents the parameters of VRNN, represents the expectation operator, z represents the random hidden state variable in VRNN, and T represents the length of the potential feature sequence; the second term is the logarithmic expectation of the generation process, and the expected calculation is approximated by sequential Monte Carlo sampling.

[0100] It must be emphasized that the goal is to maximize the evidence lower bound. Therefore, minimizing the negative ELBO will lead to the desired results. Therefore, the empirical loss function formula is as follows:

[0101]

[0102] in, zt is the hidden state of the variational recurrent network.

[0103] Furthermore, a regularization term guided by the original data graph structure is designed based on the probability distribution difference between the current potential feature and its neighborhood points.

[0104] First, we obtain the original data by augmenting Through the original data vector The original data vector with τ time steps The probability distribution difference is measured to measure the distance between the original data vectors of τ time steps, and then the time graph of the original data is constructed. The graph-guided regularization term is designed so that the latent features reconstructed in the probabilistic dynamic model of the latent features can maintain the time graph structure in the original data as much as possible.

[0105] Secondly, the latent features are augmented to obtain Through the latent feature vector Its latent feature vector at τ time steps Construct a latent predictability model: The distance d(·,·) between the current potential feature and its neighboring features is expressed as the difference in probability distribution:

[0106]

[0107] In the formula, p θRepresents the marginal probability function of the latent feature.

[0108] The difference in probability distribution between the current potential feature and its neighboring features is expressed using the Kullback-Leibler (KL) divergence:

[0109]

[0110] Among them, D KL (·|·) is the KL divergence.

[0111] According to the distribution difference between the data at each moment and the neighborhood data, the regularization term guided by the graph structure is designed as:

[0112]

[0113] S4. Stack multiple target-related autoencoders for probabilistic potential predictability model embedding, build a supervised deep network for probabilistic potential predictability model embedding, train the supervised deep network for probabilistic potential predictability model embedding layer by layer, and obtain a verified supervised deep network for probabilistic potential predictability model embedding.

[0114] In a feasible implementation, the target-related autoencoders embedded in a single probabilistic potential predictability model are stacked to build a supervised deep network for probabilistic potential predictability model embedding, the built supervised deep network model for probabilistic potential predictability model embedding is trained layer by layer, and the regression network parameters are adjusted and updated according to the prediction error, finally obtaining a verified supervised deep network for probabilistic potential predictability model embedding.

[0115] Furthermore, the input data of the test set is input into the supervised deep network embedded with the saved probabilistic potential predictability model, and the predicted value of the industrial process quality variable can be obtained through forward propagation of the supervised deep network embedded with the probabilistic potential predictability model.

[0116] S5. Obtain process variables in industrial production processes and input them into the supervised deep network embedded with the verified probabilistic potential predictability model to obtain the predicted values ​​of quality variables in industrial production processes. It can be widely used in complex industrial processes such as debutanizer processes. In this process, seven easily measurable variables such as Figure 3 The blue block unit shown includes the tower top pressure, reflux flow, flow to the next process, the temperature of the 6th tower plate, the tower bottom temperature A and the tower bottom temperature B as the soft sensor input to obtain the predicted value of butane concentration. The prediction results are as follows Figure 4 shown.

[0117] Aiming at the problem that the soft measurement of industrial process ignores the dynamic autocorrelation of process variables, the present invention proposes an industrial process soft measurement method based on probabilistic potential predictability embedded in a supervised deep network, embeds the dynamic model of potential features into a deep learning framework and applies it to the soft measurement of industrial process; due to the complex dependencies between potential sequences, a probabilistic dynamic model with graph guidance based on past and current potential features is designed for the latent space of the supervised deep network, and randomness is incorporated into the advanced latent space to model the potential predictability of the supervised deep network, and additional graph structure information is added to help analyze time dependencies, thereby improving the ability to extract dynamic process features; the loss functions embedded in the supervised deep network and the potential predictability model are merged, and the network parameters are optimized by jointly training the model to ensure the overall convergence of the network. Aiming at the fact that the deep learning model in the industrial process soft measurement model is based on static assumptions, ignores the dynamic autocorrelation of process data, and loses trend information in the process. A soft sensing method for industrial processes based on latent predictability embedding deep learning networks is proposed. Latent features from previous time points are used to model potential time dependencies. A predictability regularization term is designed based on the loss predicted by the latent distribution across the time dimension, so that the improved autoencoder can capture the dynamic autocorrelation of process variables.

[0118] Aiming at the complex temporal dependencies and distributions of latent features, a variational recurrent neural network is used to construct an uncertain dynamic model of random latent features, and the random latent states of latent features in past time steps are integrated into recurrent neurons. At the same time, the relationship between past and current latent features is associated with an information quantity measure with graph-based predictability. Therefore, the predictability of latent features can be simplified to a graph structure to guide the capture of dynamic autocorrelation properties between latent features, introduce the graph structure information of the original data into the latent probabilistic dynamic model, and learn the interpretable temporal correlation between latent sequences.

[0119] To address the problem that traditional alternating training predictability loss and supervised deep network loss cannot guarantee the convergence of the overall network and are prone to falling into local minima, the variational recurrent network for modeling the predictability of potential features and the optimizer of the supervised deep network are merged to ensure the convergence characteristics of the network.

[0120] In the embodiment of the present invention, the influence of dynamic autocorrelation and trend information in process data on quality variables can be effectively mined, the time dependency of potential features can be modeled using a graph-guided potential probability dynamic model, the extraction of potential features at the current time point can be constrained using potential features from previous multi-step time points, and all optimizers can be merged to train the parameters of the entire network, thereby improving the smoothness and predictability of potential features, ensuring the convergence of the network, and improving the accuracy of soft measurement.

[0121] Figure 5 1 is a block diagram of an industrial process soft measurement device for embedding deep learning with potential predictability according to an exemplary embodiment, wherein the device is used for an industrial process soft measurement method for embedding deep learning with potential predictability. Figure 5 The device includes an acquisition module 310, a selection module 320, a design module 330, a training module 340 and an output module 350. Among them:

[0122] The acquisition module 310 is used to acquire industrial production process data, pre-process the industrial production process data, and obtain a training data set.

[0123] The selection module 320 is used to analyze the time correlation of the training data set and select a time lag constant.

[0124] Design module 330, for designing a predictability regularization term based on the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the time graph constructed by the training data set, and the time lag constant, and constructing a target-related autoencoder for embedding the probabilistic latent predictability model based on the predictability regularization term.

[0125] The training module 340 is used to stack multiple target-related autoencoders embedded with probabilistic potential predictability models, build a supervised deep network embedded with probabilistic potential predictability models, train the supervised deep network embedded with probabilistic potential predictability models layer by layer, and obtain a verified supervised deep network embedded with probabilistic potential predictability models.

[0126] The output module 350 is used to obtain process variables in the industrial production process and input them into the supervised deep network embedded with the verified probabilistic potential predictability model to obtain the predicted values ​​of the quality variables of the industrial production process.

[0127] In the embodiment of the present invention, the influence of dynamic autocorrelation and trend information in process data on quality variables can be effectively mined, the time dependency of potential features can be modeled using a graph-guided potential probability dynamic model, the extraction of potential features at the current time point can be constrained using potential features from previous multi-step time points, and all optimizers can be merged to train the parameters of the entire network, thereby improving the smoothness and predictability of potential features, ensuring the convergence of the network, and improving the accuracy of soft measurement.

[0128] Figure 6 is a schematic diagram of the structure of an industrial process soft measurement device provided by an embodiment of the present invention, such as Figure 6 As shown, the industrial process soft measurement equipment may include the above Figure 5 The industrial process soft measurement device with potential predictability embedded in deep learning is shown. Optionally, the industrial process soft measurement device 410 may include a first processor 2001 .

[0129] Optionally, the industrial process soft measurement device 410 may further include a memory 2002 and a transceiver 2003 .

[0130] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0131] Combine the following Figure 6 The components of the industrial process soft measurement device 410 are introduced in detail:

[0132] The first processor 2001 is the control center of the industrial process soft measurement device 410, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0133] Optionally, the first processor 2001 may execute various functions of the industrial process soft measurement device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0134] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 6 CPU0 and CPU1 are shown in FIG.

[0135] In a specific implementation, as an embodiment, the industrial process soft measurement device 410 may also include multiple processors, such as Figure 6 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0136] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0137] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be accessed through the interface circuit ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0138] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0139] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 6 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0140] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0141] It should be noted that Figure 6 The structure of the industrial process soft measurement device 410 shown in the figure does not constitute a limitation on the router, and the actual knowledge structure recognition device may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0142] In addition, the technical effects of the industrial process soft measurement device 410 can refer to the technical effects of the industrial process soft measurement method with potential predictability embedded in deep learning described in the above method embodiment, which will not be repeated here.

[0143] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0144] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0145] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0146] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0147] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0148] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0149] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0150] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0151] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0153] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0154] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0155] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for industrial process soft sensing with potential predictability embedded in deep learning, characterized in that: The method comprises: S1. Acquire industrial production process data, and preprocess the industrial production process data to obtain a training data set; S2, analyzing the time correlation of the training data set and selecting a time lag constant; S3, designing a predictability regularization term based on the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the time graph constructed by the training data set, and the time lag constant, and constructing a target-related autoencoder embedded with the probabilistic latent predictability model based on the predictability regularization term; S4, stacking a plurality of target-related autoencoders embedded with the probabilistic potential predictability model, building a supervised deep network embedded with the probabilistic potential predictability model, and training the supervised deep network embedded with the probabilistic potential predictability model layer by layer to obtain a verified supervised deep network embedded with the probabilistic potential predictability model; S5. Obtain process variables in the industrial production process, and input them into the supervised deep network embedded with the verified probabilistic potential predictability model to obtain predicted values ​​of quality variables in the industrial production process.

2. The industrial process soft sensing method of embedding potential predictability into deep learning according to claim 1 is characterized in that: The predictability regularization term is designed based on the probabilistic dynamic model of the potential features constructed according to the variational recurrent network, the time graph constructed by the training data set, and the time lag constant in S3, including: The training data set is input into the encoding layer of the target-related autoencoder, and a latent feature space is obtained after transformation. Minimize the projected random variable in the latent feature space The sum of the variances of Among them, according to the time graph constructed from the training data set and the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the autocorrelation of the latent features across time is constructed, and the predictability regularization term is designed using the negative log-likelihood; The predictability regularization term is added to the loss function of the target-dependent autoencoder to obtain the loss function of the supervised deep network embedded with the probabilistic latent predictability model.

3. The industrial process soft sensing method of embedding potential predictability into deep learning according to claim 2 is characterized in that: The autocorrelation of the latent feature across time is shown in the following formula (1): In the formula, p(·|·) represents the conditional probability, represents the potential feature at the i-th moment, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is The normal distribution of .

4. The industrial process soft sensing method of embedding potential predictability into deep learning according to claim 2 is characterized in that: The predictability regularization term is shown in the following formula (2): In the formula, represents the predictability regularization term, N represents the number of samples, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is The normal distribution of .

5. The industrial process soft sensing method of embedding potential predictability into deep learning according to claim 2 is characterized in that: The loss function of the supervised deep network embedded in the probabilistic potential predictability model is shown in the following formula (3): In the formula, The loss function of the supervised deep network representing the embedding of the probabilistic latent predictability model, represents the loss function of the supervised deep network, represents the predictability regularization term, λ p Represents the adjustable coefficient of the predictability regularization term.

6. The industrial process soft sensing method of embedding potential predictability into deep learning according to claim 1 is characterized in that: a probabilistic dynamic model of the latent features for describing temporal dependencies between latent features in the latent space of the stacked target-dependent autoencoders; The probabilistic dynamic model of potential features constructed according to the variational recurrent network in S3 includes: A probabilistic dynamic model of latent features is constructed according to a variational recurrent network. Based on the evidence lower bound of the variational autoencoder, the time factor is introduced into the probabilistic dynamic model of latent features. According to the factorization formula, the variational lower bound is obtained. The goal of the probabilistic dynamic model of the latent features is set to maximize the evidence lower bound, and the empirical loss function is obtained.

7. The industrial process soft sensing method of embedding potential predictability into deep learning according to claim 1 is characterized in that: The time graph is used to guide the regularization term so that the latent features reconstructed in the probabilistic dynamic model of the latent features can maintain the time graph structure in the original data as much as possible; The training data set in S3 constructs a time graph, including: Augment the training data set to obtain the augmented data According to the original data vector in the augmented data The original data vector with τ time steps The probability distribution differences of t=1,2,…,τ are used to measure the distance between the original data vectors of τ time steps, and then construct a time graph; where τ represents the time lag constant.

8. A soft-sensing device for an industrial process with potential predictability embedded in deep learning, wherein the soft-sensing device for an industrial process with potential predictability embedded in deep learning is used to implement the soft-sensing method for an industrial process with potential predictability embedded in deep learning as claimed in any one of claims 1 to 7, characterized in that: The device comprises: An acquisition module is used to acquire industrial production process data, pre-process the industrial production process data, and obtain a training data set; A selection module, used to analyze the time correlation of the training data set and select a time lag constant; A design module for designing a predictability regularization term based on a probabilistic dynamic model of latent features constructed by a variational recurrent network, a time graph constructed by a training data set, and the time lag constant, and constructing a target-dependent autoencoder for embedding a probabilistic latent predictability model based on the predictability regularization term; A training module, used for stacking a plurality of target-related autoencoders embedded with the probabilistic potential predictability model, building a supervised deep network embedded with the probabilistic potential predictability model, and training the supervised deep network embedded with the probabilistic potential predictability model layer by layer to obtain a verified supervised deep network embedded with the probabilistic potential predictability model; The output module is used to obtain process variables in the industrial production process and input them into the supervised deep network embedded with the verified probabilistic potential predictability model to obtain the predicted values ​​of the quality variables in the industrial production process.

9. An industrial process soft measurement device, characterized in that: The industrial process soft measurement device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images