Industrial process soft-sensing method and device embedding deep learning with potential predictability

Abstract

The application provides an industrial process soft measurement method and device embedding deep learning with potential predictability, and relates to the technical field of deep learning industrial process soft measurement. The method comprises the following steps: designing a predictability regularization term according to a probability dynamic model of latent features constructed by a variational recurrent network, a time graph constructed by a training data set and a time lag constant, constructing a target related autoencoder embedded with a probability latent predictability model, building a supervised deep network embedded with the probability latent predictability model, and obtaining a quality variable prediction value of an industrial production process. The application firstly designs a regularization term for prediction by using latent features of probability distribution, and optimizes network parameters by jointly training the model to ensure the overall convergence of the network. Secondly, an improved variational graph recurrent neural network is proposed, randomness is introduced into a high-level latent space, the latent predictability of the supervised deep network is modeled, and additional graph structure information is added to help analyze the time dependence.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning industrial process soft measurement, and in particular to an industrial process soft measurement method and device with potential predictability embedded in deep learning. Background Art

[0002] With increasing demands for product quality and economic efficiency, modern industrial processes are becoming increasingly complex and large-scale. In such processes, shallow learning models struggle to extract strong nonlinear features from high-dimensional process data. To overcome these inherent limitations, deep architectures are being used to model complex industrial data. Consequently, the use of deep learning for soft sensor development is becoming increasingly important.

[0003] Among various deep learning models, stacked auto-encoders (SAEs) are one of the most widely used neural networks. To enhance the capabilities of the original encoder, stacked target-dependent auto-encoders (STAEs) were introduced. STAEs are designed to extract deep features that are particularly relevant to the prediction of desired outputs. However, STAEs are primarily designed for self-reconstruction of process and quality variables, making them difficult to describe dynamic industrial processes because the underlying trend information is lost.

[0004] Traditional data-driven modeling methods primarily include various multivariate statistical analysis methods and machine learning models. With increasing demands for product quality and environmental protection, modern industrial processes have become extremely complex and large-scale. Shallow learning models struggle to process these highly nonlinear, high-dimensional, tightly coupled, and dynamic process data. Therefore, it is necessary to be able to input process data into end-to-end soft sensing models. Deep learning methods differ from traditional shallow techniques in that they mimic the information transfer patterns of neurons in the human brain. By deploying multi-layer, nonlinear perception and mapping, they extract deep features hidden in the data, reducing reliance on signal processing and manual feature extraction, allowing for end-to-end learning and application in the field of industrial process soft sensing.

[0005] In recent years, most models used in industrial process soft sensing have been based on static assumptions. This has the drawback of ignoring dynamic autocorrelations and losing trend information in process data. Capturing dynamic autocorrelations and mining process trend information for soft sensing modeling is a worthy research question. Furthermore, the latent feature layers of deep learning models have complex distributions and temporal dependencies, necessitating specific methods to model latent predictability and design regularization terms to extract smooth, predictable latent features. Dynamic model approaches are primarily categorized as dynamic matrix expansion, time series models, and dynamic neural networks, such as dynamic principal component analysis (PCA), autoregressive moving average (ARSA), recurrent neural networks, and long short-term memory (LSTM). While most current work can achieve dynamic feature extraction to some extent, these dynamic models focus on dynamic feature extraction. The dynamic nature of latent features should be constrained in soft sensing based on stacked autoencoders.

[0006] The ARAE (Autoregressive Autoencoder) model, inspired by methods involving an explicit dynamic model and an external model compatible with the internal dynamic model structure, is designed for extracting nonlinear dynamic latent variables from long-term dependent process systems. However, because iterative training does not stop until the latent space is determined, the network may become trapped in a local minimum due to the cross-training of the reconstruction loss and the autoregressive loss. Therefore, compared with explicitly introducing an autoregressive model in the AE (Autoencoder), the AE associated with manifold learning has better results in dynamic feature extraction. To determine the dynamic neighborhood relationships between adjacent samples, an adjacency graph is constructed using temporal distance weights. However, since temporal dependencies are only roughly estimated, this approach may produce redundant or false neighbor nodes. Similarly, dynamic relationships caused by periodicity or lags are not considered. These issues limit the application of autoencoders in soft sensors for dynamic processes. Summary of the Invention

[0007] In order to solve the technical problems existing in the prior art, namely: 1. The deep learning model in the industrial process soft measurement model is based on static assumptions, ignoring the dynamic autocorrelation of process data and losing trend information in the process; 2. The complex time dependency and distribution of potential features; 3. The traditional alternating training predictability loss and supervised deep network loss lead to the inability to ensure the overall network convergence and easy to fall into local minimum, the embodiment of the present invention provides an industrial process soft measurement method and device with potential predictability embedded in deep learning. The technical solution is as follows:

[0008] In one aspect, a method for industrial process soft sensing with embedded deep learning for potential predictability is provided. The method is implemented by an industrial process soft sensing device and includes:

[0009] S1. Acquire industrial production process data, preprocess the industrial production process data, and obtain a training data set.

[0010] S2. Analyze the time correlation of the training data set and select a time lag constant.

[0011] S3. Based on the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the time graph constructed by the training dataset, and the time lag constant, a predictability regularization term is designed, and a target-dependent autoencoder for embedding the probabilistic latent predictability model is constructed based on the predictability regularization term.

[0012] S4. Stack multiple target-related autoencoders embedded with probabilistic potential predictability models to build a supervised deep network embedded with probabilistic potential predictability models, train the supervised deep network embedded with probabilistic potential predictability models layer by layer, and obtain a verified supervised deep network embedded with probabilistic potential predictability models.

[0013] S5. Obtain process variables in the industrial production process and input them into the supervised deep network embedded with the verified probabilistic potential predictability model to obtain the predicted values ​​of the quality variables of the industrial production process.

[0014] Optionally, the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the time graph constructed by the training dataset, and the time lag constant in S3 are used to design a predictability regularization term, including:

[0015] The training data set is input into the encoding layer of the target-related autoencoder, and a latent feature space is obtained after transformation. Minimize the projected random variable in the latent feature space The sum of the variances of .

[0016] Among them, based on the time graph constructed according to the training dataset and the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the autocorrelation of the latent features across time is constructed, and the predictability regularization term is designed using the negative log-likelihood.

[0017] The predictability regularization term is added to the loss function of the target-dependent autoencoder, resulting in the loss function of the supervised deep network embedded with the probabilistic latent predictability model.

[0018] Optionally, the autocorrelation of the latent features across time is as shown in the following formula (1):

[0019]

[0020] Where p(·|·) represents the conditional probability, represents the potential characteristics at the i-th moment, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is Normal distribution.

[0021] Optionally, a predictability regularization term is provided as follows (2):

[0022]

[0023] Where, represents the predictability regularization term, N represents the number of samples, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is Normal distribution.

[0024] Optionally, the loss function of the supervised deep network embedded with the probabilistic latent predictability model is as shown in Equation (3):

[0025]

[0026] Where, The loss function for the supervised deep network that represents the embedding of the probabilistic latent predictability model, represents the loss function of the supervised deep network, represents the predictability regularization term, λ p represents the adjustable coefficient of the predictability regularization term.

[0027] Optionally, a probabilistic dynamics model of the latent features to describe the temporal dependencies between the latent features in the latent space of the stacked target-dependent autoencoders.

[0028] The probabilistic dynamic model of latent features constructed based on the variational recurrent network in S3 includes:

[0029] A probabilistic dynamic model of latent features is constructed based on a variational recurrent network. Based on the evidence lower bound of the variational autoencoder, the time factor is introduced into the probabilistic dynamic model of latent features. According to the factorization formula, the variational lower bound is obtained. The goal of the probabilistic dynamic model of latent features is set to maximize the evidence lower bound, and the empirical loss function is obtained.

[0030] Optionally, the temporal graph is used to guide the regularization term so that the latent features reconstructed in the probabilistic dynamic model of the latent features can maintain the temporal graph structure in the original data as much as possible.

[0031] The training dataset in S3 constructs a time graph, including:

[0032] Augment the training data set to obtain augmented data According to the original data vector in the augmented data The distance between the original data vectors of the τ time steps is measured according to the probability distribution difference of the original data vectors of the τ time steps, and a time graph is constructed; wherein τ represents a time lag constant.

[0033] In another aspect, a latent predictability embedded deep learning industrial process soft-sensing device is provided, which is applied to a latent predictability embedded deep learning industrial process soft-sensing method, and the device comprises:

[0034] An acquisition module is configured to acquire industrial production process data, pre-process the industrial production process data, and obtain a training data set.

[0035] A selection module is configured to analyze the time correlation of the training data set and select a time lag constant.

[0036] A design module is configured to design a predictability regularization term according to a probability dynamic model of a latent feature constructed by a variational recurrent network, a time graph constructed by the training data set, and the time lag constant, and construct a target-related autoencoder embedded with a probability latent predictability model according to the predictability regularization term.

[0037] A training module is configured to stack multiple target-related autoencoders embedded with the probability latent predictability model, build a supervised deep network embedded with the probability latent predictability model, train the supervised deep network embedded with the probability latent predictability model layer by layer, and obtain a verified supervised deep network embedded with the probability latent predictability model.

[0038] An output module is configured to acquire a process variable in an industrial production process, input the process variable into the verified supervised deep network embedded with the probability latent predictability model, and obtain a quality variable prediction value of the industrial production process.

[0039] Optionally, the design module is further configured to:

[0040] input the training data set into an encoding layer of the target-related autoencoder, and obtain a latent feature space through transformation minimize the sum of variances of the projected random variables in the latent feature space.

[0041] According to the time graph constructed by the training data set and the probability dynamic model of the latent feature constructed by the variational recurrent network, the cross-time autocorrelation of the latent feature is constructed, and the predictability regularization term is designed by using the negative log-likelihood.

[0042] ​​A predictability regularization term is added to the loss function of the target-related autoencoder to obtain a loss function of the supervised deep network embedding the probabilistic latent predictability model.

[0043] Optionally, the cross-time autocorrelation of the latent features is as shown in the following formula (1):

[0044]

[0045] In the formula, p(·|·) represents a conditional probability, represents the latent feature at the i th time, τ represents a time lag constant, represents obeys a normal distribution with a mean of μ i-1 and a variance of .

[0046] Optionally, the predictability regularization term is as shown in the following formula (2):

[0047]

[0048] In the formula, L p represents the predictability regularization term, N represents the number of samples, τ represents a time lag constant, represents obeys a normal distribution with a mean of μ i-1 and a variance of .

[0049] Optionally, the loss function of the supervised deep network embedding the probabilistic latent predictability model is as shown in the following formula (3):

[0050]

[0051] In the formula, represents the loss function of the supervised deep network embedding the probabilistic latent predictability model, represents the loss function of the supervised deep network, represents the predictability regularization term, λ p represents an adjustable coefficient of the predictability regularization term.

[0052] Optionally, a probabilistic dynamic model of the latent features is used to describe the time dependence between the latent features in the latent space of the stacked target-related autoencoder.

[0053] The design module is further used for:

[0054] A probabilistic dynamic model of latent features is constructed based on a variational recurrent network. Based on the evidence lower bound of the variational autoencoder, the time factor is introduced into the probabilistic dynamic model of latent features. According to the factorization formula, the variational lower bound is obtained. The goal of the probabilistic dynamic model of latent features is set to maximize the evidence lower bound, and the empirical loss function is obtained.

[0055] Optionally, the temporal graph is used to guide the regularization term so that the latent features reconstructed in the probabilistic dynamic model of the latent features can maintain the temporal graph structure in the original data as much as possible.

[0056] Design modules, further used to:

[0057] Augment the training data set to obtain augmented data According to the original data vector in the augmented data Its original data vector of τ time steps The probability distribution difference of τ is used to measure the distance between the original data vectors of τ time steps, and then a time graph is constructed; where τ represents the time lag constant.

[0058] On the other hand, an industrial process soft measurement device is provided, which includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned industrial process soft measurement methods for embedding potential predictability into deep learning is implemented.

[0059] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned industrial process soft measurement methods with potential predictability embedded in deep learning.

[0060] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0061] The present invention can effectively mine the influence of dynamic autocorrelation and trend information in process data on quality variables, use a graph-guided potential probability dynamic model to model the time dependency of potential features, use potential features from previous multi-step time points to constrain the extraction of potential features at the current time point, and merge all optimizers to train the parameters of the entire network, thereby improving the smoothness and predictability of potential features, ensuring the convergence of the network, and improving the accuracy of soft measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0063] Figure 1 This is a flow chart of an industrial process soft sensing method with potential predictability embedded in deep learning, provided by an embodiment of the present invention;

[0064] Figure 2 Schematic diagram of the structure of an industrial process soft sensing method with embedded deep learning for potential predictability provided by an embodiment of the present invention;

[0065] Figure 3 Schematic diagram of a debutanizer process according to an embodiment of the present invention;

[0066] Figure 4 is a prediction result diagram provided by an embodiment of the present invention;

[0067] Figure 5 This is a block diagram of an industrial process soft measurement device with potential predictability embedded in deep learning, provided by an embodiment of the present invention;

[0068] Figure 6 The diagram is a structural diagram of an industrial process soft measurement device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0070] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0071] In the embodiments of the present invention, the terms "image" and "picture" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same. The terms "of," "corresponding," and "corresponding" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same.

[0072] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0073] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0074] The embodiment of the present invention provides an industrial process soft measurement method with potential predictability embedded in deep learning. The method can be implemented by an industrial process soft measurement device, which can be a terminal or a server. Figure 1 The flowchart of the industrial process soft sensing method with embedded deep learning for potential predictability is shown. The processing flow of the method may include the following steps:

[0075] S1. Acquire industrial production process data, preprocess the industrial production process data, and obtain a training data set.

[0076] In a feasible implementation, Figure 2 As shown in Figure 1, the easily measurable process variables in the industrial production process are determined as the input of the soft measurement. The relevant data for soft measurement modeling is read from the industrial process database, and the data is serialized and preprocessed into a training set. Assuming that there are M easily measurable process variables, N samples, and one is the quality variable that needs to be predicted in the data set, the original data set can be expressed as where x i,j represents the jth sample of the i-th process variable; y j Represents the jth sample of the quality variable to be measured.

[0077] S2. Analyze the time correlation of the training data set and select the time lag constant τ.

[0078] S3. Based on the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the time graph constructed by the training dataset, and the time lag constant, a predictability regularization term is designed, and a target-dependent autoencoder for embedding the probabilistic latent predictability model is constructed based on the predictability regularization term.

[0079] In one feasible implementation, the original data is input into a stacked autoencoder, a probabilistic model is applied in the latent space, and a predictability regularization term is designed using the latent features of the past τ time steps and the probabilistic prediction loss to be added to the decoding loss to learn smooth and predictable features.

[0080] Specifically, in step S3, a potential predictability loss is designed based on the distribution difference between the current potential features and the past neighborhood points.

[0081] Furthermore, based on the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the time graph constructed by the training dataset, and the time lag constant, a predictability regularization term is designed, including:

[0082] The training data set is input into the encoding layer of the target-related autoencoder, and a latent feature space is obtained after transformation. Minimize the projected random variable in the latent feature space The sum of the variances of .

[0083] In one feasible implementation, each feature in the latent feature layer of the autoencoder should have a low variance compared to the features of the previous τ time steps. As the variance decreases, predictability and smoothness will improve. The original data is input into the encoding layer of the autoencoder and transformed by f θ Obtain a latent feature space Assume that x in the process data t There is a strong correlation with the data of the first τ time steps, and the parameters are learned in the latent space by designing constraints to maintain this autocorrelation across time. The goal of this invention is to and In other words, the proposed method aims to minimize the projected random variable in this space. The trace of the expected matrix of the covariance is:

[0084]

[0085] Where, The expectation operator that represents the covariance matrix between the current latent feature and the combined output of the latent features at time τ in the past.

[0086] Among them, the past process data x is measured by mutual information t-k For the current data x t The contribution of , the time lag constant τ is determined using the mRMR (Minimal Redundancy Maximal Relevance) criterion. After determining the time lag constant, for latent features with complex time dependencies, a probabilistic model is introduced to construct the autocorrelation of latent features across time:

[0087]

[0088] Where p(·|·) represents the conditional probability, represents the potential characteristics at the i-th moment, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is Normal distribution.

[0089] Furthermore, based on the above probability model, the predictability regularization term is designed using negative log-likelihood:

[0090]

[0091] Where, represents the predictability regularization term, N represents the number of samples, τ represents the time lag constant, express Subject to the mean μ i-1 , the variance is Normal distribution.

[0092] Furthermore, the designed predictability regularization term is added to the loss function of the target-related autoencoder, and the loss function of the supervised deep network with predictability embedding is

[0093] for:

[0094]

[0095] Where, The loss function for the supervised deep network that represents the embedding of the probabilistic latent predictability model, represents the loss function of the supervised deep network, represents the predictability regularization term, λ p represents the adjustable coefficient of the predictability regularization term.

[0096] Furthermore, the present invention designs a potential probability dynamic model and designs a regularization term guided by the original data graph structure based on the probability distribution difference.

[0097] Specifically, a probabilistic dynamic model is designed for the latent space of stacked target-dependent autoencoders to describe the complex temporal dependencies between latent features. For latent features, the construction of the latent probabilistic dynamic model is divided into four modules: prior, generation, recursion, and inference. Based on the ELBO (Evidence Lower Bound) of the VAE (Variational Autoencoder), the time factor is introduced into the latent probabilistic dynamics based on the VRNN (Variational Recurrent Neural Network). According to the factorization formula, the variational lower bound can be obtained. By minimizing the negative variational lower bound, the following loss function is designed:

[0098]

[0099] Where, the first term is the inference term, which is the approximate posterior distribution q(z t |H ≤t ,z ＜t ) and the conditional prior distribution p(z t |H ＜t ,z ＜t ), where D KL (·||·) represents the Kullback-Leibler divergence between the two distributions, Represents the loss function of VRNN, φ represents the mapping function of VRNN network, θ represents the parameters of VRNN, represents the expectation operator, z represents the random hidden state variable in VRNN, and T represents the length of the potential feature sequence; the second term is the logarithmic expectation of the generation process, and the expected calculation is approximated by sequential Monte Carlo sampling.

[0100] It must be emphasized that the goal is to maximize the evidence lower bound. Therefore, minimizing the negative ELBO will lead to the desired results. Therefore, the empirical loss function formula is as follows:

[0101]

[0102] in, zt is the hidden state of the variational recurrent network.

[0103] Furthermore, a regularization term guided by the original data graph structure is designed based on the probability distribution difference between the current potential feature and its neighborhood points.

[0104] First, we obtain the original data by augmenting Through the original data vector Its original data vector of τ time steps The probability distribution difference of the latent features is measured, the distance between the original data vectors of τ time steps is measured, and then the time graph of the original data is constructed. The graph-guided regularization term is designed so that the latent features reconstructed in the probabilistic dynamic model of the latent features can maintain the time graph structure in the original data as much as possible.

[0105] Secondly, the latent features are augmented to obtain Through the latent feature vector Its latent feature vector at τ time steps Construct a latent predictability model: The distance d(·,·) between the current potential feature and its neighboring features is expressed as the difference in probability distribution:

[0106]

[0107] Where p θAn edge probability function representing a latent feature.

[0108] The probability distribution difference between the current latent feature and its neighborhood features is represented by Kullback-Leibler (KL) divergence:

[0109]

[0110] where D KL is the KL divergence.

[0111] The regularization term guided by the graph structure is designed according to the distribution difference between the data at each time and the neighborhood data:

[0112]

[0113] S4, the target-related auto-encoder embedded with multiple probabilistic latent predictability models is stacked to build a supervised deep network embedded with probabilistic latent predictability models, the supervised deep network embedded with probabilistic latent predictability models is trained layer by layer to obtain a verified supervised deep network embedded with probabilistic latent predictability models.

[0114] In a feasible implementation, a target-related auto-encoder embedded with a single probabilistic latent predictability model is stacked to build a supervised deep network embedded with probabilistic latent predictability models, the built supervised deep network embedded with probabilistic latent predictability models is trained layer by layer, the regression network parameters are adjusted and updated according to the prediction error, and finally a verified supervised deep network embedded with probabilistic latent predictability models is obtained.

[0115] Further, the input data of the test set is input to the saved supervised deep network embedded with probabilistic latent predictability models, and the forward propagation of the supervised deep network embedded with probabilistic latent predictability models can obtain the prediction value of the industrial process quality variable.

[0116] S5, the process variables in the industrial production process are obtained and input to the verified supervised deep network embedded with probabilistic latent predictability models to obtain the prediction value of the quality variable of the industrial production process. It can be widely used in complex industrial processes such as de-butanization tower processes. In the process, seven measurable variables such as Figure 3 The blue square unit shown in the figure includes the tower top pressure, reflux flow, flow to the next process, temperature of the 6th tray, tower bottom temperature A and tower bottom temperature B as the input of soft measurement to obtain the prediction value of butane concentration, and the prediction result is as shown in Figure 4 .

[0117] To address the problem of industrial process soft measurement ignoring the dynamic autocorrelation of process variables, this paper proposes an industrial process soft measurement method based on probabilistic potential predictability embedded in a supervised deep network. The dynamic model of latent features is embedded in a deep learning framework and applied to industrial process soft measurement. Due to the complex dependencies between latent sequences, a graph-guided probabilistic dynamic model based on past and current latent features is designed for the latent space of the supervised deep network. Randomness is incorporated into the high-level latent space to model the potential predictability of the supervised deep network, and additional graph structure information is added to help analyze time dependencies, improving the ability to extract dynamic process features. The loss functions embedded in the supervised deep network and the potential predictability model are merged. By jointly training the model, the network parameters are optimized to ensure the overall convergence of the network. This addresses the problem that deep learning models in industrial process soft measurement models are based on static assumptions, ignore the dynamic autocorrelation of process data, and lose trend information in the process. A soft sensing method for industrial processes based on latent predictability embedding in deep learning networks is proposed. Latent features from previous time points are used to model potential temporal dependencies. A predictability regularization term is designed based on the loss of latent distribution prediction across the time dimension, so that the improved autoencoder can capture the dynamic autocorrelation of process variables.

[0118] To address the complex temporal dependencies and distributions of latent features, a variational recurrent neural network is used to construct an uncertain dynamic model of stochastic latent features. The stochastic latent states of latent features from past time steps are integrated into recurrent neurons. Furthermore, the relationship between past and current latent features is associated with an information-rich metric with graph-based predictability. Therefore, the predictability of latent features can be simplified to a graph structure to guide the capture of dynamic autocorrelation properties between latent features. This introduces the graph structure of the original data into the latent probabilistic dynamic model, allowing for the learning of interpretable temporal correlations between latent sequences.

[0119] To address the problem that traditional alternating training predictability loss and supervised deep network loss cannot guarantee the overall network convergence and are prone to falling into local minima, the variational recurrent network for modeling the predictability of potential features and the optimizer of the supervised deep network are merged to ensure the convergence characteristics of the network.

[0120] In the embodiment of the present invention, the influence of dynamic autocorrelation and trend information in process data on quality variables can be effectively mined, the time dependency of potential features can be modeled using a graph-guided potential probability dynamic model, the potential features from previous multiple time points are used to constrain the extraction of potential features at the current time point, and all optimizers are merged to train the parameters of the entire network, thereby improving the smoothness and predictability of the potential features, ensuring the convergence of the network, and improving the accuracy of soft measurement.

[0121] Figure 5 is a potential predictability embedded deep learning industrial process soft measurement device block diagram according to an exemplary embodiment, which is used for a potential predictability embedded deep learning industrial process soft measurement method. Figure 5 The device comprises an acquisition module 310, a selection module 320, a design module 330, a training module 340 and an output module 350.

[0122] The acquisition module 310 is used for acquiring industrial production process data, pre-processing the industrial production process data, and obtaining a training data set.

[0123] The selection module 320 is used for analyzing the time correlation of the training data set and selecting a time lag constant.

[0124] The design module 330 is used for designing a predictability regularization term according to a probability dynamic model of a latent feature constructed by a variational recurrent network, a time graph constructed by the training data set and the time lag constant, and constructing a target-related autoencoder embedded with a probability potential predictability model according to the predictability regularization term.

[0125] The training module 340 is used for stacking a plurality of target-related autoencoders embedded with probability potential predictability models, building a supervised deep network embedded with the probability potential predictability model, training the supervised deep network embedded with the probability potential predictability model layer by layer, and obtaining a verified supervised deep network embedded with the probability potential predictability model.

[0126] The output module 350 is used for acquiring a process variable in an industrial production process, inputting the process variable into the verified supervised deep network embedded with the probability potential predictability model, and obtaining a quality variable prediction value of the industrial production process.

[0127] In the embodiment of the present application, the influence of the dynamic autocorrelation and trend information in the process data on the quality variable can be effectively mined, the time dependence relationship of the latent feature is modeled by using the graph-guided latent probability dynamic model, the extraction of the latent feature at the current time point is constrained by using the latent feature from the previous multiple time points, and all optimizers are combined to train the parameters of the entire network, thereby improving the smoothness and predictability of the latent feature, ensuring the convergence of the network, and improving the precision of the soft measurement.

[0128] Figure 6 is a structural schematic diagram of an industrial process soft measurement device provided by the embodiment of the present application, as shown in Figure 6 The industrial process soft measurement device can comprise the potential predictability embedded deep learning industrial process soft measurement device shown in Figure 5 Optionally, the industrial process soft measurement device 410 can comprise a first processor 2001.

[0129] Optionally, the industrial process soft-sensing device 410 can further include a memory 2002 and a transceiver 2003.

[0130] The first processor 2001 is connected with the memory 2002 and the transceiver 2003, for example, through a communication bus.

[0131] The following will be described in detail Figure 6 The various components of the industrial process soft-sensing device 410 will be described in detail as follows:

[0132] The first processor 2001 is the control center of the industrial process soft-sensing device 410, which can be one processor or a plurality of processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present application, such as one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0133] Optionally, the first processor 2001 can execute various functions of the industrial process soft-sensing device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0134] In a specific implementation, as an embodiment, the first processor 2001 can include one or more CPUs, such as the CPU0 and the CPU1 shown in FIG. 2. Figure 6

[0135] In a specific implementation, as an embodiment, the industrial process soft-sensing device 410 can also include a plurality of processors, such as the first processor 2001 and the second processor 2004 shown in FIG. 2. Each of these processors can be a single-CPU or a multi-CPU. The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). Figure 6

[0136] The memory 2002 is used to store software programs for implementing the schemes of the present application, and is controlled by the first processor 2001 to execute. The specific implementation manner can refer to the above method embodiments, and will not be described here again.​​

[0137] Optionally, the memory 2002 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this. The memory 2002 can be integrated with the first processor 2001 or exist independently and be coupled to the first processor 2001 through an interface circuit (not shown in the figure) of the industrial process soft measurement device 410, and the embodiments of the present application do not make specific limitations here. Figure 6

[0138] The transceiver 2003 is configured to communicate with a network device or a terminal device.

[0139] Optionally, the transceiver 2003 can include a receiver and a transmitter (not shown separately in the figure). The receiver is configured to implement a receiving function, and the transmitter is configured to implement a transmitting function. Figure 6

[0140] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently and be coupled to the first processor 2001 through an interface circuit (not shown in the figure) of the industrial process soft measurement device 410, and the embodiments of the present application do not make specific limitations here. Figure 6

[0141] It should be noted that the structure of the industrial process soft measurement device 410 shown in the figure does not constitute a limitation on the router, and the actual knowledge structure recognition device can include more or fewer components than those shown in the figure, or combine certain components, or different component arrangements. Figure 6 In addition, the technical effects of the industrial process soft measurement device 410 can refer to the technical effects of the industrial process soft measurement method with potential predictability embedded deep learning described in the above method embodiments, which will not be repeated here.

[0142]

[0143] ​​​​It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0144] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0145] The above embodiments can be implemented in whole or in part through software, hardware (such as circuits), firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired method (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, or magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0146] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0147] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0148] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0149] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0150] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0151] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0152] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0153] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0154] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0155] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A soft sensing method for industrial processes with potential predictability embedded in deep learning, characterized by: The method comprises: S1. Acquire industrial production process data, and preprocess the industrial production process data to obtain a training data set; S2. Analyze the time correlation of the training data set and select a time lag constant; S3, designing a predictability regularization term based on the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the time graph constructed by the training data set, and the time lag constant, and constructing a target-dependent autoencoder for embedding the probabilistic latent predictability model based on the predictability regularization term; S4, stacking a plurality of target-related autoencoders embedded with the probabilistic potential predictability model to build a supervised deep network embedded with the probabilistic potential predictability model, and training the supervised deep network embedded with the probabilistic potential predictability model layer by layer to obtain a verified supervised deep network embedded with the probabilistic potential predictability model; S5. Obtain process variables in the industrial production process and input them into the supervised deep network embedded with the verified probabilistic potential predictability model to obtain predicted values ​​of quality variables in the industrial production process; The predictability regularization term is designed based on the probabilistic dynamic model of the potential features constructed according to the variational recurrent network, the time graph constructed by the training data set, and the time lag constant in S3, including: The training data set is input into the encoding layer of the target-related autoencoder, and a latent feature space is obtained after transformation. , minimize the projected random variable in the latent feature space The sum of the variances of Among them, based on the time graph constructed from the training dataset and the probabilistic dynamic model of the latent features constructed by the variational recurrent network, the autocorrelation of the latent features across time is constructed, and the predictability regularization term is designed using the negative log-likelihood; Adding the predictability regularization term to the loss function of the target-dependent autoencoder to obtain the loss function of the supervised deep network embedded with the probabilistic latent predictability model; The time graph is used to guide the regularization term so that the latent features reconstructed in the probabilistic dynamic model of the latent features can maintain the time graph structure in the original data as much as possible; The training dataset in S3 constructs a time graph, including: Augment the training data set to obtain augmented data , according to the original data vector in the augmented data with it The original data vector of time steps The probability distribution difference of The distance between the original data vectors of time steps is used to construct the time graph; among them, represents the time lag constant.

2. The industrial process soft sensing method based on potential predictability embedded in deep learning according to claim 1 is characterized in that: The autocorrelation of the latent features across time is shown in the following formula (1): （1） Where, represents the conditional probability, Indicates the i The potential characteristics of the moment, represents the time lag constant, express Obey the mean , the variance is Normal distribution.

3. The industrial process soft sensing method based on potential predictability embedded in deep learning according to claim 1 is characterized in that: The predictability regularization term is shown in the following formula (2): （2） Where, represents the predictability regularization term, represents the number of samples, represents the time lag constant, express Obey the mean , the variance is Normal distribution.

4. The industrial process soft sensing method based on potential predictability embedded in deep learning according to claim 1 is characterized in that: The loss function of the supervised deep network embedded in the probabilistic potential predictability model is shown in the following formula (3): （3） Where, The loss function for the supervised deep network that represents the embedding of the probabilistic latent predictability model, represents the loss function of the supervised deep network, represents the predictability regularization term, represents the adjustable coefficient of the predictability regularization term.

5. The industrial process soft sensing method based on potential predictability embedded in deep learning according to claim 1 is characterized in that: a probabilistic dynamic model of the latent features, for describing the temporal dependencies between the latent features in the latent space of the stacked target-dependent autoencoders; The probabilistic dynamic model of potential features constructed according to the variational recurrent network in S3 includes: A probabilistic dynamic model of latent features is constructed based on a variational recurrent network. Based on the evidence lower bound of the variational autoencoder, the time factor is introduced into the probabilistic dynamic model of latent features. According to the factorization formula, the variational lower bound is obtained. The goal of the probabilistic dynamic model of the latent features is set to maximize the evidence lower bound, and the empirical loss function is obtained.

6. A soft-sensing device for an industrial process with potential predictability embedded in deep learning, wherein the soft-sensing device for an industrial process with potential predictability embedded in deep learning is used to implement the soft-sensing method for an industrial process with potential predictability embedded in deep learning as described in any one of claims 1 to 5, characterized in that: The device comprises: An acquisition module is used to acquire industrial production process data, pre-process the industrial production process data, and obtain a training data set; A selection module, configured to analyze the time correlation of the training data set and select a time lag constant; Designing a module for designing a predictability regularization term based on a probabilistic dynamic model of latent features constructed by a variational recurrent network, a time graph constructed from a training dataset, and the time lag constant, and constructing a target-dependent autoencoder for embedding a probabilistic latent predictability model based on the predictability regularization term; A training module is used to stack a plurality of target-related autoencoders embedded with the probabilistic potential predictability model, build a supervised deep network embedded with the probabilistic potential predictability model, and train the supervised deep network embedded with the probabilistic potential predictability model layer by layer to obtain a verified supervised deep network embedded with the probabilistic potential predictability model; The output module is used to obtain process variables in the industrial production process and input them into the supervised deep network embedded with the verified probabilistic potential predictability model to obtain the predicted values ​​of the quality variables of the industrial production process.

7. An industrial process soft measurement device, characterized in that: The industrial process soft measurement device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 5.

Images