Latent Variable Gaussian Process Soft Sensor Modeling Method for Integrating Qualitative and Quantitative Information

Abstract

Latent Variable Gaussian Process Soft Sensor Modeling Method for Integrating Qualitative and Quantitative Information, the method comprising the following steps: 1) Obtain polypropylene data; 2) Determine the training set and test set of polypropylene data and the preprocessing work; 3) Integrate quantitative factors and qualitative factors to construct new process variables; 4) Learn effective latent variables; 5) Establish a soft sensor model based on latent variables; 6) Evaluate the model performance. The present invention adopts a latent variable Gaussian process soft sensor model for integrating qualitative and quantitative information, solving the problem of difficulty in establishing a reliable soft sensor model for "small samples" with high-dimensional data characteristics. Compared with traditional soft sensor modeling methods, the proposed method can greatly improve the prediction performance of the model and has universality and generality.

Description

Technical Field

[0001] The present invention relates to the situation of "small samples" with high-dimensional data characteristics in the soft sensor modeling process, and particularly relates to a latent variable Gaussian process soft sensor modeling method that integrates qualitative and quantitative information. Background Art

[0002] In recent years, with the rapid development of communication technology and storage technology, a large amount of industrial process data has been collected and stored. In the big data environment, researchers are committed to the research on the construction and training methods of soft sensor models. However, in many scenarios, the problem of "small samples" is particularly prominent, but it has received little attention, such as high-value-added industrial processes such as pharmaceutical production and special chemical preparation. In order to meet different product requirements, these processes need to frequently change operating conditions, resulting in a limited number of products produced under each condition. However, for data-driven process modeling, the reliability of the model highly depends on the quality of the training samples. It is difficult to build a model between process variables and key quality variables with only a small number of samples.

[0003] A typical "small sample" process is the extrusion process of an extruder. The screw is an important part of the extruder, and solid particles complete the complete material transfer process of conveying, melting, and melt conveying through the screw. The performance of the extruder mainly depends on the system operating conditions and the screw configuration. According to various types of screw elements provided by the manufacturer, different screw configurations can be obtained. Therefore, selecting the appropriate screw elements and determining their positions on the screw shaft are crucial for product production. In recent years, due to advantages such as low cost and easy operation, data-driven models based on statistical characteristics and machine learning have been used to replace the computer simulation process to determine the screw element configuration. In order to meet different product requirements, the samples collected under different screw configurations are limited. Therefore, how to establish a soft sensor model of the extruder under small sample conditions is worthy of research.

[0004] To solve the problem of unreliable models due to insufficient sample size, a feasible method is to expand the sample size by collecting limited samples under different tasks. This method is similar to multi-task learning, that is, sharing useful information under similar tasks to model various tasks simultaneously. Considering the influence of the screw element configuration on soft sensor modeling, qualitative information is quantified by One-hot encoding and fused with quantitative factors to achieve the prediction of key quality variables in the extruder process. However, in the actual process, the diverse types of screw elements make it difficult to build a model that integrates quantitative and qualitative factors. Because the high-dimensional qualitative factors will overwhelm the role of quantitative factors in model construction. At the same time, the high-dimensional variables may contain redundant information, causing interference to model construction. Therefore, it is necessary to extract effective low-dimensional hidden layer feature representations from high-dimensional variables before building a soft sensor model. Summary of the Invention

[0005] To solve the problem that it is difficult to establish a reliable soft sensor model for "small samples" with high-dimensional data characteristics in industrial processes, the present invention proposes a latent variable Gaussian process soft sensor modeling method that integrates qualitative and quantitative information; by learning the effective features of high-dimensional variables that integrate qualitative information and quantitative information, the influence of redundant information on model construction is removed; it is beneficial to construct a reliable soft sensor model between latent variables and key quality variables to achieve the prediction of key quality variables of test samples.

[0006] The technical solution adopted by the present invention to solve its technical problems is:

[0007] A latent variable Gaussian process soft sensor modeling method that integrates qualitative and quantitative information, the method includes the following steps:

[0008] 1) Obtain polypropylene data

[0009] The production and processing process of polypropylene is completed in a twin-screw extruder. In this method, Ludovic simulation software is used to simulate the flow process of polypropylene in the flow channel of a co-rotating twin-screw extruder and generate data for process modeling. The flow situation of polypropylene in the extruder is related to two important quality variables, namely the material outlet temperature and the residence time. Since it is difficult to measure the material outlet temperature and the residence time in real time, three auxiliary variables closely related to the polypropylene production process are selected to construct a soft sensor model, which includes two quantitative factors: the feed flow rate, the screw speed and one qualitative factor, the screw configuration.

[0010] 2) Determine the training set and test set of polypropylene data and the preprocessing work

[0011] First, the collected polypropylene data is divided into a training set and a test set. Then, in order to assign weights to quantitative factors and qualitative factors during the modeling process, the quantitative factors and qualitative factors are preprocessed respectively.

[0012] 3) Integrate quantitative factors and qualitative factors to construct new process variables

[0013] Considering both quantitative factors and qualitative factors that affect the key quality variables of the polypropylene process, the quantitative factors and qualitative factors are integrated to construct new high-dimensional process variables.

[0014] 4) Learn effective latent variables

[0015] Based on the newly formed high-dimensional process variables, through the Gaussian process latent variable model (Gaussian process latent variable model, GPLVM), learn effective hidden layer feature representations to remove the influence of redundant information on the model.

[0016] 5) Establish a soft sensor model based on latent variables

[0017] Build a soft sensor model between the latent variable and the key quality variable to realize the prediction of the key quality variable of the test sample.

[0018] 6) Model performance evaluation

[0019] To evaluate the method proposed in the present invention more objectively, the evaluation index root mean square error (RMSE) is introduced.

[0020] Furthermore, the process of step 2) is as follows:

[0021] Step 2.1: Division of training set and test set

[0022] In the mixed polypropylene extrusion process, different screw elements are used as candidate elements for the extruder configuration. From the total number of screw configurations, m configurations are randomly selected. Under different configurations, process input and output data are collected under different operating conditions; for each operating condition, the Latin hypercube sampling method is used to determine the collected samples, and the samples are divided into a training set and a test set.

[0023] Step 2.2: Data preprocessing

[0024] For the quantitative factors of feed flow rate and screw speed, the maximum-minimum normalization method is used for preprocessing, and its formula is expressed as:

[0025]

[0026] where p is the data after normalization; a is the collected original data; a min is the minimum value in the original data; a max is the maximum value in the original data. The variables of feed flow rate and screw speed after maximum-minimum normalization are respectively expressed as p1 and p2.

[0027] For the qualitative factor of screw configuration, one-hot encoding is used for preprocessing. The first candidate screw configuration is expressed as [1, 0, 0, …, 0, 0], the second candidate screw configuration is expressed as [0, 1, 0, …, 0, 0], and so on.

[0028] Furthermore, the process of step (3) is as follows:

[0029] Fuse the quantitative factor and the qualitative factor to form a new input variable (e). Here, the symbol represents the concatenation operation of vectors. Therefore,

[0030]

[0031] Furthermore, the process of step (4) is as follows:

[0032] Step 4.1: Learn the low-dimensional latent representation of the high-dimensional process variable e

[0033] The GPLVM model has superior performance in data dimensionality reduction and feature learning. GPLVM reduces the dimensionality of high-dimensional data X by learning the latent variable Z in the low-dimensional space. Specifically, for the given N D-dimensional observation samples perform dimensionality reduction to obtain the effective representation of these samples in the low-dimensional space, denoted as where Q represents the dimension after dimensionality reduction, and Q << D. The relationship between the latent variable and the observed variable can be expressed as follows:

[0034] x n = f g (z n ) + ε n

[0035] where f g (·) represents a non-linear function that follows a Gaussian process prior, satisfying f g (·) ~ GP(0, K). ε n is Gaussian noise, satisfying ε n ~ N(0, σ 2 ). Through derivation, the specific form of the GPLVM marginal likelihood p(X|Z, θ) can be obtained:

[0036]

[0037] where θ represents the hyperparameters in the kernel function and the kernel noise distribution. x :,d represents the d-th column element of matrix X. K represents the covariance. σ 2 represents the variance. I is the identity matrix. During the learning process, the optimal latent variable and hyperparameters

[0038]

[0039] Step 4.2: Determine the kernel function

[0040] GPLVM adopts a composite kernel function, the squared exponential autocorrelation kernel function, and its expression is as follows:

[0041]

[0042] where x represents the input variable, x' represents the target variable, σ SE is the hyperparameter that determines the amplitude of the kernel function, H represents the number of input variables, λ hIndicates the correlation between the input variable and the target variable. The smaller the value, the greater the correlation.

[0043] Further, the process of step (5) is as follows:

[0044] A Gaussian process regression (GPR) model is established to predict the key quality variables in the polypropylene process. GPR and GPLVM belong to the Gaussian process framework. The GPR method is used to train the model based on the latent variables and key quality variables, and then the prediction of the key quality variables in the test set is realized.

[0045] Further, the process of step (6) is as follows:

[0046] The root mean square error is defined as follows:

[0047]

[0048] In the formula: represents the predicted value of the test data y i , and r is the total number of samples in the test set. The smaller the RMSE, the better the prediction performance of the regression model.

[0049] The beneficial effects of the present invention are mainly manifested in:

[0050] The present invention proposes a latent variable Gaussian process soft sensor modeling method that integrates qualitative and quantitative information. By learning the effective latent representation in the low dimension of the high-dimensional process variables that fuse quantitative and qualitative factors, the influence of redundant features on modeling is eliminated, which is beneficial to solving the problem of difficult establishment of a reliable soft sensor model for "small samples" with high-dimensional data characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is the network flow chart of the method of the present invention;

[0052] Figure 2 is the box plot of the prediction result of the residence time by the method of the present invention;

[0053] Figure 3 is the box plot of the prediction result of the material outlet temperature by the method of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0054] The present invention will be further described below with reference to the drawings.

[0055] Referring to Figures 1 to 3 , a latent variable Gaussian process soft sensor modeling method that integrates qualitative and quantitative information, the method includes the following steps:

[0056] (1) Obtain polypropylene data

[0057] The production and processing of polypropylene is completed in a twin-screw extruder. In this paper, Ludovic simulation software is used to simulate the flow process of polypropylene in the flow channel of a co-rotating twin-screw extruder and generate data for process modeling. The flow of polypropylene in the extruder is related to two important quality variables, namely the material outlet temperature and the residence time. Since it is difficult to measure the material outlet temperature and the residence time in real time, three auxiliary variables closely related to the polypropylene production process are selected to construct a soft measurement model. The auxiliary variables include two quantitative factors: the feed flow rate, the screw speed, and a qualitative factor, namely the screw configuration.

[0058] (2) Determine the training set, test set of polypropylene data and the preprocessing work

[0059] First, the collected polypropylene data is divided into a training set and a test set. Then, in order to assign weights to the quantitative factors and qualitative factors during the modeling process, the quantitative factors and qualitative factors are preprocessed separately.

[0060] Step 2.1: Division of the training set and the test set

[0061] For the mixed polypropylene extrusion process, a total of 30 different screw elements are used as candidate elements for the extruder configuration, so the total number of screw configurations is 30 3 = 27000. Randomly select m configurations from the whole pool, where m << 27000. Under different configurations, process input and output data are collected from four different operating conditions. For each operating condition, the Latin hypercube sampling method is used to determine the collected samples, and the samples satisfy the interval: the feed flow rate [50, 80] kg / h and the screw speed [200, 300] rpm. Therefore, the sample size for model training is M = 4m. In addition, a test data set containing 400 observations is used to evaluate the performance of different methods.

[0062] Step 2.2: Data preprocessing

[0063] For the quantitative factors of the feed flow rate and the screw speed, the maximum-minimum normalization method is used for preprocessing, and its formula is expressed as:

[0064]

[0065] In the formula, p is the data after normalization; a is the collected original data; a min is the minimum value in the original data; a max is the maximum value in the original data. The variables of the feed flow rate and the screw speed after maximum-minimum normalization are respectively expressed as p1 and p2.

[0066] For the qualitative factor screw configuration, one-hot encoding is used for preprocessing. Since the number of screw elements is 30, the length of each one-hot vector is 30. The first candidate screw configuration is represented as [1, 0, 0, …, 0, 0], the second candidate screw configuration is represented as [0, 1, 0, …, 0, 0], and so on. The preprocessed screw configuration is represented as {q1, q2, ..., q 30}.

[0067] (3) Integrate quantitative factors and qualitative factors to construct a new process variable

[0068] While considering both the quantitative factors and qualitative factors that affect the key quality variables of the polypropylene process, integrate the quantitative factors and qualitative factors to construct a new high-dimensional process variable (e). Here, the symbol represents the concatenation operation of vectors. The formula is as follows:

[0069]

[0070] (4) Learn effective latent variables

[0071] Based on the newly constructed high-dimensional process variable, learn effective hidden layer feature representations through the Gaussian process latent variable model (GPLVM) to remove the influence of redundant information on the model.

[0072] Step 4.1: Learn the low-dimensional latent representation of the high-dimensional process variable e

[0073] The GPLVM model has excellent performance in data dimensionality reduction and feature learning. GPLVM reduces the dimensionality of the high-dimensional data X by learning the latent variable Z in the low-dimensional space. Specifically, for the given N D-dimensional observation samples perform dimensionality reduction to obtain the effective representation of these samples in the low-dimensional space, denoted as where Q represents the dimension after dimensionality reduction, Q << D. The relationship between the latent variable and the observation variable can be expressed as follows:

[0074] x n = f g (z n ) + ε n

[0075] where f g (·) represents a non-linear function that follows a Gaussian process prior, satisfying f g (·) ~ GP(0, K). ε n is Gaussian noise, satisfying ε n ~ N(0, σ 2)。By derivation, the specific form of the marginal likelihood p(X|Z,θ) of GPLVM can be obtained:

[0076]

[0077] where θ represents the hyperparameters in the kernel function and the noise distribution. x :,d represents the d-th column element of matrix X. K represents the covariance. σ 2 represents the variance. I is the identity matrix. During the learning process, the optimal latent variables and hyperparameters

[0078]

[0079] Step 4.2: Determine the kernel function

[0080] GPLVM adopts the composite kernel function squared exponential autocorrelation kernel function, and its expression is as follows:

[0081]

[0082] where x represents the input variable, x′ represents the target variable, σ SE is the hyperparameter that determines the amplitude of the kernel function, H represents the number of input variables, λ h represents the correlation between the input variable and the target variable, and the smaller the value, the greater the correlation.

[0083] (5) Establish a soft sensor model based on latent variables

[0084] Establish a Gaussian process regression (GPR) model to predict the key quality variables in the polypropylene process. GPR and GPLVM belong to the Gaussian process framework. The GPR method is used to train the model based on the latent variables and the key quality variables, and then the prediction of the key quality variables in the test set is realized. The overall flowchart of the LVGPR method is as Figure 1 shown.

[0085] (6) Model performance evaluation

[0086] The root mean square error is defined as follows:

[0087]

[0088] In the formula: represents the predicted value of the test data y i , and r is the total number of samples in the test set. The smaller the RMSE, the better the prediction performance of the regression model.

[0089] The proposed LVGPR method is compared with the GPR method without dimensionality reduction and the PCA-GPR method using PCA for dimensionality reduction. All three models are established based on the hybrid process variables that integrate quantitative and qualitative factors. We study the performance of the proposed method when the number of screw elements \(m = 300\), and at this time, the number scale of the training set \(M = 1200\). To reduce the contingency of the results in a single experiment, the training and testing processes of the model are repeated 100 times. The average value of 100 experiments is used as the final prediction result. The prediction results of the three methods, GPR, PCA-GPR, and LVGPR, on the test set are recorded in Table 1. At the same time, in Figure 2 and Figure 3 the box plots of the RMSE values of the residence time and the material outlet temperature are shown. It can be seen from the table and the figure that for the prediction of the two key quality variables, the RMSE of the LVGPR method is smaller than that of the GPR and PCA-GPR methods, indicating that the performance of the LVGPR method is superior to that of the GPR and PCA-GPR methods.

[0090] Table 1

[0091]

[0092] The method of the present invention adopts a latent variable Gaussian process soft sensor model that integrates qualitative and quantitative information, and solves the problem of difficulty in establishing a reliable soft sensor model for "small samples" with high-dimensional data characteristics. Compared with the traditional soft sensor modeling method, the proposed method can greatly improve the prediction performance of the model, and has universality and generality.

[0093] The content described in the embodiments of this specification is only an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms stated in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art according to the inventive concept.

Claims

1. A latent variable Gaussian process soft sensor modeling method that fuses qualitative and quantitative information, characterized in that, It includes the following steps: 1) Obtain polypropylene data: Select the feed flow rate, screw speed, and screw configuration as the data for process modeling. The feed flow rate and screw speed are quantitative factors, and the screw configuration is a qualitative factor; 2) Divide the obtained polypropylene data into a training set and a test set, and preprocess the quantitative factors and qualitative factors respectively. The process of step 2) is as follows: Step 2.1: Division of the training set and the test set In the mixed polypropylene extrusion process, different screw elements are used as candidate elements for the extruder configuration. From the total number of screw configurations, m configurations are randomly selected. Under different configurations, process input and output data are collected under different operating conditions; for each operating condition, the Latin hypercube sampling method is used to determine the collected samples, and the samples are divided into a training set and a test set. Step 2.2: Data preprocessing For the quantitative factors of feed flow rate and screw speed, the maximum-minimum normalization method is used for preprocessing, and its formula is expressed as: ； In the formula, is the data after normalization processing; is the original data collected; is the minimum value in the original data; is the maximum value in the original data; The variables after the maximum-minimum normalization processing of the feed flow rate and the screw speed are respectively expressed as and ; For the qualitative factor of screw configuration, one-hot encoding is used for preprocessing; The first candidate screw configuration is expressed as [1, 0, 0, …, 0, 0], the second candidate screw configuration is expressed as [0, 1, 0, …, 0, 0], and so on; 3) Integrate quantitative factors and qualitative factors to construct high-dimensional process variables. The process of step 3) is as follows: Integrate quantitative factors with qualitative factors to form new input variables ( ); Use the symbol ⊕ to represent the concatenation operation of vectors, and the formula is as follows: ； 4) Based on the constructed high-dimensional process variables, learn effective latent variables through the Gaussian process latent variable GPLVM model to remove the influence of redundant information on the model 5) Establish a soft sensor model based on latent variables: Construct a soft sensor model between the latent variables and the key quality variables to achieve the prediction of the key quality variables of the test samples; 6) Model performance evaluation.

2. The latent variable Gaussian process soft sensor modeling method that fuses qualitative and quantitative information according to claim 1, characterized in that, The process of step 4) is as follows: Step 4.1: Learn the low-dimensional latent representation of the high-dimensional process variables : The GPLVM model realizes dimensionality reduction of high-dimensional data by learning latent variables in a low-dimensional space The specific process is as follows: for high-dimensional data For a given N number of D dimensional observation samples perform dimensionality reduction Obtain the effective representation of these samples in the low-dimensional space, denoted as: ； Among them, represents the dimension after dimensionality reduction, ; the relationship between the latent variable and the observed variable is expressed as follows: ； wherein, represents a non-linear function subject to a Gaussian process prior, satisfying ; is Gaussian noise, satisfying ; The specific form of the GPLVM marginal likelihood is as follows: ； Among them, represents the hyperparameter in the kernel noise distribution of the kernel function; represents the matrix the th column element; represents the covariance; represents the variance; is the identity matrix; During the learning process, the optimal latent variable and hyperparameter are obtained by maximizing the marginal likelihood of GPLVM: ； Step 4.2: Determine the kernel function The GPLVM model uses a composite kernel function, the squared exponential autocorrelation kernel function, and its expression is as follows: ； Among them, represents the input variable, represents the target variable, is a hyperparameter that determines the amplitude of the kernel function, H represents the number of input variables, represents the correlation between the input variable and the target variable. The smaller the value, the greater the correlation.

Images