A soft measurement modeling method based on integrated neural network credibility estimation and weighted learning

Abstract

The application discloses a soft measurement modeling method based on integrated neural network and weighted learning and applied to industrial soft measurement of a urea synthesis process, which comprises the following steps: firstly, weak label data learning is carried out based on a neural network set, a noise data fitting neural network set is trained and constructed, label prediction is carried out on a label noise verification set, and kernel density estimation is carried out by using a prediction value set of the model set; then, the probability density value of the original label in a KDE fitting probability density diagram is calculated, the probability density ratio of the peak value and the original label with noise is calculated, and thus the credibility weight of the data label is obtained; the verification set data is used as training data of a downstream student model, the data label credibility predicted by the upstream model is input into the student model, and the student model is trained; and finally, when the student model is tested, data directly flows into the downstream network, the input of the label credibility value in the feature dimension of the model is all set to 1, and thus the soft measurement prediction of the denoised key quality index of the industrial process is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of industrial process control, and particularly relates to a soft measurement modeling method based on integrated neural network credibility estimation and weighted learning for N / C value prediction in a urea synthesis process. BACKGROUND

[0002] In industrial production processes, due to the complexity of process environment and production conditions, many key variables reflecting production quality cannot be measured in real time by industrial instruments. The data of these quality variables are usually obtained by laboratory analysis or online analyzers, which are limited by large delay or high cost. In recent years, with the progress of data science and information technology, data-driven soft measurement models have been widely used in process industries, mainly based on historical process data and statistical machine learning methods to build input-output models to solve the process data modeling problems with characteristics of nonlinearity, noise, multimodality and multisampling rate, etc.

[0003] Traditional machine learning models need a pre-assumption when performing data analysis, that is, all data are identified as accurate and true data. However, this is difficult to apply in industrial sensor data, especially when some sensors have concentration changes and zero drift due to the immersion of sensing metal in solution, resulting in inaccurate sensor data. However, traditional machine learning models have the advantages of simple model, small number of parameters, timeliness, etc. In prediction, compared with large models, because of the simple model structure, it is suitable for integration in hardware, has more universality, and has low requirements for hardware during operation.

[0004] Neural network models have good fitting performance. When the model complexity increases, at a certain depth, a certain number of neurons can theoretically adapt to any function. When the noise distribution is unknown but fixed, through sufficient scattered data for network training, the trained model can fit the data label under the influence of noise. Through different scattered data sets, the neural networks trained by different scattered data sets can predict the same feature through a model architecture similar to a forest structure. When the model is sufficient and the data volume is sufficient, the prediction distribution of a certain label will be similar to the distribution of noise.

[0005] The credibility model is established by using historical sensing samples to output credibility to a downstream prediction model. The core problem of the credibility model is to find a suitable method to evaluate the credibility criteria and the model can learn the noise distribution. At present, the processing method for traditional noisy label data usually uses filtering or outlier screening method for pretreatment, which will cause data loss, while the frequency of outliers in industrial process can also reflect the overall distribution of noise, and outliers also contain part of the expected industrial data information. Therefore, it is necessary to invent a label data denoising soft measurement modeling method of credibility weighted learning to learn noisy data information. SUMMARY

[0006] The present application aims at the hypothesis limitation of the existing machine learning method, and provides a soft measurement modeling method based on integrated neural network credibility estimation and weighted learning.

[0007] The purpose of the present application is achieved by the following technical solutions:

[0008] A soft measurement modeling method based on integrated neural network credibility estimation and weighted learning, comprising the following steps:

[0009] (1) judging the noise type of the industrial label data that may be noisy, and judging whether it is unbiased noise.

[0010] (2) collecting sample set and test data for modeling by using industrial control system, both of which are labeled data, the sample set is and the test data set is Wherein n1=1,...,N l , n2=1,..., t , l represents labeled data, N l is the number of sample training set data, t represents the test data set, N t is the number of test data, τ is the variable dimension, R is the real number set, R 1×τ represents that X satisfies the two-dimensional distribution of 1xτ. Store these data in the model database.

[0011] (3) randomly scatter the training sample data set into training source set Where o is the source training data after shuffling, and the source data set is divided into sub-model (upstream model) training set and downstream model training set Where st refers to the sub-model training set, and dmt refers to the downstream model training set.

[0012] (4) Set up the INNs network based on the open-source deep learning platform PyTorch, and set the number of tree models tn, the number of layers of a single tree model l, the number of hidden layer nodes m, the model training learning rate p1, the model training optimizer Adam, the training batch data size Bs, the training cycle number ep1, and the jump-out training threshold tv.

[0013] (5) Based on the training set, the INNs network model is trained, the downstream model training set is predicted, and a predicted label set is obtained Where pdmt is the predicted label set of the downstream model training set by INNs.

[0014] (6) According to the obtained predicted label set KDE is performed to obtain a probability density curve or a histogram to obtain a probability distribution, and the original label is put into the fitted curve to calculate the probability density value at this time, and the label confidence is calculated.

[0015] (7) The label confidence of the downstream model training set is spliced with the original downstream model training set features as new input features to input into the downstream regression prediction model, taking NNs as an example, setting the number of hidden layers dsl, the number of hidden layer nodes p, and the number of labeled data N L selected for constructing a local model. In addition, the model training learning rate p2, the training batch data size BS, and the training cycle number ep2 are set. And the downstream model training set label is weighted.

[0016] (8) When the model is evaluated and applied, the upstream model is removed, and the trained downstream regression model is directly used for prediction, wherein the input data is stacked on the basis of the original features with a dimension feature of confidence of 1.

[0017] (9) The root mean square error (RMSE) of the model evaluation index is used to evaluate the prediction results of the test set. At the same time, it is compared with the prediction of the traditional downstream model without removing the upstream model.

[0018] The beneficial effects of the present application are: the present application can evaluate the label confidence of the industrial process noisy label data by constructing a neural network kernel density model set with a forest structure. Then, the feature is introduced into the weighted deep learning model, and a soft measurement model based on the confidence training is constructed. Compared with directly using the downstream soft measurement model, the present application can directly use the label confidence feature from the noisy training set, which can not only be suitable for the prediction accuracy of the industrial soft measurement model based on only the sensor data, but also can reduce the data cost of the training model. Compared with the traditional soft measurement model, the present application can more accurately reflect the process production state. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1is a schematic diagram of an integrated neural network credibility evaluation network;

[0020] Figure 2 is a simple schematic diagram of a urea synthesis process;

[0021] Figure 3 is a graph of the prediction results of the MLPINNsWoR model on the first 50 groups of N / C data;

[0022] Figure 4 is a graph of the single-point prediction RMSE results of the MLPINNsWoR model on the first 50 groups of N / C data. DETAILED DESCRIPTION

[0023] The present application is directed to the problem of industrial process soft measurement modeling, and a soft measurement modeling method based on integrated neural network credibility estimation and weighted learning is proposed by integrating data label credibility features into original input features for model training. A multi-neural network model with a forest structure is used to generate a series of neural network sub-models through learning of original data. The sub-model set is used to predict the training set of the downstream model, and the kernel density estimation is performed on the predicted label set through KDE. The label credibility is obtained through the corresponding credibility calculation formula, so as to obtain the credibility of each label of the training set of the downstream model. The model is weighted and trained with the training set of the downstream model and the corresponding label credibility. Further, a label data denoising soft measurement model based on integrated neural network probability density ratio credibility estimation and weighted learning is proposed to solve the problem of soft measurement modeling in a noisy label data set by using traditional machine learning. The present application can directly obtain the label credibility features from the noisy training set. Not only can it be applied to the prediction accuracy of the industrial soft measurement model based on sensing data, but also can reduce the data cost of the training model. Compared with the traditional soft measurement model, the present application can more accurately reflect the process production state, and greatly improve the prediction accuracy of the traditional machine learning soft measurement model.

[0024] The main steps of the technical solution adopted by the present application are as follows:

[0025] In the first step, the noise situation is understood through experience and on-site exploration, and the causes, influence and deviation of the data are preliminarily judged.

[0026] In the second step, the sample set and the test data for modeling are collected by the industrial control system, which are both labeled data. The sample set is and the test data set is where n1=1,...,N l , n2=1,..., t , l represents labeled data, N l is the number of sample training set data, and t represents the test data set, N tis the number of test data, τ is the dimension of variable, R is the real number set, R 1×τ represents that X satisfies the two-dimensional distribution of 1 x τ. These data are stored in the model database.

[0027] The third step is to randomly scatter the training sample data set into training source sets where o is the source training data after shuffling, and the source data set is divided into sub-model training set and downstream model training set where st refers to the sub-model training set, and dmt refers to the downstream model training set.

[0028] The fourth step is to build an INNs network based on the open source deep learning platform PyTorch, and set the number of tree models tn, the number of layers of a single tree model l, the number of hidden layer nodes m, the model training learning rate ρ1, the model training optimizer Adam, the training batch data size Bs, the training cycle number ep1, and the jump-out training threshold tv.

[0029] The fifth step is to train the INNs network model based on the training set, predict the downstream model training set, and obtain the predicted label set where pdmt is the predicted label set of the downstream model training set by INNs.

[0030] The sixth step is to obtain the probability density curve or histogram statistics probability distribution according to the obtained predicted label set , and put the original label into the fitted curve to calculate the probability density value at this time, and calculate the label reliability. The specific steps are as follows:

[0031] 1) The label in the predicted label set is , which is the label prediction of INNs for the downstream label sample x 1 ∈R 1×τ According to the kernel density estimation, the probability density function f is calculated, and the specific calculation formula is as follows:

[0032]

[0033] where h>0 is the bandwidth or window. The selected KDE kernel function K(·) is Gaussian curve:

[0034]

[0035] The bandwidth h is selected as:

[0036]

[0037] where is the mean of the predicted label set.

[0038] 2) Obtain the probability density function f by kernel density estimation, and calculate the peak value of the kernel density function and its corresponding coordinate label value y peak , and calculate the probability density value f under the kernel density function corresponding to label .

[0039] 3) Calculate the label confidence. Two confidence PW and FW calculation formulas are derived:

[0040] PW:

[0041]

[0042] Since there are some distribution cases that are not standard normal distribution, a second confidence calculation method FW is proposed:

[0043]

[0044] where lenth is the confidence length of the probability density curve. The specific judgment basis is as follows:

[0045] Set the left interval coordinate as y left , the right interval coordinate as y right , the change threshold as δ confidence , the change step as ε step , and the significance level as α confidence , which satisfies the following conditions:

[0046]

[0047] Thus, lenth = y right -y left .

[0048] Step 7: Concatenate the label confidence of the downstream model training set with the original downstream model training set features as new input features into the downstream regression prediction model. Take NNs as an example, set the hidden layer number dsl, the hidden layer node number p, and the labeled number N selected by the local model construction L . In addition, set the model training learning rate ρ2, the training batch data size BS, and the training cycle number ep2. And perform weighted training according to the downstream model training set label. The specific steps are as follows:

[0049] 1) Concatenate the calculated downstream model training set label confidence data with the feature data to obtain That is, the confidence calculated by the upstream is input as a one-dimensional feature of the downstream model.

[0050] 2) Define the loss function for the downstream NNs model and train the downstream model. The loss function should be able to consider the confidence level when updating parameters during gradient updates. There are two custom loss functions: formerLoss and newLoss.

[0051] FL:

[0052]

[0053] NL:

[0054]

[0055] Where w i The standard deviation of the Euclidean distance. These are model predictions. The dataset contains built-in label values. In this case, in the loss function, those with high credibility have a larger weight in the final loss evaluation, while those with low credibility have a smaller weight in the final loss evaluation. This results in less change in the loss parameters fed back to the upstream model and less impact on the model parameter updates.

[0056] In the eighth step of model evaluation and prediction application, the upstream model is removed, and the trained downstream regression model is used directly for prediction. The input data is stacked with a layer of dimensional features with a confidence level of 1 on top of the original features.

[0057] The ninth step is to evaluate the prediction results on the test set using the root mean square error (RMSE) model evaluation metric. Simultaneously, a comparison is made with the predictions of the traditional downstream model (Baseline) after removing the upstream model.

[0058]

[0059] Where y n , These are the true value and the model prediction value of the test sample, respectively, and Nt is the number of test samples.

[0060] To verify the effectiveness of the model, urea synthesis process data will be used for simulation and performance evaluation. We selected urea synthesis process data provided by CNOOC Fudao Company. The weakly supervised samples in this dataset consist of 552,799 minute-level analyzer data points from September 1, 2019 to September 23, 2020. We randomly selected 36,000 weakly labeled data points from this dataset, and a small amount of labeled data (469 laboratory data points sampled twice weekly from July 22, 2016 to March 24, 2021) was also included. We randomly selected 50% of the analyzer data points as the sub-model training set, the remaining 50% as the downstream model training set, and the 469 laboratory data points as the test set. The effectiveness of the invention is verified by comparing the weakly labeled urea synthesis process data and the laboratory data.

[0061] The synthesis of urea involves two steps: 1. Ammonia reacts with carbon dioxide to produce ammonium carbamate; 2. Ammonium carbamate decomposes into urea and water. The first step is exothermic, and the second step is endothermic.

[0062] like Figure 2 As shown, the reaction first occurs in a high-pressure ammonium carbamate condenser (pool condenser). The pool condenser is one of the main devices where the reaction takes place; the other main device is the urea synthesis tower. The pressure, temperature, and feed flow rate of the pool condenser all indicate the urea synthesis process, therefore these parameters are selected as process variables. The mixture generated in the pool condenser is sent to the downstream urea synthesis tower for further reaction; therefore, the temperature, pressure, feed flow rate, and liquid level of the reaction liquid in the synthesis tower are selected as process variables.

[0063] Throughout the process, a key mass variable is the molar ratio of ammonia to carbon dioxide at the bottom outlet of the synthesis tower, which serves as an indicator of the carbon dioxide conversion rate of the feedstock gas. To predict the carbon dioxide conversion rate (N / C) of the feedstock gas in this process, the process variables shown in Table 1 below were selected.

[0064] Table 1. Description of variables in the urea synthesis process

[0065]

[0066]

[0067] The overall process is as follows:

[0068] 1. Weakly labeled samples from the randomly selected urea synthesis process were randomly shuffled and divided into a sub-model training set and a downstream model training set in a 1:1 ratio. The laboratory dataset served as the test set for evaluating model performance. In other words, the entire dataset consisted of three parts: 18,000 weakly labeled samples forming the sub-model training set, 18,000 samples forming the downstream model training set, and 469 laboratory data points forming the test set.

[0069] 2. Build an INN network model on the deep learning platform PyTorch. Divide the sub-model training set into a non-repeating sample training set corresponding to the tree model for model training. Set the number of hidden layer nodes in the tree neural network to 40 and the activation function to ReLU. When training the INN tree network, set the batch size to 10% of the corresponding sample data volume, the number of training epochs to 20, and the learning rate to 0.01.

[0070] 3. After the INNs network is trained, the training data of the downstream model is input into the trained INNs to obtain the label prediction set of the downstream model training data. The kernel density of the label prediction set is estimated and the credibility of the corresponding label is calculated using the defined credibility calculation formula 14. The result is then merged with the original input features to form a new feature matrix.

[0071] 4. Use the new feature matrix of the downstream model training data obtained by splicing as the input of the downstream model, and then calculate the loss function of the downstream neural network according to (17) to carry out Adam optimization training.

[0072] 5. Finally, based on the above dataset and model hyperparameter settings, the credibility-based ensemble neural network weighted model proposed in this invention was constructed, and its prediction performance was compared with that of a basic neural network model on the test set. Its single-point prediction effect is shown in [the figure]. Figure 3 middle, Figure 4 The prediction RMSE loss for a single point is shown. It can be seen that the method proposed in this invention effectively combines the advantages of label credibility and the weighted model, showing improvement compared to the comparison model. Table 2 lists the test RMSE evaluation metrics of the proposed MLPINNsWoR model and the basic neural network BaseMLP.

[0073] Table 2: Comparison of Model Prediction Performance

[0074] Model Name RMSE BaseMLP 0.013679 MLPINNsWoR 0.007014

[0075] Comparative results show that the software testing modeling method proposed in this invention can effectively utilize the process information contained in a large number of noisy samples to construct a credibility-based weighted deep learning model, and its performance will be improved compared with the improved model.

[0076] The above embodiments are used to explain and illustrate the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A soft sensor modeling method based on ensemble neural network credibility estimation and weighted learning, characterized in that, Includes the following steps: (1) Determine the type of noise that may be present in the industrial label data and determine whether it is unbiased noise; (2) The urea synthesis control system was used to collect a sample set and laboratory data for modeling. All data were labeled, and the sample set was... and test dataset Where n1 = 1, ..., N l n² = 1, ..., N t , l represents labeled data, N l N represents the number of data points in the training set, t represents the number of data points in the test dataset, and N represents the number of data points in the test dataset. t Let τ be the number of test data, τ be the dimension of the variable, and R be the set of real numbers. 1×τ Let X satisfy a 1×τ two-dimensional distribution; store these data in the model database; (3) Randomly shuffle the training sample dataset into training source sets. Where o represents the shuffled source training data, and the source dataset is divided into sub-model training sets in a 1:1 ratio. and downstream model training set Where st refers to the sub-model training set, i.e., the upstream model training set, and dmt refers to the downstream model training set; (4) Build an INN network based on the open-source deep learning platform PyTorch, and set the number of tree models tn, the number of layers of a single tree model l, the number of hidden nodes m, the model training learning rate ρ1, the model training optimizer Adam, the training batch data size Bs, the number of training cycles ep1, and the training exit threshold tv. (5) Train the INN network model based on the training set, and make predictions on the training set of the downstream model to obtain the prediction label set. Where pdmt is the predicted label set of the downstream model training set using INNs; (6) Based on the obtained predicted label set Perform KDE to obtain the probability density curve or histogram to obtain the probability distribution, and then apply the original labels. The probability density value is calculated by fitting the estimated curve, and the label confidence is calculated accordingly. (7) Concatenate the label confidence of the downstream model training set with the features of the original downstream model training set, and use the concatenation as new input features to input into the downstream regression prediction model MLP. Set the number of hidden layers dsl, the number of hidden layer nodes p, and the number of labeled elements N selected to construct the local model. L Additionally, the model training learning rate ρ2, training batch size BS, and training ep2 are set; and weighted training is performed according to the labels of the downstream model training set. (8) When evaluating and predicting the model, remove the upstream model and directly use the trained downstream regression model for prediction. The input data is stacked with a layer of dimensional features with a confidence level of 1 on the basis of the original features. (9) The prediction results of the test set are evaluated using the root mean square error (RMSE) model evaluation index; and the prediction results are compared with those of the traditional downstream model MLP directly after removing the upstream model (INNsWoR).

2. The soft sensor modeling method based on ensemble neural network credibility estimation and weighted learning according to claim 1, characterized in that, Steps (2) and (3) specifically involve: processing the collected raw dataset... and corresponding weak tags After randomly shuffling the data and dividing it into a 1:1 ratio, the training sets for the sub-models and the downstream models are normalized separately. Where i is the i-th feature dimension, j is the j-th sample data, and μ i The mean of the i-th feature, δ i The mean squared error of the i-th feature; after the above normalization operation, the training sets of the sub-model and the downstream model are respectively: and 3. The soft sensor modeling method based on ensemble neural network credibility estimation and weighted learning according to claim 1, characterized in that, Step (4) specifically involves: constructing an INN network model based on the sub-model training set samples; an INN network is a forest structure with NNs as trees; wherein each NN tree will input X from non-overlapping sub-samples. i ={X t ∈R 1×τ ,t=1,2,...,T}, where T refers to the number of samples in the original sub-model training set assigned to each single tree, and i refers to the i-th tree. The training samples of each tree are unique. After multiple hidden layer mappings, the state h of the neurons in the last hidden layer is... i ={h t ∈R 1×m }, where m represents the number of hidden layer neurons, and finally outputs the noisy label prediction value; where a multilayer perceptron (MLP) is used to process the hidden state h of the previous layer. t Mapping to vector κ t : κ t =ReLU(h t ·IN α +b α ) (1) Where ReLU(·) is the activation function, W α and b α These are all parameters that need to be solved during the training of NNs networks; When training the INN network model, the Adam optimizer is used to optimize the loss function, and the model training learning rate ρ1, training batch data size Bs, and training ep1 are set.

4. The soft sensor modeling method based on ensemble neural network credibility estimation and weighted learning according to claim 1, characterized in that, Step (5) specifically involves: after the INNs network model has been trained, the feature training set of the downstream model is... The labels are used to predict the labels, and the label set of the downstream model feature training set is obtained.

5. The soft sensor modeling method based on ensemble neural network credibility estimation and weighted learning according to claim 1, characterized in that, Step (6) specifically involves: based on the obtained predicted label set Perform KDE to obtain the probability density curve or histogram to obtain the probability distribution, and then apply the original labels. The probability density value is calculated by fitting it into the estimated curve, and then the label confidence is calculated from this value. The specific steps are as follows: 6.1) Predict the label set The tags in the middle are INNs are for downstream labeled samples x 1 ∈R 1×τ For label prediction, the probability density function f is calculated using kernel density estimation. The specific calculation formula is as follows: Where h>0 represents the bandwidth or window; the KDE kernel function K(·) chosen here is a Gaussian curve: The bandwidth h is selected as follows: in, To predict the mean of the label set; 6.2) At this point, calculate the peak value of the fitted kernel density function. and its corresponding coordinate label value y peak Simultaneously calculate The probability density value f under the kernel density function label ; 6.3) Calculate label credibility; two credibility calculation formulas, PW and FW, are derived: PW: Because some distributions do not conform to a standard normal distribution, a second credibility calculation method is proposed: FW: Where *lenth* is the confidence length of the probability density curve; the specific criteria for judgment are as follows: Let the coordinates of the left interval be y left The right interval coordinate is y right , change threshold δ confidence The step size of the change is ε step The significance level is α. confidence It meets the following conditions: Therefore, length = y can be calculated. right -y left .

6. The soft sensor modeling method based on ensemble neural network credibility estimation and weighted learning according to claim 1, characterized in that, Step (7) mainly focuses on the training process of the downstream model. Taking downstream NNs as an example, it specifically includes: 7.1) The calculated downstream model training set label confidence data and feature data are concatenated to obtain... That is, the credibility obtained from the upstream calculation is used as a one-dimensional feature input to the downstream model; 7.2) Define the loss function for the downstream NNs model and train the downstream model. The loss function should be able to consider the confidence level when updating parameters during gradient updates. There are two custom loss functions: FL and NL. FL: NL: Where w i This represents the credibility of the labels predicted by the upstream model of INNsWoR on the training set of the downstream model. y is the prediction value of the downstream model for the i-th training sample. i These are the labels of the downstream model training dataset; at this point, in the loss function, those with high credibility have a large weight in the final loss evaluation, while those with low credibility have a small weight in the final loss evaluation, resulting in small changes in the loss parameters fed back to the upstream model and little impact on the model parameter updates.

7. The soft sensor modeling method based on ensemble neural network credibility estimation and weighted learning according to claim 1, characterized in that, Steps (8) and (9) are specifically as follows: 1) When applying the model for prediction, the upstream INNs confidence model should be removed, and the downstream prediction model, or a pre-trained traditional machine learning model, should be directly called for prediction. In this case, the model input includes an additional confidence weight feature. Because the prediction needs to predict the true label model, the dimension of the weight feature is set to 1, meaning the predicted concatenated feature data is {X}. predict ∈R 1×(τ+1) }; 2) Since the whole method is an improved framework for traditional machine learning to predict noisy labeled data, it is necessary to compare it with traditional machine learning methods when evaluating the results. That is, to determine the baseline and use the root mean square error (RMSE) model evaluation metric to evaluate the prediction results on the test set.

Images