Semi-supervised industrial production quality index prediction method based on pseudo label confidence evaluation
By using the interval prediction model to evaluate the confidence of pseudo-labels in the semi-supervised algorithm, selecting high confidence samples for fine-tuning of parameters, the problem of inaccurate pseudo-labels in the prior art is solved, and the accuracy and model stability of industrial production quality index prediction are improved.
Patent Information
- Application Number
- CN202411863564.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-05-16
AI Technical Summary
The existing semi-supervised algorithm lacks effective pseudo-label confidence evaluation criteria in the prediction of industrial production quality indicators, resulting in pseudo-label inaccurateness, misleading model learning and training, and unable to obtain optimal performance.
Using the idea of quantile regression, an interval prediction model is established, and the labelless data is assigned to the basic quality prediction model. The output results of the basic interval predictor are used to calculate the confidence of the pseudo-label, and the high confidence pseudo-label sample is selected and the parameters are fine-tuned.
It improves the accuracy of soft measurement of industrial production quality indicators, ensures model stability, avoids model misleading caused by inaccurate pseudo labels, and improves the performance of quality prediction models.
Smart Images
Figure CN120013311A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of chemical production process control, and specifically discloses a semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation. Background Art
[0002] Most industrial sites rely on manual production supervision, which makes it difficult to meet production needs in terms of product quality and production efficiency. In addition, the real-time optimization and control of the entire production process depends on the quality of key product variables for operational adjustments. However, due to the harsh measurement conditions and high costs in actual industrial production, most key quality variables are often obtained through offline laboratory analysis. With the continuous advancement of information technology, quality prediction modeling has been chosen to solve this problem. However, due to the complexity of energy-quality coupling conversion in the production process, it is unreliable to establish a mechanism-based quality prediction model. The data-driven quality prediction model based on deep learning has been widely used in quality prediction modeling of industrial process systems because of its excellent high-dimensional big data processing capabilities and efficient representation capabilities for complex functions.
[0003] However, in the actual modeling process, only a very small number of labeled quality samples can be used for quality prediction modeling, but these scarce samples cannot fully reflect the changes in the process itself. Although semi-supervised methods can add pseudo-labels to unlabeled data to augment the data set for model training. However, this type of method currently lacks a widely recognized pseudo-label confidence evaluation standard, which may cause the pseudo-labels of new samples added to the training set to be inaccurate, thereby misleading the learning and training of the model. Therefore, how to design a reasonable pseudo-label confidence discrimination strategy is crucial to improving the performance of quality prediction models.
[0004] The interval prediction method based on quantiles has two characteristics. Feature 1: The training process is only subject to quantile constraints on the true value. Feature 2: The width of the interval of the model output result changes continuously according to different samples. Based on feature 1, it is possible to evaluate whether the trained model meets the quantile hypothesis based on the pre-set quantile. Based on feature 2, the model will output intervals with different widths when different samples are input, which provides the possibility of measuring the possible distribution range of the true value of the quality indicator. Based on features 1 and 2, if the trained model meets the hypothesis, it provides the possibility of using the possible distribution interval of the true value of the quality indicator and the pseudo-label provided by the basic quality prediction model to measure the confidence. However, the existing semi-supervised algorithms do not measure the confidence of the pseudo-labels. Most methods will use all samples with pseudo-labels for further training of the quality prediction model, resulting in misleading the training process of the semi-supervised soft measurement model and failing to obtain optimal performance. In view of this, the inventor provides a semi-supervised industrial production quality indicator prediction method based on pseudo-label confidence evaluation to solve the above problems. Summary of the invention
[0005] The purpose of the present invention is to provide a semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation, so as to improve the accuracy of soft measurement of industrial process production quality index while ensuring the stability of the model.
[0006] In order to achieve the above object, the basic scheme of the present invention provides a semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation, comprising the following steps:
[0007] Step 1, collecting historical process data and historical quality test data from the industrial production process;
[0008] Step 2: divide the data into training set and test set according to a certain ratio.
[0009] Step 3: Build a basic quality prediction model and a basic interval predictor, and use labeled process data for training;
[0010] Step 4: assign pseudo labels to unlabeled data based on the basic quality prediction model, and calculate the confidence of the pseudo labels based on the output results of the basic interval predictor;
[0011] Step 5: Add unlabeled data with high-confidence pseudo labels to the original training set, and use this to fine-tune the parameters of the basic quality prediction model;
[0012] Step 6: Input the process data of the test sample and obtain the predicted value of the output.
[0013] Furthermore, in step 2, the labeled data is first randomly divided into a training set and a test set in proportion, and then the unlabeled data is added to the training set as an additional training set.
[0014] Furthermore, the training set and the test set are 80% and 20% of the data containing labels, respectively.
[0015] Furthermore, in step 3, a basic quality prediction model and a basic interval prediction model are built based on LSTM and a fully connected neural network using the given process data and corresponding quality indicators.
[0016] Furthermore, the output of the basic quality prediction model is the predicted value of the quality index, and the output of the basic interval prediction model is the upper and lower limits of the possible distribution of the predicted true value under certain quantile conditions. The loss function expression for training the basic quality prediction model is as follows:
[0017]
[0018] The relationship between the process data and the upper and lower limits of the possible distribution of the true value is expressed as follows:
[0019]
[0020] Where Q y(i) (τ|x L (i)) is for input x L (i) The conditional quantile at quantile τ, representing p(y(i)<Q y(i) (τ|x L (i)))=τ, f(·) is the mapping of regression calculation, is the regression parameter, which is obtained by minimizing the following loss function:
[0021]
[0022] in yes The estimate of the function g(·) is as follows:
[0023]
[0024] The expressions for the upper and lower limits of the possible distribution of the true value obtained by training the model are as follows:
[0025] Q i =[Q y(i) (τ / 2|x L (i)),Q y(i) (1-τ / 2|x L (i))]
[0026] L2 regularization is used during training to prevent overfitting.
[0027] Furthermore, in step S3, the quantile τ is 0.1, and the learning rate during training is 0.001.
[0028] Further, in step 4, the following sub-steps are included:
[0029] Step 4.1: For the unlabeled process data {x U (1),x U (2),...x U (N U )}, using the basic quality prediction to calculate the corresponding prediction value Using the basic interval prediction model to calculate the corresponding interval sequence {Q U (1),Q U (2),...Q U (N U )};
[0030] Step 4.2: Calculate the width of the interval corresponding to each unlabeled sample by the following formula:
[0031]
[0032] Step 4.3: Use kernel density estimation and Gaussian kernel intervals to represent the distribution of interval widths:
[0033]
[0034] where δ is the bandwidth, is the kernel density estimation function;
[0035] Step 4.4: Utilize pseudo labels and interval Q U Calculate the confidence of the pseudo-label:
[0036]
[0037] like In the interval Q U (i), then the corresponding sample x U (i) The confidence of the pseudo-label is set to 0.
[0038] Furthermore, in step 5, unlabeled data with pseudo-label confidence higher than 80% are added to the labeled training set, and the expanded training set is used to fine-tune the parameters of the basic quality prediction model.
[0039] The principle and effect of this scheme are:
[0040] 1. Based on the idea of quantile regression, the present invention establishes a prediction model between the original process data and the interval of possible distribution of the true quality value under certain assumptions. The interval predicted by the model helps to provide an assumption and theoretical basis for measuring the confidence of pseudo labels.
[0041] 2. Based on the prediction interval and the existing pseudo-labels, the present invention designs a strategy to measure the confidence of pseudo-labels. This strategy is helpful to select samples with high-confidence pseudo-labels to be added to the training set, so as to avoid the model being misled during the training process due to inaccurate pseudo-labels.
[0042] 3. In summary, the present invention establishes an interval prediction model based on the idea of quantile regression, and uses the prediction interval and the pseudo-label provided by the basic quality prediction model to measure the confidence of the pseudo-label. Then, samples with high-confidence pseudo-labels are selected to be added to the training set to enhance the ability of the data to characterize changes in the industrial production process. These new data are used to fine-tune the parameters of the basic quality prediction model, and ultimately the quality prediction model has higher performance. Compared with traditional methods, this method is simple to operate, has a short training time, and has a high confidence in the pseudo-labels of the data newly added to the training set. It is suitable for industrial production process quality prediction where the sampling rates of process data and quality data are unbalanced. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0044] Figure 1 A schematic diagram of an alumina six-effect four-flash structure evaporation process equipment is shown;
[0045] Figure 2 A flowchart of a semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation proposed in an embodiment of the present application is shown;
[0046] Figure 3 The comparison between the output interval and the true value of the interval prediction model of the semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation proposed in the embodiment of the present application is shown;
[0047] Figure 4 The figure shows a comparison between the output interval of a semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation proposed in an embodiment of the present application and the pseudo-label output by a basic quality prediction model;
[0048] Figure 5The probability density distribution curve of the output interval width of the interval prediction model in the semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation proposed in an embodiment of the present application is shown;
[0049] Figure 6 A schematic diagram of the prediction results of the quality index of the evaporation mother liquor at the outlet of the IV effect flash evaporator of a semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0050] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.
[0051] A semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation is used to take the alumina six-effect four-flash structure evaporation process as an example. The process equipment schematic diagram is shown in Figure 1 Shown: Evaporation of stock solution The raw liquid enters the entire evaporation system from the flash evaporator, flows into the VI-effect evaporator after flash evaporation, and then sequentially passes to the I-effect evaporator. The raw liquid completes heat exchange in the evaporator, and then enters four cascade flash evaporators, where the solution is further evaporated by reducing the pressure, and finally forms the evaporated mother liquid. Flowing out from the IV effect flash evaporator. At the same time, the new steam F s It enters the evaporation system from the I-effect evaporator, flows sequentially to the second-stage (II-effect) evaporator, continues to provide heat energy for the evaporation process, and finally flows out from the VI-effect evaporator.
[0052] The predicted quality index of this embodiment is the Nk value of the evaporated mother liquor at the outlet of the IV effect flash evaporator, which represents the caustic soda concentration of the evaporated mother liquor. Since the caustic soda solution is mixed with ionic impurities such as silicon and calcium, it is easier to precipitate during the evaporation process, thereby clogging the pipeline or causing scarring on the sensor, making the sensor inaccurate and unusable. Therefore, the accurate Nk value often needs to be sent to the laboratory for analysis to obtain, and the interval between the test results is at least 8 hours, making it difficult for on-site operators to evaluate whether the operation is reasonable. This embodiment predicts the Nk value of the evaporated mother liquor at the outlet of the IV effect flash evaporator by establishing a semi-supervised industrial production quality index prediction model based on pseudo-label confidence assessment.
[0053] The prediction method of this embodiment is as follows Figure 2 As shown, the following steps are included:
[0054] Step 1: Collect historical process data and historical quality test data from the industrial production process.
[0055] Step 2: divide the data into training set and test set according to a certain ratio.
[0056] Step 2.1, divide the process data with quality labels and the process data without labels into two data sets;
[0057] Step 2.2: randomly divide the labeled data into 80% training set and 20% test set.
[0058] In step 2.3, the unlabeled data is added as an additional training set to the training set described in step 2.2.
[0059] Step 3: Build a basic quality prediction model and basic interval predictor and use labeled process data for training. Specifically include:
[0060] Step 3.1, given the process data {x L (1),x L (2),...x L (N L )} and the corresponding quality indicators {y(1),y(2),...y(N L )}, based on LSTM and fully connected neural network, a basic quality prediction model and a basic interval prediction model are built. The output of the basic quality prediction model is the predicted value of the quality index, and the output of the basic interval prediction model is the upper and lower limits of the possible distribution of the predicted true value under certain quantile conditions;
[0061] Step 3.2: Train the basic quality prediction model according to the following loss function:
[0062]
[0063] Where y(t) is the true value, is the predicted value, L pred is the predicted loss value.
[0064] Step 3.3, establish the relationship between the process data and the upper and lower limits of the possible distribution of the true value according to the following formula:
[0065]
[0066] Where Q y(i) (τ|x L (i)) is for input x L (i) The conditional quantile at quantile τ, representing p(y(i)<Q y(i) (τ|x L (i))) = τ. f(·) is the mapping of the regression calculation.
[0067] Step 3.4: Get the regression parameters by minimizing the following loss function
[0068]
[0069] in yes The specific expression of the function g(·) is:
[0070]
[0071] Step 3.5: Set the quantile τ to 0.1 and train the model to obtain the upper and lower limits of the possible distribution of the true value:
[0072] Q i =[Q y(i) (τ / 2|x L (i)),Q y(i) (1-τ / 2|x L (i))]
[0073] The learning rate is set to 0.001, and L2 regularization is used to prevent overfitting.
[0074] Step 4: assign pseudo labels to unlabeled data based on the basic quality prediction model, and calculate the confidence of the pseudo labels based on the output results of the basic interval predictor. Specifically, it includes:
[0075] Step 4.1: For the unlabeled process data {x U (1),x U (2),...x U (N U )}, using the basic quality prediction to calculate the corresponding prediction value Using the basic interval prediction model to calculate the corresponding interval sequence {Q U (1),Q U (2),...Q U (N U )};
[0076] Step 4.2: Calculate the width of the interval corresponding to each unlabeled sample:
[0077]
[0078] Step 4.3: Use kernel density estimation and Gaussian kernel intervals to represent the distribution of interval widths:
[0079]
[0080] where δ is the bandwidth, is the kernel density estimation function.
[0081] Step 4.4: Utilize pseudo labels and interval Q U Calculate the confidence of the pseudo-label:
[0082]
[0083] like In the interval Q U (i), then the corresponding sample x U (i) The confidence of the pseudo-label is set to 0.
[0084] Step 5: Add unlabeled data with high-confidence pseudo labels to the original training set, and use this to fine-tune the parameters of the basic quality prediction model. Specifically, it includes:
[0085] Step 5.1: Add unlabeled data with pseudo-label confidence higher than 80% to the labeled training set;
[0086] Step 5.2: Use the expanded training set to fine-tune the parameters of the basic quality prediction model.
[0087] Step 6: Input the process data of the test sample and obtain the predicted value of the output.
[0088] Figure 3 and Figure 4 The comparison between the output interval of the interval prediction model applied to the aluminum oxide evaporation process and the true value, as well as the comparison between the output interval and the pseudo-label output by the basic quality prediction model are shown. The comparison between the output interval and the true value proves that the relationship between the two is generally in line with the given assumptions, that is, 90% of the true values will fall within the prediction interval. The comparison between the output interval and the pseudo-label shows that the quality predictor and the interval predictor are independent of each other, and there is no causal relationship between the two. Therefore, the pseudo-label provided by the quality predictor and the output interval of the interval predictor can be used to evaluate the confidence of the pseudo-label.
[0089] The probability density distribution curve of the output interval width of the interval prediction model applied to the aluminum oxide evaporation process is shown in the figure below: Figure 5 As shown in , if the result of interval prediction is consistent with the hypothesis, then when the pseudo label of the sample falls within the interval and the width of the interval is smaller, the confidence of the pseudo label is higher;
[0090] The prediction of Nk value of the mother liquor at the outlet of the flash evaporator of the fourth effect in the alumina evaporation process is as follows: Figure 6As shown, the results show that compared with some other traditional methods that use semi-supervision to improve data utilization, the method proposed in this embodiment can better use unlabeled data to characterize missing process change information and achieve a more accurate prediction of the Nk value of alumina evaporation mother liquor. It is suitable for promotion to other types of industrial processes.
[0091] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.
Claims
1. A semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation, characterized by: The following steps are involved: Step 1, collecting historical process data and historical quality test data from the industrial production process; Step 2: divide the data into training set and test set according to a certain ratio. Step 3: Build a basic quality prediction model and a basic interval predictor, and use labeled process data for training; Step 4: assign pseudo labels to unlabeled data based on the basic quality prediction model, and calculate the confidence of the pseudo labels based on the output results of the basic interval predictor; Step 5: Add unlabeled data with high-confidence pseudo labels to the original training set, and use this to fine-tune the parameters of the basic quality prediction model; Step 6: Input the process data of the test sample and obtain the predicted value of the output.
2. According to claim 1, a semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation is characterized in that: In step 2, the labeled data is first randomly divided into a training set and a test set in proportion, and then the unlabeled data is added to the training set as an additional training set.
3. According to claim 2, a semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation is characterized in that: The training set and test set contain 80% and 20% of the data with labels, respectively.
4. A semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation according to any one of claims 1 to 3, characterized in that: In step 3, a basic quality prediction model and a basic interval prediction model are built based on LSTM and a fully connected neural network using given process data and corresponding quality indicators.
5. The method for predicting semi-supervised industrial production quality indicators based on pseudo-label confidence evaluation according to claim 4 is characterized in that: The output of the basic quality prediction model is the predicted value of the quality index, and the output of the basic interval prediction model is the upper and lower limits of the possible distribution of the predicted true value under certain quantile conditions. The loss function expression for training the basic quality prediction model is as follows: The relationship between the process data and the upper and lower limits of the possible distribution of the true value is expressed as follows: Where Q y(i) (τ|x L (i)) is for input x L (i) The conditional quantile at quantile τ, representing p(y(i)<Q y(i) (τx L (i)))=τ, f(·) is the mapping of regression calculation, is the regression parameter, which is obtained by minimizing the following loss function: in yes The estimate of the function g(·) is as follows: The expressions for the upper and lower limits of the possible distribution of the true value obtained by training the model are as follows: Q i =[Q y(i) (τ / 2x L (i)),Q y(i) (1-τ / 2x L (i))] L2 regularization is used during training to prevent overfitting.
6. The method for predicting semi-supervised industrial production quality indicators based on pseudo-label confidence evaluation according to claim 5 is characterized in that: In step S3, the quantile τ is 0.1, and the learning rate during training is 0.
001.
7. A semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation according to claim 5 or 6, characterized in that: In step 4, the following sub-steps are included: Step 4.1: For the unlabeled process data {x U (1),x U (2),...x U (N U )}, using the basic quality prediction to calculate the corresponding prediction value Using the basic interval prediction model to calculate the corresponding interval sequence {Q U (1),Q U (2),...Q U (N U )}; Step 4.2: Calculate the width of the interval corresponding to each unlabeled sample by the following formula: Step 4.3: Use kernel density estimation and Gaussian kernel intervals to represent the distribution of interval widths: where δ is the bandwidth, is the kernel density estimation function; Step 4.4: Utilize pseudo labels and interval Q U Calculate the confidence of the pseudo-label: like In the interval Q U (i), then the corresponding sample x U (i) The confidence of the pseudo-label is set to 0.
8. A semi-supervised industrial production quality index prediction method based on pseudo-label confidence evaluation according to claim 5 or 6, characterized in that: In step 5, unlabeled data with pseudo-label confidence higher than 80% are added to the labeled training set, and the expanded training set is used to fine-tune the parameters of the basic quality prediction model.