Industrial key process index adaptive prediction method based on instant learning

Through the real-time learning method and the Marshall-cosine-similar sample selection strategy, the problem of low prediction accuracy in the industrial production process is solved, and long-term accurate prediction and dynamic adaptation of key process indicators are achieved, providing an important basis for operational adjustment .

CN118982276APending Publication Date: 2024-11-19CENT SOUTH UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411005149.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

Traditional offline data-driven models are difficult to adapt to environmental changes in the industrial production process, resulting in low accuracy in predicting key process indicators and lack real-time update mechanisms, so they cannot dynamically adapt to new changes in the production process.

Method used

Adaptive prediction method of industrial key process indicators based on instant learning, using the Marthagata-cosine two-step method similar sample selection strategy, combined with nearest neighbor label-free data and rough estimates, adaptively select student model parameters for fine-tuning offline training. Implement online updates to overcome the problem of difference between model learning information and actual working conditions.

Benefits of technology

It improves the prediction accuracy of key process indicators, achieves long-term accurate prediction, provides an important foundation for the operational adjustment of industrial processes, and overcomes the problem of performance degradation of traditional models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982276A_ABST
    Figure CN118982276A_ABST
Patent Text Reader

Abstract

The invention is suitable for the technical field of industrial process data-driven modeling, and provides an industrial key process index adaptive prediction method based on instant learning, and the method comprises the steps: determining process parameters related to industrial key process indexes, and obtaining offline training data; training the teacher model by using the offline training data, and carrying out distillation learning on the student model according to the trained teacher model; first-step screening of the instant learning similar sample set is realized based on the mahalanobis distance; performing process index prediction on a current query sample by utilizing the trained student model, and realizing second-step screening of an instant learning similar sample set according to a prediction result and cosine similarity; performing online training on the student model by using the instant learning similar sample set to obtain a process index prediction model; and performing process index prediction on the current query sample by using the process index prediction model. The prediction accuracy of the key process indexes can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the technical field of industrial process data-driven modeling, and in particular, relates to an adaptive prediction method for industrial key process indicators based on real-time learning. Background Art

[0002] In the industrial production process, online and accurate measurement of key process indicators is an important basis for ensuring safe and stable control of the production process and improving production efficiency. However, most industrial sites have harsh environments and complex production mechanisms, and many process indicators such as grade and concentration cannot be directly measured. Due to the lack of online detection methods, operators mainly rely on experience to analyze and observe, estimate key process indicators, and then adjust the production process, which has problems such as strong subjectivity and low reliability. The data-driven soft sensor modeling method based on deep learning has significant advantages, such as not requiring a large amount of prior knowledge and strong nonlinear fitting capabilities. By establishing a complex relationship between key process indicators that are difficult to measure and process variables that are easy to measure, such as temperature, pressure, and liquid level, a prediction model is constructed to achieve the estimation of key process indicators, which can provide an important reference for operational adjustments in actual industrial production processes.

[0003] In the actual continuous industrial production process, process indicators are affected by many factors, including frequent changes in raw material properties, different product demands due to seasonal differences, equipment salt damage, and differences in worker operation adjustments. These factors together make the continuous industrial production process present dynamic time-varying characteristics, resulting in performance degradation of traditional offline data-driven models. Specifically, there are three characteristics: the first is poor model adaptability. Offline models are usually trained based on historical data, but these data cannot reflect all possible operating conditions in the production process. When the production environment changes, the model cannot adapt to the new conditions, resulting in a decrease in prediction accuracy; the second is the data offset problem: over time, the distribution of data collected during the production process may change. The data distribution captured by the offline model during training may be significantly different from the current actual data distribution, resulting in a decrease in the model's prediction performance and an inability to accurately reflect the current production status; the third is the lack of a real-time update mechanism. Traditional offline models cannot use new production data in a timely manner to adjust and optimize model parameters, making the model unable to dynamically adapt to new changes in the production process, resulting in low prediction accuracy of key process indicators, which in turn affects the operational adjustments of the production process. Summary of the invention

[0004] The embodiment of the present application provides an industrial key process indicator adaptive prediction method based on real-time learning, which can solve the problem of low prediction accuracy of key process indicators.

[0005] The present application embodiment provides an industrial key process indicator adaptive prediction method based on real-time learning, including:

[0006] Determine process parameters related to industrial key process indicators and obtain offline training data; the offline training data includes process parameter data at multiple historical moments and industrial key process indicator values ​​corresponding to process parameter data at some historical moments;

[0007] The teacher model is trained using offline training data to obtain a trained teacher model, and the student model is distilled and learned based on the trained teacher model to obtain a trained student model; the teacher model is a convolutional neural network;

[0008] The process parameter data of multiple historical moments before the current prediction moment is used as the nearest unlabeled data set of the current query sample; the current query sample is the process parameter data of the current prediction moment;

[0009] Based on Mahalanobis distance, the initial instant learning similar sample set is selected from the nearest neighbor unlabeled dataset;

[0010] The trained student model is used to predict the process indicators of the current query sample, and based on the rough estimated value and cosine similarity of the predicted process indicators, the final instant learning similar sample set is screened from the initial instant learning similar sample set;

[0011] The trained student model is trained online using the final instant learning similar sample set to obtain a process index prediction model;

[0012] The current query sample is input into the process indicator prediction model to predict the process indicator and obtain the predicted value of the industrial key process indicator at the current prediction moment.

[0013] Optionally, based on the Mahalanobis distance, an initial set of similar samples for instant learning is selected from the nearest neighbor unlabeled dataset, including:

[0014] Calculate the weighted center of the nearest neighbor unlabeled dataset;

[0015] Calculate the difference vector between the current query sample and the weighted center;

[0016] The process parameter data of each historical moment in the nearest unlabeled data set is taken as a historical sample, and the Mahalanobis distance between each historical sample in the nearest unlabeled data set and the current query sample is calculated;

[0017] The set of historical samples whose Mahalanobis distance is less than the length of the difference vector is used as the initial instant learning similarity sample set.

[0018] Optionally, calculate the weighted center of the nearest neighbor unlabeled dataset, including:

[0019] The weighted center x of the neighbor unlabeled data set is calculated by the weighted center calculation formula q,c ; The weighted center calculation formula is:

[0020]

[0021] Among them, N q,U represents the number of historical samples in the nearest neighbor unlabeled dataset, represents the i-th historical sample in the nearest unlabeled dataset, t q represents the current prediction time, express The corresponding historical moment, σ represents the coordination coefficient.

[0022] Optionally, calculate the difference vector between the current query sample and the weighted center, including:

[0023] By formula x q,dif =x q -x q,c Calculate the difference vector x between the current query sample and the weighted center q,dif ;

[0024] Among them, x q Represents the current query sample.

[0025] Optionally, based on the rough estimated value of the predicted process indicator and the cosine similarity, a final instant learning similar sample set is screened from the initial instant learning similar sample set, including:

[0026] Based on the rough estimate of the process indicators obtained by prediction and the number of historical samples in the initial instant learning similar sample set Determine the number K of historical samples contained in the final instant learning similar sample set;

[0027] Let each historical sample in the initial instant learning similar sample set be subtracted from the weighted center to obtain a set of difference vectors

[0028] Calculate the difference vector set The difference vectors and weighted center x q,c The cosine similarity set C of

[0029] According to the cosine similarity set C, K historical samples are selected from the initial instant learning similar sample set as the final instant learning similar sample set.

[0030] Optionally, a rough estimate of the process indicator obtained based on the prediction and the number of historical samples in the initial instant learning similar sample set Determine the number of historical samples K contained in the final instant learning similar sample set, including:

[0031] The number of historical samples K included in the final instant learning similar sample set is determined by the following formula:

[0032]

[0033] in, represents the rough estimate of the predicted process index, Δ represents the degree of widening, N L represents the total number of industrial key process indicator values ​​in offline training data, y i Represents the i-th industrial key process indicator value in the offline training data.

[0034] Optional, a set of difference vectors for:

[0035]

[0036] in, Represents a set of difference vectors The i-th difference vector in Represents the i-th historical sample in the initial instant learning similar sample set.

[0037] Optionally, the cosine similarity set C is:

[0038]

[0039] Among them, c i express With weighted center x q,c The cosine similarity of express The j-th dimension component of , d represents the total dimension of the component, Represents x q,c The j-th dimension component of .

[0040] Optionally, according to the cosine similarity set C, K historical samples are selected from the initial instant learning similar sample set as the final instant learning similar sample set, including:

[0041] Sort the cosine similarities in the cosine similarity set C in descending order;

[0042] The K historical samples corresponding to the first K cosine similarities after sorting are used as the final instant learning similarity sample set.

[0043] The above solution of the present application has the following beneficial effects:

[0044] In an embodiment of the present application, a two-step Markov-cosine similar sample selection strategy is used to take into account both the neighbors in spatial position and the changing trend of recent working conditions; by introducing neighbor unlabeled data information and the density of roughly estimated value distribution, a similar sample set is adaptively selected for fine-tuning the student model parameters of offline training, and the similar sample set is used to perform online update of the parameters of the offline trained student model, so that the process indicator prediction model obtained through offline training and online update can continuously learn the online data distribution while maintaining the learning of the spatiotemporal information of historical data, overcoming the problem of offline model performance degradation caused by the increasing difference between the information learned by the model and the current actual working conditions, which can greatly improve the prediction accuracy of key process indicators, realize long-term accurate prediction of key industrial process indicators, and provide an important basis for the operational adjustment of industrial processes.

[0045] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of an adaptive prediction method for industrial key process indicators based on real-time learning provided in one embodiment of the present application;

[0048] Figure 2 A schematic diagram of the first step of screening similar samples based on Mahalanobis distance provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of the second step of screening similar samples based on cosine similarity provided in an embodiment of the present application;

[0050] Figure 4 It is a scatter plot of the prediction results of potash concentrate grade of the process indicator of LWPLS algorithm in an example of this application;

[0051] Figure 5 It is a scatter plot of the prediction results of the process indicator potash concentrate grade by the OAFDBN model in an example of this application;

[0052] Figure 6 It is a scatter plot of the prediction results of potash concentrate grade of the process indicator by the offline MT-TSNLNet model in an example of this application;

[0053] Figure 7This is a scatter plot of the prediction results of potash concentrate grade using the online MT-TSNLNet model for process indicators in an example of this application. DETAILED DESCRIPTION

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0056] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0057] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0058] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0059] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0060] In response to the current problem of low prediction accuracy of key process indicators, an embodiment of the present application provides an adaptive prediction method for industrial key process indicators based on just-in-time learning. Through the Markov-cosine two-step method similar sample selection strategy, it takes into account both the neighbors in spatial position and the changing trend of recent working conditions. By introducing the neighbor unlabeled data information and the density of the roughly estimated value distribution, a similar sample set is adaptively selected for fine-tuning the student model parameters for offline training, and the similar sample set is used to perform online update of the parameters of the offline trained student model, so that the process indicator prediction model obtained through offline training and online update has high prediction accuracy. When it is used for key process indicators, it can greatly improve the prediction accuracy of key process indicators, achieve long-term accurate prediction of industrial key process indicators, and provide an important basis for operational adjustment of industrial processes.

[0061] The following is an illustrative description of the adaptive prediction method for key industrial process indicators based on real-time learning provided by the present application in conjunction with specific embodiments.

[0062] like Figure 1 As shown, the adaptive prediction method for industrial key process indicators based on real-time learning provided by the embodiment of the present application includes the following steps:

[0063] Step 11, determine the process parameters related to the industrial key process indicators and obtain offline training data.

[0064] The above-mentioned key industrial process indicators are indicators used to measure the quality of industrial products, such as grade, concentration, etc.; the above-mentioned process parameters refer to process parameters that have a strong correlation with the key industrial process indicators, such as temperature, humidity, etc.

[0065] The above-mentioned offline training data includes process parameter data at multiple historical moments, and industrial key process indicator values ​​(ie, test data) corresponding to the process parameter data at some historical moments.

[0066] It is understandable that, in order to facilitate subsequent model training, when collecting historical data, a data set can be constructed by collecting unlabeled and labeled process parameter data of the continuous production process and the test data of the corresponding labeled process indicators. Among them, the above-mentioned labels refer to the industrial key process indicator values ​​corresponding to the process parameter data. Of course, in order to facilitate data processing and model training, after collecting relevant data, the collected raw data can be normalized, and the normalized data can be used for subsequent steps (such as model training).

[0067] Taking the industrial scenario of potassium salt flotation continuous production process as an example, labeled data and unlabeled data are obtained. Each sample point contains 25 dimensions of process parameter features. The collected data is divided into training set, validation set and test set in a ratio of 8:1:1. Each data is normalized to the range of 0 to 1 to protect the original data.

[0068] Step 12: Use the offline training data to train the teacher model to obtain a trained teacher model, and perform distillation learning on the student model based on the trained teacher model to obtain a trained student model.

[0069] The teacher model is a convolutional neural network, and the student model is also a convolutional neural network. It should be noted that the teacher model is a large and complex model that is well trained and can provide high-precision predictions, while the student model is a relatively small and more concise model that aims to replicate the performance of the teacher model as much as possible through training.

[0070] In some embodiments of the present application, a conventional teacher model and student model training method may be used to obtain a trained student model so that the student model can predict the value of the industrial key process indicator. The training process of the student model is exemplified below.

[0071] In some embodiments of the present application, offline training data can be used as input to offline train a student model that integrates spatiotemporal information learning through calculation and back propagation of supervision loss, consistency loss, and spatiotemporal neighborhood embedding loss.

[0072] The above step 12 performs offline training on the model to obtain the trained student model. The specific implementation steps are as follows:

[0073] Step 12.1, use supervised loss to learn the label information in the input data;

[0074] Step 12.2, use consistency loss to constrain the output of the student model to be consistent with the output of the teacher model;

[0075] Step 12.3, using the spatial neighborhood embedding loss to retain the spatial neighborhood structure of the input data after dimensionality reduction mapping, and learning the spatial correlation information of labeled and unlabeled data;

[0076] Step 12.4, using the temporal neighborhood embedding loss to retain the temporal neighborhood structure of the input data after dimensionality reduction mapping, and learning the temporal correlation information of labeled and unlabeled data;

[0077] Step 12.5, copy the basic semi-supervised spatiotemporal neighborhood learning network to obtain a model with exactly the same network structure, define the copied model as the teacher model, and use exponential moving average to update the parameters of the teacher model;

[0078] Step 12.6, the original model is defined as the student model, and the overall loss of the student model is calculated;

[0079] Step 12.7, back-propagation updates the student model parameters.

[0080] Among them, the supervision loss The expression is:

[0081]

[0082] N L Indicates the number of samples with labeled data, y L,i is the true label of the i-th sample, f L,i is the predicted value of the model for the i-th sample, and L(·) represents the specific loss function.

[0083] Consistency loss The expression is:

[0084]

[0085] N U represents the number of samples of unlabeled data, f LU,i and Represent the predicted values ​​for the original input and the perturbed input respectively.

[0086] Spatial Neighborhood Embedding Loss The expression is:

[0087]

[0088] N L and N U Represents the number of samples with labeled data and unlabeled data, K S represents the number of spatial neighbor samples, x i represents the i-th input data, x S,ij represents the jth spatial neighbor sample corresponding to the i-th input data, and They represent the low-dimensional embedding samples of the two, g(x i ) represents x i The low-dimensional mapping of g(x S,ij ) represents x S,ij Low-dimensional mapping.

[0089] W S,ij It is the distance weight that measures the neighbor relationship between two sample spaces. The calculation formula can be expressed as follows:

[0090] W S,ij =exp(-||xi -x S,ij || 2 / σ S )

[0091] Among them, σ S is an adjustable hyperparameter that controls the degree of diversity in the spatial distance weights.

[0092] Temporal Neighborhood Embedding Loss The expression is:

[0093]

[0094] Among them, K T represents the number of time neighbor samples, x T,ij represents the jth time neighbor sample corresponding to the i-th input data, represents the low-dimensional embedding sample of the sample, g(x T,ij )express Low-dimensional mapping.

[0095] W T,ij It is the distance weight that measures the time neighbor relationship between two samples. The calculation formula can be expressed as follows:

[0096]

[0097] Among them, t i Represents sample x i The sampling time, t ij Represents sample x T,ij The sampling time, σ T It is an adjustable hyperparameter that controls the degree of difference in time distance weights. The closer the sampling time of two samples is, the larger the time distance weight value is.

[0098] The expression of exponential moving average is:

[0099]

[0100] The original model is defined as the student model, and its parameters are recorded as The copy model is defined as the teacher model, and its parameters are recorded as Where q represents the qth training iteration, and α is a hyperparameter called exponential moving average decay. The characteristic of exponential moving average is that α usually takes a value close to 1, such as 0.9 to 0.999, which makes the model parameters closer to the current moment have a higher weight.

[0101] The overall loss of the above student model The expression is:

[0102]

[0103] in, represents the supervision loss, which is used to learn the label information in the input data; represents the consistency loss, which constrains the output of the student model to be consistent with the output of the teacher model. i represents the true label of the sample (i.e. the actual industrial key process indicator value), represents the predicted value of the industrial key process indicator output by the student model, Represents the predicted value of the industrial key process indicator output by the teacher model. λ1 and λ2 are coordination coefficients used to balance the importance of different loss terms. Based on the calculated loss, the parameters of the student model are updated through the back propagation algorithm until the model converges.

[0104] Step 13, taking the process parameter data of multiple historical moments before the current prediction moment as the nearest neighbor unlabeled data set of the current query sample.

[0105] The current query sample is the process parameter data at the current prediction time (current time). In some embodiments of the present application, the N nearest to the current prediction time can be q,U The process parameter data at each historical moment is used as the nearest unlabeled dataset X of the current query sample. q,U , whose expression is:

[0106]

[0107] In the above formula, Represents X q,U The first historical sample in Represents X q,U The second historical sample in Represents X q,U The Nth q,U A historical sample.

[0108] X q,U It reflects the change in the spatial position of the data before the current moment. From the overall trend, the closer the sampling time (i.e. the sampling time of the historical sample) is to the current query sample, the closer its spatial position is to the query sample.

[0109] Step 14: Based on the Mahalanobis distance, an initial instant learning similar sample set is selected from the nearest neighbor unlabeled data set.

[0110] In some embodiments of the present application, Figure 2 As shown in FIG, the screening process of the initial instant learning similar sample set includes the following steps:

[0111] Step 14.1, calculate the weighted center of the nearest neighbor unlabeled dataset.

[0112] In some embodiments of the present application, the weighted center x of the neighboring unlabeled data set can be calculated by the weighted center calculation formula: q,c ; The weighted center calculation formula is:

[0113]

[0114] Among them, the weight w i Determined by sampling time, and the current query sample x q The closer the sampling times are, the more weight they should contribute when calculating the weighted center:

[0115]

[0116]

[0117] Among them, N q,U represents the number of historical samples in the nearest neighbor unlabeled dataset, represents the i-th historical sample in the nearest unlabeled dataset, t q represents the current prediction time, express At the corresponding historical moment, σ represents the coordination coefficient. The larger the value of σ, the greater the discrimination of the weight.

[0118] Step 14.2, calculate the difference vector between the current query sample and the weighted center.

[0119] In some embodiments of the present application, the formula x q,dif =x q -x q,c Calculate the difference vector x between the current query sample and the weighted center q,dif ; In the formula, x q Represents the current query sample.

[0120] x q,dif is a vector with size and direction, which points from the weighted center of the nearest unlabeled data to the current query sample. Its direction reflects the direction of data change between the nearest unlabeled data set and the current query sample. q,dif The length of x q With x q,c distance.

[0121] Step 14.3, taking the process parameter data at each historical moment in the nearest unlabeled data set as a historical sample, and calculating the Mahalanobis distance between each historical sample in the nearest unlabeled data set and the current query sample.

[0122] In some embodiments of the present application, for any historical sample x in the neighbor unlabeled dataset, the historical sample is similar to the current query sample x.q The Mahalanobis distance between them is denoted as φ1(x q ,x). It can be understood that the Mahalanobis distance is a commonly used distance measurement method, so its specific calculation principle will not be described in detail here.

[0123] Step 14.4, the set of historical samples whose Mahalanobis distance is less than the length of the difference vector is used as the initial instant learning similar sample set. That is, when performing the first step of screening similar samples, the neighborhood space around the current query sample is determined using the length of the difference vector as a threshold.

[0124] The above initial instant learning similar sample set The expression is:

[0125]

[0126] In the above formula, for The first historical sample in for The second historical sample in for The A historical sample, The number of historical samples in the initial instant learning similar sample set.

[0127] Step 15, using the trained student model to predict the process indicators of the current query sample, and based on the rough estimated value and cosine similarity of the predicted process indicators, the final instant learning similar sample set is screened out from the initial instant learning similar sample set.

[0128] In some embodiments of the present application, the student model can predict the process indicators after training. Therefore, the current query sample is input into the trained student model, and the prediction value output by the student model after analyzing and processing the current query sample is a rough estimate of the process indicator.

[0129] In some embodiments of the present application, Figure 3 As shown, the process of selecting the final instant learning similar sample set from the initial instant learning similar sample set includes the following steps:

[0130] Step 15.1: Based on the rough estimate of the process indicator obtained by prediction and the number of historical samples in the initial instant learning similar sample set Determine the number K of historical samples contained in the final instant learning similar sample set.

[0131] Specifically, the number K of historical samples included in the final instant learning similar sample set can be determined by the following formula:

[0132]

[0133] in, It indicates the rough estimated value of the predicted process index, and Δ indicates the degree of widening, which can be set according to the actual situation and is mainly used to widen range, to estimate The density of data distribution in the interval, N L represents the total number of industrial key process indicator values ​​in offline training data, y i R represents the value of the i-th industrial key process indicator in the offline training data, that is, the true label of the i-th process parameter data with a true label (the true label refers to the true process indicator value corresponding to the process parameter data). q Mainly used for initial instant learning of similar sample sets The number of samples is constrained. The lower the data density, the fewer similar samples are selected. Specifically, R q The key industrial process indicator value is located at The proportion of samples within a certain interval to the total label samples, R q The larger the value, the The more historical samples there are around the value, the more likely it is that the distribution of data under the same operating conditions will be denser.

[0134] It is worth mentioning that the above number K is based on The idea is that the more historical samples there are around, the more likely the distribution of data under the same working condition is to be denser. The final number of similar samples K satisfies the following properties: when the current query sample is in a grade value interval with a large number of samples, it is considered that the distribution of samples under the same working condition around the current query sample is denser, and the K value is larger at this time, and more samples are selected to provide more information for the model. On the contrary, the K value is smaller to avoid learning other working condition information.

[0135] Step 15.2: Subtract each historical sample from the weighted center in the initial instant learning similar sample set to obtain a difference vector set

[0136] The above difference vector set for:

[0137]

[0138] in, Represents a set of difference vectors The i-th difference vector in Represents the i-th historical sample in the initial instant learning similar sample set.

[0139] Step 15.3, calculate the difference vector set The difference vectors and weighted center x q,cThe cosine similarity set C of .

[0140] The above cosine similarity set C is:

[0141]

[0142] Among them, c i express With weighted center x q,c The cosine similarity of i The closer the value is to 1, the closer the change direction of the corresponding historical sample relative to the recent unlabeled data is to the current query sample. express The j-th dimension component of , d represents the total dimension of the component, Represents x q,c The j-th dimension component of .

[0143] Step 15.4, based on the cosine similarity set C, select K historical samples from the initial instant learning similar sample set as the final instant learning similar sample set.

[0144] In some embodiments of the present application, after obtaining the cosine similarity set C, the cosine similarities in the cosine similarity set C can be sorted in descending order; then the K historical samples corresponding to the first K cosine similarities after sorting are used as the final instant learning similarity sample set.

[0145] That is, when performing the second step of screening similar samples, historical samples whose change direction is close to the current query sample are screened out through cosine similarity.

[0146] Step 16, using the final instant learning similar sample set to perform online training on the trained student model to obtain a process indicator prediction model.

[0147] The above online training refers to updating the parameters of the student model after offline training with the final real-time learning similar sample set to fit the distribution of local data. That is, the historical samples of the final real-time learning similar sample set are input into the student model after offline training, and the network parameters are updated using the forward propagation and back propagation algorithms to obtain the adaptive prediction model of industrial key process indicators.

[0148] It should be noted that during the online training process, the loss of the student model can also be calculated by the overall loss method of the offline training process, and the parameters of the student model can be updated based on the loss until the student model converges (the convergence condition can be that the number of training times reaches a preset number, or the loss is less than the loss threshold (such as 0.1)).

[0149] It is worth mentioning that since the performance of the student model after offline training is difficult to maintain stability for a long time, performance degradation will inevitably occur. The historical samples in the final instant learning similar sample set are the samples closest to the current query samples. Therefore, online training based on the final instant learning similar sample set can enable the student model to learn information under the same working conditions as the current query samples, making the model more adaptable to the current local working conditions, so that when it is used to predict the current query samples, it can greatly improve the prediction accuracy of key process indicators.

[0150] Step 17, input the current query sample into the process indicator prediction model to perform process indicator prediction, and obtain the predicted value of the industrial key process indicator at the current prediction moment.

[0151] Specifically, by analyzing and processing the current query sample through the process indicator prediction model, the predicted value of the industrial key process indicator at the current prediction moment can be obtained.

[0152] It should be noted that after parameter fine-tuning (i.e. the above-mentioned online training), the process index prediction model is obtained. This model is discarded after making a prediction for the current query sample. When the next query sample arrives, the parameters are still fine-tuned based on the offline trained student model, rather than the model after the last fine-tuning. Therefore, overall, offline model training can be carried out in a longer cycle, and the training parameters can be adjusted to better fit the global data. In this way, when applied to short-cycle online model updates, the number of iterations can be reduced to a certain extent, improving efficiency.

[0153] It is worth mentioning that the adaptive prediction method for industrial key process indicators in the embodiment of the present application designs a Markov-Cosine two-step similar sample selection strategy, introduces neighboring unlabeled data information and estimates the density of data distribution, and adaptively selects a set of similar samples for fine-tuning offline model parameters. On this basis, the real-time learning framework is introduced to update the parameters of the student model after offline training online. Through the two stages of offline model training and online model update, the long-term accurate prediction of key process indicators is achieved, the prediction performance of key process indicators of dynamic and time-varying industrial processes is improved, and an important basis is provided for the operational adjustment of industrial processes.

[0154] In order to verify the effectiveness of the adaptive prediction method for industrial key process indicators of the embodiment of the present application, the above-mentioned adaptive prediction method for industrial key process indicators is exemplified by taking the potassium salt flotation process as an example.

[0155] In order to evaluate the performance of different predictive modeling methods, the standard root mean square error (RMSE) and relative coefficient (R2) were used to quantitatively evaluate the predictive performance of all methods. The smaller the RMSE value and the larger the R2 value, the better the model prediction performance.

[0156] In this example, the prediction tasks of key process indicators are compared using the local weighted partial least squares algorithm (LWPLS) and the online adaptive fine-tuning deep belief network (OAFDBN) model (both the LWPLS algorithm and the OAFDBN model are typical just-in-time learning prediction models) and the student model trained in the embodiment of the present application (denoted as MT-TSNLNet).

[0157] In order to verify the effectiveness of the proposed online adaptive strategy based on real-time learning, the prediction task of key process indicators is compared using offline MT-TSNLNet and online MT-TSNLNet. The prediction results of the four prediction models are shown as follows: Figure 4 , Figure 5 , Figure 6 , Figure 7 The results of root mean square error (RMSE) and relative coefficient (R2) are shown in Table 1.

[0158] method RMSE <![CDATA[R 2 ]]> LWPLS 0.1299 0.5668 OAFDBN 0.1181 0.6415 Offline MT-TSNLNet 0.0923 0.7810 Online MT-TSNLNet 0.0657 0.8890

[0159] Table 1

[0160] It can be seen from the results that the prediction performance of online MT-TSNLNet is the best, which is caused by multiple factors. Online MT-TSNLNet is based on offline MT-TSNLNet. Since the offline model obtains rich information from a large amount of unlabeled data during training, it has good initialization parameters and prior knowledge. LWPLS builds a new model in real time based on the query sample. There is no prior information that can be used. At the same time, it cannot extract abstract features from the foam image features, resulting in low prediction accuracy. OAFDBN is based on the pre-trained offline model for online fine-tuning, but its offline training and online fine-tuning use labeled data, which limits its performance to a certain extent. At the same time, OAFDBN uses the basic Euclidean distance as a similarity measure during the online fine-tuning process. The proposed Mahalanobis-cosine two-step similar sample selection strategy performs better than the Euclidean distance, which also verifies the effectiveness of this application. Through the above experiments, it is verified that the method of this application is superior to the traditional predictive modeling method in predicting key process indicators in dynamic industrial production processes.

[0161] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An adaptive prediction method for industrial key process indicators based on real-time learning, characterized in that: include: Determine process parameters related to industrial key process indicators and obtain offline training data; The offline training data includes process parameter data at multiple historical moments, and industrial key process indicator values ​​corresponding to the process parameter data at some historical moments; The teacher model is trained using the offline training data to obtain a trained teacher model, and the student model is distilled and learned according to the trained teacher model to obtain a trained student model; the teacher model is a convolutional neural network; The process parameter data of multiple historical moments before the current prediction moment are used as the nearest unlabeled data set of the current query sample; the current query sample is the process parameter data of the current prediction moment; Based on the Mahalanobis distance, an initial instant learning similar sample set is selected from the nearest neighbor unlabeled data set; Using the trained student model to predict the process index of the current query sample, and based on the rough estimated value and cosine similarity of the predicted process index, selecting a final instant learning similar sample set from the initial instant learning similar sample set; Using the final instant learning similar sample set to perform online training on the trained student model to obtain a process indicator prediction model; The current query sample is input into the process indicator prediction model to perform process indicator prediction, and the predicted value of the industrial key process indicator at the current prediction moment is obtained.

2. The prediction method according to claim 1, characterized in that: The method of selecting an initial instant learning similar sample set from the neighbor unlabeled data set based on the Mahalanobis distance includes: Calculating the weighted center of the nearest neighbor unlabeled data set; Calculating a difference vector between the current query sample and the weighted center; Taking the process parameter data of each historical moment in the nearest neighbor unlabeled data set as a historical sample, and calculating the Mahalanobis distance between each historical sample in the nearest neighbor unlabeled data set and the current query sample; A set of historical samples whose Mahalanobis distance is less than the length of the difference vector is used as an initial instant learning similarity sample set.

3. The prediction method according to claim 2, characterized in that: The calculating the weighted center of the neighbor unlabeled data set includes: The weighted center x of the neighbor unlabeled data set is calculated by the weighted center calculation formula q,c ; The weighted center calculation formula is: in, N q,U represents the number of historical samples in the nearest unlabeled dataset, represents the i-th historical sample in the neighbor unlabeled dataset, t q represents the current prediction time, express The corresponding historical moment, σ represents the coordination coefficient.

4. The prediction method according to claim 3, characterized in that: The calculating the difference vector between the current query sample and the weighted center includes: By formula x q,dif =x q -x q,c Calculate the difference vector x between the current query sample and the weighted center q,dif ; Among them, x q Represents the current query sample.

5. The prediction method according to claim 4, characterized in that: The method of selecting a final instant learning similar sample set from the initial instant learning similar sample set based on the rough estimated value of the predicted process indicator and the cosine similarity includes: The rough estimate of the process index obtained by prediction and the number of historical samples in the initial instant learning similar sample set Determine the number K of historical samples contained in the final instant learning similar sample set; Subtract each historical sample in the initial instant learning similar sample set from the weighted center to obtain a difference vector set Calculate the difference vector set The difference vectors in the weighted center x q,c The cosine similarity set C of According to the cosine similarity set C, K historical samples are selected from the initial instant learning similar sample set as the final instant learning similar sample set.

6. The prediction method according to claim 5, characterized in that: The rough estimate of the process indicator obtained according to the prediction and the number of historical samples in the initial instant learning similar sample set Determine the number of historical samples K contained in the final instant learning similar sample set, including: The number of historical samples K included in the final instant learning similar sample set is determined by the following formula: in, represents the rough estimate of the predicted process index, Δ represents the degree of widening, N L represents the total number of industrial key process indicator values ​​in offline training data, y i Represents the i-th industrial key process indicator value in the offline training data.

7. The prediction method according to claim 6, characterized in that: The difference vector set for: in, Represents a set of difference vectors The i-th difference vector in Represents the i-th historical sample in the initial instant learning similar sample set.

8. The prediction method according to claim 7, characterized in that: The cosine similarity set C is: Among them, c i express With the weighted center x q,c The cosine similarity of express The j-th dimension component of , d represents the total dimension of the component, Represents x q,c The j-th dimension component of .

9. The prediction method according to claim 8, characterized in that: The step of selecting K historical samples from the initial instant learning similarity sample set as the final instant learning similarity sample set according to the cosine similarity set C includes: Sort the cosine similarities in the cosine similarity set C in descending order; The K historical samples corresponding to the first K cosine similarities after sorting are used as the final instant learning similarity sample set.