Online prediction method for key quality indicators of industrial processes with multiple sampling rates
Through the multi-particle size completion model, data completion and modeling of multi-sampling rate industrial process data is solved, and the problem of low prediction accuracy of key quality indicators in the multi-sampling rate industrial process is achieved, and the effect of high-precision and long-range multi-step prediction is achieved. .
Patent Information
- Application Number
- CN202211591643.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-12
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-12-12
AI Technical Summary
In the process of processing multi-sampling rate industry, it is difficult for the prior art to accurately predict key quality indicators, resulting in low prediction accuracy, and the completion is separated from the modeling process, and lack of accuracy control.
Using a multi-grained completion model, combining multi-layer convolutional network, sample sampling category coding network, Transformer network and linear output layer, data completion reconstruction of the original input data is carried out, and multi-sampling rate is achieved through the fusion of multi-level correlation relationships. Accurate estimation of unsampled points in the data, and a high-precision index prediction model is constructed.
The prediction accuracy of key quality indicators and the generalization of the model are improved, and accurate modeling of multi-sampling rate data and long-range multi-step real-time prediction are achieved.
Smart Images

Figure CN115963788B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the technical field of online prediction of industrial process quality indicators, and in particular, relates to an online prediction method for key quality indicators of multi-sampling rate industrial processes. Background Art
[0002] In modern industrial processes, it is very important to monitor key quality variables to maintain a safe state of the process and provide effective control and optimization methods. However, due to the harsh measurement environment and expensive analysis costs, it is difficult to measure these important quality variables in real time. Therefore, over time, soft sensing technology has emerged, which uses auxiliary process variables related to key quality variables to build mathematical prediction models to estimate key quality variables that cannot be directly measured. Generally speaking, soft sensing technology can be divided into two categories: first-principle-based soft sensing models and data-driven soft sensing models. With the increase in process complexity, first-principle soft sensing models that require clear reaction mechanisms are difficult to provide accurate prediction performance. On the contrary, data-driven soft sensing models that do not require knowledge of reaction mechanisms and only rely on data characteristics have become the mainstream method in current industrial processes, thanks to the rich historical data resources provided by distributed control systems.
[0003] In recent years, the research on data-driven soft sensor models has achieved many important results in industrial data modeling. However, most of these methods are built on the assumption that industrial processes have a uniform sampling rate, without considering the ubiquitous multi-sampling rate process. In fact, due to the limitations of measurement technology and hardware costs, many variables in actual industrial processes, such as temperature, pressure, flow, composition and concentration, cannot be obtained at a uniform sampling frequency domain rate. At present, the most common approach to this type of problem is to use downsampling technology to align the sampling points of high-frequency sampling variables with those of low-frequency sampling variables to form a complete data set, but this method will lose a lot of known data information and cannot achieve accurate modeling at incomplete sampling moments. In addition, some researchers first use mean filling, linear interpolation and other completion algorithms to upsample and complete multi-sampling rate data sets, and then model on the constructed complete data set. However, the current completion methods are mostly shallow and local, and it is difficult to effectively mine the complex correlations in the data, which makes the final completion effect difficult to achieve the expected effect. In addition, under this modeling approach, completion and modeling are two separate processes. The completion process lacks guidance from the modeling task, while modeling is extremely dependent on the effect of completion. This results in uncontrollable final modeling accuracy and low prediction accuracy of key quality indicators. Summary of the invention
[0004] The embodiment of the present application provides an online prediction method for key quality indicators of a multi-sampling rate industrial process, which can solve the problem of low prediction accuracy of key quality indicators.
[0005] The embodiment of the present application provides a method for online prediction of key quality indicators of a multi-sampling rate industrial process, including:
[0006] Determine a plurality of process variables related to a key quality indicator of an industrial process, and use each of the plurality of process variables as a modeling auxiliary variable;
[0007] Obtaining an auxiliary variable time series set; the auxiliary variable time series set includes a time series composed of multiple samplings of each modeling auxiliary variable within a preset historical time period;
[0008] Sort the time series in the auxiliary variable time series set in descending order of sampling frequency to obtain the original input data;
[0009] The original input data is completed and reconstructed using a multi-granularity completion model to obtain reconstructed original data; the multi-granularity completion model includes a multi-layer convolutional network, a sample sampling category encoding network, a Transformer network encoder, and a linear output layer connected in sequence;
[0010] Using the reconstruction error of the original data as the loss function, the multi-granularity completion model is trained unsupervised to obtain the trained multi-granularity completion model;
[0011] The decoder of the Transformer network is used to process the encoded input data output by the sample sampling category encoding network and the deep features output by the encoder to obtain the final deep features of the original data;
[0012] The final deep features are input into the fully connected layer for processing to obtain the predicted values of the key quality indicators corresponding to the original data;
[0013] Taking the prediction error of the predicted value as the loss function, the indicator prediction model composed of the sequentially connected multi-layer convolutional network, the sample sampling category encoding network, the encoder, the decoder and the fully connected layer is trained to obtain the trained indicator prediction model;
[0014] The values of each modeling auxiliary variable are collected in real time, and the collected values are input into the trained indicator prediction model for indicator prediction to obtain the real-time prediction values of key quality indicators.
[0015] Optionally, a multi-granularity completion model is used to complete and reconstruct the original input data to obtain reconstructed original data, including:
[0016] The original input data is processed using a multi-layer convolutional network to obtain the coarse-grained estimated values of the modeling auxiliary variables at the unsampled points in the preset historical time period, and the coarse-grained completed input data is obtained based on the coarse-grained estimated values and the original input data;
[0017] The sample sampling category encoding network is used to perform sampling category encoding on the input data after the coarse-grained completion to obtain the encoded input data;
[0018] The encoded input data is fed into the encoder of the Transformer network, and the coarse-grained estimate is corrected by the encoder;
[0019] The fine-grained corrected input data is input into the linear output layer for data reconstruction to obtain the reconstructed original data.
[0020] Optionally, the original input data is processed using a multi-layer convolutional network to obtain coarse-grained estimated values of unsampled points of the modeling auxiliary variables within a preset historical time period, and coarse-grained completed input data is obtained based on the coarse-grained estimated values and the original input data, including:
[0021] By formula Get the input data X after coarse-grained completion conv ;
[0022] Among them, MultiConv() represents a multi-layer convolutional network, N C The number of layers of the multi-layer convolutional network is the model hyperparameter, X block represents the original input data, M miss represents the sampling label matrix;
[0023]
[0024] M miss (x i,j ) indicates M miss The element in row i and column j in i,j Observable representation X block There is a sample value in the i-th row and j-th column, x i,j Unsampled means X block There is no sample value in the i-th row and j-th column.
[0025] Optionally, a sample sampling category encoding network is used to perform sampling category encoding on the input data after the coarse-grained completion to obtain encoded input data, including:
[0026] By formula N s =LCM{r1,r2,…,r k ,…r m ,r y}Calculate the least common multiple N s ; LCM{} means to find the least common multiple of the set, r k represents the multiple of the sampling interval of the kth modeling auxiliary variable compared to the benchmark sampling interval, 1≤k≤m, m represents the total number of modeling auxiliary variables, ry Indicates the multiple of the sampling interval of the key quality indicator during the preset historical time period compared with the benchmark sampling interval;
[0027] Build size is N s ×d model The learnable parameter matrix X s ;X s The row vectors in the d model Represents the dimension in which the original input data is mapped to the encoder;
[0028] The parameter matrix X s and the coarse-grained completed input data X conv Add together to get the encoded input data.
[0029] Optional, reconstructed original data X recon The expression is:
[0030]
[0031] Among them, Linear() represents the linear output layer, Encoder() represents the encoder of the Transformer network, and N e The number of encoder layers is a network hyperparameter, and Embedding() represents the sample sampling category encoding network.
[0032] Optionally, the decoder of the Transformer network is used to process the encoded input data output by the sample sampling category encoding network and the deep features output by the encoder to obtain the final deep features of the original data, including:
[0033] The encoded input data output by the sample sampling category encoding network is used as the query matrix of the decoder of the Transformer network, and the deep features output by the encoder are used as the key-value pair matrix of the decoder;
[0034] By formula Get the final deep feature X of the original data hidden ;
[0035] in, Decoder() represents the decoder, N d The number of layers of the decoder is a network hyperparameter, Q represents the query matrix that the decoder needs to input, K represents the key matrix that the decoder needs to input, V represents the value matrix that the decoder needs to input, and Linear Q () represents the linear mapping layer corresponding to Q, Linear K () indicates the linear mapping layer corresponding to K, Linear V() represents the linear mapping layer that generates V.
[0036] Optionally, the final deep features are input into the fully connected layer for processing to obtain the predicted values of key quality indicators corresponding to the original data, including:
[0037] By formula Get the predicted value of the key quality indicator corresponding to the original data;
[0038] Among them, W represents the weight of the fully connected layer, b represents the bias of the fully connected layer, t represents the prediction time, and t p represents the end time of multi-step prediction, p represents the length of multi-step prediction window, Represents the multi-step prediction value of the key quality indicator from time t to t+p.
[0039] The above solution of the present application has the following beneficial effects:
[0040] In an embodiment of the present application, since the multi-granularity completion model for data completion and reconstruction of the original input data is composed of a multi-layer convolutional network, a sample sampling category encoding network, an encoder of a Transformer network, and a linear output layer, the multi-granularity completion model can fully mine the variable-level local correlation and sample-level long-range correlation existing in the multi-sampling rate data, and achieve accurate estimation of unsampled points in the multi-sampling rate data through the fusion and mutual complementation of multi-level correlations, thereby making the final constructed indicator prediction model have high prediction accuracy and good generalization, thereby achieving the effect of improving the prediction accuracy of key quality indicators.
[0041] In addition, on this basis, combined with the Transformer network's ability to model long-range evolution laws, we can make full use of the information contained in the data and achieve long-range, multi-step real-time prediction of key quality indicators.
[0042] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0044] Figure 1 A flowchart of a method for online prediction of key quality indicators of a multi-sampling rate industrial process provided by an embodiment of the present application;
[0045] Figure 2A schematic diagram of the structure of an encoder and a decoder is provided for an embodiment of the present application;
[0046] Figure 3 A simplified flow chart of hydrocracking provided in one embodiment of the present application;
[0047] Figure 4 A schematic diagram of a multi-sampling rate Transformer network structure provided in an embodiment of the present application;
[0048] Figure 5 A comparison curve of the predicted value and the actual value of the isopentane content in light naphtha by the LSTNet model at two sampling rates provided in an example of this application;
[0049] Figure 6 A comparison curve diagram of the predicted value and the actual value of the isopentane content in light naphtha by the LogTrans model at two sampling rates provided in an example of this application;
[0050] Figure 7 A comparison curve diagram of the predicted value and the actual value of the isopentane content in light naphtha by the Informer model at two sampling rates provided in an example of this application;
[0051] Figure 8 This is a comparison curve chart of the predicted value and true value of the isopentane content in light naphtha using a multi-sampling rate Transformer model at two sampling rates provided in an example of this application. DETAILED DESCRIPTION
[0052] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0053] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0054] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0055] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0056] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0057] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0058] In response to the current problem of low prediction accuracy of key quality indicators, the present application provides an online prediction method for key quality indicators of multi-sampling rate industrial processes. The method uses a multi-granularity completion model composed of a multi-layer convolutional network, a sample sampling category encoding network, a Transformer network encoder and a linear output layer to complete and reconstruct the original input data, so that the multi-granularity completion model can fully explore the variable-level local correlation and sample-level long-range correlation in the multi-sampling rate data, and achieve accurate estimation of unsampled points in the multi-sampling rate data through the fusion and mutual complementation of multi-level correlations, thereby making the final constructed indicator prediction model have high prediction accuracy and good generalization, thereby achieving the effect of improving the prediction accuracy of key quality indicators.
[0059] The following is an exemplary description of the multi-sampling rate industrial process key quality indicator online prediction method provided by the present application in conjunction with specific embodiments.
[0060] The embodiment of the present application provides a method for online prediction of key quality indicators of a multi-sampling rate industrial process, which can be executed by a terminal device or by a device (such as a chip) applied to the terminal device. The following embodiment takes the method executed by a terminal device as an example. As an example, the terminal device can be a tablet, a server, or a laptop computer, etc., which is not limited in the embodiment of the present application.
[0061] like Figure 1 As shown, the multi-sampling rate industrial process key quality indicator online prediction method provided by the embodiment of the present application includes the following steps:
[0062] Step 11, determining a plurality of process variables related to key quality indicators of the industrial process, and using each of the plurality of process variables as a modeling auxiliary variable.
[0063] In some embodiments of the present application, several process variables that have an impact on key quality indicators of the industrial process may be selected as auxiliary modeling variables based on process mechanisms and operating experience.
[0064] Step 12: Obtain an auxiliary variable time series set, where the auxiliary variable time series set includes a time series consisting of multiple samplings of each modeling auxiliary variable within a preset historical time period.
[0065] In some embodiments of the present application, the above-mentioned preset historical time period may be a historical time period set according to actual needs, such as January 10, 2022 to August 9, 2022. For the convenience of subsequent description, the preset historical time period is recorded as 1 to T.
[0066] The above time series set It is obtained by collecting all modeling auxiliary variables several times at different sampling frequencies within the time from 1 to T. In order to facilitate the training of the subsequent model, while collecting all modeling auxiliary variables, The key quality indicators are collected for the sampling interval to obtain the time series of the key quality indicators Then we get the quality index time series set Specifically, The expression is as follows:
[0067]
[0068] in, represents the benchmark sampling interval (generally, the sampling interval of the full sampling variable is used as the benchmark sampling interval), m represents the number of auxiliary variables for modeling, Indicates that the kth modeling auxiliary variable is (r k and r yare all positive integers) is a time series composed of multiple samplings at the sampling interval. If At this moment, the auxiliary variable of the model has no sample value. i is a positive integer.
[0069] It is understandable that in order to facilitate data processing, after obtaining the time series set Afterwards, you can Perform data normalization preprocessing to obtain the processed time series set And the processed In As the auxiliary variable time series set in step 12. Among them, the combination Ω of the sampling frequency (i.e., sampling rate) of each modeling auxiliary variable is: The sampling frequency of the key quality indicators is
[0070] In some embodiments of the present application, The method of data normalization preprocessing is as follows:
[0071] Through the following formula The value x in row i and column j is i ' ,j Perform normalization:
[0072]
[0073] Through the following formula The i-th value y in i 'Normalization processing:
[0074]
[0075] Among them, x i,j Represents x i ' ,j The normalized value, x max,j and x min,j Respectively represent the maximum and minimum values of the j-th modeling auxiliary variable, y i Represents y i ′The normalized value, y max and min Respectively represent the maximum and minimum values of key quality indicators.
[0076] Step 13, sort the time series in the auxiliary variable time series set in descending order of sampling frequency to obtain the original input data.
[0077] In some embodiments of the present application, the sampling frequencies of the same type of modeling auxiliary variables are the same. To improve the accuracy of subsequent data completion, when sorting the time series in the auxiliary variable time series set in order from high to low sampling frequencies, the time series corresponding to the modeling auxiliary variables with the same sampling frequencies can be arranged together.
[0078] Step 14: Use the multi-granularity completion model to complete and reconstruct the original input data to obtain reconstructed original data.
[0079] The multi-granularity completion model includes a multi-layer convolutional network, a sample sampling category encoding network, an encoder of a Transformer network, and a linear output layer connected in sequence. Among them, the multi-layer convolutional network is mainly used to use its own local spatiotemporal feature capture ability to preliminarily obtain a coarse-grained estimate of the unsampled point data, the sample sampling category encoding network is mainly used to convert the data output by the multi-layer convolutional network into data that can be recognized by the encoder of the Transformer network, and the encoder is mainly used to perform fine-grained corrections on the coarse-grained estimate of the unsampled data points. The linear output layer is mainly used to reconstruct the deep features output by the encoder to obtain the reconstructed original data. Among them, the above-mentioned linear output layer can be specifically a fully connected layer.
[0080] It is worth mentioning that the above-mentioned multi-granularity completion model can fully explore the variable-level local correlation and sample-level long-range correlation in multi-sampling rate data, and realize accurate estimation of unsampled points in multi-sampling rate data through the fusion and mutual complementation of multi-level correlations, thereby making the final constructed indicator prediction model have high prediction accuracy and good generalization.
[0081] Step 15, using the reconstruction error of the original data as the loss function, unsupervised training is performed on the multi-granularity completion model to obtain a trained multi-granularity completion model.
[0082] In some embodiments of the present application, the reconstruction error of the original data can be used as the loss function, and a suitable optimizer, learning rate, number of iterations, etc. can be selected to update the weight parameters of the multi-granularity completion model in an unsupervised manner until the model converges to obtain a trained multi-granularity completion model.
[0083] Step 16, using the decoder of the Transformer network to process the encoded input data output by the sample sampling category encoding network and the deep features output by the encoder to obtain the final deep features of the original data.
[0084] In some embodiments of the present application, the decoder of the above-mentioned Transformer network is mainly used to further extract relevant relationships and output the final deep features of the original data after the multi-granularity completion is completed.
[0085] Step 17: Input the final deep features into the fully connected layer for processing to obtain the predicted values of the key quality indicators corresponding to the original data.
[0086] In some embodiments of the present application, the above-mentioned fully connected layer is mainly used to predict the key quality indicator using the final deep features output by the decoder, and output the predicted value of the key quality indicator.
[0087] Step 18, taking the prediction error of the predicted value as the loss function, train the indicator prediction model composed of the sequentially connected multi-layer convolutional network, sample sampling category encoding network, encoder, decoder and full connection layer to obtain the trained indicator prediction model.
[0088] In some embodiments of the present application, the prediction error of the predicted value can be used as the loss function, and a suitable optimizer, learning rate, number of iterations, etc. are selected to update the weight parameters of the entire indicator prediction model in a supervised reverse manner until the model converges to obtain a trained indicator prediction model.
[0089] Step 19, collect the value of each modeling auxiliary variable in real time, and input the collected value in real time into the trained indicator prediction model to perform indicator prediction, so as to obtain the real-time prediction value of the key quality indicator.
[0090] In some embodiments of the present application, after obtaining the trained indicator prediction model, the numerical values of all modeling auxiliary variables collected in real time can be input into the trained indicator prediction model to perform real-time prediction of key quality indicators.
[0091] It is worth mentioning that in some embodiments of the present application, since the multi-granularity completion model for data completion and reconstruction of the original input data is composed of a multi-layer convolutional network, a sample sampling category encoding network, a Transformer network encoder and a linear output layer, the multi-granularity completion model can fully explore the variable-level local correlation and sample-level long-range correlation in the multi-sampling rate data, and accurately estimate the unsampled points in the multi-sampling rate data through the fusion and mutual complementation of multi-level correlations, thereby making the final constructed indicator prediction model have high prediction accuracy and good generalization, thereby achieving the effect of improving the prediction accuracy of key quality indicators.
[0092] In addition, on this basis, combined with the Transformer network's ability to model long-range evolution laws, we can make full use of the information contained in the data and achieve long-range, multi-step real-time prediction of key quality indicators.
[0093] The following is an exemplary description of the data completion and reconstruction of the original input data in conjunction with a specific embodiment.
[0094] In some embodiments of the present application, the above step 14, using the multi-granularity completion model to complete and reconstruct the original input data to obtain the reconstructed original data, includes the following steps:
[0095] Step 1: Use a multi-layer convolutional network to process the original input data to obtain the coarse-grained estimated values of the modeling auxiliary variables at the unsampled points in the preset historical time period, and obtain the coarse-grained completed input data based on the coarse-grained estimated values and the original input data.
[0096] In some embodiments of the present application, the formula Get the input data X after coarse-grained completion conv .
[0097] Among them, MultiConv() represents a multi-layer convolutional network, N C The number of layers of the multi-layer convolutional network is the model hyperparameter, X block represents the original input data, M miss represents the sampling label matrix.
[0098]
[0099] M miss (x i,j ) indicates M miss The element in row i and column j in i,j Observable representation X block There is a sample value in the i-th row and j-th column, x i,j Unsampled means X block There is no sample value in the i-th row and j-th column.
[0100] Step 2: Use the sample sampling category encoding network to perform sampling category encoding on the coarse-grained completed input data to obtain the encoded input data.
[0101] In some embodiments of the present application, a specific implementation method of obtaining the encoded input data includes the following steps:
[0102] The first step is to use the formula N s =LCM{r1,r2,…,r k ,…r m ,r y}Calculate the least common multiple N s .
[0103] Among them, LCM{} represents the least common multiple of the solution set, r k represents the multiple of the sampling interval of the kth modeling auxiliary variable compared to the benchmark sampling interval, 1≤k≤m, m represents the total number of modeling auxiliary variables, r yIndicates the multiple of the sampling interval of the key quality indicator during the preset historical time period compared with the benchmark sampling interval.
[0104] The second step is to build a size N s ×d model The learnable parameter matrix X s .
[0105] Among them, X s The row vectors in the d model Represents the dimensions that the original input data is mapped to in the encoder.
[0106] The third step is to transform the parameter matrix X s and the coarse-grained completed input data X conv Add together to get the encoded input data.
[0107] It is worth mentioning that the above-mentioned sample sampling category encoding network can be extended to any multi-sampling rate model to assist it in identifying the sampling conditions of samples at different times and improve modeling accuracy.
[0108] Step 3: Input the encoded input data into the encoder of the Transformer network, and use the encoder to make fine-grained corrections to the coarse-grained estimate.
[0109] Step 4: Input the fine-grained corrected input data into the linear output layer for data reconstruction to obtain the reconstructed original data.
[0110] In some embodiments of the present application, the reconstructed original data X recon The expression is:
[0111]
[0112] Among them, Linear() represents the linear output layer, Encoder() represents the encoder of the Transformer network, and N e The number of encoder layers is a network hyperparameter, and Embedding() represents the sample sampling category encoding network.
[0113] Based on the above reconstruction process of the original data, the expression of the reconstruction error J1 of the original data in the above step 15 can be: Among them, x i,j and Respectively represent the original sampled value and model reconstructed value of the jth modeling auxiliary variable at the i-th sampling moment, and N represents the number of sampling moments. The above loss is used to describe the reconstruction accuracy of the model for observable points in multi-granularity completion. This is because the data of unsampled points in the actual process cannot be collected. Therefore, the reconstruction error of the model for observable points represents the performance of the model multi-granularity completion. In each iteration of the training process, the calculated loss function is used to back-propagate and update the network parameters until the model (multi-granularity completion model) converges.
[0114] The final deep feature extraction process is exemplarily described below in conjunction with a specific embodiment.
[0115] In some embodiments of the present application, the above step 16, using the decoder of the Transformer network to process the encoded input data output by the sample sampling category encoding network and the deep features output by the encoder to obtain the final deep features of the original data, is specifically implemented as follows:
[0116] First, the encoded input data output by the sample sampling category encoding network is used as the query matrix of the decoder of the Transformer network, and the deep features output by the encoder are used as the key-value pair matrix of the decoder; then, the formula Get the final deep feature X of the original data hidden .
[0117] in, Decoder() represents the decoder, N d The number of layers of the decoder is a network hyperparameter, Q represents the query matrix that the decoder needs to input, K represents the key matrix that the decoder needs to input, V represents the value matrix that the decoder needs to input, and Linear Q () represents the linear mapping layer corresponding to Q, Linear K () indicates the linear mapping layer corresponding to K, Linear V () represents the linear mapping layer that generates V.
[0118] The following is an exemplary description of the process of obtaining the predicted value in conjunction with a specific embodiment.
[0119] In some embodiments of the present application, the above step 17, inputting the final deep features into the fully connected layer for processing, and obtaining the predicted value of the key quality indicator corresponding to the original data is specifically implemented as follows: Get the predicted values of key quality indicators corresponding to the original data.
[0120] Among them, W represents the weight of the fully connected layer, b represents the bias of the fully connected layer, and t represents the prediction time. i is a positive integer, t p represents the end time of multi-step prediction, p represents the multi-step prediction window length, Represents the multi-step prediction value of the key quality indicator from time t to t+p.
[0121] In some embodiments of the present application, the above p can be set according to actual needs. is the value of the key quality indicator from time t to t+p.
[0122] For example, when using the trained indicator prediction model to perform real-time prediction of key quality indicators, assuming that p is 5 steps and the reference interval is If the prediction time is 1 minute, then the key quality indicator value in the next 5 minutes is predicted.
[0123] Based on the above final deep features and the process of obtaining the predicted value, the expression of the prediction error of the predicted value in the above step 18 is:
[0124]
[0125] Among them, y i , Respectively represent i(t≤i≤t p ) at the moment of the actual value of the key quality indicator and the model prediction value, X i , Respectively represent i(t≤i≤t p ) moment modeling the true value and reconstructed value of the auxiliary variable, M miss (y i ) represents the observable item in the key quality indicator, 1-Μ miss (X i ) represents the observable item in the modeling auxiliary variables. The above loss includes two losses: the prediction error of the observable quality indicator and the reconstruction error of the observable auxiliary variables. λ represents the weight coefficient, which is used to balance the importance of the two losses. Then, the obtained loss function is back-propagated to update the parameters of the entire network again until convergence.
[0126] Among them, the specific structures of the encoder and decoder in the trained indicator prediction model are as follows: Figure 2 As shown in the figure, since both of them are structures in the traditional Transformer network, their structures and principles will not be described in detail here.
[0127] In some embodiments of the present application, in order to improve the accuracy of prediction, before using the trained indicator prediction model to perform real-time prediction of key quality indicators, the collected test sample set (which can be The prediction and generalization performance of the trained indicator prediction model is verified by using partial data in the training data. After the verification is qualified, the values of the modeling auxiliary variables collected in real time are used to predict the future key quality indicator values, so as to realize the real-time online prediction of the key quality indicators. On the contrary, if the verification is unqualified, repeat step 18 to retrain the model until the indicator prediction model is verified to be qualified.
[0128] Among them, the specific implementation method of using the collected test sample set to verify the prediction and generalization performance of the trained indicator prediction model is as follows:
[0129] Assume that the normalized and reorganized test sample set It can be expressed as follows:
[0130]
[0131] in, represents the auxiliary variable time series set in the test sample set, Indicates that the kth modeling auxiliary variable in the test sample set is is a time series composed of multiple samplings at the sampling interval, represents the time series set of quality indicators in the test sample set, It shows that the key quality indicators of the test sample set are in the period from T1 to T2. is a time series composed of multiple sampling intervals, T1 and T2 represent the start time and end time of test sample collection respectively. Then, the prediction output on the test set is Available from the trained model:
[0132]
[0133] Among them, (X t ) test , They represent the test input at time t (T1≤t≤T2) and the corresponding key quality indicator prediction value, respectively. Multirate-Former () represents the trained indicator prediction model, and the generalization error of the model can be obtained by comparing the key quality indicator values predicted by the model with the actual key quality indicator values of the test set. The generalization error is calculated as follows:
[0134]
[0135]
[0136] Among them, MAE represents the mean absolute error, RMSE represents the root mean square error, (y i,j ) test , They represent the actual value and model prediction value of the key quality indicator of the i-th test sample in the j-th step within the p-step prediction window, respectively. test Indicates the number of test samples.
[0137] It is worth mentioning that the online prediction method for key quality indicators of multi-sampling rate industrial processes (referred to as Multirate-Former) provided in the embodiment of the present application fully mines the complex multi-level correlations contained in multi-sampling rate data through a strategy that combines multi-granularity completion with correlation modeling. It can not only realize the accurate modeling of key quality indicators of multi-sampling rate industrial processes, but also obtain accurate estimates of unsampled points; in addition, the sampling category coding structure used to identify the sampling status of samples at different times is universal, which enriches the multi-sampling rate industrial process processing means and improves the applicability and accuracy of the soft measurement model.
[0138] The following is a hydrocracking process (hydrocracking process such as Figure 3 ) is used as an example to illustrate the above-mentioned multi-sampling rate industrial process key quality indicator online prediction method.
[0139] Specifically, you can use Figure 4 The multi-sampling rate Transformer network structure shown executes the above multi-sampling rate industrial process key quality indicator online prediction method, which includes the following steps:
[0140] S1: 43 variables that affect the isopentane content of light naphtha in the hydrocracking process (as shown in Table 1) were selected as input variables, and a total of 2,600 labeled samples from December 3, 2015 to December 28, 2018 were extracted.
[0141] Table 1
[0142]
[0143]
[0144] S2: Data collected in step S1 Normalized data preprocessing is performed in the time series after processing And get the sampling rate of each variable Ω represents the combination of the sampling rates of all variables, where the normalized calculation formula is as follows:
[0145]
[0146]
[0147] Where x min,j and x max,jRespectively represent the minimum and maximum values of the jth process variable, y min and max They represent the minimum and maximum values of the quality variable (i.e. the isopentane content of light naphtha), respectively.
[0148] The first 2300 samples are used as training sets to train the model and find the optimal parameter combination, and the last 300 samples are used as test sets for subsequent testing of the generalization performance of the trained model.
[0149] S3: The sequence obtained in S2 The variables are rearranged in blocks, and the variables with the same sampling rate are arranged together. The model input data is reconstructed from high to low sampling rates to obtain the new input data X block .
[0150] S4: Construct a multi-granularity completion model for unsampled data points. First, use the local spatiotemporal feature capture capability of the multi-layer convolutional network to preliminarily obtain the coarse-grained estimate of the unsampled data points. Then, assign the sampling category code to the preliminary completion data and input it into the Transformer encoder for sample-level similarity capture to further fine-grainedly correct the coarse-grained estimate. Finally, use the reconstruction error of the sampling point data as the loss function to unsupervisedly update the weight parameters of the multi-granularity completion network. S4 specifically includes:
[0151] The following multi-layer convolutional network structure is used to obtain the coarse-grained estimate of the unsampled points:
[0152] X c o nv =MultiConv NC (X bl o ck )*M miss (X bl o ck )+X bl o ck
[0153]
[0154] Where X conv is the input data after coarse-grained completion, M miss is the sampling label matrix, N C It indicates that the number of convolutional network layers is a model hyperparameter, and MultiConv() is a multi-layer convolutional network.
[0155] On this basis, the coarse-grained estimate is first sampled and coded:
[0156] First, calculate the least common multiple N of all variables compared to the benchmark sampling interval s :
[0157] N s =LCM{r1,r2,…,r k ,…r m ,r y}
[0158] Where LCM{} means to find the least common multiple of the set; then construct a set of size N s ×d model The learnable parameter matrix X s , you need to ensure X when building s The row vectors in are different from each other; the obtained X s Matrix and data matrix X conv Adding them together gives the encoded data.
[0159] Then, the encoder of the Transformer network is used to make fine-grained corrections to the coarse-grained estimates of the unsampled data points and to the original input data X block To refactor:
[0160]
[0161] Among them, Linear() and Encoder() are the linear output layer and Transformer encoder respectively, Embedding() is the sample encoding layer, and X recon is the reconstruction of the original data, N e Indicates the number of encoder layers as a network hyperparameter.
[0162] Then, the reconstruction error of the sampling point data is used as the loss function, and the appropriate optimizer, learning rate, and number of iterations are selected to update the weight parameters of the multi-granularity completion network in an unsupervised manner:
[0163]
[0164] where x i,j and Represent the original sampled value and model reconstructed value of the jth variable at the i-th sampling moment. The above loss is used to describe the reconstruction accuracy of the model for observable points in multi-granularity completion. This is because the data of unsampled points in the actual process cannot be collected. Therefore, the reconstruction error of the model for observable points represents the performance of the model multi-granularity completion. In each iteration of the training process, the calculated loss function is used to back-propagate and update the network parameters until the model converges.
[0165] S5: After multi-granular completion is completed, the encoded data is used as the query matrix of the Transformer network decoder, and the deep features generated by the encoder are used as the key-value pair matrix of the decoder to further extract relevant relationships:
[0166]
[0167] Q=Linear Q (Embedding(X conv ))
[0168]
[0169]
[0170] Where X hidden Represents the final deep features, Decoder() represents the Transformer decoder, N d The number of decoder layers is a network hyperparameter. Q, K, and V represent the query vector, key vector, and value vector required to be input to the Transformer encoder, respectively. They are the features Embedding (X) obtained by the embedding layer after the coarse-grained completion data. conv ) and the features obtained after fine-grained completion (Embedding(X conv )) Linear mapping generates, Linear Q (), Linear K () and Linear V () represent the linear mapping layers corresponding to Q, K, and V respectively.
[0171] S6: Finally, the deep features obtained by the decoder are input into a fully connected layer to obtain the predicted value of the key quality indicator, and the prediction error is used as the loss function. The appropriate optimizer, learning rate, and number of iterations are selected to update the weight parameters of the entire network in a supervised reverse manner:
[0172]
[0173] Where W and b are the weight and bias of the fully connected layer respectively. (1≤t≤T, i is a positive integer) is the current time, (1≤t p ≤T) is the end time of multi-step prediction, p is the length of multi-step prediction window, It represents the multi-step prediction value of the key quality indicator from time t to t+p, where the model prediction loss function is expressed as follows:
[0174]
[0175] where y i , Respectively represent i(t≤i≤t p ) The true value of the quality variable and the model predicted value, X i , Respectively represent i(t≤i≤t p ) at the moment . The above loss includes two losses: the prediction error of the observable quality indicator and the reconstruction error of the observable auxiliary variable. λ is used to balance the importance of the two losses. Then the loss function is back-propagated to update the parameters of the entire network again until convergence.
[0176] S7: Load the weights obtained in step S6, and use the collected test sample set to verify the prediction and generalization performance of the trained model. After the verification is qualified, the real-time collected process variable values can be used to predict the future key quality indicator values, thereby realizing real-time online prediction of key quality indicators. Otherwise, return to S6 to retrain the model.
[0177] The standard mean absolute error (MAE) and root mean square error (RMSE) are used to evaluate the prediction accuracy of the obtained soft sensor model, which are calculated as follows:
[0178]
[0179]
[0180] Where (y i,j ) test , They respectively represent the actual value and model predicted value of the quality variable of the i-th test sample in the j-step prediction window.
[0181] Among them, the prediction results of the key quality indicators are respectively predicted by using the long-term and short-term time series network (LSTNet) model, the LogTrans model, the Informer model (the LogTrans model and the Informer model are both commonly used multi-step prediction models) and the online prediction method of this application (i.e., the multi-sampling rate Transformer model) as shown below: Figure 5 , 6 , 7, and 8, and the standard mean absolute error (MAE) and root mean square error (RMSE) are shown in Table 2. It can be seen that the multi-sampling rate industrial process key quality indicator prediction modeling method based on the Transformer network provided by this application can achieve more accurate predictions than traditional time series prediction methods such as the Informer model, the LogTrans model, and the LSTNet model, which also verifies the effectiveness of this application. From the results, it can also be seen that whether it is increasing the prediction step size or increasing the type of sampling frequency, the method proposed in this application can achieve the best results.
[0182] Table 2
[0183]
[0184] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A multi-sampling rate industrial process key quality indicator online prediction method, characterized in that: include: Determine a plurality of process variables related to a key quality indicator of an industrial process, and use each of the plurality of process variables as a modeling auxiliary variable; Obtaining an auxiliary variable time series set; the auxiliary variable time series set includes a time series composed of multiple samplings of each modeling auxiliary variable within a preset historical time period; Sorting the time series in the auxiliary variable time series set in descending order of sampling frequency to obtain original input data; The original input data is reconstructed by using a multi-granularity completion model to complete the data, thereby obtaining reconstructed original data; the multi-granularity completion model includes a multi-layer convolutional network, a sample sampling category encoding network, an encoder of a Transformer network, and a linear output layer connected in sequence; Using the reconstruction error of the original data as a loss function, performing unsupervised training on the multi-granularity completion model to obtain a trained multi-granularity completion model; Using the decoder of the Transformer network to process the encoded input data output by the sample sampling category encoding network and the deep features output by the encoder to obtain the final deep features of the original data; Inputting the final deep features into a fully connected layer for processing to obtain predicted values of key quality indicators corresponding to the original data; Taking the prediction error of the predicted value as the loss function, training the indicator prediction model composed of the multi-layer convolutional network, the sample sampling category encoding network, the encoder, the decoder and the fully connected layer connected in sequence to obtain a trained indicator prediction model; The numerical value of each modeling auxiliary variable is collected in real time, and the numerical value collected in real time is input into the trained indicator prediction model for indicator prediction to obtain the real-time prediction value of the key quality indicator.
2. The method according to claim 1, characterized in that The method of using the multi-granularity completion model to complete and reconstruct the original input data to obtain the reconstructed original data includes: The original input data is processed by using a multi-layer convolutional network to obtain a coarse-grained estimated value of the modeling auxiliary variable at an unsampled point within the preset historical time period, and coarse-grained completed input data is obtained based on the coarse-grained estimated value and the original input data; Using the sample sampling category encoding network to perform sampling category encoding on the coarse-grained completed input data to obtain encoded input data; Inputting the encoded input data into the encoder of the Transformer network, and performing fine-grained correction on the coarse-grained estimate through the encoder; The fine-grained corrected input data is input into the linear output layer for data reconstruction to obtain the reconstructed original data.
3. The method according to claim 2, characterized in that The method of processing the original input data by using a multi-layer convolutional network to obtain a coarse-grained estimated value of the modeling auxiliary variable at an unsampled point within the preset historical time period, and obtaining coarse-grained completed input data according to the coarse-grained estimated value and the original input data, includes: By formula Get the input data X after coarse-grained completion conv ; Among them, MultiConv(·) represents a multi-layer convolutional network, N C The number of layers of the multi-layer convolutional network is a model hyperparameter, X block represents the original input data, M miss represents the sampling label matrix; M miss (x i,j ) indicates M miss The element in row i and column j in i,j Observable representation X block There is a sample value in the i-th row and j-th column, x i,j Unsampled means X block There is no sample value in the i-th row and j-th column.
4. The method according to claim 3, characterized in that The step of using the sample sampling category encoding network to perform sampling category encoding on the coarse-grained completed input data to obtain the encoded input data includes: By formula N s =LCM{r1,r2,…,r k ,…r m ,r y }Calculate the least common multiple N s ; LCM{·} means to find the least common multiple of the set, r k represents the multiple of the sampling interval of the kth modeling auxiliary variable compared to the benchmark sampling interval, 1≤k≤m, m represents the total number of modeling auxiliary variables, r y Indicates the multiple of the sampling interval of the key quality indicator during the preset historical time period compared with the benchmark sampling interval; Build size is N s ×d model The learnable parameter matrix X s ;X s The row vectors in the d model represents the dimension in which the original input data is mapped to the encoder; The parameter matrix X s and the coarse-grained completed input data X conv Add together to get the encoded input data.
5. The method according to claim 3, characterized in that: Reconstructed original data X recon The expression is: Wherein, Linear(·) represents the linear output layer, Encoder(·) represents the encoder of the Transformer network, and N e The number of layers of the encoder is a network hyperparameter, and Embedding(·) represents the sample sampling category encoding network.
6. The method according to claim 5, characterized in that The decoder of the Transformer network is used to process the encoded input data output by the sample sampling category encoding network and the deep features output by the encoder to obtain the final deep features of the original data, including: Using the encoded input data output by the sample sampling category encoding network as a query matrix of a decoder of the Transformer network, and using the deep features output by the encoder as a key-value pair matrix of the decoder; By formula Get the final deep feature X of the original data hidden ; Where Q = Linear Q (Embedding(X conv )), Decoder(·) represents the decoder, N d Indicates that the number of layers of the decoder is a network hyperparameter, Q represents the query matrix required to be input by the decoder, K represents the key matrix required to be input by the decoder, V represents the value matrix required to be input by the decoder, Linear Q (·) represents the linear mapping layer that generates Q. K (·) represents the linear mapping layer that generates K. V (·) represents the linear mapping layer that generates V.
7. The method according to claim 6, characterized in that The step of inputting the final deep features into the fully connected layer for processing to obtain the predicted values of the key quality indicators corresponding to the original data includes: By formula Obtaining predicted values of key quality indicators corresponding to the original data; Where W represents the weight of the fully connected layer, b represents the bias of the fully connected layer, t represents the prediction time, and t p represents the end time of multi-step prediction, p represents the length of multi-step prediction window, Represents the multi-step prediction value of the key quality indicator from time t to t+p.
