A method and system for quality prediction based on spatio-temporal structure preserving deep network

Abstract

The application discloses a quality prediction method and system based on a space-time structure preserving deep network, which considers the time and space structure features between input data in the process of layer-by-layer feature extraction to improve the accuracy of quality prediction. The loss function used in pre-training comprehensively considers input reconstruction, quality-related feature extraction, and space-time structure preservation. Then, an output fusion layer is designed to fully utilize the output estimates given by the shallow-to-deep space-time structure preserving encoder, and the corresponding parameters are initialized by a least squares regression algorithm before fine-tuning. In addition, the application also discloses an improved scheme related to using variable selection, orthogonal signal correction processing and other technologies to improve the quality prediction accuracy of the model. Finally, the application designs a quality prediction system based on the same concept, which combines software and hardware to perform the implementation process including pre-training, fine-tuning, and implementing online quality prediction tasks.

Description

Technical Field

[0001] This invention belongs to the field of data-driven soft measurement technology, and specifically relates to a quality prediction method and system based on a spatiotemporally structure-preserving deep network. Background Technology

[0002] In real-world process industry production scenarios, real-time measurement of quality-related variables is a crucial prerequisite for improving product quality and optimizing process control. However, due to technological or economic limitations, some quality variables cannot be measured in real time using hardware sensors and must be measured manually offline in a laboratory, leading to significant time lags. Against this backdrop, soft measurement technology has emerged. It utilizes easily measurable process variables as inputs and quality variables as outputs, estimating target variables by constructing mathematical models describing the system's input-output behavior. With the widespread adoption of distributed measurement systems and the development of computer storage and computing power, data-driven approaches have become the mainstream development direction for soft measurement. With the advent of Industry 4.0, combining advanced machine learning techniques with soft measurement technology offers new avenues for improving process efficiency and decision-making. Among various available machine learning methods, deep learning, with its significant advantages in hierarchical data representation, demonstrates enormous potential for capturing the complex nonlinear relationships between process and quality variables.

[0003] Chinese invention patent application number / patent number 202411516430.9 discloses a soft measurement method based on a supervised gated recurrent unit deep network. This method uses quality information as additional input to two gates and intermediate units to guide the learning of the hidden states and cell states of the gated recurrent unit, achieving soft measurement modeling of the penicillin fermentation process. Chinese invention patent application number / patent number 202510012580.4 discloses a soft measurement method for industrial processes using a multi-module self-attention gated network. This method uses self-attention to fuse deep features and combines dynamic weighting with a gating mechanism to implement real-time prediction of industrial process quality variables. Furthermore, Chinese invention patent application number / patent number 202510030107.9 discloses a soft measurement method and apparatus for industrial processes using latent predictability embedding deep learning. This method constructs a deep network model by embedding a target-related autoencoder into a probabilistic latent predictability model to perform quality prediction.

[0004] A paper published in the journal *Expert Systems with Applications* in 2026, titled "GF-DBiNN: Gated fusion-based deep bi-channel neural network for industrial softsensor modeling," discloses a soft sensor method based on a deep bi-channel neural network model with a gating mechanism. Additionally, a paper published in the journal *Control and Decision* in 2025, titled "Soft Sensor Modeling of a Dual-Stream Memory-Regulated GRU Based on Two-Stage Attention," discloses a soft sensor method based on a dual-stream memory-regulated gated recurrent unit with an embedded two-stage attention mechanism, used to solve the problem of real-time prediction of sulfur dioxide concentration in flue gas emitted during the desulfurization process of thermal power plants.

[0005] While deep learning-based quality prediction techniques have achieved satisfactory results in many real-time quality variable prediction tasks, these soft measurement modeling methods still face some significant challenges in practical industrial applications. For example, these deep learning methods capture the hierarchical structure features of data by minimizing the global fitting error, but they neglect the implicit spatial nearest neighbor structure features in the original data, which is detrimental to the establishment of quality prediction models and accurate predictions. Furthermore, the temporal dynamics of the input data at different sampling times can also provide valuable information for improving the accuracy of quality prediction models. Therefore, it can be said that how to fully and effectively extract this information from deep learning models is an effective way to improve the accuracy of soft measurement methods in quality prediction. Summary of the Invention

[0006] The main technical problem to be solved by this invention is: how to take into account the temporal and spatial structural features between input data during the layer-by-layer feature extraction process of deep neural network models, so as to improve the accuracy of quality prediction and enhance the generalization ability of the model.

[0007] The technical solution adopted by the present invention to solve the above problems is: a quality prediction method and system based on spatiotemporal structure-preserving deep networks; wherein, the implementation process of the quality prediction method based on spatiotemporal structure-preserving deep networks disclosed in the present invention includes the following steps 1 to 3.

[0008] Step 1: Obtain from the sampling database of industrial processes The process data and their corresponding quality data, after being normalized, form an input matrix. and output vector .

[0009] Step 2, using and To The parameters of the deep network model, composed of encoders connected in series with spatiotemporal structures, are sequentially pre-trained and fine-tuned; among them... It is an integer not less than 3.

[0010] Step 3: Perform quality prediction periodically according to the sampling time interval, as shown in Steps 3-1 to 3-2 below.

[0011] Step 3-1: Obtain the latest sampling time A set of process data, after being normalized, forms an input vector. .

[0012] Step 3-2: Using the deep network model obtained in Step 2... Calculate the corresponding terminal valuation Then, on The latest sampling time was obtained by performing inverse normalization. Quality prediction .

[0013] The core technical feature of this invention is: any third A spatiotemporal structure-preserving encoder consists of an encoding layer, a gating layer, a decoding layer, a weighting layer, and an output layer, which are respectively configured according to the formula... , , , and Calculate the corresponding implicit vectors Gated vectors Input valuation Gated feature vectors and output valuation .

[0014] in, To pass the first Each spatiotemporal structure preserves the latent vectors computed by the encoder, and when hour, ; For any given input vector, for function, This indicates multiplication of elements at the same position; the first... The parameters of the spatiotemporal structure-preserving encoder include four weight matrices. and 4 bias quantities ; This refers to the sigmoid function, whose output range is... .

[0015] Step 2: During pre-training, use gradient descent to sequentially tune the first to the second... Each spatiotemporal structure preserves the encoder's parameters, thereby enabling the loss function to... Minimize; the deep network model also includes a method according to Estimation of computing terminals The output fusion layer, and in step 2, the weight vector of the output fusion layer is adjusted before fine-tuning. Initialize to ;

[0016] Step 2, during fine-tuning, involves using gradient descent to further tune the parameters of the deep network model, including the three weight matrices of each spatiotemporal structure-preserving encoder. and 3 bias values and weight vector This makes the loss function minimize;

[0017] Among them, coefficient and All within the range Internal value, for The Middle One data point; These represent the values ​​obtained through the deep network model at the 1st, 2nd, and 3rd stages, respectively. The corresponding spatiotemporal structure-preserving encoder calculates the value of the time-space structure-preserving encoder. The latent vector, input estimate, and output estimate, and when hour, ; Indicates calculation Length; Indicates will Combined into column vectors dimensional output estimation matrix The Middle Line number The elements of the column are equal to , express The first in Column vector, terminal valuation Spatiotemporal weighting coefficient Calculate according to formula ①: ①

[0018] In equation ①, express The first in Column vector, width factor The range of values ​​for is , Represented by natural constant Exponential functions with base 1 are numbered. ,serial number The nearest neighbors in equation ① are based on The sampling time and spatial distance are determined.

[0019] As can be seen from the loss functions of the pre-trained spatiotemporal structure-preserving encoders, the pre-trained encoders can not only extract gating feature vectors that are beneficial to the prediction quality, but also preserve the temporal and spatial nearest neighbor relationships between the original process data. In other words, The corresponding latent vectors are extracted by the encoder through various spatiotemporal structures. and Afterwards, it still maintains the original neighbor relationship, which is why it is called a spatiotemporal structure preservation encoder.

[0020] Furthermore, by examining the computations involved in maintaining the encoder across various spatiotemporal structures, it can be seen that the output layer uses gated feature vectors when calculating the output estimate. Non-latent vectors However, the encoder maintains its position by concatenating various spatiotemporal structures through latent vectors. This is mainly because latent vectors that are conducive to preserving spatiotemporal structural relationships are not necessarily the most conducive to predicting high-quality data. Therefore, gating vectors are used on the basis of latent vectors. The weighting effect, that is, through right Each element is weighted to filter and obtain a feature vector that is more conducive to predicting quality data, thereby improving the accuracy of quality prediction.

[0021] Furthermore, the deep network model involved in this invention is based on serial connections. Based on the spatiotemporal structure-preserving encoder, an output fusion layer is added, and the weight vector of this output fusion layer is, to those skilled in the art, [describe the characteristics of the encoder]. It is initialized using the least squares regression algorithm, that is: This output fusion layer can further integrate the first to the second layers. The spatiotemporal structure preserves the output estimate calculated by the encoder, and regression yields a more accurate estimate of the quality data.

[0022] These three technical advantages demonstrate that, in the process of progressively deepening feature extraction, this invention not only considers preserving the structural features of the temporal and spatial nearest neighbors of the original process data, but also filters features that are beneficial to quality prediction through a gating mechanism, and further combines and utilizes multiple output estimates to ultimately generate a more accurate quality data prediction value.

[0023] As an improvement to the present invention, starting from the second spatiotemporal structure-preserving encoder, the input vector First, obtain through variable selection. Then, according to The latent vectors are updated, and then the coding layer and gating layer calculations are performed; where, The variable numbers to be retained in the variable selection are determined according to steps A1 to A2.

[0024] Step A1: Calculate separately Each row vector and the output vector The correlation coefficient between them.

[0025] Step A2: Find the one with the largest absolute value. The row vector numbers corresponding to each correlation coefficient are retained as the variable numbers required for variable selection; among them... Indicates a value less than A positive integer representing the total number of row vectors in the middle.

[0026] This improved technique aims to prevent deep network models from losing information related to quality data from the original input data during the progressive feature extraction process. Since the variable numbers required for variable selection are determined based on the principle of maximum relevance, the results obtained after variable selection... Only kept The most relevant to quality data The presence of multiple elements ensures that each spatiotemporal structure retains the comprehensiveness of the encoder's input information, avoiding the loss of original input information in deep feature extraction, and further improving the model's quality and prediction accuracy.

[0027] In step A1 above, the correlation coefficient is preferably the Kendall correlation coefficient calculated according to equation ②: ②

[0028] in, express The Middle Row vectors and output vectors Kendall correlation coefficient between them, number ; express The total number of row vectors in the middle. ,coefficient Determined by equation ③: ③

[0029] In equation ③, express The Middle Line number Column elements, express The Middle Line number Column elements.

[0030] In this technical field, the most common way to calculate the correlation coefficient is through the formula calculate The Middle row vector and The correlation coefficient between the two variables is different from that between the two variables. In contrast, the improved correlation coefficient technique used in this paper employs the Kendall correlation coefficient, which measures the strength and direction of the correlation between two random variables and can be used to solve correlation analysis problems when outliers or uncertainties exist in the dataset. Therefore, the correlation coefficient calculated using equation ② above is more robust and can better preserve the correlation coefficient between the two variables. The variables most relevant to quality data.

[0031] In step A2 above, For a less than A positive integer representing the total number of row vectors; as a further improvement of the present invention, it is preferably determined according to steps A2-1 to A2-2. The value of .

[0032] Step A2-1: Take the absolute value of the correlation coefficient calculated in step A1, and then calculate its average value. .

[0033] Step A2-2: For values ​​with absolute values ​​greater than... The number of correlation coefficients was determined as follows: The value of .

[0034] Obviously, the above regarding The technical solution for selecting values ​​is to use the average value as the cutoff value, and to treat correlation coefficients with absolute values ​​greater than the average value as strong correlations, retaining the corresponding row vector numbers. This method is simple and quick to implement, providing an objective and practical approach to variable selection.

[0035] As another improvement of the present invention, after the fine-tuning in step 2, the ridge regression coefficients are determined according to steps B1 to B3 as shown below. ; Calculate any given input vector Corresponding terminal valuation Then, the first to the last... Gated feature vectors calculated by a spatiotemporal structure-preserving encoder The combined vectors are augmented input vectors. Then through Calculate terminal residual estimation Then, according to right Further updates will be made.

[0036] Step B1, with In The column vectors are used sequentially as input vectors, and the corresponding gated feature vectors are calculated by the encoders that preserve the spatiotemporal structure in the deep network model. and output valuation Then, the corresponding terminal valuation is calculated through the output fusion layer of the deep network model. .

[0037] Step B2: Combine all the gated feature vectors calculated in Step B1 to form an augmented input matrix. ,Will Individual terminal valuation Merged into terminal valuation vector ;in, any of the first Column vector is The column vectors are merged into one.

[0038] Step B3, according to the formula Calculate ridge regression coefficients ; where, regularity coefficient The range of values ​​for is , Represents the identity matrix.

[0039] Therefore, it can be seen that this improved technical solution utilizes the first to the second... The spatiotemporal structure preserves the latent vectors computed by the encoder to further fit the residuals that the output fusion layer cannot fit. To avoid overfitting, this improved technique uses ridge regression to calculate the regression coefficients. Therefore, this improved technique can further reduce the quality prediction error and improve the model's fitting accuracy.

[0040] because Each column vector is from level 1 to level 2. The spatiotemporal structure preserves the latent vectors of the encoder, which may contain information unrelated to the quality data. Extracting this unrelated information can effectively improve the fitting accuracy of the ridge regression in step B3. Therefore, as a further improvement of this invention, steps B2-1 to B2-3 are added between steps B2 and B3 to determine the load vector. and eigenvectors and the input matrix Implement update processing; correspondingly, after merging to obtain the augmented input vector... After that, it needs to be passed immediately. right Implement the update process.

[0041] Step B2-1, through calculate covariance matrix Then, calculate The eigenvector corresponding to the largest eigenvalue Then through Calculate the score vector .

[0042] Step B2-2, through Calculate the new score vector Then through Recalculate the eigenvectors ;in, for An identity matrix of dimension 1.

[0043] Step B2-3, through Recalculate the score vector Then, determine the score vector. Has the circuit converged? If not, return to step B2-2; if yes, proceed according to... Calculate the load vector Afterwards, through right Implement the update process.

[0044] It should be noted that the above improvement measures are based on the idea of ​​orthogonal signal correction, from... or Remove component information that is orthogonal to the quality data, thereby reducing... or The negative impact of interference information on the subsequent ridge regression model is reduced, further improving the accuracy of quality prediction.

[0045] Inspired by the concept of orthogonal signal correction, and also as another improvement of this invention, the weight vector of the output fusion layer is initialized... In this case, it is preferable to first determine the load vector according to steps C1 to C3. and eigenvectors Then through For the output valuation matrix After the update process is implemented, according to initialization Then, further through right Implement update processing; among which, express An identity matrix of dimension T, where the superscript T denotes the transpose of the matrix or vector.

[0046] Step C1, through calculate covariance matrix Then, calculate The eigenvector corresponding to the largest eigenvalue Then through Calculate the score vector .

[0047] Step C2, through Calculate the new score vector Then through Recalculate the eigenvectors ;in, for An identity matrix of dimension 1.

[0048] Step C3, through Recalculate the score vector Then, determine the score vector. Has convergence been achieved? If not, return to step C2; if yes, proceed according to... Calculate the load vector .

[0049] The above implementation process eliminates the interference of irrelevant component information between the input and output of the least squares regression model, thereby providing... This provides a better initialization condition for further optimization during the subsequent fine-tuning stage. In this technical field, the purpose of training a deep learning model through two stages—pre-training and fine-tuning—is to obtain a better initialization condition for fine-tuning during the pre-training stage, thereby improving the fitting accuracy of the deep learning model. Therefore, this improved technical solution aims to provide a better initialization weight vector through the idea of ​​orthogonal signal correction. This approach improves the accuracy of deep network models for quality prediction tasks.

[0050] As a further improvement to the present invention and its various improved technical solutions described above, it is preferred to set... equal The dimension, set ;in, .

[0051] Since the dimension of the quality data is equal to 1, and The dimension is generally greater than 1, by setting equal The dimension is actually to reduce the impact of dimensionality imbalance on the loss function. The negative impact caused; similarly, settings This is also to eliminate the influence of dimensionality and spatiotemporal weighting coefficients; the coefficients are set in this way. and The above scheme can very evenly consider the importance of the three minimization objectives: quality correlation, input reconstruction, and preservation of spatiotemporal nearest neighbor results. Therefore, it provides an objective coefficient setting method that does not rely on experience.

[0052] Based on the same inventive concept, the present invention also provides a quality prediction system based on a spatiotemporally structure-preserving deep network, comprising a hardware subsystem and a software subsystem; wherein, the hardware subsystem consists of a data acquisition module, a central processing unit, a storage medium and a display module; the software subsystem includes a program for implementing quality prediction according to steps 3-1 to 3-2, and a program for implementing pre-training and fine-tuning.

[0053] The central processing unit periodically controls the data acquisition module to retrieve the latest sampling time from the industrial process sampling database according to the set sampling time interval. The system takes a set of process data, calls the minimum and maximum values ​​stored in the storage medium for normalization and denormalization, as well as the parameters of the deep network model, and then calculates the predicted quality value according to the quality prediction program in the software subsystem. Then, it is sent to the display module for real-time display.

[0054] The central processing unit can also control the data acquisition module to acquire data from the sampling database of the industrial process according to the initialization instructions. The process data and its corresponding quality data are grouped together, and the minimum and maximum values ​​are calculated. Normalization is then performed to obtain the input matrix. and output vector Then, the central processing unit first tunes the first through the pre-training program implemented in the software subsystem, one by one. After maintaining the spatiotemporal structure encoder, the fine-tuning procedure is then implemented according to the software subsystem, and the result is... The parameters of the pre-trained spatiotemporal structure deep network model composed of encoders are fine-tuned; finally, the minimum value, maximum value, and the fine-tuned parameters of the deep network model are sent to the storage medium to overwrite the storage.

[0055] The soft measurement system disclosed in this invention is based on the same inventive concept as the aforementioned soft measurement method, mainly in that: First, any third The computations involved in a spatiotemporal structure-preserving encoder include , , , and Secondly, the pre-training procedure needs to be based on... The sampling time and spatial distance between each column vector determine the nearest neighbors of each column vector, and then the corresponding spatiotemporal weighting coefficients are calculated according to Equation ①. Then, gradient descent is used to tune the parameters of the encoder for each spatiotemporal structure preservation, thereby improving the loss function. Minimize; finally, after executing the pre-trained program, the central processing unit must first follow... Initialize weight vector Then, the fine-tuning program in the software subsystem is executed to further adjust the parameters of the deep network model.

[0056] Furthermore, this invention also provides an improved quality prediction system in which a pre-trained program calculates the spatiotemporal weight coefficients. Then, set the coefficients. equal The dimension, then set ;in, .

[0057] This improved quality prediction system does not require two coefficients to be given in advance. and Instead, it is determined dynamically in an objective way. This is more conducive to engineering applications when there is insufficient prior knowledge or experience. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the process flow of a sulfur recovery unit.

[0059] Figure 2 This is a schematic diagram of the deep network model of the present invention, which consists of a series of three spatiotemporal structure-maintaining encoders.

[0060] Figure 3 This is a schematic diagram illustrating the quality prediction effect of the first embodiment of the present invention.

[0061] Figure 4 This is a comparison chart of the quality prediction performance indicators of the first embodiment of the present invention and three comparative methods.

[0062] Figure 5 This is a schematic diagram of the deep network model involved in the second embodiment of the present invention.

[0063] Figure 6 This is a schematic diagram of the deep network model involved in the third embodiment of the present invention.

[0064] Figure 7 This is a comparison chart of the quality prediction performance indicators of the first to fifth embodiments of the present invention.

[0065] Figure 8 This is a schematic diagram of the structural composition of the quality prediction system involved in this invention. Detailed Implementation

[0066] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0067] In oil refineries, sulfur recovery units are used to purify acidic gas streams and simultaneously recover sulfur. Figure 1 A schematic diagram of a sulfur recovery unit is shown, in which two acidic gases, labeled MEA and SWS, are treated by a reactor and two combustion chambers before emission. MEA gas originates from a previous gas scrubbing unit, while SWS is generated by an acidic water stripping process. Both acidic gases are rich in hydrogen sulfide, which is harmful to human health. Therefore, real-time monitoring Figure 1 The terminal gas flow shown Concentration is a mandatory environmental requirement. However, the concentration used for measurement... Hardware sensors for concentration suffer from reliability issues and require regular maintenance; therefore, an auxiliary method or system for mass prediction is needed to achieve [the desired effect]. Online soft measurement of concentration, a mass data point.

[0068] Figure 1 The sulfur recovery unit shown in the diagram measures five process variables in real time. A delay needs to be introduced for each process variable to overcome the interference from the dynamic delay of the process object. Therefore, any set of process data consists of 20 data points. The first embodiment of the invention implements the prediction of quality data according to steps 1 to 3 as shown below.

[0069] Step 1: Obtain from the sampling database of industrial processes The process data and their corresponding quality data, after being normalized, form an input matrix. and output vector .

[0070] In this technical field, normalization is applied to individual variables, including process variables and quality variables; considering this embodiment, the selected activation function... for The function whose output range is Therefore, normalization requires utilizing the values ​​of each variable. The maximum and minimum values ​​of each data point are used to determine the input matrix. and output vector The range of data variation for each row vector in the interval is [range]. superior.

[0071] Step 2, using and To The parameters of a deep network model composed of encoders connected in series with spatiotemporal structures are pre-trained and fine-tuned sequentially; the structure of the deep network model is as follows: Figure 2 As shown.

[0072] Specifically, Figure 2 Each spatiotemporal structure preserved encoder shown consists of an encoding layer, a gating layer, a decoding layer, a weighting layer, and an output layer, respectively, according to the formula... , , , and Calculate the corresponding implicit vectors Gated vectors Input valuation Gated feature vectors and output valuation ;in, To pass the first Each spatiotemporal structure preserves the latent vectors computed by the encoder, and when hour, In other words, the latent vector of the previous spatiotemporal structure-preserving encoder is used as the input of the next spatiotemporal structure-preserving encoder, while the input of the first spatiotemporal structure-preserving encoder is equal to... .

[0073] Step 2, during pre-training, involves using gradient descent to sequentially tune the first through the second... Each spatiotemporal structure preserves the encoder's parameters, thereby enabling the loss function to... Minimize; in this embodiment, two coefficients are set. Spatiotemporal weighting coefficient Calculate according to formula ① above; where, The nearest neighbor is Selecting from and The most recent sampling time Each column vector, and with The 5 column vectors with the closest spatial distance.

[0074] Figure 2 The deep network model shown also includes a model based on the formula Estimation of computing terminals The output fusion layer, and before fine-tuning, according to... For weight vector Perform initialization.

[0075] Step 2, during fine-tuning, uses gradient descent to further adjust the parameters of the deep network model, thereby adjusting the loss function. Minimize. It's important to note that step 2, when fine-tuning, only needs to maintain the encoder's three weight matrices for each spatiotemporal structure. and 3 bias values and weight vector Perform the adjustment.

[0076] Therefore, if we denote any given input vector as Then, using a deep network model for Estimation of computing terminals The specific implementation process is shown in steps 2-1 to 2-3.

[0077] Step 2-1, Settings Then set .

[0078] Step 2-2: Follow the formulas respectively , , , and Calculate the corresponding implicit vectors Gated vectors Input valuation Gated feature vectors and output valuation .

[0079] Steps 2-3, Judgment Is it less than If so, then set Then, return to step 2-2; otherwise, return to the calculated result. Output valuation After merging into column vectors, follow the formula Estimation of computing terminals .

[0080] Step 3: Based on the sampling time interval, periodically execute steps 3-1 to 3-2 to perform quality prediction.

[0081] Step 3-1: Obtain the latest sampling time A set of process data, after being normalized, forms an input vector. .

[0082] Step 3-2: Using the depth obtained in Step 2... Estimation of computing terminals Then, on The quality prediction value was obtained by performing inverse normalization. ; where is the input vector Estimation of computing terminals The implementation process is as shown in steps 2-1 to 2-3 above.

[0083] To illustrate the quality prediction effect of the method of the present invention, quality data at 4072 latest sampling times are predicted according to step 3, and the degree of matching between the predicted quality values ​​and the actual measured values ​​at these 4072 latest sampling times is displayed. Figure 3 middle.

[0084] Figure 3 The diagonal line in the diagram represents the reference line for zero error, which is 100% accuracy. Therefore, from... Figure 3 It can be seen that the prediction accuracy provided by the method of the present invention is quite satisfactory, basically varying around the zero error reference line.

[0085] Ensuring high-accuracy quality prediction depends on Figure 2 The design includes a gating layer, an output fusion layer, and a loss function during pre-training. To verify the technical advantages of these three improvements, ablation comparison experiments are needed. The three comparison methods are based on this embodiment, respectively removing the gating layer, removing the output fusion layer, and setting... Based on 50 repeated experiments, the root mean square error (RMSE) of the method of this invention and three comparative methods are compared using box plots. Figure 4 middle.

[0086] from Figure 4 The comparison reveals that removing any one of the three improvements involved in this invention will affect the corresponding quality prediction accuracy. It can be said that the key technical features of this invention bring significant technical advancements and are the foundation for ensuring the high quality prediction accuracy of this invention.

[0087] The second embodiment of the present invention relates to, for example Figure 5 The deep network model shown is compared with Figure 2 The structure shown in this embodiment starts from the second spatiotemporal structure-preserving encoder, with the input vector... First, obtain through variable selection. Then, according to The latent vectors are updated, and then the calculations for the coding layer and the gating layer are performed. Figure 2 The variable numbers retained in the variable selection shown are determined according to the aforementioned steps A1 to A2, that is, first calculating... Each row vector and the output vector The correlation coefficient between them, and then the one with the largest absolute value The row vector numbers corresponding to each correlation coefficient are retained as the variable numbers required for variable selection.

[0088] The third embodiment of the present invention involves a deep network model as follows: Figure 6 As shown, the gating feature vectors calculated by the encoder are preserved using various spatiotemporal structures. The step, that is, the need to... Merge into augmented input vectors Then through Calculate the terminal residual estimate Then, according to Valuation of the terminal Further updates will be made; among them, the ridge regression coefficients It is determined according to the aforementioned steps B1 to B3.

[0089] The fourth embodiment of the present invention is based on the third embodiment, by adding the aforementioned steps B2-1 to B2-3 between steps B2 and B3 to determine the load vector. and eigenvectors and the input matrix Perform update processing; accordingly, obtain the augmented input vector. After that, it needs to be passed immediately. right Implement the update process.

[0090] The fifth embodiment of the present invention is based on the initialization of the weight vector in the first embodiment. The improvement requires first determining the load vector according to the aforementioned steps C1 to C3. and eigenvectors Afterwards, through For the output valuation matrix Implement update processing; then according to initialization Then, further through right Perform the update process to complete. Initialization.

[0091] The above five embodiments were used to predict the quality data for the same 4072 most recent sampling times, and the root mean square error (RMSE) between the predicted and actual measured values ​​for these 4072 most recent sampling times was displayed in the form of a box plot. Figure 7 middle.

[0092] It should be noted that the box plot shows the maximum, minimum, median, 25% lower bound, 75% upper bound, and possible gross errors of the RMSE index. Therefore, compared to the RMSE index of the first embodiment, the RMSE of the second to fourth embodiments all show varying degrees of improvement, demonstrating that the corresponding improvement schemes can bring about beneficial technical progress.

[0093] The last embodiment of the present invention designs a quality prediction system through a combination of hardware and software, the structure of which is as follows: Figure 8 As shown, it specifically includes a hardware subsystem and a software subsystem; wherein, the hardware subsystem consists of a data acquisition module, a central processing unit, a storage medium and a display module; the software subsystem includes a program for implementing quality prediction according to steps 3-1 to 3-2, and a program for implementing pre-training and fine-tuning.

[0094] Figure 8 The central processing unit shown in the diagram periodically controls the data acquisition module to retrieve the latest sampling time from the industrial process sampling database according to the set sampling time interval. The system takes a set of process data, calls the minimum and maximum values ​​stored in the storage medium for normalization and denormalization, as well as the parameters of the deep network model, and then calculates the predicted quality value according to the quality prediction program in the software subsystem. Then, it is sent to the display module for real-time display.

[0095] In addition, the central processing unit can also control the data acquisition module to acquire data from the sampling database of the industrial process according to the initialization instructions. The process data and its corresponding quality data are grouped together, and the minimum and maximum values ​​are calculated. Normalization is then performed to obtain the input matrix. and output vector Then, the central processing unit first tunes the first through the pre-training program implemented in the software subsystem, one by one. After maintaining the encoder parameters for each spatiotemporal structure, the fine-tuning procedure implemented in the software subsystem is then followed to adjust the parameters of the encoder. The parameters of the pre-trained spatiotemporal structure deep network model composed of encoders are fine-tuned; finally, the minimum value, maximum value, and the fine-tuned parameters of the deep network model are sent to the storage medium to overwrite the storage.

[0096] Figure 8 The quality prediction system shown has three technical points: First, any number of... The computations involved in a spatiotemporal structure-preserving encoder include , , , and Secondly, the pre-training procedure needs to be based on... The sampling time and spatial distance between each column vector determine the nearest neighbors of each column vector, and then the corresponding spatiotemporal weighting coefficients are calculated according to Equation ①. Then, gradient descent is used to tune the parameters of the encoder for each spatiotemporal structure preservation, thereby improving the loss function. Minimize; finally, after executing the pre-trained program, the central processing unit must first follow... Initialize weight vector Then, a fine-tuning procedure is executed to further adjust the parameters of the deep network model; among which, the loss function... The two coefficients are set as follows: First, set the coefficients. equal The dimension, then set .

[0097] The above explanations and descriptions of the technical solutions disclosed in this invention in conjunction with the accompanying drawings are illustrative rather than restrictive. The technical solutions disclosed in this invention are not limited to the above-described embodiments. Modifications made without departing from the spirit of the disclosed technical solutions and the scope of protection of the claims shall fall within the protection scope of this invention.

Claims

1. A quality prediction method based on spatiotemporally structure-preserving deep networks, comprising the following steps 1 to 3: Step 1: Obtain from the sampling database of industrial processes The process data and their corresponding quality data, after being normalized, form an input matrix. and output vector ; Step 2, using and To The parameters of a deep network model composed of encoders connected in series with spatiotemporal structure preservation are sequentially pre-trained and fine-tuned; among them... It is an integer not less than 3; Step 3: Perform steps 3-1 to 3-2 periodically according to the sampling time interval to implement quality prediction; Step 3-1: Obtain the latest sampling time A set of process data, after being normalized, forms an input vector. ; Step 3-2: Using the deep network model obtained in Step 2... Estimation of computing terminals Then, on The quality prediction value was obtained by performing inverse normalization. ; Its characteristic is: any number of A spatiotemporal structure-preserving encoder consists of an encoding layer, a gating layer, a decoding layer, a weighting layer, and an output layer, which are respectively configured according to the formula... , , , and Calculate the corresponding implicit vectors Gated vectors Input valuation Gated feature vectors and output valuation ;in, To pass the first Each spatiotemporal structure preserves the latent vectors computed by the encoder, and when hour, ; For any given input vector, for function, This indicates multiplication of elements at the same position; the first... The parameters of the spatiotemporal structure-preserving encoder include four weight matrices. and 4 bias values ; In step 2, during pre-training, gradient descent is used to sequentially tune the first to the second... Each spatiotemporal structure preserves the encoder's parameters, thereby enabling the loss function to... Minimize; the deep network model also includes a method according to Estimation of computing terminals The output fusion layer, and the weight vector is adjusted before fine-tuning. Initialize to During fine-tuning, gradient descent is used to further tune the parameters of the deep network model, including the three weight matrices of each spatiotemporal structure-preserving encoder. and 3 bias values and weight vector This makes the loss function Minimize; where the coefficients and All within the range Internal value, for The Middle One data point; They respectively represent the passage through the first The corresponding spatiotemporal structure-preserving encoder calculates the value corresponding to The latent vector, input estimate, and output estimate, and when hour, ; Indicates calculation Length; Indicates will Combined into column vectors dimensional output estimation matrix The Middle Line number The elements of the column are equal to , express The first in Column vector, terminal valuation The superscript T denotes the transpose of a matrix or vector; spatiotemporal weighting coefficients Calculate according to formula ①: ① In equation ①, the width coefficient The range of values ​​for is In equation ①, the nearest neighbor is based on The sampling time and spatial distance are determined.

2. The quality prediction method according to claim 1, characterized in that, Starting from the second spatiotemporal structure-preserving encoder, the input vector First, obtain through variable selection. Then, according to For latent vectors The update is performed, followed by calculations at the coding and gating layers; among which, The variable numbers to be retained during variable selection are determined according to steps A1 to A2: Step A1: Calculate separately Each row vector and the output vector The correlation coefficient between them; Step A2: Find the one with the largest absolute value. The row vector numbers corresponding to each correlation coefficient are retained as the variable numbers required for variable selection; among them... Indicates a value less than A positive integer representing the total number of row vectors in the middle.

3. The quality prediction method according to claim 1, characterized in that, After the fine-tuning is completed in step 2, the ridge regression coefficients are determined according to steps B1 to B3. ; Calculate any given input vector Corresponding terminal valuation Then, the first to the last... Gated feature vectors calculated by a spatiotemporal structure-preserving encoder Merge into augmented input vectors Then through Calculate terminal residual estimation Then, according to Valuation of the terminal Further updates will be made. Step B1, with In The column vectors are used sequentially as input vectors, and the corresponding gated feature vectors are calculated by the encoders that preserve the spatiotemporal structure in the deep network model. and output valuation Then, the corresponding terminal valuation is calculated through the output fusion layer of the deep network model. ; Step B2: Combine all the gated feature vectors calculated in Step B1 to form an augmented input matrix. ,Will Individual terminal valuation Merged into terminal valuation vector ;in, any of the first Column vector is The column vectors merged into one; Step B3, according to the formula Calculate ridge regression coefficients ; where, regularity coefficient The range of values ​​for is , Represents the identity matrix.

4. The quality prediction method according to claim 3, characterized in that, in Between steps B2 and B3, add steps B2-1 to B2-3 to determine the load vector. and eigenvectors and the input matrix Implement update processing; correspondingly, after merging to obtain the augmented input vector... Then, it needs to be passed immediately. right Implement update processing; Step B2-1, through calculate covariance matrix Then, calculate The eigenvector corresponding to the largest eigenvalue Then through Calculate the score vector ; Step B2-2, through Calculate the new score vector Then through Recalculate the eigenvectors ;in, for An identity matrix of 3D; Step B2-3, through Recalculate the score vector Then, determine the score vector. Has the circuit converged? If not, return to step B2-2; if yes, proceed according to... Calculate the load vector Afterwards, through right Implement the update process.

5. The quality prediction method according to claim 1, characterized in that, in Initialize the weight vector of the output fusion layer First, determine the load vector according to steps C1 to C3. and eigenvectors Then through For the output valuation matrix After the update process is implemented, according to initialization Further through right Implement update processing; among which, express An identity matrix of 3D; Step C1, through calculate covariance matrix Then, calculate The eigenvector corresponding to the largest eigenvalue Then through Calculate the score vector ; Step C2, through Calculate the new score vector Then through Recalculate the eigenvectors ;in, for An identity matrix of 3D; Step C3, through Recalculate the score vector Then, determine the score vector. Has convergence been achieved? If not, return to step C2; if yes, proceed according to... Calculate the load vector .

6. The quality prediction method according to claim 2, characterized in that, The correlation coefficient in step A1 is the Kendall correlation coefficient calculated according to equation ②: ② in, express The Middle Row vectors and output vectors Kendall correlation coefficient between them, number ; express The total number of row vectors in the middle. ,coefficient Determined by equation ③: ③ In equation ③, express The Middle Line number Column elements, express The Middle Line number Column elements.

7. A quality prediction method according to any one of claims 2 or 6, characterized in that, Determine according to steps A2-1 to A2-2. The possible values ​​of: Step A2-1: Take the absolute value of the correlation coefficient calculated in step A1, and then calculate its average value. ; Step A2-2: For values ​​with absolute values ​​greater than... The number of correlation coefficients was determined as follows: The value of .

8. A quality prediction method according to any one of claims 1 to 6, characterized in that, Set the coefficient equal The dimension, setting the coefficients ;in, .

9. A quality prediction system based on a spatiotemporally structure-preserving deep network, comprising a hardware subsystem and a software subsystem; wherein, The hardware subsystem consists of a data acquisition module, a central processing unit, a storage medium, and a display module; the software subsystem includes a program for implementing quality prediction according to steps 3-1 to 3-2, and a program for implementing pre-training and fine-tuning. The central processing unit periodically controls the data acquisition module to retrieve the latest sampling time from the industrial process sampling database according to the set sampling time interval. The system takes a set of process data, calls the minimum and maximum values ​​stored in the storage medium for normalization and denormalization, as well as the parameters of the deep network model, and then calculates the predicted quality value according to the quality prediction program in the software subsystem. Then, it is sent to the display module for real-time display; The central processing unit can also control the data acquisition module to acquire data from the sampling database of the industrial process according to the initialization instructions. The process data and its corresponding quality data are grouped together, and the minimum and maximum values ​​are calculated. Normalization is then performed to obtain the input matrix. and output vector Then, the central processing unit first tunes the first through the pre-training program implemented in the software subsystem, one by one. After maintaining the encoder parameters for each spatiotemporal structure, the fine-tuning procedure implemented in the software subsystem is then followed to adjust the parameters of the encoder. The parameters of the pre-trained spatiotemporal structure deep network model composed of encoders are fine-tuned; finally, the minimum value, maximum value, and the parameters of the fine-tuned deep network model are sent to the storage medium to overwrite the storage. Its characteristic is that, firstly, any first... The computations involved in a spatiotemporal structure-preserving encoder include , , , and Secondly, the pre-training procedure is first based on... The sampling time and spatial distance between each column vector determine the nearest neighbors of each column vector, and then the corresponding spatiotemporal weighting coefficients are calculated according to Equation ①. Then, gradient descent is used to tune the parameters of the encoder for each spatiotemporal structure preservation, thereby improving the loss function. Minimize; finally, after executing the pre-trained program, the central processing unit must first follow... Initialize weight vector Then, the fine-tuning program in the software subsystem is executed to further adjust the parameters of the deep network model.

10. A quality prediction system according to claim 9, characterized in that, The pre-training program in the software subsystem calculates the spatiotemporal weight coefficients. Then, set the coefficients. equal The dimension, then set ;in, .

Images