A non-stationary industrial process quality prediction method based on deep slow feature network

Abstract

This invention discloses a method for predicting the quality of non-stationary industrial processes based on a deep slow feature network, belonging to the field of industrial artificial intelligence technology. The method involves constructing a training set; training a slow feature network model based on multiple cascaded slow feature analysis layers using the training set, including: initializing a pre-defined persistent covariance matrix within each slow feature analysis layer; calculating an instantaneous covariance matrix based on the current batch data of the training set; updating the persistent covariance matrix using the instantaneous covariance matrix; outputting a sequence of slow feature vectors extracted by each slow feature analysis layer; and inputting newly acquired time-series data, after standardization, into the trained slow feature network model to obtain real-time predicted values ​​of key quality indicators. This method solves the problems of feature extraction and prediction targets being disconnected, training instability, and difficulty adapting to non-stationary data distribution drift in existing technologies, achieving stable, accurate, and real-time prediction of key quality indicators for non-stationary industrial processes.

Description

Technical Field

[0001] This application belongs to the field of industrial artificial intelligence technology, specifically relating to a method for predicting the quality of non-stationary industrial processes based on deep slow feature networks. Background Technology

[0002] In modern industrial manufacturing, real-time monitoring of key quality indicators is crucial for optimizing production control. Due to harsh industrial environments and high testing costs, many key quality variables are difficult to acquire online via hardware sensors. Offline testing is often necessary, leading to measurement delays that fail to meet the demands of real-time control. Therefore, data-driven quality prediction technologies are widely used in industrial process quality forecasting.

[0003] Current quality prediction methods primarily employ deep learning techniques, such as recurrent neural networks (RNNs) or convolutional neural networks (CNNs), to extract nonlinear features from industrial time-series data. However, these models typically implicitly assume that the training data and future application data follow the same statistical distribution. But in real-world, complex industrial processes, influenced by factors such as raw material fluctuations, equipment aging, and changes in operating conditions, production data often exhibits significant non-stationarity, causing the statistical distribution of the data to drift over time. When the distribution of process data changes, the predictive performance of deep learning models trained on historical, fixed data significantly degrades, making it difficult to adapt to dynamically changing real-world conditions.

[0004] For modeling non-stationary processes, Slow Feature Analysis (SFA) algorithms have been proposed to extract the essential features that change slowly in the data, thereby eliminating rapidly changing noise interference. Although traditional SFA provides a theoretical direction for this, it has inherent limitations: it is essentially a linear or shallow nonlinear transformation, and its feature extraction depth and expressive power are severely insufficient for high-dimensional, strongly coupled, and deeply nonlinear industrial big data.

[0005] To improve feature extraction capabilities, existing technologies attempt to combine SFA with deep neural networks. Currently, the mainstream combination methods fall into two main categories:

[0006] The first type is the two-stage strategy: This involves first using the SFA algorithm to extract unsupervised features from the original data, and then using the extracted slow features as input to subsequent deep neural networks (Long Short-Term Memory, LSTM, or CNN). The fundamental flaw of this method is that feature extraction and target prediction are completely separate. The slowest features extracted by the SFA algorithm (e.g., extremely low-frequency system drift or environmental changes) may not be necessary for predicting key quality indicators (such as product concentration or reaction temperature). Due to the lack of supervisory feedback from the predicted target, the front-end SFA layer cannot extract the most useful features for the downstream prediction task, resulting in limited final prediction accuracy.

[0007] The second type is the regularization strategy: simply adding a regularization term to the loss function of the deep network to measure the degree of slow variation of features. While this formally achieves joint training, it is extremely difficult to optimize in practice. This is because there is often a competition between the slow variation constraint (hoping that features remain unchanged) and the prediction accuracy (hoping that features can sensitively reflect changes in the output). Simple weighted summation can easily cause the training process to oscillate repeatedly between the two objectives, making convergence difficult. In addition, this weight parameter is extremely sensitive to model performance; even a small adjustment can cause the model to fail to converge or its performance to collapse, lacking robustness for large-scale industrial applications.

[0008] In summary, the fundamental dilemma faced by existing technologies lies in the following: the two-stage method, due to its fragmented architecture, leads to a severe mismatch between the extracted slow features and the final prediction target, limiting the upper limit of model performance; while the regularization constraint method, due to conflicting optimization objectives, suffers from difficulty in balancing the training process, poor stability, and a high threshold for practical application. More importantly, both types of methods adhere to the assumption of static data distribution and lack online adaptive mechanisms to cope with the inherent non-stationarity of industrial processes, causing their prediction performance to continuously degrade as the operating conditions drift under real-world conditions. Therefore, designing a novel network structure that achieves native fusion of slow feature extraction and prediction tasks at the architectural level, ensures training stability at the optimization level, and endows the model with the ability to adapt to distribution drift online at the mechanism level has become the key to breaking through the bottleneck of industrial quality prediction technology. This invention is proposed to solve this systemic problem. Summary of the Invention

[0009] In view of at least one of the above technical problems, this application provides a method for predicting the quality of non-stationary industrial processes based on deep slow feature networks.

[0010] Firstly, a method for predicting the quality of non-stationary industrial processes based on deep slow feature networks is provided, including:

[0011] Construct the training set;

[0012] The training process involves training a slow feature network model based on multiple cascaded slow feature analysis layers using a training set. This includes: initializing the pre-defined persistent covariance matrix within each slow feature analysis layer; calculating the instantaneous covariance matrix based on the current batch of data in the training set; updating the persistent covariance matrix using the instantaneous covariance matrix; and outputting the slow feature vector sequence extracted by each slow feature analysis layer.

[0013] The newly collected time-series data is standardized and then input into the trained slow feature network model to obtain real-time predicted values ​​of key quality indicators.

[0014] Optionally, constructing the training set includes:

[0015] Collect raw, multivariate time-series data from industrial process sites;

[0016] Standardization preprocessing is performed on the original multivariate time series data to obtain standardized time series data;

[0017] A training set was constructed by selecting standardized time-series historical data of industrial processes. B is the batch size, T' is the time step, and n is the variable dimension. It is the set of real numbers.

[0018] Optionally, training a slow feature network model based on multiple cascaded slow feature analysis layers using the training set further includes:

[0019] The slow feature vector sequences extracted from all slow feature analysis layers are flattened and concatenated to form a single comprehensive feature vector; or

[0020] Extract the slow feature vector sequence of the last slow feature analysis layer;

[0021] Flatten the slow feature vector sequence of the last slow feature analysis layer into a deep feature vector.

[0022] Optionally, training a slow feature network model based on multiple cascaded slow feature analysis layers using the training set further includes:

[0023] The comprehensive feature vector or deep feature vector, along with the real quality index values ​​in the training set that correspond one-to-one with the comprehensive feature vector or deep feature vector, are used as the input to the slow feature network model.

[0024] The comprehensive feature vector or deep feature vector is calculated by a fully connected prediction layer, and the predicted value of the quality index is output. Based on the predicted value and the actual quality index value, the prediction error term is calculated.

[0025] Slow feature constraint terms characterizing the degree of temporal fluctuation are calculated based on the slow feature vector sequences output by each slow feature analysis layer.

[0026] The prediction error term and the slow feature constraint term are weighted and combined to construct a joint loss function for optimization.

[0027] Optionally, the calculation of slow feature constraint terms characterizing the degree of temporal fluctuation based on the slow feature vector sequence output by each slow feature analysis layer includes:

[0028] Calculate the difference vector between the slow feature vectors of two adjacent time steps in any slow feature analysis layer, and then calculate the square of the length of the difference vector.

[0029] Traverse all consecutive time step pairs of the slow feature analysis layer and accumulate the squared differences of all adjacent time steps; divide the accumulated result by the product of the time step size minus one and the batch sample size to obtain the average temporal change of the slow feature analysis layer.

[0030] Traverse each slow feature analysis layer to obtain the average time series change corresponding to each slow feature analysis layer; sum the average time series changes of each layer, and then divide by the total number of slow feature analysis layers to obtain a single scalar value.

[0031] Optionally, the calculation of the instantaneous covariance matrix based on the current batch of data in the training set is achieved through the following expression:

[0032] ;

[0033] in, The instantaneous covariance matrix of the current batch of data. This is the initial feature matrix after flattening, where B is the batch size. For time step, This represents the total number of samples after flattening the data in the current batch. Subtracting 1 gives an unbiased estimate of the covariance. This is the transpose of the initial feature matrix after flattening.

[0034] Optionally, training the slow feature network model using the training set further includes:

[0035] The instantaneous covariance matrix is ​​decomposed into an orthogonal matrix composed of eigenvectors and a diagonal matrix composed of eigenvalues;

[0036] Introducing a very small regularization constant, a whitening transformation matrix is ​​constructed based on the orthogonal matrix and the diagonal matrix;

[0037] Multiply the feature matrix of the current batch of data by the transpose of the constructed whitening transformation matrix.

[0038] Optionally, updating the persistent covariance matrix using the instantaneous covariance matrix specifically involves updating the persistent covariance matrix using an exponential moving average update strategy, as shown in the following expression:

[0039] ;

[0040] in, For the updated persistent covariance matrix, The persistent covariance matrix before the update. The instantaneous covariance matrix is ​​calculated based on the current batch of data. This is the attenuation coefficient.

[0041] Beneficial effects

[0042] 1. High adaptability to non-stationary data.

[0043] By introducing a persistent covariance update mechanism based on exponential moving average (EMA), the model can smoothly track the slow drift of industrial process data distribution online. This effectively overcomes the dependence of traditional deep learning models on the assumption of static data distribution and significantly improves the model's long-term prediction robustness and accuracy stability under non-stationary environments such as changing operating conditions and equipment aging.

[0044] 2. Deep and robust ability to extract essential features.

[0045] By designing stackable slow feature analysis layers, a deep slow feature network is constructed, overcoming the limitations of traditional slow feature analysis methods in terms of nonlinearity and representation depth. This network can automatically extract deep, slowly varying driving features with physical meaning from high-dimensional, noisy industrial data, thereby effectively suppressing high-frequency noise interference and improving feature quality.

[0046] 3. Stable and efficient end-to-end collaborative optimization features.

[0047] By designing a joint loss function, the prediction accuracy target and feature slowness constraint are unified within a single training framework, achieving end-to-end collaborative optimization of feature extraction and quality prediction. This not only solves the problem of the separation of objectives between the two in traditional two-stage methods, but also avoids the training instability and parameter tuning difficulties caused by simply adding regularization terms, ensuring the reliability of model convergence and the superiority of final performance.

[0048] 4. Flexible and configurable practical architecture.

[0049] Two network architectures are provided: feature fusion and final output. The former, by fusing multi-layer features, is suitable for capturing complex patterns across multiple time scales; the latter, by focusing on deeply abstract features, is suitable for long-term trend prediction under strong noise. Users can flexibly choose according to specific industrial scenarios, greatly enhancing the practicality and applicability of the method. Attached Figure Description

[0050] Figure 1 A flowchart of a nonstationary industrial process quality prediction method based on deep slow feature networks provided in this application embodiment;

[0051] Figure 2 This application provides a schematic diagram of the internal structure of the Slow Feature Analysis (SFA) layer in an embodiment.

[0052] Figure 3 A schematic diagram of the Deep Slow Feature Network Fusion (DSFN-Fusion) architecture provided in this application embodiment;

[0053] Figure 4 A schematic diagram of the final output architecture (DSFN-Final) of the Deep Slow Feature Network provided in this application embodiment;

[0054] Figure 5 This application provides an example of an industrial process quality prediction method based on deep slow feature networks, which establishes a system structure diagram. Detailed Implementation

[0055] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The technical solutions of the embodiments of this application will now be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0056] This invention aims to overcome three major shortcomings of existing industrial process quality prediction methods: first, the feature extraction and prediction model architecture are disconnected; second, the joint training process is unstable; and third, the model struggles to adapt to non-stationary drift in data distribution. To address these shortcomings, this invention provides a solution based on deep slow feature networks, specifically designed to solve the following problems:

[0057] 1. Achieve end-to-end collaborative optimization: By constructing a trainable deep slow feature analysis layer, the feature extraction process directly serves the final prediction target, completely solving the feature-target mismatch problem in the traditional two-stage method;

[0058] 2. Establish a stable and convergent joint training framework: By designing a structured joint loss function and matching it with a stable optimization strategy (such as gradient clipping), the training oscillation and difficulty in convergence caused by target conflicts in simple regularization methods are systematically avoided.

[0059] 3. Embedded online adaptive capability: By introducing an internal state dynamic update mechanism based on the persistent covariance matrix, the model can track the distribution drift of non-stationary operating conditions in real time, significantly improving the robustness of the model in long-term deployment.

[0060] Example 1

[0061] To address the aforementioned technical problems, this invention provides a method for predicting the quality of non-stationary industrial processes based on deep slow feature networks, comprising the following steps:

[0062] Step S1: Data acquisition and preprocessing.

[0063] Raw multivariate time series data collected from industrial process sites are used as input. The data is standardized to eliminate the influence of dimensions and to ensure that it meets the zero mean and unit variance distribution. The resulting standardized time series data blocks are then used by the subsequent network.

[0064] In practice, historical time-series data of industrial processes are collected as a training set. B is the batch size, T' is the time step, and n is the variable dimension. Given a set of real numbers, standardize the data to ensure it has zero mean and unit variance.

[0065] Step S2: Construct a Deep Slow Feature Network (DSFN).

[0066] Construct a deep neural network consisting of multiple stacked SFA layers. In this embodiment, it is preferable to use... Several SFA layers are cascaded, with the hidden feature dimension of each layer set to 64. For example... Figure 2As shown, each SFA layer contains a nonlinear transformation unit, a covariance matrix calculation unit, a whitening transformation unit, and a linear projection unit connected in sequence. When input data flows through this SFA layer, the following feature transformation processes will be performed sequentially:

[0067] (1) Nonlinear transformation unit

[0068] The input to this unit is either the output features of the previous layer or the original standardized data. .

[0069] The input is fed into the first linear layer, through the transpose of the trainable weight matrix. and bias vector A linear mapping is performed to upscale the original feature dimensions to a higher dimension; then the output of the first linear layer is passed through the hyperbolic tangent activation function. Perform nonlinear activation processing to obtain intermediate activation values. ;Activate value It is fed into the second linear layer, through the transpose of the trainable weight matrix. and bias vector A second linear transformation is performed, and the final output of this unit is then given. , This serves as the initial feature. This process, through two linear mappings and one nonlinear activation, enhances the network's ability to express deep nonlinear relationships in complex industrial data.

[0070] (2) Covariance matrix calculation unit

[0071] This unit receives the preliminary characteristics output by the nonlinear transformation unit. As input, since the input data has three dimensions—batch, time step, and feature dimension—this unit first performs a flattening operation, reorganizing the three-dimensional tensor structure into a two-dimensional matrix. This ensures that the number of rows in the matrix corresponds to the sum of the time steps for all samples within the batch, and the number of columns corresponds to the feature dimensions of the SFA layer. Based on the flattened preliminary feature matrix... Calculate the instantaneous covariance matrix of the current batch of data. This matrix reflects the degree of linear correlation between the feature dimensions in the current input batch, serving as the basis for subsequent whitening transformation.

[0072] (3) Whitening transformation unit

[0073] This unit receives the instantaneous covariance matrix output by the covariance matrix calculation unit. And the persistent covariance matrix maintained at this layer. Singular value decomposition is performed on the input target matrix, decomposing it into an orthogonal matrix composed of eigenvectors. and a diagonal matrix composed of eigenvalues Then, based on the decomposition results, construct the whitening transformation matrix. A very small regularization constant is introduced during the construction process. To prevent numerical calculation errors during matrix inversion, the initial features received by this unit are... Transpose of the constructed whitening transformation matrix Perform matrix multiplication to complete whitening and obtain the output features. After this operation, the features The components were forcibly decorrelated and have unit variance, providing a standardized data foundation for the next step of extracting the essential features of slow change.

[0074] (4) Linear projection unit

[0075] This unit receives the features output by the whitening transformation unit. As input, this unit internally maintains a trainable weight matrix. This matrix is ​​constrained to be approximately orthogonal during initialization. In each forward propagation, the input features are... With weight matrix Multiplication is performed to project the high-dimensional whitening features into a low-dimensional space, outputting the final slow feature vector. .

[0076] Unlike traditional slow eigenvalue analysis algorithms that obtain a fixed analytical solution by solving the generalized eigenvalue problem, the linear projection matrix in this step... These are constructed as trainable parameters within a deep, slow feature network. During the model training phase, the matrix... The numerical adjustment does not depend in isolation on the statistically slow variability of the current batch of data, but is incorporated into the backpropagation optimization process of the joint loss function described in step S5. Specifically, in each training iteration, the matrix... The parameter update direction is determined by the combined effect of the gradient of the prediction error term and the gradient of the slow feature constraint term in the joint loss function. This mechanism makes the matrix... During training, it can gradually converge to a projection direction that satisfies the slow variation constraint of features and minimizes the final prediction error, thereby achieving synergistic optimization of the unsupervised feature stationarity requirement and the supervised prediction task objective at the structural level.

[0077] Step S3: Construct a persistent covariance matrix and perform an exponential moving average update.

[0078] In industrial processes, slow changes in raw material properties, equipment status, or environmental conditions can cause data statistical distribution drift. If the covariance is directly calculated and whitened using the current batch of data, the random noise contained in a single batch of data will cause a large instantaneous deviation in the calculated whitening projection direction, leading to discontinuous jumps in the subsequently extracted slow features. To address this issue, this step designs a persistent covariance matrix as an internal memory state for each slow feature analysis layer and uses an exponential moving average strategy to smoothly update it, enabling the model to stably track the slow evolution of the data distribution.

[0079] The following uses an SFA layer as an example to explain the complete execution flow of this mechanism in detail:

[0080] Step S3-1: Establish and initialize the persistent covariance matrix.

[0081] Before training begins, for each SFA layer in the network, a square matrix matching the size of its feature dimensions is created, called the persistent covariance matrix. This matrix is ​​stored and maintained as a long-term state variable within that layer, does not participate in gradient backpropagation calculations, and is only used to record the accumulated information of historical data statistical characteristics. Specifically, during initialization, a square matrix of the same size is created using the preset number of hidden feature dimensions for that SFA layer as the number of rows and columns, and initialized as an identity matrix. That is, all diagonal elements are set to 1, and all off-diagonal elements are set to zero. After initialization, an identity matrix is ​​obtained, which serves as the initial persistent covariance matrix for that layer, denoted as the historical state matrix, and stored within that layer for subsequent steps.

[0082] Step S3-2: Obtain the instantaneous covariance matrix of the current batch.

[0083] As attached Figure 2As shown, in each forward computation of the model, whether in the training or online prediction phase, when data flows through the covariance matrix calculation unit of this SFA layer, the unit calculates a covariance matrix reflecting the current local data fluctuations based on the small batch of input data. In each forward computation of the model, whether in the training or online prediction phase, when data flows through the covariance matrix calculation unit of a certain SFA layer, this unit receives a feature data block output from the previous nonlinear transformation unit as input. This input feature data block contains a set of feature vectors for several samples in the current batch over several consecutive time steps. For this input object, the covariance matrix calculation unit first flattens it from a three-dimensional structure into a two-dimensional matrix, such that the number of rows after flattening equals the product of the batch sample number and the time step, and the number of columns equals the feature dimension of this layer. Subsequently, a transpose multiplication operation is performed on this two-dimensional matrix, that is, the transpose of the flattened matrix is ​​multiplied by the flattened matrix itself, and then the product is divided by the total number of samples minus one, thus obtaining an unbiased estimate of the covariance matrix. This matrix only reflects the statistical fluctuation characteristics of the current small batch of input data itself, so it is called the instantaneous covariance matrix, and it is passed as the output result to the next matrix update stage.

[0084] Step S3-3: Update the persistent covariance matrix using the exponential moving average strategy.

[0085] Following the instantaneous covariance matrix calculated in the previous step, the model's internal state update module immediately initiates the update process. This module receives two input objects: the historical state matrix initialized and continuously maintained in step S3-1, and the instantaneous covariance matrix just output in step S3-2. Furthermore, the update module uses a preset decay coefficient to control the fusion ratio of old and new information. For these two input matrices, the update module performs the following weighted fusion operation: multiplying each element value in the historical state matrix by the decay coefficient, and simultaneously multiplying each element value in the instantaneous covariance matrix by the result minus the decay coefficient. Then, the two weighted matrices are added one by one at the corresponding element positions to obtain a new square matrix. This new square matrix retains both historically accumulated statistical information and incorporates the latest data features of the current batch, achieving smooth tracking of slowly changing data distribution. After the update is complete, this new square matrix replaces the original historical state matrix and is stored as the updated persistent covariance matrix within the SFA layer, denoted as the updated historical state matrix, and is output to the subsequent whitening transformation stage.

[0086] Step S3-4: Apply the updated persistent covariance matrix to the whitening transformation.

[0087] After obtaining the updated persistent covariance matrix output in step S3-3, this matrix is ​​fed into the whitening transformation unit of this SFA layer as the basis for calculation. The whitening transformation unit receives this matrix as input and performs singular value decomposition on it, thus decomposing the matrix into an orthogonal eigenvector matrix and a diagonal matrix storing the eigenvalues. Subsequently, the whitening transformation unit extracts the diagonal matrix, adds a very small regularization constant to each eigenvalue on its diagonal to prevent numerical computational anomalies, and then calculates the negative 1 / 2 power of the diagonal matrix after adding the constant, i.e., calculates its inverse square root matrix. Next, the inverse square root matrix is ​​multiplied left by the eigenvector matrix and right by the transpose of the eigenvector matrix, finally constructing a whitening transformation matrix. This whitening transformation matrix will be directly applied to the feature data input in this batch to complete decorrelation and unit variance standardization. Since the whitening transformation matrix is ​​calculated based on the persistent covariance matrix after exponential moving average smoothing, rather than directly using the instantaneous covariance matrix of the current batch, it can effectively suppress the drastic jitter of the projection direction caused by random noise in a single batch, ensuring that the model can still stably extract the slowly changing essential features when facing slow drift in the operating conditions.

[0088] Step S4: Based on the predicted scenario requirements, construct and select the network output architecture.

[0089] After extracting multiple layers of slow features, the deep slow feature network needs to map the extracted features to the final predicted quality index values. This step provides two optional output architecture construction methods, each suitable for different industrial prediction scenarios. Users can flexibly choose according to the actual data characteristics and prediction objectives.

[0090] Step S4-1: Construction and execution process of Feature Fusion Architecture (DSFN-Fusion).

[0091] When a prediction task requires capturing both short-term local details and long-term global trends in industrial data simultaneously, a feature fusion architecture can be used. This architecture uses the output features of all L slow feature analysis layers in a deep slow feature network as input, where L is three in this embodiment. Each layer's output feature, collected from the first, second, and Lth layers, is a three-dimensional data block containing a batch sample dimension, a time step dimension, and the feature dimension of that layer. For each layer's three-dimensional data block, the last two dimensions—the time step dimension and the feature dimension—are merged into one dimension, while the batch sample dimension remains unchanged. The temporal feature matrix corresponding to each sample is flattened into a one-dimensional vector, resulting in a two-dimensional matrix with rows equal to the number of batch samples and columns equal to the product of the time step and the feature dimension of that layer. After flattening each layer, all flattened two-dimensional matrices are concatenated end-to-end along the column direction. The columns of the first-layer flattened matrix, the second-layer flattened matrix, and so on up to the Lth-layer flattened matrix are arranged sequentially and merged into a comprehensive feature matrix with the same number of rows and the same number of columns as the sum of the number of columns in each layer. This comprehensive feature matrix integrates high-frequency detail information captured by shallow networks with low-frequency abstract trend information extracted by deep networks, achieving simultaneous representation of features across multiple time scales. The concatenated comprehensive feature matrix is ​​input into a fully connected prediction network consisting of two linear transformation layers and an intermediate linear rectified function activation. After feature dimensionality reduction or transformation by the first linear layer and the introduction of nonlinearity by the activation function, the second linear layer maps the features into a final one-dimensional predicted value sequence, which is the output of the quality index. (See attached image) Figure 3 As shown, the outputs of each slow feature analysis layer are branched out, flattened, and then converged into the splicing node. The splicing node then sends the outputs to the fully connected prediction layer, which finally outputs the predicted values.

[0092] Step S4-2: Construction and execution process of the final output architecture (DSFN-Final).

[0093] When the prediction task focuses on long-term trend forecasting, or when there is strong high-frequency noise interference in the industrial field and the model needs to automatically filter out irrelevant fluctuations, the final output architecture can be selected. This architecture uses the output features of the last layer, the Lth layer of the deep slow feature network, as the sole input object, and no longer collects the outputs of shallow or intermediate layers. Through layer-by-layer nonlinear transformations and slow-varying constraints, the deep network has gradually filtered out rapid random fluctuations in the data. The output of the Lth layer is considered to contain the most essential and most gently changing system-driving features. A flattening operation is performed on the three-dimensional data block output by the Lth layer, merging the last two dimensions to obtain a one-dimensional feature vector for each sample, forming a two-dimensional feature matrix. The number of rows is the number of samples in the batch, and the number of columns is the product of the time step and the number of feature dimensions of the Lth layer. The flattened two-dimensional feature matrix is ​​directly input into a fully connected prediction network consisting of two layers of linear transformation and an intermediate linear rectified function activation. After the feature matrix is ​​processed by the first linear layer transformation and activation function, the second linear layer outputs the final one-dimensional quality index prediction value sequence. (See attached image) Figure 4 As shown, the output branch is drawn from the last slow feature analysis layer, flattened, and then fed into the fully connected prediction layer. The intermediate layers do not participate in the construction of the final features.

[0094] Both architectures address the fundamental problem of the separation between feature extraction and prediction targets in the background technology, but they have different focuses in feature utilization mechanisms: the DSFN-Fusion architecture ensures that both shallow perceptual details and deep abstract semantics can directly participate in the final prediction through a feature fusion strategy, effectively avoiding the risk of information loss that may be caused by deep networks; while the DSFN-Final architecture constructs a deep feature filtering mechanism, which automatically discards high-frequency and rapid fluctuations that are redundant to the prediction task by utilizing the layer-by-layer transformation of deep networks, maximizing the purity of core slowly changing features and the stability of prediction results. In practical applications, the choice can be made according to the dynamic characteristics of the specific industrial process and the degree of emphasis on prediction stability.

[0095] Step S5: Construct the joint loss function and perform end-to-end co-optimization training.

[0096] Step S5-1: Calculate the prediction error term.

[0097] After the deep slow feature network completes forward propagation and outputs predicted values ​​of the quality indicators, the model training and optimization module first receives two input objects: one is the sequence of predicted values ​​finally output by the network, and the other is the sequence of actual measured values ​​of the quality indicators corresponding to the input data obtained from the training set. For these two input objects, the module performs the following operations: for each sample in the batch, it calculates the square of the difference between the predicted value and the true value, then sums these squared differences for all samples in the batch, and finally divides the sum by the number of samples in the batch to obtain a scalar value. This scalar value reflects the average deviation between the current model prediction result and the actual working condition; the smaller the value, the more accurate the prediction. This output result is called the prediction error term, denoted as the first loss component, and is passed to step S5-3 to construct the joint loss.

[0098] Step S5-2: Calculate the slow feature constraint term.

[0099] Simultaneously with step S5-1, the model training and optimization module also collects the slow feature vector sequence output by each slow feature analysis layer of the deep slow feature network as input. For each slow feature vector sequence collected from a layer, this module performs the following operations along the time dimension: For slow feature vectors at two adjacent time steps in the sequence, calculate the difference vector between them, and then calculate the square of the length of the difference vector; traverse all consecutive time step pairs of this layer, sum up the squared differences of all adjacent steps, and then divide by the number of time steps minus one and the batch sample number to obtain the average temporal change of this layer. Subsequently, sum up the average temporal changes calculated by all slow feature analysis layers, and then divide by the total number of layers to obtain a scalar value. This scalar value measures the degree of fluctuation of the slow features extracted by each layer of the network in the time dimension. The smaller the value, the smoother the feature change and the more it conforms to the slow variation constraint. This output result is called the slow feature constraint term, denoted as the second loss component, and is also passed to step S5-3.

[0100] Step S5-3: Construct the joint loss function.

[0101] Following the first loss component output from step S5-1 and the second loss component output from step S5-2, the model training and optimization module also receives a preset balancing weight coefficient as a third input. This coefficient ranges from zero to one and is used to adjust the relative importance between the prediction accuracy target and the feature stability target. For these three inputs, the module performs the following weighted combination operation: multiplying the first loss component by one minus the balancing weight coefficient, and simultaneously multiplying the second loss component by the balancing weight coefficient, then adding the two products to obtain a scalar value. This scalar value simultaneously includes the penalty for prediction error and the penalty for rapid feature fluctuations, achieving a unified expression of the supervised learning objective and the unsupervised slow-changing constraints. This output result is called the joint loss value and is passed to step S5-4 for network parameter updates.

[0102] Step S5-4: Perform end-to-end network parameter updates.

[0103] The model training and optimization module receives the joint loss value output from step S5-3 as input and initiates the backpropagation process. This module first calculates the partial derivative of the joint loss value with respect to each trainable parameter in the deep slow feature network. These trainable parameters include the weights and biases of the first linear layer, the weights and biases of the second linear layer, the linear projection matrix, and the weights and biases of the fully connected network in the final prediction output layer. After obtaining all gradients, this module uses the Adaptive Moment Estimator (Adam) to update each parameter element-wise based on the gradient information: the new value of the parameter equals the current value minus the learning rate multiplied by the gradient estimate. This update process applies to all the trainable parameters simultaneously in one backpropagation, achieving end-to-end collaborative optimization from input to predicted output. One update completes one training batch iteration; after traversing all batches in the training set, one training cycle is completed. Through repeated iterations over multiple training cycles, the joint loss value gradually decreases to convergence, ultimately yielding the trained slow feature network model.

[0104] The joint training process described in steps S5-1 to S5-4 above has computational logic that is consistent with the appendix to this application specification. Figure 3 and attached Figure 4 In the network output architecture shown, whether it's the feature fusion architecture or the final output architecture, all slow feature analysis layer outputs before the prediction output layer are used for two purposes simultaneously: first, they are fed into subsequent layers or concatenation layers to generate the final predicted value and calculate the prediction error term; second, they are collected to calculate the temporal differences of features in each layer to obtain the slow feature constraint term. This parallel collection and dual computation structure ensures that while pursuing prediction accuracy, the network is always guided by the feature stationarity constraint.

[0105] Step S6: Online prediction.

[0106] The trained deep slow feature network is deployed in an online system; for real-time collected industrial process data, the network performs forward computation based on its internally updated persistent covariance matrix to output real-time predicted values ​​of key quality indicators.

[0107] Understandable, Figure 1 This is another way of expressing the steps of the above method embodiment, and the specific details will not be described in detail.

[0108] To illustrate the method in this embodiment in more detail, a specific method example is given below:

[0109] This example uses the prediction of key indicators in the Alumina Digestion Process (ALDP) as an example to illustrate the implementation process of the method of the present invention in detail. It includes the following steps:

[0110] Step S100: Establish a mathematical model for the industrial process prediction task and perform data preprocessing.

[0111] Historical time-series data of industrial processes were collected as the training set. B is the batch size, T' is the time step, and n is the variable dimension. Given a set of real numbers, the original data is standardized to satisfy the statistical properties of zero mean and unit variance.

[0112] Step S200: Construct a deep slow feature network (DSFN) containing multiple cascaded slow feature analysis units.

[0113] Build a A deep neural network consisting of stacked SFA layers, with each layer having a hidden dimension of 64. The slow feature analysis layer sequentially includes a nonlinear transformation unit, a covariance matrix calculation unit, a whitening transformation unit, and a linear projection unit, used to perform the following feature transformations:

[0114] Step S201: Pass the input through the first linear layer Mapped to a high-dimensional space, the features are obtained after passing through a hyperbolic tangent activation function and then through a second linear layer. The calculation formula is as follows:

[0115] ;

[0116] ;

[0117] In the formula, These are trainable parameters.

[0118] Step S202: Flatten the data and calculate the covariance matrix of the current batch. :

[0119] ;

[0120] Step S203: For the covariance matrix Perform singular value decomposition (SVD), construct the whitening transformation matrix U, and then whiten the data to obtain... Eliminate feature correlation:

[0121] The singular value decomposition formula is: , The formula for calculating the whitening transformation matrix U is as follows:

[0122]

[0123] In the formula, ϵ is the regularization constant. It is an identity matrix.

[0124] Step S204: Extract slow features using a trainable orthogonal weight matrix W. :

[0125] ;

[0126] Key parameter configuration and design basis: To achieve stable, efficient, and adaptive prediction, the core parameters in this embodiment are set as follows, and their design is based on targeted considerations of the non-stationary characteristics of industrial processes:

[0127] SFA layer count L=3: This setting constructs a progressive feature extraction architecture with "shallow-medium-deep" abstraction capabilities. Fewer than 3 layers will result in insufficient feature representation capabilities, making it difficult to extract deep, slow features; too many layers can easily lead to overfitting and increase computational burden. The three-layer structure showed the best performance balance in experiments, but in practical applications, the number of layers can be adaptively adjusted according to the nonlinearity of the data and the time-dependent complexity.

[0128] Hidden dimension q=64: This dimension balances the model's expressive power and computational efficiency, sufficiently capturing the coupling information in multivariate industrial process data. This dimension is not fixed and can be flexibly configured according to the scale and coupling degree of the input variables in different industrial scenarios.

[0129] EMA attenuation coefficient This setting is crucial for balancing the model's tracking capability and robustness against interference. This coefficient assigns 95% weight to historical statistics, effectively smoothing out random noise fluctuations within a batch; simultaneously, it retains 5% weight for current information to slowly track data distribution shifts. For industrial processes with varying dynamic characteristics, this coefficient can be fine-tuned to meet different needs, such as prioritizing rapid tracking or long-term stability.

[0130] Joint loss weights This weight is a core parameter for co-optimizing prediction accuracy and feature stationarity. It ensures that the model prioritizes prediction accuracy (corresponding to 90% weight) while guiding the network to learn slowly changing features with a small but consistent gradient (corresponding to 10% weight). This value has been experimentally verified to effectively improve feature stationarity. In practical applications, the strength of the feature stationarity constraint can be adjusted according to the specific task requirements.

[0131] Step S300: Update the persistent covariance matrix and exponential moving average.

[0132] During model training, a persistent covariance matrix is ​​established to adapt to the distribution drift of non-stationary data. As the internal state of the model. Using the exponential moving average strategy, the formula is as follows:

[0133] ;

[0134] In the formula, The attenuation coefficient is taken as 0.95 here. For the updated persistent covariance matrix, The persistent covariance matrix before the update. This is the covariance matrix calculated for the current batch.

[0135] This mechanism is used to smooth the distribution drift of non-stationary data and suppress random noise fluctuations.

[0136] Step S400: Based on the predicted scenario requirements, construct and select the network output architecture.

[0137] The output architecture of the deep slow feature network can be one of the following two:

[0138] Feature Fusion Architecture (DSFN-Fusion): as attached Figure 3 As shown, the slow feature vectors output by each SFA layer in the network are extracted. The output features of all SFA layers are concatenated along the feature dimension to form a comprehensive feature vector f, which is then input into the fully connected layer for prediction.

[0139] ;

[0140] Final output architecture (DSFN-Final): Extracts only the highly abstract slow feature vector from the output of the last SFA layer in the network. After flattening it into a feature vector, it is directly input into the fully connected layer for prediction.

[0141] Step S500: End-to-end training and parameter optimization based on the joint loss function.

[0142] Define the joint loss function Includes prediction error term Slow feature constraint term :

[0143] ;

[0144] In the formula, ;

[0145] Prediction error term The calculation formula is:

[0146] ;

[0147] In the formula, y is the actual value. This is a predicted value;

[0148] Intended to punish rapid fluctuations in characteristics:

[0149] ;

[0150] In this example, to ensure stable convergence of the deep slow feature network under the joint loss function, the following specific training strategy is adopted:

[0151] Optimizer and Learning Rate Scheduling: The Adam optimizer is preferred, as its adaptive learning rate characteristic helps achieve stable optimization under the complex joint loss function of this scheme. The initial learning rate is set to... It employs a stepped decay strategy, multiplying the learning rate by a decay factor of 0.9 every 10 training epochs. This strategy helps achieve rapid convergence in the early stages of training and allows for fine-tuning of parameters in the later stages.

[0152] Batch processing mechanism: The batch size B is set to 64. This size achieves a good balance between computational efficiency and statistical reliability of gradient estimation, and is also conducive to stable estimation of the batch covariance matrix.

[0153] Training stability enhancement measures: To prevent outliers in industrial data from causing instability in the optimization process, this embodiment preferably introduces gradient pruning techniques, such as limiting the gradient norm to within a threshold of 1.0. This effectively prevents gradient explosion, suppresses training oscillations, and ensures the robustness of the overall optimization process.

[0154] Step S600: Online deployment and real-time prediction.

[0155] The trained model is deployed to an online system. For real-time acquired industrial process data, the model performs forward computation based on its internally updated persistent covariance matrix, outputting real-time predicted values ​​of key quality indicators, thus achieving high-precision quality prediction that adapts to changes in operating conditions.

[0156] Implementation effect

[0157] To verify the effectiveness of the method of this invention, experiments were conducted on simulation and real industrial process data, and comparisons were made with various benchmark models. The experimental configuration is as follows: all models were implemented using PyTorch and trained on a single NVIDIA RTX 3090 GPU. DSFN model settings. Each layer has an SFA layer with a hidden dimension q=64. The comparison models cover various representative methods, including: the classic statistical model Autoregressive Moving Average with Exogenous Inputs (ARMAX); the traditional machine learning method Support Vector Regression (SVR); and the widely used deep learning models such as Long Short-Term Memory (LSTM), Temporal Convolutional Network (TCN), and Transformer. To quantitatively evaluate the accuracy of the method in predicting the caustic ratio during the alumina dissolution process (ALDP), the following methods were used: MAE, MAPE, and RMSE are used as evaluation indicators:

[0158] Table 1 shows a comparison of the results in the simulation of long-term trend prediction tasks:

[0159] Table 1 Comparison of Simulation Experiment Performance

[0160]

[0161] As shown in Table 1, the DSFN-Final architecture, focusing on extracting deep, abstract, slow features, achieved the best RMSE (0.1784) and the highest R² (0.9745) in the trend prediction task. The DSFN-Fusion architecture, by fusing multi-layer features, also demonstrated superior performance in balancing multi-scale information (RMSE of 0.2976 and R² of 0.9657), significantly outperforming the listed comparison models. This verifies the effectiveness of the deep slow feature network designed in this invention in theoretically extracting the inherently slow-changing driving factors of data and suppressing random noise.

[0162] The experimental results above demonstrate that the method of this invention has significant advantages over existing mainstream models in trend prediction tasks.

[0163] Example 2

[0164] This invention provides a non-stationary industrial process quality prediction system based on deep slow feature networks, comprising:

[0165] The training set construction module is used to build the training set;

[0166] The slow feature network model training module is used to train a slow feature network model based on multiple cascaded slow feature analysis layers using the training set. It includes: initializing the preset persistent covariance matrix in each slow feature analysis layer; calculating the instantaneous covariance matrix based on the current batch data of the training set; updating the persistent covariance matrix using the instantaneous covariance matrix; and outputting the slow feature vector sequence extracted by each slow feature analysis layer.

[0167] The slow feature network model training module includes an output architecture selection and generation submodule, which is used to generate a feature fusion architecture or a final output architecture according to the configuration; wherein, the output of the feature fusion architecture is a comprehensive feature vector obtained by concatenating the feature vectors output by each slow feature analysis layer in the feature dimension, and the output of the final output architecture is the deep feature vector output by the last slow feature analysis layer.

[0168] The online prediction service module is used to standardize newly collected time-series data and input it into the trained slow feature network model to obtain real-time predicted values ​​of key quality indicators.

[0169] In one feasible implementation, the non-stationary industrial process quality prediction system based on deep slow feature networks can also be composed of the following modules: a data acquisition and preprocessing module, a slow feature network model module, a model internal state update module, a model training and optimization module, and an online prediction service module.

[0170] The data acquisition and preprocessing module is connected to the input of the slow feature network model module to provide standardized time-series data for the model. The model internal state update module is embedded in each slow feature analysis unit of the slow feature network model module to maintain and update the hierarchical persistent covariance matrix. The slow feature network model module is connected to the model training and optimization module to perform parameter optimization during the training phase. The input of the online prediction service module is connected to the trained slow feature network model module, and its output is used to output the real-time prediction results of key quality indicators.

[0171] Data acquisition and preprocessing module: Connects to the data source of the industrial process to acquire multivariate time series data, and performs standardization and sliding window slicing on the data to build a training dataset and real-time prediction input data that can be used by the model;

[0172] Slow Feature Network Model Module: Connected to the output of the data acquisition and preprocessing module; the module includes a deep network composed of multiple cascaded slow feature analysis units. Each slow feature analysis unit sequentially integrates a nonlinear transformation submodule, a covariance calculation submodule, a whitening transformation submodule, and a linear projection submodule, with the function of each submodule corresponding one-to-one with the units defined in step S2 of the first aspect. The module also includes an output architecture selection and generation submodule, used to generate fused features or final layer features according to the configuration, and to map the features to predicted values ​​of key quality indicators through a prediction output submodule.

[0173] Model internal state update module: embedded in each of the slow feature analysis units; this module maintains and manages a persistent covariance matrix as the internal state of the corresponding unit; during each model forward calculation, this module receives the current batch covariance matrix output from the covariance calculation submodule in the current unit, and performs the exponential moving average update operation defined in step S3 of the first aspect to dynamically update the persistent covariance matrix.

[0174] Model training and optimization module: connected to the slow feature network model module; this module is configured to receive training data, call the model to perform forward computation, calculate the loss according to the joint loss function defined in step S5 of the first aspect, and use the backpropagation algorithm to perform end-to-end optimization of all trainable parameters;

[0175] Online prediction service module: connected to the trained slow feature network model module and real-time data stream; this module is responsible for loading the solidified model parameters, receiving the preprocessed process data in real time, driving the model to perform forward calculations, and outputting real-time prediction results of key quality indicators.

[0176] To further understand the technical solution of the present invention, the present invention will be described in detail below with reference to specific implementation examples.

[0177] This embodiment provides a non-stationary industrial process quality prediction system based on a deep slow feature network for implementing the method described in Example 1. Figure 5 As shown, the system can be deployed on industrial field computing nodes, edge servers, or industrial cloud platforms.

[0178] The system includes: a data acquisition and preprocessing module, a slow feature network model module, a model internal state update module, a model training and optimization module, and an online prediction service module. The output of the data acquisition and preprocessing module is connected to the input of the slow feature network model module; the model internal state update module is embedded within each slow feature analysis unit of the slow feature network model module; the slow feature network model module is bidirectionally connected to the model training and optimization module; the input of the online prediction service module is connected to the trained slow feature network model module, and its output is used to output the prediction results.

[0179] Data Acquisition and Preprocessing Module: This module establishes a physical connection with data sources from industrial processes (such as Distributed Control Systems (DCS), historical databases, or LIMS systems) via industrial protocols (such as OPC-UA or internal APIs). This module incorporates data cleaning and transformation operators, capable of performing standardization and sliding time window slicing on the acquired raw multivariate time-series data, converting it into a format suitable for model input. The batch tensor data provides a standardized data source for subsequent calculations.

[0180] Slow Feature Network Model Module: The module internally constructs a system consisting of... A deep network composed of cascaded slow feature analysis units, with each unit having a hidden dimension of 64. Each slow feature analysis unit integrates, in sequence:

[0181] Nonlinear transformation submodule: Executes the nonlinear high-dimensional mapping defined in step S2 of Example 1;

[0182] Covariance calculation submodule: Real-time statistical analysis of batch covariance matrix in feature space;

[0183] Whitening transformation submodule: Eliminates redundant coupling between features through singular value decomposition (SVD);

[0184] The linear projection submodule is used to extract slow features using a trainable orthogonal weight matrix. This model module also includes a configurable output architecture selection and generation submodule, which can generate feature representations of either a feature fusion architecture (DSFN-Fusion) or a final output architecture (DSFN-Final) based on a preset selection, and map the features to the final key quality metric predictions through a prediction output submodule (usually a fully connected network).

[0185] Model Internal State Update Module: This module is embedded within each level of slow feature analysis unit and manages the unit's internal memory state. During system operation, this module receives the output of the covariance calculation submodule in real time and applies a decay coefficient. The exponential moving average (EMA) algorithm is used to update the persistent covariance matrix online. This module ensures that the system can smoothly track the drift of non-stationary process conditions without interrupting the prediction task.

[0186] Model Training and Optimization Module: This module is deployed in a computing environment equipped with a single NVIDIA RTX 3090 GPU and is responsible for parameter optimization in the offline phase. It receives the preprocessed training set and optimizes it based on... The model uses a joint loss function oriented towards performance metrics such as RMSE, and employs the Adam optimizer and backpropagation mechanism to iteratively optimize the network weights until the model converges and outputs a fixed optimal parameter file.

[0187] Online Prediction Service Module: This module is activated during the production phase and is responsible for loading the fixed parameters and attaching them to the real-time data pipeline. It drives the slow feature network model to perform forward inference and pushes the generated predicted values ​​of key quality indicators (such as caustic ratio) to the plant-level monitoring screen or feeds them back to the control loop in real time.

[0188] It is worth noting that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of protection defined in this application.

Claims

1. A method for predicting the quality of non-stationary industrial processes based on deep slow feature networks, characterized in that, include: Construct the training set; The training process involves training a slow feature network model based on multiple cascaded slow feature analysis layers using the training set, including: initializing the pre-defined persistent covariance matrix within each slow feature analysis layer; and calculating the instantaneous covariance matrix based on the current batch of data in the training set. The persistent covariance matrix is ​​updated using the instantaneous covariance matrix, and the slow feature vector sequence extracted by each slow feature analysis layer is output. The newly collected time-series data is standardized and then input into the trained slow feature network model to obtain real-time predicted values ​​of key quality indicators.

2. The method for predicting the quality of non-stationary industrial processes based on deep slow feature networks according to claim 1, characterized in that, The construction of the training set includes: Collect raw, multivariate time-series data from industrial process sites; Standardization preprocessing is performed on the original multivariate time series data to obtain standardized time series data; A training set was constructed by selecting standardized time-series historical data of industrial processes. B is the batch size, T' is the time step, and n is the variable dimension. It is the set of real numbers.

3. The method for predicting the quality of non-stationary industrial processes based on deep slow feature networks according to claim 1, characterized in that, The method of training a slow feature network model based on multiple cascaded slow feature analysis layers using a training set also includes: The slow feature vector sequences extracted from all slow feature analysis layers are flattened and concatenated to form a single comprehensive feature vector; or Extract the slow feature vector sequence of the last slow feature analysis layer; Flatten the slow feature vector sequence of the last slow feature analysis layer into a deep feature vector.

4. The method for predicting the quality of non-stationary industrial processes based on deep slow feature networks according to claim 3, characterized in that, The method of training a slow feature network model based on multiple cascaded slow feature analysis layers using a training set also includes: The comprehensive feature vector or deep feature vector, along with the real quality index values ​​in the training set that correspond one-to-one with the comprehensive feature vector or deep feature vector, are used as the input to the slow feature network model. The comprehensive feature vector or deep feature vector is calculated by a fully connected prediction layer, and the predicted value of the quality index is output. Based on the predicted value and the actual quality index value, the prediction error term is calculated. Slow feature constraint terms characterizing the degree of temporal fluctuation are calculated based on the slow feature vector sequences output by each slow feature analysis layer. The prediction error term and the slow feature constraint term are weighted and combined to construct a joint loss function for optimization.

5. The method for predicting the quality of non-stationary industrial processes based on deep slow feature networks according to claim 4, characterized in that, The slow feature constraint term, which characterizes the degree of temporal fluctuation, is calculated based on the slow feature vector sequence output by each slow feature analysis layer; including: Calculate the difference vector between the slow feature vectors of two adjacent time steps in any slow feature analysis layer, and then calculate the square of the length of the difference vector. Traverse all consecutive time step pairs of the slow feature analysis layer and accumulate the squared differences of all adjacent time steps; divide the accumulated result by the product of the time step size minus one and the batch sample size to obtain the average temporal change of the slow feature analysis layer. Traverse each slow feature analysis layer to obtain the average time series change corresponding to each slow feature analysis layer; sum the average time series changes of each layer, and then divide by the total number of slow feature analysis layers to obtain a single scalar value.

6. The method for predicting the quality of non-stationary industrial processes based on deep slow feature networks according to claim 1, characterized in that, The instantaneous covariance matrix is ​​calculated based on the current batch of data in the training set, using the following expression: ； in, The instantaneous covariance matrix of the current batch of data. This is the initial feature matrix after flattening, where B is the batch size. For time step, This represents the total number of samples after flattening the data in the current batch. Subtracting 1 gives an unbiased estimate of the covariance. This is the transpose of the initial feature matrix after flattening.

7. The method for predicting the quality of non-stationary industrial processes based on deep slow feature networks according to claim 1, characterized in that, The method of training a slow feature network model using a training set also includes: The instantaneous covariance matrix is ​​decomposed into an orthogonal matrix composed of eigenvectors and a diagonal matrix composed of eigenvalues; Introducing a very small regularization constant, a whitening transformation matrix is ​​constructed based on the orthogonal matrix and the diagonal matrix; Multiply the feature matrix of the current batch of data by the transpose of the constructed whitening transformation matrix.

8. The method according to claim 1, characterized in that, The method of updating the persistent covariance matrix using the instantaneous covariance matrix specifically involves: updating the persistent covariance matrix using an exponential moving average update strategy, with the corresponding expression as follows: ； in, For the updated persistent covariance matrix, The persistent covariance matrix before the update. The instantaneous covariance matrix is ​​calculated based on the current batch of data. This is the attenuation coefficient.

Images