A method for monitoring faults in nonstationary industrial processes based on self-adjusting nonstationary converter networks

Abstract

This invention discloses a fault monitoring method for non-stationary industrial processes based on a self-adjusting non-stationary transformer network. First, an enhanced Dickey-Fuller test is used to distinguish between stationary and non-stationary variables. For non-stationary variables, a sequence stabilization module is designed to convert them into stationary variables, improving the model's reconstruction capability. Simultaneously, a de-stationary attention module is proposed to compensate for the non-stationary information lost during stabilization, ensuring that the model retains and fully utilizes the inherent non-stationary information when processing non-stationary data. Second, a two-stage training strategy based on weight self-adjustment is proposed. The first stage aims to reconstruct the input window data, and the reconstruction bias from the first stage is used as prior knowledge for updating the attention weights in the second stage attention module. This makes the model pay more attention to time series with larger biases, thereby accelerating the model's convergence speed and improving the accuracy and reliability of fault monitoring.

Description

Technical Field

[0001] This invention relates to the field of fault monitoring technology for non-stationary industrial processes, and specifically to a fault monitoring method based on a self-adjusting non-stationary converter network. Background Technology

[0002] With the rapid development of industrial automation, the trends of intelligent industrial equipment and industrial integration are becoming increasingly apparent, production scale is constantly expanding, and processes are becoming more complex. Once a malfunction occurs, it can affect the production performance of the entire industrial chain. Therefore, real-time fault monitoring is crucial for ensuring the safety and reliability of modern industrial systems. Modern equipment and systems are equipped with many advanced sensors capable of collecting large amounts of operational data in real time and at high frequency. Utilizing modern data analysis techniques, hidden patterns and regularities can be discovered from complex operational data, enabling the monitoring of complex industrial systems. Therefore, data-driven fault monitoring methods have attracted much attention and have become the mainstream approach.

[0003] Data-driven fault monitoring methods mainly include signal processing, multivariate statistical analysis, and machine learning. Signal processing extracts key features by analyzing various signals (such as vibration, temperature, and pressure) in industrial processes to identify abnormal behavior. Multivariate statistical analysis uses statistical tools to model and analyze the correlations of multiple process variables, thereby monitoring potential fault modes. Machine learning, on the other hand, automatically learns and identifies fault modes in complex systems through training on large amounts of historical data. These methods are typically based on the assumption that industrial processes are stationary, meaning that the statistical properties of process data, such as the mean and variance, do not change over time. However, in actual industrial processes, due to variations in raw material characteristics, equipment status, environmental conditions, and process parameters, the statistical properties of the data may change, exhibiting non-stationary characteristics, which can mask fault information. This non-stationarity makes it difficult for traditional methods to accurately monitor faults, leading to missed detections. Therefore, fault monitoring of non-stationary industrial processes remains a significant challenge.

[0004] To address this challenge, many scholars have conducted exploratory research in recent years. Currently, mainstream methods for monitoring non-stationary process faults include adaptive modeling, cointegration analysis, subspace decomposition, and trend analysis. Adaptive modeling monitors non-stationary processes by dynamically constructing a process model that tracks changes in non-stationarity; cointegration analysis relies on stable long-term equilibrium relationships between data points to monitor non-stationary processes through cointegration models; subspace decomposition separates stationary subspaces from the entire data space to extract inherent stationary features; and trend analysis extracts feature information from time series data to analyze the changing trends of the process. However, these methods increase computational complexity and reduce model processing efficiency when dealing with large amounts of data. Furthermore, they struggle to capture deep-seated features and long-term dependencies in complex industrial systems, leading to false alarms and missed alarms. Therefore, directly applying these methods to non-stationary process monitoring often fails to yield accurate results.

[0005] With the rapid development of deep learning (DL), some scholars have adopted deep learning-based methods for industrial process fault monitoring. Data reconstruction-based fault monitoring is an effective means of monitoring complex industrial processes. This method learns the distribution characteristics of normal data and uses reconstruction errors to identify abnormal data. Transformer models, due to their unique self-attention mechanism, can effectively capture long-range dependencies in time series data, while overcoming the sequential dependencies of traditional sequence models and enabling parallel computing, thus improving computational efficiency. In the field of industrial process fault monitoring, data reconstruction methods based on transformer models have been extensively studied. For example, Thomas et al. proposed a novel model combining a deep convolutional neural network (CNN) with a transformer to automatically monitor fault type, phase, and location; Zhu et al. proposed a novel model combining a variable-centric transformer and a large-kernel spatiotemporal convolution, achieving good monitoring results on industrial system datasets; Shang et al. proposed a transformer model based on a residual autoencoder, realizing fault monitoring of industrial processes under high-dimensional and high-noise conditions; in addition, Peng et al. established an efficient fault monitoring model for wastewater treatment processes by utilizing position encoding, residual connections, and multi-head attention mechanisms in the transformer structure, achieving accurate fault monitoring through a combination of offline modeling and online monitoring.

[0006] While data reconstruction methods based on transformer networks offer advantages in capturing long-range dependencies and improving computational efficiency in complex industrial processes, they suffer from limitations when handling non-stationary data. These methods typically assume a stable data distribution, meaning the statistical properties of the data do not change significantly over time. However, in real-world industrial processes, the statistical properties of data can change, exhibiting non-stationary characteristics. This non-stationarity makes it difficult for models to accurately capture data trends, leading to false positives or false negatives. Furthermore, because deep learning models rely on large amounts of historical data for training, they struggle to adapt to significant non-stationary characteristics in a timely manner, impacting the real-time performance and accuracy of monitoring. Therefore, although data reconstruction methods perform well with stationary data, their accuracy and robustness need improvement when dealing with non-stationary data. Summary of the Invention

[0007] To address the issues of non-stationarity in industrial process data and the difficulty in distinguishing between outlier and non-stationary data, leading to low accuracy in fault monitoring, an innovative monitoring method based on a Self-Adjusting Nonstationary Transformer (SANT) network is proposed. First, a sequence stabilization module is designed to transform non-stationary variables identified through an enhanced Dickey-Fuller test into stationary variables, thereby eliminating the interference of non-stationarity on fault information and improving the model's reconstruction accuracy and monitoring accuracy. Second, a de-stationarity attention module is introduced into the traditional transformer network to compensate for the non-stationary information lost during stabilization, ensuring that the model retains and utilizes the inherent non-stationary information in the data while processing it, further enhancing the monitoring effect. Furthermore, a two-stage training strategy with self-adjusting weights and an S-section is proposed, and a focus score mechanism is introduced to focus on time series with significant biases, effectively extracting long-term dependencies and deep-level features from the data, significantly improving the monitoring accuracy and robustness of non-stationary process fault data.

[0008] To achieve the above objectives, the present invention employs the following technical solution:

[0009] The fault monitoring method based on self-adjusting non-stationary converter networks is characterized by the following steps:

[0010] A. Offline modeling stage:

[0011] 1) Obtain historical data of industrial processes as raw data Where n represents the number of samples, m represents the sample dimension, and m also represents the number of data points in each sample. The maximum value of each column vector of the original data X is calculated as max(x). j ) and minimum value min(x) j Standardized data X is obtained

[0012]

[0013] Where j = 0, 1, ..., n-1, x j Let represent the j-th column vector of the original data X, and let ε be a small constant to prevent the denominator from being zero. After multiple experiments, ε is set to 0.001.

[0014] 2) Standardize the data Divided into training sample set and test sample set The sum of p and q is n;

[0015] 3) The ADF test can be used to distinguish the training sample set. Non-stationary variable datasets and stationary variable dataset The sum of z and v is p;

[0016] 4) For the detected non-stationary variables, sequence stabilization is performed to transform them into stationary sequences. Finally, all processed data (including variables that were originally stationary and those that have undergone stabilization) are integrated and concatenated to form the final stationary training data. The specific steps are as follows:

[0017] First, the non-stationary variable dataset is processed using a sliding window with a window size of 8. The dataset is divided into B windows along the time dimension, and the dataset corresponding to each window is... The dataset corresponding to the b-th window Where b = 1, 2, ..., B, This represents the i-th sample in the b-th window;

[0018] Secondly, the non-stationary data in each window All data are transformed into stationary data through translation and scaling operations. The specific calculation formula is as follows:

[0019]

[0020] Where μ x This represents the mean of non-stationary data. σ represents the variance of non-stationary data. x This represents the standard deviation of non-stationary data, where m represents the sample dimension and / represents element-wise division. This represents element-wise multiplication;

[0021] Finally, the originally stationary variables and the variables that have undergone stabilization are combined and concatenated to obtain the final stationary training data.

[0022] 5) Construct a transformer reconstruction module based on a self-regulating strategy. Introduce a destationary attention module into the traditional transformer network to compensate for the non-stationary information lost due to stabilization operations. Utilize a self-regulating two-stage training strategy and introduce a focus score mechanism as prior knowledge for updating attention weights, making the model pay more attention to time series with larger biases, thereby accelerating the model's convergence speed. Specific steps are as follows:

[0023] Stable training data The training data W = {w1,...,w} is obtained by dividing the sliding window. H}, where the dataset corresponding to the ω-th window is Where ω = 1, 2, ... H, and C represents the size of the sliding window. After multiple experiments, C is set to 10 here.

[0024] When generating embedding vectors for each window of training data, positional encoding information is incorporated to capture the contextual positional information of the data. The specific formula for positional encoding is as follows:

[0025]

[0026] Where pos refers to the current position of the data in the sequence, 2ψ represents the even dimension of the embedding vector, and 2ψ represents their odd dimension;

[0027] In the transformer, the encoder module, with its internal multi-head attention mechanism, can dynamically capture the feature information of the input vector, thereby greatly increasing the encoder's grasp of global feature information. Self-attention updates each component of the sequence by aggregating global contextual information from the complete input sequence. The formula for self-attention can be expressed as:

[0028]

[0029] Where, d φ The dimension of the signal is represented by 32, which is the dimension of the key-value vector. Q, K and V represent the matrices corresponding to the query vector, key-value vector and value vector, respectively.

[0030] Multi-head attention is a mechanism that includes multiple self-attention modules. It can encapsulate complex relationships between different elements in a sequence. Assuming there are h self-attention modules, multi-head attention transforms a given input vector into three different sets of vectors, each set containing h vectors of dimension D / h, where D represents the total dimension of the input vectors. Then, the vectors from the different inputs are combined into different matrices. and Therefore, the formula for the multi-head attention mechanism can be expressed as:

[0031]

[0032] head r =Attention(Q) r ,K r V r (6)

[0033] Where Q′, K′ and V′ are respectively and Cascade, It is a linear projection matrix. The linear projection matrix in the self-attention module generates query, key, and value vectors by applying a learnable weight matrix to the input features, thereby achieving effective information extraction and attention.

[0034] Training data carrying location information is input into a transformer reconstruction module based on a self-adjusting strategy. The module extracts deep features from the input data and utilizes a self-attention mechanism to effectively capture the dependencies between data points in the time series while preserving the location information of the input data. By processing the extracted features, the module reconstructs the input data, thereby generating an output that matches the original input data. This process not only ensures the preservation of the temporal relationships of the data but also enhances the model's ability to understand and process complex time series data. The specific steps are as follows:

[0035] The converter reconstruction module based on the self-adjusting strategy consists of two encoders and two decoders. The two encoders are a sequence encoder and a window encoder, respectively. The two decoders have the same model structure, but the model structure design of the feedforward network is different. The feedforward network structure in decoder 1 is 1024×512, while the feedforward network structure in decoder 2 is 512×256.

[0036] First, the sequence encoder captures long-range dependencies in the fully stationary training data through a self-attention mechanism, transforming the fully stationary training data... The focus score F (initially a 0 matrix, with dimensions equal to the fully stationary training data) (The same) are concatenated, and then positional encoding is used to generate the input E1 of the sequence encoder. In the sequence encoder, the following operations are performed:

[0037]

[0038] Where Add represents a residual connection, LayerNorm represents a layer normalization operation, DSmultihead(·) represents the computational operation of the destabilizing attention module, and FeedForward represents the computational operation of the feedforward network. This represents the feature matrix extracted by the destationary attention module in the sequence encoder. This represents the self-attention feature matrix output by the sequence encoder. The destationary attention module mainly recovers the non-stationary information lost during the sequence stabilization process and solves the over-stabilization problem caused by sequence stabilization. The specific operation of the destationary attention module is as follows:

[0039]

[0040] in The standard deviation of the training samples is represented. denoted as the mean of the training samples, log represents the logarithmic operation, Attn represents the self-attention calculation, the de-stationing factors τ and Δ are shared by the de-stationing attention modules of all layers, and MLP(·) represents the multilayer perceptron layer;

[0041] Secondly, the output of the sequence encoder As input to the multi-head self-attention module in the window encoder, the window training data W is embedded with positional encoding information as input E2 to the window encoder. The output of the window encoder is the feature matrix. The specific operation of the window encoder is as follows:

[0042]

[0043] Mask(Multihead(·)) represents masking multihead attention computation. This represents the feature matrix extracted by the occlusion multi-head attention module in the window encoder. This indicates the output of the window encoder;

[0044] Finally, the output of the window encoder The input is processed by the decoder to obtain the final reconstructed output. The specific steps are as follows:

[0045]

[0046] Where sigmoid(·) represents the sigmoid activation function, which maps input values ​​to the range of 0 to 1, converting continuous values ​​into probability outputs, D u This represents the output of the decoder;

[0047] 6) The sum of the self-adjusting two-stage reconstruction biases is used as the loss function for model training. The monitoring statistics are calculated using the feature matrix output by the encoder of the converter network and the reconstruction error of the decoder. The monitoring control limits are determined according to the kernel density estimation method.

[0048] In the first stage, the goal of the model is to generate time-series data that is identical to the training data in the input window, using the reconstruction bias as the focus score F. The reconstruction bias is calculated as follows:

[0049] L fp=||D1-W||2 (11)

[0050] Where L fp The first stage represents the reconstruction bias, W represents the window training data, and D1 is the reconstruction output of the first decoder in the first stage. First, the encoder extracts the features of the input training data. The specific operation is shown in formulas (7), (8), and (9). Then, the extracted feature matrix is ​​input into the decoder to obtain the reconstruction output D1. The specific operation is shown in formula (10).

[0051] In the second stage, the reconstruction deviation L from the first stage is... fp As the focus score F, the focus score F is compared with the fully stationary training data. The concatenation is used as input to the sequence encoder. Features of the input data are extracted by the sequence encoder and window encoder, and then passed through a second decoder to obtain the reconstructed output of the second stage. The focus score generated in the first stage represents the deviation between the reconstructed output and the given input. This serves as prior knowledge for updating the attention weights in the second stage, and the weight update enhances the neural network's ability to respond to specific inputs, enabling the model to more effectively capture and utilize deep features in the input data. This method is called "self-adjustment." The formula for calculating the deviation in the second stage is as follows:

[0052]

[0053] in, L represents the reconstruction output of the second stage. sp This indicates the reconstruction deviation in the second stage;

[0054] During the model training phase, for the training dataset Its goal is to minimize the model loss, using the sum of the two-stage reconstruction biases as the model training loss function:

[0055]

[0056] Where λ is the training period, with a value of 10, and ζ represents the training parameters, set to 0.8;

[0057] Based on the trained model, T is calculated using the feature matrix output by the transformer network window encoder. 2 The SPE statistic, T, is calculated using the reconstruction bias of the converter network decoder. 2 The formulas for calculating the SPE statistic and the SPE statistic are as follows:

[0058]

[0059] Where x(k) is the current observation vector, It is the mean of the eigenvectors, ∑H It is the covariance matrix of the eigenvectors;

[0060]

[0061] Where res(k) represents the residual vector. It is the mean of the residual vector, ∑ RES Yes, the covariance matrix of the residual vector;

[0062] For the calculated T 2 The control limits for the SPE statistic and the SPE statistic are determined using kernel density estimation. The formula for kernel density estimation is as follows:

[0063]

[0064] Where g represents the number of samples, h > 0 is the kernel density bandwidth parameter, set to 0.5, and T s 2 and SPE s For each sample in each sample point in the training set, T 2 The Gaussian kernel statistic and SPE statistic, where K(·) represents the Gaussian kernel function, are calculated as follows:

[0065]

[0066] in, This represents the distance between the observation point and the point to be estimated;

[0067] B. Online monitoring phase:

[0068] 1) Test sample set The ADF test was used to distinguish the dataset of stationary variables. Non-stationary variable dataset

[0069] 2) Use a sliding window of size 8 to process the non-stationary variable dataset. The dataset is divided into B windows along the time dimension, and the dataset corresponding to each window is... The dataset corresponding to the b-th window Where b = 1, 2, ..., B, This represents the i-th sample in the b-th window;

[0070] 3) Use formula (2) to The variables are then stabilized, and the originally stable variables are combined with the stabilized variables to obtain the final stable test data.

[0071] 4) Stabilize the test data The sliding window is divided to obtain window training data W'={w'1,...,w' H}, using formulas (3) and (4) to embed position encoding information into the window training data W';

[0072] 5) Stabilize the test data Input the sequence encoder and use formulas (7) and (8) to obtain the self-attention feature output of the stationary test data.

[0073] 6) The output of the sequence encoder As the input to the multi-head self-attention module in the window encoder, the window test data W' with embedded position encoding information is used as the input E'2 of the window encoder, and the output feature matrix is ​​obtained using formula (9).

[0074] 7) Output of the window encoder Input the decoder and use formula (10) to obtain the final reconstructed output D. test The reconstruction error is calculated using formula (13);

[0075] 8) Calculate T using formulas (14) and (15) 2 The parameters in the formula remain unchanged for the SPE statistic and the SPE statistic. When the statistic exceeds the control limit obtained in the offline modeling stage, it indicates that the sample point has failed; otherwise, no failure has occurred.

[0076] 9) The effectiveness of fault monitoring is evaluated using two metrics: Fault Detection Rate (FDR) and Fault False Alarm Rate (FAR). The calculation formulas for these two metrics are as follows:

[0077]

[0078] Where J represents the monitoring statistic, J th The monitoring control limit is represented by 37. Through experimental verification, the monitoring control limit is set to 37. f = 0 and f ≠ 0 represent no fault and fault occurrence, respectively.

[0079] Beneficial effects

[0080] This invention proposes a method for monitoring non-stationary industrial process faults based on a self-regulating non-stationary transformer network. First, a sequence stabilization module is designed to transform non-stationary data identified by the ADF test into stationary data, helping the model to more accurately reconstruct time-series data and improve the accuracy of fault monitoring. Simultaneously, a de-stationarization attention module is proposed to compensate for the non-stationary information lost during stabilization, ensuring that the model retains and fully utilizes the inherent non-stationary information when processing non-stationary data. Furthermore, this invention designs a reconstruction module based on a self-regulating strategy. During model training, this module employs a self-regulating two-stage training strategy and introduces a focus score mechanism to focus on time series with significant deviations, enabling the model to fully extract long-term dependencies and deep-level features from the data, significantly improving the accuracy and robustness of monitoring non-stationary process fault data. In summary, this invention fully considers the characteristics of the data itself when monitoring non-stationary industrial process faults, possessing advantages such as strong representational ability, high targeting, and high monitoring accuracy. This is of great significance for the monitoring of non-stationary industrial process faults. Attached Figure Description

[0081] Figure 1 The diagram shown is a flowchart of the SANT model of this invention;

[0082] Figure 2 The diagram shown is a schematic of the sequence stabilization module;

[0083] Figure 3 The diagram shown is a schematic of the refactoring module;

[0084] Figure 4 The diagram shown is a flowchart of the TE process;

[0085] Figure 5 The image shows the fault monitoring performance of the model under fault 8 conditions in the TE dataset.

[0086] Figure 6 The image shows the fault monitoring performance of the model under fault 10 in the TE dataset; Detailed Implementation

[0087] To address the shortcomings of existing technologies, this invention provides a method for monitoring faults in non-stationary industrial processes based on a self-adjusting non-stationary converter network. This method can effectively extract long-term dependencies and deep-level features from non-stationary process data, thereby improving the accuracy of fault monitoring in non-stationary industrial processes.

[0088] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0089] Reference Figure 1 This invention provides a method for monitoring faults in nonstationary industrial processes based on a self-adjusting nonstationary Transformer (SANT) network. First, historical industrial process data is acquired and standardized. The standardized process data is then divided into training and testing data. Second, a sequence stabilization module is constructed. The training data is tested using the Augmented Dickey-Fuller (ADF) unit root test to distinguish between stationary and nonstationary variables. For the detected nonstationary variables, sequence stabilization is applied to transform them into stationary sequences. All processed data (including both originally stationary and stabilized variables) are integrated and concatenated to form the final stationary training data. The structure of the sequence stabilization module is as follows: Figure 2 As shown in the diagram. Then, a reconstruction module based on a self-regulating strategy is constructed. This module utilizes a destationary attention module to extract long-term dependencies and deep-level features from the stationary training data, while preventing the loss of non-stationary information. A self-regulating two-stage training strategy is employed to introduce subsequences with significant focus score bias, extracting effective deep-level features and reconstructing the input stationary training data. The structure of the self-regulating reconstruction module is shown in the diagram. Figure 3 As shown in the figure, the monitoring statistics are calculated using the feature matrix obtained from the training and reconstruction module and the reconstruction error. The monitoring control limits are determined using kernel density estimation. Based on the trained model and the monitoring control limits, fault monitoring of non-stationary industrial processes is achieved. The fault monitoring process for non-stationary industrial processes based on SANT is as follows: Figure 1 As shown.

[0090] The TE process, a benchmark simulation platform proposed by Eastman, is widely used to verify the performance of fault monitoring methods. For example... Figure 4The flowchart of the TE process is shown. According to Table 1, the TE process includes 41 measured variables and 12 control variables. However, control variables 5 (compressor recirculation valve), 9 (stripper water flow valve), and 12 (stirring speed) are always 0, having no impact on fault feature extraction. Therefore, these three variables were removed in this invention, and only 50 variables were used for experiments. The TE process can be divided into 5 operating units: condenser, reactor, separator, stripper, and compressor. The TE production process involves four gaseous feedstocks, A, B, C, and D, which participate in the reaction to produce two main products, G and H, and also generate a byproduct F. The TE process covers 21 fault types, including 16 known faults and 5 unknown faults. The known faults are mainly classified into five categories: step, random variable, slow drift, viscous, and constant position. This invention conducted comparative experiments on these 21 faults, and the results show that the proposed method performs well in most cases. The entire operation lasted 38 hours, with sampling of the 50 variables every 3 minutes. The training and testing data comprised a total of 760 samples, with anomalies appearing in the testing data starting from the 150th sample. The TE process involves multiple variables, not all of which are non-stationary. Therefore, this invention performed ADF tests on 50 variables, and the ADF test results for the non-stationary variables are listed in Table 2.

[0091] In terms of hyperparameter design, the SANT model uses the Adam optimizer for model optimization, with an initial learning rate set to 0.001, which decays to 0.1 after every 100 training epochs. The batch size is 128, and the model iterated a total of 10 times. Regarding model structure design, SANT uses a 512-dimensional positional encoding in the data processing part to add positional information to each position in the input sequence. After rigorous experimental verification, the window size for the window normalization module is set to 8; for the stationary time series, the window size is set to 10. The window encoder and sequence encoder have similar structures and most parameters are the same, with the encoder having 5 layers, the multi-head attention module having 8 heads, and both input and output dimensions being 512. The feedforward network uses a Linear (512×1024) structure, and the Dropout Rate is set to 0.1 to prevent overfitting of the neural network. In the decoder section, the feedforward network adopts a Linear (1024×512) structure, using the sigmoid activation function, and a Dropout Rate of 0.1 is also set. Furthermore, to further optimize model performance, the following key parameters were considered: First, the weight initialization employed the Xavier initialization method to ensure model stability in the early stages of training. Second, batch normalization was used in the normalization layers to accelerate the training process and stabilize model convergence. Simultaneously, to enhance the model's robustness, residual connections and layer normalization were introduced in each multi-head attention module; these measures help prevent gradient vanishing or exploding. In addition, to improve the model's adaptability to practical applications, regularization was applied, with the L2 regularization coefficient set to 0.0001 to suppress overfitting. Finally, the SANT model training process employed an early stopping strategy, setting the tolerance for the validation set loss to no more than 10 iterations to prevent overfitting and improve training efficiency. All of these hyperparameter settings have been fine-tuned through multiple experiments to ensure the model performs optimally when processing complex time series data.

[0092] Based on the above description, and in accordance with the invention, the specific process is implemented as follows:

[0093] A. Offline modeling stage:

[0094] 1) First, obtain historical industrial process data as raw data. Where n represents the number of samples, m represents the sample dimension, and also represents the number of data points contained in each sample;

[0095] 2) Standardize the original data using formula (1), and divide the processed data into training datasets. and test dataset The sum of p and q is n;

[0096]

[0097] Where j = 0, 1, ..., n-1, and ε represents a small constant to prevent the denominator from being zero. After multiple experiments, ε was set to 0.001.

[0098] 3) For the training dataset Perform the ADF test to distinguish non-stationary variable datasets. and stationary variable dataset

[0099] 4) Use a sliding window of size 8 to divide the non-stationary variable dataset. The dataset is divided into B windows along the time dimension, and the dataset corresponding to each window is... The dataset corresponding to the b-th window Where b = 1, 2, ..., B, This represents the i-th sample in the b-th window;

[0100] 5) Using formula (2), the non-stationary data of each window are processed. Transform into stationary data The processed data is then concatenated to obtain the final stable training data.

[0101]

[0102] Where μ x This represents the mean of non-stationary data. σ represents the variance of non-stationary data. x This represents the standard deviation of non-stationary data, where m represents the sample dimension and / represents element-wise division. This represents element-wise multiplication;

[0103] 6) Stabilize the training data The training data W = {w1,...,w} is obtained by dividing the sliding window. H}, where the dataset corresponding to the ω-th window is Where ω=1,2,…H, C represents the sliding window size. After multiple experiments, C is set to 10. The position encoding information is embedded in the window training data W' using formulas (3) and (4).

[0104]

[0105] Where pos refers to the current position of the data in the sequence, 2ψ represents the even dimension of the embedding vector, and 2ψ+1 represents their odd dimension;

[0106] 7) Training data carrying location information The input data is fed into the converter module, and the self-attention features of the input data are extracted using formulas (5) and (6).

[0107]

[0108] Where, d φ This represents the dimension in which the signal is transformed into a specific key-value vector, where Q, K, and V represent the matrices corresponding to the query vector, key-value vector, and value vector, respectively.

[0109]

[0110] head r =Attention(Q) r ,K r V r (6)

[0111] Where Q′, K′ and V′ are respectively and Cascade, It is a linear projection matrix;

[0112] 8) Stabilize the training data The focus score F (initially a 0 matrix, with dimensions equal to the fully stationary training data) The same data are concatenated, and then the input E1 of the sequence encoder is generated by position encoding. E1 is input into the sequence encoder, and the self-attention feature output of the stationary test data is obtained by using formulas (7) and (8).

[0113]

[0114] Where Add represents a residual connection, LayerNorm represents a layer normalization operation, DSmultihead(·) represents the computational operation of the destabilizing attention module, and FeedForward represents the computational operation of the feedforward network. This represents the feature matrix extracted by the destationary attention module in the sequence encoder. This represents the self-attention feature matrix output by the sequence encoder;

[0115]

[0116] in The standard deviation of the training samples is represented. denoted as the mean of the training samples, log represents the logarithmic operation, Attn represents the self-attention calculation, the de-stationing factors τ and Δ are shared by the de-stationing attention modules of all layers, and MLP(·) represents the multilayer perceptron layer;

[0117] 9) The output of the sequence encoder As the input to the multi-head self-attention module in the window encoder, the window training data W with embedded positional encoding information is used as the input E2 of the window encoder, and the output feature matrix is ​​obtained using formula (9).

[0118]

[0119] Mask(Multihead(·)) represents masking multihead attention computation. This represents the feature matrix extracted by the occlusion multi-head attention module in the window encoder. This indicates the output of the window encoder;

[0120] 10) Output the feature matrix from the window encoder. As the input to the decoder, the reconstructed output D is obtained using formula (10). i ;

[0121]

[0122] Sigmoid(·) represents the sigmoid activation function, which maps input values ​​to the range of 0 to 1 and converts continuous values ​​into probabilistic outputs.

[0123] 11) Calculate the reconstruction error according to formulas (11)-(13) and perform self-adjusting two-stage model training;

[0124] L fp =||D1-W||2 (11)

[0125] Where L fp The first stage represents the reconstruction bias, W represents the window training data, and D1 is the reconstruction output of the first decoder in the first stage. First, the encoder extracts the features of the input training data. The specific operation is shown in formulas (7), (8), and (9). Then, the extracted feature matrix is ​​input into the decoder to obtain the reconstruction output D1. The specific operation is shown in formula (10).

[0126]

[0127] in, L represents the reconstruction output of the second stage. sp This indicates the reconstruction deviation in the second stage;

[0128]

[0129] Where λ is the training period, with a value of 10, and ζ represents the training parameters, with a value of 0.8;

[0130] 12) Calculate the monitoring statistics using the feature matrix and reconstruction error according to formulas (14)-(15);

[0131]

[0132] Where x(k) is the current observation vector, It is the mean of the eigenvectors, ∑ H It is the covariance matrix of the eigenvectors;

[0133]

[0134] Where res(k) represents the residual vector. It is the mean of the residual vector, ∑ RES Yes, the covariance matrix of the residual vector;

[0135] 13) Determine the monitoring control limits using the kernel density estimation method according to formulas (16)-(17);

[0136]

[0137] Where g represents the number of samples, h > 0 is the kernel density bandwidth parameter, set to 0.5, and T s 2 and SPE s For each sample in each sample point in the training set, T 2 Statistics and SPE statistics;

[0138] B. Online monitoring phase:

[0139] 1) Test sample set The ADF test was used to distinguish the dataset of stationary variables. Non-stationary variable dataset

[0140] 2) Use a sliding window of size 8 to process the non-stationary variable dataset. The dataset is divided into B windows along the time dimension, and the dataset corresponding to each window is... The dataset corresponding to the b-th window Where b = 1, 2, ..., B, This represents the i-th sample in the b-th window;

[0141] 3) Use formula (2) to The variables are then stabilized, and the originally stable variables are combined with the stabilized variables to obtain the final stable test data.

[0142] 4) Stabilize the test data The sliding window is divided to obtain window training data W'={w'1,...,w' H}, using formulas (3) and (4) to embed position encoding information into the window training data W';

[0143] 5) Stabilize the test data Input the sequence encoder and use formulas (7) and (8) to obtain the self-attention feature output of the stationary test data.

[0144] 6) The output of the sequence encoder As the input to the multi-head self-attention module in the window encoder, the window test data W' with embedded position encoding information is used as the input E'2 of the window encoder, and the output feature matrix is ​​obtained using formula (9).

[0145] 7) Output of the window encoder Input the signal to the decoder and use formula (10) to obtain the final reconstructed output D'. test The reconstruction error is calculated using formula (11);

[0146] 8) Calculate T using formulas (14) and (15) 2 The parameters in the formula remain unchanged for the SPE statistic and the SPE statistic. When the statistic exceeds the control limit obtained in the offline modeling stage, it indicates that the sample point has failed; otherwise, no failure has occurred.

[0147] The above steps illustrate the specific application of this method in TE process fault monitoring. Test data is used to evaluate the model's fault monitoring performance, and the monitoring control limits are used to determine whether the test data indicates a fault. Figure 5 This demonstrates the fault monitoring performance of the SANT model on the TE dataset under fault 8 scenarios, including T 2 Statistical graphs and SPE statistical graphs. Figure 6This paper demonstrates the fault detection performance of the SANT model on the TE dataset under fault 10 conditions. Two metrics, Fault Detection Rate (FDR) and Fault False Alarm Rate (FAR), are used to evaluate the model's fault detection performance on non-stationary process data under different fault conditions. In addition to the method proposed in this invention, the experimental section also compares seven other methods: DPCA, SMRCAE, SSA, AAE, NSCA, LSTMED, and USAD. DPCA, SSA, and NSCA are statistical learning methods, while SMRCAE, AAE, LSTMED, and USAD are deep learning methods. The monitoring statistics for all comparative methods are constructed in the same way as the SANT method, and all eight methods use a 95% control limit confidence level. All methods are first trained on normal data, and then their monitoring performance on fault data is tested. Table 3 details the comparison of fault monitoring performance of eight models on the TE dataset under 21 different fault types. For relatively easy-to-monitor faults such as faults 1, 2, 4, 6, 7, 8, 12, 13, 14, 17, and 18, most monitoring methods achieve a monitoring rate of over 90%, but their false alarm rates fluctuate significantly. The DPCA method, for example, has a false alarm rate of 0.08 for faults 4 and 13. For minor faults such as faults 3, 5, 9, and 15, the monitoring performance of all models is unsatisfactory. The fault monitoring rate of this invention is superior to other models in these fault types, indicating that the fault monitoring performance of this invention is better. For faults 19, 20, and 21, the monitoring rate of all monitoring models still has room for improvement. The fault monitoring results of the SANT method under fault 10 are shown in [Table 3]. Figure 6 The upper and lower rows in the figure show the T values ​​of the monitoring method, respectively. 2 The SANT statistic and SPE statistic, threshold control limits are represented by dashed lines, and vertical lines represent fault locations. Fault 10 is a random variation in the C feed temperature. The SANT method can detect anomalies at 150 points, and within 150 sampling points, the SANT method's T... 2Both the statistical measures and the SPE statistical measures plots did not exceed the control limits, and no false alarms occurred. However, after 700 sampling points, the SANT method may identify some fault points as normal. The proposed SANT-based method achieves an FDR of 97% under fault 10, which is superior to other comparative methods. Simultaneously, its false alarm rate is only 0.01, also lower than other methods, indicating that this method outperforms others in non-stationary process monitoring. Analysis of Table 3 shows that the monitoring rate of this invention is higher than other models in most fault conditions. In faults 1, 2, 6, 14, 17, and 18, the monitoring rate of this invention reaches 100%, and the average false alarm rate is the lowest among all models. In conclusion, this invention can effectively model non-stationary process data, ensuring full exploitation of the temporal dependencies and deep-level characteristics of process data, and improving the accuracy of non-stationary industrial process fault monitoring. Therefore, it is of great significance for non-stationary industrial process fault monitoring.

[0148] Table 1 Description of TE process variables

[0149]

[0150]

[0151] Table 2. ADF test results for non-stationary variables during the TE process.

[0152]

[0153] Table 3 Comparison of fault monitoring performance on the TE dataset

[0154]

[0155]

Claims

1. A method for monitoring faults in non-stationary industrial processes based on a self-adjusting non-stationary converter network, characterized in that, Includes the following steps: A. Offline modeling stage: A1 first acquires historical industrial process data and performs data standardization processing. Then, the standardized process data is divided into training data and test data. A2 constructs a sequence stabilization module, which uses the Augmented Dickey-Fuller (ADF) unit root test to distinguish between stationary and non-stationary variables in the training data. For the detected non-stationary variables, sequence stabilization is performed to transform them into stationary sequences. Finally, all the processed data are integrated and spliced ​​together to form the final stationary training data. All data includes variables that were originally stationary and variables that have undergone stabilization. A3 constructs a transformer reconstruction module based on a self-regulating strategy. It extracts long-term dependencies and deep features of stationary training data through a destationary attention module, while preventing the loss of non-stationary information. It adopts a self-regulating two-stage training strategy, introduces focus scores, focuses on sequences with large deviations, and reconstructs the input stationary training data. A4 uses the sum of two-stage reconstruction biases as the loss function for model training, calculates monitoring statistics using the feature matrix output by the encoder of the converter network and the reconstruction error of the decoder, and determines the monitoring control limit based on the kernel density estimation method. B. Online monitoring phase: B1 uses the ADF test to distinguish between stationary and non-stationary variables in the test data. For the detected non-stationary variables, serial stabilization is performed to transform them into stationary series. Finally, all the processed data are integrated and spliced ​​together to form the final stationary test data. B2 uses the transformer reconstruction module of the trained self-adjusting strategy to calculate the monitoring statistics of stationary test data; B3 uses the comparison between the monitoring statistic and the control limit to determine whether a fault has occurred. When the statistic exceeds the control limit obtained in the offline modeling stage, it indicates that a fault has occurred in the sample point; otherwise, no fault has occurred.

2. The method for monitoring faults in non-stationary industrial processes based on a self-adjusting non-stationary converter network according to claim 1, characterized in that: The specific steps of A1 are as follows: 1) Obtain historical data under normal industrial process conditions as training data. Where n represents the number of samples, m represents the sample dimension, and m also represents the number of data points in each sample. The maximum value of each column vector of the training data X is calculated as max(x). j ) and minimum value min(x) j Standardized data X is obtained Where j = 0, 1, ..., n-1, x j Let represent the j-th column vector of the original data X, and ε be a small constant to prevent the denominator from being zero, which is set to 0.001; 2) Standardize the data Divided into training sample set and test sample set The sum of p and q is n.

3. The method for monitoring faults in non-stationary industrial processes based on a self-adjusting non-stationary converter network according to claim 1, characterized in that, In step A2, the stationary and non-stationary variables are distinguished using the ADF unit root test. The specific steps are as follows: 1) The ADF test can be used to distinguish the training sample set. Non-stationary variable datasets and stationary variable dataset The sum of z and v is p; 2) First, use a sliding window of size 8 to divide the non-stationary variable dataset. The dataset is divided into B windows along the time dimension, and the dataset corresponding to each window is... The dataset corresponding to the b-th window Where b = 1, 2, ..., B, This represents the i-th sample in the b-th window; Where i = 1, 2, ..., z; 3) Convert the non-stationary data in each window All data are transformed into stationary data through translation and scaling operations. 4) Integrate and concatenate the originally stationary variables and the variables that have undergone stabilization to obtain the final stationary training data.

4. The method for monitoring faults in non-stationary industrial processes based on a self-adjusting non-stationary converter network according to claim 1, characterized in that, The specific steps for A3 are as follows: 1) Divide the stable training data into sliding windows to obtain window training data, and embed position encoding information into each window training data; 2) Input the fully stationary training data into the sequence encoder. The sequence encoder captures long-range dependencies in the fully stationary training data through a self-attention mechanism. First, input the fully stationary training data... The input E1 of the sequence encoder is concatenated with the focus score F, and then generated through position encoding. In the sequence encoder, the following operations are performed: where the initial matrix is ​​a 0-matrix, and the dimension is the same as the fully stationary training data. same; Where Add represents a residual connection, LayerNorm represents a layer normalization operation, DSmultihead(·) represents the computational operation of the destabilizing attention module, and FeedForward represents the computational operation of the feedforward network. This represents the feature matrix extracted by the destationary attention module in the sequence encoder. This represents the self-attention feature matrix output by the sequence encoder. The destationary attention module mainly recovers the non-stationary information lost during the sequence stabilization process and solves the over-stabilization problem caused by sequence stabilization. The specific operation of the destationary attention module is as follows: in The standard deviation of the training samples is represented. denoted as the mean of the training samples, log represents the logarithmic operation, Attn represents the self-attention calculation, the de-stationing factors τ and Δ are shared by the de-stationing attention modules of all layers, and MLP(·) represents the multilayer perceptron layer; 3) Output of the sequence encoder As input to the multi-head self-attention module in the window encoder, the window training data W is embedded with positional encoding information and used as input E2 to the window encoder. The specific operation of the window encoder is as follows: Mask(Multihead(·)) represents masking multihead attention computation. This represents the feature matrix extracted by the occlusion multi-head attention module in the window encoder. This indicates the output of the window encoder; 4) Output of the window encoder The input is processed by the decoder to obtain the final reconstructed output. The specific steps are as follows: Where sigmoid(·) represents the sigmoid activation function calculation, which maps input values ​​to the range of 0 to 1 and converts continuous values ​​into probabilistic outputs.

5. The method for monitoring faults in non-stationary industrial processes based on a self-adjusting non-stationary converter network according to claim 1, characterized in that, In step A3, a self-regulating two-stage training strategy is used to introduce focus scores to address subsequences with large biases, thereby extracting effective deep-level features. The specific steps are as follows: 1) In the first stage, the goal of the model is to generate a reconstruction of the input window, using the reconstruction bias as the focus score F. The reconstruction bias is calculated as follows: L fp =||D1-W||2 (6) Where L fp The first stage represents the reconstruction bias, W represents the window training data, and D1 is the reconstruction output of the first decoder in the first stage. First, the encoder extracts the features of the input training data. The specific operations are shown in formulas (2), (3), and (4). Then, the extracted feature matrix is ​​input into the decoder to obtain the reconstruction output D1. The specific operations are shown in formula (5). 2) In the second stage, the reconstruction deviation L from the first stage is... fp As the focus score F, the focus score F is compared with the fully stationary training data. The concatenation is used as input to the sequence encoder. Features of the input data are extracted by the sequence encoder and window encoder, and then passed through a second decoder to obtain the reconstructed output of the second stage. The formula for calculating the deviation in the second stage is as follows: L sp =||D2-W||2 (7) Where D2 represents the reconstruction output of the second stage, L sp This indicates the reconstruction deviation in the second stage.

6. The method for monitoring faults in non-stationary industrial processes based on a self-adjusting non-stationary converter network according to claim 1, characterized in that, The specific steps of step A4 are as follows: 1) The sum of the two-stage reconstruction biases is used as the loss function for model training. The specific calculation formula is as follows: L=ζ -λ ||D1-W||2+(1-ζ -λ )||D2-W||2 (8) Where λ is the training period, with a value of 10, and ζ represents the training parameters, with a value of 0.8; 2) Calculate T using the feature matrix output from the second stage of the transformer network window encoder. 2 The SPE statistic, T, is calculated using the reconstruction bias of the second stage of the converter network decoder. 2 The formulas for calculating the SPE statistic and the SPE statistic are as follows: Where x(k) is the current observation vector, It is the mean of the eigenvectors, ∑ H It is the covariance matrix of the eigenvectors; Where res(k) represents the residual vector. It is the mean of the residual vector, ∑ RES Yes, the covariance matrix of the residual vector; 3) For the calculated T 2 The control limits for the SPE statistic and the SPE statistic are determined using kernel density estimation. The formula for kernel density estimation is as follows: Where g represents the number of samples, h > 0 is the kernel density bandwidth parameter, set to 0.

5. and SPE s For each sample in each sample point in the training set, T 2 The Gaussian kernel statistic and SPE statistic, where K(·) represents the Gaussian kernel function, are calculated as follows: in, This represents the distance between the observation point and the point to be estimated.

7. The method for monitoring faults in non-stationary industrial processes based on a self-adjusting non-stationary converter network according to claim 1, characterized in that, B specifically includes the following steps: 1) Test sample set The ADF test was used to distinguish the dataset of stationary variables. Non-stationary variable dataset 2) Use a sliding window of size 8 to process the non-stationary variable dataset. The dataset is divided into B windows along the time dimension, and the dataset corresponding to each window is... The dataset corresponding to the b-th window Where b = 1, 2, ..., B, This represents the i-th sample in the b-th window; where i = 1, 2, ..., z; 3) By using translation and scaling operations... The variables are then stabilized, and the originally stable variables are combined with the stabilized variables to obtain the final stable test data. 4) Stabilize the test data The sliding window is divided to obtain window training data W'={w'1,...,w' H }, and embed position encoding information into the training data for each window; 5) Stabilize the test data Input the sequence encoder and use formulas (2) and (3) to obtain the self-attention feature output of the stationary test data. 6) The output of the sequence encoder As the input to the multi-head self-attention module in the window encoder, the window test data W' with embedded position encoding information is used as the input E′2 of the window encoder, and the output feature matrix is ​​obtained using formula (4). 7) Output of the window encoder Input the signal to the decoder and use formula (5) to obtain the final reconstructed output D. test The reconstruction error is calculated using formula (6); 8) Calculate T using formulas (9) and (10) 2 The parameters in the formula remain unchanged for the statistic and the SPE statistic. When the statistic exceeds the control limit obtained in the offline modeling stage, it indicates that a fault has occurred in the sample point; otherwise, no fault has occurred.

Images