Non-stationary industrial process fault monitoring method based on self-adjusting non-stationary converter network

Abstract

The invention discloses a non-stationary industrial process fault monitoring method based on a self-adjusting non-stationary converter network. The method comprises the following steps: firstly, dividing a variable into a stationary variable and a non-stationary variable by adopting enhanced Dickey-Fuller test; and a sequence stabilization module is designed for non-stationary variables, and the non-stationary variables are converted into stationary variables, so that the reconstruction capability of the model is improved. Meanwhile, a stationary attention removing module is provided to compensate non-stationary information lost by stationary operation, and it is ensured that the model can retain and make full use of inherent non-stationary information when processing non-stationary data. Secondly, a two-stage training strategy based on weight self-adjustment is provided, in the first stage, input window data are reconstructed, the reconstruction deviation of the first stage is used as priori knowledge of attention weight updating of an attention module in the second stage, the model is made to pay more attention to a time sequence with the large deviation, and therefore the convergence speed of the model is increased; and the accuracy and reliability of fault monitoring are improved.

Description

Technical Field

[0001] The invention relates to the technical field of non-stationary industrial process fault monitoring, and in particular to a fault monitoring method based on a self-regulating non-stationary converter network. Background Art

[0002] With the rapid development of industrial automation, the trend of intelligent industrial equipment and industrial integration is becoming increasingly obvious, the scale of production is constantly expanding, and the process is more complicated. Once a failure occurs, it may affect the production performance of the entire industrial chain. Therefore, real-time fault monitoring is essential to ensure the safety and reliability of modern industrial systems. Modern equipment and systems are equipped with many advanced sensors that can collect a large amount of operating data in real time and at high frequency. Using modern data analysis technology, hidden patterns and laws can be discovered from complex operating data to achieve monitoring of complex industrial systems. Therefore, data-driven fault monitoring methods have attracted much attention and have become the current mainstream.

[0003] Data-driven fault monitoring methods mainly include signal processing, multivariate statistical analysis, and machine learning. Signal processing extracts key features by analyzing various signals (such as vibration, temperature, pressure, etc.) in industrial processes to identify abnormal behaviors; multivariate statistical analysis uses statistical tools to model and analyze the correlation of multiple process variables to monitor potential failure modes; and machine learning automatically learns and identifies failure modes in complex systems by training on a large amount of historical data. These methods are usually based on the assumption that industrial processes are stationary, that is, the statistical properties of process data such as the mean and variance do not change over time. However, in actual industrial processes, due to changes in raw material characteristics, equipment status, environmental conditions, and process parameters, the statistical properties of the data may change, showing non-stationary characteristics, resulting in fault information being masked by non-stationary trends. This non-stationarity makes it difficult for traditional methods to accurately monitor faults, which in turn leads to underreporting. Therefore, fault monitoring of non-stationary industrial processes remains a major challenge.

[0004] In response to this problem, many scholars have conducted exploratory research in recent years. At present, the mainstream non-stationary process fault monitoring methods include adaptive modeling, cointegration analysis, subspace decomposition and trend analysis. Adaptive modeling monitors non-stationary processes by dynamically constructing a process model that tracks non-stationary changes; cointegration analysis relies on the stable long-term equilibrium relationship between data and monitors non-stationary processes through cointegration models; subspace decomposition methods separate stationary subspaces from the entire data space to extract the inherent stationary features in the data; trend analysis extracts feature information through time series and analyzes the changing trend of the process. However, these methods increase the computational complexity and reduce the processing efficiency of the model when processing a large number of data samples. In addition, they are difficult to capture the deep characteristics and long-term dependencies in complex industrial systems, resulting in false positives and false negatives. Therefore, it is usually difficult to obtain accurate results when these methods are directly applied to non-stationary process monitoring.

[0005] With the rapid development of deep learning (DL), some scholars have adopted deep learning-based methods for industrial process fault monitoring. The fault monitoring method based on data reconstruction is an effective means of complex industrial process monitoring. This method learns the distribution characteristics of normal data and uses reconstruction errors to identify abnormal data. The Transformer model can effectively capture long-distance dependencies in time series due to its unique self-attention mechanism, while getting rid of the sequential dependencies of traditional sequence models, realizing parallel computing and improving computational efficiency. In the field of industrial process fault monitoring, data reconstruction methods based on transformer models have been widely studied. For example, Thomas et al. proposed a novel model combining a deep convolutional neural network (CNN) with a transformer to automatically monitor fault type, phase and location; Zhu et al. proposed a new model combining a variable-centered transformer and large-kernel spatiotemporal convolution, which achieved good monitoring results on industrial system datasets; Shang et al. proposed a transformer model based on residual autoencoders, which realized fault monitoring of industrial processes under high-dimensional and high-noise conditions; in addition, Peng et al. used position encoding, residual connection and multi-head attention mechanism in the transformer structure to establish an efficient sewage treatment process fault monitoring model, and achieved accurate fault monitoring by combining offline modeling and online monitoring.

[0006] Although data reconstruction methods based on transformer networks have advantages in capturing long-distance dependencies in complex industrial processes and improving computational efficiency, they have shortcomings when dealing with non-stationary data. These methods usually assume that the data distribution is stable, that is, the statistical characteristics of the data do not change significantly over time. However, in actual industrial processes, the statistical characteristics of the data will change and show non-stationary characteristics. This non-stationarity makes it difficult for the model to accurately capture data trends, thereby causing false positives or negatives. In addition, since deep learning models rely on a large amount of historical data for training, when non-stationary features are significant, it is difficult for the model to adapt to these changes in a timely manner, affecting the real-time and accuracy of monitoring. Therefore, although data reconstruction methods perform well when dealing with stationary data, their accuracy and robustness still need to be improved when dealing with non-stationary data. Summary of the invention

[0007] Aiming at the problem of low fault monitoring accuracy caused by the non-stationarity of industrial process data and the difficulty in distinguishing abnormal data from non-stationary data, an innovative monitoring method based on the self-adjusting nonstationary transformer network (SANT) is proposed. Firstly, a sequence stabilization module is designed to convert the non-stationary variables distinguished by the enhanced Dickey-Fuller test into stationary variables, thereby eliminating the interference of non-stationarity on fault information and improving the reconstruction accuracy and monitoring accuracy of the model. Secondly, a de-stationary attention module is introduced into the traditional transformer network to compensate for the non-stationary information lost due to the stabilization operation, ensuring that the model retains and utilizes the intrinsic non-stationary information in the data while processing non-stationary data, further enhancing the monitoring effect. In addition, a weight self-adjusting s-section two-stage training strategy is proposed, and a focus score mechanism is introduced to focus on time series with large deviations, effectively extract long-term dependencies and deep-level features in the data, and significantly improve the monitoring accuracy and robustness of non-stationary process fault data.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] A fault monitoring method based on a self-regulating non-stationary converter network is characterized by comprising the following steps:

[0010] A. Offline modeling stage:

[0011] 1) Obtain historical data of industrial processes as raw data Where n represents the number of samples, m represents the sample dimension, and also represents the number of data points contained in each sample. Calculate the maximum value max(x j ) and the minimum value min(x j ), the standardized data X is obtained

[0012]

[0013] where j = 0, 1, ..., n-1, x j represents the j-th column vector of the original data X, ε represents a small constant to prevent the denominator from being zero. After multiple experiments, ε is set to 0.001;

[0014] 2) Standardize the data Divide into training sample sets and test sample set The sum of p and q is n;

[0015] 3) The ADF test can be used to distinguish the training sample set A dataset of non-stationary variables in and stationary variable datasets The sum of z and v is p;

[0016] 4) For the detected non-stationary variables, sequence stabilization is performed to transform them into stationary sequences. Finally, all processed data (including originally stationary variables and variables that have been stabilized) are integrated and spliced ​​to form the final stationary training data. The specific steps are as follows:

[0017] First, the non-stationary variable dataset is transformed into It is divided into B windows in the time dimension, and the data set corresponding to the window is The data set corresponding to the bth window Where b = 1, 2, ... B, represents the i-th sample in the b-th window;

[0018] Secondly, the non-stationary data of each window All are converted into stable data through translation and scaling operations The specific calculation formula is as follows:

[0019]

[0020] where μ x represents the mean of non-stationary data, represents the variance of non-stationary data, σ x represents the standard deviation of non-stationary data, m represents the sample dimension, and / represents element-by-element division. represents element-wise multiplication;

[0021] Finally, the originally stable variables and the stabilized variables are integrated and spliced ​​to obtain the final stable training data.

[0022] 5) Construct a transformer reconstruction module based on a self-regulating strategy, introduce a destabilizing attention module into the traditional transformer network, compensate for the non-stationary information lost due to the stabilization operation, use a self-regulating two-stage training strategy, and introduce a focus score mechanism as a priori knowledge for updating the attention weight, so that the model pays more attention to time series with large deviations, thereby accelerating the convergence of the model. The specific steps are as follows:

[0023] Stabilize the training data Divide the sliding window to obtain window training data W = {w 1 ,...,w H}, where the data set corresponding to the ω-th window is Where ω=1,2,…H, C represents the sliding window size. After multiple experiments, C is set to 10.

[0024] When generating an embedding vector for each window training data, the position encoding information is combined to capture the contextual position information of the data. The specific formula for position encoding is as follows:

[0025]

[0026] Where pos refers to the position of the current data in the sequence, 2ψ represents the even dimensions of the embedding vectors, and 2ψ represents their odd dimensions;

[0027] The encoder module in the transformer can dynamically capture the feature information of the input vector by virtue of its internal multi-head attention mechanism, thereby greatly increasing the encoder's grasp of global feature information. The function of self-attention is to update each component of the sequence by aggregating global context information from the complete input sequence. The formula for self-attention can be expressed as:

[0028]

[0029] Among them, d φ Indicates the dimension of the signal converted into a specific key-value vector, with a value of 32. Q, K, and V represent the matrices corresponding to the query vector, key-value vector, and value vector, respectively.

[0030] Multi-head attention is a mechanism that includes multiple self-attentions. It can encapsulate multiple complex relationships between different elements in a sequence. Assuming there are h self-attention modules, multi-head attention converts a given input vector into three different groups of vectors, each of which has h vectors of dimension D / h, where D represents the total dimension of the input vector. Then, vectors from different inputs are combined into different matrices: and Therefore, the formula of the multi-head attention mechanism can be expressed as:

[0031]

[0032] head r =Attention(Q r ,K r ,V r ) (6)

[0033] Among them, Q′, K′ and V′ are and Cascade, is the linear projection matrix. The linear projection matrix in the self-attention module generates query, key, and value vectors by applying a learnable weight matrix to the input features, thereby achieving effective information extraction and attention;

[0034] The training data with position information is input into the transformer reconstruction module based on the self-adjustment strategy to extract the deep features of the input data. The reconstruction module uses the self-attention mechanism to effectively capture the dependencies between the data points in the time series and retain the position information of the input data. By processing the extracted features, the reconstruction module can reconstruct the input data to generate output that matches the original input data. This process not only ensures that the temporal relationship of the data is preserved, but also enhances the model's ability to understand and process complex time series data. The specific steps are as follows:

[0035] The transformer reconstruction module based on the self-adjustment strategy consists of two encoders and two decoders. The two encoders are a sequence encoder and a window encoder respectively. The model structures of the two decoders are the same, but the model structures of the feedforward networks are designed differently. The feedforward network structure in decoder 1 is 1024×512, while the feedforward network structure in decoder 2 is 512×256.

[0036] First, the sequence encoder captures the long-range dependencies in the fully stationary training data through the self-attention mechanism. and the focal score F (initialized as a zero matrix with the same dimensions as the fully stationary training data The same) is concatenated, and then the input E of the sequence encoder is generated through position encoding. 1 , in the sequence encoder, do the following:

[0037]

[0038] Where Add represents the residual connection, LayerNorm represents the layer normalization operation, DSmultihead(·) represents the calculation operation of the destabilizing attention module, and FeedForward represents the calculation operation of the feedforward network. represents the feature matrix extracted by the destabilizing attention module in the sequence encoder, Represents the self-attention feature matrix output by the sequence encoder. The destabilizing attention module is mainly used to retrieve the non-stationary information lost in the sequence stabilization process and solve the problem of over-stabilization caused by sequence stabilization. The specific operation of the destabilizing attention module is as follows:

[0039]

[0040] in represents the standard deviation of the training samples, represents the mean of the training samples, log represents the logarithmic operation, Attn represents the self-attention calculation, the destabilization factors τ and Δ are shared by the destabilization attention modules of all layers, and MLP(·) represents the multi-layer perceptron layer;

[0041] Secondly, the output of the sequence encoder As the input of the multi-head self-attention module in the window encoder, the window training data W is embedded in the position encoding information as the input E of the window encoder 2 , the output of the window encoder is the feature matrix, and the specific operation of the window encoder is as follows:

[0042]

[0043] Mask(Multihead(·)) indicates masking the multi-head attention calculation. represents the feature matrix extracted by the masked multi-head attention module in the window encoder, represents the output of the window encoder;

[0044] Finally, the output of the window encoder Input the decoder to get the final reconstructed output. The specific operations are as follows:

[0045]

[0046] Where sigmoid(·) represents the activation function sigmoid calculation, which maps the input value to the range of 0 to 1 and converts the continuous value into a probability output, D u represents the output of the decoder;

[0047] 6) The sum of the self-adjusting two-stage reconstruction deviation is used as the loss function for model training, the monitoring statistics are calculated using the feature matrix output by the transformer network encoder and the reconstruction error of the decoder, and the monitoring control limit is determined based on the kernel density estimation method;

[0048] In the first stage, the goal of the model is to generate time series data that is the same as the input window training data, using the reconstruction deviation as the focal score F. The reconstruction deviation is calculated as follows:

[0049] L fp =||D 1 -W|| 2 (11)

[0050] Where L fp represents the reconstruction deviation of the first stage, W represents the window training data, and D 1 is the reconstructed output of the first decoder in the first stage. First, the encoder extracts the features of the input training data. The specific operation is shown in formulas (7), (8), and (9). Then, the extracted feature matrix is ​​input into the decoder to obtain the reconstructed output D 1 , the specific operation is shown in formula (10);

[0051] In the second stage, the reconstruction deviation L of the first stage is fp As the focal score F, the focal score F is combined with the full stationary training data The splicing is used as the sequence encoder input, the features of the input data are extracted through the sequence encoder and the window encoder, and then the second decoder is used to obtain the reconstructed output of the second stage. The focus score generated in the first stage represents the deviation of the reconstructed output from the given input, which serves as the prior knowledge for updating the attention weights in the second stage and enhances the responsiveness of the neural network to specific inputs through weight updates, enabling the model to more effectively capture and utilize deep features in the input data. This method is called "self-regulation". The second stage deviation calculation formula is as follows:

[0052]

[0053] in, represents the reconstructed output of the second stage, L sp represents the reconstruction deviation in the second stage;

[0054] In the model training phase, for the training data set Its goal is to minimize the model loss, and the sum of the two-stage reconstruction deviations is used as the model training loss function:

[0055]

[0056] Where λ is the training period, which is set to 10, and ζ represents the training parameter, which is set to 0.8;

[0057] According to the trained model, the feature matrix output by the transformer network window encoder is used to calculate T 2 Statistics, the SPE statistics are calculated using the reconstruction deviation of the transformer network decoder, T 2 The calculation formulas for the statistic and the SPE statistic are as follows:

[0058]

[0059] Where x(k) is the current observation vector, is the mean of the eigenvectors, ∑ H is the covariance matrix of the eigenvectors;

[0060]

[0061] where res(k) represents the residual vector, is the mean of the residual vector, ∑ RES is the covariance matrix of the residual vector;

[0062] For the calculated T 2 The statistics and SPE statistics use kernel density estimation to determine the control limits. The kernel density estimation calculation formula is as follows:

[0063]

[0064] Among them, g represents the number of samples, h>0, is the kernel density bandwidth parameter, set to 0.5, T s 2 and SPE s is the T of each sample in each sample point in the training set 2 Statistics and SPE statistics, K(·) represents the Gaussian kernel function, calculated as follows:

[0065]

[0066] in, Represents the distance between the observed point and the point to be estimated;

[0067] B. Online monitoring stage:

[0068] 1) Set the test sample set Using the ADF test to distinguish stationary variable data sets and non-stationary variable datasets

[0069] 2) The non-stationary variable dataset is transformed into It is divided into B windows in the time dimension, and the data set corresponding to the window is The data set corresponding to the bth window Where b = 1, 2, ... B, represents the i-th sample in the b-th window;

[0070] 3) Using formula (2) Perform stabilization processing, then integrate and splice the originally stable variables and the stabilized variables to obtain the final stable test data

[0071] 4) Stabilize test data Divide the sliding window to obtain window training data W'={w' 1 ,...,w' H}, use formulas (3) and (4) to embed the position encoding information of the window training data W';

[0072] 5) Stabilize test data Input sequence encoder, use formula (7) and (8) to get the self-attention feature output of stable test data

[0073] 6) The output of the sequence encoder As the input of the multi-head self-attention module in the window encoder, the window test data W' with embedded position encoding information is used as the input E' of the window encoder 2 , using formula (9) to get the output feature matrix

[0074] 7) Output of the window encoder Input the decoder and use formula (10) to get the final reconstructed output D test , use formula (13) to calculate the reconstruction error;

[0075] 8) Calculate T using formulas (14) and (15) 2 Statistics and SPE statistics, the parameters in the formula remain unchanged. When the statistics exceed the control limit obtained in the offline modeling stage, it indicates that the sample point has a fault, otherwise, no fault has occurred;

[0076] 9) The fault detection rate (FDR) and fault false alarm rate (FAR) are used to evaluate the fault monitoring effect. The calculation formulas of the two indicators are as follows:

[0077]

[0078] Among them, J represents the monitoring statistic, J th It represents the monitoring control limit. Through experimental verification, the monitoring control limit is set to 37. f = 0 and f ≠ 0 represent no fault and fault respectively.

[0079] Beneficial Effects

[0080] The present invention proposes a non-stationary industrial process fault monitoring method based on a self-regulating non-stationary converter network. First, a sequence stabilization module is designed. The non-stationary data identified by the ADF test is converted into stationary data through the sequence stabilization operation, which helps the model to reconstruct the time series data more accurately and improves the accuracy of fault monitoring. At the same time, a de-stationary attention module is proposed to compensate for the non-stationary information lost by the stabilization operation, ensuring that the model can retain and fully utilize its inherent non-stationary information when processing non-stationary data. In addition, the present invention also designs a reconstruction module based on a self-regulating strategy. During the model training process, the module adopts a self-regulating two-stage training strategy and introduces a focus score mechanism to focus on time series with large deviations, so that the model can fully extract long-term dependencies and deep-level features in the data, significantly improving the accuracy and robustness of monitoring non-stationary process fault data. In general, the present invention fully considers the characteristics of the data itself when performing non-stationary industrial process fault monitoring, and has the advantages of strong characterization ability, strong pertinence and high monitoring accuracy. This is of great significance for non-stationary industrial process fault monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 Shown is a flow chart of the SANT model of the present invention;

[0082] Figure 2 Shown is a schematic diagram of the sequence stabilization module;

[0083] Figure 3 Shown is a schematic diagram of the reconstruction module;

[0084] Figure 4 Shown is a flow chart of the TE process;

[0085] Figure 5 The figure shows the fault monitoring effect of the model under fault 8 in the TE data set;

[0086] Figure 6 The figure shows the fault monitoring effect of the model under the condition of fault 10 in the TE data set; DETAILED DESCRIPTION

[0087] In view of the shortcomings of the prior art, the present invention provides a non-stationary industrial process fault monitoring method based on a self-regulating non-stationary converter network, which can effectively extract the long-term dependencies and deep-level characteristics of non-stationary process data, thereby improving the accuracy of non-stationary industrial process fault monitoring.

[0088] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0089] Reference Figure 1 The present invention provides a non-stationary industrial process fault monitoring method based on a self-adjusting nonstationary Transformer (SANT). First, the historical data of the industrial process is obtained and the data is standardized, and then the process data after data standardization is divided into training data and test data. Secondly, a sequence stabilization module is constructed, and the training data is subjected to an ADF (Augmented Dickey-Fuller) unit root test to distinguish between stationary variables and non-stationary variables. For the detected non-stationary variables, sequence stabilization processing is adopted to transform them into stationary sequences. All processed data (including variables that are originally stationary and variables that have been stabilized) are integrated and spliced ​​to form the final stationary training data; the structure of the sequence stabilization module is as follows Figure 2 As shown. Then, a reconstruction module based on the self-regulation strategy is constructed, and the de-stationary attention module is used to extract the long-term dependencies and deep features of the stationary training data, while preventing the loss of non-stationary information of the data. Using the self-regulatory two-stage training strategy, the focus score is introduced to focus on the subsequence with large deviations, extract effective deep features, and reconstruct the input stationary training data; the reconstruction module structure based on the self-regulation strategy is shown in Figure 3 The characteristic matrix and reconstruction error obtained by the training reconstruction module are used to calculate the monitoring statistics, and the monitoring control limit is determined by kernel density estimation. The non-stationary industrial process fault monitoring is realized according to the trained model and monitoring control limit. The non-stationary industrial process fault monitoring process based on SANT is shown in Figure 1 shown.

[0090] The TE process is a benchmark simulation platform proposed by Eastman Company and is widely used to verify the performance of fault monitoring methods. Figure 4The flowchart of the TE process is shown. According to Table 1, the TE process includes 41 measured variables and 12 controlled variables, but the control variables 5 (compressor recirculation valve), 9 (stripper water flow valve) and 12 (stirring speed) are always 0, which has no effect on the extraction of fault features. Therefore, these three variables are removed in the present invention, and only 50 variables are used for experiments. The TE process can be divided into five operating units, namely condenser, reactor, separator, stripper and compressor. The TE production process involves four gaseous raw materials, namely A, B, C and D, which participate in the reaction to generate two main products G and H, and also produce a by-product F. The TE process covers 21 types of faults, including 16 known faults and 5 unknown faults. The known faults are mainly classified into five categories: step, random variable, slow drift, viscosity and constant position. The present invention conducted comparative experiments on these 21 faults, and the results showed that the proposed method performed well in most cases. The entire operation process lasted 38 hours, and 50 variables were sampled every 3 minutes. The training data and test data total 760 samples, and the test data begins to show abnormalities from the 150th sample. There are multiple variables in the TE process, and not all variables are non-stationary. Therefore, the present invention performs an ADF test on 50 variables, and the ADF test results of the non-stationary variables are listed in Table 2.

[0091] In terms of hyperparameter design, the SANT model uses the Adam optimizer for model optimization. The initial learning rate is set to 0.001. After every 100 rounds of training, the learning rate decays to the original 0.1. The batch size of the model is 128, and a total of 10 iterations are performed. In terms of model structure design, SANT sets a 512-dimensional position encoding in the data processing part to add position information to each position in the input sequence. After rigorous experimental verification, the window size of the window normalization module is set to 8; for the time series after stabilization, the size of the window division is set to 10. The window encoder and the sequence encoder have similar structures, and most of the parameter settings are the same. The number of layers of the encoder is 5, the number of heads of the multi-head attention module is 8, and the input and output dimensions are both 512. The feedforward network adopts a Linear (512×1024) structure, and the Dropout Rate is set to 0.1 to prevent the neural network from overfitting. In the decoder part, the feedforward network adopts the Linear (1024×512) structure, the activation function uses sigmoid, and the Dropout Rate is also set to 0.1. In addition, to further optimize the model performance, the following key parameters are also considered: First, the weight initialization adopts the Xavier initialization method to ensure the stability of the model in the early stage of training. Secondly, the normalization layer in the model adopts batch normalization to accelerate the training process and stabilize the model convergence. At the same time, in order to enhance the robustness of the model, residual connections and layer normalization are introduced in each multi-head attention module. These measures help prevent the gradient from disappearing or exploding. In addition, in order to improve the adaptability of the model in practical applications, the model is also regularized, and the L2 regularization coefficient is set to 0.0001 to suppress the risk of overfitting. Finally, the training process of the SANT model adopts the Early Stopping strategy, and the number of times the validation set loss no longer decreases is set to 10 to prevent overfitting and improve training efficiency. All these hyperparameter settings have been fine-tuned through multiple experiments to ensure the best performance of the model when processing complex time series data.

[0092] Based on the above description and according to the content of the invention, the specific process is implemented as follows:

[0093] A. Offline modeling stage:

[0094] 1) First, obtain the industrial process historical data as raw data Where n represents the number of samples, and m represents the sample dimension, which also represents the number of data points contained in each sample;

[0095] 2) Use formula (1) to standardize the original data and divide the processed data into training data sets and test dataset The sum of p and q is n;

[0096]

[0097] Where j = 0, 1, ..., n-1, ε is a small constant to prevent the denominator from being zero. After multiple experiments, ε is set to 0.001;

[0098] 3) Training data set Perform ADF test to distinguish non-stationary variable data sets and stationary variable datasets

[0099] 4) The non-stationary variable dataset is transformed into It is divided into B windows in the time dimension, and the data set corresponding to the window is The data set corresponding to the bth window Where b = 1, 2, ... B, represents the i-th sample in the b-th window;

[0100] 5) Using formula (2), the non-stationary data of each window Convert to stable data Splice the processed data to get the final stable training data

[0101]

[0102] where μ x represents the mean of non-stationary data, represents the variance of non-stationary data, σ x represents the standard deviation of non-stationary data, m represents the sample dimension, and / represents element-by-element division. represents element-wise multiplication;

[0103] 6) Stabilize training data Divide the sliding window to obtain window training data W = {w 1 ,...,w H}, where the data set corresponding to the ω-th window is Where ω = 1, 2, ... H, C represents the sliding window size. After multiple experimental verifications, C is set to 10 here. Formulas (3) and (4) are used to embed the position encoding information of the window training data W';

[0104]

[0105] Where pos refers to the position of the current data in the sequence, 2ψ represents the even dimensions of the embedding vectors, and 2ψ+1 represents their odd dimensions;

[0106] 7) The training data with location information Input into the transformer module, and use formula (5) and formula (6) to extract the self-attention features of the input data;

[0107]

[0108] Among them, d φ Indicates the dimension of the signal converted into a specific key-value vector, Q, K and V represent the matrices corresponding to the query vector, key-value vector and value vector respectively;

[0109]

[0110] head r =Attention(Q r ,K r ,V r ) (6)

[0111] Among them, Q′, K′ and V′ are and Cascade, is the linear projection matrix;

[0112] 8) Stabilize training data and the focal score F (initialized as a zero matrix with the same dimensions as the fully stationary training data The same) is concatenated, and then the input E of the sequence encoder is generated through position encoding. 1 , E 1 Input sequence encoder, use formula (7) and (8) to get the self-attention feature output of stable test data

[0113]

[0114] Where Add represents the residual connection, LayerNorm represents the layer normalization operation, DSmultihead(·) represents the calculation operation of the destabilizing attention module, and FeedForward represents the calculation operation of the feedforward network. represents the feature matrix extracted by the destabilizing attention module in the sequence encoder, Represents the self-attention feature matrix output by the sequence encoder;

[0115]

[0116] in represents the standard deviation of the training samples, represents the mean of the training samples, log represents the logarithmic operation, Attn represents the self-attention calculation, the destabilization factors τ and Δ are shared by the destabilization attention modules of all layers, and MLP(·) represents the multi-layer perceptron layer;

[0117] 9) The output of the sequence encoder As the input of the multi-head self-attention module in the window encoder, the window training data W with embedded position encoding information is used as the input E of the window encoder 2 , using formula (9) to get the output feature matrix

[0118]

[0119] Mask(Multihead(·)) indicates masking the multi-head attention calculation. represents the feature matrix extracted by the masked multi-head attention module in the window encoder, represents the output of the window encoder;

[0120] 10) Output the feature matrix of the window encoder As the decoder input, the reconstructed output D is obtained using formula (10) i ;

[0121]

[0122] Where Sigmoid(·) represents the activation function sigmoid calculation, which maps the input value to the range of 0 to 1 and converts the continuous value into a probability output;

[0123] 11) Calculate the reconstruction error according to formulas (11)-(13) and perform self-adjusting two-stage model training;

[0124] L fp =||D 1 -W|| 2 (11)

[0125] Where L fp represents the reconstruction deviation of the first stage, W represents the window training data, and D 1 is the reconstructed output of the first decoder in the first stage. First, the encoder extracts the features of the input training data. The specific operation is shown in formulas (7), (8), and (9). Then, the extracted feature matrix is ​​input into the decoder to obtain the reconstructed output D 1 , the specific operation is shown in formula (10);

[0126]

[0127] in, represents the reconstructed output of the second stage, L sp represents the reconstruction deviation in the second stage;

[0128]

[0129] Where λ is the training period, the value is 10, ζ represents the training parameter, the value is 0.8;

[0130] 12) According to formulas (14)-(15), the monitoring statistics are calculated using the characteristic matrix and the reconstruction error;

[0131]

[0132] Where x(k) is the current observation vector, is the mean of the eigenvectors, ∑ H is the covariance matrix of the eigenvectors;

[0133]

[0134] where res(k) represents the residual vector, is the mean of the residual vector, ∑ RES is the covariance matrix of the residual vector;

[0135] 13) According to formulas (16)-(17), the monitoring control limits are determined using the kernel density estimation method;

[0136]

[0137] Among them, g represents the number of samples, h>0, is the kernel density bandwidth parameter, set to 0.5, T s 2 and SPE s is the T of each sample in each sample point in the training set 2 Statistics and SPE statistics;

[0138] B. Online monitoring stage:

[0139] 1) Set the test sample set Using the ADF test to distinguish stationary variable data sets and non-stationary variable datasets

[0140] 2) The non-stationary variable dataset is transformed into It is divided into B windows in the time dimension, and the data set corresponding to the window is The data set corresponding to the bth window Where b = 1, 2, ... B, represents the i-th sample in the b-th window;

[0141] 3) Using formula (2) Perform stabilization processing, then integrate and splice the originally stable variables and the stabilized variables to obtain the final stable test data

[0142] 4) Stabilize test data Divide the sliding window to obtain window training data W'={w' 1 ,...,w' H}, use formulas (3) and (4) to embed the position encoding information of the window training data W';

[0143] 5) Stabilize test data Input sequence encoder, use formula (7) and (8) to get the self-attention feature output of stable test data

[0144] 6) The output of the sequence encoder As the input of the multi-head self-attention module in the window encoder, the window test data W' with embedded position encoding information is used as the input E' of the window encoder 2 , using formula (9) to get the output feature matrix

[0145] 7) Output of the window encoder Input the decoder and use formula (10) to get the final reconstructed output D' test , use formula (11) to calculate the reconstruction error;

[0146] 8) Calculate T using formulas (14) and (15) 2 Statistics and SPE statistics, the parameters in the formula remain unchanged. When the statistics exceed the control limit obtained in the offline modeling stage, it indicates that the sample point has a fault, otherwise, no fault has occurred;

[0147] The above steps are the specific application of this method in TE process fault monitoring. The test data is used to test the model fault monitoring performance, and the test data is judged whether it is a fault according to the monitoring control limit. Figure 5 The fault monitoring performance of the SANT model in TE dataset is shown in Figure 8, including T 2 Statistics graph and SPE statistics graph. Figure 6The fault monitoring of the SANT model under 10 faults on the TE data set is demonstrated. Two types of indicators, fault detection rate (FDR) and fault false alarm rate (FAR), are used to evaluate the fault monitoring of the model for non-stationary process data under different fault conditions. In addition to the method proposed in the present invention, the experimental part also uses seven methods, including DPCA, SMRCAE, SSA, AAE, NSCA, LSTMED and USAD, for comparison. Among them, DPCA, SSA and NSCA are statistical learning methods, and SMRCAE, AAE, LSTMED and USAD are deep learning methods. The monitoring statistics of all comparison methods are constructed in the same way as the SANT method, and all eight methods use a 95% control limit confidence level. All methods first use normal data to train the model, and then test its monitoring performance on fault data. Table 3 shows in detail the comparison of fault monitoring effects of 8 models under TE data sets for 21 different fault types. For relatively easy-to-monitor faults such as faults 1, 2, 4, 6, 7, 8, 12, 13, 14, 17 and 18, the monitoring rates of most monitoring methods can reach over 90%, but their false alarm rates fluctuate greatly. The false alarm rate of the DPCA method for faults 4 and 13 is 0.08. For minor faults such as faults 3, 5, 9 and 15, the monitoring performance of all models is not ideal. The fault monitoring rate of the present invention in these types of faults is better than that of other models, indicating that the fault monitoring performance of the present invention is better. For faults 19, 20 and 21, the monitoring rates of all monitoring models still have room for improvement. The fault monitoring results of the SANT method in the case of fault 10 are shown in Table 3. Figure 6 The upper and lower lines in the figure show the T 2 Statistics and SPE statistics, the threshold control limit is represented by a dotted line, and the vertical line represents the fault location. Fault 10 is the random change of C feed temperature. The SANT method can detect the abnormality at 150 points, and within 150 sampling points, the T of the SANT method is 2Both the statistic and the SPE statistic graph did not exceed the control limit, and no false alarm occurred. However, after 700 sampling points, the SANT method may identify some fault points as normal. The proposed SANT method can reach an FDR of 97% under fault 10, which is better than other comparison methods. At the same time, its false alarm rate is only 0.01, which is also lower than other methods, indicating that this method is superior to other methods in terms of non-stationary process monitoring performance. It can be seen from the analysis of Table 3 that the monitoring rate of the present invention in most fault conditions is higher than that of other models. In the faults 1, 2, 6, 14, 17, and 18, the monitoring rate of the present invention can reach 100%, and the average false alarm rate is the lowest among all models. Finally, it can be concluded that the present invention can effectively model non-stationary process data, ensure the full mining of the temporal dependency and deep-level characteristics of the process data, and improve the accuracy of non-stationary industrial process fault monitoring. Therefore, it is of great significance for non-stationary industrial process fault monitoring.

[0148] Table 1TE process variable description

[0149]

[0150]

[0151] Table 2 ADF test results of non-stationary variables in TE process

[0152]

[0153] Table 3 Comparison of fault monitoring effects on TE dataset

[0154]

[0155]

Claims

1. A method for monitoring faults in a non-stationary industrial process based on a self-regulating non-stationary converter network, characterized in that: The following steps are involved: A. Offline modeling stage: A1 first obtains the historical data of the industrial process and performs data standardization, and then divides the standardized process data into training data and test data; A2 builds a sequence stabilization module, and uses the ADF (Augmented Dickey-Fuller) unit root test on the training data to distinguish between stable variables and non-stationary variables. For the detected non-stationary variables, sequence stabilization is performed to transform them into stable sequences. Finally, all processed data (including originally stable variables and variables that have been stabilized) are integrated and spliced ​​to form the final stable training data. A3 builds a transformer reconstruction module based on a self-regulatory strategy. It extracts the long-term dependencies and deep features of stationary training data through the destationary attention module, while preventing the loss of non-stationary information. It adopts a self-regulatory two-stage training strategy, introduces focus scores, pays attention to sequences with large deviations, and reconstructs the input stationary training data. A4 uses the sum of the two-stage reconstruction deviations as the loss function for model training, calculates the monitoring statistics using the feature matrix output by the transformer network encoder and the reconstruction error of the decoder, and determines the monitoring control limit based on the kernel density estimation method; B. Online monitoring stage: B1 uses the ADF test to distinguish the test data from the stationary variables and non-stationary variables. For the detected non-stationary variables, the sequence is stabilized to transform them into stationary sequences. Finally, all the processed data are integrated and spliced ​​to form the final stationary test data. B2 uses the transformer reconstruction module of the trained self-regulation strategy to calculate the monitoring statistics of the stationary test data; B3 compares the monitoring statistics with the control limits to determine whether a fault has occurred. When the statistics exceed the control limits obtained in the offline modeling stage, it indicates that a fault has occurred at the sample point. Otherwise, no fault has occurred.

2. A method for monitoring faults in a non-stationary industrial process based on a self-regulating non-stationary converter network according to claim 1, characterized in that: The specific steps of A1 are as follows: 1) Obtain historical data under normal conditions of industrial processes as training data Where n represents the number of samples, m represents the sample dimension, and also represents the number of data points contained in each sample. Calculate the maximum value max(x j ) and the minimum value min(x j ), standardize the data X to get where j = 0, 1, ..., n-1, x j represents the j-th column vector of the original data X, ε represents a small constant to prevent the denominator from being zero, which is taken as 0.001; 2) Standardize the data Divide into training sample sets and test sample set The sum of p and q is n.

3. A method for monitoring faults in a non-stationary industrial process based on a self-regulating non-stationary converter network according to claim 1, characterized in that: In step A2, the ADF unit root test is used to distinguish between stationary variables and non-stationary variables. The specific steps are as follows: 1) The ADF test can distinguish the training sample set A dataset of non-stationary variables in and stationary variable datasets where the sum of z and v is p; 2) First, use a sliding window of size 8 to transform the non-stationary variable data set It is divided into B windows in the time dimension, and the data set corresponding to the window is The data set corresponding to the bth window Where b = 1, 2, ... B, represents the i-th sample in the b-th window; 3) The non-stationary data of each window All are converted into stable data through translation and scaling operations 4) Integrate and concatenate the originally stable variables and the stabilized variables to obtain the final stable training data 4. A method for monitoring faults in a non-stationary industrial process based on a self-regulating non-stationary converter network according to claim 1, characterized in that: The specific steps of A3 are as follows: 1) Divide the stationary training data into sliding windows to obtain window training data, and embed position coding information into each window training data; 2) Input the fully stationary training data into the sequence encoder. The sequence encoder captures the long-range dependencies in the fully stationary training data through the self-attention mechanism. First, the fully stationary training data and the focal score F (initialized as a zero matrix with the same dimensions as the fully stationary training data The same) are concatenated, and then the input E1 of the sequence encoder is generated through position encoding. In the sequence encoder, the following operations are performed: Among them, Add represents the residual connection, LayerNorm represents the layer normalization operation, DSmultihead(·) represents the calculation operation of the destabilizing attention module, and FeedForward represents the calculation operation of the feedforward network. represents the feature matrix extracted by the destabilizing attention module in the sequence encoder, Represents the self-attention feature matrix output by the sequence encoder. The destabilizing attention module is mainly used to retrieve the non-stationary information lost in the sequence stabilization process and solve the problem of over-stabilization caused by sequence stabilization. The specific operation of the destabilizing attention module is as follows: in represents the standard deviation of the training samples, represents the mean of the training samples, log represents the logarithmic operation, Attn represents the self-attention calculation, the destabilization factors τ and Δ are shared by the destabilization attention modules of all layers, and MLP(·) represents the multi-layer perceptron layer; 3) Output of the sequence encoder As the input of the multi-head self-attention module in the window encoder, the window training data W is embedded in the position encoding information as the input E2 of the window encoder. The specific operation of the window encoder is as follows: Mask(Multihead(·)) indicates masking the multi-head attention calculation. represents the feature matrix extracted by the masked multi-head attention module in the window encoder, represents the output of the window encoder; 4) The output of the window encoder Input the decoder to get the final reconstructed output. The specific operations are as follows: Where sigmoid(·) represents the activation function sigmoid calculation, which maps the input value to the range of 0 to 1, converting the continuous value into a probability output.

5. A method for monitoring faults in a non-stationary industrial process based on a self-regulating non-stationary converter network according to claim 1, characterized in that: In step A3, a self-regulating two-stage training strategy is used to introduce subsequences with large focus score deviations to extract effective deep-level features. The specific steps are as follows: 1) In the first stage, the goal of the model is to generate a reconstruction of the input window, using the reconstruction deviation as the focus score F. The reconstruction deviation is calculated as follows: L fp =||D1-W||2 (6) Where L fp represents the reconstruction deviation of the first stage, W represents the window training data, D1 is the reconstructed output of the first decoder in the first stage, firstly, the encoder extracts the features of the input training data, and the specific operation is shown in formulas (2), (3), and (4), and then the extracted feature matrix is ​​input into the decoder to obtain the reconstructed output D1, and the specific operation is shown in formula (5); 2) In the second stage, the reconstruction deviation L of the first stage is fp As the focal score F, the focal score F is combined with the full stationary training data The splicing is used as the sequence encoder input, the features of the input data are extracted through the sequence encoder and the window encoder, and then the second decoder is used to obtain the reconstructed output of the second stage. The second stage deviation calculation formula is as follows: in, represents the reconstruction output of the second stage, L sp Represents the reconstruction deviation of the second stage.

6. A method for monitoring faults in a non-stationary industrial process based on a self-regulating non-stationary converter network according to claim 1, characterized in that: The specific steps of step A4 are as follows: 1) The sum of the two-stage reconstruction deviations is used as the loss function for model training. The specific calculation formula is as follows: Where λ is the training period, the value is 10, ζ represents the training parameter, the value is 0.8; 2) Calculate T using the feature matrix output from the second stage of the transformer network window encoder 2 Statistics, the SPE statistics are calculated using the reconstruction deviation of the second stage of the transformer network decoder, T 2 The calculation formulas for the statistic and the SPE statistic are as follows: Where x(k) is the current observation vector, is the mean of the eigenvectors, ∑ H is the covariance matrix of the eigenvectors; where res(k) represents the residual vector, is the mean of the residual vector, ∑ RES is the covariance matrix of the residual vector; 3) For the calculated T 2 The statistics and SPE statistics use kernel density estimation to determine the control limits. The kernel density estimation calculation formula is as follows: Among them, g represents the number of samples, h>0, is the kernel density bandwidth parameter, set to 0.5, and SPE s is the T of each sample point in the training set 2 Statistics and SPE statistics, K(·) represents the Gaussian kernel function, calculated as follows: in, Represents the distance between the observed point and the point to be estimated.

7. A method for monitoring faults in a non-stationary industrial process based on a self-regulating non-stationary converter network according to claim 1, characterized in that: B specifically includes the following steps: 1) Set the test sample set Using the ADF test to distinguish stationary variable data sets and non-stationary variable datasets 2) The non-stationary variable dataset is transformed into It is divided into B windows in the time dimension, and the data set corresponding to the window is The data set corresponding to the bth window Where b = 1, 2, ... B, represents the i-th sample in the b-th window; 3) Use pan and zoom operations to Perform stabilization processing, then integrate and splice the originally stable variables and the stabilized variables to obtain the final stable test data 4) Stabilize test data Divide the sliding window to obtain window training data W'={w'1,...,w' H }, and embed the position encoding information for each window training data; 5) Stabilize test data Input sequence encoder, use formula (2) and (3) to get the self-attention feature output of stable test data 6) The output of the sequence encoder As the input of the multi-head self-attention module in the window encoder, the window test data W' with embedded position encoding information is used as the input E'2 of the window encoder, and the output feature matrix is ​​obtained using formula (4) 7) Output of the window encoder Input the decoder and use formula (5) to get the final reconstructed output D test , use formula (6) to calculate the reconstruction error; 8) Calculate T using formulas (9) and (10) 2 Statistics and SPE statistics, the parameters in the formula remain unchanged. When the statistics exceed the control limit obtained in the offline modeling stage, it indicates that a fault has occurred at the sample point. Otherwise, no fault has occurred.

Images