Fault detection method based on multi-feature fusion attention mechanism assisted autoencoder

Abstract

The application provides a fault detection method based on a multi-feature fusion attention mechanism auxiliary self-encoder, and belongs to the field of industrial process fault detection.The application combines the stability advantage of traditional machine learning in feature extraction with the inclusiveness advantage of deep learning in feature extraction, and is suitable for complex industrial process fault detection.The application processes non-Gaussian features through ICA, processes dynamic features through SFA, and processes nonlinear features through a self-encoder with the steps of dimension increasing and then dimension decreasing, so that more comprehensive features of the process are mined, and the accuracy of fault detection is improved.In addition, the application optimizes the back propagation process of the deep learning network, and enhances the robustness of the overall model to noise data.The application increases noise data, constructs a weight matrix, changes the degree of back propagation, enhances the robustness of the model, and reduces the false alarm rate of fault detection.

Description

Technical Field

[0001] This invention belongs to the field of industrial process fault detection, and focuses on a fault detection method and system based on a multi-feature fusion attention mechanism-assisted autoencoder, which can be applied to industrial process monitoring. Background Technology

[0002] In the era of intelligent manufacturing, higher demands are being placed on the reliability of industrial processes to meet the requirements of production diversity, safety, and high quality. Process monitoring technology is crucial for ensuring process reliability. With the continuous development of sensor and data storage technologies, massive amounts of data can be collected and stored, and data-driven process monitoring methods are becoming increasingly mature. This makes it possible to extract information from collected industrial data and even detect faults in situations lacking specialized experience and mechanical knowledge.

[0003] Previous research has primarily categorized data-driven fault detection methods into two main types. First, Multivariate Statistical Process Monitoring (MSPM). Principal Component Analysis (PCA) is the most typical method for acquiring static features of stationary processes. It extracts key information from dense, high-dimensional process data using linear transformations. However, PCA assumes that the process data follows a multidimensional Gaussian distribution, significantly reducing the effectiveness of monitoring when applied to non-Gaussian processes. Furthermore, MSPM methods like PCA offer only a shallow representation of process features. These methods may perform poorly if the industrial process is complex. Second, many researchers have recently shifted their focus to deep learning methods. Deep learning models are primarily developed based on neural network models. Compared to traditional statistical learning models, deep learning models have a stronger ability to extract latent information from data and can effectively handle nonlinear features. Autoencoders (AEs) are a typical deep learning model. Due to their strong representation learning capabilities, simple structure, and ease of training, AEs have been widely used in fault detection. However, despite their powerful unsupervised feature extraction capabilities, the interpretability of the features obtained is relatively weak. When faced with inspection tasks with specific requirements, it is difficult to select feasible features to construct a fault detection strategy. Traditional models such as PCA and Slow Feature Analysis (SFA) can quickly and easily acquire specific features, such as static and dynamic features. Therefore, some researchers have combined AE (Automatic Feature Analysis) with traditional models, proposing some improved AE-based fault detection methods. Furthermore, real-world industrial process data exhibits complex characteristics, often displaying non-Gaussian, dynamic, and nonlinear features, which the aforementioned methods—improved AE-based fault detection methods—cannot simultaneously address the detection of complex features. It is worth noting that training neural network models requires a large amount of data, while effective data from real-world industrial processes is often limited, restricting the in-depth exploration of process features. Summary of the Invention

[0004] To overcome the limitations of existing technologies, this invention proposes a fault detection method and system based on a multi-view feature fusion attention mechanism-assisted autoencoder network (MFA-AE). It employs a strategy combining the MSPM method (which requires only a small amount of data) with AE to handle complex processes. To address the limitation of PCA (Programmable Component Analysis) in handling multidimensional non-Gaussian distributions, Independent Component Analysis (ICA) is used. ICA, by introducing higher-order statistics and nonlinear elements, can more effectively extract hidden statistical information. This results in higher feature extraction performance when processing non-Gaussian data. To better handle the dynamic characteristics of industrial processes, SFA obtains slowly changing features through nonlinear extension, principal component analysis, and feature ranking, effectively processing time-series data. Subsequently, an attention mechanism is used to fuse these features with the original data, providing more effective information for subsequent modeling. The integrated features are then processed by an autoencoder network to explore their nonlinear characteristics. Finally, a fault detection model is constructed based on the network's residuals.

[0005] To address the above objectives, this invention provides the following technical solution: a fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder, comprising the following steps:

[0006] Step S1: Obtain the dataset X containing noise in the industrial process. noise ;

[0007] Step S2, for data X noise Based on experience, a small segment of data without noise is selected to obtain noise-free data X;

[0008] Step S3, for data X and noisy data X noise Data were obtained by standardizing to zero mean and unit variance. and noise data

[0009] Step S4: Calculate the standardized data. and noise data Based on the cosine similarity, construct the weight matrix Φ. θ ;

[0010] Step S5, for the standardized noise data Independent component analysis (ICA) and slow eigenvalue analysis (SFA) were used to process the data to obtain the independent components c. q and slow features s k ;

[0011] Step S6, train the attention mechanism autoencoder by using cq and s k and standardized data The input is fed into the attention mechanism autoencoder to compute the attention output 'a', and the weight matrix Φ is used to apply the gradient to the network during backpropagation. θ The network parameters are weighted and updated to obtain the output X of the trained attention mechanism autoencoder. out ;

[0012] Step S7, calculate data and X out The residual matrix R;

[0013] Step S8: Calculate the test statistic u using the residual matrix R;

[0014] Step S9: Use kernel density estimation (KDE) to calculate the control limit E of the test statistic u. limit ;

[0015] Step S10: Obtain the dataset X of samples to be tested in the industrial process. new The data were then standardized to zero mean and unit variance to obtain the data.

[0016] Step S11: Input the data into the model trained in steps S5 and S6 to obtain the output X. new,out ;

[0017] Step S12, calculate data and X new,out The residual matrix R new And calculate the test statistic u. new ;

[0018] Step S13, compare the test statistic u new The statistic u for each tested sample new,k and control limit E limit If a size detection fault occurs, u new,k >E limit If the test statistic is positive, the sample represented by the test statistic is a faulty sample; otherwise, it is a normal sample. Here, k takes the value of 1-n, and n is the number of samples tested.

[0019] Furthermore, the specific formula for the standardization process in step S3 is as follows:

[0020]

[0021] Where X represents the original values ​​of the sample data in the dataset. Let X be the standardized values ​​of the sample data, mean(X) be the mean of the sample data, and variance(X) be the variance of the sample data.

[0022] Furthermore, the method for calculating cosine similarity in step S4 is as follows:

[0023]

[0024] Where A and B are vectors, ||A|| and ||B|| are the L2 norms of A and B, and A·B represents the dot product between the vectors. Therefore, the constructed weight matrix Φ θ for:

[0025]

[0026] in represent The i-th eigenvector in the middle, represent The j-th eigenvector in the dataset, where N is the total number of eigenvectors, and the weight matrix Φ is... θ The first row and first column represent The first eigenvector and The cosine similarity of the first feature vector in the model is calculated, and so on up to the Nth row and Nth column.

[0027] Furthermore, in step S5, the standardized data... Independent component analysis (ICA) was used for feature selection to obtain independent component c. q This includes the following sub-steps:

[0028] S5.1.1, regarding data Calculate the covariance matrix Then perform eigenvalue decomposition to obtain eigenvalues ​​λ1,λ2,…,λ m , where λ1≥λ2≥…≥λ m Let m be the number of features, the corresponding eigenvector matrix be U, and the diagonal matrix of eigenvalues ​​be Λ=diag(λ1,λ2,…,λ). m );

[0029] S5.1.2, obtain the whitening matrix P = Λ -1 / 2 U T The data obtained after whitening is Where A is the transformation matrix and S is the source signal; let B = PA, then Z = BS, and the reconstructed data...

[0030] S5.1.3, the orthogonal matrix B is obtained through iterative calculation using the following formula as the optimization function:

[0031] b i (r+1)=E{Zg(b i (r) TZ)}-E{g ′ (b i (r) T Z)}b i (r)

[0032] Where r represents the number of iterations, b i It is the i-th column of an orthogonal matrix, and the g function is... The derivative of , where logcosh represents the natural logarithm of the hyperbolic cosine function, a1 controls its sensitivity to outliers, and b is set before iterative calculation. i (0) Normalization is performed, where E represents the expectation and ' represents the first derivative;

[0033] S5.1.4, Set the iteration precision ε, if after a certain iteration:

[0034] ||b i (r+1)-b i (r)‖<ε

[0035] This indicates that the iteration has reached the specified precision, the iteration stops, and b is recorded. i The value of (r+1) is used to iterate the next component;

[0036] S5.1.5, to prevent convergence to the same extreme value, after each iteration, [the following is done:] Perform the de-correlation operation, that is:

[0037]

[0038]

[0039] S5.1.6, Calculate the unmixing matrix H = B T P, then the reconstructed data is That is, the independent component c is what we are looking for. q .

[0040] Furthermore, in step S5, the standardized data... Slow Feature Analysis (SFA) is used for feature selection to obtain slow features s k This includes the following sub-steps:

[0041] S5.2.1, whitening data The covariance matrix of the first derivative Perform singular value decomposition: G is the eigenvector matrix, and Ω is the eigenvalue matrix = diag(ζ1,ζ2,…,ζ). m ), where ζ m These are the eigenvalues, arranged in ascending order, representing the rate of change of the slow eigenvalues.

[0042] S5.2.2, Transformation matrix W = GY -1 / 2 U T Where Y is the covariance matrix Singular value decomposition yields the following characteristic: The slow characteristic is... n is the number of sampling times, i.e., the number of data samples;

[0043] S5.2.3, according to the formula Find the number of major slow features k, where card represents the cardinality of the set. The rate at which data changes represents the speed at which it changes, and t represents the set. The 0.1 upper quantile is used to obtain the k-dimensional slow feature.

[0044]

[0045] Furthermore, the specific implementation method of step S6 is as follows:

[0046] S6.1, the independent component c obtained after processing with ICA and SFA. q and slow features s k and standardized data In the input attention mechanism, a linear transformation is used to obtain the query vector Q. i =L q c q Key vector K j =J k s k Sum value vector Where L q J k and W v It is a linear transformation matrix;

[0047] S6.2, through Q i and K j The attention score is calculated by dividing the dot product by the scaling factor. Where d k It is the scaling factor;

[0048] S6.3, the attention weight r is obtained from the attention score using the Softmax function. ij =Softmax(score(Q) i ,K j ));

[0049] S6.4, Value Vector V i The output of the attention mechanism is obtained by multiplying and summing the values ​​by the attention weights. Where n is the total number of key vectors, and a is a i The set of values ​​i, where i ranges from 1 to n, is input into an autoencoder assisted by a self-attention mechanism for training.

[0050] S6.5 uses the output 'a' of the attention mechanism to input the encoder to map the data to the feature space;

[0051] S6.6, the decoder reconstructs the input from the feature representation in the feature space to obtain the output X of the autoencoder. o ′ ut ;

[0052] In S6.7, during each training round, the autoencoder uses data... With output X o ′ ut Calculate the loss function and backpropagate to update the mapping function parameters until training is complete; during backpropagation, the parameter gradients of the autoencoder model need to be multiplied by the weight matrix Φ. θ By weighting, the importance of different features can be adjusted, and the proportion of noise can be reduced;

[0053] S6.8, the output 'a' of the attention mechanism is input into the trained autoencoder network to obtain the output X. out .

[0054] Furthermore, in step S7, the formula for calculating the residual matrix is:

[0055] Furthermore, in step S8, for each sample dimension of the residual matrix R, the vector R... k Multiply them to obtain the test statistic. Where n is the total number of vectors in the sample dimension.

[0056] Furthermore, in step S9, given the confidence level α, the control limits are calculated using the KDE for the test statistic corresponding to each obtained sample. Where F α (u) is the cumulative distribution function of the probability density function obtained by KDE.

[0057] Furthermore, it also includes step S10, using the fault detection rate (FDR) and the false alarm rate (FAR) to measure detection performance. FDR refers to the ratio of faulty samples detected during monitoring, i.e.:

[0058]

[0059] FAR refers to the ratio of normal samples to abnormal samples, i.e.:

[0060]

[0061] When FDR > 50% and FAR < 5%, the fault is successfully detected.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] (1) This invention combines the stability advantage of traditional machine learning feature extraction with the inclusiveness advantage of deep learning feature extraction, making it suitable for fault detection in complex industrial processes. This invention processes non-Gaussian features through ICA, dynamic features through SFA, and nonlinear features through an autoencoder that first increases and then decreases dimensionality, thereby mining more comprehensive features of the process and improving the accuracy of fault detection.

[0064] (2) This invention enhances the robustness of the overall model to noisy data by optimizing the backpropagation process of the deep learning network. This invention constructs a weight matrix based on the similarity between noisy data and noise-free data, thereby changing the degree of backpropagation, enhancing the robustness of the model, and reducing the false alarm rate of fault detection.

[0065] (3) In order to better optimize the feature extraction effect, this invention proposes a feature fusion strategy based on self-attention mechanism to further fuse the independent components obtained by ICA and the slow features carrying important information obtained by SFA. Experiments on real public datasets show that this invention has good detection effect. Attached Figure Description

[0066] Figure 1 This is a flowchart of an embodiment of the present invention.

[0067] Figure 2 The graph shows the change in training loss of the experimental network as a function of training rounds in an embodiment of the present invention.

[0068] Figure 3 A comparison chart of the detection results of PCA, GDAE, and MFA-AE on fault 10.

[0069] Figure 4 This is a comparison chart of the ablation test results of MFA-AE on fault 1.

[0070] Figure 5 This is a comparison chart of the ablation test results of MFA-AE on fault 7. Detailed Implementation

[0071] To more clearly illustrate the objectives, technical implementation methods, and advantages of this invention, a detailed explanation will be provided below in conjunction with the accompanying drawings and embodiments. It should be noted that the specific embodiments listed below are for illustrative purposes only and are not intended to limit the scope of this invention. Furthermore, the technical points inherent in the various embodiments of this invention mentioned below can be freely combined and applied, provided they are not logically contradictory.

[0072] To address the challenges of extracting non-Gaussian, non-linear, and dynamic features from complex data in industrial processes, and to improve fault detection rates while reducing false alarm rates, this invention provides a fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder. In the offline modeling phase, a dataset of normal samples from the industrial process is first acquired and standardized. Then, ICA is used for feature extraction of non-Gaussian features, and SFA is used for feature extraction of dynamic features. A self-attention mechanism is then used to fuse the extracted features, and an autoencoder that first increases dimensionality and then reduces it is used to process non-linear features. Next, a test statistic and control limits are calculated from the residuals of the trained MFA-AE network. In the online monitoring phase, the dataset of samples to be detected is acquired and standardized. This dataset is then input into the trained MFA-AE network to obtain residuals and calculate test statistics. Finally, the test statistic for each data sample is compared with the control limits to determine whether a fault has occurred. Figure 1 As shown, the present invention includes the following specific steps:

[0073] Step S1: Obtain the dataset X containing noise in the industrial process. noise ;

[0074] Step S2, for data X noise Based on experience, a small segment of data without noise is selected to obtain noise-free data X;

[0075] Step S3, for data X and noisy data X noise Data were obtained by standardizing to zero mean and unit variance. and noise data

[0076] In a specific embodiment, the specific formula for the standardization process in step S3 is as follows:

[0077]

[0078] Where X represents the original values ​​of the sample data in the dataset. Here, X represents the standardized values ​​of the sample data, where mean(X) is the mean of the sample data and variance(X) is the variance of the sample data. The standardization process for noisy data is the same.

[0079] Step S4: Calculate the standardized data. and noise data Based on the cosine similarity, construct the weight matrix Φ. θ ;

[0080] In a specific embodiment, the method for calculating cosine similarity in step S4 is as follows:

[0081]

[0082] Where A and B are vectors, ||A|| and ||B|| are the L2 norms of A and B, and A·B represents the dot product between the vectors. Therefore, the constructed weight matrix Φ θ for:

[0083]

[0084] in represent The i-th eigenvector in the middle, represent The j-th eigenvector in the dataset, where N is the total number of eigenvectors, and the weight matrix Φ is... θ The first row and first column represent The first eigenvector and The cosine similarity of the first feature vector in the model is calculated, and so on up to the Nth row and Nth column.

[0085] Step S5, for the standardized noise data Independent component analysis (ICA) and slow feature analysis (SFA) were used respectively to obtain the independent components c. q and slow features s k ;

[0086] In a specific embodiment, step S5 involves processing the standardized noise data. Feature selection was performed using ICA to obtain the independent component c. q This includes the following sub-steps:

[0087] S5.1.1, for noisy data Calculate the covariance matrix Then perform eigenvalue decomposition to obtain eigenvalues ​​λ1,λ2,…,λ m , where λ1≥λ2≥…≥λ m Let m be the number of features, the corresponding eigenvector matrix be U, and the diagonal matrix of eigenvalues ​​be Λ=diag(λ1,λ2,…,λ). m );

[0088] S5.1.2, obtain the whitening matrix P = Λ -1 / 2 U T The data obtained after whitening is Where A is the transformation matrix and S is the source signal. Let B = PA, then Z = BS, and the reconstructed data...

[0089] S5.1.3, the orthogonal matrix B is obtained through iterative calculation using the following formula as the optimization function:

[0090] b i (r+1)=E{Zg(b i (r) T Z)}-E{g ′ (b i (r) T Z)}b i (r)

[0091] Where r represents the number of iterations, b i It is the i-th column of an orthogonal matrix, and the g function is... The derivative of , where logcosh represents the natural logarithm of the hyperbolic cosine function, a1 controls its sensitivity to outliers, and b needs to be calculated before iterative computation. i (0) Normalization is performed, where E represents the expectation and ' represents the first derivative;

[0092] S5.1.4, Set the iteration precision ε, if after a certain iteration:

[0093] ||b i (r+1)-b i (r)‖<ε

[0094] This indicates that the iteration has reached the specified precision, the iteration stops, and b is recorded. i The value of (r+1) is used to iterate the next component;

[0095] S5.1.5, to prevent convergence to the same extreme value, after each iteration, [the following is done:] Perform the de-correlation operation, that is:

[0096]

[0097]

[0098] Where l represents the orthogonal matrix b i The column numbers, i and j represent the i-th and j-th columns, respectively;

[0099] S5.1.6, Calculate the unmixing matrix H = B T P, then the reconstructed data is That is, the independent component c is what we are looking for. q .

[0100] Furthermore, in step S5, the standardized data... SFA is used for feature selection to obtain slow features s k This includes the following sub-steps:

[0101] S5.2.1, whitening data The covariance matrix of the first derivative Perform singular value decomposition: G is the eigenvector matrix, and Ω is the eigenvalue matrix = diag(ζ1,ζ2,…,ζ). m ), where ζ m These are the eigenvalues, arranged in ascending order, representing the rate of change of the slow eigenvalues.

[0102] S5.2.2, Transformation matrix W = GY -1 / 2 U T Where Y is the covariance matrix Singular value decomposition yields the following characteristic: The slow characteristic is... n is the number of sampling times, i.e., the number of data samples;

[0103] S5.2.3, according to the formula Find the number of major slow features k, where card represents the cardinality of the set. The rate at which data changes represents the speed at which it changes, and t represents the set. The 0.1 upper quantile is used to obtain the k-dimensional slow feature. The 0.1 upper quantile refers to the numerical points that divide the probability distribution range of a random variable into several equal parts, where the numerical points satisfy the condition P{X>x}≤0.1.

[0104] Step S6, train the attention mechanism autoencoder by using c q and s k and standardized data The input is fed into the attention mechanism autoencoder to compute the attention output 'a'. During backpropagation, the network gradient is weighted using the weight matrix Φ. θ The network parameters are weighted and updated to obtain the output X of the trained attention mechanism autoencoder. out ;

[0105] In a specific embodiment, the sub-steps of step S6 are as follows:

[0106] S6.1, the independent component c obtained after processing with ICA and SFA. q and slow features s k and standardized data In the input attention mechanism, a linear transformation is used to obtain the query vector Q. i =L q c q Key vector K j =J k s k Sum value vector Where L q J k and Wv It is a linear transformation matrix;

[0107] S6.2, through Q i and K j The attention score is calculated by dividing the dot product by the scaling factor. Where d k It is the scaling factor;

[0108] S6.3, the attention weight r is obtained from the attention score using the Softmax function. ij =Softmax(score(Q) i ,K j ));

[0109] S6.4, Value Vector V i The output of the attention mechanism is obtained by multiplying and summing the values ​​by the attention weights. Where n is the total number of key vectors, and a is a i The set of values ​​i, where i ranges from 1 to n, is input into an autoencoder assisted by a self-attention mechanism for training.

[0110] S6.5 uses the output 'a' of the attention mechanism to input the encoder to map the data to the feature space;

[0111] S6.6, the decoder reconstructs the input from the feature representation in the feature space to obtain the output X of the autoencoder. o ′ ut ;

[0112] In S6.7, during each training round, the autoencoder uses data... With output X o ′ ut The loss function is calculated, and the mapping function parameters are updated via backpropagation until training is complete. During backpropagation, the gradients of the autoencoder model's parameters are multiplied by the weight matrix Φ. θ Weighting is used to adjust the importance of different features and reduce the proportion of noise. The formula is as follows:

[0113] Param' grad =Param grad ×Φ θ

[0114] Among them, Param grad The gradient representing the model parameters, Param' grad The gradient represents the updated model parameters;

[0115] S6.8, the output 'a' of the attention mechanism is input into the trained autoencoder network to obtain the output X. out .

[0116] Step S7, calculate data and X out The residual matrix R;

[0117] In a specific embodiment, in step S7, the standardized data is calculated. The final output X obtained from the autoencoder network out The residuals are used to obtain the residual matrix.

[0118] Step S8: Calculate the test statistic u using the residual matrix R;

[0119] In a specific embodiment, in step S8, for each sample dimension of the residual matrix R, the vector R... k Multiply them to obtain the test statistic. Where n is the total number of vectors in the sample dimension.

[0120] Step S9: Calculate the control limit E of the test statistic u using kernel density estimation (KDE). limit ;

[0121] In the example shown in the figure, in step S9, given a confidence level α, the control limit is calculated using KDE for the test statistic corresponding to each obtained sample. Where F α (u) is the cumulative distribution function of the probability density function obtained by KDE. The probability density function is obtained from kernel density estimation.

[0122] Step S10: Obtain the dataset X of samples to be tested in the industrial process. new The data were then standardized to zero mean and unit variance to obtain the data.

[0123] In a specific embodiment, the standardization process in step S10 is the same as in step S3, and the specific formula is as follows:

[0124]

[0125] Among them, X new The sample data in the dataset to be detected The values ​​of the sample data to be tested are the mean and variance of the normal sample data after standardization. mean(X) is the mean of the normal sample data, and variance(X) is the variance of the normal sample data.

[0126] Step S11: Input the data into the MFA-AE model trained in steps S5 and S6 to obtain the output X. new,out ;

[0127] In a specific embodiment, the variable selection in the feature selection part of the MFA-AE model in step S11 is consistent with the selection during training. Its attention mechanism autoencoder is the network model trained in step S6, and uses the input... Obtain the output X of the MFA-AE model new,out .

[0128] Step S12, calculate data and X new,out The residual matrix R new And calculate the test statistic u. new ;

[0129] In a specific embodiment, in step S11, the residual matrix R new and test statistic u new The calculation is the same as steps S6 and S7, that is... and

[0130] Step S13, compare the test statistic u new The statistic u for each tested sample new,k and control limit E limit If a size detection fault occurs, u new,k >E limit If the test statistic is positive, the sample represented by the test statistic is a faulty sample; otherwise, it is a normal sample. Here, k takes the value of 1-n, and n is the number of samples tested.

[0131] The following explanation further illustrates the beneficial effects of this invention in specific application scenarios.

[0132] This invention is applied to the publicly available Tennessee-Eastman (TE) process dataset. The TE process simulation is a chemical system consisting of a reactor, condenser, compressor, separator, and stripper, and is widely used as a benchmark for process control and fault diagnosis. The TE simulation requires the measurement of 52 variables, including 22 continuous measurements, 19 sampled measurements, and 11 manipulated variables. The TE dataset contains fault samples covering 21 different fault conditions. For samples under normal conditions, the dataset is divided into two groups with sample sizes of 500 and 960. Similarly, for each fault condition, the dataset is also divided into two groups with sample sizes of 480 and 960.

[0133] This invention uses Fault Detection Rate (FDR) and Fault False Alarm Rate (FAR) to measure detection performance. FDR refers to the ratio of faulty samples detected during monitoring, i.e.:

[0134] FAR refers to the ratio of normal samples to abnormal samples, i.e.:

[0135] When FDR > 50% and FAR < 5%, the fault is successfully detected. It's worth noting that if FDR is less than 50%, the detection performance is even worse than random guessing. When FAR < 5%, the higher the FDR, the better the performance.

[0136] The platform information used in this invention is as follows: CPU: Intel(R) Xeon(R) Silver4210R CPU @ 2.40GHz; Graphics Processing Unit: NVIDIA GeForce RTX 3090; Programming Language: Python 3.9. The MFA-AE network model of this invention uses the PyTorch framework. The number of neurons in the AE structure of the MFA-AE is set to 52-256-128-128-256-52. The Adam optimizer is used during network model training. The regularization parameter is set to 0.01. The number of training epochs is 500. The training loss varies with the number of epochs as follows: Figure 2 As shown, it can be seen that the model tends to fit after about 250 rounds.

[0137] This invention presents a multi-scale feature fusion attention mechanism-assisted autoencoder network fault detection method that integrates traditional machine learning and deep learning methods. Therefore, for each fault type, the method is compared with PCA-based fault detection methods and Graph Dynamic Autoencoder (GDAE)-based fault detection methods, as shown in Table 1. Simultaneously, the ablation experiment was conducted, comparing the detection results of fault detection using only AE, directly concatenating features obtained after SFA and ICA processing, and the proposed method for each fault type, as shown in Table 2. The ablation experiment, by progressively eliminating components of the proposed method, evaluates their contribution and impact on the final result. This helps determine the relative importance of each component in the proposed method and verifies its effectiveness.

[0138] For PCA and GDAE, the statistical indicator used in the experiment was the Hotelling statistic T. 2(Hotelling's T-squared). As shown in Table 1, the FDR of all three methods is above 95% for faults 1, 2, 6, 7, 8, 12, 13, and 14. These are relatively serious faults. The MFA-AE proposed in this invention can effectively detect these faults. The FDR of all three methods is above 50% for faults 4, 10, 11, 17, 18, and 20. These faults can be detected normally. The lowest FDR of MFA-AE is better than the lowest FDR of the other two methods. This proves that the lower limit of MFA-AE is higher, and its detection capability is slightly better. The three methods have difficulty detecting faults 3, 5, 9, 15, 16, 19, and 21. Although PCA has a higher FDR than MFA-AE for some of the 21 faults, its FAR is mostly too high. In actual industrial production, a similar detection rate but a high false negative rate is more harmful. For faults with an FAR of less than 5%, the number of faults with an FDR greater than 50% is consistent for both GDAE and MFA-AE. This demonstrates that MFA-AE is as robust as GDAE in fault detection capability. Taking the FDR of fault 10 as an example, under the requirement of an FAR of less than 5%, the method of this invention outperforms PCA by nearly 15 percentage points and GDAE by approximately 5 percentage points. Figure 3 As shown, the data curve after a fault occurs resembles the letter M. At the concave area, the method of this invention provides more data above the control limit than other methods, and the data below the control limit is closer to the control limit. Considering all factors, the fault detection capability of the MFA-AE proposed in this invention is slightly superior.

[0139] Table 1 shows the FDR and FAR (FAR in parentheses) for 21 fault types in the TE dataset using PCA, GDAE, and MFA-AE.

[0140]

[0141]

[0142] As shown in Table 2, MFA-AE demonstrated superior performance under different fault scenarios in the ablation experiments. For faults 1, 7, 14, and 18, MFA-AE successfully reduced FAR while maintaining excellent FDR. More notably, after adding the attention mechanism, MFA-AE effectively offset the improvement in FAR achieved by SFA-ICA-AE. For faults 2 and 6, all three methods performed well in both FDR and FAR, indicating a relative balance in their method dependence according to the present invention. When dealing with fault 9, MFA-AE not only successfully reduced FAR but also achieved this without sacrificing FDR. However, in faults 4, 5, 19, and 21, MFA-AE slightly sacrificed FDR to reduce FAR; while in faults 11 and 12, it improved FDR by slightly sacrificing FAR. Furthermore, in fault 16, although MFA-AE successfully compensated for the shortcomings of SFA-ICA-AE while maintaining FDR, it slightly sacrificed FAR. In contrast, for faults 3, 10, 13, and 17, while MFA-AE successfully reduced the impact of SFA-ICA-AE, it slightly sacrificed FDR and FAR. Although MFA performed worse than SFA-ICA-AE in fault 15, it was significantly better than the pure AE method, both in terms of FDR and FAR. As for fault 8, MFA-AE successfully reduced the impact of SFA-ICA-AE on FAR and improved the FDR of pure AE; overall, MFA-AE is superior in both cases. Furthermore, for fault 20, MFA-AE maintained FAR at the expense of FDR, a trade-off that appears more reasonable compared to the other two methods. Figure 4 and Figure 5 As shown, taking faults 1 and 7 as examples, it can be seen that MFA-AE can better detect normal samples before the fault occurs, while maintaining its superior fault detection capability, thus achieving better control limit performance and improving its ability to avoid misdiagnosing normal samples as faulty samples. In summary, MFA-AE significantly improves fault detection capability by integrating the attention mechanism with features extracted by ICA and SFA.

[0143] Table 2. FDR and FAR of MFA-AE ablation experiments on the TE dataset.

[0144]

[0145] This invention also enhances the robustness of the overall model to noisy data by optimizing the backpropagation process of MFA-AE. By adding noisy data and constructing a weight matrix, this invention alters the degree of backpropagation, thereby enhancing model robustness and reducing the false alarm rate of fault detection. As shown in Table 3, the optimized MFA-AE effectively detects faults 1, 2, 6, 7, 8, 12, and 14, which are considered severe faults, and significantly reduces the false alarm rate for almost all faults. There are 13 faults with a detection rate above 50% and a false alarm rate below 5%, one more than GDAE and MFA-AE. For fault 7, it not only maintains the fault detection rate but also reduces the false alarm rate. For faults 1, 2, 3, 8, 9, 12, 13, 14, 15, and 16, the reduction in false alarm rate is greater than the reduction in detection rate. Based on the above analysis, it can be demonstrated that the optimized backpropagation MFA-AE has better noise robustness.

[0146] Table 3. FDR and FAR of MFA-AE and the optimized MFA-AE on the TE dataset.

[0147]

[0148]

[0149] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the invention.

Claims

1. A fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder, characterized in that, Includes the following steps: Step S1: Obtain the dataset X containing noise in the industrial process. noise ; Step S2, for data X noise Based on experience, a small segment of data without noise is selected to obtain noise-free data X; Step S3, for data X and noisy data X noise Data were obtained by standardizing to zero mean and unit variance. and noise data Step S4: Calculate the standardized data. and noise data Based on the cosine similarity, construct the weight matrix Φ. θ ; Step S5, for the standardized noise data Independent component analysis (ICA) and slow eigenvalue analysis (SFA) were used to process the data to obtain the independent components c. q and slow features s k ; Step S6, train the attention mechanism autoencoder by using c q and s k and standardized data The input is fed into the attention mechanism autoencoder to compute the attention output 'a', and the weight matrix Φ is used to apply the gradient to the network during backpropagation. θ The network parameters are weighted and updated to obtain the output X of the trained attention mechanism autoencoder. out ; Step S7, calculate data and X out The residual matrix R; Step S8: Calculate the test statistic u using the residual matrix R; Step S9: Use kernel density estimation (KDE) to calculate the control limit E of the test statistic u. limit ; Step S10: Obtain the dataset X of samples to be tested in the industrial process. new The data were then standardized to zero mean and unit variance to obtain the data. Step S11: Input the data into the model trained in steps S5 and S6 to obtain the output X. new,out ; Step S12, calculate data and X new,out The residual matrix R new And calculate the test statistic u. new ; Step S13, compare the test statistic u new The statistic u for each tested sample new,k and control limit E limit If a size detection fault occurs, u new,k >E limit If the test statistic is positive, the sample represented by the test statistic is a faulty sample; otherwise, it is a normal sample. Here, k takes the value of 1-n, and n is the number of samples tested.

2. The fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder as described in claim 1, characterized in that: The specific formula for the standardization process in step S3 is as follows: Where X represents the original values ​​of the sample data in the dataset. Let X be the standardized values ​​of the sample data, mean(X) be the mean of the sample data, and variance(X) be the variance of the sample data.

3. The fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder as described in claim 1, characterized in that: The method for calculating cosine similarity in step S4 is as follows: Where A and B are vectors, ||A|| and ||B|| are the L2 norms of A and B, and A·B represents the dot product between the vectors. Therefore, the constructed weight matrix Φ θ for: in represent The i-th eigenvector in the middle, represent The j-th eigenvector in the dataset, where N is the total number of eigenvectors, and the weight matrix Φ is... θ The first row and first column represent The first eigenvector and The cosine similarity of the first feature vector in the model is calculated, and so on up to the Nth row and Nth column.

4. The fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder as described in claim 1, characterized in that: In step S5, the standardized data Independent component analysis (ICA) was used for feature selection to obtain independent component c. q This includes the following sub-steps: S5.1.1, regarding data Calculate the covariance matrix Then perform eigenvalue decomposition to obtain eigenvalues ​​λ1,λ2,…,λ m , where λ1≥λ2≥…≥λ m Let m be the number of features, the corresponding eigenvector matrix be U, and the diagonal matrix of eigenvalues ​​be Λ=diag(λ1,λ2,…,λ). m ); S5.1.2, obtain the whitening matrix P = Λ -1 / 2 U T The data obtained after whitening is Where A is the transformation matrix and S is the source signal; let B = PA, then Z = BS, and the reconstructed data... S5.1.3, the orthogonal matrix B is obtained through iterative calculation using the following formula as the optimization function: b i (r+1)=E{Zg(b i (r) T Z)}-E{g ′ (b i (r) T Z)}b i (r) Where r represents the number of iterations, b i It is the i-th column of an orthogonal matrix, and the g function is... The derivative of , where logcosh represents the natural logarithm of the hyperbolic cosine function, a1 controls its sensitivity to outliers, and b is set before iterative calculation. i (0) Normalization is performed, where E represents the expectation and ' represents the first derivative; S5.1.4, Set the iteration precision ε, if after a certain iteration: ‖b i (r+1)-b i (r)‖<ε This indicates that the iteration has reached the specified precision, the iteration stops, and b is recorded. i The value of (r+1) is used to iterate the next component; S5.1.5, to prevent convergence to the same extreme value, after each iteration, [the following is done:] Perform the de-correlation operation, that is: S5.1.6, Calculate the unmixing matrix H = B T P, then the reconstructed data is That is, the independent component c is what we are looking for. q .

5. The fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder as described in claim 1, characterized in that: In step S5, the standardized data Slow Feature Analysis (SFA) is used for feature selection to obtain slow features s k This includes the following sub-steps: S5.2.1, whitening data The covariance matrix of the first derivative Perform singular value decomposition: G is the eigenvector matrix, and Ω is the eigenvalue matrix = diag(ζ1,ζ2,…,ζ). m ), where ζ m These are the eigenvalues, arranged in ascending order, representing the rate of change of the slow eigenvalues. S5.2.2, Transformation matrix W = GY -1 / 2 U T Where Y is the covariance matrix Singular value decomposition yields the following characteristic: The slow characteristic is... n is the number of sampling times, i.e., the number of data samples; S5.2.3, according to the formula Find the number of major slow features k, where card represents the cardinality of the set. The rate at which data changes represents the speed at which it changes, and t represents the set. The 0.1 upper quantile is used to obtain the k-dimensional slow feature.

6. The fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder as described in claim 1, characterized in that: The specific implementation method of step S6 is as follows: S6.1, the independent component c obtained after processing with ICA and SFA. q and slow features s k and standardized data In the input attention mechanism, a linear transformation is used to obtain the query vector Q. i =L q c q Key vector K j =J k s k Sum value vector Where L q J k and W v It is a linear transformation matrix; S6.2, through Q i and K j The attention score is calculated by dividing the dot product by the scaling factor. Where d k It is the scaling factor; S6.3, the attention weight r is obtained from the attention score using the Softmax function. ij =Softmax(score(Q) i ,K j )); S6.4, Value Vector V i The output of the attention mechanism is obtained by multiplying and summing the values ​​by the attention weights. Where n is the total number of key vectors, and a is a i The set of values ​​for i, i ranging from 1 to n, is input into an autoencoder assisted by a self-attention mechanism for training; S6.5 uses the output 'a' of the attention mechanism to input the encoder to map the data to the feature space; S6.6, the decoder reconstructs the input from the feature representation in the feature space to obtain the output X of the autoencoder. o ′ ut ; In S6.7, during each training round, the autoencoder uses data... With output X o ′ ut Calculate the loss function and backpropagate to update the mapping function parameters until training is complete; during backpropagation, the parameter gradients of the autoencoder model need to be multiplied by the weight matrix Φ. θ By weighting, the importance of different features can be adjusted, and the proportion of noise can be reduced; S6.8, the output 'a' of the attention mechanism is input into the trained autoencoder network to obtain the output X. out .

7. The fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder as described in claim 1, characterized in that: In step S7, the formula for calculating the residual matrix is:

8. The fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder as described in claim 1, characterized in that: In step S8, for each sample dimension of the residual matrix R, the vector R... k Multiply them to obtain the test statistic. Where n is the total number of vectors in the sample dimension.

9. The fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder as described in claim 1, characterized in that: In step S9, given the confidence level α, the control limits are calculated using the KDE for the test statistic corresponding to each sample. Where F α (u) is the cumulative distribution function of the probability density function obtained by KDE.

10. The fault detection method based on a multi-feature fusion attention mechanism-assisted autoencoder as described in claim 1, characterized in that: The method also includes step S10, which uses the fault detection rate (FDR) and the false alarm rate (FAR) to measure detection performance. FDR refers to the ratio of faulty samples detected during monitoring. FAR refers to the ratio of normal samples to abnormal samples, i.e.: When FDR > 50% and FAR < 5%, the fault is successfully detected.

Images