An industrial process fault detection method based on convolutional neural network and weight model

Abstract

The application discloses an industrial process fault detection method based on a convolutional neural network and a weight model, relates to an industrial process fault detection method, and discloses the following technical scheme: data of an industrial production process are collected and standardized; the standardized data are input into a time step attention convolution mechanism and a channel step attention convolution mechanism; the two parts of data capturing local information are respectively integrated into the channel step attention mechanism and the time step attention mechanism; Gumbel-SoftMax functions are used to calculate weights carrying time step characteristic information and weights carrying channel step characteristic information; and finally, the two weight values are combined with the time step characteristic information and the channel step characteristic information, and a detection result is obtained through logistic regression. The application effectively solves the problems of insufficient local information extraction capability of a Transformer model and insufficient attention to channel dimensions. Experiments show that the method can effectively detect industrial process faults.

Description

TECHNICAL FIELD

[0001] The application relates to an industrial process fault detection method, in particular to an industrial process fault detection method based on a convolutional neural network and a weight model; the method is based on a convolutional neural network, a channel step attention convolution mechanism and a weight function Transformer model. BACKGROUND

[0002] With the continuous improvement of modern industrial automation level, the production scale is continuously expanded, and the complexity of the system is also continuously increased. The fault detection and diagnosis (FDD) in modern industry becomes more and more complex and important. Therefore, the method based on data-driven modeling is concerned, which uses a decentralized control system to conveniently collect and store a large amount of industrial production process data, thereby providing a solid foundation for the process detection technology based on data-driven. This method directly establishes a model using historical data to realize system state detection and diagnosis. Specifically, the method based on data-driven modeling can be further divided into a statistical method, a shallow learning method and a deep learning method. Since the industrial process data usually has a strong nonlinear relationship, the shallow structure may not be able to effectively process, and therefore the deep learning method is more popular. A small fault in an industrial process may affect the entire system, eventually leading to equipment damage or product quality decline. Therefore, it is necessary to timely find the system deviation caused by the fault, and then take corresponding measures to avoid the occurrence of accidents. Therefore, establishing an effective process FDD model to distinguish the fault types in the industrial process is the key to reducing the operation personnel's mistakes and ensuring the safety and reliability of the system.

[0003] In recent years, with the increase of computer computing power, deep learning has become an important cornerstone in artificial intelligence, and various neural network models have been proposed to handle tasks in different fields. The introduction of the Transformer model has a milestone significance in the development of artificial intelligence, and is crucial in natural language processing, computer vision, speech processing and some other disciplines. The self-attention mechanism of the Transformer model can focus on the entire sequence at the same time, and can be easily parallelized. And it is easy to visualize the contribution of each input position to the output, thus realizing the explanation and analysis of the model. The Transformer model can solve the long-term dependence problem when processing data, and can depict more long-distance correlations. However, it also has a disadvantage that the computational complexity increases exponentially when calculating the global attention. At the same time, the Transformer model does not capture local information in detail, and there is often a lot of feature information between any two samples. The self-attention mechanism in the Transformer model only models the self-correlation of a single sequence, ignoring the dependence between channels. The multivariate time series fault detection problem not only needs to consider the relationship in the time dimension, but also needs to pay attention to the relationship between the channel dimensions to improve the accuracy of model detection. SUMMARY

[0004] The purpose of the present application is to address the complex data characteristics of large-scale, high-dimensional, nonlinear data in industrial processes and fault detection problems. A method for industrial process fault detection based on convolutional neural network and weight model is proposed. The method is based on an improved Transformer model for industrial process fault detection, which can extract features from industrial data in the case of large-scale, high-dimensional and nonlinear data, and improve the accuracy of fault detection.

[0005] The technical scheme adopted by the present application is:

[0006] A method for industrial process fault detection based on convolutional neural network and weight model, based on a Transformer model calculated by convolutional neural network and weight, the model is applied to industrial process fault detection, including the following steps:

[0007] Step one: collect N sample data X in industrial process X∈R N×k , X = [X1, X2, …, X i , …, X N ] T , where k represents the number of channels of the data, N represents the total number of samples collected, X i represents the i-th sample in the data X, i∈[1,N], and X is standardized, as shown in formulas (1)-(3):

[0008]

[0009]

[0010]

[0011] wherein formula (1) and (2) get the mean and variance of the training data set respectively, and the mean X train and variance S train of the training data set are used according to formula (3) to standardize X.

[0012] Step two: segment the input data X to reduce the computational complexity. Assuming that the length of each segment of the division is L seg , each sequence is divided into L segments, and the input is denoted as The division process is shown in formula (4):

[0013]

[0014] Step three: local information feature extraction is performed in the data time dimension and channel dimension respectively. In the time dimension: position encoding is added to the input information, and position encoding (PE) is added to mark the position information by using the sine and cosine functions. The position encoding is shown in formula (5) and (6):

[0015]

[0016]

[0017] wherein pos is the position index of the time series data in the sequence, and i is a certain dimension of the vector. The input information is transposed in the channel dimension. The number of channels in the time dimension is mapped to a high-dimensional space through a fully connected neural network (FC) to obtain wherein d model is the dimension obtained by mapping the number of channels. The time steps in the channel dimension are mapped to a high-dimensional space through a fully connected neural network to obtain wherein d model is the dimension obtained by mapping the time steps. Then, Xtand Xcare input to a multi-layer convolution structure, and each layer of convolution is configured with a convolution kernel of different size to extract local feature information. At the same time, the feature information obtained by convolution may have redundancy, so an average pooling layer is used to eliminate the redundant information of the features and improve the robustness of the training model. In addition, a Bn layer is added to prevent gradient explosion and gradient disappearance, and a residual connection module is added to focus on the difference. Finally, the local information in the time dimension Xconvt and channel dimension local information X convc The above formula is:

[0018] X convt = X' + max(0, Bn(pooling(Convn(X' + pe))) (7)

[0019] X convc = X' + max(0, Bn(Convn(pooling(X' T ))) (8)

[0020] Wherein, X convt represents the time dimension local information, X' represents the input sequence data, pooling represents the average pooling, Convn represents the convolution operation, Bn represents the batch normalization, pe represents the position encoding information, X convc represents the channel dimension local information, X' T represents the transpose of the input information X'.

[0021] Step four: the time step local information X convc and channel step local information X convc extracted by convolutional neural network are respectively input into multi-head time step attention mechanism Trm time and multi-head channel step attention mechanism Trm channel to obtain the time dimension global information and the channel dimension global information In the multi-head self-attention mechanism, the time step local information X convt is first transformed into different subspaces by multiple different linear transformations Each subspace only focuses on d v = d k dimension information, in each subspace, first, the dot product of the transpose of Q i and K i is calculated to obtain the attention weight matrix between each feature, then the attention weight matrix is normalized using the softmax function, and finally the weighted sum of V i is obtained, then the correlation output between the features in each subspace is obtained The outputs in each subspace are spliced using the Concat function, and then linearly transformed to obtain the output X h of the multi-head self-attention mechanism, and then a residual connection module is added to simplify network training and reduce network degradation. The related formula is as follows:

[0022] Q i = X Convt · W Q , Ki = X Convt · W K , V i = X Convt · W V (9)

[0023]

[0024] X h = Concat(Head1, Head2,..., Head h ) W O (11)

[0025] X time = X Convt + X h (12)

[0026] wherein, represents data with global information after passing through the multi-head attention mechanism, X h represents the output of the multi-head self-attention mechanism, is a linear transformation matrix, is a scaling factor, which is introduced to prevent the gradient from being too small and the attention weight from being scattered. Similarly, the change in the channel dimension is the same as the above, that is, the time step local information X convc is replaced by the channel step local information X convc , to obtain the channel dimension global information

[0027] Step five: add the Gumbel-SoftMax function for calculating the weight, calculate X time and X channel to obtain the time dimension weight ratio t and the channel dimension weight ratio c, and then multiply the feature information of the two dimensions by the corresponding weights and concatenate them through Concat; finally, the classification result X out ∈ R N×1 is obtained through the full connection layer. The related formula is as follows:

[0028] (t, c) = gum(X time , X channel ) (13)

[0029] X out = max(linear(Concat(X time *t, X channel *c))) (14)

[0030] wherein t represents the weight ratio of the time dimension, c represents the weight ratio of the channel dimension, and gum represents the Gumbel-SoftMax function.

[0031] X out The classification result is represented by linear, and the concatenation function is represented by Concat. The value in X out is changed to a certain value between 0 and 1 by using the Sigmoid function. If the value after the Sigmoid function processing is greater than or equal to 0.5, the final detection result of the model is 1, indicating that the model detects that the sample is a fault sample. If the value is less than 0.5, the final detection result of the model is 0, indicating that the model detects that the sample is a normal sample. The above formula is as follows:

[0032]

[0033]

[0034] Where sig represents the Sigmoid function.

[0035] Step six: Calculate the cross-entropy loss function between Y and the actual label, and constantly update the model network parameters, and save the best model for subsequent testing.

[0036] Step seven: fault detection stage, the process is as follows:

[0037] 1) Collect new data X test ∈R 4000×16 in the industrial process, and use the mean and variance S train of the training data for standardization

[0038] 2) The standardized data X test is input to the model saved in the training stage for fault detection.

[0039] 3) The result X out of the encoder is mapped to X out ∈R N×1 using a fully connected layer, and finally the value of X out is converted to a certain value between 0 and 1 by using the Sigmoid function. If the value is greater than or equal to 0.5, it is set to 1, indicating that the model detects that the sample is a fault sample. If the value is less than 0.5, it is set to 0, indicating that the model detects that the sample is a normal sample.

[0040] The beneficial effects produced by the above technical solutions are that the industrial process fault detection method based on the improved Transformer model is provided.

[0041] The advantages are as follows:

[0042] (1) The method utilizes the local connection characteristics of the convolutional neural network, respectively convolves the input data X in the time dimension and the channel dimension, sets a multi-layer convolution structure, and configures convolution kernels of different sizes for each layer of convolution to extract local feature information, thereby making up for the problem of insufficient local information extraction capability of the Transformer model.

[0043] (2) The method utilizes the time step attention mechanism and the channel step attention mechanism, needs to consider not only the relationship in the time dimension of the multivariate time series, but also the relationship between channels to improve the feature information extraction capability of the model.

[0044] (3) The method utilizes the Gumbel-SoftMax function to accurately calculate the weight of all data in the time dimension and the channel dimension.

[0045] In summary, the method greatly improves the accuracy of industrial process fault detection. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a structural diagram of a penicillin fermentation process;

[0047] Figure 2 is a flow chart of the Transformer model based on the convolutional neural network and weight calculation of the application;

[0048] Figure 3 is an architecture diagram of the Transformer model based on the convolutional neural network and weight calculation of the application;

[0049] Figure 4 is a structure diagram of the convolutional neural network extracting time step and channel step local information;

[0050] Figure 5 is a structure diagram of the attention mechanism extracting time step and channel step global information;

[0051] Figure 6 is a t-SNE visualization distribution diagram of the original data;

[0052] Figure 7 is a t-SNE visualization distribution diagram of the original data after the first layer of convolutional neural network;

[0053] Figure 8 is a t-SNE visualization distribution diagram of the original data after the second layer of convolutional neural network;

[0054] Figure 9 is a t-SNE visualization distribution diagram of the original data after the third layer of convolutional neural network. DETAILED DESCRIPTION

[0055] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0056] This invention takes the penicillin fermentation process as an example. A schematic diagram of the penicillin fermentation production process is shown below. Figure 1 As shown, the industrial process fault detection method based on the improved Transformer model is as follows: Figure 2 As shown in the diagram, the Transformer model architecture based on convolutional neural networks and weight calculation is as follows: Figure 3 As shown in the diagram, the structure of a convolutional neural network for extracting local information at time steps and channel steps is as follows: Figure 4 As shown in the diagram, the structure of the attention mechanism for extracting global information at time steps and channel steps is as follows: Figure 5 As shown.

[0057] Specific experimental steps:

[0058] Step 1: Collect sample data from the industrial process. The data used in this simulation experiment was generated by Pensim V2.0. The penicillin fermentation process has a total of 16 variables: aeration rate, stirrer power, substrate feed rate, substrate feed temperature, substrate concentration, DO concentration, biomass concentration, penicillin concentration, culture medium volume, carbon dioxide concentration, pH, temperature, heat of reaction, acid flow rate, alkali flow rate, and cold water flow rate. In this normal operation mode, the initial conditions, setpoint, temperature, PID controller, pH, etc., were all set to default. Fault 1 introduced a stirrer power fault with a fault slope set to +9. Fault 2 introduced an input substrate feed rate fault with a fault amplitude set to +8%. Both fault types were introduced into the training data after 200 hours, and into the test data after 300 hours. Each operating mode lasted 400 hours, with a sampling interval of 0.2 hours. That is, the training dataset X was collected. train The test dataset X contains 4000 samples. test This involves 4000 sample data points. Data X train ∈R 4000×16 , for X train Perform standardization. The standardization process is shown in the following formula:

[0059]

[0060]

[0061]

[0062] Step 2: Input data X train ∈R 4000×16 This is achieved by segmenting the data into segments, thus reducing computational complexity. Each segment has a length of L.seg = 100, each sequence is divided into 40 segments, and the input is X' e R 40×100×16 .

[0063] Step three: local information feature extraction is performed on the data time dimension and channel dimension respectively. In the time dimension: the input information is added with position encoding, and the position encoding (PositionalEncoding, PE) is added by using the sine and cosine functions to mark the position information. The position encoding is as follows:

[0064]

[0065]

[0066] Where pos is the position index of the time series data in the sequence, and i is a certain dimension of the vector. The input information is transposed in the channel dimension. In the time dimension, the number of channels is mapped to a high-dimensional space through a fully connected neural network (FullyConnectedNeuralNetwork, FC), and Xte R 40×100×512 is obtained. In the channel dimension, the time step is mapped to a high-dimensional space through a fully connected neural network to obtain Xce R 40×16×512 . Then X and X are input into a multi-layer convolution structure, and each layer of convolution is configured with a convolution kernel of different size to extract local feature information. At the same time, the feature information obtained by convolution may have redundancy, so an average pooling layer is used to eliminate the redundant information of the feature and improve the robustness of the training model. In addition, a Bn layer is added to prevent gradient explosion and gradient disappearance, and a residual connection module is added to focus on the difference. Finally, the time dimension local information X convt and the channel dimension local information X convc are obtained. The above formula is:

[0067] X convt = X' + max (0, Bn (pooling (Convn (X' + pe))) ) (22)

[0068] X convc = X' + max (0, Bn (Convn (pooling (X' T ))) ) (23)

[0069] Where X convt represents the time dimension local information, X' represents the input sequence data, pooling represents the average pooling, Convn represents the convolution operation, Bn represents the batch normalization, pe represents the position encoding information, and X convc represents the channel dimension local information, and X' T represents the transpose of the input information X'.

[0070] Step four: the time step local information X convc and channel step local information X convc enter the multi-head time step attention mechanism Trm time and multi-head channel step attention mechanism Trm channel respectively to obtain the time dimension global information X time ∈R 40×100×512 and channel dimension global information X channel ∈R 40×16×512 In the multi-head self-attention mechanism, the time step local information X convt is first transformed into Q i ∈R 40×100×8 , K i ∈R 400×100×8 , V i ∈R 40×100×8 in different subspaces through multiple different linear transformations, and only d v =d k dimension information is concerned in each subspace, and d v =d k =8 in this embodiment. In each subspace, the transpose of Q i and K i are first dot multiplied to calculate the attention weight matrix between each feature, then the attention weight matrix is normalized using the softmax function, and finally weighted summation is performed with V i to obtain the correlation output Head i ∈R 40×100×8 between features in a subspace, i∈[1, h], and h=8 in this embodiment. The outputs in each subspace are spliced using the Concat function, and then linearly transformed to obtain the output X h of the multi-head self-attention mechanism, and then a residual connection module is added to simplify network training and reduce network degradation. The related formula is as follows:

[0071] Q i =X Convt ·W Q , K i =X Convt ·W K , V i =X Convt ·W V (24)

[0072]

[0073] X h =Concat(Head1, Head2,..., Head h )WO (26)

[0074] X time = X Convt + X h (27)

[0075] where X time ∈ R 40×100×512 represents the data with global information after passing through the multi-head attention mechanism, X h represents the output of the multi-head self-attention mechanism, W Q ∈ R 512×8 , W K ∈ R 512×8 , W V ∈ R 512×8 , W O ∈ R 64×512 are linear transformation matrices, is a scaling factor, which is introduced to prevent the gradient from being too small and the attention weight from being scattered. Similarly, the change in the channel dimension is the same as above, and in the case where the channel dimension remains unchanged, the time step local information X convc is replaced by the channel step local information X convc , and after the same operation, the global information X channel ∈ R 40×16×512 in the channel dimension is obtained.

[0076] Step five: add the Gumbel-SoftMax function for calculating the weight, calculate X time and X channel to obtain the weight proportion t in the time dimension and the weight proportion c in the channel dimension, and then concatenate the feature information in the two dimensions after multiplying them by the corresponding weights; finally, the classification result X out ∈ R 40×1 is obtained through the full connection layer. The related formula is as follows:

[0077] (t, c) = gum(X time , X channel ) (28)

[0078] X out = max(linear(Concat(X time *t, X channel *c))) (29)

[0079] where t represents the weight proportion in the time dimension, c represents the weight proportion in the channel dimension, gum represents the Gumbel-SoftMax function,

[0080] X outThis represents the classification result, where `linear` represents a fully connected layer and `Concat` represents the concatenation function. Then, the `Sigmoid` function is used to convert X... out If the value in the expression changes to a value between [0,1] and after processing by the Sigmoid function, its value is greater than or equal to 0.5, then the model's final detection result is 1, indicating that the model detected the sample as a faulty sample. If the value is less than 0.5, then the model's final detection result is 0, indicating that the model detected the sample as a normal sample. The above formula is shown below:

[0081]

[0082]

[0083] Where sig represents the Sigmoid function.

[0084] Step 6: Calculate the cross-entropy loss function between Y and the actual label, continuously update the model network parameters, and save the best-performing model for subsequent testing.

[0085] Step 7: Fault detection phase, the process is as follows:

[0086] 1) Collecting new data X from industrial processes test ∈R 4000×16 Using the mean of the training data S and variance train Standardization processing

[0087] 2) Standardize the data X test The data is input into the model saved during the training phase for fault detection.

[0088] 3) The result X from the encoder out Map to X using a fully connected layer out ∈R N×1 Finally, the Sigmoid function is used to convert X... out The value is converted to a value between [0,1]. If the value is greater than or equal to 0.5, it is set to 1, indicating that the model has detected the sample as a faulty sample; if the value is less than 0.5, it is set to 0, indicating that the model has detected the sample as a normal sample.

[0089] Analysis of simulation experiment results:

[0090] In this embodiment, the fault detection accuracy rate is 97.5%. The results show that the process monitoring method of this invention demonstrates good fault detection performance.

[0091] Next, the fault detection performance of this invention will be analyzed. This invention uses t-SNE (T-Distributed Stochastic Neighbor Embedding) to visualize and analyze the convolutional layers. The t-SNE visualization results are as follows: Figures 6 to 9 As shown, label 0 represents normal data (blue), and label 1 represents faulty data (orange). First, as... Figure 6 As shown, in the original data, blue and orange dots are mixed together irregularly, indicating that fault data cannot be distinguished from normal data, which is detrimental to fault detection tasks; then, as... Figure 6 As shown, after the first convolutional layer, the orange dots gradually separate from the blue dots, but... Figure 6 In the bottom right corner, two blue dots remain separated from the orange dots, indicating that after one convolutional layer, most of the normal data can be distinguished from the faulty data; finally, from Figures 6 to 9 As can be seen, as the number of convolutional layers increases, the orange and blue dots gradually separate, indicating that after multiple convolutional layers, the model can correctly identify fault data and normal data, thus completing the fault detection task.

[0092] In summary, multi-layer convolutional structures have a strong ability to distinguish between normal and faulty data. By increasing the number of layers in the convolutional neural network, inseparable features can be mapped to a nonlinear separable space, effectively detecting faulty and normal data. This also verifies the rationality of the design idea of ​​increasing the number of convolutional layers in the model.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A method for detecting industrial process faults based on convolutional neural networks and weight models, characterized in that, The method, based on a convolutional neural network and a Transformer model for weight calculation, applies this model to industrial process fault detection and includes the following steps: Step 1: Collect N sample data X∈R from the industrial process. N×k X = [X1, X2, ..., X i ,…X N ] T Where k represents the number of data channels, N represents the total number of samples collected, and X i Let X represent the i-th sample in data X, where i∈[1,N]. X is standardized as shown in formulas (1)-(3): Formulas (1) and (2) yield the mean and variance of the training dataset, respectively. Formula (3) is used to calculate the mean of the training dataset. S and variance train Standardize X; Step 2: Divide the input data X into segments to reduce computational complexity; assume that the length of each segment is L. seg Each sequence needs to be divided into L segments, and the input is denoted as L. The division process is shown in formula (4): Step 3: Extract local information features in the time dimension and channel dimension of the data respectively; in the time dimension: add positional encoding to the input information, and use sine and cosine functions to mark the position information; the positional encoding is shown in formulas (5) and (6): Where pos is the position index of the time-series data in the sequence, and i is a certain dimension of the vector; the input information is transposed in the channel dimension; in the time dimension, the number of channels is mapped to a high-dimensional space through a fully connected neural network (FC), resulting in... Where d model The dimension is obtained by mapping the number of channels; the time steps are then mapped to a higher-dimensional space using a fully connected neural network on the channel dimension. Where d model The dimension is obtained by mapping the time steps. Then, Xt and Xc are input into a multi-layer convolutional structure, with each layer configured with a kernel of different sizes to extract local feature information. Meanwhile, the feature information obtained from convolution may contain redundancy; therefore, an average pooling layer is used to remove redundant information and improve the robustness of the trained model. In addition, a Bn layer is added to prevent gradient explosion and vanishing, and a residual connection module is added to focus on the differences. Finally, the local information X in the time dimension is obtained. convt Local information X in the channel dimension convc The above formula is: X convt =X'+max(0,Bn(pooling(Convn(X'+pe)))) (7) X convc =X'+max(0,Bn(Convn(pooling(X' T )))) (8) Among them, X convt Representing local information in the time dimension, X' represents the input sequence data, pooling represents average pooling, Convn represents the convolution operation, Bn represents batch normalization, pe represents positional encoding information, X convc Represents local information at the channel dimension, X' T This represents the transpose of the input information X'; Step 4: Extract the local time-step information X from the convolutional neural network convc and channel step local information X convc Enter the multi-head time step attention mechanism TRM respectively time and multi-channel step attention mechanism TRM channel Get global information in the time dimension Global information in the channel dimension In a multi-head self-attention mechanism, the local information X at each time step is first transformed using multiple different linear transformations. convt Transformed into different subspaces Only d is considered within each subspace. v =d k Information in each dimension, first Q in each subspace i and K i The attention weight matrix between each feature is calculated by performing a dot product on the transpose of V, then normalized using the softmax function, and finally combined with V. i By performing a weighted summation, we obtain the correlation output between features within a subspace. The outputs from each subspace are concatenated using the Concat function, and then subjected to a linear transformation to obtain the output X of the multi-head self-attention mechanism. h Subsequently, a residual connection module was added to simplify network training and reduce network degradation; the relevant formulas are shown below: Q i =X Convt ·W Q ,K i =X Convt ·W K ,V i =X Convt ·W V (9) X h =Concat(Head1,Head2,...,Head h )W O (11) X time =X Convt +X h (12) in, X represents data with global information after passing through a multi-head attention mechanism. h This represents the output of the multi-head self-attention mechanism. It is a linear transformation matrix. It is a scaling factor, introduced to prevent gradients from becoming too small and attention weights from becoming scattered; similarly, the change in channel dimension is the same as above, that is, the local information X at the time step. convc Replace with channel step local information X convc To obtain global information in the channel dimension Step 5: Add the Gumbel-SoftMax function to calculate the weights for X. time and X channel The calculations yield the weight percentages for the time dimension (t) and the channel dimension (c). The feature information from each dimension is then multiplied by its corresponding weight and concatenated using Concat. Finally, a fully connected layer is used to obtain the classification result X. out ∈R N×1 The relevant formulas are shown below: (t,c)=gum(X time ,X channel ) (13) X out =max(linear(Concat(X time *t,X channel *c))) (14) Where t represents the weighting percentage of the time dimension, c represents the weighting percentage of the channel dimension, gum represents the Gumbel-SoftMax function, and X... out The classification result is represented by `linear`, which indicates a fully connected layer, and `Concat`, which indicates a concatenation function. Then, the `Sigmoid` function is used to convert X... out If the value in the expression changes to a value between [0,1] and after processing by the Sigmoid function, its value is greater than or equal to 0.5, then the model's final detection result is 1, indicating that the model detected the sample as a faulty sample; if the value is less than 0.5, then the model's final detection result is 0, indicating that the model detected the sample as a normal sample. The above formula is shown below: Where sig represents the Sigmoid function; Step 6: Calculate the cross-entropy loss function between Y and the actual label, continuously update the model network parameters, and save the best-performing model for subsequent testing; Step 7: Fault detection phase, the process is as follows: 1) Collecting new data X from industrial processes test ∈R 4000×16 Using the mean of the training data S and variance train Standardization processing 2) Standardize the data X test The model saved during the training phase is input for fault detection; 3) The result X from the encoder out Map to X using a fully connected layer out ∈R N×1 Finally, the Sigmoid function is used to convert X... out The value is converted to a value between [0,1]. If the value is greater than or equal to 0.5, it is set to 1, indicating that the model has detected the sample as a faulty sample; if the value is less than 0.5, it is set to 0, indicating that the model has detected the sample as a normal sample.

Images