A fault diagnosis classification method based on convolution fusion network and attention mechanism

Abstract

The application discloses a kind of based on convolution fusion network and attention mechanism's fault diagnosis classification method, first constructs data fusion fault detection framework, first data is collected and preprocessed, then by one-dimensional convolutional neural network based on attention, to obtain the independent features of two modalities.Further realize the self-fusion of feature, solve the extraction problem of modal mutual relationship.The output of self-fusion feature is output through the squeeze excitation module, attention weighting and gradient inversion layer, and then input into the discriminator for training to obtain the modal interaction feature.The two outputs of the squeeze excitation module are spliced into features, input into the long short-term memory network for fusion, and the output is used for result inference in the inference network.Finally, the network is trained with the minimum target loss as the goal by iterating and combining the inference result, and fault diagnosis classification is realized.Multi-modal data fusion effectively utilizes the interaction features between modalities, which can improve the accuracy of fault detection and classification.

Description

Technical Field

[0001] This invention relates to the field of fault diagnosis and classification, specifically to a fault diagnosis and classification method based on convolutional fusion networks and attention mechanisms to complete fault detection. Background Technology

[0002] Basic components are widely used in factory machinery and equipment, playing a crucial role in the operation of these parts. Because these parts experience wear and tear during use, performance degradation can occur, potentially leading to malfunctions or accidents. Therefore, one of the key guarantees for ensuring machine reliability and safety in industrial systems is the continuous monitoring and evaluation of the condition of these components.

[0003] As the types and performance of sensors continue to improve, they can provide rich and diverse data. Making reasonable use of the information from different sensors can help to better complete the bearing detection task. Data fusion methods are mainly classified into data-level, feature-level and decision-level fusion according to different levels of the system. (1) Data-level fusion refers to fusion at the original data level, that is, information integration and analysis is performed on the original information before various sensors have done much preprocessing. This is the lowest level of fusion. This method can retain the most information, but it is easily affected by noise. (2) Feature-level fusion is an intermediate level. It extracts features from the original information from the sensors and then performs comprehensive analysis and processing on the feature information. Feature-level fusion can be divided into two categories: target state information fusion and target characteristic fusion. Target state information fusion is mainly used in the field of multi-sensor target tracking. The fusion system first preprocesses the sensor data to complete data calibration, and then mainly realizes parameter correlation and state vector estimation. Target characteristic fusion is feature-level joint recognition. The specific fusion method is still the corresponding technology of pattern recognition. However, before fusion, the target features must be correlated and the feature vectors must be classified into meaningful combinations. (3) Decision-level fusion is a high-level fusion. Decision-level fusion must start from the needs of specific decision-making problems, fully utilize the various feature information of the measurement object extracted by feature fusion, and adopt effective fusion techniques. Decision-level fusion is the final result of three-level fusion, directly targeting specific decision-making objectives, and the fusion result directly affects the decision-making level. Decision-level fusion is less affected by noise, but it first requires preprocessing the original sensor information to obtain the respective judgment results. Some information is lost during the feature extraction stage, so the preprocessing cost is high.

[0004] Typical traditional data fusion methods include Kalman filtering, Dempster evidence reasoning, and neural networks. Kalman filtering uses the statistical properties of measurement models to recursively estimate multi-source data, achieving real-time fusion representation of redundant data. It is a classic data-layer fusion algorithm, but it can only handle linear problems, has low observability, and is prone to divergence. Dempster evidence reasoning uses the Dempster synthesis rule to merge various evidence bodies into a new evidence body, using uncertainty intervals and probability intervals to determine the likelihood function of hypotheses in multi-evidence situations, and then performs reasoning and fusion based on this. It is a typical decision-layer fusion algorithm. This algorithm is suitable for fusion of imprecise and incomplete information, but struggles with inconsistent information. Neural network methods use neural networks to extract data features and perform feature fusion within the network, making it a typical feature-layer fusion algorithm. However, shallow neural networks are ill-suited for data with high feature dimensionality, large structural differences, and uneven distribution (few fault data, many normal samples). Deep neural networks have powerful feature extraction and synthesis capabilities, and through representation learning methods, they can be applied to the fusion of unstructured, high-dimensional, multimodal (e.g., vibration, voltage, current), and heterogeneous data (e.g., images, text, audio). Therefore, feature layer fusion algorithms based on deep learning can effectively solve the problem of fusion for multimodal, imbalanced, and heterogeneous data.

[0005] In deep learning-based multimodal data fusion methods, different feature extraction networks are typically used first to extract data features from different modalities. The concatenated features are then input into a fusion network for integration and decision-making. This method of extracting features individually does not consider the interactions between modalities during feature extraction, resulting in the loss of some feature information and a degraded overall network performance. Therefore, capturing the interactive features between modalities before multimodal data fusion is of great significance. Summary of the Invention

[0006] This method mainly adopts multimodal data fusion to improve the accuracy of industrial parts detection and classification. Existing data fusion methods mainly have the following problems: (1) Typical traditional data fusion methods include Kalman filtering, DS evidence reasoning, neural networks, etc., which cannot adapt to data with high feature dimensions, large structural differences, and unbalanced distribution. (2) Most methods using deep learning technology fuse data from a single sensor without considering the use of sensor data from other modalities to supplement the information. (3) For data fusion methods from different sensors, most methods extract features separately and then fuse them, which makes it impossible to obtain the interaction features between different modalities.

[0007] In view of the above problems, this invention proposes a fault diagnosis and classification method based on convolutional fusion networks and attention mechanisms. Taking bearing fault detection as an example, vibration and current data are fused to better complete the fault diagnosis task. The specific flowchart is as follows: Figure 1 As shown in the diagram, this method employs a one-dimensional convolutional neural network (CNN) to extract correlations between feature maps. An attention mechanism is used to assign different weights to the feature maps, extracting features and interaction features of each modality, thus solving the challenge of extracting modal relationships. Then, a long short-term memory network and a multilayer perceptron are used to extract and fuse temporal features. Finally, an inference neural network is used, through iterative iteration and inference results, to achieve fault diagnosis and classification. The specific model structure diagram is shown below. Figure 2 As shown.

[0008] Specifically, the following steps are included:

[0009] Step 1: Data preprocessing. Normalize the input vibration and current data.

[0010] Step 2: Independent Feature Extraction. The processed vibration and current data are input into a one-dimensional convolutional neural network and attention mechanism module for training to obtain independent features of the two modalities.

[0011] The first feature fusion stage is self-feature fusion, meaning it processes data from two modalities independently. For example... Figure 3 As shown, each part consists of two attention-based one-dimensional CNN modules. Assuming the input is s, convolution is performed first, then max pooling is applied to reduce the dimensionality, followed by batch normalization (BN), as shown in the following formula:

[0012]

[0013]

[0014]

[0015] Feature maps are used to generate evaluation vectors. The importance of the feature maps is evaluated through global average pooling and two fully connected (FC) layers, with ReLU and softmax activation functions, respectively. As the feature layers deepen, some information is lost. The input data is then convolved, and the resulting feature maps are concatenated with the previous feature maps to generate new feature maps, which serve as the output of this module.

[0016] Two convolutional attention networks were used to process the vibration signal input (s) respectively. v ), current signal input (s) c They are trained to extract their independent features, represented as follows:

[0017] Fv (s v ) = CAM v (s v (4)

[0018] F c (s c ) = CAM c (s c (5)

[0019] Here, CAM represents feature extraction of different modal data through a convolutional attention network.

[0020] Step 3: Attention-weighted / modal interaction feature acquisition. The two outputs are passed through the Squeeze-Activation Module (SEBlock) for attention weighting and a gradient reversal layer, and then input into the modal discriminator for training to obtain modal interaction features.

[0021] Since the fault manifestation times in a signal sequence are mainly concentrated around the trigger time of the fault point, fault features can be learned more efficiently by focusing on the fault sequence. This invention incorporates a squeezing excitation module to perform attention weighting on the time-series signal, thereby amplifying fault features and reducing non-fault features, resulting in an attention-weighted representation, as shown in the following formula:

[0022]

[0023]

[0024] Feature fusion employs a concatenation approach. In this stage, features extracted from the two inputs are concatenated along different dimensions from the first stage. The features in the two parts are as follows: and The output of this stage can be represented as:

[0025]

[0026] Where Conc represents the vector concatenation operation. and This represents a weighted attention representation of two modalities.

[0027] The modality discriminator and gradient inversion layer obtain the generative adversarial loss, and attention-weighted representation is applied. and Input the gradient inversion layer R, and obtain the gradient inversion representation r. v and r c :

[0028]

[0029]

[0030] Where R is the gradient inversion layer (GRL) function (its output is the same as the identity function, but the gradient direction is opposite). Then, the gradient inversion is represented as r. v and r c The input is fed into the modality discriminator D (Discriminator) to obtain the input d of the adversarial loss function:

[0031]

[0032] in, Here, represents the network parameters of the modality discriminator; D is the modality discriminator, used to identify the modality of a sample. The adversarial loss function generated in this way enables the modality discriminator to perform a gradient ascent operation (gradient inversion layer) during gradient descent learning tasks. This makes it difficult for the modality discriminator to distinguish between two modalities, allowing the network to better capture modality invariant features.

[0033] Step 4: Modality Fusion / Inference. The output of the SE module in Step 3 is concatenated with features and then fed into a Long Short-Term Memory (LSTM) network for fusion. The output of this fusion network is then used to infer the result.

[0034] Inputting the concatenated joint representation into the LSTM yields the following representation:

[0035]

[0036] Where M represents memory fusion, This represents the network parameters of the LSTM.

[0037] Step 5: Iterate repeatedly. Based on the inference results, train the network with the goal of minimizing the target loss.

[0038] The memory fusion representation is input into the inference neural network (Inf) for fault inference, as shown below:

[0039]

[0040] Where O represents the inference representation and Inf represents the inference network parameters. Attached Figure Description

[0041] To more clearly illustrate the embodiments or technical solutions of the present invention, the accompanying drawings used in the description of the embodiments or prior art are briefly introduced below.

[0042] Figure 1 The flowchart below shows the multimodal feature fusion and fault detection classification method of this approach.

[0043] Figure 2 The diagram shows the framework of a fault diagnosis classification network based on convolutional fusion networks and attention mechanisms.

[0044] Figure 3 A diagram of a one-dimensional CNN module based on attention. Detailed Implementation

[0045] The specific implementation steps of this invention mainly include:

[0046] 1. Feature Extraction

[0047] (1) Convolutional Neural Network (CNN)

[0048] CNNs are inspired by the working mechanism of the brain's visual cortex, which processes images in a hierarchical and distributed manner. CNNs extract features through numerous convolutional pooling modules, followed by final classification using fully connected layers. Convolutional kernels slide across the entire image with a fixed stride to extract features, which vary depending on the kernel's weights. Pooling layers follow the convolutional layers to reduce feature dimensionality without losing features. One-dimensional CNNs are better suited for processing sequential data.

[0049] In the nth convolutional pooling module of the CNN, the output formula of the convolutional layer is as follows:

[0050]

[0051] Where l represents the l-th layer of the network, i = 1, 2, ..., i are the indices of the input feature maps, and i' = 1, 2, ..., i' represent the indices of the output feature maps. If l = 1, it means the network is the first layer, and the input is the initial image matrix. * represents the i-th convolutional kernel. In layer l, the i-th feature map from layer (l-1) Perform discrete convolution with a certain stride, and then add a bias matrix. The final convolutional output is obtained. `f` is an activation function applied to the convolutional output to perform a non-linear transformation on each output. Adding an activation function maps the features to a more complex non-linear space, enhancing the linear separability of the features in the mapped space. Three common activation functions include the Sigmoid function, the hyperbolic tangent Tanh function, and the Rectified Linear Unit (ReLU). This invention uses ReLU to prevent gradient vanishing and exploding. The output after the activation function is:

[0052]

[0053] in, Let represent the i-th feature map output by the l-th layer network, where the (x, y)-th element is denoted.

[0054] A pooling layer is then periodically inserted between convolutional layers. Its function is to gradually reduce the data dimensionality, thereby reducing the number of parameters in the network, reducing computational resource consumption, and effectively controlling overfitting. Like convolutional layers, pooling layers also have a spatial window. Typically, the maximum or average value of this window is extracted as the output, and the window is continuously slid to reduce the size of the input data. Pooling equations can be expressed using max pooling, average pooling, and weighted pooling. Max pooling is the most common in convolutional neural networks. The max pooling formula is:

[0055]

[0056] Where w represents the width of the pooling window and h represents the height.

[0057] After a series of stacked convolutional modules extract features from the input image, the final feature map is flattened into a one-dimensional feature vector, and then connected by fully connected layers to perform classification or regression tasks. The forward propagation expression of the fully connected layer is:

[0058]

[0059] in, This represents the weight between the i-th neuron in layer l and the j-th neuron in layer l+1. This represents the output value of the i-th neuron in the (l+1)-th layer. This represents the bias. The hidden layers of a fully connected layer are activated using the ReLU function, and the last layer uses the Softmax function as the activation function because the Softmax function can transform the neurons in the last layer of a fully connected layer into a probability distribution that sums to 1.

[0060]

[0061] Where q = q1, q2, ..., q z represents the output of the fully connected layer, and x represents the number of neurons in the last layer of the fully connected layer, which is the number of labels to be classified.

[0062] (2) Attention mechanism

[0063] The feature maps generated by different convolutional kernels contribute differently to the final classification result. Therefore, the attention mechanism aims to improve the network's sensitivity to features generated by different convolutional kernels, thereby adaptively distinguishing their importance.

[0064] Suppose the input is S = [s1, s2, ..., s...]. xIt consists of x feature maps. These feature maps are generated by x convolutional kernels, and each feature map has a length of w. First, the feature maps are compressed into a vector Z corresponding to the x convolutional kernels. The i-th element of Z can be calculated as:

[0065]

[0066] The channel recalibration vector is obtained using the following formula:

[0067] Z'=F(Z) (20) where F is a transformation function used to evaluate the weights of each channel feature map.

[0068] The i-th element of Z' represents the importance of the i-th channel feature map. Finally, the feature X is recalibrated using the vector Z':

[0069] X'=X·Z'=[x1z'1,x2z'2,…,x C z' C ] (twenty one)

[0070] Therefore, the obtained feature X' fully considers global information and effectively highlights the useful information in the feature map.

[0071] 2. Feature Fusion

[0072] (1) Long Short-Term Memory Neural Network (LSTM)

[0073] Long Short-Term Memory (LSTM) neural networks are variants of traditional Recurrent Neural Networks (RNNs). Compared to classic RNNs, they effectively capture semantic relationships between long sequences and mitigate gradient vanishing or exploding phenomena. LSTMs are also more complex, with their core structure consisting of four parts, including a forget gate (f...). t ), Input gate (i t ), output gate (o t ), cell state (c t The specific structure is as follows: Figure 3 As shown.

[0074]

[0075] h t =o t ·tanh(c t )

[0076] Where, x t c t h t These represent the input unit, cell state, and output unit at time t, respectively; c t-1 h t-1δ represents the cell state and output unit at time t-1, respectively; W is the nonlinear activation function; b is the weight matrix; and b is the bias.

[0077] (2) Extrusion Excitation Module (SE BLOCK)

[0078] The squeeze-excitation module is a channel attention module commonly used in visual models. It enhances the channel features of the input feature map, and the final SE module output does not change the size of the input feature map. This module mainly consists of three parts: squeeze, excitation, and scale.

[0079] The first step is the convolution operation, which is a transformation operation and a standard convolution operation. The input, output, and convolution operation formula are as follows:

[0080] F tr : X→U, X∈R W*H*C ,U∈R W*H*C (twenty three)

[0081]

[0082] Among them, F tr This is a convolution operation; W′*H′*C′ and W*H*C represent the height, width and channels of X and U respectively; U is actually a three-dimensional matrix composed of C feature maps of size H*W, with dimensions [H, W, C].

[0083] v c Let x represent the c-th convolutional kernel. s C' represents the s-th input covered by the current convolutional kernel, and C' represents the number of convolutional kernels.

[0084] Then, global average pooling is used to compress each feature map in U, which is equivalent to showing the numerical distribution of the C channels in that layer, or global information. The process is as follows:

[0085]

[0086] Among them, F sq For global average pooling operation, z c It is an array of real numbers.

[0087] The convolution operation is followed by an activation operation, which learns and updates the weights of each feature channel using parameters. The specific formula is as follows:

[0088] s = sigmond(W2*ReLU(W1z) c (26)

[0089] Among them, z cAs obtained in the previous step, W1 and W2 represent linear layers; s is the weight array of the output of the activation operation. These weights are learned through fully connected layers and non-linear layers. The role of these two fully connected layers is to fuse the feature information of each channel.

[0090] The dot product operation is used to transform the one-dimensional real number sequence s obtained after the activation operation. c With feature mapping u c We perform a dot product operation to achieve attention weighting, as shown in the following formula:

[0091]

[0092] in, For SE module output, F scale It's a dot product operation, u c s is a channel in U c This indicates the weight of the channel.

[0093] 3. Fault Diagnosis Classification

[0094] The fault diagnosis and classification module inputs the memory fusion representation into the inference neural network (Inf) for fault inference, as shown below:

[0095]

[0096] Where O represents the inference representation result, and M represents the fusion representation. To infer network parameters, Inf is a multilayer perceptron with Leaky ReLU and Tanh activations. Through iterative iteration, and by combining the inference results, the network is trained with the goal of minimizing the target loss to achieve fault diagnosis and classification.

Claims

1. A fault diagnosis and classification method based on convolutional fusion networks and attention mechanisms, characterized in that, It includes the following steps: A data fusion fault detection framework is constructed, including a data acquisition module, a preprocessing module, an attention-based one-dimensional convolutional neural network feature self-fusion module, a squeeze excitation module, a feature fusion module, and a fault diagnosis and classification module; Data sets are collected and preprocessed. Taking bearing fault diagnosis and classification as an example, the data acquisition module obtains current and vibration signals through current sensors and acceleration sensors, and divides them into training datasets and test datasets according to the proportions. The processed current and vibration signal data are input into the attention-based one-dimensional convolutional neural network feature self-fusion module to obtain independent features of the two modes, and then further realize the self-fusion of features. The self-fused feature output is passed through the squeezing and excitation module, then attention weighted and gradient inversion layer, and then input into the discriminator for training to obtain modal interaction features. The output of the squeezing excitation module is concatenated with features and then fed into the Long Short-Term Memory (LSTM) network for fusion. Its output is then used by the inference network to infer the result. Iterative iteration; combining the inference results, the network is trained with the goal of minimizing the target loss to achieve fault diagnosis and classification; The attention-based one-dimensional convolutional neural network feature self-fusion module consists of two attention-based convolutional neural network modules. First, convolution is performed, then max pooling is applied to reduce dimensionality, and then batch normalization is performed. The resulting feature map is used to generate an evaluation vector, and the importance of the feature map is evaluated through global average pooling and two fully connected layers. The two activation functions are ReLU and softmax, respectively. The two outputs are passed through the squeeze excitation module (SE), then undergo attention weighting and gradient inversion layer processing before being input into the discriminator for training. The modality discriminator and gradient inversion layer obtain the generative adversarial loss, which is then used to weight the attention-based representation. and Input gradient inversion layer R to obtain gradient inversion representation and The formula is as follows: （5） （6） Where R is the gradient inversion layer GRL function; then the gradient inversion representation is... and The input is fed into the modality discriminator D to obtain the input d of the adversarial loss function: （7） in, denoted by , where is the network parameter of the modality discriminator; D is the modality discriminator used to identify the modality of a sample; the adversarial loss function generated in this way enables the modality discriminator to perform gradient ascent operation, i.e., gradient inversion layer, when performing gradient descent learning task; thus, the modality discriminator has difficulty distinguishing between two modalities, which allows the network to better capture modality invariant features.

2. The fault diagnosis and classification method based on convolutional fusion networks and attention mechanisms as described in claim 1, characterized in that, An attention-based one-dimensional convolutional neural network feature self-fusion module extracts features from different modal data, including vibration signal input. Current signal input The formula is as follows: （1）； （2）； in, This indicates that features are extracted from data of different modalities through a convolutional attention network.

3. The fault diagnosis and classification method based on convolutional fusion networks and attention mechanisms as described in claim 1, characterized in that, Attention-weighted representation is applied to time-series signals to amplify fault features and reduce non-fault features, resulting in the attention-weighted representation, as shown in the following formula: （3）； 。 4. The fault diagnosis classification method based on convolutional fusion network and attention mechanism as described in claim 1 concatenates the output values ​​of the feature self-fusion stage, inputs them into the LSTM network for fusion, and uses the output for the inference network to infer the result; the weighted attention representations of the two modalities are respectively and The output of this stage is represented as: (8); The concatenated joint representation is input into the LSTM for memory fusion, resulting in the following representation: (9), M represents fusion. To infer network parameters.

5. The fault diagnosis and classification method based on convolutional fusion networks and attention mechanisms as described in claim 1, characterized in that, The memory fusion representation is input into the inference neural network for fault inference, as shown below: (10); among which, M represents the inference result, and M represents the fusion representation. To infer network parameters, A multilayer perceptron activated by Leaky ReLU and Tanh.

Images