Convform-SENet-based gear fault diagnosis method
Through the Convformer-SENet algorithm, combined with the advantages of CNN and Transformer, using multi-scale feature extraction and self-attention mechanisms, the problem of CNN's difficulty in capturing global information and high consumption of Transformer resources is solved, and high-accuracy gear fault diagnosis is achieved.
Patent Information
- Application Number
- CN202411113487.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-12-13
AI Technical Summary
Existing CNNs are difficult to fully capture global fault information in gear fault diagnosis, and Transformer is expensive to calculate resources and is easy to overfit when processing long sequences.
The Convformer-SENet algorithm is proposed. By combining the local feature extraction capability of convolutional neural network and the global dependency modeling capability of Transformer, multi-scale feature extraction and self-attention mechanism are used, combined with the channel attention mechanism of SENet, to reduce redundant information and Enhance the nonlinear learning ability of the model.
It effectively improves the accuracy of gear fault diagnosis, avoids information loss and overfitting problems, and significantly improves the accuracy of diagnosis and resource utilization efficiency.
Smart Images
Figure CN119128714A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of mechanical fault diagnosis, and in particular to a gear fault diagnosis method based on Convformer-SENet. Background Art
[0002] Gears are important components of mechanical equipment and play a key role in the automotive industry, wind power generation, agricultural machinery and other fields. Gears are key components for transmitting power, and the operating status of gears directly affects the safety and reliability of the entire mechanical system. Through timely fault diagnosis, faults can be discovered in the early stages of the fault, thereby avoiding potential risks caused by further expansion of the fault.
[0003] Machine learning has the advantages of automation, high accuracy, real-time monitoring and strong adaptability in the field of fault diagnosis, but there is still room for improvement in data requirements, computing resources and model interpretability. In recent years, scholars have conducted extensive research in the field of gear fault diagnosis based on deep learning methods such as Deep Autoencoder (DAE) and Convolutional Neural Networks (CNN). These deep learning methods have shown great potential in the field of fault diagnosis due to their powerful feature extraction capabilities and ability to process complex data. For example, deep autoencoders can automatically extract high-order features from raw data, thereby improving the accuracy and efficiency of fault diagnosis. Deep Convolutional Neural Networks (DCNN) effectively captures spatial features and local patterns in data through its hierarchical structure, and is particularly suitable for processing vibration signals and image data.
[0004] Shao et al. proposed an integrated deep autoencoder for intelligent fault diagnosis of rolling bearings. This method can effectively overcome the limitations of individual deep learning models and effectively improve the accuracy and stability of diagnostic results. Li et al. proposed a bearing fault diagnosis transfer learning method based on adaptive batch normalization (AdaBN) and combinatorial optimization algorithm to alleviate the domain offset problem. In order to solve the problem that the quadratic complexity of the self-attention mechanism of Transformer leads to extremely high computational cost, Sun et al. proposed a lightweight architecture of LiteFormer, which takes into account both algorithm cost and diagnostic performance. Cen et al. proposed the MSFormer algorithm based on masked self-supervised learning (SSL), which works well on a multi-stage centrifugal fan dataset. Chen et al.
[14] proposed a multi-channel multi-task convolutional neural network (MCMT-CNN) model for multiple faults in gearbox fault diagnosis, achieving a diagnostic accuracy of more than 97%.
[0005] The above methods have achieved good results, but there are still some problems. For example, although CNN can fully extract local features of different scales, it is slightly insufficient when integrating and parsing the overall scene and global dependencies of the image; Transformer has the ability of parallel processing, and can effectively capture long-distance dependencies in the sequence through the attention mechanism, but the calculation and memory consumption are high when processing long sequences, and it is easy to overfit when the amount of data is small. This paper proposes a new feature extraction algorithm Convformer-SENet for gear fault diagnosis to solve the problems of global information feature extraction and easy overfitting in the above two methods. Summary of the invention
[0006] The purpose of this invention is to improve the shortcomings of CNN in gear fault diagnosis, that is, it is difficult to fully capture global fault information due to local characteristics limitations, and Transformer has high resource consumption when processing long sequences and is prone to overfitting when the amount of data is small. This paper proposes a new gear fault diagnosis algorithm Convformer-SENet to improve the accuracy of gear fault diagnosis.
[0007] CNN and Transformer each have unique advantages in feature extraction. In order to accurately monitor and detect each fault, it is necessary to extract local area features and global information from sufficient fault information:
[0008] Therefore, the design concept of Convformer is to obtain multi-scale features, including long-distance dependencies and local dependencies;
[0009] The Convformer module is composed of stacked Convformer blocks with different parameters:
[0010] The difference between Convformer and Transformer is that Convformer uses two consecutive convolutional layers to process input data, gradually reducing the input size, increasing the dimension of features, enhancing the nonlinear representation capability, and avoiding the inefficiency of processing long data;
[0011] Secondly, some properties of Transformer are modified to reduce redundant information and better focus on neighboring points. Before calculating K and V matrices, one-dimensional convolution is applied to the input features for downsampling;
[0012] In addition, using CNN in MHSA can enhance the nonlinear learning ability of the model. Convformer can mine multi-scale features and avoid information loss caused by resolution reduction;
[0013] The main computational process of the Convformer model includes processing the input features by the convolution module, calculating the query, key and value matrices, applying the self-attention mechanism to obtain the attention weights, and concatenating and normalizing the multi-head attention outputs, and finally further processing the features through convolution projection;
[0014] This model combines the advantages of convolutional neural networks in local feature extraction and the ability of Transformer in global dependency modeling, thereby effectively capturing multi-scale feature information;
[0015] The specific process of the gear fault diagnosis method based on Convformer-SENet is as follows:
[0016] The gear fault diagnosis method of the Convformer model combines the advantages of convolutional neural networks in local feature extraction and the ability of Transformer in global dependency modeling, thereby effectively capturing multi-scale feature information;
[0017] The main calculation process of the Convformer model is as follows:
[0018] Formula 1: Given input feature X i , through the convolution module to obtain new features
[0019]
[0020] Formula 2: The feature Multiply by the query weight matrix Then we get the query matrix Q. j is the key matrix;
[0021]
[0022] Formula 3, 4: Use one-dimensional convolutional neural network to Processed and multiplied by the key weight matrix Sum value matrix V j ;
[0023]
[0024]
[0025] Formula 5: Apply the SoftMax function to the result to obtain the attention weight Z j . Use these weights to value the matrix V j Perform weighted summation;
[0026]
[0027] Formula 6, 7, 8: Using convolutional projection instead of linear projection in the feedforward neural network further reduces the number of parameters of Convformer;
[0028] Z=Concat(Z1,Z2...,Z h )W O ;
[0029]
[0030]
[0031] SENet is an innovative architecture for improving the performance of convolutional neural networks. SENet improves the representation capability of the model by adaptively recalibrating the feature responses between channels.
[0032] Its core idea is to improve the feature expression ability of the network by explicitly modeling the dependencies between channels;
[0033] SENet mainly consists of three steps, namely, squeezing operation, excitation operation and feature reweighting operation:
[0034] Squeeze operation: Obtain the global features of each channel through global average pooling. Assume that the input feature map is U∈R H×W×C , where H, W, and C are the height, width, and number of channels of the feature map, respectively. The Squeeze operation compresses the spatial information of each channel into a global description through global average pooling:
[0035]
[0036] Excitation operation: The excitation operation adaptively recalibrates the importance of each channel. First, two fully connected layers are used to capture the nonlinear relationship between channels, and the weight of each channel is obtained through the activation function (usually ReLU and Sigmoid). The formula is as follows:
[0037] s=σ(W2δ(W1z))
[0038] In this formula, z=[z1,z2,...,z C ] T is the output of the squeeze operation, and are the weight matrices of the two fully connected layers, δ is the ReLU activation function, σ is the Sigmoid activation function, and r is a scaling factor used to reduce the number of parameters and computations;
[0039] Feature reweighting operation: The feature reweighting operation reweights the weight S generated by the Excitation operation to each channel of the original feature map. The formula is as follows:
[0040]
[0041] Among them, s c is the weight of the cth channel, U c is the cth channel of the input feature map, is the re-weighted c-th channel feature; BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is the general structure of CNN;
[0043] Figure 2 It is the Transformer model structure;
[0044] Figure 3 It is the structure of a multi-head self-attention mechanism;
[0045] Figure 4 is the structure of the Convformer module;
[0046] Figure 5 Convformer model parameters;
[0047] Figure 6 It is the SENet network structure;
[0048] Figure 7 It is the Convformer-SENet model structure;
[0049] Figure 8 Design the diagnostic process for this purpose;
[0050] Fig. 9 It is a power transmission simulation test bench for Southeast University;
[0051] Fig.10 Identify the accuracy and loss curves for the DDS dataset;
[0052] Fig.11 It is the confusion matrix of DDS dataset;
[0053] Fig.12 This is the t-SNE graph of the DDS dataset;
[0054] Fig.13 Implantation of a comprehensive test platform for rotating machinery faults;
[0055] Fig.14 For normal gear and faulty gear;
[0056] Fig.15 This is the accuracy and loss curve for the Northeast Forestry University dataset;
[0057] Fig.16 This is the t-SNE diagram of the Northeast Forestry University data set;
[0058] Fig.17 This is the confusion matrix of the Northeast Forestry University dataset; DETAILED DESCRIPTION
[0059] This method uses the Convformer-SENet algorithm to integrate the channel attention mechanism, which can effectively extract multi-scale visual images and significantly enhance the mapping relationship between feature channels. This method is more refined in feature representation, effectively suppresses redundant features, and can more accurately focus on key positions and channels in gear vibration signals:
[0060] The main computational process of the Convformer model includes processing the input features by the convolution module, calculating the query, key and value matrices, applying the self-attention mechanism to obtain the attention weights, concatenating and normalizing the outputs of multiple attention heads, and finally further processing the features through convolutional projection.
[0061] The main calculation process of the Convformer model is as follows:
[0062] Formula 1: Given input feature X i , through the convolution module to obtain new features
[0063]
[0064] Formula 2: The feature Multiply by the query weight matrix Then we get the query matrix Q. j is the key matrix;
[0065]
[0066] Formula 3, 4: Use one-dimensional convolutional neural network to Processed and multiplied by the key weight matrix Sum value matrix V j ;
[0067]
[0068]
[0069] Formula 5: Apply the SoftMax function to the result to obtain the attention weight Z j . Use these weights to value the matrix V j Perform weighted summation;
[0070]
[0071] Formula 6, 7, 8: Using convolutional projection instead of linear projection in the feedforward neural network further reduces the number of parameters of Convformer;
[0072] Z=Concat(Z1,Z2...,Z h )W O ;
[0073]
[0074]
[0075] SENet mainly consists of three steps, namely, squeezing operation, excitation operation and feature reweighting operation:
[0076] Squeeze operation: Obtain the global features of each channel through global average pooling. Assume that the input feature map is U∈R H×W×C , where H, W, and C are the height, width, and number of channels of the feature map, respectively. The Squeeze operation compresses the spatial information of each channel into a global description through global average pooling:
[0077]
[0078] Excitation operation: The excitation operation adaptively recalibrates the importance of each channel. First, two fully connected layers are used to capture the nonlinear relationship between channels, and the weight of each channel is obtained through the activation function (usually ReLU and Sigmoid). The formula is as follows:
[0079] s=σ(W2δ(W1z))
[0080] In this formula, z=[z1,z2,...,z C ] T is the output of the squeeze operation, and are the weight matrices of the two fully connected layers, δ is the ReLU activation function, σ is the Sigmoid activation function, and r is a scaling factor used to reduce the number of parameters and computations;
[0081] Feature reweighting operation: The feature reweighting operation reweights the weight S generated by the Excitation operation to each channel of the original feature map. The formula is as follows:
[0082]
[0083] Among them, s c is the weight of the cth channel, U c is the cth channel of the input feature map, is the reweighted c-th channel feature;
[0084] Implementation verification example 1:
[0085] The data collected by Southeast University through the Drivertrain Dynamics Simulator (DDS) was used for experimental verification.
[0086] The test platform mainly consists of six parts, namely motor controller, motor, planetary gearbox, reduction gearbox, load and load controller.
[0087] The five types of gear faults collected by the test platform are healthy gear (HEA), tooth root crack fault (TCF), missing tooth fault (MTF), broken tooth fault (BTF) and gear surface wear fault (SWF).
[0088] The data set contains two working conditions, and the speed load configuration is set to 20-0 and 30-2. This experiment uses the vibration signals of the x-axis and y-axis directions of the planetary gearbox with a speed of 1200r / min and a load of 0V. The fault types are artificially introduced at the gear bearing position to simulate the fault conditions in the real working environment;
[0089] In this experiment, 1024 sampling points were selected for each sample, 1023 samples were selected for each fault type, and the model was trained using 150 epochs;
[0090] In order to highlight the advantages of the proposed method, this paper selects commonly used methods in the field of fault diagnosis and conducts a series of experiments to evaluate their performance. ;
[0091] These methods include: Method 1: UniFormer, Method 2: CNN-CBAM, Method 3: ResNet50. This paper uses diagnostic accuracy as the evaluation indicator. The experimental results are shown in Table 1.
[0092]
[0093] As can be seen from Table 1, in the experiment, the Convformer-SENet network model proposed in this paper outperforms the comparison method in terms of diagnostic accuracy, with the highest accuracy reaching 100%, which shows that this method has higher accuracy in gear fault diagnosis tasks.
[0094] By analyzing the recognition accuracy and loss curve of the DDS data set, it can be seen that when the model is trained 70 times, the recognition accuracy of the training set and the test has reached 100%, the Loss function value has also tended to be stable, and the final model has reached a stable convergence state, which shows that the method proposed in this paper has a good recognition effect on gear faults.
[0095] By analyzing the confusion matrix, we can see that Convformer-SENet performs best in the classification task, with a prediction accuracy of 100% and all labels are correctly predicted. UniFormer has an accuracy of 98% and 2% misclassification on label 4. CNN-CBAM has an accuracy of 97% and confusion between label 1 and label 2, with 10% and 9% misclassification respectively. ResNet50 performs relatively poorly, with an accuracy of 95% and more obvious confusion between label 2 and label 3, with 14% of label 2 being misclassified as label 3, 10% of label 3 being misclassified as label 2, and 1% of label 4 being misclassified as label 3.
[0096] In order to intuitively demonstrate the superiority of the Convformer-SENet model, the t-distributed stochastic neighbor embedding (t-SNE) method is used to visualize the target domain output data processed by the Convformer-SENet model;
[0097] The dimensionality reduction visualization results show the category distribution of each model in the feature space. It can be seen that the feature distribution of Convformer-SENet is dispersed and highly discriminative, and the clustering of each category is clear, showing that the model has good feature extraction ability. The feature distribution of UniFormer is mostly clear, but there is overlap in label 4;
[0098] The feature distribution of CNN-CBAM is relatively scattered, but there is overlap between label 1 and label 2. The feature distribution of ResNet50 has more overlap, especially between the green (label 2) and red (label 3) categories, which shows that the model has weaker ability to distinguish samples in this fault category. This shows that the method of this method has achieved good results and has a high accuracy rate.
[0099] Implementation verification example 2:
[0100] In order to further verify the effectiveness of the Convformer-SENet model, a gear fault dataset was collected using the comprehensive test platform for rotating machinery fault implantation at Northeast Forestry University;
[0101] The data set contains a total of 6 types, namely normal, missing teeth, spalling, pitting, cracks, and wear.
[0102] This experiment selected gear data with a load of 0.5A, a speed of 35hz (1050r / min), and a sampling frequency of 10kHZ. This experiment randomly selected 6 groups of data from 6 types for experimental verification:
[0103] The classification accuracy and loss values of the network model training and testing on the gear fault dataset of Northeast Forestry University show the changes in loss and accuracy during training and testing. It can be seen that with the increase in the number of training rounds, the accuracy of the model is close to 100% after about 60 rounds of training, and the loss value tends to be stable. At this time, the Convformer-SENet model has reached a good convergence state and has good generalization ability.
[0104] The experimental results show that the Convformer-SENet model proposed in this paper can accurately identify different gear faults. The same comparative experiment as DDS was conducted on the laboratory data set, and the t-SNE algorithm was used to visualize the features of the model's classification process. It can be seen that after classification by the Convformer-SENet network, the model performed well in terms of classification accuracy and no classification errors occurred. This further proves the superiority of the proposed method.
[0105] The test set data is input into the model for classification, and the confusion matrix is obtained. It can be seen that the accuracy of the Convformer-SENet model reaches 100%, which is significantly better than the other three methods, demonstrating the excellent performance of Convformer-SENet in the field of gear fault diagnosis;
[0106] In summary, in this method, Convformer combines the advantages of CNN and Transformer, taking into account both local and global features, and embedding SENet channel attention mechanism to achieve adaptive learning, which can effectively improve the accuracy of gear fault diagnosis. In two sets of verification experiments, the recognition accuracy of the Convformer-SENet model reached 100% after 150 and 70 rounds of training respectively, and the fault recognition accuracy was better than other advanced models. The method proposed in this paper has high effectiveness and robustness in the field of gear fault diagnosis.
Claims
1. The gear fault diagnosis method based on Convformer-SENet is characterized by being able to effectively extract multi-scale visual images and significantly enhance the mapping relationship between feature channels. This method is more refined in feature representation, effectively suppresses redundant features, and can more accurately focus on key positions and channels in gear vibration signals. The Convformer model mainly includes the following calculation process: Step 1: includes the processing of input features by the convolution module. Step 2: Calculate the query, key and value matrices, Step 3: Apply self-attention mechanism to obtain attention weights, Step 4: Concatenate and normalize the multi-head attention outputs. Step 5: Finally, the features are further processed through convolution projection; SENet mainly consists of three steps, namely, the squeezing operation, the excitation operation, and the feature reweighting operation.
2. According to the gear fault diagnosis method based on Convformer-SENet in claim 1, the step 1 includes the following contents and formulas: Given input feature X i , through the convolution module to obtain new features 3. According to the gear fault diagnosis method based on Convformer-SENet described in claim 2, the calculation query, key and value matrix of step 2 contains the following content and formula: The characteristics Multiply by the query weight matrix Then we get the query matrix Q. j is the key matrix; 4. The gear fault diagnosis method based on Convformer-SENet according to claim 3 is characterized in that: The content and formula of applying the self-attention mechanism in step 3 to obtain the attention weight are as follows: Using a one-dimensional convolutional neural network Processed and multiplied by the key weight matrix Sum value matrix V j 5. The gear fault diagnosis method based on Convformer-SENet according to claim 4 is characterized in that: The contents and formulas of the splicing and normalization of the multi-head attention output in step 4 are as follows: Apply the SoftMax function to the result to get the attention weight Z j . Use these weights to value the matrix V j Perform weighted summation; 6. The gear fault diagnosis method based on Convformer-SENet according to claim 5 is characterized in that: In step 5, the features are further processed by convolution projection, and the formula is as follows: Use convolutional projection instead of linear projection in feedforward neural networks to further reduce the number of Convformer parameters; Z=Concat(Z1,Z2...,Z h )IN O 7. The gear fault diagnosis method based on Convformer-SENet according to claim 6 is characterized in that: SENet mainly consists of three steps, namely, squeezing operation, excitation operation and feature reweighting operation: The squeezing operation is to obtain the global features of each channel through global average pooling. Assume that the input feature map is U∈R H×W×C , where H, W, and C are the height, width, and number of channels of the feature map, respectively. The squeeze operation compresses the spatial information of each channel into a global description through global average pooling; The application formula is as follows:
8. The gear fault diagnosis method based on Convformer-SENet according to claim 7 is characterized in that: The SENet mainly consists of three steps, namely, squeezing operation, excitation operation and feature reweighting operation: Among them, the excitation operation: The excitation operation adaptively recalibrates the importance of each channel. First, two fully connected layers are used to capture the nonlinear relationship between channels, and the weight of each channel is obtained through the activation function (usually ReLU and Sigmoid). The application formula is as follows: s=σ(W2δ(W1z)) In this formula, z=[z1,z2,...,z C ] T is the output of the squeeze operation, and are the weight matrices of the two fully connected layers, δ is the ReLU activation function, σ is the Sigmoid activation function, and r is a scaling factor used to reduce the amount of parameters and computation.
9. The gear fault diagnosis method based on Convformer-SENet according to claim 8, characterized in that: The SENet mainly consists of three steps, namely, squeezing operation, excitation operation and feature reweighting operation: Among them, feature reweighting operation: The feature reweighting operation reweights the weight S generated by the Excitation operation to each channel of the original feature map. The application formula is as follows: Among them, s c is the weight of the cth channel, U c is the cth channel of the input feature map, is the reweighted c-th channel feature.
10. According to claims 1-9, a gear fault diagnosis method based on Convformer-SENet is characterized by combining the powerful local feature extraction capability of CNN and the advantage of Transformer that can capture the dependency between any two positions in the input sequence through the self-attention mechanism. Taking into account both local and global features, while embedding the SENet channel attention mechanism to achieve adaptive learning, the mapping relationship between feature channels is significantly enhanced. It is more refined in feature representation, effectively suppresses redundant features, and can more accurately focus on key positions and channels in gear vibration signals.