Bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network

Abstract

The invention relates to the technical field of intelligent fault diagnosis, in particular to a bearing fault diagnosis method based on multi-modal fusion and a multi-scale residual network, and the method comprises the steps: building a subway train traction motor bearing experiment platform, collecting the multi-modal data of a motor bearing, carrying out the preprocessing of the multi-modal data based on principal component analysis, and carrying out the fault diagnosis of the motor bearing. A data fusion method is constructed to convert collected multi-modal data into RGB images to serve as input of a model, meanwhile, a multi-scale noise reduction module is constructed, the network is helped to explore multi-scale features and filter out irrelevant information, and the adaptability of a residual network in a noise scene is improved; a dual-scale residual block is constructed, deep and shallow layer features are learned on different scales, and bearing fault information of different spatial dimensions is captured. Building a multi-scale residual network, carrying out the training and verification through the preprocessed multi-modal data, and carrying out the evaluation of a bearing multi-modal sample through a trained model, and obtaining a fault classification result of the subway train traction motor bearing.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent fault diagnosis, and in particular to a bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network. Background Art

[0002] With the rapid expansion of global rail transit, subway trains, as efficient and environmentally friendly public transportation, have become an important choice for cities to solve traffic congestion. Since trains run at high speeds and under heavy load conditions, the service life and operating status of traction motor bearings are crucial to the safety of trains. However, with the passage of time, the risk of failures such as wear, cracks, and even fractures of traction motor bearings has increased significantly. These failures not only threaten the normal operation of trains, but also lead to an increase in the frequency of train maintenance and an increase in maintenance costs. Therefore, it is of great significance to study intelligent diagnostic methods that can warn of failures in advance and improve operational safety.

[0003] At present, the bearing fault diagnosis method is mainly based on vibration signals, but the collection of a single vibration signal has the problem of insufficient information. In addition, subway trains are subject to greater noise interference in complex operating environments, such as tunnels and urban areas, which limits the traditional intelligent diagnosis model based on vibration data in extracting fault features.

[0004] In recent years, deep learning (DL) technology has been widely used in fault diagnosis of mechanical equipment, especially residual network (ResNet) because it can solve the gradient vanishing problem of deep models and has become a common method in fault diagnosis models. However, in a strong noise environment, it is difficult for DL ​​models to effectively extract bearing fault features from complex multimodal data. Summary of the invention

[0005] The purpose of the present invention is to provide a bearing fault diagnosis method based on multimodal fusion and multiscale residual network, aiming to realize the classification of the health status of train motor bearings according to the characteristics of multimodal data and the advantages of MF-MSRNet model in a strong noise environment.

[0006] To achieve the above object, the present invention provides a bearing fault diagnosis method based on multimodal fusion and multiscale residual network, comprising the following steps:

[0007] Step 1: Set up a subway train traction motor bearing experimental platform to obtain multi-modal data of the motor bearing;

[0008] Step 2: Preprocess the multimodal data based on principal component analysis, build a data fusion method, and divide it into training set, validation set, and test set;

[0009] Step 3: Construct a multi-scale noise reduction module;

[0010] Step 4: Construct a dual-scale residual block;

[0011] Step 5: Build a multi-scale residual network;

[0012] Step 6: Train and verify the multi-scale residual network based on the training set, validation set and test set, and finally use the trained model to evaluate the multi-modal samples of the bearing to obtain the fault classification results.

[0013] Optionally, the subway train traction motor bearing experimental platform in step 1 is used to simulate the working conditions of the subway train under uniform speed driving and the corresponding bearing fault types. The multimodal data is obtained by sampling different types of sensors, and each signal has the same sampling point.

[0014] Optionally, during the execution of step 2, the collected multimodal data is converted into an RGB image as the input of the model. The specific process is to use principal component analysis (PCA) to preprocess the multimodal original signal for dimensionality reduction, retain the first three principal components and convert them into three-channel time domain signals, convert the data samples of the three channels into a two-dimensional matrix, normalize their pixel values, and finally fuse the three two-dimensional matrices into an RGB image.

[0015] Optionally, the multi-scale denoising module includes an expanded depthwise convolution, a multi-scale module and a soft denoising module, wherein the multi-scale module evenly divides the obtained feature map into 4 subsets in the channel direction, presents multi-scale features through a hierarchical structure, and expands the receptive field of the convolutional neural network. The soft denoising module uses a squeeze excitation network and a soft threshold function to automatically learn and identify features and eliminate redundant information.

[0016] Optionally, the dual-scale residual block is used to extract deep and shallow features of the input vector and retain local and global feature information.

[0017] Optionally, the process of performing bearing fault diagnosis using the multimodal fusion and multiscale residual network includes the following steps:

[0018] Step 5.1: The RGB image is first passed through the first convolutional layer for preliminary feature extraction;

[0019] Step 5.2: Use the global average pooling layer to reduce the dimension of the features and achieve translation invariance;

[0020] Step 5.3: Use a fully connected layer to convert the pooled features into a one-dimensional vector, and use the Softmax function to classify the bearing health status.

[0021] Optionally, in step 6, indicators are used to evaluate the fault diagnosis performance of the model, and the indicators include average test accuracy and average test F1-score.

[0022] The present invention provides a bearing fault diagnosis method based on multimodal fusion and multiscale residual network. By setting up a subway train traction motor bearing experimental platform, multimodal data of motor bearings are collected, and then the multimodal data is preprocessed based on principal component analysis. A data fusion method is constructed to convert the collected multimodal data into RGB images as the input of the model. At the same time, a multiscale noise reduction module is constructed to help the network explore multiscale features and filter out irrelevant information, thereby improving the adaptability of the residual network in noise scenes; a dual-scale residual block is constructed to learn deep and shallow features at different scales and capture bearing fault information of different spatial dimensions. Then, a multimodal fusion and multiscale residual network is constructed, and the preprocessed multimodal data is used for training and verification. Finally, the trained model is used to evaluate the multimodal samples of the bearings to obtain the fault classification results of the subway train traction motor bearings. The present invention has high robustness and adaptability, can be applied to the real-time monitoring system of subway trains, and is of great significance to improving the safety of equipment operation and maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 It is a schematic flow chart of the steps of a bearing fault diagnosis method based on multimodal fusion and multi-scale residual network of the present invention.

[0025] Figure 2 It is a flow chart of multimodal data fusion of the present invention.

[0026] Figure 3 It is an RGB image of different bearing fault types of a subway train traction motor according to a specific embodiment of the present invention.

[0027] Figure 4 It is a structural diagram of the multi-scale noise reduction module of the present invention.

[0028] Figure 5 is a structural diagram of the dual-scale residual block of the present invention.

[0029] Figure 6 It is a structural diagram of the multi-scale residual network of the present invention.

[0030] Figure 7 It is a comparison chart of the accuracy and loss of each model in the specific embodiment of the present invention during the training process under the G1 working condition.

[0031] Figure 8 It is the confusion matrix of MSRNet of the specific embodiment of the present invention under four working conditions.

[0032] Fig. 9 It is a comparison chart of t-SNE visualization results of various models in the specific embodiments of the present invention under the G1 working condition.

[0033] Fig.10 It is a comparison chart of the diagnostic accuracy of each model under different signal-to-noise ratio conditions in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0034] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0035] See also Figure 1 The present invention provides a bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network, comprising the following steps:

[0036] S1: Set up a subway train traction motor bearing experimental platform to obtain multi-modal data of motor bearings;

[0037] S2: Preprocess the multimodal data based on principal component analysis, build a data fusion method, and divide the training set, validation set, and test set;

[0038] S3: construct a multi-scale noise reduction module;

[0039] S4: construct a dual-scale residual block;

[0040] S5: Build a multi-scale residual network;

[0041] S6: The multi-scale residual network is trained and verified based on the training set, the validation set and the test set. Finally, the trained model is used to evaluate the multi-modal samples of the bearing to obtain the fault classification results.

[0042] The following is further described in conjunction with specific embodiments and execution steps:

[0043] The subway train traction motor bearing experimental platform set up in step S1 is composed of a motor, a brake, a test bearing, a control system, a cooling system, a vibration acceleration sensor, a sound print sensor, an acoustic emission sensor, a signal acquisition device and a control platform, etc. It is used to simulate the working conditions of a subway train under uniform speed driving and the corresponding bearing fault types, corresponding to 8 types of bearing faults, namely, normal bearing, inner ring fault, outer ring fault, rolling element fault, inner-outer ring fault, inner ring-roller fault, outer ring-roller fault and inner-outer ring-roller fault; the multimodal data is obtained by sampling different types of sensors, and each signal has the same sampling point.

[0044] The bearings used in the experimental platform are cylindrical roller bearings (SKF NU216), which are exactly the same as the bearing models used in the traction motors of trains on a certain city's rail transit lines.

[0045] The bearing crack fault was simulated by wire cutting, with a crack width of 0.4 mm and a depth of 0.1 mm.

[0046] In step S2, a multimodal fusion method based on principal component analysis (PCA) is proposed to convert the collected multimodal data such as vibration, voiceprint, and acoustic emission into RGB images as the input of the model to enhance the expression ability of fault information, such as Figure 2 As shown, the steps include:

[0047] S21 Specific, multi-modal data original signal X raw It can be expressed as:

[0048] X raw ={x ij ,i=1,2,…u; j=1,2,…v} (1)

[0049] Where, X raw is the signal data matrix, u is the number of sensors, and v is the number of sampling points.

[0050] S22 uses PCA to preprocess the original signal of multimodal data in order to reduce the interference of invalid information in massive data. The PCA algorithm can keep the local manifold structure of the data sample unchanged and effectively reduce the dimension of the nonlinear data of the bearing. The data after PCA dimension reduction can be expressed as:

[0051] X pca =A T X raw ={x ij ,i=1,2,3; j=1,2,...v} (2)

[0052] Where, X pca is a three-channel time domain signal, A T Is with X raw A matrix of the three salient features of .

[0053] S23 randomly divides the time domain signals of the three channels and generates signal samples of different fault types. The size of each sample is S t = k×k×3, where k×k∈[4,v] is the size of the RGB image. The segmentation points of each sample are randomly selected without overlap and uniformly distributed as [1,vk×k]. Then, signal samples of length k×k are generated according to the segmentation points.

[0054] S24 converts the three-channel signal samples into a pixel matrix (PM) and normalizes the pixel values. The signal segment is converted into a two-dimensional matrix by means of texture grayscale image. The pixel matrix PM of each channel i (m,n) can be calculated by the following formula:

[0055]

[0056] Among them, the signal value N is the total number of samples, h=1,2,…,k 2 , i represents the number of channels of the data after dimensionality reduction.

[0057] S25 fuses the three gray image pixel matrices into an RGB image for input into the model, see Figure 3 , the RGB image can be represented as:

[0058] 3D-Matrix(m,n)=(2D-Matrix(m,n,1),2D-Matrix(m,n,2),2D-Matrix(m,n,3))(4)

[0059] According to the experimental requirements, the obtained RGB image 3D-Matrix (m, n) is randomly divided into training set, validation set and test set in a 7:1:2 manner to provide effective data preprocessing means for the subsequent process.

[0060] Step S3 constructs a multi-scale noise reduction module (MDM), see Figure 4 , helps the network explore multi-scale features and filter out irrelevant information, improving the adaptability of the residual network in noisy scenes;

[0061] Specifically, without involving cross-channel information fusion, an expanded depthwise convolution with a kernel size of 3×3 is used to extract fault-related features in each channel. Then, a multi-scale module (MM) and a soft denoising module (SDM) are used to explore the multi-scale features of the data and filter out irrelevant information. MM aims to present multi-scale features through a hierarchical structure and expand the receptive field of the convolutional neural network. First, after the input feature X undergoes a 1×1 convolution, the resulting feature map x is evenly divided into 4 subsets in the channel direction, denoted as x i ,i=1,2,3,4. Except for x 1 , perform 3×3 convolution on other sub-feature maps to obtain a larger receptive field, and use K i () indicates. Using y i K i The output of () can be expressed as:

[0062]

[0063] The output of MM can be expressed as:

[0064] y=Concat(y i ),i=1,2,3,4 (6)

[0065] In the formula, Concat(·) represents a concatenation operator. In order to better fuse information of different scales, the present invention uses a 1×1 convolution layer to adaptively fuse the cascade features, and uses a residual connection mechanism to promote the parameter training of MM. The formula is:

[0066] M=Conv 1×1 (y)+X (7)

[0067] In order to eliminate the noise characteristics in the original signal, the soft threshold function is used as a nonlinear transformation layer to eliminate redundant information. The soft thresholding process can be expressed as:

[0068]

[0069] In the formula, a and f(a) represent the input and output feature vectors respectively, and t represents the threshold. However, how to set the threshold range t is a challenge. This paper adopts the SDM method to solve this problem.

[0070] SDM uses a squeeze excitation network and a soft threshold function to automatically learn and identify features and eliminate redundant information. First, the present invention performs a squeeze operation on the input feature M to obtain the global information of the input feature:

[0071] Q v = GAP(abs(M)) (9)

[0072] Where abs(·) represents the absolute function and GAP(·) represents the global average pooling. Then, an excitation operation is performed to highlight the channel information:

[0073]

[0074] In the formula, FC(·) represents the fully connected layer, and then the present invention uses the Sigmoid function to convert the output Scaling to the range [0,1]:

[0075]

[0076] Where β represents the output of SE operation. The threshold t can be obtained by the following formula:

[0077] t=β·Q v (12)

[0078] By integrating the Squeeze-and-Excitation Networks (SE) module and the soft threshold function, SDM can effectively learn important features and remove redundant information. After determining the threshold vector, SDM can automatically eliminate interference information in the noise signal M:

[0079] O'=SDM(M) (13)

[0080] Finally, the present invention adopts a residual connection strategy to promote the feature transfer of SDM, and the output O of DM can be expressed as:

[0081] O=O'+M (14)

[0082] In step S4, a dual-scale residual block is constructed, and the structure is shown in FIG. Figure 5 , respectively learn deep and shallow features at different scales and capture bearing fault information in different spatial dimensions to enhance the ability to extract fusion features;

[0083] S41 First, the feature map is processed by a convolution layer with a convolution kernel size of 1×1 to increase its dimension, and the number of feature channels is expanded to twice the original.

[0084] S42 divides it into two groups of features with the same channel, one of which is input into the traditional Basicblock to capture the deep relationship of features and enhance the feature extraction capability. The other group of features is input into the convolution layer with a convolution kernel size of 1×1 to capture the shallow relationship of features.

[0085] S43 performs channel fusion on the two sets of features and performs dimensionality reduction operation through a convolution layer with a convolution kernel size of 1×1 to enhance the expressiveness of features at different scales. In order to improve the feature preservation capability, residual connections are performed. Unlike the dual-scale residual block DSRB1, the DSRB2 module uses a convolution layer with a convolution kernel size of 1×1 to increase the dimension to keep the channels consistent. In addition, the LeakyReLU activation function is used to speed up training and avoid overfitting problems.

[0086] Step S5 constructs a multi-modal fusion and multi-scale residual network (MF-MSRNet) and proposes a MF-MSRNet bearing fault diagnosis method. Figure 6 ,This method involves the changes in data distribution in different fields and can maintain a consistent label space in each field. The process is as follows:

[0087] S51 Specifically, the RGB image is firstly extracted through the first convolutional layer. The MSRNet model uses DSRB as the core learning algorithm and combines the DSRB module with the multi-scale denoising module (MDM). The DSRB module can enhance the ability to extract features of different scales and retain local and global feature information; MDM helps to further explore multi-level features and filter out irrelevant information.

[0088] S52 uses a global average pooling layer to reduce the dimension of features and achieve translation invariance.

[0089] S53 uses a fully connected layer to convert the pooled features into a one-dimensional vector and uses the Softmax function to classify the bearing health status.

[0090] Step S6 is a training verification process, which includes the following steps:

[0091] S61, based on the training set, the validation set, and the test set, using a multimodal dataset to evaluate the MF-MSRNet model;

[0092] S62 verifies the performance of the MF-MSRNet model based on a comparative method.

[0093] Furthermore, the present invention is based on the following embodiments and verifies the performance of the model of the present invention through comparison methods such as MF-InResNet, MF-ResNet18, MF-PResNet, MF-Res2Net, MF-MobileNetV2, MF-ShuffleNetV2 and ResNet50.

[0094] Specifically, the IEPE acquisition instrument is used to collect vibration signals of three channels. The acceleration sensors are installed on the bearing seat of the bearing to be tested (in the x, y and z directions respectively), the sound print sensors are installed at the top and front end of the bearing seat (in the y and z directions respectively), and the acoustic emission sensor is installed at the top of the bearing seat of the bearing to be tested (in the z direction). The sampling frequency of the three sensors is set to 20kHz. A total of 4 working conditions are set according to different speeds, and the constant speeds are 900rpm (G1), 1500rpm (G2), 2100rpm (G3) and 2700rpm (G4), and the sampling time of each experiment is set to 60s.

[0095] Based on the above data, the method of the present invention and the comparative method were used to evaluate the fault diagnosis performance in a strong noise environment. Figure 7 It is a comparison of the accuracy and loss of each model during the training process under the G1 condition.

[0096] Figure 7 It can be seen from the results shown that the MSRNet model proposed in the present invention has the fastest convergence speed, with an accuracy rate of more than 90% after the third training cycle, and the training process has reached stability after the ninth training cycle. The ResNet18 curve rises smoothly, and its diagnostic performance is second only to the MSRNet model. The training curves of Res2Net, MobileNetV2, InResNet, and ResNet50 are relatively close, with slow convergence speed and low overall accuracy. The diagnostic curve of PResNet fluctuates greatly; and the diagnostic performance of the ShuffleNetV2 model is the worst. In summary, compared with other models, the MSRNet model has obvious advantages in accuracy and convergence speed.

[0097] The confusion matrix provides a clear measure of how accurately each fault type is classified. Figure 8 The confusion matrices of MSRNet under four working conditions are listed, and the study finds that MSRNet can identify eight types of train motor bearing faults with high accuracy.

[0098] In order to describe the experimental results more clearly, the experimental results are visualized. Fig. 9 , by observing Fig. 9It can be easily seen that the MF-MSRNet fault diagnosis model has a strong feature extraction capability and can effectively distinguish the characteristics of eight types of faults, which is better than other comparison models. Therefore, the availability and effectiveness of the proposed method are verified by the four working condition data of the train traction motor bearing.

[0099] In order to evaluate the robustness of the model under different noise environments, a train bearing data set with a signal-to-noise ratio (SNR) ranging from -10 to 0 dB was generated, and different models were experimented under various SNR conditions. The diagnostic accuracy is shown in Fig.10 . Under the condition of -10dB SNR, the accuracy of the proposed model dropped to 77.68%, but it was still better than other models. The accuracy of MF-InResNet, MF-ResNet18 and MF-PResNet were 64.37%, 65.62% and 55.12% respectively, while the accuracy of MF-Res2Net, MF-MobileNetV2, MF-ShuffleNetV2 and ResNet50 were all lower than 34%. The comparison of the above experimental results verified the effectiveness and feasibility of the MF-MSRNet method proposed in this paper.

[0100] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A bearing fault diagnosis method based on multimodal fusion and multiscale residual network, characterized in that: The following steps are involved: Step 1: Set up a subway train traction motor bearing experimental platform to obtain multi-modal data of the motor bearing; Step 2: Preprocess the multimodal data based on principal component analysis, build a data fusion method, and divide it into training set, validation set, and test set; Step 3: Construct a multi-scale denoising module; Step 4: Construct a dual-scale residual block; Step 5: Build a multi-scale residual network; Step 6: Train and verify the multi-scale residual network based on the training set, validation set and test set, and finally use the trained model to evaluate the multi-modal samples of the bearing to obtain the fault classification results.

2. The bearing fault diagnosis method based on multimodal fusion and multiscale residual network according to claim 1 is characterized in that: The subway train traction motor bearing experimental platform in step 1 is used to simulate the working conditions of the subway train under uniform speed driving and the corresponding bearing fault types. The multimodal data is obtained by sampling different types of sensors, and each signal has the same sampling point.

3. The bearing fault diagnosis method based on multimodal fusion and multiscale residual network according to claim 2 is characterized in that: During the execution of step 2, the collected multimodal data is converted into an RGB image as the input of the model. The specific process is to use the principal component analysis method to preprocess the multimodal original signal for dimensionality reduction, retain the first three principal components and convert them into three-channel time domain signals, convert the data samples of the three channels into a two-dimensional matrix, and normalize their pixel values. Finally, the three two-dimensional matrices are fused into an RGB image.

4. The bearing fault diagnosis method based on multimodal fusion and multiscale residual network according to claim 3 is characterized in that: The multi-scale denoising module includes an expanded depthwise convolution, a multi-scale module and a soft denoising module, wherein the multi-scale module evenly divides the obtained feature map into four subsets in the channel direction, presents multi-scale features through a hierarchical structure, and expands the receptive field of the convolutional neural network. The soft denoising module adopts a squeeze excitation network and a soft threshold function to automatically learn and identify features and eliminate redundant information.

5. The bearing fault diagnosis method based on multimodal fusion and multiscale residual network according to claim 4 is characterized in that: The dual-scale residual block is used to extract deep and shallow features of the input vector and retain local and global feature information.

6. The bearing fault diagnosis method based on multimodal fusion and multiscale residual network according to claim 5 is characterized in that: The process of bearing fault diagnosis using multimodal fusion and multiscale residual network includes the following steps: Step 5.1: The RGB image is first passed through the first convolutional layer for preliminary feature extraction; Step 5.2: Use the global average pooling layer to reduce the dimension of the features and achieve translation invariance; Step 5.3: Use a fully connected layer to convert the pooled features into a one-dimensional vector, and use the Softmax function to classify the bearing health status.

7. The bearing fault diagnosis method based on multimodal fusion and multiscale residual network according to claim 6 is characterized in that: In step 6, indicators are used to evaluate the fault diagnosis performance of the model, and the indicators include average test accuracy and average test F1-score.

Images