Bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network

Abstract

The present application relates to the technical field of intelligent fault diagnosis, and particularly relates to a bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network, a metro train traction motor bearing experiment platform is set up, multi-modal data of the motor bearing is collected, the multi-modal data is preprocessed based on principal component analysis, a data fusion method is constructed to convert the collected multi-modal data into an RGB image as the input of the model, a multi-scale noise reduction module is constructed to help the network explore multi-scale features and filter out irrelevant information, and the adaptability of the residual network in a noise scene is improved; a double-scale residual block is constructed to learn deep and shallow features respectively at different scales and capture bearing fault information of different spatial dimensions. Then a multi-scale residual network is built, the preprocessed multi-modal data is used for training and verification, and finally the trained model is used to evaluate the bearing multi-modal samples to obtain the fault classification result of the metro train traction motor bearing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent fault diagnosis, and particularly relates to a bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network. BACKGROUND

[0002] With the rapid expansion of global rail transit scale, subway trains as efficient and environmentally friendly public transportation tools have become an important choice for cities to solve traffic congestion. Since the train runs at high speed and heavy load, the service life and operating state of the traction motor bearing are crucial to the safety of the train. However, with the passage of time, the risk of wear, cracks and even breakage of the traction motor bearing increases significantly. These faults not only threaten the normal operation of the train, but also increase the frequency of train maintenance and maintenance costs. Therefore, it is of great significance to study an intelligent diagnosis method that can provide early warning of faults and improve the safety of operation.

[0003] Currently, bearing fault diagnosis methods are mainly based on vibration signals, but the collection of single vibration signals has the problem of insufficient information. In addition, subway trains are subjected to strong noise interference in complex operating environments such as tunnels and urban areas, which limits the traditional intelligent diagnosis model based on vibration data in extracting fault features.

[0004] In recent years, deep learning (DL) technology has been widely applied in the fault diagnosis of mechanical equipment, and in particular, residual network (ResNet) has become a common method in fault diagnosis models because it can solve the problem of gradient disappearance in deep models. However, in a strong noise environment, the DL model has difficulty in effectively extracting the fault features of the bearing from complex multi-modal data. SUMMARY

[0005] The purpose of the present application is to provide a bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network, aiming to classify the health state of the train motor bearing according to the characteristics of multi-modal data and the advantages of the MF-MSRNet model in a strong noise environment.

[0006] To achieve the above purpose, the present application provides a bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network, comprising the following steps:

[0007] Step 1: Set up a subway train traction motor bearing experiment platform to obtain multi-modal data of the motor bearing;

[0008] Step 2: Preprocess the multi-modal data based on principal component analysis, construct a data fusion method, and divide the training set, validation set and test set;

[0009] Step 3: Construct a multi-scale noise reduction module;

[0010] Step 4: constructing a double-scale residual block;

[0011] Step 5: building a multi-scale residual network;

[0012] Step 6: training and verifying the multi-scale residual network based on the training set, the verification set and the test set, finally using the trained model to evaluate the bearing multi-modal samples to obtain the fault classification result.

[0013] Optionally, the subway train traction motor bearing experiment platform in step 1 is used to simulate the working condition of the subway train in the uniform speed running state and the corresponding bearing fault type, and the multi-modal data is obtained by sampling through different types of sensors. Each signal has the same sampling point.

[0014] Optionally, during the execution of step 2, the collected multi-modal data is converted into an RGB image as the input of the model. The specific process is to use principal component analysis (PCA) to preprocess the multi-modal original signal for dimension reduction, retain the first three principal components and convert them into three-channel time-domain signals, convert the data samples of the three channels into two-dimensional matrices, and normalize the pixel values. Finally, the three two-dimensional matrices are fused into an RGB image.

[0015] Optionally, the multi-scale denoising module includes an inflation depthwise convolution, a multi-scale module and a soft denoising module. The multi-scale module uniformly divides the obtained feature map into 4 subsets in the channel direction, presents multi-scale features through a hierarchical structure, expands the receptive field of the convolutional neural network, and the soft denoising module adopts a squeeze-and-excitation network and a soft threshold function to automatically learn and identify features and eliminate redundant information.

[0016] Optionally, the double-scale residual block is used to extract deep and shallow features of the input vector and retain local and global feature information.

[0017] Optionally, the process of bearing fault diagnosis by the multi-modal fusion and multi-scale residual network includes the following steps:

[0018] Step 5.1: the RGB image is first subjected to a preliminary feature extraction through a first convolutional layer;

[0019] Step 5.2: a global average pooling layer is used to reduce the dimension of the features and realize the translational invariance characteristic;

[0020] Step 5.3: a fully connected layer is used to convert the pooled features into a one-dimensional vector, and a Softmax function is used to realize the classification of the bearing health state.

[0021] Optionally, the performance of the fault diagnosis model is evaluated in step 6 using indicators including average test accuracy and average test F1-score.

[0022] The application provides a bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network, an experimental platform of metro train traction motor bearing is built, multi-modal data of the motor bearing is collected, then the multi-modal data is pretreated based on principal component analysis, a data fusion method is constructed to convert the collected multi-modal data into RGB images as the input of the model, a multi-scale noise reduction module is constructed to help the network explore multi-scale features and filter out irrelevant information, and the adaptability of the residual network in a noise scene is improved; a double-scale residual block is constructed to learn deep and shallow features and capture bearing fault information of different spatial dimensions at different scales. Then a multi-modal fusion and multi-scale residual network is built, the pretreated multi-modal data is used for training and verification, finally the trained model is used to evaluate the bearing multi-modal samples, and the fault classification result of the metro train traction motor bearing is obtained. The application has high robustness and adaptability, can be applied to a real-time monitoring system of a metro train, and has important significance for improving equipment operation safety and maintenance efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0024] Figure 1 is a step flowchart of a bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network of the present application.

[0025] Figure 2 is a flowchart of multi-modal data fusion of the present application.

[0026] Figure 3 is an RGB image of different bearing fault types of a metro train traction motor in a specific embodiment of the present application.

[0027] Figure 4 is a structure diagram of a multi-scale noise reduction module of the present application.

[0028] Figure 5 is a structure diagram of a double-scale residual block of the present application.

[0029] Figure 6 is a structure diagram of a multi-scale residual network of the present application.

[0030] Figure 7 is a comparison chart of accuracy and loss of the training process of each model of the specific embodiment of the present application under G1 working condition.

[0031] Figure 8 is the confusion matrix of the MSRNet of the specific embodiment of the present application under four working conditions.

[0032] Figure 9 is a comparison chart of t-SNE visualization results of each model of the specific embodiment of the present application under G1 working condition.

[0033] Figure 10 is a comparison chart of diagnostic accuracy of each model of the specific embodiment of the present application under different signal-to-noise ratio conditions. DETAILED DESCRIPTION

[0034] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numbers represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0035] Please refer to Figure 1 , the present application provides a bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network, comprising the following steps:

[0036] S1: set up a subway train traction motor bearing experiment platform, and acquire multi-modal data of the motor bearing;

[0037] S2: pre-process the multi-modal data based on principal component analysis, construct a data fusion method, and divide the training set, the validation set and the test set;

[0038] S3: construct a multi-scale noise reduction module;

[0039] S4: construct a double-scale residual block;

[0040] S5: build a multi-scale residual network;

[0041] S6: train and verify the multi-scale residual network based on the training set, the validation set and the test set, and finally use the trained model to evaluate the bearing multi-modal samples to obtain the fault classification result.

[0042] The following is further described in combination with specific embodiments and execution steps:

[0043] The subway train traction motor bearing experiment platform set up in step S1 is composed of a motor, a brake, a test bearing, a control system, a cooling system, a vibration acceleration sensor, an acoustic fingerprint sensor, an acoustic emission sensor, a signal acquisition device and a control platform, etc. It is used to simulate the working condition of the subway train in uniform speed running state and the corresponding bearing fault types, which are normal bearing (Normal Bearing), inner ring fault (Inner Ring Fault), outer ring fault (Outter Ring Fault), roller fault (Roller Fault), inner-outter ring fault (Inner-Outter Ring Fault), inner ring-roller fault (Inner Ring-Roller Fault), outer ring-roller fault (Outter Ring-Roller Fault) and inner-outter ring-roller fault (Inner-Outter Ring-Roller Fault), etc. Eight kinds of bearing fault types. The multi-modal data is obtained by sampling through different types of sensors, and each signal has the same sampling points.

[0044] The bearing used in the experiment platform is a cylindrical roller bearing (SKF NU216), which is exactly the same as the bearing model used in the traction motor of the train of a certain urban rail transit line.

[0045] The simulated bearing crack fault is processed by wire cutting method, with a crack width of 0.4mm and a depth of 0.1mm.

[0046] In step S2, a multi-modal fusion method based on principal component analysis (PCA) is proposed, which converts the collected vibration, acoustic fingerprint, acoustic emission and other multi-modal data into RGB images as the input of the model, enhances the expression ability of fault information, and realizes the fault diagnosis of bearing as shown in Figure 2 As shown, it includes the following steps:

[0047] S21 Specifically, the multi-modal data original signal X raw can be expressed as:

[0048] X raw ={x ij ,i=1,2,…u;j=1,2,...v} (1)

[0049] In the formula, X raw is a signal data matrix, u is the number of sensors, and v is the number of sampling points.

[0050] S22In order to reduce the interference of invalid information in mass data, PCA is used to preprocess the original signal of multi-modal data. The PCA algorithm can keep the local manifold structure of data samples unchanged, and effectively reduce the dimension of the nonlinear data of the bearing. The data after PCA dimension reduction can be expressed as:

[0051] X pca =A T X raw ={x ij ,i=1,2,3;j=1,2,...v} (2)

[0052] In the formula, X pca is a three-channel time-domain signal, A T is a matrix with three significant features of X raw .

[0053] S23Randomly divide the time-domain signals of the three channels, and generate signal samples of different fault types. The size of each sample is S t =k×k×3, where k×k∈[4,v] is the size of the RGB image. The segmentation point of each sample is randomly selected, without overlap, and uniformly distributed in [1,v-k×k]. Then, according to the segmentation point, a signal sample with length k×k is generated.

[0054] S24Convert the three-channel signal sample into a pixel matrix (PM), and normalize the pixel value. The signal segment is converted into a two-dimensional matrix by the texture grayscale image method, and the pixel matrix PM i (m,n) of each channel can be calculated by the following formula:

[0055]

[0056] Where, the signal value N is the total number of samples, h=1,2,…,k 2 , i represents the number of channels of the data after dimension reduction.

[0057] S25Fuse the three grayscale image pixel matrices into an RGB image to input into the model, see Figure 3 , the RGB image can be expressed as:

[0058] 3D-Matrix(m,n)=(2D-Matrix(m,n,1),2D-Matrix(m,n,2),2D-Matrix(m,n,3))(4)

[0059] According to the experimental requirements, the obtained RGB images 3D-Matrix(m,n) are randomly divided into training set, validation set and test set in the ratio of 7:1:2, providing effective data preprocessing means for subsequent use.

[0060] Step S3 constructs a multi-scale denoising module (MDM), see Figure 4 , which helps the network to explore multi-scale features and filter out irrelevant information, improving the adaptability of the residual network in a noisy environment;

[0061] Specifically, without involving cross-channel information fusion, an inflation depthwise convolution with a kernel size of 3x3 is used to extract fault-related features in each channel. Then, a multi-scale module (MM) and a soft denoising module (SDM) are used to explore the multi-scale features of the data and filter out irrelevant information. MM aims to present multi-scale features through a hierarchical structure and expand the receptive field of the convolutional neural network. First, after the input feature X undergoes a 1x1 convolution, the obtained feature map x is evenly divided into 4 subsets in the channel direction, denoted as x i ,i=1,2,3,4. Except for x1, 3x3 convolution is performed on the other sub-feature maps to obtain a larger receptive field, denoted as K i () represents. The output of K i () can be represented as: i

[0062]

[0063] The output of MM can be represented as:

[0064] y=Concat(y i ),i=1,2,3,4 (6)

[0065] where Concat(·) represents the concatenation operator. In order to better fuse information of different scales, the invention adopts a 1x1 convolution layer to adaptively fuse the concatenated features, and at the same time adopts a residual connection mechanism to facilitate the parameter training of MM, and its formula is:

[0066] M=Conv 1×1 (y)+X (7)

[0067] In order to eliminate the noise characteristics in the original signal, a soft threshold function is used as a nonlinear conversion layer to eliminate redundant information. The soft thresholding process can be represented as:

[0068]

[0069] where a and f(a) represent the input and output feature vectors respectively, and t represents the threshold. However, how to set the threshold range t is a challenge, and the SDM method is used to solve this problem.

[0070] ​The SDM adopts a squeeze excitation network and a soft threshold function, and automatically learns and identifies features and eliminates redundant information.

[0071] Q v =GAP(abs(M)) (9)

[0072] In the formula, abs(·) represents an absolute function, and GAP(·) represents a global average pooling. Then, an excitation operation is performed to highlight channel information:

[0073]

[0074] In the formula, FC(·) represents a full connection layer, and then the output of the present application is scaled to the range of [0, 1] by using a Sigmoid function:

[0075]

[0076] In the formula, β represents the output of the SE operation. The threshold t can be obtained by the following formula:

[0077] t=β·Q v (12)

[0078] By integrating the Squeeze-and-Excitation Networks (SE) module and the soft threshold function, the SDM can effectively learn important features and remove redundant information. After determining the threshold vector, the SDM can automatically eliminate interference information in the noise signal M:

[0079] O'=SDM(M) (13)

[0080] Finally, the present application adopts a residual connection strategy to promote the feature transmission of the SDM, and the output O of the DM can be represented as:

[0081] O=O'+M (14)

[0082] In step S4, a double-scale residual block is constructed, and the structure is shown in Figure 5 Deep and shallow features are learned and different spatial dimension bearing fault information is captured at different scales, and the extraction ability of the fused features is enhanced;

[0083] In S41, the feature map is processed by a convolution layer with a convolution kernel size of 1*1 to increase the dimension, and the number of feature channels is expanded to twice the original number.

[0084] ​S42 divides the features into two groups of the same channel, one group of features is input into the traditional Basicblock to capture the deep relationship of the features and enhance the feature extraction ability. The other group of features is input into the convolution layer with the kernel size of 1x1 to capture the shallow relationship of the features.

[0085] S43 fuses the two groups of features and performs dimension reduction operation through the convolution layer with the kernel size of 1x1 to enhance the expression ability of the features in different scales. In order to improve the feature preservation ability, residual connection is performed. Unlike the double-scale residual block DSRB1, the DSRB2 module uses the convolution layer with the kernel size of 1x1 to perform dimension increasing operation to keep the consistency of the channels. In addition, the LeakyReLU activation function is adopted to speed up the training speed and avoid the overfitting problem.

[0086] Step S5 constructs the multi-modal fusion and multi-scale residual network (MF-MSRNet) and proposes a bearing fault diagnosis method based on the MF-MSRNet, as shown in Figure 6 The method involves the change of data distribution in different fields and can maintain consistent label space in each field, and the process is as follows:

[0087] S51 specifically, the RGB image is first subjected to preliminary feature extraction through the first convolution layer. The MSRNet model takes the DSRB as the core learning algorithm, and combines the DSRB module with the multi-scale denoising module (MDM). The DSRB module can enhance the extraction ability of features in different scales and retain local and global feature information; the MDM is helpful to further explore multi-level features and filter out irrelevant information.

[0088] S52 uses the global average pooling layer to reduce the dimension of the features and realize the translation invariant property.

[0089] S53 uses the fully connected layer to convert the pooled features into a one-dimensional vector, and uses the Softmax function to realize the classification of the bearing health state.

[0090] Step S6 is the training and verification process, which includes the following steps:

[0091] S61 based on the training set, the validation set and the test set, the MF-MSRNet model is evaluated by using the multi-modal data set;

[0092] S62 verifies the performance of the MF-MSRNet model based on the comparative method.

[0093] Further, the present application is based on the following embodiments, and the performance of the model of the present application is verified by comparison methods such as MF-InResNet, MF-ResNet18, MF-PResNet, MF-Res2Net, MF-MobileNetV2, MF-ShuffleNetV2 and ResNet50.

[0094] Specifically, the IEPE acquisition instrument is used to collect three-channel vibration signals, and the acceleration sensors are respectively installed on the bearing seat of the bearing to be measured (x, y and z directions), the voiceprint sensors are respectively installed on the top position and the front end of the bearing seat (y and z directions), and the acoustic emission sensor is installed on the top of the bearing seat of the bearing to be measured (z direction), and the sampling frequency of the three sensors is set to 20 kHz. According to different rotating speeds, a total of four working conditions are set, and the rotating speeds are constant at 900 rpm (G1), 1500 rpm (G2), 2100 rpm (G3) and 2700 rpm (G4), and the sampling time of each experiment is set to 60 s.

[0095] Based on the above data, the method of the present application and the comparison method are respectively used to evaluate the fault diagnosis performance in a strong noise environment, Figure 7 is the accuracy and loss comparison of each model during the training process under G1 working condition.

[0096] Figure 7 As can be seen from the results shown, the convergence speed of the MSRNet model proposed in the present application is the fastest, the accuracy has exceeded 90% after the third training period, and the training process has reached stability after the ninth training period. The ResNet18 curve rises smoothly, and the diagnosis performance is only second to the MSRNet model. The training curves of Res2Net, MobileNetV2, InResNet and ResNet50 are relatively close, the convergence speed is slow, and the overall accuracy is low. The diagnosis curve of PResNet fluctuates greatly; and the diagnosis performance of the ShuffleNetV2 model is the worst. In summary, compared with other models, the MSRNet model has obvious advantages in accuracy and convergence speed.

[0097] The confusion matrix can clearly measure the classification accuracy of each fault type. Figure 8 The confusion matrix of MSRNet under four working conditions is listed, and it is found that MSRNet can identify the fault types of eight train motor bearings with high accuracy.

[0098] In order to more clearly describe the experimental results, the experimental results are visualized, and the results are shown in Figure 9 , by observing Figure 9It can be seen that the MF-MSRNet fault diagnosis model has strong feature extraction capability, can effectively distinguish the characteristics of eight types of faults, and the effect is better than other comparison models. Therefore, the availability and effectiveness of the proposed method are verified by the four working condition data of the train traction motor bearing.

[0099] In order to evaluate the robustness of the model under different noise environments, the train bearing data set with a signal-to-noise ratio SNR of-10 to 0 dB is generated, and experiments are conducted on different models under various SNR conditions, and the diagnosis accuracy is shown in Table 4. Figure 10 Under the condition of-10 dB SNR, the accuracy of the proposed model is reduced to 77.68%, but it is still better than other models. The accuracy of MF-InResNet, MF-ResNet18 and MF-PResNet is 64.37%, 65.62% and 55.12% respectively, while the accuracy of MF-Res2Net, MF-MobileNetV2, MF-ShuffleNetV2 and ResNet50 is less than 34%. Through the comparison of the above experimental results, the effectiveness and feasibility of the MF-MSRNet method proposed in the application are verified.

[0100] The above disclosure is only a preferred embodiment of the application, and of course cannot limit the scope of the right of the application. Those skilled in the art can understand that all or part of the above-mentioned embodiments are implemented, and equivalent changes made according to the claims of the application still belong to the scope covered by the application.

Claims

1. A bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network, characterized in that, Comprising the following steps: Step 1: set up a subway train traction motor bearing experiment platform to obtain multi-modal data of the motor bearing; The subway train traction motor bearing experiment platform in step 1 is used to simulate the working condition of the subway train in the uniform speed running state and the corresponding bearing fault type, and the multi-modal data is obtained by sampling through different types of sensors, and each signal has the same sampling point; Step 2: pre-process the multi-modal data based on principal component analysis, construct a data fusion method, and divide the training set, validation set and test set; In the execution process of step 2, the collected multi-modal data is converted into an RGB image as the input of the model, and the specific process is to pre-process the multi-modal original signal by using principal component analysis method for dimension reduction, retain the first three principal components and convert them into three channel time domain signals, convert the data samples of the three channels into two-dimensional matrices, and normalize the pixel values, and finally fuse the three two-dimensional matrices into an RGB image; Step 3: construct a multi-scale denoising module; The multi-scale denoising module includes an inflation depthwise convolution, a multi-scale module and a soft denoising module, wherein the multi-scale module uniformly divides the obtained feature map into 4 subsets in the channel direction, presents multi-scale features through a hierarchical structure, expands the receptive field of the convolutional neural network, and the soft denoising module adopts a squeeze-and-excitation network and a soft threshold function to automatically learn and identify features and eliminate redundant information; Step 4: construct a double-scale residual block; The double-scale residual block is used to extract deep and shallow features of the input vector and retain local and global feature information; Step 5: build a multi-scale residual network; The process of bearing fault diagnosis by multi-modal fusion and multi-scale residual network includes the following steps: Step 5.1: the RGB image is first subjected to a preliminary feature extraction by a first convolutional layer; Step 5.2: a global average pooling layer is used to reduce the dimension of the features and realize the translational invariance characteristic; Step 5.3: a fully connected layer is used to convert the pooled features into a one-dimensional vector, and a Softmax function is used to realize the classification of bearing health status; Step 6: train and verify the multi-scale residual network based on the training set, validation set and test set, and finally use the trained model to evaluate the bearing multi-modal samples to obtain the fault classification result.

2. The bearing fault diagnosis method based on multi-modal fusion and multi-scale residual network according to claim 1, wherein In step 6, the performance of the model in fault diagnosis is evaluated by using indicators, and the indicators include average test accuracy and average test F1-score.

Images