Rolling bearing fault diagnosis system and method based on multi-source fusion residual network

Abstract

The invention provides a rolling bearing fault diagnosis system and method based on a multi-source fusion residual network, and relates to the technical field of bearing fault diagnos.The rolling bearing fault diagnosis system and method adopt a one-dimensional convolution kernel and a residual block feature extraction module, the one-dimensional convolution kernel ensures that data of different sensors are not mixed in the feature extraction process, and the residual block feature extraction module is used for extracting the data of different sensors; the effectiveness of unique features of each type of signals is guaranteed, the gradient disappearance problem in a deep network is relieved through residual connection, a feature fusion module of a grouping cross attention mechanism is further adopted, sensor signals are grouped according to types, each group is weighted independently, and the sensor signals are subjected to clustering processing according to types. And local and global multi-scale features are further extracted by using multi-scale convolution, so that the uniqueness of different sensor signals is ensured, the expression ability of fusion features is improved, and the accuracy and robustness of rolling bearing fault diagnosis are improved.

Description

Technical Field

[0001] The invention relates to the technical field of bearing fault diagnosis, and in particular to a rolling bearing fault diagnosis system and method based on a multi-source fusion residual network. Background Art

[0002] Rolling bearings are key components widely used in rotating machinery, and their performance directly affects the operating stability and service life of the equipment. However, as time goes by, rolling bearings will inevitably suffer from wear, fatigue and other faults, which will affect the normal operation of the equipment and may even cause equipment shutdown or damage in severe cases. Therefore, how to accurately diagnose the fault status of rolling bearings is of great significance to equipment maintenance and fault prevention.

[0003] Rolling bearing fault diagnosis methods mainly include data analysis methods based on a single sensor and data fusion methods based on multi-source fusion. Fault diagnosis methods based on a single sensor usually rely on data collected by a single type of sensor (such as a vibration or acoustic sensor) and use signal processing and feature extraction technology to identify faults. However, the single signal diagnosis method is difficult to fully reflect the complex fault characteristics of rolling bearings, especially under complex working conditions, which is prone to information loss or misjudgment. Compared with this method, the rolling bearing fault diagnosis method based on multi-source fusion can effectively improve the richness of feature extraction and the reliability of diagnostic results by simultaneously utilizing multiple types of signal data from multiple sensors, thereby achieving better fault diagnosis effects.

[0004] As a powerful tool in multi-source fusion methods, deep learning has been widely used in the field of rolling bearing fault diagnosis due to its outstanding ability in deep feature mining. However, existing multi-source fusion methods based on deep learning still have the following problems: (1) Early fusion methods usually fuse or directly concatenate multi-source signals through weighted rules, and then extract features from the fused data, or convert multi-sensor data into images in advance for processing. During the feature extraction process, data from different sensors will be mixed with each other, resulting in the weakening of the effectiveness of the unique features of each signal, affecting the diagnostic accuracy. (2) Late fusion methods usually adopt a multi-branch network structure, where each sensor data channel corresponds to an independent network branch. Although this can maintain the independent processing of signals, it increases the complexity of the network. Especially when processing more data channels, it is easy to cause optimization problems such as gradient disappearance or explosion, which limits the performance improvement of the model. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to propose a rolling bearing fault diagnosis system based on a multi-source fusion residual network, including a rolling bearing initial data acquisition module, a rolling bearing data preprocessing module, a feature extraction module, a feature fusion module, a classification module and a parameter updating module;

[0006] The rolling bearing initial data acquisition module comprises a plurality of sensors, which collect multi-sensor signal data of the rolling bearing through the sensors, wherein the multi-sensor signal data comprises vibration signals and sound signals under five different fault states, and sends the multi-sensor signal data to the rolling bearing data preprocessing module;

[0007] The rolling bearing data preprocessing module includes a data splicing module, a normalization processing module and a label allocation module;

[0008] The data splicing module is used to intercept the vibration signal and the sound signal under each fault state in the multi-sensor signal data, and splice the intercepted vibration signal and the sound signal to obtain multiple two-dimensional matrices of the fault state, and then obtain multiple two-dimensional matrices of five different fault states, and send all the two-dimensional matrices to the normalization processing module;

[0009] The normalization processing module is used to normalize the data in each two-dimensional matrix to obtain a normalized two-dimensional matrix, and send the normalized two-dimensional matrix to the label allocation module;

[0010] The label assignment module is used to assign labels to the normalized two-dimensional matrix, and the labels represent the fault states corresponding to the normalized two-dimensional matrix; the label assignment module is also used to send the normalized two-dimensional matrix and its corresponding labels to the feature extraction module;

[0011] The feature extraction module is used to extract features from the normalized two-dimensional matrix to obtain a downsampled high-order feature map, and input the downsampled high-order feature map into the feature fusion module;

[0012] The feature fusion module includes a group cross attention mechanism and a flattening layer, which processes the downsampled high-order feature map through the group cross attention mechanism to obtain a merged feature map, and sends the merged feature map to the flattening layer, which is used to flatten the merged feature map into a one-dimensional vector, and sends the flattened one-dimensional vector to the classification module;

[0013] The classification module includes a fully connected layer and a softmax layer; the flattened one-dimensional vector is sequentially passed through the fully connected layer and the softmax layer to obtain five first values, each of which represents a predicted value of the probability that the rolling bearing is in a fault state, and the maximum value of the five first values ​​is obtained, and then the second value is obtained, the second value represents the fault state corresponding to the maximum value, and the second value is sent to the parameter update module;

[0014] The parameter updating module is used to compare the second value and the label of the normalized two-dimensional matrix, and calculate the error between the two, to determine whether the error between the second value and the label of the normalized two-dimensional matrix meets the preset conditions. When the error between the second value and the label of the normalized two-dimensional matrix meets the preset conditions, it is characterized that the rolling bearing fault diagnosis system based on the multi-source fusion residual network can obtain the rolling bearing fault diagnosis result. When the error between the second value and the label of the normalized two-dimensional matrix does not meet the preset conditions, the parameters in the feature extraction module, the feature fusion module and the classification module are updated according to the error between the second value and the label of the normalized two-dimensional matrix through the error back propagation algorithm until the error between the second value and the label of the normalized two-dimensional matrix meets the preset conditions.

[0015] Optionally, when the error between the second value and the label of the normalized two-dimensional matrix meets a preset condition, the method further includes:

[0016] The multi-sensor signal data of the rolling bearing to be tested is collected by the rolling bearing initial data acquisition module, and then processed by the data splicing module and the normalization processing module in the rolling bearing data preprocessing module to obtain a two-dimensional matrix of the rolling bearing to be tested, and then successively passed through the feature extraction module, the feature fusion module and the classification module to obtain the target value of the rolling bearing to be tested, and the target value is used as the fault diagnosis result.

[0017] Optionally, the feature extraction module includes residual block 1, average pooling layer 1, residual block 2, average pooling layer 2, residual block 3, and average pooling layer 3;

[0018] Among them, feature extraction is performed on the normalized two-dimensional matrix to obtain a downsampled high-order feature map, including:

[0019] The normalized two-dimensional matrix is ​​input into residual block 1, which performs shallow feature extraction on the normalized two-dimensional matrix to obtain a preliminary feature map, and the preliminary feature map is input into average pooling layer 1, which downsamples the preliminary feature map to obtain a downsampled preliminary feature map, and the downsampled preliminary feature map is input into residual block 2, which further extracts features from the downsampled preliminary feature map to obtain an intermediate feature map, and the intermediate feature map is input into average pooling layer 2, which downsamples the intermediate feature map, and the downsampled intermediate feature map is input into residual block 3, which performs high-order feature extraction on the downsampled intermediate feature map to obtain a high-order feature map, and the high-order feature map is input into average pooling layer 3, which downsamples the high-order feature map to obtain a downsampled high-order feature map.

[0020] Optionally, the residual block 1 includes two convolutional layers 1, two batch normalization layers, two ReLU activation layers and a residual connection module, and the convolutional layer 1 includes a one-dimensional convolution kernel with a convolution kernel size of a first size;

[0021] The residual block 2 includes two convolutional layers 2, two batch normalization layers, two ReLU activation layers and a residual connection module, and the convolutional layer 2 includes a one-dimensional convolution kernel with a convolution kernel size of the second size;

[0022] The residual block 3 includes two convolutional layers 3, two batch normalization layers, two ReLU activation layers and a residual connection module layer, and the convolutional layer 3 includes a one-dimensional convolution kernel with a convolution kernel size of a third size.

[0023] Optionally, the grouped cross attention mechanism includes a grouping layer, a global average pooling layer, a convolution layer, a softmax layer, a group weighting layer, a multi-scale convolution layer and a merging layer;

[0024] Among them, the downsampled high-order feature map is processed through the group cross attention mechanism to obtain the merged feature map, including:

[0025] The grouping layer is used to receive the downsampled high-order feature map, divide the high-order feature map into three groups of high-order feature maps according to the type of sensor, and send the three groups of high-order feature maps to the global average pooling layer and the grouping weighted layer;

[0026] The global average pooling layer is used to perform a global average pooling operation on each group of high-order feature maps to obtain three groups of feature maps after the global average pooling operation, and send the three groups of feature maps after the global average pooling operation to the convolution layer;

[0027] The convolution layer is used to perform 1×1 convolution and sigmoid function nonlinear activation on each group of feature maps after the global average pooling operation, obtain the attention weight of each group, and then obtain three groups of attention weights, and send the three groups of attention weights to the softmax layer;

[0028] The softmax layer is used to normalize the attention weights of each group to obtain three groups of normalized attention weights, and the three groups of normalized attention weights are sent to the group weighting layer;

[0029] The group weighted layer is used to multiply the normalized attention weights and the grouped feature maps belonging to the same group channel by channel to obtain three groups of weighted feature maps, and send the three groups of weighted feature maps to the multi-scale convolutional layer;

[0030] The multi-scale convolution layer is used to perform a 3×3 convolution operation and a 5×5 convolution operation on each group of weighted feature maps, add the feature map after the 3×3 convolution operation and the feature map after the 5×5 convolution operation to obtain a multi-scale feature map, and then obtain three groups of multi-scale feature maps, and send the three groups of multi-scale feature maps to the merging layer;

[0031] The merging layer is used to splice the three sets of multi-scale feature maps along the sensor dimension, restore and merge them into feature maps of original size, and obtain a merged feature map.

[0032] Optionally, the grouped cross attention mechanism is implemented by the following formula:

[0033] According to the type of sensor, the high-order feature map is divided into three groups of high-order feature maps. A global average pooling operation is performed on each group of high-order feature maps to obtain a feature map after the global average pooling operation, which is specifically expressed by the following formula:

[0034] G i =GlobalAvgPool(X i );

[0035] Among them, X i represents the i-th group of high-order feature maps, G i Represents the feature map after the global average pooling operation corresponding to the i-th group of high-order feature maps;

[0036] Perform 1×1 convolution and sigmoid function nonlinear activation on each group of feature maps after the global average pooling operation to obtain the attention weight of each group, which is specifically implemented by the following formula:

[0037] A i =σ(Conv 1×1 (G i ));

[0038] Among them, σ is the sigmoid function, A i is the attention weight of the i-th group;

[0039] The attention weights of each group are normalized using the following formula:

[0040]

[0041] in, represents the normalized attention weight of the i-th group, exp(A i ) represents the attention weight A i Applying the exponential function, A i,c represents the cth channel of the i-th group of attention weights, C represents the number of channels, represents the exponential summation of all channels of the i-th group of attention weights;

[0042] Each group of high-order feature maps X i And the normalized attention weights Multiply each channel to generate a weighted feature map, which is specifically expressed by the following formula:

[0043]

[0044] in, represents element-wise multiplication, X i ′ represents the weighted feature map of the i-th group;

[0045] For each group of weighted feature maps, 3×3 convolution operations and 5×5 convolution operations are performed respectively, and the feature maps after the 3×3 convolution operation are added to the feature maps after the 5×5 convolution operation to obtain a multi-scale feature map, which is specifically implemented by the following formula:

[0046]

[0047] in, Represents the multi-scale feature map of the i-th group;

[0048] The three sets of multi-scale feature maps are spliced ​​along the sensor dimension and restored to the feature map of the original size to obtain the merged feature map, which is specifically achieved by the following formula:

[0049]

[0050] Among them, X output Represents the merged feature map.

[0051] A rolling bearing fault diagnosis method based on a multi-source fusion residual network is implemented based on the aforementioned rolling bearing fault diagnosis system based on a multi-source fusion residual network, comprising:

[0052] Step 1: Collect multi-sensor signal data of the rolling bearing through a sensor, wherein the multi-sensor signal data includes vibration signals and sound signals under five different fault states;

[0053] Step 2: intercept and splice the vibration signal and sound signal of each fault state to obtain multiple two-dimensional matrices of the fault state, and then obtain multiple two-dimensional matrices of five different fault states;

[0054] Step 3: Normalize each two-dimensional matrix to obtain a normalized two-dimensional matrix;

[0055] Step 4: assign a label to each normalized two-dimensional matrix, wherein the label represents the fault state corresponding to the normalized two-dimensional matrix, take the normalized two-dimensional code matrix as an input sample, take the label corresponding to the normalized two-dimensional code matrix as an output sample, and obtain multiple samples, and all samples constitute a sample set;

[0056] Step 5: Divide the sample set into training set and test set;

[0057] Step 6: inputting the input samples in the training set into the multi-source fusion residual network model to obtain a second value, wherein the multi-source fusion residual network model is composed of the feature extraction module, the feature fusion module and the classification module;

[0058] Step 7: Input the second value and the output sample of the normalized two-dimensional matrix into a loss function to obtain a loss function value, where the loss function is sparse classification cross entropy;

[0059] Step 8: According to the value of the loss function, the back propagation algorithm is applied to calculate the gradient of each parameter layer by layer; the optimizer is used to adjust the weights and biases of the multi-source fusion residual network model according to the gradient of each parameter to minimize the loss function; the parameters of the multi-source fusion residual network model are continuously updated until the multi-source fusion residual network model converges to obtain the trained multi-source fusion residual network model;

[0060] Step 9: Input the normalized two-dimensional matrix in the test set into the trained multi-source fusion residual network model to obtain the numerical value corresponding to the normalized two-dimensional matrix in the test set, that is, the fault diagnosis result.

[0061] The beneficial effects of adopting the above technical solution are:

[0062] (1) The present invention uses a residual block of a one-dimensional convolution kernel for feature extraction, wherein the one-dimensional convolution kernel ensures that data from different sensors are not mixed during the feature extraction process, thereby ensuring the effectiveness of the unique features of each signal. The residual block has significant advantages in a multi-sensor environment and can effectively extract the deep features of each sensor. At the same time, the residual connection is used to alleviate the gradient vanishing problem in the deep network, thereby ensuring the training stability and accuracy of the network when processing multi-source data.

[0063] (2) The present invention adopts a feature fusion method based on a grouped cross-attention mechanism. The sensor signals are grouped and processed by type (vibration signal group, central sound sensor group, edge sound sensor group), and each group is independently weighted to enhance the response of key signals. Multi-scale convolution is further used to extract local and global multi-scale features to ensure the uniqueness of different sensor signals. At the same time, the expression ability of fused features is improved, thereby improving the accuracy and robustness of rolling bearing fault diagnosis. (3) The present invention ensures the independence of various sensor signals in the feature extraction stage by connecting the feature extraction module of the one-dimensional convolution kernel in series with the feature fusion module of the grouped cross-attention mechanism. In the feature fusion stage, the feature expression ability of each group of sensors is enhanced by grouping processing and multi-scale convolution, thereby effectively improving the accuracy and stability of the diagnosis system. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a structural schematic diagram of a rolling bearing fault diagnosis system based on a multi-source fusion residual network in an embodiment of the present invention;

[0065] Figure 2 It is a structural diagram of a group-based cross-attention mechanism in an embodiment of the present invention;

[0066] Figure 3 A schematic flow chart of a rolling bearing fault diagnosis method based on a multi-source fusion residual network in an embodiment of the present invention;

[0067] Figure 4 The training and testing accuracy curves in the embodiment of the present invention;

[0068] Figure 5 The training and testing loss curves in the embodiment of the present invention;

[0069] Figure 6 It is a confusion matrix diagram of the test set in an embodiment of the present invention;

[0070] Figure 7 This is a T-SNE visualization diagram of the fully connected layer in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0072] In view of the problems existing in the prior art, the present invention provides a rolling bearing fault diagnosis system based on a multi-source fusion residual network. Figure 1 , including a rolling bearing initial data acquisition module, a rolling bearing data preprocessing module, a feature extraction module, a feature fusion module, a classification module and a parameter updating module;

[0073] The rolling bearing initial data acquisition module comprises a plurality of sensors, which collect multi-sensor signal data of the rolling bearing through the sensors, wherein the multi-sensor signal data comprises vibration signals and sound signals under five different fault states, and sends the multi-sensor signal data to the rolling bearing data preprocessing module;

[0074] Among them, the five different fault states include normal, outer ring fault, inner ring fault, rolling element fault, and inner and outer ring combined fault; among them, the number of sensors collecting vibration signals is 1, and the number of sensors collecting sound signals is 11, so there is 1 group of vibration signals and 11 groups of sound signals.

[0075] In this embodiment, the experimental platform for collecting bearing fault data is mainly composed of a motor, a vibration sensor, a sound sensor, a vibration data acquisition system, a sound data acquisition system and bearings in five states. The motor speed is set to 1200r / min, the sampling frequency of the vibration signal and the sound signal is set to 16384Hz, and the number of sampling points is set to 81000; the sound sensor is arranged in a plane 400mm away from the end of the bearing to be tested, and the center is coaxial with the rolling bearing; the vibration sensor is fixed above the bearing seat of the bearing to be tested by pasting; the sound sensor and the vibration sensor are respectively connected to the sound and vibration data acquisition system; the sound and vibration data acquisition system are respectively connected to the computer terminal to obtain the sound signal and vibration signal data of the bearing.

[0076] The rolling bearing data preprocessing module includes a data splicing module, a normalization processing module and a label allocation module;

[0077] The data splicing module is used to intercept the vibration signal and the sound signal under each fault state in the multi-sensor signal data, and splice the intercepted vibration signal and the sound signal to obtain multiple two-dimensional matrices of the fault state, and then obtain multiple two-dimensional matrices of five different fault states, and send all the two-dimensional matrices to the normalization processing module;

[0078] In the specific implementation, the length of 1 vibration signal and 11 sound signals is 81000. Every 300 data points are intercepted to obtain 270 vibration signals and 11*270 sound signals. There are a total of 1350 vibration signals and 11*270*5 sound signals for the five fault states. The vibration signals and sound signals of each fault state are spliced ​​to obtain 270 two-dimensional matrices of size 12×300, where rows represent data points and columns represent sensors, that is, the first 12 sensors. For example, (1,1) in the two-dimensional matrix corresponds to the data of the first data point collected by the first sensor.

[0079] The normalization processing module is used to normalize the data in each two-dimensional matrix to obtain a normalized two-dimensional matrix, and send the normalized two-dimensional matrix to the label allocation module;

[0080] The data in each two-dimensional matrix is ​​normalized. Specifically, each row in the two-dimensional matrix is ​​normalized so that it is within the interval [-1,1]. This can be achieved by the following formula:

[0081]

[0082] Among them, X is a data in a row of a two-dimensional matrix, X normalized represents the normalized data of X, and max(|X|) represents the maximum absolute value of the row where X is located;

[0083] The label assignment module is used to assign labels to the normalized two-dimensional matrix, and the labels represent the fault states corresponding to the normalized two-dimensional matrix, wherein the labels can be 0, 1, 2, 3, and 4, corresponding to normal, outer ring fault, inner ring fault, rolling element fault, and inner and outer ring combined fault, respectively; the label assignment module is also used to send the normalized two-dimensional matrix and its corresponding labels to the feature extraction module;

[0084] The feature extraction module is used to extract features from the normalized two-dimensional matrix to obtain a downsampled high-order feature map, and input the downsampled high-order feature map into the feature fusion module;

[0085] The feature extraction module includes residual block 1, average pooling layer 1, residual block 2, average pooling layer 2, residual block 3, and average pooling layer 3;

[0086] Among them, feature extraction is performed on the normalized two-dimensional matrix to obtain a downsampled high-order feature map, including:

[0087] The normalized two-dimensional matrix is ​​input into residual block 1, which performs shallow feature extraction on the normalized two-dimensional matrix to obtain a preliminary feature map to retain the basic information of the input features while avoiding the gradient vanishing problem. The preliminary feature map is input into average pooling layer 1, which downsamples the preliminary feature map to obtain a downsampled preliminary feature map to reduce the size of the feature map, retain global important information and improve computational efficiency. The downsampled preliminary feature map is input into residual block 2, which performs further feature extraction on the downsampled preliminary feature map to obtain an intermediate feature map to capture Deeper feature information enhances the ability to distinguish different sensor signals. The intermediate feature map is input into the average pooling layer 2. The average pooling layer 2 downsamples the intermediate feature map to further reduce the size of the feature map and retain important information. The downsampled intermediate feature map is input into the residual block 3. The residual block 3 extracts high-order features from the downsampled intermediate feature map to obtain a high-order feature map. The high-order feature map can comprehensively characterize the fault characteristics of the rolling bearing. The high-order feature map is input into the average pooling layer 3. The average pooling layer 3 downsamples the high-order feature map to obtain a downsampled high-order feature map.

[0088] The residual block 1 includes two convolutional layers 1, two batch normalization layers, two ReLU activation layers and a residual connection module, and the convolutional layer 1 includes a one-dimensional convolution kernel with a convolution kernel size of a first size.

[0089] Among them, the residual block 1 performs shallow feature extraction on the normalized two-dimensional matrix to obtain a preliminary feature map, including:

[0090] The normalized two-dimensional matrix is ​​subjected to preliminary feature extraction through convolution layer 1, and a one-dimensional convolution kernel is used to ensure that different sensor signals are not mixed during the feature extraction process, and the first feature map is obtained and sent to the batch normalization layer; the batch normalization layer standardizes the first feature map to stabilize the network training process and prevent the gradient from disappearing or exploding, and the standardized first feature map is sent to the ReLU activation layer; the ReLU activation layer is used to introduce nonlinear transformations to enhance the expression ability of the model, and the activated feature map is sent to convolution layer 1, which further extracts features from the activated feature map to ensure that different sensor signals are still processed independently and are not mixed, and the first feature map is obtained. To a deeper feature map, and send it to the batch normalization layer, the batch normalization layer normalizes the deeper feature map again to ensure the stability of the feature extraction process, and sends the standardized deeper feature map to the residual connection module; the residual connection module is used to jump connect the normalized two-dimensional matrix with the standardized deeper feature map to obtain the fused feature map, maintain the basic information of the normalized two-dimensional matrix, avoid the gradient disappearance problem, and send the fused feature map to the ReLU activation layer, the ReLU activation layer performs nonlinear activation on the fused feature map to enhance the expression ability of the feature and obtain the activated feature map, which is the preliminary feature map.

[0091] The residual block 2 includes a convolution layer 2, a batch normalization layer, a ReLU activation layer, a convolution layer 2, a batch normalization layer, a residual connection module and a ReLU activation layer, and the convolution layer 2 includes a one-dimensional convolution kernel with a convolution kernel size of a second size;

[0092] Among them, the residual block 2 further extracts features from the downsampled preliminary feature map to obtain an intermediate feature map, including:

[0093] The convolution layer 2 performs preliminary feature extraction on the downsampled preliminary feature map, and adopts a one-dimensional convolution kernel to ensure that different sensor signals are not mixed during the feature extraction process, and obtains the second feature map and sends it to the batch normalization layer; the batch normalization layer standardizes the second feature map to stabilize the network training process and prevent the gradient from disappearing or exploding, and sends the standardized second feature map to the ReLU activation layer; the ReLU activation layer is used to introduce nonlinear transformations to enhance the expression ability of the model, and sends the activated feature map to the convolution layer 2, which further extracts features from the activated feature map to ensure that different sensor signals are still processed independently without mixing, and obtains To a deeper feature map, and send it to the batch normalization layer, the batch normalization layer normalizes the deeper feature map again to ensure the stability of the feature extraction process, and sends the standardized deeper feature map to the residual connection module; the residual connection module is used to jump connect the downsampled preliminary feature map with the standardized deeper feature map to obtain the fused feature map, maintain the basic information of the normalized two-dimensional matrix, avoid the gradient disappearance problem, and send the fused feature map to the ReLU activation layer, the ReLU activation layer performs nonlinear activation on the fused feature map to enhance the expression ability of the feature and obtain the activated feature map, which is the intermediate feature map.

[0094] The residual block 3 includes a convolution layer 3, a batch normalization layer, a ReLU activation layer, a convolution layer 3, a batch normalization layer, a residual connection module and a ReLU activation layer, and the convolution layer 3 includes a one-dimensional convolution kernel with a convolution kernel size of a third size.

[0095] Among them, the residual block 3 extracts high-order features from the downsampled intermediate feature map to obtain a high-order feature map, including:

[0096] The convolution layer 3 performs preliminary feature extraction on the downsampled preliminary feature map, and adopts a one-dimensional convolution kernel to ensure that different sensor signals are not mixed during the feature extraction process, and obtains the third feature map and sends it to the batch normalization layer; the batch normalization layer standardizes the third feature map to stabilize the network training process and prevent the gradient from disappearing or exploding, and sends the standardized third feature map to the ReLU activation layer; the ReLU activation layer is used to introduce nonlinear transformation to enhance the expression ability of the model, and sends the activated feature map to the convolution layer 3, which further extracts features from the activated feature map to ensure that different sensor signals are still processed independently without mixing, and obtains To a deeper feature map, and send it to the batch normalization layer, the batch normalization layer normalizes the deeper feature map again to ensure the stability of the feature extraction process, and sends the standardized deeper feature map to the residual connection module; the residual connection module is used to jump connect the downsampled intermediate feature map with the standardized deeper feature map to obtain the fused feature map, maintain the basic information of the normalized two-dimensional matrix, avoid the gradient disappearance problem, and send the fused feature map to the ReLU activation layer, the ReLU activation layer performs nonlinear activation on the fused feature map to enhance the expression ability of the feature and obtain the activated feature map, that is, the high-order feature map.

[0097] The feature fusion module includes a group cross attention mechanism and a flattening layer, which processes the downsampled high-order feature map through the group cross attention mechanism to obtain a merged feature map, and sends the merged feature map to the flattening layer, which is used to flatten the merged feature map into a one-dimensional vector, and sends the flattened one-dimensional vector to the classification module;

[0098] The group cross attention mechanism includes a grouping layer, a global average pooling layer, a convolution layer, a softmax layer, a group weighting layer, a multi-scale convolution layer and a merging layer;

[0099] Among them, the downsampled high-order feature map is processed through the group cross attention mechanism to obtain the merged feature map, including:

[0100] The grouping layer is used to receive the downsampled high-order feature map, divide the high-order feature map into three groups of high-order feature maps according to the type of sensor, and send the three groups of high-order feature maps to the global average pooling layer and the grouping weighted layer;

[0101] In specific implementation, it can be divided into a vibration signal group (1 vibration sensor), a central sound sensor group (6 sound sensors) and an edge sound sensor group (5 sound sensors) to ensure that each type of signal is processed independently.

[0102] The global average pooling layer is used to perform a global average pooling operation on each group of high-order feature maps to obtain three groups of feature maps after the global average pooling operation, compress the global information of each channel into a single value to retain the global spatial information, and send the three groups of feature maps after the global average pooling operation to the convolution layer;

[0103] The convolution layer is used to perform 1×1 convolution and sigmoid function nonlinear activation on each group of feature maps after the global average pooling operation, obtain the attention weight of each group, and then obtain three groups of attention weights, and send the three groups of attention weights to the softmax layer;

[0104] The softmax layer is used to normalize the attention weights of each group to obtain three groups of normalized attention weights, so that the weights compete in the channel dimension of each group, highlight the characteristics of the key channels, and send the three groups of normalized attention weights to the group weighting layer;

[0105] The group weighted layer is used to multiply the normalized attention weights and grouped feature maps belonging to the same group channel by channel, enhance the features of important channels in the vibration signal group, the center sound sensor group and the edge sound sensor group, obtain three groups of weighted feature maps, and send the three groups of weighted feature maps to the multi-scale convolution layer;

[0106] The multi-scale convolution layer is used to perform 3×3 convolution operations and 5×5 convolution operations on the weighted feature maps of each group, and add the feature maps after the 3×3 convolution operation and the feature maps after the 5×5 convolution operation to obtain a multi-scale feature map to capture local and global multi-scale features, further improve the expression ability of features of different sensor groups, and then obtain three groups of multi-scale feature maps, which are sent to the merging layer;

[0107] The merging layer is used to splice the three sets of multi-scale feature maps along the sensor dimension, restore and merge them into feature maps of original size, and obtain a merged feature map.

[0108] The grouped cross attention mechanism, combined with Figure 2 , achieved through the following formula:

[0109] According to the type of sensor, the high-order feature graphs are divided into three groups, namely, the high-order feature graphs of the vibration signal group, the high-order feature graphs of the center sound sensor group, and the high-order feature graphs of the edge sound sensor group; assuming that the dimension of the high-order feature graph is Where H represents height, W represents width, and C represents the number of channels, then the dimension of each set of high-order feature maps is Among them, H i Represents the height of the i-th group of feature maps;

[0110] In specific implementation, combined with Figure 2 , the dimension of the high-order feature map is Among them, 12 represents the height, 37 represents the width, and 64 represents the number of channels. The dimension of each set of high-order feature maps is Among them, H i Indicates the height of the i-th group of feature graphs, the vibration signal group H1=1, the center sound sensor group (6 sound sensors) H2=6, and the edge sound sensor group (5 sound sensors) H3=5;

[0111] Perform a global average pooling operation on each group of high-order feature maps to obtain the feature map after the global average pooling operation, which is specifically expressed by the following formula:

[0112] G i =GlobalAvgPool(X i );

[0113] Among them, X i represents the i-th group of high-order feature maps, G i represents the feature map after the global average pooling operation corresponding to the i-th group of high-order feature maps; G i The dimension is The first group of high-order feature graphs are high-order feature graphs of the vibration signal group, the second group of high-order feature graphs are high-order feature graphs of the center sound sensor group, and the third group of high-order feature graphs are high-order feature graphs of the edge sound sensor group;

[0114] In specific implementation, combined with Figure 2 , G i The dimension is

[0115] Perform 1×1 convolution and sigmoid function nonlinear activation on each group of feature maps after the global average pooling operation to obtain the attention weight of each group, which is specifically implemented by the following formula:

[0116] A i =σ(Conv 1×1 (G i ));

[0117] Among them, σ is the sigmoid function, A i is the attention weight of the i-th group, with dimension In specific implementation, combined with Figure 2 , A i The dimension is

[0118] The attention weights of each group are normalized using the following formula:

[0119]

[0120] in, represents the normalized attention weight of the i-th group, exp(A i ) represents the attention weight A i Applying the exponential function, A i,c represents the cth channel of the i-th group of attention weights, C represents the number of channels, represents the exponential summation of all channels of the i-th group of attention weights;

[0121] Each group of high-order feature maps X i And the normalized attention weights Multiply each channel to generate a weighted feature map, which is specifically expressed by the following formula:

[0122]

[0123] in, represents element-wise multiplication, X i ′ represents the weighted feature map of the i-th group, and its dimension is In specific implementation, combined with Figure 2 , X i The dimension of ′ is

[0124] For each group of weighted feature maps, 3×3 convolution operations and 5×5 convolution operations are performed respectively, and the feature maps after the 3×3 convolution operation are added to the feature maps after the 5×5 convolution operation to obtain a multi-scale feature map, which is specifically implemented by the following formula:

[0125]

[0126] in, Represents the multi-scale feature map of the i-th group, with dimension In specific implementation, combined with Figure 2 , The dimension is

[0127] The three sets of multi-scale feature maps are spliced ​​along the sensor dimension and restored to the feature map of the original size to obtain the merged feature map, which is specifically achieved by the following formula:

[0128]

[0129] Among them, X output Represents the merged feature map, with dimension In specific implementation, combined with Figure 2 , X output The dimension is

[0130] The classification module includes a fully connected layer and a softmax layer; the flattened one-dimensional vector is sequentially passed through the fully connected layer and the softmax layer to obtain five first values, each of which represents a predicted value of the probability that the rolling bearing is in a fault state, and the maximum value of the five first values ​​is obtained, and then the second value is obtained, the second value represents the fault state corresponding to the maximum value, and the second value is sent to the parameter update module;

[0131] Among them, the fully connected layer is used to process the one-dimensional feature vector from the flattened layer, extract higher-level feature representation, and send the result to the softmax layer; the softmax layer is used to normalize the output of the fully connected layer, generate the probability distribution of various faults, that is, the five first values, and finally obtain the fault diagnosis result, that is, the second value.

[0132] The parameter updating module is used to compare the second value and the label of the normalized two-dimensional matrix, and calculate the error between the two, to determine whether the error between the second value and the label of the normalized two-dimensional matrix meets the preset conditions. When the error between the second value and the label of the normalized two-dimensional matrix meets the preset conditions, it is characterized that the rolling bearing fault diagnosis system based on the multi-source fusion residual network can obtain the rolling bearing fault diagnosis result. When the error between the second value and the label of the normalized two-dimensional matrix does not meet the preset conditions, the parameters in the feature extraction module, the feature fusion module and the classification module are updated according to the error between the second value and the label of the normalized two-dimensional matrix through the error back propagation algorithm until the error between the second value and the label of the normalized two-dimensional matrix meets the preset conditions.

[0133] The preset condition may be that the number of parameter updates reaches a preset number, or that the error between the second value and the label of the normalized two-dimensional matrix is ​​less than a preset threshold.

[0134] Wherein, when the error between the second value and the label of the normalized two-dimensional matrix meets a preset condition, the method further includes:

[0135] The multi-sensor signal data of the rolling bearing to be tested is collected by the rolling bearing initial data acquisition module, and then processed by the data splicing module and the normalization processing module in the rolling bearing data preprocessing module to obtain a two-dimensional matrix of the rolling bearing to be tested, and then successively passed through the feature extraction module, the feature fusion module and the classification module to obtain the target value of the rolling bearing to be tested, and the target value is used as the fault diagnosis result.

[0136] According to the rolling bearing fault diagnosis system based on multi-source fusion residual network, combined with Figure 3The present invention also provides a rolling bearing fault diagnosis method based on a multi-source fusion residual network, comprising:

[0137] Step 1: Collect multi-sensor signal data of the rolling bearing through a sensor, wherein the multi-sensor signal data includes vibration signals and sound signals under five different fault states;

[0138] Step 2: intercept and splice the vibration signal and sound signal of each fault state to obtain multiple two-dimensional matrices of the fault state, and then obtain multiple two-dimensional matrices of five different fault states;

[0139] Step 3: Normalize each two-dimensional matrix to obtain a normalized two-dimensional matrix;

[0140] Step 4: assign a label to each normalized two-dimensional matrix, wherein the label represents the fault state corresponding to the normalized two-dimensional matrix; use the normalized two-dimensional code matrix as an input sample, and use the label corresponding to the normalized two-dimensional code matrix as an output sample to obtain multiple samples, and all samples constitute a sample set;

[0141] Step 5: Divide the sample set into training set and test set;

[0142] Among them, 80% (i.e., the first 1080 samples) of the samples for each fault state in the sample set are used as the training set, and 20% (i.e., the subsequent 270 samples) of the samples for each fault state in the sample set are used as the test set; the training set is used to train the model parameters of the multi-source fusion residual network model, and the test set is used to evaluate the diagnostic performance of the multi-source fusion residual network model after training.

[0143] Step 6: Input the input samples in the training set into the multi-source fusion residual network model to obtain a second value, i.e., a fault diagnosis result, wherein the multi-source fusion residual network model is composed of the feature extraction module, the feature fusion module and the classification module;

[0144] It should be noted that in step 6, the multi-source fusion residual network model of the input sample in the training set is an untrained model. Before the first input of the training sample, the parameters of the multi-source fusion residual network model need to be initialized. Later, the parameters of the multi-source fusion residual network model can be updated through the loss function value. The parameters of the multi-source fusion residual network model are shown in Table 1:

[0145] Table 1 Specific parameters of the multi-source fusion residual network model

[0146] Serial number Layer Name Kernel size / number of channels Stride Output shape 1 Residual Block 1 1×3 / 16 1×1 12×300×16 2 Average Pooling 1 1×2 / 16 1×2 12×150×16 3 Residual Block 2 1×5 / 32 1×1 12×150×32 4 Average Pooling 2 1×2 / 32 1×2 12×75×32 5 Residual Block 3 1×7 / 64 1×1 12×75×64 6 Average Pooling 3 1×2 / 64 1×2 12×37×64 7 Grouped Cross-Attention Mechanism \ \ 12×37×64 8 Flatten the layer \ \ 28416 9 Fully connected layer + ReLU 256 / 1 \ 256 10 Fully connected layer + softmax 5 / 1 \ 5

[0147] Step 7: Input the second numerical value and the output sample of the normalized two-dimensional matrix into a loss function to obtain a loss function value, where the loss function is sparse classification cross entropy.

[0148] like Figure 4 , which is the accuracy curve of training and testing in the embodiment of the present invention. The figure shows the change of the accuracy of the model during the training process. It can be observed that with the increase of training rounds, the accuracy of training and testing gradually increases, indicating that the model is constantly learning and optimizing;

[0149] Step 8: According to the value of the loss function, the back propagation algorithm is applied to calculate the gradient of each parameter layer by layer; the optimizer is used to adjust the weights and biases of the multi-source fusion residual network model according to the gradient of each parameter to minimize the loss function; the parameters of the multi-source fusion residual network model are continuously updated until the multi-source fusion residual network model converges to obtain the trained multi-source fusion residual network model;

[0150] In this embodiment, the training process includes multiple iterations (epochs) of the data set, by repeatedly updating the model parameters, gradually reducing the loss function value until the model converges, saving the model parameters, and obtaining the trained multi-source fusion residual network model;

[0151] like Figure 5 As shown in FIG. 1 , it is a loss curve of training and testing in an embodiment of the present invention. The figure shows the change trend of the loss value of the model with the training round. As the training progresses, the loss gradually decreases, verifying the convergence and stability of the model.

[0152] Step 9: Input the normalized two-dimensional matrix in the test set into the trained multi-source fusion residual network model. The value corresponding to the normalized two-dimensional matrix in the test set is the fault diagnosis result.

[0153] In this embodiment, by integrating the network structure of multi-scale convolution modules and attention mechanism, the model can effectively extract local and global features of vibration and sound signals and realize accurate classification of rolling bearing faults;

[0154] Figure 6 The confusion matrix of the test set is shown, which shows the classification effect of the model on each fault category. The confusion matrix clearly shows the accurate classification number of each type of fault, verifying the high-precision fault diagnosis ability of the model;

[0155] To further verify the separability of feature extraction, Figure 7The T-SNE visualization results of the fully connected layer are shown. The figure represents high-dimensional features in two-dimensional space through dimensionality reduction, clearly showing the distinction between different fault categories, and further confirming the good distribution and classification ability of the features extracted by the model on different categories.

[0156] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.

Claims

1. A rolling bearing fault diagnosis system based on multi-source fusion residual network, characterized in that: It includes a rolling bearing initial data acquisition module, a rolling bearing data preprocessing module, a feature extraction module, a feature fusion module, a classification module and a parameter updating module; The rolling bearing initial data acquisition module comprises a plurality of sensors, which collect multi-sensor signal data of the rolling bearing through the sensors, wherein the multi-sensor signal data comprises vibration signals and sound signals under five different fault states, and sends the multi-sensor signal data to the rolling bearing data preprocessing module; The rolling bearing data preprocessing module includes a data splicing module, a normalization processing module and a label allocation module; The data splicing module is used to intercept the vibration signal and the sound signal under each fault state in the multi-sensor signal data, and splice the intercepted vibration signal and the sound signal to obtain multiple two-dimensional matrices of the fault state, and then obtain multiple two-dimensional matrices of five different fault states, and send all the two-dimensional matrices to the normalization processing module; The normalization processing module is used to normalize the data in each two-dimensional matrix to obtain a normalized two-dimensional matrix, and send the normalized two-dimensional matrix to the label allocation module; The label assignment module is used to assign labels to the normalized two-dimensional matrix, and the labels represent the fault states corresponding to the normalized two-dimensional matrix; the label assignment module is also used to send the normalized two-dimensional matrix and its corresponding labels to the feature extraction module; The feature extraction module is used to extract features from the normalized two-dimensional matrix to obtain a downsampled high-order feature map, and input the downsampled high-order feature map into the feature fusion module; The feature fusion module includes a group cross attention mechanism and a flattening layer, which processes the downsampled high-order feature map through the group cross attention mechanism to obtain a merged feature map, and sends the merged feature map to the flattening layer, which is used to flatten the merged feature map into a one-dimensional vector, and sends the flattened one-dimensional vector to the classification module; The classification module includes a fully connected layer and a softmax layer; the flattened one-dimensional vector is sequentially passed through the fully connected layer and the softmax layer to obtain five first values, each of which represents a predicted value of the probability that the rolling bearing is in a fault state, and the maximum value of the five first values ​​is obtained, and then the second value is obtained, the second value represents the fault state corresponding to the maximum value, and the second value is sent to the parameter update module; The parameter updating module is used to compare the second value and the label of the normalized two-dimensional matrix, and calculate the error between the two, to determine whether the error between the second value and the label of the normalized two-dimensional matrix meets the preset conditions. When the error between the second value and the label of the normalized two-dimensional matrix meets the preset conditions, it is characterized that the rolling bearing fault diagnosis system based on the multi-source fusion residual network can obtain the rolling bearing fault diagnosis result. When the error between the second value and the label of the normalized two-dimensional matrix does not meet the preset conditions, the parameters in the feature extraction module, the feature fusion module and the classification module are updated according to the error between the second value and the label of the normalized two-dimensional matrix through the error back propagation algorithm until the error between the second value and the label of the normalized two-dimensional matrix meets the preset conditions.

2. The rolling bearing fault diagnosis system based on multi-source fusion residual network according to claim 1 is characterized in that: When the error between the second value and the label of the normalized two-dimensional matrix meets a preset condition, the method further includes: The multi-sensor signal data of the rolling bearing to be tested is collected by the rolling bearing initial data acquisition module, and then processed by the data splicing module and the normalization processing module in the rolling bearing data preprocessing module to obtain a two-dimensional matrix of the rolling bearing to be tested, and then successively passed through the feature extraction module, the feature fusion module and the classification module to obtain the target value of the rolling bearing to be tested, and the target value is used as the fault diagnosis result.

3. The rolling bearing fault diagnosis system based on multi-source fusion residual network according to claim 1 is characterized in that: The feature extraction module includes residual block 1, average pooling layer 1, residual block 2, average pooling layer 2, residual block 3, and average pooling layer 3; Among them, feature extraction is performed on the normalized two-dimensional matrix to obtain a downsampled high-order feature map, including: The normalized two-dimensional matrix is ​​input into residual block 1, which performs shallow feature extraction on the normalized two-dimensional matrix to obtain a preliminary feature map, and the preliminary feature map is input into average pooling layer 1, which downsamples the preliminary feature map to obtain a downsampled preliminary feature map, and the downsampled preliminary feature map is input into residual block 2, which further extracts features from the downsampled preliminary feature map to obtain an intermediate feature map, and the intermediate feature map is input into average pooling layer 2, which downsamples the intermediate feature map, and the downsampled intermediate feature map is input into residual block 3, which performs high-order feature extraction on the downsampled intermediate feature map to obtain a high-order feature map, and the high-order feature map is input into average pooling layer 3, which downsamples the high-order feature map to obtain a downsampled high-order feature map.

4. The rolling bearing fault diagnosis system based on multi-source fusion residual network according to claim 3 is characterized in that: The residual block 1 includes two convolutional layers 1, two batch normalization layers, two ReLU activation layers and a residual connection module, and the convolutional layer 1 includes a one-dimensional convolution kernel with a convolution kernel size of a first size; The residual block 2 includes two convolutional layers 2, two batch normalization layers, two ReLU activation layers and a residual connection module, and the convolutional layer 2 includes a one-dimensional convolution kernel with a convolution kernel size of the second size; The residual block 3 includes two convolutional layers 3, two batch normalization layers, two ReLU activation layers and a residual connection module layer, and the convolutional layer 3 includes a one-dimensional convolution kernel with a convolution kernel size of a third size.

5. The rolling bearing fault diagnosis system based on multi-source fusion residual network according to claim 1 is characterized in that: The grouped cross attention mechanism includes a grouping layer, a global average pooling layer, a convolution layer, a softmax layer, a grouping weighted layer, a multi-scale convolution layer and a merging layer; Among them, the downsampled high-order feature map is processed through the group cross attention mechanism to obtain the merged feature map, including: The grouping layer is used to receive the downsampled high-order feature map, divide the high-order feature map into three groups of high-order feature maps according to the type of sensor, and send the three groups of high-order feature maps to the global average pooling layer and the grouping weighted layer; The global average pooling layer is used to perform a global average pooling operation on each group of high-order feature maps to obtain three groups of feature maps after the global average pooling operation, and send the three groups of feature maps after the global average pooling operation to the convolution layer; The convolution layer is used to perform 1×1 convolution and sigmoid function nonlinear activation on each group of feature maps after the global average pooling operation, obtain the attention weight of each group, and then obtain three groups of attention weights, and send the three groups of attention weights to the softmax layer; The softmax layer is used to normalize the attention weights of each group to obtain three groups of normalized attention weights, and the three groups of normalized attention weights are sent to the group weighting layer; The group weighted layer is used to multiply the normalized attention weights and the grouped feature maps belonging to the same group channel by channel to obtain three groups of weighted feature maps, and send the three groups of weighted feature maps to the multi-scale convolutional layer; The multi-scale convolution layer is used to perform a 3×3 convolution operation and a 5×5 convolution operation on each group of weighted feature maps, add the feature map after the 3×3 convolution operation and the feature map after the 5×5 convolution operation to obtain a multi-scale feature map, and then obtain three groups of multi-scale feature maps, and send the three groups of multi-scale feature maps to the merging layer; The merging layer is used to splice the three sets of multi-scale feature maps along the sensor dimension, restore and merge them into feature maps of original size, and obtain a merged feature map.

6. The rolling bearing fault diagnosis system based on multi-source fusion residual network according to claim 5 is characterized in that: The grouped cross attention mechanism is implemented by the following formula: According to the type of sensor, the high-order feature map is divided into three groups of high-order feature maps. A global average pooling operation is performed on each group of high-order feature maps to obtain a feature map after the global average pooling operation, which is specifically expressed by the following formula: G i =GlobalAvgPool(X i ); Among them, X i represents the i-th group of high-order feature maps, G i Represents the feature map after the global average pooling operation corresponding to the i-th group of high-order feature maps; Perform 1×1 convolution and sigmoid function nonlinear activation on each group of feature maps after the global average pooling operation to obtain the attention weight of each group, which is specifically implemented by the following formula: A i =σ(Conv 1×1 (G i )); Among them, σ is the sigmoid function, A i is the attention weight of the i-th group; The attention weights of each group are normalized using the following formula: in, represents the normalized attention weight of the i-th group, exp(A i ) represents the attention weight A i Applying the exponential function, A i,c represents the cth channel of the i-th group of attention weights, C represents the number of channels, represents the exponential summation of all channels of the i-th group of attention weights; Each group of high-order feature maps X i And the normalized attention weights Multiply each channel to generate a weighted feature map, which is specifically expressed by the following formula: in, represents element-wise multiplication, X i ′ represents the weighted feature map of the i-th group; For each group of weighted feature maps, 3×3 convolution operations and 5×5 convolution operations are performed respectively, and the feature maps after the 3×3 convolution operation are added to the feature maps after the 5×5 convolution operation to obtain a multi-scale feature map, which is specifically implemented by the following formula: X i multi =Conv 3×3 (X i ′)+Conv 5×5 (X i ′); Among them, X i multi Represents the multi-scale feature map of the i-th group; The three sets of multi-scale feature maps are spliced ​​along the sensor dimension and restored to the feature map of the original size to obtain the merged feature map, which is specifically achieved by the following formula: X output =Concat(X1 multi ,X2 multi ,X3 multi ); Among them, X output Represents the merged feature map.

7. A rolling bearing fault diagnosis method based on a multi-source fusion residual network, implemented based on the rolling bearing fault diagnosis system based on a multi-source fusion residual network in claim 1, characterized in that: include: Step 1: Collect multi-sensor signal data of the rolling bearing through a sensor, wherein the multi-sensor signal data includes vibration signals and sound signals under five different fault states; Step 2: intercept and splice the vibration signal and sound signal of each fault state to obtain multiple two-dimensional matrices of the fault state, and then obtain multiple two-dimensional matrices of five different fault states; Step 3: Normalize each two-dimensional matrix to obtain a normalized two-dimensional matrix; Step 4: assign a label to each normalized two-dimensional matrix, wherein the label represents the fault state corresponding to the normalized two-dimensional matrix, take the normalized two-dimensional code matrix as an input sample, take the label corresponding to the normalized two-dimensional code matrix as an output sample, and obtain multiple samples, and all samples constitute a sample set; Step 5: Divide the sample set into training set and test set; Step 6: inputting the input samples in the training set into the multi-source fusion residual network model to obtain a second value, wherein the multi-source fusion residual network model is composed of the feature extraction module, the feature fusion module and the classification module; Step 7: Input the second value and the output sample of the normalized two-dimensional matrix into a loss function to obtain a loss function value, where the loss function is sparse classification cross entropy; Step 8: According to the value of the loss function, the back propagation algorithm is applied to calculate the gradient of each parameter layer by layer; the optimizer is used to adjust the weights and biases of the multi-source fusion residual network model according to the gradient of each parameter to minimize the loss function; the parameters of the multi-source fusion residual network model are continuously updated until the multi-source fusion residual network model converges to obtain the trained multi-source fusion residual network model; Step 9: Input the normalized two-dimensional matrix in the test set into the trained multi-source fusion residual network model to obtain the numerical value corresponding to the normalized two-dimensional matrix in the test set, that is, the fault diagnosis result.

Images