Rolling bearing fault diagnosis method based on time-frequency image and transfer learning

Abstract

The invention discloses a rolling bearing fault diagnosis method based on a time-frequency image and transfer learning. The method comprises the following steps: S1, collecting an original rolling bearing vibration signal; s2, converting the vibration signal of the rolling bearing into a time-frequency image; s3, dividing the time-frequency image into a training set, a verification set and a test set, and constructing a convolutional neural network fault diagnosis model; s4, normalizing the sample data of the training set and the verification set, inputting the normalized sample data into the network model, and carrying out iterative training to obtain an optimal model; inputting the sample data of the test set into the optimal model for classification to obtain a model diagnosis result; s5, converting the attention superposition thermodynamic diagram into a weight matrix; and S6, calculating the weight value of each fault sample signal, evaluating the closeness degree of the weight value, and determining the sample type. The invention belongs to the technical field of mechanical state monitoring and fault diagnosis, and solves the problems of insufficient interpretability and non-transparent process of an existing deep learning model in rolling bearing fault diagnosis.

Description

Technical Field

[0001] The present invention belongs to the technical field of mechanical condition monitoring and fault diagnosis, and relates to a rolling bearing fault diagnosis method based on time-frequency images and transfer learning. Background Art

[0002] In recent years, deep learning has achieved remarkable results in the field of mechanical condition monitoring and fault diagnosis, especially in feature extraction and pattern recognition. However, the model methods of deep learning are accompanied by problems such as unclear mechanism and poor interpretability, making it often impossible to understand the reasons for the model judgment behind it in practical applications and difficult to form a scientific decision-making basis.

[0003] With the introduction of the attention mechanism into the neural network, by dynamically allocating different weights to input features during the training process and allowing the model to focus on sensitive areas in the data according to the weight sizes, the recognition ability of the neural network for key fault information is effectively improved, thereby improving the performance of the model. By introducing the attention mechanism, it is possible to visualize the attention weight overlay heat map during the training process and clarify which input features play a key role in the model's decision-making during the training of the deep learning model. On this basis, a weight matrix is generated through obtaining the attention overlay heat map of the sample, binary processing by Otsu's method, normalization processing, etc., for realizing the classification decision of unknown bearing samples. That is, when performing fault prediction classification, the input image is multiplied by this weight matrix, so as to strengthen the influence of key features and suppress the interference of unimportant information, and realize mechanical condition monitoring and fault diagnosis. By introducing the attention mechanism, not only can the accuracy of fault recognition of the deep learning model be optimized, but also it can help to understand the judgment basis of the deep learning model, provide a certain degree of interpretability for the deep learning model, and enhance its credibility in practical applications. Summary of the Invention

[0004] The present invention provides a rolling bearing fault diagnosis method based on time-frequency images and transfer learning, which solves the problems of insufficient interpretability and opaque process in the rolling bearing fault diagnosis by existing deep learning models.

[0005] The technical solution adopted by the present invention is that the rolling bearing fault diagnosis method based on time-frequency images and transfer learning is implemented according to the following steps: S1. Collect the original rolling bearing vibration signal; S2. Convert the rolling bearing vibration signal into a time-frequency image; S3. Divide various time-frequency images into a training set, a validation set and a test set, and construct a convolutional neural network fault diagnosis model; S4. Normalize the sample data of the training set and the validation set, and then input them into the constructed network model. Through iterative training, obtain the best model of the network model; input the sample data of the test set into the best model for classification to obtain the model diagnosis result; S5. Obtain the sample attention superposition heat map, and convert the attention superposition heat map into a weight matrix; S6. Calculate the weight values of various fault sample signals. According to the weight calculation result of the unknown bearing sample signal, evaluate which type of fault sample has the closest weight value, and then determine the sample type.

[0006] The beneficial effects of the present invention specifically include the following aspects: 1) The present invention proposes an attention convolutional neural network model based on squeeze-and-excitation (hereinafter referred to as the SE-CNN model). During the fault diagnosis process relying on the SE-CNN model, when grasping the time-frequency feature regions of the fault information concerned by the model in the stage of obtaining the attention heat map of features, by analyzing the change rules of the heat maps in each stage, it is found that the SE-CNN model does not rely on the size distribution rule of the time-frequency domain energy of the original fault signal during fault diagnosis, but focuses on the differentiated high-frequency regions, and at the same time performs weighted processing on the features of multiple regions, with obvious effects. 2) By calculating the attention superposition threshold map and the weight matrix, the feature extraction and classification decision-making are closely combined, which not only reduces the unnecessary computational burden, simplifies the process of fault prediction and classification, but also improves the response speed of the deep learning model. When making a classification decision for unknown samples, only continuous wavelet transform and feature weighting processing need to be performed on the unknown samples to quickly and accurately achieve their fault classification. 3) The method of the present invention takes the rolling bearing as a specific research object, but its principle and method have certain scalability, and according to the training results of the model, it is verified that the model has strong generalization ability in different types of mechanical fault diagnosis, which is crucial for actual industrial applications. This method provides a reliable tool for the real-time monitoring and diagnosis system in the field of mechanical condition monitoring and fault diagnosis, and also brings more possibilities for industrial automation and intelligence. Description of the Drawings

[0007] Figure 1 is the flow chart of the rolling bearing fault diagnosis method of the present invention; Figure 2 is the network structure diagram of the SE-CNN of the rolling bearing fault diagnosis method of the present invention; Figure 3 is the flow chart for realizing the classification decision of unknown bearing samples of the method of the present invention; Figure 4 is the original signal image of category 1 obtained in Embodiment 1 of the present invention; Figure 5 is the time-frequency image of category 1 obtained in Embodiment 1 of the present invention; Figure 6 It is the attention superposition heat map of category 1 obtained in Embodiment 1 of the present invention; Figure 7 It is the normalized total threshold map of category 1 obtained in Embodiment 1 of the present invention; Figure 8 It is the weight matrix image of category 1 obtained in Embodiment 1 of the present invention. Detailed implementation manners

[0008] The present invention will be described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0009] Refer to Figure 1 , the rolling bearing fault diagnosis method based on time-frequency images and transfer learning of the present invention is implemented according to the following steps: S1. Collect the original rolling bearing vibration signal. The specific process is as follows: The original rolling bearing vibration signal includes normal bearing signals and different types of fault signals. The collected signal is based on the vibration signal obtained by an acceleration sensor, and the measuring point position is on the wall panel or bearing housing near the bearing, including but not limited to vibration signals in the horizontal and vertical directions.

[0010] S2. Use continuous wavelet transform, Hilbert-Huang transform, and short-time Fourier transform to convert the original one-dimensional vibration signal of the rolling bearing into a time-frequency image (time-frequency information distribution image). The specific process is as follows: Use the three methods of continuous wavelet transform, Hilbert-Huang transform, and short-time Fourier transform as the time-frequency image acquisition methods to convert the vibration signal into a time-frequency image. Before analyzing the three obtained time-frequency images, confirm that they are RGB three-channel digital images with an image size of not less than 128×128; perform modeling, feature extraction, and fault diagnosis on the three-channel digital images respectively. At this time, it is necessary to compare the respective weight relationships corresponding to the three-channel digital images; (Note: In special cases, that is, in a mechanical system with a single working condition and fewer mechanical parts, 1 or 2 of the images can be selected). The calculation formula of continuous wavelet transform is as shown in Equation (1): (1) Among them, represents the original signal, t is the time variable, x is the original signal, a is the scale factor, b is the translation factor, represents the wavelet function, represents the complex conjugate of the wavelet function; The continuous wavelet transform uses the Morlet wavelet basis as the basis function, and the calculation formula is as shown in Equation (2): (2) Among them, is the center frequency; The Hilbert-Huang transform consists of two parts: empirical mode decomposition and Hilbert spectral analysis. Empirical mode decomposition decomposes the signal into several intrinsic mode functions, and then applies the Hilbert transform to each intrinsic mode function to obtain the instantaneous frequency and amplitude; Empirical mode decomposition is an iterative process used to decompose the original signal x ( t ) into multiple intrinsic mode function components, and the specific operation method is as follows: ① Find the extreme points: Find all local maxima and minima in the original signal; ② Generate the envelope: Generate the upper envelope (i.e., the envelope of the maxima) and the lower envelope (i.e., the envelope of the minima) through interpolation; ③ Calculate the mean envelope: Calculate the average of the upper envelope and the lower envelope ; ④ Extract the first intrinsic mode function: Subtract the envelope mean from the original signal x ( t ) to obtain . If meets the intrinsic mode function condition, it is used as the first intrinsic mode function; Otherwise, repeat the above steps until the condition is met; ⑤ Remaining signal: Use as the new input signal, and repeat the above process until the remaining signal becomes a monotonic function or its standard deviation is lower than the set threshold; Since then, the expression of the original signal ( x ) is converted to: t (3) (3) For each intrinsic mode function component , apply the Hilbert transform to obtain its analytic signal , and the expression is: (4) where j is the imaginary unit; Given an original signal x ( t ) and a window function w ( t ), the calculation formula of the short-time Fourier transform is as shown in Equation (5): (5) where represents the sliding window position on the time axis, is the angular frequency, is the window function at position .

[0011] S3. Data preprocessing: Divide the time-frequency images of various bearings into a training set, a validation set, and a test set, and construct a convolutional neural network fault diagnosis model based on the SE attention module. The specific process is as follows: Preferably, divide the obtained time-frequency images into a training set, a validation set, and a test set at a ratio of 6:2:2; Refer to Figure 2 , construct a convolutional neural network fault diagnosis model based on the SE attention module, including an input layer, a convolutional layer, an SE attention module layer, a pooling layer, and a fully connected layer; In particular, if the time-frequency images of 2 or 3 classes are used as the input of the model, an input layer is established for each class of time-frequency images to ensure that the model can focus on the features of each class of time-frequency images, thereby improving the accuracy and robustness of fault diagnosis. The construction of the convolutional neural network fault diagnosis model based on the SE attention module is described as follows: ① Receive the time-frequency images of the bearing signals through the input layer; ② Extract different features in the images through the convolutional layer, and enhance the non-linear expression ability through the ReLu activation function. The calculation formula is as shown in Equation (6): (6) where, represents the output of the ReLu activation function; x represents the input value; represents taking the larger value between zero and the input value; ③ Construct an SE attention module after each convolutional layer to effectively enhance the attention degree of the convolutional neural network model to the fault-sensitive features, and then reduce the influence of irrelevant information. The SE attention module adds a content-aware mechanism to compress the feature map into a single value, enabling the network model to adaptively adjust the weight of each feature map, thereby obtaining a global understanding of each channel and improving the mutual dependence of channels at a very small computational cost.

[0012] ④ The pooling layer reduces the size of the feature map through downsampling operations to improve computational efficiency. The specific method is the max pooling operation; ⑤ The fully connected layer integrates the extracted features and calculates the predicted output result through weights. Specifically, the Softmax function is used to predict the fault classification. The calculation formula is as shown in Equation (7): (7) where, is the i th element of the input vector, N is the total number of classes.

[0013] S4. Normalize the sample data of the training set and the validation set and input them into the constructed network model. After multiple iterative trainings, obtain the best model of this network model; then input the sample data of the test set into the trained best model for classification to obtain the final model diagnosis result. The specific process is as follows: First, perform normalization processing on the sample data of the training set, the validation set, and the test set to ensure that the feature values are within a unified range. Then, input the data of the training set and the validation set into the constructed convolutional neural network fault diagnosis model based on the SE attention module; through multiple iterative trainings, it is recommended to select the Adam backpropagation algorithm as the model optimizer and ReLu as the activation function, and set the hyperparameters batch size and learning rate. At the end of each training cycle, use the validation set to evaluate the performance of the convolutional neural network fault diagnosis model and save the best-performing model version; finally, input the data of the test set into this best model version for classification to obtain the final fault diagnosis result.

[0014] S5. Obtain the attention superimposed heat map of the sample, and through Otsu's method for binary processing and normalization processing, convert the attention superimposed heat map into a weight matrix to realize the weight calculation of the unknown bearing sample signal, as Figure 3 shown. The specific process of step S5 is as follows: S51. Obtain the attention superimposed heat map: For the three types of time-frequency images of the continuous wavelet transform, Hilbert-Huang transform, and short-time Fourier transform of the input rolling bearing, use the trained convolutional neural network fault diagnosis model based on the SE attention module to statistically analyze the regions with higher weights in the SE attention module of this convolutional neural network fault diagnosis model, and obtain the attention superimposed heat map of all samples; S52. Determine the optimal threshold through binary processing: Apply Otsu's method to perform binary processing on the attention superimposed heat maps obtained for the three types of time-frequency images respectively. Otsu's method will automatically select a threshold to divide the image into two categories, foreground and background, so as to determine the optimal threshold of the attention superimposed heat map of the bearing; then determine the optimal segmentation threshold by maximizing the between-class variance to divide the image into two categories, foreground and background, as described below: ① Calculate the gray histogram of the attention superimposed heat map of each type of bearing , where i represents the gray level, with a range of [0, L - 1], L is the total number of gray levels (usually set to 256), and then count the number of pixels for each gray level; ② Initialize the total number of pixels M , gray level L , between-class variance and the optimal threshold ; ③ For each possible thresholds , the threshold s ranges from 0 to L -1, and calculate the cumulative probabilities and means of the background and foreground; The cumulative probability of the background and the mean are calculated as follows: (8) (9) The cumulative probability of the foreground and the mean are calculated as follows: (10) (11) ④ Calculate the between-class variance , and the calculation formula is as follows: (12) ⑤ Traverse all possible thresholds s , find the threshold that maximizes the between-class variance , and the calculation formula is as follows: (13) ⑥ Set the pixel values greater than in the attention superposed heat map to 1, which is called the foreground; at the same time, set the part less than or equal to to 0, which is called the background. Thus, the precise segmentation of the key feature area in the rolling bearing fault diagnosis is realized; S53. Generate the total threshold superposed map: Under the same type of bearing, add all the attention superposed heat maps after binarization according to the pixel positions to obtain the total threshold superposed map of the attention superposed heat map of this type, and convert it into a grayscale image; S54. Normalization processing: Process each class of superposed threshold map (this is the threshold map of a single class, and the three time-frequency maps are also calculated separately) using the min-max normalization method. Specifically, calculate the minimum and maximum values of all pixel values of the grayscale image of each class, and then linearly map each pixel value to the range of [0, 1] to ensure that all images can be compared on the same scale. The normalization calculation formula is as shown in Equation (14): (14) where, is the pixel value before normalization, is the pixel value after normalization; S55. Calculate the weight matrix: According to the above-mentioned normalized total threshold superposed map, calculate a weight matrix with the same dimensionW ; specifically, for each category of original time-frequency images X , by performing the Hadamard product (element-wise multiplication) operation with the weight matrix, the final total threshold superposition map is obtained, and the calculation formula is as shown in Equation (15): (15) where, denotes the Hadamard product, that is, element-wise multiplication, is the calculated total threshold superposition map, is the original time-frequency image.

[0015] S6. Calculate the weight values of each type of fault sample signal using Step S5. Subsequently, based on the calculated weight results of the unknown bearing sample signal, evaluate which type of fault sample's weight value it is close to, and then determine that it belongs to that type of sample. The specific process is as follows: S61. Preprocess the unknown sample: For the unknown bearing sample, first perform continuous wavelet transform, Hilbert-Huang transform, and short-time Fourier transform on it to obtain its corresponding 3 types of time-frequency images; S62. Calculate the weight map of the unknown sample: For each type of time-frequency image, perform the Hadamard product operation with the weight matrix to obtain the attention superposition heat map of the unknown sample; S63. Classification decision: According to the attention superposition heat map of the unknown sample, evaluate which type of known fault sample's attention superposition heat map it is closest to, and then determine which type of fault sample the unknown sample belongs to, thus completing. The fault categories of the rolling bearings of the method of the present invention are divided into six categories: Category 1 is the bearing with cage fault, Category 2 is the bearing with inner ring fault, Category 3 is the bearing with outer ring fault, Category 4 is the normal bearing, Category 5 is the bearing with rolling element fault, and Category 6 is the bearing with compound fault of inner ring and outer ring.

[0016] Example 1 The diagnostic object of this Example 1 is the bearing of model PH205, as Figure 1 , and it is implemented according to the following steps: S1. Collect the original vibration signal of the rolling bearing, see Figure 4 as the original signal image of Category 1; S2. Use three methods, namely continuous wavelet transform, Hilbert-Huang transform, and short-time Fourier transform, as the time-frequency image acquisition methods to convert the vibration signal into a time-frequency image, as Figure 5As shown, when obtaining the time-frequency images of the three changes, it is necessary to confirm that they are RGB three-channel digital images before analysis, and the image size is not less than 128×128; for the three-channel digital images, modeling, feature extraction, and fault diagnosis are carried out respectively. At this time, it is necessary to compare the respective weight relationships corresponding to the three-channel digital images; this Embodiment 1 does not belong to a mechanical system with a single working condition and fewer mechanical parts, so 3 types of time-frequency images are selected; S3. First, divide the time-frequency images into a training set, a validation set, and a test set according to a division ratio of 6:2:2; then, construct a convolutional neural network fault diagnosis model based on the SE attention module, as Figure 2 shown; in this Embodiment 1, 3 types of time-frequency images are used as the input of the model, and an input layer is established for each type of image respectively to ensure that the model can focus on the features of each type of time-frequency image, thereby improving the accuracy and robustness of fault diagnosis; S4. First, normalize the sample data of the training set, validation set, and test set to ensure that the feature values are in a unified range; then, input the sample data of the training set and validation set into the constructed convolutional neural network fault diagnosis model; the image size input into the input layer of the convolutional neural network model is (64, 64, 3), that is, the width and height of the input image are 64 pixels each, and 3 is the three color channels of RGB of the image. The filter of the first convolutional layer is 32, and the output is (64, 64, 32), that is, the width and height of the original image are retained, and 32 feature maps are generated. The output of the first max-pooling layer is (32, 32, 32); the second convolutional filter is 64, and the output is (32, 32, 64). The output of the second max-pooling layer is (16, 16, 64); the third convolutional filter is 128, and the output is (16, 16, 128). The output of the third max-pooling layer is (8, 8, 128). Through multiple iterative trainings, the Adam algorithm of backpropagation is selected as the model optimizer, ReLu is used as the activation function, the hyperparameters batch size is set to 32 and the learning rate is set to 0.001. The validation set is used to evaluate the model performance at the end of each training cycle, and the best-performing model version is saved; finally, the sample data of the test set is input into this best model version for classification, so as to obtain the final fault diagnosis result; S5. As Figure 3 shown, the specific process is: S51. Obtain the attention overlay heat map: For the 3 types of time-frequency images of the continuous wavelet transform, Hilbert-Huang transform, and short-time Fourier transform of the input rolling bearing, use the trained convolutional neural network fault diagnosis model to statistically analyze the regions with higher weights in the SE attention module of the convolutional neural network fault diagnosis model, so as to obtain the attention overlay heat map of all samples. Taking the continuous wavelet transform as an example, simplify the attention overlay heat map of category 1 into a 6×6 matrix, as Figure 6 shown, which is the attention overlay heat map of category 1,

[0017] Simplify the attention superposition heat map of category 2 into a 6×6 matrix as follows:

[0018] Simplify the attention superposition heat map of category 3 into a 6×6 matrix as follows:

[0019] Simplify the attention superposition heat map of category 4 into a 6×6 matrix as follows:

[0020] Simplify the attention superposition heat map of category 5 into a 6×6 matrix as follows:

[0021] Simplify the attention superposition heat map of category 6 into a 6×6 matrix as follows:

[0022] S52. Binarization processing to determine the optimal threshold; For three types of time-frequency images, obtain all the attention superposition heat maps respectively, and apply Otsu's method for binarization processing. Otsu's method will automatically select a threshold to segment the image into two categories, foreground and background, so as to determine the optimal threshold of the attention superposition heat map of the bearing; Determine the optimal segmentation threshold by maximizing the between-class variance, so as to segment the image into two categories, foreground and background. Through calculation, the threshold value of the faulty bearing of category 1 is 0.70; The threshold value of the faulty bearing of category 2 is 0.72; The threshold value of the faulty bearing of category 3 is 0.76; The threshold value of the faulty bearing of category 4 is 0.66; The threshold value of the faulty bearing of category 5 is 0.68; The threshold value of the faulty bearing of category 6 is 0.78; S53. Generate the total threshold superposition map: Under the same category of bearings, add up all the attention superposition heat maps after binarization processing according to the pixel positions to obtain the total threshold superposition map of this category, and convert it into a grayscale image; Taking one attention superposition heat map as an example, the total threshold map is the threshold superposition map in S52, as Figure 7 shown, which is the attention superposition threshold map of category 1; S54. Normalization processing: The simplified matrix after normalization of the attention superposition threshold map of category 1 is:

[0023] The simplified matrix after normalization of the attention superposition threshold map of category 2 is:

[0024] The simplified matrix after normalization of the attention superposition threshold map of category 3 is:

[0025] The simplified matrix after normalization of the attention superposition threshold map of category 4 is:

[0026] The simplified matrix after normalization of the attention superposition threshold map of category 5 is:

[0027] The simplified matrix after normalization of the attention superposition threshold map of category 6 is:

[0028] S55. Calculate the weight matrix: The simplified matrix of the original time-frequency image of category 1 is:

[0029] The simplified matrix of the original time-frequency image of category 2 is:

[0030] The simplified matrix of the original time-frequency image of category 3 is:

[0031] The simplified matrix of the original time-frequency image of category 4 is:

[0032] The simplified matrix of the original time-frequency image of category 5 is:

[0033] The simplified matrix of the original time-frequency image of category 6 is:

[0034] Moreover, the simplified matrix after normalization of the original time-frequency image of category 1 is:

[0035] The simplified matrix after normalization of the original time-frequency image of category 2 is:

[0036] The simplified matrix after normalization of the original time-frequency image of category 3 is:

[0037] The simplified matrix after normalization of the original time-frequency image of category 4 is:

[0038] The simplified matrix after normalizing the original time-frequency image of category 5 is:

[0039] The simplified matrix after normalizing the original time-frequency image of category 6 is:

[0040] As Figure 8 , which is the weight matrix image of category 1, the calculated weight matrix of category 1 is:

[0041] The calculated weight matrix of category 2 is:

[0042] The calculated weight matrix of category 3 is:

[0043] The calculated weight matrix of category 4 is:

[0044] The calculated weight matrix of category 5 is:

[0045] The calculated weight matrix of category 6 is:

[0046] S6. The specific process is as follows: S61. Preprocess the unknown sample: For the unknown bearing sample, first perform continuous wavelet transform, Hilbert-Huang transform, and short-time Fourier transform on it to obtain its corresponding 3 types of time-frequency images; taking the continuous wavelet transform as an example, there is an unknown sample, perform continuous wavelet transform on it, and the obtained time-frequency image has a simplified matrix as:

[0047] The simplified matrix after normalization is:

[0048] S62. Calculate the weight map of the unknown sample: Perform Hadamard product operation on each type of time-frequency map with the weight matrix to obtain the attention superposition heat map of the unknown sample; obtain the attention superposition heat map of the unknown sample, and the simplified matrix is:

[0049] The threshold value is calculated to be 0.721.

[0050] S63. Classification decision: Based on the attention superposition heat map of the unknown sample, evaluate which attention superposition heat map of the known fault samples it is closest to, and determine which type of fault sample the unknown sample belongs to.

[0051] Conclusion of Embodiment 1: The threshold value of the unknown sample is calculated to be 0.721, which is close to the attention superposition heat map and threshold value of the known fault sample category 2 in step S52. It can be determined that the fault type of the unknown sample is the inner ring fault of category 2.

[0052] Embodiment 2 The diagnostic object of this Embodiment 2 is the bearing of model PH205. Based on the aforementioned six types of fault classifications, it is implemented according to the following steps: S1. Collect the original vibration signal of the rolling bearing; S2. Use continuous wavelet transform, Hilbert-Huang transform, and short-time Fourier transform to convert the original one-dimensional vibration signal of the rolling bearing into a time-frequency information distribution image; in a mechanical system with a single working condition and fewer mechanical parts, 1 or 2 images can be selected. Taking the short-time Fourier transform as an example, the simplified matrix of the original time-frequency image of category 1 is:

[0053] The simplified matrix of the original time-frequency image of category 2 is:

[0054] The simplified matrix of the original time-frequency image of category 3 is:

[0055] The simplified matrix of the original time-frequency image of category 4 is:

[0056] The simplified matrix of the original time-frequency image of category 5 is:

[0057] The simplified matrix of the original time-frequency image of category 6 is:

[0058] S3. Data preprocessing, divide the time-frequency images of various bearings into training sets, validation sets, and test sets, and then construct a convolutional neural network fault diagnosis model based on the SE attention module; S4. Normalize the training set and validation set sample data and input them into the network model. After multiple iterative trainings, obtain the optimal model of the network. Then, input the test set sample data into the trained optimal model for classification to obtain the final model diagnosis result. The simplified matrix after normalizing the original time-frequency image of category 1 is:

[0059] The simplified matrix after normalizing the original time-frequency image of category 2 is:

[0060] The simplified matrix after normalizing the original time-frequency image of category 3 is:

[0061] The simplified matrix after normalizing the original time-frequency image of category 4 is:

[0062] The simplified matrix after normalizing the original time-frequency image of category 5 is:

[0063] The simplified matrix after normalizing the original time-frequency image of category 6 is:

[0064] S5. Obtain the attention overlay heat map of the sample. Through the binarization processing and normalization processing steps of Otsu's method, convert the heat map into a weight matrix to realize the weight calculation of the unknown bearing sample signal. The simplified matrix of the attention overlay heat map of category 1 is:

[0065] The simplified matrix of the attention overlay heat map of category 2 is:

[0066] The simplified matrix of the attention overlay heat map of category 3 is:

[0067] The simplified matrix of the attention overlay heat map of category 4 is:

[0068] The simplified matrix of the attention overlay heat map of category 5 is:

[0069] The simplified matrix of the attention overlay heat map of category 6 is:

[0070] The simplified matrix after normalizing the attention overlay heat map of Category 1 is as follows:

[0071] The simplified matrix after normalizing the attention overlay heat map of Category 2 is as follows:

[0072] The simplified matrix after normalizing the attention overlay heat map of Category 3 is as follows:

[0073] The simplified matrix after normalizing the attention overlay heat map of Category 4 is as follows:

[0074] The simplified matrix after normalizing the attention overlay heat map of Category 5 is as follows:

[0075] The simplified matrix after normalizing the attention overlay heat map of Category 6 is as follows:

[0076] The calculated weight matrix of Category 1 is as follows:

[0077] The calculated weight matrix of Category 2 is as follows:

[0078] The calculated weight matrix of Category 3 is as follows:

[0079] The calculated weight matrix of Category 4 is as follows:

[0080] The calculated weight matrix of Category 5 is as follows:

[0081] The calculated weight matrix of Category 6 is as follows:

[0082] The calculated threshold value of the faulty bearing of Category 1 is 0.75; the calculated threshold value of the faulty bearing of Category 2 is 0.65; the calculated threshold value of the faulty bearing of Category 3 is 0.70; the calculated threshold value of the faulty bearing of Category 4 is 0.77; the calculated threshold value of the faulty bearing of Category 5 is 0.67; the calculated threshold value of the faulty bearing of Category 6 is 0.72; S6. Calculate the weight values of various fault sample signals using the result of step S5. Subsequently, based on the weight calculation result of the unknown bearing sample signal, evaluate which type of fault sample's weight value it is close to, and then determine that it belongs to this type of sample. Taking the short-time Fourier transform as an example, there is an unknown sample. Perform the short-time Fourier transform on it to obtain its time-frequency image. The simplified matrix is as follows:

[0083] The simplified matrix after normalization is as follows:

[0084] It is obtained that the attention superimposed heat map of the unknown sample is similar to that of sample category 2. The simplified matrix is as follows:

[0085] The threshold value calculated through it is 0.649.

[0086] Conclusion of this Example 2: Through the method based on time-frequency analysis and transfer learning, the threshold value of the unknown sample is calculated to be 0.649, which is close to the attention superimposed heat map and threshold value of the known fault sample category 2 at the end of step S5. It can be judged that the fault type of the unknown sample is the inner ring fault of category 2.

[0087] Example 3 The diagnostic object of this Example 3 is the bearing of model PH205. Based on the aforementioned six types of fault classifications, in a mechanical system with a single working condition and fewer mechanical parts, 2 types of images are selected. The implementation steps are as follows: S1. Collect the original vibration signal of the rolling bearing; S2. Use the continuous wavelet transform and short-time Fourier transform to convert the original one-dimensional vibration signal of the rolling bearing into a time-frequency information distribution image; S3. Data preprocessing, divide the time-frequency images of various bearings into a training set, a validation set, and a test set, and construct a convolutional neural network fault diagnosis model based on the SE attention module; S4. Normalize the training set and validation set sample data and input them into the network model. After multiple iterative trainings, obtain the best model of this network; then input the test set sample data into the trained best model for classification to obtain the final model diagnosis result; S5. Obtain the attention superimposed heat map, perform binarization processing to determine the best threshold, generate the total threshold superimposed map, perform normalization processing, and calculate the weight matrix; S6. Calculate the weight values of various fault sample signals using the result of step S5. Subsequently, based on the weight calculation result of the unknown bearing sample signal, evaluate which type of fault sample's weight value it is close to, and then determine that it belongs to this type of sample.

[0088] Conclusion of Embodiment 3: For relevant data, refer to Embodiment 1 and Embodiment 2. Taking continuous wavelet transform as an example, through the method based on time-frequency analysis and transfer learning, the threshold value of the unknown sample is calculated to be 0.721, which is close to the attention overlay heat map and threshold value of the known fault sample category 2. Thus, it can be judged that the fault type of the unknown sample is the inner ring fault of category 2. Moreover, taking short-time Fourier transform as an example, through the method based on time-frequency analysis and transfer learning, the threshold value of the unknown sample is calculated to be 0.649, which is close to the attention overlay heat map and threshold value of the known fault sample category 2. Thus, it can be judged that the fault type of the unknown sample is the inner ring fault of category 2.

[0089] Embodiment 4 The diagnostic object of this Embodiment 4 is the bearing of model PH205. Based on the foregoing six types of fault classifications, referring to the step process of Embodiment 1, in a mechanical system with a single working condition and fewer mechanical parts, 2 types of images are selected and implemented according to the following steps: S1. Collect the original vibration signal of the rolling bearing; S2. Use Hilbert-Huang transform and short-time Fourier transform to convert the original one-dimensional vibration signal of the rolling bearing into a time-frequency information distribution image; S3. First, divide the time-frequency image into a training set, a validation set, and a test set according to the division ratio of 6:2:2; then, construct a convolutional neural network fault diagnosis model based on the SE attention module; S4. Normalize the sample data of the training set and the validation set and input them into the network model. After multiple iterative trainings, the best model of the network is obtained; then, input the sample data of the test set into the trained best model for classification to obtain the final model diagnosis result; S5. Obtain the attention overlay heat map, perform binary processing to determine the optimal threshold, generate the total threshold overlay map, perform normalization processing, and calculate the weight matrix; S6. Use the weight values of various fault sample signals calculated in step S5, and then, according to the weight calculation result of the unknown bearing sample signal, evaluate which type of fault sample its weight value is close to, and further judge that it belongs to this type of sample.

[0090] Conclusion of Embodiment 4: For relevant data, refer to Embodiment 2. Taking short-time Fourier transform as an example, through the method based on time-frequency analysis and transfer learning, the threshold value of the unknown sample is calculated to be 0.649, which is close to the attention overlay heat map and threshold value of the known fault sample category 2. Thus, it can be judged that the fault type of the unknown sample is the inner ring fault of category 2.

[0091] Embodiment 5 The diagnostic object of this Embodiment 5 is the bearing of model PH205. Based on the foregoing six types of fault classifications, referring to the step process of Embodiment 1, in a mechanical system with a single working condition and fewer mechanical parts, 1 type of image is selected and implemented according to the following steps: S1. Collect the original vibration signal of the rolling bearing; S2. Use the short-time Fourier transform to convert the original one-dimensional vibration signal of the rolling bearing into a time-frequency information distribution image; S3. First, divide the time-frequency image into a training set, a validation set, and a test set according to a division ratio of 6:2:2; then, construct a convolutional neural network fault diagnosis model based on the SE attention module; S4. Normalize the sample data of the training set and the validation set and input them into the network model. After multiple iterative trainings, obtain the best model of the network; then input the sample data of the test set into the trained best model for classification to obtain the final model diagnosis result; S5. Obtain the attention superposition heat map of the sample. Through the binarization processing and normalization processing steps of Otsu's method, convert the heat map into a weight matrix to realize the weight calculation of the unknown bearing sample signal; S6. Use the weight values of various fault sample signals calculated in step S5. Subsequently, according to the weight calculation result of the unknown bearing sample signal, evaluate which class of fault sample its weight value is close to, and then judge that it belongs to this class of samples.

[0092] Conclusion of this Example 5: For relevant data, refer to Example 2. Taking the short-time Fourier transform as an example, through the method based on time-frequency analysis and transfer learning, the threshold value of the unknown sample is calculated to be 0.649, which is close to the attention superposition heat map and the threshold value of the known fault sample category 2. It can be judged that the fault type of the unknown sample is the inner ring fault of category 2.

[0093] Example 6 In this Example 6, the diagnosed object is a bearing of model PH205. Based on the foregoing six types of fault classifications, referring to the step process of Example 1, in a mechanical system with a single working condition and fewer mechanical parts, 1 type of image is selected and implemented according to the following steps: S1. Collect the original vibration signal of the rolling bearing; S2. Use the continuous wavelet transform to convert the original one-dimensional vibration signal of the rolling bearing into a time-frequency information distribution image; S3. First, divide the time-frequency image into a training set, a validation set, and a test set according to a division ratio of 6:2:2; then, construct a convolutional neural network fault diagnosis model based on the SE attention module; S4. Normalize the sample data of the training set and the validation set and input them into the network model. After multiple iterative trainings, obtain the best model of the network; then input the sample data of the test set into the trained best model for classification to obtain the final model diagnosis result; S5. Obtain the attention superposition heat map of the sample. Through the binarization processing and normalization processing steps of Otsu's method, convert the heat map into a weight matrix to realize the weight calculation of the unknown bearing sample signal; S6. Use the weight values of various fault sample signals calculated in step S5. Subsequently, according to the weight calculation result of the unknown bearing sample signal, evaluate which class of fault sample its weight value is close to, and then judge that it belongs to this class of samples.

[0094] Conclusion of this Example 6: For relevant data, refer to Example 1. Taking continuous wavelet transform as an example, through the method based on time-frequency analysis and transfer learning, the threshold value of the unknown sample is calculated to be 0.721, which is close to the attention superposition heat map and threshold value of the known fault sample category 2. Therefore, it can be judged that the fault type of the unknown sample is the inner ring fault of category 2.

Claims

1. A rolling bearing fault diagnosis method based on time-frequency images and transfer learning, characterized in that: Follow these steps to implement: S1, collecting the original rolling bearing vibration signal; S2, converting the rolling bearing vibration signal into a time-frequency image; S3, divide various time-frequency images into training set, validation set and test set, and build a convolutional neural network fault diagnosis model; S4, normalizing the sample data of the training set and the validation set and inputting them into the constructed network model, iteratively training to obtain the best model of the network model; inputting the sample data of the test set into the best model for classification to obtain the model diagnosis result; S5. Obtain the attention superposition heat map of the sample, and convert the attention superposition heat map into a weight matrix; S6. Calculate the weight values ​​of various types of fault sample signals, and based on the weight calculation result of the unknown bearing sample signal, evaluate which type of fault sample weight value is closest to it, and then determine the sample type.

2. The rolling bearing fault diagnosis method based on time-frequency image and transfer learning according to claim 1 is characterized in that: In step S1, the specific process is: The original rolling bearing vibration signal includes normal bearing signals and different types of fault signals. The acquisition signal is based on the vibration signal obtained by the acceleration sensor. The measuring point is located on the wall panel or bearing seat near the bearing, including but not limited to horizontal and vertical vibration signals.

3. The rolling bearing fault diagnosis method based on time-frequency image and transfer learning according to claim 1 is characterized in that: In step S2, the specific process is: Continuous wavelet transform, Hilbert-Huang transform and short-time Fourier transform are used as time-frequency image acquisition methods to convert vibration signals into time-frequency images. The three changes to obtain time-frequency images are confirmed as RGB three-channel digital images before analysis, and the image size is not less than 128×128; modeling, feature extraction and fault diagnosis are performed for the three channel digital images respectively, and at this time, the respective weight relationships corresponding to the three channel digital images need to be compared; The calculation formula of continuous wavelet transform is as follows: （1） in, represents the original signal, t is the time variable, x is the original signal, a is the scale factor, b is the translation factor, represents the wavelet function, represents the complex conjugate of the wavelet function; The continuous wavelet transform uses the Morlet wavelet basis as the basis function, and the calculation formula is as follows: （2） in, is the center frequency; The Hilbert-Huang transform consists of two parts: empirical mode decomposition and Hilbert spectrum analysis. The empirical mode decomposition decomposes the signal into several intrinsic mode functions, and then the Hilbert transform is applied to each intrinsic mode function to obtain the instantaneous frequency and amplitude.

4. The rolling bearing fault diagnosis method based on time-frequency image and transfer learning according to claim 1 is characterized in that: In step S3, the specific process is: The time-frequency images are divided into training set, validation set and test set; a convolutional neural network fault diagnosis model based on the SE attention module is constructed, including input layer, convolution layer, SE attention module layer, pooling layer and fully connected layer.

5. The rolling bearing fault diagnosis method based on time-frequency image and transfer learning according to claim 1 is characterized in that: In step S4, the specific process is: First, the sample data of the training set, validation set, and test set are normalized to ensure that the eigenvalues ​​are in a uniform range; Then the data of the training set and the validation set are input into the constructed convolutional neural network fault diagnosis model based on the SE attention module; Perform multiple iterations of training, use the validation set to evaluate the performance of the convolutional neural network fault diagnosis model at the end of each training cycle, and save the best performing model version; Finally, the data of the test set is input into the best model version for classification to obtain the final fault diagnosis result.

6. The rolling bearing fault diagnosis method based on time-frequency image and transfer learning according to claim 1 is characterized in that: In step S5, the specific process is: S51, obtaining attention superposition heat map; For the three types of time-frequency images of the continuous wavelet transform, Hilbert Huang transform, and short-time Fourier transform of the input rolling bearing, the trained convolutional neural network fault diagnosis model is used to count the areas with higher weights in the SE attention module of the convolutional neural network fault diagnosis model, so as to obtain the attention superposition heat map of all samples; S52, binarization processing to determine the optimal threshold; The attention superposition heat map obtained from the three types of time-frequency images is binarized using Otsu's method, where Otsu's method automatically selects a threshold to segment the image into two categories, foreground and background, thereby determining the optimal threshold of the attention superposition heat map of the bearing; then the optimal segmentation threshold is determined by maximizing the inter-class variance, thereby segmenting the image into two categories, foreground and background, as described below: ① Calculate the grayscale histogram of the attention superposition heat map of various types of bearings ,in i Represents the grayscale level, ranging from [0, L -1], L is the total number of gray levels, and then count the number of pixels at each gray level; ② Initialize the total number of pixels M , Grayscale L , between-class variance and the optimal threshold ; ③For each possible threshold s , threshold s The range is from 0 to L -1, calculate the cumulative probability and mean of background and foreground; Cumulative probability of background and mean The calculation formulas are as follows: （8） （9） Cumulative probability of the prospect and mean The calculation formulas are as follows: （10） （11） ④Calculate the inter-class variance , the calculation formula is as follows: （12） ⑤Traverse all possible thresholds s , find the threshold that maximizes the between-class variance , the calculation formula is as follows: （13） ⑥Add the pixel value in the attention superposition heat map to the value greater than The part is set to 1, which is called the foreground; at the same time, The part is set to 0, which is called background, thereby achieving accurate segmentation of key feature areas in rolling bearing fault diagnosis; S53, generating a total threshold overlay map; Under the same category of bearings, all the attention superposition heat maps after binarization are added according to the pixel position to obtain the total threshold superposition map of the attention superposition heat map of this category, and then converted into a grayscale image; S54, normalization processing; The threshold map after superposition of each category is processed using the minimum-maximum normalization method. Specifically, the minimum and maximum values ​​of all pixel values ​​of the grayscale image of each category are calculated, and then each pixel value is linearly mapped to the range of [0, 1] to ensure that all images can be compared on the same scale. The normalization calculation formula is as follows: （14） in, is the pixel value before normalization, is the normalized pixel value; S55, calculating a weight matrix; According to the above total threshold overlay graph, calculate a weight matrix with consistent dimensions W ; Specifically: For each category of original time-frequency image X , by performing Hadamard product operation with the weight matrix, the final normalized total threshold overlay map is obtained. The calculation formula is as follows: （15） in, represents the Hadamard product, that is, the elements are multiplied by position, is the calculated total threshold overlay, is the original time-frequency image.

7. The rolling bearing fault diagnosis method based on time-frequency image and transfer learning according to claim 1 is characterized in that: In step S6, the specific process is: S61, preprocessing the unknown sample; For unknown bearing samples, continuous wavelet transform, Hilbert-Huang transform and short-time Fourier transform are performed to obtain the corresponding three types of time-frequency images; S62, calculating a weight map of unknown samples; For each type of time-frequency image, the attention superposition heat map of the unknown sample is obtained by performing Hadamard product operation with the weight matrix; S63, classification decision; According to the attention superposition heat map of the unknown sample, it is evaluated to which category of known fault samples the attention superposition heat map is closest, and then it is determined which category of fault samples the unknown sample belongs to.

Images