A small sample transfer learning fault diagnosis method and system based on CNN feature fusion, a computer and a storage medium

Abstract

A small sample transfer learning fault diagnosis method and system based on CNN feature fusion, a computer and a storage medium, relate to the field of rotating machinery fault diagnosis. The existing deep learning method cannot effectively process small samples and is difficult to be used for fault diagnosis of rolling bearings. The method comprises the following steps: collecting vibration signals of rolling bearings under original working conditions and target working conditions, and segmenting the signals to construct source domain sample data set and target domain sample data set; extracting time-frequency features of the signals to construct time-frequency image data set; constructing a CNN model with feature fusion; normalizing the original vibration signals and corresponding time-frequency images, and training the CNN model using the source domain data to obtain a CNN training model; adjusting the CNN training model according to part of the samples in the target domain to obtain a CNN fault diagnosis fine-tuning model; and using the fine-tuning model to diagnose the remaining samples in the target domain to obtain a fault diagnosis result. It is applied to the field of intelligent detection of rolling bearings.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of rotating machinery fault diagnosis, and in particular to a small sample transfer learning fault diagnosis method based on CNN feature fusion. BACKGROUND

[0002] Rolling bearings are key basic components of rotating machinery transmission devices, and are prone to failure when running for a long time under extreme and complex working conditions, thereby causing economic losses and safety accidents. Therefore, in order to avoid economic losses and safety accidents caused by rolling bearing failures, it is necessary to use fault diagnosis technology to monitor the running condition of rolling bearings.

[0003] At present, the fault diagnosis methods of rolling bearings mainly include traditional methods based on signal processing technology and methods based on deep learning. Most of the traditional fault diagnosis methods need to manually select features, and a lot of professional knowledge is needed when analyzing complex systems, and the diagnosis results have great uncertainty and poor generalization. The deep learning method can greatly reduce the demand for professional knowledge in feature extraction and reduce the uncertainty caused by human participation, and can directly complete the end-to-end intelligent fault diagnosis.

[0004] The collected original vibration signal of the bearing usually has non-stationary characteristics and contains a large amount of background noise and noise generated by other components, thereby resulting in poor diagnosis results. Time-frequency analysis is an effective tool for processing non-stationary signals. The deep learning method based on time-frequency image often has a large number of layers, and with the increase of the number and size of hidden layers, the number of required training parameters also increases. This leads to the need for a large amount of labeled data in training the deep learning network based on images, and the existing deep learning method is based on a single time domain signal, which results in poor feature quality and difficulty in training the deep learning model, and it is difficult to apply when the target domain has insufficient labeled samples.

[0005] In most actual industrial scenes, the training data is strictly limited, especially the fault often occurs only at the end of the bearing life. Moreover, due to the complex and diverse working conditions of rolling bearings, the distribution of fault data learned and predicted in the diagnosis model is different. The commonly used deep learning method cannot well handle the small sample case, and it is difficult to be used for fault diagnosis of rolling bearings. SUMMARY

[0006] The present application proposes a small sample transfer learning (TL-MTCN) fault diagnosis method based on CNN feature fusion in order to solve the problem that the existing deep learning method cannot well handle the small sample case and is difficult to be used for fault diagnosis of rolling bearings.

[0007] The technical scheme of the present application is as follows:

[0008] A small sample transfer learning fault diagnosis method based on CNN feature fusion, the method comprises:

[0009] S1: Collecting the vibration signals of rolling bearings under original working conditions and target working conditions, and dividing the original vibration signals, and constructing source domain sample data set and target domain sample data set according to the divided signals;

[0010] S2: Extracting the time-frequency features of the original vibration signals, and constructing a time-frequency image data set;

[0011] S3: Constructing a CNN model with feature fusion according to the original vibration signals and the time-frequency image data set;

[0012] S4: Normalizing the original vibration signals and corresponding time-frequency images, training the CNN model according to the normalized source domain data, and obtaining a CNN training model;

[0013] S5: Adjusting the CNN training model according to part of the samples in the target domain sample data set, and obtaining a CNN fault diagnosis fine-tuning model;

[0014] S6: Diagnosing the remaining samples in the target domain using the CNN fault diagnosis fine-tuning model, and obtaining a bearing fault diagnosis result.

[0015] Further, a preferred mode is also provided, wherein the CNN model with feature fusion is constructed according to the original vibration signals and the time-frequency image data set, comprising:

[0016] A ConvNeXt network with a standard convolution module is used to extract the features of the time-frequency image data set, wherein the input feature size of the ConvNeXt network is 224×224, and the output feature size is 768×1;

[0017] A 1D-CNN network structure combined with ECA-Net is constructed to extract features of the original vibration signals, wherein the length of the input original vibration signal of the 1D-CNN network structure is 1024, and the output feature size is 96×1;

[0018] The extracted time-frequency image features and original vibration signal features are fused to obtain a CNN model with feature fusion.

[0019] Further, a preferred mode is also provided, wherein the 1D-CNN network structure is specifically:

[0020] The 1D-CNN network is composed of 4 convolution layers, and the number of convolution kernels in each convolution layer is 256, 128, 64 and 32 respectively, wherein the convolution kernel size of the first convolution layer is 64, and the rest is 3, and the step length is 1;

[0021] After each convolutional layer, there is a max-pooling layer with a pooling kernel size of 4x1 and a pooling step of 4. And the convolutional layer and the pooling layer do not perform padding operation; at the end of the network, two fully connected layers are added, respectively having 96 and 10 neurons;

[0022] ECA-Net modules are added after the last two convolutional layers, and the nonlinear activation function used after each convolutional layer is the scaled exponential linear unit SeLu, and the classification layer uses Softmax.

[0023] Further, a preferred mode is also provided, wherein the convolutional calculation method is:

[0024]

[0025] wherein, is the output of the i-th neuron, is the input of the i-th neuron, f(·) is an activation function, ω ij is an input signal is the connection weight of the j-th neuron, b j is an output bias.

[0026] Further, a preferred mode is also provided, wherein the original vibration signal and the corresponding time-frequency image in the data set are normalized, and the normalization processing further comprises:

[0027] 70% of the sample data in the source domain sample data set is selected as the training set for pre-training, and 30% of the sample data is used as the validation set to evaluate the source domain training result and reserve the best parameters; when the preset iteration number is reached, the best source domain training parameter is saved according to the performance of the validation set and migrated to the target domain.

[0028] Further, a preferred mode is also provided, wherein the CNN training model is adjusted according to part of the sample in the target domain sample data set, and the CNN fault diagnosis fine-tuning model is obtained, comprising:

[0029] The pre-trained samples in the source domain sample data set are migrated to the entire TL-MTCN deep convolutional network, and it is determined whether to delete the last fully connected layer and its parameters according to the migration task;

[0030] In the target domain, 10% or less of the samples are used as the training set to pre-train and adjust the CNN training model, and the remaining samples are used as the validation set.

[0031] Further, a preferred mode is also provided, wherein the normalization processing of the time-frequency image is specifically:

[0032] The ImageNet dataset is standardized by using the mean and variance, wherein the mean is (0.485, 0.456, 0.406) and the variance is (0.229, 0.224, 0.225).

[0033] Based on the same inventive concept, the application further provides a small sample transfer learning fault diagnosis system based on CNN feature fusion, which comprises:

[0034] Module one: used for collecting vibration signals under original working conditions and target working conditions of rolling bearings, and segmenting the original vibration signals, and constructing a source domain sample data set and a target domain sample data set according to the segmented signals;

[0035] Module two: used for extracting time-frequency features of the original vibration signals and constructing a time-frequency image data set;

[0036] Module three: used for constructing a CNN model based on feature fusion according to the original vibration signals and the time-frequency image data set;

[0037] Module four: used for normalizing the original vibration signals and corresponding time-frequency images, training a CNN model according to the normalized source domain data, and obtaining a CNN training model;

[0038] Module five: used for adjusting the CNN training model according to part of the samples in the target domain sample data set, and obtaining a CNN fault diagnosis fine-tuning model;

[0039] Module six: used for diagnosing the remaining samples in the target domain by using the CNN fault diagnosis fine-tuning model, and obtaining a bearing fault diagnosis result.

[0040] Based on the same inventive concept, the application further provides a computer readable storage medium for storing a computer program, wherein the computer program executes any one of the above-mentioned small sample transfer learning fault diagnosis methods based on CNN feature fusion.

[0041] Based on the same inventive concept, the application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes any one of the above-mentioned small sample transfer learning fault diagnosis methods based on CNN feature fusion.

[0042] The application has the advantages that the application solves the problem that the existing deep learning method cannot well process small samples and is difficult to be used for fault diagnosis of rolling bearings.

[0043] The application provides a rolling bearing fault diagnosis method based on original vibration signal and time-frequency image feature fusion, and through a transfer learning technology, a small amount of training samples in a target domain are used to fine-tune a source domain fault diagnosis model, so that high-precision intelligent fault diagnosis between different working conditions and different machine structures is realized. The method still has good effect without source domain training. The pre-training weight is used to initialize the ConvNeXt, so that the training cost of the source domain diagnosis model can be effectively reduced. The 1D-CNN designed based on the attention mechanism can effectively suppress overfitting while keeping a small number of trainable parameters. In the case of a small number of samples and a high noise environment, the 1D-CNN still has excellent feature extraction capability.

[0044] The application is applied to the field of intelligent detection of rolling bearings. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 A small sample transfer learning fault diagnosis process flow chart based on CNN feature fusion according to the first embodiment;

[0046] Figure 2 A TL-MTCN deep convolutional network overall framework diagram according to the second embodiment;

[0047] Figure 3 A task A to D fine-tuning result confusion matrix comparison diagram according to the eleventh embodiment;

[0048] Figure 4 A fine-tuning result T-SNE visualization comparison diagram after adding noise to the data set E according to the eleventh embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments of the application.

[0050] The first embodiment is described in Figure 1 The first embodiment is described in

[0051] S1: Collect vibration signals under original working conditions and target working conditions of rolling bearings, and segment the original vibration signals, and construct a source domain sample data set and a target domain sample data set according to the segmented signals;

[0052] S2: Extract time-frequency features of the original vibration signals, and construct a time-frequency image data set;

[0053] S3: constructing a CNN model with feature fusion according to the original vibration signal and the time-frequency image data set;

[0054] S4: normalizing the original vibration acceleration signal and the corresponding time-frequency feature image, training the CNN model according to the normalized source domain data, and obtaining a CNN training model;

[0055] S5: adjusting the CNN training model according to part of the sample data in the target domain sample data set, and obtaining a CNN fault diagnosis fine-tuning model;

[0056] S6: diagnosing the remaining samples in the target domain using the CNN fault diagnosis fine-tuning model to obtain a bearing fault diagnosis result.

[0057] The original vibration signal in the embodiment is the vibration signal under the original working condition and the target working condition of the rolling bearing.

[0058] The small sample transfer learning fault diagnosis method based on CNN feature fusion in the embodiment uses the transfer learning technology to fine-tune the source domain fault diagnosis model using a small amount of training samples in the target domain, and realizes high-precision intelligent fault diagnosis between different working conditions and different machine structures.

[0059] The CNN training model in the embodiment will continuously adjust the model parameters until the parameters converge, obtain the best source domain training parameters, and save the training parameters, the purpose being to obtain the optimal training model and guarantee the accuracy of fault diagnosis.

[0060] Embodiment two, see Figure 2 The embodiment is a further limitation of the small sample transfer learning fault diagnosis method based on CNN feature fusion in embodiment one, and the CNN model with feature fusion constructed according to the original vibration signal and the time-frequency image data set comprises:

[0061] The ConvNeXt network with a standard convolution module is used to extract the features of the time-frequency image data set, wherein the input feature size of the ConvNeXt network is 224x224, and the output feature size is 768x1;

[0062] The 1D-CNN network structure combined with ECA-Net is constructed to extract the features of the original vibration signal, wherein the length of the input original vibration signal of the 1D-CNN network structure is 1024, and the output feature size is 96x1;

[0063] The extracted time-frequency image features and original vibration signal features are fused to obtain a CNN model with feature fusion.

[0064] The TL-MTCN deep convolutional network overall framework described in the embodiment is as shown in Figure 2 The TL-MTCN deep convolutional network described in the embodiment is a CNN model for constructing feature fusion.

[0065] In the embodiment, the improved 1D-CNN network structure and the ConvNeXt network structure are respectively established to perform deep feature extraction on the combined data, and then a fully connected layer is used to classify the fused features to realize bearing state pattern recognition. Specifically,

[0066] The feature fusion CNN model described in the embodiment is completely dependent on the ConvNeXt network of the standard convolution module. The ConvNeXt network is used to extract the features of the time-frequency image. The input feature size of the ConvNeXt network is 224x224, the output feature size is 768x1, and the classification layer at the end of the ConvNeXt network is deleted. A 1D-CNN network structure combined with ECA-Net is constructed to extract features of the original vibration signal. The length of the input original vibration acceleration signal is 1024, the output feature size is 96x1, and the classification layer at the end of the 1D-CNN network structure is also deleted. Then the extracted features are spliced and input into a fully connected layer. The activation function is Softmax, and the classification of various rolling bearing faults is realized.

[0067] Embodiment three, the embodiment is a further limitation of the small sample transfer learning fault diagnosis method based on CNN feature fusion described in embodiment two, and the 1D-CNN network structure is specifically:

[0068] The 1D-CNN network is composed of 4 convolutional layers, and the number of convolutional kernels in each convolutional layer is 256, 128, 64 and 32 respectively. The convolutional kernel size of the first convolutional layer is 64, and the rest is 3, and the step length is 1.

[0069] After each convolutional layer, there is a maximum pooling layer, and the pooling kernel size is 4x1 and the pooling moving step is 4. The convolutional layer and the pooling layer do not perform padding operation. At the end of the network, two fully connected layers are added, and the two fully connected layers have 96 neurons and 10 neurons respectively.

[0070] ECA-Net modules are added after the last two convolutional layers. In addition, the nonlinear activation function used after each convolutional layer is the scaled exponential linear unit SeLu, and the classification layer uses Softmax.

[0071] In this embodiment, in order to make full use of the characteristics of different channels, an ECA-Net module is added after the last two convolutional layers. This embodiment constructs a new 1D-CNN network structure, and the feature extraction is more accurate.

[0072] Embodiment four, this embodiment is a further limitation of the small sample transfer learning fault diagnosis method based on CNN feature fusion of embodiment three, the convolution calculation method is:

[0073]

[0074] wherein, is the output of the i-th neuron, is the input of the i-th neuron, f(·) is an activation function, ω ij is the input signal is the connection weight of the j-th neuron, b j is the output bias.

[0075] This embodiment is described in combination with embodiment two and embodiment three.

[0076] Firstly, different dimensional parallel convolutional layers are constructed by using ConvNeXt network and improved 1D-CNN network, and key state feature information is extracted from two different dimensional data. The first group of two-dimensional convolutional layers operates on the time-frequency image to adaptively extract useful features. The second group of one-dimensional convolutional layers extracts key information from the original vibration signal. Finally, the two different dimensional features extracted are fused for the next step of classification. Among them, there are different parameter quantities of pre-trained models based on the pre-trained ConvNeXt network, and the Tiny model with the least parameters is selected in this embodiment.

[0077] wherein, the constructed 1D-CNN network model is composed of 4 convolutional layers, and the number of convolution kernels in each convolutional layer is 256, 128, 64 and 32 respectively. The convolution kernel size of the first convolutional layer is 64, and the rest is 3, and the step length is 1. After each convolutional layer, there is a maximum pooling layer, and the pooling kernel size is 4x1, and the pooling moving step length is 4. And the convolutional layer and the pooling layer do not perform padding operation. At the end of the network, two fully connected layers are added, which have 96 and 10 neurons respectively. The convolution calculation method is as follows:

[0078]

[0079] wherein, is the output of the i-th neuron, is the input of the i-th neuron, f(·) is an activation function, ω ij is the input signal is the connection weight of the j-th neuron, bj is the output bias. The max-pooling calculation method is as follows:

[0080]

[0081] wherein, is the input value of the jth neuron of the input feature surface, f max (·) is the maximum value of the function, Y i out is the output value of the ith neuron of the output feature surface.

[0082] In order to make full use of the characteristics of different channels, an ECA-Net module is added after the last two convolutional layers. In addition, the nonlinear activation function used after each convolutional layer is the scaled exponential linear unit (SeLu), and the classification layer uses Softmax. The specific structure is shown in Table 1:

[0083] Table 1 1D-CNN model structure

[0084]

[0085] Finally, by merging two feature maps of different shapes, the feature map of the previous convolution operation is fused into one in the channel dimension. Then, the new feature map is flattened and input into the fully connected layer to calculate the mapping relationship between feature extraction and bearing fault states, and finally the fault is classified through Softmax. The calculation formula of the fully connected layer is as follows:

[0086]

[0087] wherein, represents the weight matrix of the jth neuron of the lth layer and the ith neuron of the l+1th layer, is the corresponding bias term, represents the jth neuron of the lth layer. The Softmax function is a normalized exponential function, which is used to calculate the probability of the original vibration signal corresponding to each fault category, and its definition is:

[0088]

[0089] wherein, s i represents the sample number of the ith fault category corresponding to the current vibration signal, p i represents the probability of the ith fault category corresponding to the current vibration signal, and the sum of the probabilities of each fault category corresponding to the current sample is 1, i.e.

[0090] Embodiment five, the embodiment is further limited to the CNN feature fusion based small sample transfer learning fault diagnosis method of embodiment one, the original vibration acceleration signal and the corresponding time-frequency image are normalized, and the normalized signal and the corresponding time-frequency image are further normalized.

[0091] 70% of the source domain sample data set is selected as the training set for pre-training, and 30% of the samples are used as the validation set to evaluate the source domain training result and retain the best parameters; when the preset iteration number is reached, the best source domain training parameters are saved according to the performance of the validation set and migrated to the target domain.

[0092] Specifically, S4 further includes: when migrating the network parameters pre-trained based on ImageNet to the source domain ConvNeXt deep convolutional network, deleting the fully connected layer parameters in the pre-trained network. 70% of the data in the source domain sample data set is used as the training set for pre-training, and 30% of the samples are used as the validation set to evaluate the source domain training result and retain the best parameters. When the preset iteration number is reached, the best source domain training parameters are saved according to the performance of the validation set and migrated to the target domain.

[0093] Embodiment six, the embodiment is further limited to the CNN feature fusion based small sample transfer learning fault diagnosis method of embodiment five, the CNN training model is adjusted according to part of the sample in the target domain sample data set, and a CNN fault diagnosis fine-tuning model is obtained, comprising:

[0094] The pre-trained samples in the source domain sample data set are migrated to the entire TL-MTCN deep convolutional network, and it is determined whether to delete the last fully connected layer and its parameters according to the migration task;

[0095] 10% or less of the samples in the target domain are used as the training set for pre-training to adjust the CNN training model, and the remaining samples are used as the validation set.

[0096] This embodiment is described in combination with embodiment five, specifically:

[0097] The network parameters pre-trained based on the source domain are migrated to the entire TL-MTCN deep convolutional network, and it is determined whether to delete the last fully connected layer and its parameters according to the migration task. 10% or less of the data in the target domain are used as the training set for pre-training, and the remaining samples are used as the final validation set. The validation set is not used to evaluate the training effect during the training process to avoid data leakage. When the preset iteration number is reached, the last saved parameters are used to test the test set samples to obtain the target domain fault diagnosis result.

[0098] Further, the TL-MTCN fault diagnosis model trained on the source domain is used to diagnose faults on the target domain. A parameter migration method is adopted in combination with fine-tuning of the fault diagnosis model of the target domain. The parameter migration method is an effective and efficient solution to overcome the lack of labeled data in the target domain.

[0099] In an embodiment seven, the normalization of the time-frequency image is specifically:

[0100] The mean and variance of the ImageNet dataset are used for standardization, wherein the mean is (0.485, 0.456, 0.406) and the variance is (0.229, 0.224, 0.225). The normalization method of the original vibration signal is to standardize the training set and the validation set using the mean and variance of the training set.

[0101] In an embodiment eight, the small sample transfer learning fault diagnosis system based on CNN feature fusion comprises:

[0102] Module one is used to collect the original working condition and the vibration acceleration signal under the target working condition of the rolling bearing, and to segment the original vibration signal, and to construct a source domain sample data set and a target domain sample data set according to the segmented signal;

[0103] Module two is used to extract the time-frequency feature of the original vibration signal and construct a time-frequency image data set;

[0104] Module three is used to construct a CNN model based on feature fusion according to the original vibration signal and the time-frequency image data set;

[0105] Module four is used to normalize the original vibration signal and the corresponding time-frequency image, train the CNN model according to the normalized source domain data, and obtain a CNN training model;

[0106] Module five is used to adjust the CNN training model according to part of the samples in the target domain sample data set, and obtain a CNN fault diagnosis fine-tuning model;

[0107] Module six is used to diagnose the remaining samples in the target domain by using the CNN fault diagnosis fine-tuning model, and obtain a bearing fault diagnosis result.

[0108] The original vibration signal in the embodiment is the vibration signal under the original working condition and the target working condition of the rolling bearing.

[0109] Embodiment nine, the computer readable storage medium of the embodiment, the computer readable storage medium is used to store a computer program, the computer program executes the small sample transfer learning fault diagnosis method based on CNN feature fusion of any one of the embodiments one to seven.

[0110] Embodiment ten, see Figure 3 And Figure 4 The embodiment is described. The computer device of the embodiment comprises a memory and a processor, the memory has a computer program stored therein, when the processor runs the computer program stored in the memory, the processor executes the small sample transfer learning fault diagnosis method based on CNN feature fusion according to any one of the embodiments one to seven.

[0111] Embodiment eleven, the embodiment is a specific embodiment of the small sample transfer learning fault diagnosis method based on CNN feature fusion of the embodiment one, also used to explain the embodiments two to seven, specifically:

[0112] Step one, the vibration acceleration signals under the existing working conditions and target working conditions of bearings are collected respectively, and the original vibration acceleration signals are segmented to construct source domain sample data set and target domain sample data set;

[0113] Step two, the time-frequency features of the original vibration signals are extracted using synchronous compression wavelet transform, and the corresponding image data set is constructed;

[0114] Step three, the feature fusion CNN model initialized in different ways is constructed;

[0115] Step four, the original vibration signals in the source domain sample data set and the corresponding time-frequency image are used as input data, and after normalization, the input data is input into the fault diagnosis model for training and saving the training parameters;

[0116] Step five, the few labeled samples in the target domain sample data set are used to fine-tune the model, and finally the remaining samples of the target domain to be tested are input into the fine-tuned network to obtain the bearing fault diagnosis result.

[0117] In step four of the embodiment, the original vibration signals of the source domain sample data and the corresponding time-frequency image are used as input data to realize the source domain training of the model.

[0118] The data used in the embodiment are respectively from the bearing vibration database of Case Western Reserve University in the United States and the American Mechanical Failure Prevention Technology Association.

[0119] The bearing data platform of Case Western Reserve University in the United States is composed of a 1.5KW motor, a torque sensor, a power test meter and an accelerometer. The system contains two test bearings, respectively located at the motor driving end and the motor fan end, and the bearing model used is: 6205-2RS JEM SKF. Single-point damage is set on the ball, inner ring and outer ring by electric spark machining, so the bearing has four states: normal state, ball failure state, inner ring failure state and outer ring failure state. According to the different fault diameters, three kinds of fault states contain three kinds of fault degree signals, that is, the signals of fault diameters of 0.007, 0.014 and 0.021 inches. The vibration data is collected by using the accelerometer, and the accelerometer is placed near the bearings, and the signal is collected at a sampling frequency of 12kHz. The data used in the experiment of the embodiment is the driving end bearing data.

[0120] The American Mechanical Fault Prevention Technology Association dataset contains three groups of experimental bearing vibration data, namely baseline bearing data, outer ring fault data under various loads and inner ring fault data under various loads. Three normal data are collected at a sampling frequency of 97656Hz and a load of 270lbs; seven outer ring fault data are collected at a sampling frequency of 48828Hz under loads of 25, 50, 100, 150, 200, 250 and 300lbs, respectively; seven inner ring fault data are collected at a sampling frequency of 48828Hz under loads of 0, 50, 100, 150, 200, 250 and 300lbs, respectively. The vibration data under normal conditions is down-sampled to 48828Hz to match other fault conditions. Since the sampling rate is higher than that of CWRU, the fault information contained under the same sample length is less, resulting in an increase in the difficulty of diagnosis.

[0121] Table 2 Bearing experimental dataset

[0122]

[0123] According to the bearing fault diagnosis problem across working conditions and across devices proposed in the application, A→B, A→C, A→D, B→C, B→D, C→D, A→E, B→E, C→E and D→E migration tasks are used to verify the effectiveness of the method of the application, wherein A→B represents the knowledge migration of the source domain dataset A to the target domain dataset B. In the experiment, the proportion of the training set and the test set of the source domain is set to 70:30, the training set of the target domain is not more than 10%, and the remaining samples are used for final verification. The average diagnosis accuracy of 10 repeated experiments is used as the comparison experiment result in the application. The training set, test set and verification set randomly extracted from the dataset are balanced for different categories. In order to ensure the fairness of the experiment, all the training is carried out under the same set of randomly selected samples.

[0124] The standardization process of the original vibration signal is as follows:

[0125]

[0126]

[0127]

[0128]

[0129] wherein, is the standardized training data set, is the standardized test data set, x af represents the training data set, x ae represents the test data set, σ f is the standard deviation of the training data set, A is the number of training set samples, is the mean of the training data set.

[0130] The optimizer is selected as AdamW because its learning rate can be self-adaptively optimized. It can constantly update the neural network weights according to the training data, avoid local optimum, and dynamically adjust the learning rate of each parameter. The learning rate of the AdamW algorithm is set to 0.0005, and the weight decay is set to 0.05. The number of iterations is selected as 50 times, and the batch size is 32. When the preset number of iterations is reached, the best source domain training parameter is saved according to the performance of the validation set to migrate to the target domain.

[0131] In step five, the model is initialized using the source domain training parameter, so as to realize the fine-tuning training of the model by a small amount of sample data in the target domain.

[0132] First, experiments under cross-condition conditions are performed, and the above model is fine-tuned under the target condition to perform the diagnosis task under different working conditions. Specifically, 20 samples in the target domain are randomly selected to fine-tune the entire pre-trained network, and the remaining samples are used for final verification. Select the current several relatively mainstream detection methods for experimental comparison, which are ResNet-50 without freezing parameters and VGG-16 network with three shallow blocks frozen, and the average accuracy of ten repeated experiments will be used as the evaluation index for the classification task. The experimental results of the migration task under different conditions are shown in Table 3.

[0133] Table 3 Experimental results of different migration tasks under cross-condition

[0134]

[0135] As can be seen from Table 3, the average accuracy of the method recognition is more than 99.11%, which fully shows that the knowledge of different working conditions of the rolling bearing can be migrated to each other, that is, the knowledge migration can realize the mutual diagnosis of different working conditions of the rolling bearing, and has very good effect. The confusion matrix of the A→D task is as shown in Table 3. Figure 3

[0136] In order to further verify the effectiveness of the model, the following experiments are carried out for the rolling bearing cross-device transfer learning task. The proposed method and other models are compared in a series of experiments. In this section, the data set A in CWRU is selected as the source domain, and E in the MFPT data set is selected as the target domain. The data preprocessing method and the selection of hyperparameters are the same as the above experiments. However, since the number of bearing categories of the CWRU data set and the MFPT is different. Therefore, the classification layer parameter migration is not performed in the fine-tuning process. The proportion of the target domain fine-tuning samples is set to 2.5%, 5%, 7.5% and 10% in turn. The experimental results of different transfer tasks are shown in Table 4.

[0137] Table 4 Comparison of diagnostic method accuracy

[0138]

[0139] As can be seen from Table 4, even if only 2.5% of the target samples are selected for fine-tuning, the diagnostic accuracy of the proposed method on the transfer path is more than 98.21%, which shows that the model has commendable transferability across devices. The transfer accuracy is continuously improved with the increase of the proportion of fine-tuning samples. When the proportion reaches 5%, the diagnostic accuracy of each transfer path reaches 99.01%. When the proportion reaches 10%, the accuracy further reaches 99.49%, but this also brings greater computational cost. Therefore, 5% is selected as the fine-tuning proportion in the subsequent experiments, which can ensure high accuracy while using lower computational cost.

[0140] Next, in order to explore the good transfer performance between different devices. Based on different source domains (A, B, C and D) for pre-training, and through 5% small sample fine-tuning. The training results are shown in Table 5, and the classification performance is 99.01% (A→E), 98.37% (B→E), 98.51% (C→E) and 98.63% (D→E) respectively. As can be seen from the results, the transfer accuracy is very close to the performance of the model trained with sufficient labeled samples. Therefore, the proposed method can better solve the problem of insufficient target samples while maintaining the diagnostic accuracy.

[0141] Table 5 Comparison results of different model transfer methods

[0142] ​

[0143] In addition, considering that in actual engineering environment, the vibration and mutual friction between parts will inevitably produce noise, leading to the bearing vibration signal collected by the sensor being easily contaminated by noise, which masks the fault information in the vibration signal. To verify the anti-noise performance of the proposed method, different intensities of Gaussian white noise are added to the test samples to simulate the noise in the actual engineering environment. Among them, the data samples used for training are not added noise. The signal-to-noise ratio (SNR) is the ratio of signal power to noise power, and the signal-to-noise ratio is generally measured in decibels (dB), as shown below.

[0144]

[0145] In the formula, P s and P n respectively represent the power of the signal and the power of the noise. This experiment designed five kinds of noise signals with signal-to-noise ratios of -4dB, -2dB, 0dB, 2dB and 4db. The greater the signal-to-noise ratio, the less the signal is disturbed by noise. Signals with low SNR are usually more complex, so it is very important to improve the robustness of the model. Table 6 shows the comparison results of different methods under different signal-to-noise ratios for migration tasks A-E.

[0146] Table 6 Comparison results under different signal-to-noise ratios

[0147]

[0148] The TL-MTCN model did not significantly reduce the accuracy rate when the signal-to-noise ratio was 4dB and 2dB. For noise data with a signal-to-noise ratio of 0dB and -2dB, the classification accuracy of the model decreased slightly, but still maintained an average accuracy of more than 96.73%. For noise data with a signal-to-noise ratio of -4dB, the accuracy of the TL-MTCN model was affected to some extent. The ResNet-50 model, which performed well under noise-free conditions, showed a significant performance decline as the noise intensity increased.

[0149] T-SNE is a data dimensionality reduction technique that visualizes the output layer features of each model to reveal the feature classification ability of different models. Figure 4 The T-SNE deep feature analysis chart for the cross-dataset migration task under a signal-to-noise ratio of -4dB is shown, where D2 and D1 represent the principal components after dimensionality reduction. The experimental comparison results further show that, compared with other models, the TL-MTCN has better intra-class compactness and inter-class separability, and has stronger fault diagnosis migration ability.

[0150] Although preferred embodiments of the disclosure have been described herein, those skilled in the art will appreciate that other modifications than those specifically described herein can be made to the embodiments without departing from the spirit and scope of the disclosure. Accordingly, it is intended that the appended claims be construed to cover all such modifications as fall within the true spirit and scope of the disclosure.

[0151] Obviously, numerous modifications and variations of the present disclosure are possible in light of the above teachings. It is therefore to be understood that within the scope of the disclosure, the disclosure can be practiced otherwise than as specifically described herein.

[0152] Those skilled in the art will appreciate that embodiments of the disclosure can be supplied as a method, a system, or a computer program product. Accordingly, the disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the disclosure can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code. The disclosure is described in reference to the flow diagrams and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams flow or flows and / or block or blocks specified in the flow diagrams and / or block diagrams. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams flow or flows and / or block or blocks specified in the flow diagrams and / or block diagrams. Accordingly, embodiments of the disclosure include a computer program product. The computer program product can include a computer-readable storage medium having stored thereon instructions that, when executed by a processor of a computer or other programmable data processing apparatus, can cause the processor to carry out steps or functions of embodiments of the disclosure. Figure 1 one or more flow or flows and / or block or blocks Figure 1 means for performing the function specified by the block or blocks. Such computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams flow or flows and / or block or blocks specified in the flow diagrams and / or block diagrams. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams flow or flows and / or block or blocks specified in the flow diagrams and / or block diagrams. Accordingly, embodiments of the disclosure include a computer program product. The computer program product can include a computer-readable storage medium having stored thereon instructions that, when executed by a processor of a computer or other programmable data processing apparatus, can cause the processor to carry out steps or functions of embodiments of the disclosure. Figure 1 one or more flow or flows and / or block or blocks Figure 1 means for performing the function specified by the block or blocks. Such computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams flow or flows and / or block or blocks specified in the flow diagrams and / or block diagrams. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams flow or flows and / or block or blocks specified in the flow diagrams and / or block diagrams. Accordingly, embodiments of the disclosure include a computer program product. The computer program product can include a computer-readable storage medium having stored thereon instructions that, when executed by a processor of a computer or other programmable data processing apparatus, can cause the processor to carry out steps or functions of embodiments of the disclosure. Figure 1 one or more flow or flows and / or block or blocks Figure 1 means for performing the function specified by the block or blocks. Such computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams flow or flows and / or block or blocks specified in the flow diagrams and / or block diagrams. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams flow or flows and / or block or blocks specified in the flow diagrams and / or block diagrams. Accordingly, embodiments of the disclosure include a computer program product. The computer program product can include a computer-readable storage medium having stored thereon instructions that, when executed by a processor of a computer or other programmable data processing apparatus, can cause the processor to carry out steps or functions of embodiments of the disclosure.

[0153] It should be pointed out finally that the above embodiments are only used for illustrating the technical solutions of the present disclosure but not for limiting the protection scope thereof, and although the present disclosure is described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the specific embodiments of the present disclosure can be changed, modified or replaced equivalently by those skilled in the art after reading the present disclosure, but these changes, modifications or equivalent replacements are all within the protection scope of the disclosed claims.

Claims

1. A small sample transfer learning fault diagnosis method based on CNN feature fusion, characterized in that, The method comprises: S1: Collecting vibration signals of rolling bearings under original working conditions and target working conditions, and segmenting the original vibration signals, and constructing a source domain sample data set and a target domain sample data set according to the segmented signals; S2: Extracting time-frequency features of the original vibration signals, and constructing a time-frequency image data set; S3: Constructing a CNN model with feature fusion according to the original vibration signals and the time-frequency image data set; S4: Normalizing the original vibration signals and corresponding time-frequency images, training the CNN model according to the normalized source domain data, and obtaining a CNN training model; S5: Adjusting the CNN training model according to part of the samples in the target domain sample data set, and obtaining a CNN fault diagnosis fine-tuning model; S6: Diagnosing the remaining samples in the target domain by using the CNN fault diagnosis fine-tuning model, and obtaining a bearing fault diagnosis result; The CNN model with feature fusion comprises: A ConvNeXt network using a standard convolution module is used to extract features of the time-frequency image data set, wherein the input feature size of the ConvNeXt network is 224x224, and the output feature size is 768x1; A 1D-CNN network structure combined with an ECA-Net is constructed to extract features of the original vibration signals, wherein the length of the input original vibration signal of the 1D-CNN network structure is 1024, and the output feature size is 96x1; The extracted time-frequency image features and original vibration signal features are fused to obtain a CNN model with feature fusion; The 1D-CNN network structure specifically comprises: The 1D-CNN network is composed of four convolution layers, and the number of convolution kernels in each convolution layer is 256, 128, 64 and 32 respectively, wherein the convolution kernel size of the first convolution layer is 64, and the rest is 3, and the step length is 1; After each convolution layer, there is a maximum pooling layer, and the pooling kernel size is 4x1 and the pooling moving step length is 4; and the convolution layer and the pooling layer do not perform padding operation; an ECA-Net module is added after the last two convolution layers, and a non-linear activation function, namely a scaling exponential linear unit SeLu, is used after each convolution layer; At the end of the overall network formed by the ConvNeXt network and the 1D-CNN network structure combined with the ECA-Net, two fully connected layers are added, respectively having 96 and 10 neurons; The classification layer uses Softmax.

2. The small sample transfer learning fault diagnosis method based on CNN feature fusion according to claim 1, characterized in that, The convolution calculation method in the 1D-CNN network comprises: wherein, is the output of the i-th neuron, is the input of the i-th neuron, is an activation function, is the input signal is the connection weight to the j-th neuron, is the output bias.

3. The small sample transfer learning fault diagnosis method based on CNN feature fusion according to claim 1, characterized in that, The normalization of the original vibration signals and corresponding time-frequency images further comprises: selecting 70% of the samples in the source domain sample data set as a training set for pre-training, and using 30% of the samples as a validation set to evaluate the source domain training result and reserve the best parameters; when a preset number of iterations is reached, the best source domain training parameters are saved and migrated to the target domain according to the performance of the validation set.

4. The small sample transfer learning fault diagnosis method based on CNN feature fusion according to claim 3, characterized in that, The adjustment of the CNN training model according to part of the samples in the target domain sample data set to obtain a CNN fault diagnosis fine-tuning model comprises: The pre-trained samples in the source domain sample data set are migrated to the CNN training model with feature fusion, and whether to delete the last fully connected layer and its parameters is determined according to the migration task; The CNN training model is pre-trained and adjusted on the target domain with 10% or less samples as the training set, and the remaining samples are used as the verification set.

5. The small sample transfer learning fault diagnosis method based on CNN feature fusion according to claim 1, characterized in that, The normalization processing of the time-frequency image is specifically: The mean and variance of the ImageNet data set are used for standardization processing, wherein the mean is (0.485, 0.456, 0.406), and the variance is (0.229, 0.224, 0.225).

6. A small sample transfer learning fault diagnosis system based on CNN feature fusion, characterized in that, The system is realized based on the method of claim 1, and the system comprises: Module one: used for collecting vibration signals of rolling bearings under original working conditions and target working conditions, segmenting the original vibration signals, and constructing a source domain sample data set and a target domain sample data set according to the segmented signals; Module two: used for extracting time-frequency features of the original vibration signals and constructing a time-frequency image data set; Module three: used for constructing a CNN model with feature fusion according to the original vibration signals and the time-frequency image data set; Module four: used for normalizing the original vibration signals and corresponding time-frequency images, training a CNN model according to the normalized source domain data, and obtaining a CNN training model; Module five: used for adjusting the CNN training model according to part of the samples in the target domain sample data set, and obtaining a CNN fine-tuning model for fault diagnosis; Module six: used for diagnosing the remaining samples in the target domain by using the CNN fine-tuning model for fault diagnosis, and obtaining a bearing fault diagnosis result.

7. A computer readable storage medium characterized in that, The computer readable storage medium is used for storing a computer program, and the computer program executes the small sample transfer learning fault diagnosis method based on CNN feature fusion according to any one of claims 1-5.

8. A computer device, comprising: The computer readable storage medium is used for storing a computer program, and the computer program executes the small sample transfer learning fault diagnosis method based on CNN feature fusion according to any one of claims 1-5. The computer readable storage medium is used for storing a computer program, and the computer program executes the small sample transfer learning fault diagnosis method based on CNN feature fusion according to any one of claims 1-5.

Images