A time-frequency map fault diagnosis method and device based on multi-scale feature fusion and incremental learning

Abstract

The application belongs to the technical field of equipment part fault diagnosis, and discloses a time-frequency diagram fault diagnosis method and equipment based on multi-scale feature fusion and incremental learning, which comprises the following steps: (1) adopting old category data as training data, training an initial neural network to obtain a model ModelZ capable of classifying faults corresponding to the old category data; (2) dividing new category data into a predetermined batch, and then training the model ModelZ based on incremental learning to obtain a model ModelX(k) capable of identifying the new category data and a model ModelY(k) capable of classifying and diagnosing faults corresponding to the new category data and the old category data; (3) inputting a real-time time-domain signal of an equipment to be tested into the model ModelX(k) after converting the real-time time-domain signal into a two-dimensional time-frequency image, so as to obtain current old category data and new category data, and then the model ModelY(k) realizes fault category classification. The application has high efficiency and accuracy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field related to fault diagnosis of equipment parts, and more particularly to a time-frequency diagram fault diagnosis method and device based on multi-scale feature fusion and incremental learning. BACKGROUND

[0002] The health operation and fault diagnosis of mechanical equipment are the most important in industrial production and life. If the mechanical equipment fails during operation, it may cause the production line to stop and incur incalculable losses. In this context, fault diagnosis of mechanical equipment has always been a focus and hotspot of research. In recent years, fault diagnosis methods based on deep learning have developed unprecedentedly. Deep learning models have strong transferability and generalization ability, which makes deep learning more and more applied to fault diagnosis of mechanical equipment and mechanisms, and has achieved remarkable results. With the development of convolutional neural networks, image recognition and classification have become one of the popular directions in recent years. However, due to the characteristics of deep neural networks, general fault diagnosis models forget the old knowledge learned when learning new data, which is called catastrophic forgetting.

[0003] To solve this problem, researchers have proposed an incremental learning method, which aims to enable the model to learn new classes or tasks while maintaining good performance on the learned classes or tasks. For example, when training new data, some representative old data sets are added for mixed training. This method solves the problem of catastrophic forgetting of deep learning networks to some extent, but as the number of new data classes increases, the storage space pressure requirement becomes increasingly large, and the class imbalance problem becomes more serious, and the classifier will be biased towards the class with more data. And in the process of practical application, in many confidential environments, old class data is often confidential, so when new data comes, the user needs to update the model, which will cause a series of problems. Or some scholars found that GAN network can be used to add some old virtual data sets generated while training new data sets, which can solve the problem of old class data involving secrets, but the experimental results obtained by using this method of virtual generated old data sets in practical application are not ideal.

[0004] In the existing multi-class incremental task, it is not difficult to find that the main factors affecting the classification results are: whether the storage space is sufficient, the data set involving the problem of the actual working environment, the imbalance problem of new and old class data, etc. And in the process of actual fault diagnosis, the fault signal is always updated in real time. In this incremental environment, it is necessary to study an incremental learning method that does not require old class data sets and can classify and diagnose fault signals in a short time. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides a time-frequency graph fault diagnosis method and device based on multi-scale feature fusion and incremental learning, which aims to solve the problem that the existing fault recognition method uses old category data, resulting in low classification speed and poor accuracy.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, a time-frequency graph fault diagnosis method based on multi-scale feature fusion and incremental learning is provided, which comprises the following steps:

[0007] Step one, divide the time domain signals of different fault types of the equipment to be tested into multiple windows, and perform continuous wavelet transform on each window to obtain the frequency spectrum in each window, and then combine all the obtained frequency spectrums to obtain a two-dimensional time-frequency image to generate a time-frequency image dataset; divide the data in the time-frequency image dataset into old category data and new category data;

[0008] Step two, use the old category data as training data, and train an initial neural network Model with Transformer as the network structure to obtain a model ModelZ capable of classifying the faults corresponding to the old category data;

[0009] Step three, divide the new category data into a predetermined batch, and then train the model ModelZ based on incremental learning to obtain a model ModelX(k) capable of recognizing the new category data and a model ModelY(k) capable of classifying and diagnosing the faults corresponding to the new category data and the old category data;

[0010] Step four, after converting the real-time time domain signal of the equipment to be tested into a two-dimensional time-frequency image, input it into the model ModelX(k) to obtain the current old category data and new category data, and then input the current old category data and new category data into the model ModelY(k), which realizes fault category classification; wherein, when the model ModelY(k) classifies the fault category, it dynamically adjusts the fully connected layer weight of the model ModelY(k) according to the data category received by the ModelY(k).

[0011] Further, the ModelX(k) outputs the LSE value of each data it receives, and then identifies whether the corresponding data is new category data or old category data according to the comparison result of the LSE value and the set threshold.

[0012] Further, the calculation formula of the fully connected layer weight of the model ModelY(k) is:

[0013] O a+b (x)=[O1(x),...,O a (x),Oa+1 (x),...,O b (x)]=W ModelY T φ(x)

[0014] In the formula, W ModelY represents the full connection layer weight of the fault data time-frequency image classification model ModelY(k) of the kth increment, T represents the temperature coefficient, and φ(x) represents a feature extractor.

[0015] Further, the output result of the model ModelY(k) to the new class data k is defined as The output result to the old class data is defined as W ModelY is divided into W old and W new according to the different class weights of the new and old image data; and the expression of the output result of the model ModelY(k) is:

[0016]

[0017] In the formula, η represents a positive number with a value range of (0-1).

[0018] Further, the value of η is (0.82, 0.96).

[0019] Further, the self-attention formula based on the Transformer is: Where Q, K, and V are query, key, and value vector matrices respectively, d k is a scaling factor.

[0020] Further, the formula of the continuous wavelet transform is: Where s represents a scaling factor, τ represents a translation factor, and represents the complex conjugate of a sub-wavelet.

[0021] Further, the total loss function expression of the model ModelY(k) is: TotelLoss=α·Loss D +β·Loss CE +γ·Loss balance , where Loss D represents a distillation loss term, Loss CE represents a cross-entropy loss term, Loss balance represents a balance loss term, and α, β, and γ are weight coefficients of the respective loss terms.

[0022] The application further provides a time-frequency diagram fault diagnosis system based on multi-scale feature fusion and incremental learning, comprising a memory and a processor, the memory stores a computer program, and the processor executes the computer program to execute the time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning as described above.

[0023] The application further provides a computer readable storage medium storing machine executable instructions, which, when invoked and executed by a processor, cause the processor to implement the time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning as described above.

[0024] Overall, compared with the prior art, the time-frequency diagram fault diagnosis method and device based on multi-scale feature fusion and incremental learning provided by the application mainly have the following beneficial effects:

[0025] 1. The application adopts an incremental learning method of training only using newly added category samples (i.e., newly added category data k), without any old fault category data images (i.e., original category data and the first to k-1 newly added fault category data images), which guarantees data privacy and greatly reduces the storage space required by the model, overcoming the problems caused by using old data.

[0026] 2. The multi-scale fusion feature extraction method is adopted, which guarantees that the detailed feature information (local information) of the image can be effectively extracted, and also guarantees that the contour information (global information) of the image can be effectively extracted, which improves the classification accuracy of the model to a certain extent.

[0027] 3. The LSE threshold method is combined with the dynamic weight of the full connection layer of new and old category data, which avoids the phenomenon of category imbalance in feature extraction and classification of new and old category data, and the method can more tendentially classify new and old category data, improving the classification accuracy of the model.

[0028] 4. The one-dimensional fault signal collected is converted into a two-dimensional time-frequency image, which takes advantage of the advantages of convolutional neural network in two-dimensional image fault feature extraction and classification, realizes fault diagnosis and fault type classification of bearings, has greater generalizability, and can be used for fault diagnosis of various industrial parts. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a flowchart of a time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning provided by the application. DETAILED DESCRIPTION

[0030] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be given below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0031] Please refer to Figure 1 The present application provides a time-frequency map fault diagnosis method based on multi-scale feature fusion and incremental learning, which mainly comprises the following steps:

[0032] Step one, the time domain signals of different fault types of the equipment to be tested are divided into multiple windows, and continuous wavelet transform is performed on each window to obtain the frequency spectrum in each window, and then all the obtained frequency spectrums are combined to obtain a two-dimensional time-frequency image to generate a time-frequency image dataset; the data in the time-frequency image dataset is divided into old category data and new category data.

[0033] In one embodiment, step one comprises the following sub-steps:

[0034] F1: Install rolling bearings of different fault types on a mechanical simulation comprehensive fault test bench, set different working conditions for experiments, collect time domain signals of fault bearings, i.e. original time series data;

[0035] F2: Divide the time domain signals into multiple short time periods (windows), and perform continuous wavelet transform on each window to obtain the frequency spectrum in each time period.

[0036] F3: Combine the frequency spectrums of all time periods to generate a two-dimensional time-frequency image, make a time-frequency image dataset, and divide the data in the time-frequency image dataset into old category data and new category data.

[0037] In one embodiment, 10 kinds of fault signals are collected and made into 64*64*3 time-frequency images, and the dataset is made according to 1200 images per category, and the number ratio of old category data to new category data is 5:1.

[0038] Step two, use the old category data as training data, train on the initial neural network Model with Transformer as the network structure to obtain a model ModelZ capable of classifying the faults corresponding to the old category data.

[0039] Step three, the new class data is divided into a predetermined batch, and then the model ModelZ is trained based on incremental learning to obtain a model ModelX(k) capable of identifying new class data and a model ModelY(k) capable of classifying and diagnosing faults corresponding to new class data and old class data.

[0040] Wherein, when new class data k appears, the model needs to be retrained, but the model only needs to train the newly added class data during the training process, without repeating the training of old class data. Through retraining, a model ModelX(k) capable of classifying fault data images of new class data k can be obtained, and through incremental learning, a model ModelY(k) capable of classifying faults corresponding to new class data k and faults corresponding to old class data can also be obtained.

[0041] ModelX(k) outputs the LSE value of each data it receives, and then identifies whether the corresponding data is new class data or old class data according to the comparison result of the LSE value and the set threshold. Wherein, the output result of the model ModelY(k) for new class data k is defined as The output result for old class data is defined as

[0042] Step four, after the real-time time domain signal of the device to be tested is converted into a two-dimensional time-frequency image, it is input into the model ModelX(k) to obtain the current old class data and new class data, and then the current old class data and new class data are input into the model ModelY(k), which realizes fault class classification. Wherein, when the model ModelY(k) classifies faults, it dynamically adjusts the full connection layer weight of the model ModelY(k) according to the data class received by ModelY(k).

[0043] A classification method using multi-scale feature fusion is used for the input time-frequency image, each input time-frequency image is constructed into three categories of original image, 1 / 2 resolution image and 1 / 4 resolution image, and after feature extraction of each image, it is input into the Transformer encoder for feature fusion. In addition, the self-attention formula based on Transformer is: Wherein Q, K and V are query, key and value vector matrices respectively, d k is a scaling factor.

[0044] In this embodiment, the formula of continuous wavelet transform is: Wherein s represents a scaling factor, τ represents a translation factor, represents the complex conjugate of the sub-wavelet.

[0045] In one embodiment, the new class data k refers to the fault data time-frequency image classification class introduced in the kth increment. When k = 1 (i.e., the first increment), the old class is the original class; when k ≥ 2, the old class image data includes the original fault data image class and the set of all fault data image classification classes newly added from the first increment to the k-1th increment. For example, when training, the third increment, the old class data contains the original fault data class and the first and second newly added fault data classes. In addition, the model ModelX(k) is a neural network model ModelZ trained using time-frequency image training data belonging to the kth newly added fault class, thereby obtaining a model that can classify new class data k; the model ModelY(k) is a time-frequency image classification model ModelY(k-1) learned in the k-1th increment using time-frequency image training data belonging to the kth newly added fault class and the total loss function Loss, thereby obtaining a model ModelY(k) that can classify the fault corresponding to the new class data k. The total loss function expression of the model ModelY(k) is: TotelLoss = a·Loss D + b·Loss CE + g·Loss balance , where Loss D represents the distillation loss term, Loss CE represents the cross-entropy loss term, and Loss balance represents the balance loss term, and a, b, and g are weight coefficients of each loss term.

[0046] The identification steps of the new class data and the old class data are as follows:

[0047] F4: Using the kth newly added class time-frequency image classification model ModelX(k) to process the input image data to obtain the output result based on the model ModelX(k). The output result sequentially represents the determination result of the kth newly added class time-frequency image classification model ModelX(k) that the current image data is classified into the first to bth newly added classes, where b represents the number of the kth newly added classes, and b ≥ 1.

[0048] F5: Then, according to the output result of each image data obtained in the F4 step, using the LSE (Log-Sum-Exp) function, the LSE value of the current classification result is calculated, i.e., LSE(O1(x)), …, LSE(O b (x)); the LSE value is compared with the set threshold value μ, i.e., if max(LSE(O1(x)), …, LSE(O b(x)))≤μ (μ is a threshold coefficient preset in advance), the LSE value corresponding to the fault data image belongs to the old fault category data image, otherwise the LSE value corresponding to the test set fault data image belongs to the newly added fault category data image.

[0049] The output result of the model ModelY(k) is defined as O a+b (x), wherein a represents the number of original old category fault data time-frequency images at the kth increment, a>1, and b represents the number of newly added category fault data time-frequency images at the kth increment, b>1, so that the expression of the output result of the model ModelY(k) and the full connection layer weight is as follows:

[0050] O a+b (x)=[O1(x),...,O a (x),O a+1 (x),...,O b (x)]=W ModelY T φ(x)

[0051] In the formula, W ModelY represents the full connection layer weight of the fault data time-frequency image classification model ModelY(k) of the kth increment, T represents a temperature coefficient, and φ(x) represents a feature extractor.

[0052] Wherein, W ModelY can be divided into W old and W new according to the different weights of new and old image data categories. The output result of the model ModelY(k) for the newly added category fault data image in the data set after the kth increment is defined as The output result of the model ModelY(k) for the old category fault data image is defined as

[0053]

[0054] In the formula, η represents a positive number with a value range of (0-1), and by dynamically adjusting the value of η, the fault data time-frequency images of new and old categories can tend to the correct classification categories. When η is in the interval (0.82, 0.96), better fault diagnosis classification results can be obtained.

[0055] The application also provides a time-frequency graph fault diagnosis system based on multi-scale feature fusion and incremental learning, which comprises a memory and a processor, the memory stores a computer program, and the processor executes the computer program to execute the time-frequency graph fault diagnosis method based on multi-scale feature fusion and incremental learning as described above.

[0056] The application further provides a computer readable storage medium, which stores machine executable instructions, and the machine executable instructions, when called and executed by a processor, cause the processor to implement the time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning as described above.

[0057] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A time-frequency graph fault diagnosis method based on multi-scale feature fusion and incremental learning, characterized in that: The method comprises the following steps: Step 1: Divide the time domain signals of different fault types of the device under test into multiple windows, perform continuous wavelet transform on each window to obtain the spectrum within each window, and then combine all the obtained spectra to obtain a two-dimensional time-frequency image to generate a time-frequency graph dataset; divide the data in the time-frequency graph dataset into old category data and new category data; Step 2: Use the old category data as training data and train it on the initial neural network Model with Transformer as the network structure to obtain the model Model Z that can classify the faults corresponding to the old category data; Step 3: Divide the new category data into predetermined batches, and then train the model ModelZ based on incremental learning to obtain the model ModelX(k) that can identify the new category data and the model ModelY(k) that can classify and diagnose the faults corresponding to the new category data and the old category data; Step 4: Convert the real-time time domain signal of the device under test into a two-dimensional time-frequency image and input it into the model ModelX(k) to obtain the current old category data and the new category data, and then input the current old category data and the new category data into the model ModelY(k). The model ModelY(k) implements fault category classification; wherein, when the model ModelY(k) performs fault category classification, the fully connected layer weight of the model ModelY(k) is dynamically adjusted according to the data category received by ModelY(k).

2. The time-frequency graph fault diagnosis method based on multi-scale feature fusion and incremental learning according to claim 1 is characterized in that: ModelX(k) outputs the LSE value of each data it receives, and then identifies whether the corresponding data is new category data or old category data based on the comparison result of the LSE value and the set threshold.

3. The time-frequency graph fault diagnosis method based on multi-scale feature fusion and incremental learning according to claim 1 is characterized in that: The calculation formula of the fully connected layer weight of model ModelY(k) is: O a+b (x)=[O1(x),...,O a (x),O a+1 (x),...,O b (x)]=W ModelY T φ(x) Where W ModelY represents the fully connected layer weights of the time-frequency image classification model ModelY(k) for the k-th incremental fault data, T represents the temperature coefficient, and φ(x) represents the feature extractor.

4. The time-frequency graph fault diagnosis method based on multi-scale feature fusion and incremental learning according to claim 1 is characterized in that: in, The output of model ModelY(k) for new category data k is defined as The output result of the old class data is defined as According to the different weights of the new and old image data categories, W ModelY Divided into W old and W new ; The output expression of model ModelY(k) is: Where η represents a positive number ranging from (0-1).

5. The time-frequency graph fault diagnosis method based on multi-scale feature fusion and incremental learning according to claim 4 is characterized in that: The value of η is (0.82, 0.96).

6. The time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning according to any one of claims 1 to 5, characterized in that: The Transformer-based self-attention formula is: Where Q, K, and V are query, key, and value vector matrices, respectively. k is the scaling factor.

7. The method for time-frequency diagram fault diagnosis based on multi-scale feature fusion and incremental learning according to claim 6, characterized in that: The formula for continuous wavelet transform is: Where s represents the scaling factor, τ represents the translation factor, represents the complex conjugate of the sub-wavelet.

8. The time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning according to any one of claims 1 to 5, characterized in that: The total loss function expression of model ModelY(k) is: TotalLoss=α·Loss D +β·Loss CE +γ·Loss balance , where Loss D Represents the distillation loss term, Loss CE Represents the cross entropy loss term, Loss balance represents the balance loss term, α, β, and γ are the weight coefficients of each loss term respectively.

9. A time-frequency graph fault diagnosis system based on multi-scale feature fusion and incremental learning, characterized by: The fault diagnosis system includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it executes the time-frequency graph fault diagnosis method based on multi-scale feature fusion and incremental learning according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor, the machine-executable instructions prompt the processor to implement the time-frequency graph fault diagnosis method based on multi-scale feature fusion and incremental learning as described in any one of claims 1-8.

Images