Time-frequency diagram fault diagnosis method and equipment based on multi-scale feature fusion and incremental learning

Abstract

The invention belongs to the related technical field of equipment part fault diagnosis, and discloses a time-frequency diagram fault diagnosis method and equipment based on multi-scale feature fusion and incremental learning. Training the initial neural network to obtain a model ModelZ capable of classifying faults corresponding to the old category data; (2) dividing the new category data into a predetermined batch, and training the model ModelZ based on incremental learning to obtain a model ModelX (k) capable of identifying the new category data and a model ModelY (k) capable of performing classification diagnosis on faults corresponding to the new category data and the old category data; (3) converting a real-time time-domain signal of the to-be-tested equipment into a two-dimensional time-frequency image and inputting the two-dimensional time-frequency image into the model ModelX (k), current old category data and new category data are obtained, and then fault category classification is achieved through the model ModelY (k). The invention provides efficiency and accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field related to equipment component fault diagnosis, and more specifically, relates to a time-frequency graph fault diagnosis method and device based on multi-scale feature fusion and incremental learning. Background Art

[0002] Healthy operation and maintenance of mechanical equipment and fault diagnosis are of paramount importance in industrial production and life. If mechanical equipment fails during operation, it may cause stagnation of the production line and inestimable losses. In this context, fault diagnosis of mechanical equipment has always been a research focus and hotspot. In recent years, fault diagnosis methods based on deep learning have achieved unprecedented development. Deep learning models have strong transferability and generalization capabilities, which makes deep learning increasingly used in fault diagnosis of mechanical equipment and mechanisms, and has achieved remarkable results; and with the development of convolutional neural networks, image recognition and classification have also become one of the hot directions in recent years. However, due to the characteristics of deep neural networks themselves, general fault diagnosis models will forget the old knowledge they have learned when learning new data. This phenomenon is called catastrophic forgetting.

[0003] To solve this problem, researchers have proposed an incremental learning method that aims to enable the model to learn new categories or new tasks while maintaining the performance of the learned categories or tasks. For example, when training new data, some old representative data sets are added for mixed training. This method solves the catastrophic forgetting phenomenon of deep learning networks to a certain extent. However, as the number of new data categories continues to increase, this method places increasing pressure on storage space, and the category imbalance problem becomes more serious. The classifier will tend to favor categories with more data. In addition, in actual applications, in many confidential environments, old category data is often kept confidential. This will cause a series of problems when users need to update the model when new data arrives. Alternatively, some scholars have found that the GAN network can be used to add some old virtual data sets generated while training new data sets, which can solve the problem of confidentiality of old category data. However, the experimental results obtained by using this method of virtually generating some old data sets in actual applications are not ideal.

[0004] In the existing multi-category incremental tasks, it is not difficult to find that the main factors affecting the classification results are: whether the storage space is sufficient, the confidentiality of the data set in the actual working environment, the imbalance of new and old category data, etc.; and in the actual fault diagnosis process, the fault signal is always updated in real time. In this incremental environment, it is very necessary to study an incremental learning method that does not require old category data sets and can classify and diagnose fault signals in a short time. Summary of the invention

[0005] In response to the above defects or improvement needs of the prior art, the present invention provides a time-frequency diagram fault diagnosis method and device based on multi-scale feature fusion and incremental learning, which aims to solve the problem that the existing fault identification method uses old category data, resulting in low classification speed and poor accuracy.

[0006] To achieve the above object, according to one aspect of the present invention, a method for time-frequency diagram fault diagnosis based on multi-scale feature fusion and incremental learning is provided, the method comprising the following steps:

[0007] Step 1: divide the time domain signals of different fault types of the equipment under test into multiple windows, and perform continuous wavelet transform on each window to obtain the spectrum in each window, and then combine all the obtained spectra to obtain a two-dimensional time-frequency image to generate a time-frequency image data set; divide the data in the time-frequency image data set into old category data and new category data;

[0008] Step 2: Use the old category data as training data and train it on the initial neural network Model with Transformer as the network structure to obtain a model Model Z that can classify the faults corresponding to the old category data;

[0009] Step 3: Divide the new category data into predetermined batches, and then train the model ModelZ based on incremental learning to obtain a model ModelX(k) capable of identifying the new category data and a model ModelY(k) capable of classifying and diagnosing faults corresponding to the new category data and the old category data;

[0010] Step 4: Convert the real-time time domain signal of the device under test into a two-dimensional time-frequency image and input it into the model ModelX(k) to obtain the current old category data and the new category data, and then input the current old category data and the new category data into the model ModelY(k), and the model ModelY(k) implements fault category classification; wherein, when the model ModelY(k) performs fault category classification, the fully connected layer weights of the model ModelY(k) are dynamically adjusted according to the data category received by ModelY(k).

[0011] Furthermore, ModelX(k) outputs the LSE value of each data it receives, and then identifies whether the corresponding data is new category data or old category data according to the comparison result between the LSE value and the set threshold.

[0012] Furthermore, the calculation formula of the fully connected layer weight of model ModelY(k) is:

[0013] O a+b (x)=[O1(x),...,O a (x),Oa+1 (x),...,O b (x)]=W ModelY T φ(x)

[0014] Where W ModelY represents the fully connected layer weights of the time-frequency image classification model ModelY(k) for the k-th incremental fault data, T represents the temperature coefficient, and φ(x) represents the feature extractor.

[0015] Furthermore, the output of model ModelY(k) for new category data k is defined as The output result for the old class data is defined as According to the different weights of the new and old image data categories, W ModelY Divided into W old and W new ; The output expression of model ModelY(k) is:

[0016]

[0017] In the formula, η represents a positive number ranging from (0-1).

[0018] Furthermore, the value of η is (0.82, 0.96).

[0019] Furthermore, the Transformer-based self-attention formula is: where Q, K, and V are query, key, and value vector matrices, respectively. k is the scaling factor.

[0020] Furthermore, the formula for continuous wavelet transform is: Where s represents the scaling factor, τ represents the translation factor, Represents the complex conjugate of the sub-wavelet.

[0021] Furthermore, the total loss function expression of model ModelY(k) is: TotalLoss = α·Loss D +β·Loss CE +γ·Loss balance , where Loss D Represents the distillation loss term, Loss CE Represents the cross entropy loss term, Loss balance represents the balanced loss term, α, β, and γ are the weight coefficients of each loss term.

[0022] The present invention also provides a time-frequency graph fault diagnosis system based on multi-scale feature fusion and incremental learning, the fault diagnosis system includes a memory and a processor, the memory stores a computer program, and the processor executes the time-frequency graph fault diagnosis method based on multi-scale feature fusion and incremental learning as described above when executing the computer program.

[0023] The present invention also provides a computer-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning as described above.

[0024] In general, compared with the prior art, the above technical solution conceived by the present invention, the time-frequency diagram fault diagnosis method and device based on multi-scale feature fusion and incremental learning provided by the present invention mainly have the following beneficial effects:

[0025] 1. The present invention adopts an incremental learning method that only uses newly added category samples (i.e., newly added category data k) for training, without requiring any old fault category data images (i.e., the original category data and the fault category data images added from the 1st to the k-1th time), thereby ensuring data privacy and greatly reducing the storage space required for the model, overcoming the problems caused by using old data.

[0026] 2. The multi-scale fusion feature extraction method is adopted, which not only ensures that the detailed feature information (local information) of the image can be effectively extracted, but also ensures that the contour information (global information) of the image can be effectively extracted, which improves the classification accuracy of the model to a certain extent.

[0027] 3. The LSE threshold method is combined with the dynamic weight of the fully connected layer of the new and old category data to avoid the category imbalance phenomenon during feature extraction and classification of the new and old category data. This method can be used to classify the new and old category data in a more trend-oriented manner, thereby improving the accuracy of model classification.

[0028] 4. The collected one-dimensional fault signal is converted into a two-dimensional time-frequency image, and the advantages of convolutional neural network in extracting and classifying two-dimensional image fault features are utilized to realize the fault diagnosis and fault type classification of bearings. It has greater generalizability and can be used for fault diagnosis of various industrial parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of a time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0031] See also Figure 1 The present invention provides a method for fault diagnosis of a time-frequency graph based on multi-scale feature fusion and incremental learning, and the method mainly comprises the following steps:

[0032] Step 1: divide the time domain signals of different fault types of the equipment under test into multiple windows, and perform continuous wavelet transform on each window to obtain the spectrum in each window, and then combine all the obtained spectra to obtain a two-dimensional time-frequency image to generate a time-frequency graph data set; divide the data in the time-frequency image data set into old category data and new category data.

[0033] In one embodiment, step one includes the following sub-steps:

[0034] F1: Rolling bearings with different fault types are installed on a mechanical simulation comprehensive fault test bench, and different working conditions are set for experiments to collect the time domain signals of the faulty bearings, i.e. the original time series data;

[0035] F2: Divide the time domain signal into multiple short time periods (windows) and perform continuous wavelet transform on each window to obtain the spectrum in each time period.

[0036] F3: Combine the frequency spectra of all time periods to generate a two-dimensional time-frequency image, create a time-frequency image dataset, and divide the data in the time-frequency image dataset into old category data and new category data.

[0037] In one implementation, 10 types of fault signals are collected and made into 64*64*3 time-frequency images, and a data set is made according to 1200 images of each type, and the ratio of the number of old category data to the number of new category data is 5:1.

[0038] Step 2: Use the old category data as training data and train it on the initial neural network Model with Transformer as the network structure to obtain a model Model Z that can classify the faults corresponding to the old category data.

[0039] Step three, divide the new category data into predetermined batches, and then train the model ModelZ based on incremental learning to obtain the model ModelX(k) that can identify the new category data and the model ModelY(k) that can classify and diagnose the faults corresponding to the new category data and the old category data.

[0040] Among them, when new category data k appears, the model needs to be retrained, but during the training process, the model only needs to train the newly added category data, and there is no need to repeatedly train the old category data. Through retraining, a model ModelX(k) that can classify the fault data images of the new category data k can be obtained, and incremental learning is also required to obtain a model ModelY(k) that can classify the faults corresponding to the new category data k and the faults corresponding to the old category data.

[0041] ModelX(k) outputs the LSE value of each data it receives, and then identifies whether the corresponding data is new category data or old category data based on the comparison result between the LSE value and the set threshold. Among them, the output result of model ModelY(k) for new category data k is defined as The output result for the old class data is defined as

[0042] Step 4: Convert the real-time time domain signal of the device under test into a two-dimensional time-frequency image and input it into the model ModelX(k) to obtain the current old category data and the new category data, and then input the current old category data and the new category data into the model ModelY(k), and the model ModelY(k) implements fault category classification; wherein, when the model ModelY(k) performs fault category classification, the fully connected layer weights of the model ModelY(k) are dynamically adjusted according to the data category received by ModelY(k).

[0043] The input time-frequency image is classified by multi-scale feature fusion, and each input time-frequency image is constructed into three categories: original image, 1 / 2 resolution image, and 1 / 4 resolution image. After feature extraction of each image, it is input into the Transformer encoder for feature fusion. In addition, the self-attention formula based on Transformer is: where Q, K, and V are query, key, and value vector matrices, respectively. k is the scaling factor.

[0044] In this implementation, the formula for continuous wavelet transform is: Where s represents the scaling factor, τ represents the translation factor, Represents the complex conjugate of the sub-wavelet.

[0045] In one embodiment, the new category data k refers to the time-frequency image classification category of the fault data introduced in the kth increment. When k=1 (i.e., the first increment), the old category is the original category; when k≥2, the old category image data includes the original fault data image category and the set of all fault data image classification categories newly added from the first increment to the k-1th increment. For example, when training, at the third increment, the old category data contains the original fault data category and the fault data category newly added for the first and second times. In addition, the model ModelX(k) uses the time-frequency image training data belonging to the kth newly added fault category to train the neural network model ModelZ, thereby obtaining a model that can classify the new category data k; the model ModelY(k) uses the time-frequency image training data belonging to the kth newly added fault category and the total loss function Loss to train the time-frequency image classification model ModelY(k-1) learned in the k-1th increment, thereby obtaining a model ModelY(k) that can classify the fault corresponding to the new category data k. The total loss function expression of model ModelY(k) is: TotalLoss = α·Loss D +β·Loss CE +γ·Loss balance , where Loss D Represents the distillation loss term, Loss CE Represents the cross entropy loss term, Loss balance represents the balanced loss term, α, β, and γ are the weight coefficients of each loss term.

[0046] The steps for identifying new and old category data are:

[0047] F4: Use the k-th newly added category time-frequency image classification model ModelX(k) to process the input image data and obtain the output result based on the model ModelX(k). The output result sequentially represents the judgment result of the k-th newly added category time-frequency image classification model ModelX(k) on classifying the current image data into the 1st to the b-th newly added categories, where b represents the number of the k-th newly added categories, and b≥1.

[0048] F5: Then, according to the output results of each image data obtained in step F4, the LSE value of the current classification result is calculated using the LSE (Log-Sum-Exp) function, that is, LSE (O1 (x)), ..., LSE (O b (x)); compare the LSE value with the set threshold μ, that is: if max(LSE(O1(x)),…,LSE(O b(x)))≤μ (μ is a preset threshold coefficient), the fault data image corresponding to the LSE value belongs to the old fault category data image, otherwise the test set fault data image corresponding to the LSE value belongs to the new fault category data image.

[0049] The output result of model ModelY(k) is defined as: a+b (x), where a represents the number of time-frequency images of the original old category fault data at the kth increment, a ≥ 1; b represents the number of time-frequency images of the newly added category fault data at the kth increment, b ≥ 1; so the output result of the model ModelY(k) and the expression of the fully connected layer weight are:

[0050] O a+b (x)=[O1(x),...,O a (x),O a+1 (x),...,O b (x)]=W ModelY T φ(x)

[0051] Where W ModelY represents the fully connected layer weights of the time-frequency image classification model ModelY(k) for the k-th incremental fault data, T represents the temperature coefficient, and φ(x) represents the feature extractor.

[0052] Among them, W can be calculated according to the different weights of the new and old image data categories. ModelY Divided into W old and W new After that, for the k-th incremental data set, the output result of the model ModelY(k) for the newly added category of fault data images is defined as The image output result for the old class fault data is defined as According to the dynamic weight, the final output expression can be obtained as follows:

[0053]

[0054] In the formula, η represents a positive number with a value range of (0-1). By dynamically adjusting the value of η, the time-frequency images of fault data of new and old categories can be made to tend to their respective correct classification categories. Among them, when η is in the interval of (0.82, 0.96), better fault diagnosis classification results can be obtained.

[0055] The present invention also provides a time-frequency graph fault diagnosis system based on multi-scale feature fusion and incremental learning, the fault diagnosis system includes a memory and a processor, the memory stores a computer program, and the processor executes the time-frequency graph fault diagnosis method based on multi-scale feature fusion and incremental learning as described above when executing the computer program.

[0056] The present invention also provides a computer-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning as described above.

[0057] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning, characterized in that: The method comprises the following steps: Step 1: divide the time domain signals of different fault types of the equipment under test into multiple windows, and perform continuous wavelet transform on each window to obtain the spectrum in each window, and then combine all the obtained spectra to obtain a two-dimensional time-frequency image to generate a time-frequency image data set; divide the data in the time-frequency image data set into old category data and new category data; Step 2: Use the old category data as training data and train it on the initial neural network Model with Transformer as the network structure to obtain a model Model Z that can classify the faults corresponding to the old category data; Step 3: Divide the new category data into predetermined batches, and then train the model ModelZ based on incremental learning to obtain a model ModelX(k) capable of identifying the new category data and a model ModelY(k) capable of classifying and diagnosing faults corresponding to the new category data and the old category data; Step 4: Convert the real-time time domain signal of the device under test into a two-dimensional time-frequency image and input it into the model ModelX(k) to obtain the current old category data and the new category data, and then input the current old category data and the new category data into the model ModelY(k), and the model ModelY(k) implements fault category classification; wherein, when the model ModelY(k) performs fault category classification, the fully connected layer weights of the model ModelY(k) are dynamically adjusted according to the data category received by ModelY(k).

2. The method for time-frequency diagram fault diagnosis based on multi-scale feature fusion and incremental learning as claimed in claim 1, characterized in that: ModelX(k) outputs the LSE value of each data it receives, and then identifies whether the corresponding data is new category data or old category data based on the comparison result of the LSE value and the set threshold.

3. The method for time-frequency diagram fault diagnosis based on multi-scale feature fusion and incremental learning as claimed in claim 1, characterized in that: The calculation formula of the fully connected layer weight of model ModelY(k) is: O a+b (x)=[O1(x),...,O a (x),O a+1 (x),...,O b (x)]=W ModelY T φ(x) Where W ModelY represents the fully connected layer weights of the time-frequency image classification model ModelY(k) for the k-th incremental fault data, T represents the temperature coefficient, and φ(x) represents the feature extractor.

4. The method for time-frequency diagram fault diagnosis based on multi-scale feature fusion and incremental learning as claimed in claim 1, characterized in that: in, The output of model ModelY(k) for new category data k is defined as The output result for the old class data is defined as According to the different weights of the new and old image data categories, W ModelY Divided into W old and W new ; The output expression of model ModelY(k) is: In the formula, η represents a positive number ranging from (0-1).

5. The method for time-frequency diagram fault diagnosis based on multi-scale feature fusion and incremental learning as claimed in claim 4, characterized in that: The value of η is (0.82, 0.96).

6. The method for time-frequency diagram fault diagnosis based on multi-scale feature fusion and incremental learning according to any one of claims 1 to 5, characterized in that: The Transformer-based self-attention formula is: where Q, K, and V are query, key, and value vector matrices, respectively. k is the scaling factor.

7. The method for time-frequency diagram fault diagnosis based on multi-scale feature fusion and incremental learning as claimed in claim 6, characterized in that: The formula for continuous wavelet transform is: Where s represents the scaling factor, τ represents the translation factor, Represents the complex conjugate of the sub-wavelet.

8. The method for time-frequency diagram fault diagnosis based on multi-scale feature fusion and incremental learning according to any one of claims 1 to 5, characterized in that: The total loss function expression of model ModelY(k) is: TotalLoss = α·Loss D +β·Loss CE +γ·Loss balance , where Loss D Represents the distillation loss term, Loss CE Represents the cross entropy loss term, Loss balance represents the balanced loss term, α, β, and γ are the weight coefficients of each loss term.

9. A time-frequency diagram fault diagnosis system based on multi-scale feature fusion and incremental learning, characterized by: The fault diagnosis system includes a memory and a processor, the memory stores a computer program, and the processor executes the time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning as described in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine executable instructions. When the machine executable instructions are called and executed by the processor, the machine executable instructions prompt the processor to implement the time-frequency diagram fault diagnosis method based on multi-scale feature fusion and incremental learning as described in any one of claims 1-8.

Images