A multi-modal gearbox fault diagnosis method based on deep transfer learning

Abstract

This invention discloses a multimodal gearbox fault diagnosis method based on deep transfer learning, addressing the problem of poor gearbox fault diagnosis capability under unlabeled sample conditions, belonging to the field of gearbox fault diagnosis technology. The method includes: collecting raw vibration signals under different operating conditions, defining them as source and target domains; fusing multimodal information in the time and frequency domains using a data-level fusion method and dividing the collected signals into samples; constructing a deep multimodal adversarial transfer network model, extracting fault information features from the source and target domains through iterative adversarial training to adapt to the joint probability distribution of the source and target domains, and utilizing the abundant fault label information in the source domain to ensure accurate fault category discrimination; finally, obtaining a trained transfer diagnosis model for the target domain. This method is applicable to gearbox fault diagnosis under different operating conditions, i.e., transfer diagnosis between different operating conditions or different fault types, exhibiting high accuracy and good generalization performance.

Description

Technical Field

[0001] This invention relates to a gearbox fault diagnosis method, specifically a multimodal gearbox fault diagnosis method based on deep transfer learning, belonging to the field of gearbox fault diagnosis technology. Background Technology

[0002] With the rapid development of modern industrial technology, people have put forward increasingly higher reliability requirements for the operation of gear transmission systems. Mechanical equipment is developing towards high precision, high efficiency, automation, and intelligence. However, in actual operation, gearboxes often operate under complex and harsh conditions, and key components such as gears and bearings are prone to failure during equipment operation. Therefore, it is crucial to study effective fault diagnosis models, focusing on gears and bearings of gearboxes, in order to ensure their safe and reliable operation.

[0003] Currently, with the continuous development of artificial intelligence, intelligent diagnostic methods based on deep learning are being increasingly applied to the field of gearbox fault diagnosis. However, in actual engineering, there are often more normal data samples and fewer typical fault label data. Moreover, most vibration data are not labeled, and manual labeling of data is time-consuming and laborious.

[0004] Therefore, fault diagnosis models based on unlabeled samples are an important problem that urgently needs to be solved. Researching intelligent fault diagnosis methods for gearboxes based on unlabeled samples has significant theoretical and engineering value for reducing transmission system failures, improving operational reliability and efficiency, and preventing unplanned downtime. Summary of the Invention

[0005] The purpose of this invention is to address the problem of poor gearbox fault diagnosis capability in the case of unlabeled data samples for the critical components of gearboxes, such as gears and bearings, in practical engineering. This invention provides a multimodal gearbox fault diagnosis method based on deep transfer learning. It employs multimodal information fusion and transfer learning theory for fault diagnosis, constructing a deep multimodal adversarial transfer network based on unsupervised learning, which exhibits good accuracy and generalization performance.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a multimodal gearbox fault diagnosis method based on deep transfer learning, comprising the following steps:

[0007] S1. Acquire raw vibration signals:

[0008] The raw vibration signals of the gearbox under different working conditions were collected and set as source domain and target domain data, where the source domain is labeled data and the target domain is unlabeled data;

[0009] S2, Multimodal Information Fusion:

[0010] Extract the time-domain waveform sequence from the original vibration signal acquired and set in step S1 to obtain the time-domain vibration signal. Then, perform a Fast Fourier Transform (FFT) on the time-domain vibration signal to obtain the frequency-domain vibration signal. Finally, obtain the time-domain sample x. i and frequency domain sample x t Z-score standardization is performed, and data-level fusion of the time and frequency domains is carried out. The processed source and target domain data samples are divided into training and test sets, and the data sampling of the training and test sets does not overlap.

[0011] S3. Construct a multimodal adversarial transfer network model:

[0012] The multimodal adversarial transfer network model consists of a feature extractor G. f (θ f ), category classifier G y (θ y ) and domain discriminator G d (θ d The feature extractor G is composed of... f (θ f A one-dimensional convolutional neural network is used to receive data samples from the source or target domain and extract fault information from them; a category classifier G y (θ y The received features are classified into task faults through a fully connected layer, and mapped to probability outputs through a Softmax function; the domain discriminator G d (θ d The system determines whether the input is from the source domain or the target domain. The model is constructed using a 1D-CNN network structure as the feature extractor, and the model training uses the Adam optimizer as the stochastic gradient descent optimization method to correct the deviation between the first-order momentum and the second-order momentum to update the network parameters.

[0013] The objective loss function of a multimodal adversarial transfer network model The expression is:

[0014] In the formula, The traditional cross-entropy loss is used for classifiers, where λ is a hyperparameter. For the domain adversarial classification loss of the domain discriminator, θ f θ d θ y These represent the network parameters of the feature extractor, domain discriminator, and category classifier, respectively.

[0015] The expression is:

[0016] The expression is:

[0017] In the above formula, n s n represents the number of samples in the source domain. t G represents the number of samples in the target domain. y G represents a category classifier. f x represents the feature extractor. i φ(θ) represents the input to the feature extractor. f θ y ) represents the regularization term for the model parameters. For the domain discriminant loss of data i;

[0018] S4. Using the fused data samples from the source and target domains as input, a transfer diagnostic model for the target domain is trained.

[0019] The processed training sets from the source and target domains in step S2 are input into the multimodal adversarial transfer network model constructed in step S3, based on the target loss function constructed in step S3. The network parameters are optimized through iterative adversarial training using the stochastic gradient descent algorithm. After training, the optimal network parameters are saved. and The trained transfer diagnostic model for the target domain is obtained;

[0020] S5. Input the target domain test sample into the trained transfer diagnostic model to obtain the fault diagnosis result:

[0021] In step S4, the target domain data sample processed in step S2 is input into the transfer diagnostic model trained to perform fault detection, and finally the fault diagnosis result is obtained.

[0022] In step S1, the tag data includes normal tag data and fault type tag data.

[0023] In step S2, the frequency domain sample x t The expression is: x t =FFT(x) i ), FFT(x) i ) represents the sum of each time-domain sample x i Perform a Fast Fourier Transform to the frequency domain, and take the first half of the frequency domain data result due to the symmetry of the spectrum.

[0024] The formula for calculating the standardized Z-score is: In the formula, x is the Z score value. m For time-domain sample x i The average value, x s For time-domain sample x iThe standard deviation.

[0025] In step S3, due to the feature extractor G f (θ f ) and domain discriminator G d (θ d The network parameters are iteratively optimized using an adversarial mechanism. Therefore, a gradient inversion layer (GRL) is constructed between the fully connected layer and the domain discriminator to reduce the loss of the domain discriminator. Backpropagation to feature extractor G f (θ f Reverse the gradient at time ) to perform different operations for forward or backward propagation, thus realizing G f (θ f ) and G d (θ d ( ) confrontation training;

[0026] During forward propagation, the gradient reversal layer acts as an identity mapping, R λ (x) = x, R λ This is a gradient inversion layer, where x is the input data; during backpropagation, the gradient direction is automatically inverted by multiplying by a negative unit identity matrix I. λ is a hyperparameter, and I is the identity matrix.

[0027] In step S4, the processed training sets of the source domain and the target domain are input into the multimodal adversarial transfer network model constructed in step S3. During the training process, the data samples of the source domain and the target domain need to be processed by the one-dimensional convolutional neural network of the feature extractor for feature extraction. Therefore, the two share the structure and weights of the feature extractor.

[0028] In step S4, the feature extractor G is minimized through backpropagation during the training process. f (θ f The loss and category classifier G y (θ y ) loss optimization network parameters θ f and θ y The backpropagation minimization expression is:

[0029]

[0030] During training, the domain discriminator G is maximized through backpropagation. d (θ d ) loss optimization network parameters θ d The backpropagation maximization expression is:

[0031] The optimal network parameters are obtained by optimizing the category classification loss and domain adversarial loss through training, thereby adapting the marginal probability distribution and conditional probability distribution of the source domain and the target domain to obtain a transfer diagnostic model for the target domain.

[0032] The beneficial effects of this invention are:

[0033] 1) The method of this invention is dedicated to gearbox fault diagnosis under unlabeled samples; by fusing time-domain and frequency-domain multimodal information as input, the model is trained and optimized in a deep adversarial network with the help of adversarial strategies, while considering the joint distribution adaptation of source domain and target domain data, so that it can better adaptively learn fault diagnosis knowledge, thereby effectively solving the fault diagnosis task of gearbox under unlabeled samples.

[0034] 2) The method of the present invention uses a data-level fusion approach to fuse the time and frequency domains of the signal as the input of the model, so as to fully extract information features and avoid the problems of information omission, incompleteness and high randomness caused by a single mode.

[0035] 3) The multimodal adversarial transfer network model constructed in the method of this invention takes into account the two key points of domain invariance and discriminability, which enables the model to better diagnose the test samples in the target domain; and by calculating the target loss function, the network model is optimized to achieve better fault diagnosis effect.

[0036] 4) In the method of the present invention, a gradient inversion layer is constructed between the feature extraction layer and the classifier, so that the gradient of the domain classification loss of the domain classifier is automatically inverted before it is backpropagated to the feature extractor. Attached Figure Description

[0037] Figure 1 This is a flowchart of the fault diagnosis method of the present invention;

[0038] Figure 2 This is the overall structure diagram of the multimodal adversarial transfer network model constructed in this invention;

[0039] Figure 3 These are the diagnostic results of each migration task under different working conditions in the embodiments of the present invention;

[0040] Figure 4 These are the accuracy change curves of each model transfer diagnosis training process for task A to B in this embodiment of the invention.

[0041] Figure 5 These are the accuracy change curves of each model during the transfer diagnosis training process from task B to A in this embodiment of the invention.

[0042] Figure 6 This is the t-SNE visualization result in an embodiment of the present invention. Detailed Implementation

[0043] The present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments.

[0044] like Figure 1-6 As shown, to verify the feasibility and effectiveness of the fault diagnosis method of the present invention, gears and bearings are used as experimental research objects. The Southeast University (SEU) gearbox dataset is used as sample data in different domains to design a diagnostic transfer task, further explore the fault diagnosis of key components of gearbox, and consider the transfer fault diagnosis research under different working conditions and different fault types.

[0045] Example: This invention provides a multimodal gearbox fault diagnosis method based on deep transfer learning, comprising the following steps:

[0046] S1. Acquire raw vibration signals:

[0047] Raw vibration signals of the gearbox under different operating conditions were collected. This example uses gearbox data from Southeast University, which consists of two subsets: a bearing dataset and a gear dataset. Each subset contains one normal state (N) and four fault states. During the experiment, eight channels of data were collected; in this example, data from channel 2 was used. Since the SEU dataset has sufficient data points, non-overlapping sampling was used to acquire the experimental data, with a sampling length of 2048, to avoid test leakage and ensure the fairness of the test results.

[0048] The collected datasets were set as source and target domain data, with the source domain consisting of labeled data and the target domain consisting of unlabeled data. The number of samples in both the source and target domains was set to 1000. The experimental dataset is shown in Table 1 below: Table 1: Experimental Gearbox Dataset

[0049]

[0050] S2, Multimodal Information Fusion:

[0051] Extract the time-domain waveform sequence from the original vibration signal acquired and set in step S1 to obtain the time-domain vibration signal. Then, perform a Fast Fourier Transform (FFT) on the time-domain vibration signal to obtain the frequency-domain vibration signal. Finally, obtain the time-domain sample x. i and frequency domain sample x t Perform Z-score standardization.

[0052] The time and frequency domain modes of the processed signal are spliced ​​together to achieve data-level fusion, which enhances the complementarity between modal information. The processed source and target domain data samples are divided into two parts: a training set and a test set, which are used to train and test the model. To avoid test leakage, the sampling of these two parts of the data does not overlap, so as to avoid the randomness of sampling and ensure the fairness of the test results.

[0053] The original time-domain vibration signal is used as one modal input of the model, while the frequency-domain vibration signal after fast Fourier transform is used as another modal input. After standardization, the signal after data-level fusion of the two modes is used as the final input of the model to ensure that the model can obtain sufficient and comprehensive fault information features in the feature extraction stage.

[0054] S3. Construct a multimodal adversarial transfer network model:

[0055] The multimodal adversarial transfer network model consists of a feature extractor G. f (θ f ), category classifier G y (θ y ) and domain discriminator G d (θ d The feature extractor G is composed of... f (θ f A one-dimensional convolutional neural network is used to receive data samples from the source or target domain and extract fault information from them; a category classifier G y (θ y The received features are classified into task faults through a fully connected layer, and mapped to probability outputs through a Softmax function; the domain discriminator G d (θ d It determines whether the input is from the source domain or the target domain, and uses the feature information from the source domain and the target domain to perform continuous adversarial training, thereby achieving the purpose of confusing the fault features of the source domain and the target domain.

[0056] This network model uses the concept of GAN for transfer learning when studying the distribution adaptation problem caused by the difference in data distribution between the source and target domains. It uses the adversarial network as an implicit metric function for the source and target domains to measure the similarity of their distributions. The generator in GAN is equivalent to the feature extractor in this model, and the discriminator corresponds to the metric function of data distribution. The two are trained through adversarial games to optimize the network parameters.

[0057] The model uses a 1D-CNN network structure as the feature extractor. Training employs the Adam optimizer with stochastic gradient descent to correct for biases in first and second momentum, updating network parameters accordingly. To mitigate overfitting, Dropout is used to discard some neurons from the neural network with a probability of p = 0.5. The learning rate is 1e-2, batch size ranges from 16 to 128, and the number of iterations is set to 100. The overall structural parameters of the model are shown in Table 2 below.

[0058] Table 2 shows the overall structural parameters of the model.

[0059]

[0060] S4. Using the fused data samples from the source and target domains as input, a transfer diagnostic model for the target domain is trained.

[0061] The processed training sets from the source and target domains in step S2 are input into the multimodal adversarial transfer network model constructed in step S3, based on the target loss function constructed in step S3. The network parameters are optimized through iterative adversarial training using the stochastic gradient descent algorithm. After training, the optimal network parameters are saved. and The trained transfer diagnostic model for the target domain is obtained.

[0062] S5. Input the target domain test sample into the trained transfer diagnostic model to obtain the fault diagnosis result:

[0063] In step S4, the target domain data sample processed in step S2 is input into the transfer diagnostic model trained to perform fault detection, and finally the fault diagnosis result is obtained.

[0064] 1. Set up a migration diagnostic task for the experiment.

[0065] 1) Set up migration diagnostic experiments under different operating conditions:

[0066] To verify the transfer diagnostic capability of the constructed model under different operating conditions of the gearbox, four sets of transfer diagnostic tasks were set up, using bearings and gears as experimental objects respectively. The experimental information is shown in Table 3 below. To ensure the reliability of the experimental results, the model's accuracy was taken as the average of 20 experiments as the final accuracy.

[0067] Table 3 shows the migration diagnosis experiments under different working conditions.

[0068]

[0069] The training results show that for bearings, the transfer learning accuracy for tasks T1 and T2 reaches 89.40% and 94.12%, respectively; for gears, the transfer accuracy for tasks T3 and T4 is 80.29% and 82.27%, respectively. This indicates that the method can effectively learn and extract information features under different working conditions, achieving cross-domain diagnosis.

[0070] 2) Set up migration diagnosis experiments for different fault types:

[0071] To verify the migration diagnosis capability of the constructed model under different fault types in the gearbox, two migration tasks were set up under the same operating condition of 20Hz-0V to verify the migration accuracy between different fault types. Experimental information is shown in Table 4 below: Table 4 shows the migration diagnosis experiments between different fault types.

[0072]

[0073] The experimental results show that the accuracy rate for task A→B is 92.58%, and the accuracy rate for task B→A is 86.35%, indicating that the model has good transfer diagnostic capabilities.

[0074] 2. To further verify the superiority of the proposed method, a comparative analysis of traditional transfer methods was conducted.

[0075] In transfer diagnostic experiments under different operating conditions, the diagnostic results obtained by comparing this method with traditional transfer learning methods TCA and JAN, as well as the deep transfer learning method DDC, are as follows: Figure 3 As shown, its accuracy is far superior to existing methods, further verifying the superiority of the proposed method in the migration diagnosis of gearboxes under different operating conditions.

[0076] In transfer diagnosis experiments for different fault types, this method was also compared and analyzed to assess the transfer learning capabilities of transfer methods TCA, JAN, and DDC. Figure 4 and 5 As shown, its accuracy is far superior to existing methods. The results demonstrate that the proposed method can still effectively transfer fault diagnosis knowledge under different fault types, exhibiting good generalization performance across different fault types.

[0077] 3. To more intuitively demonstrate the model's excellent transfer diagnostic performance, t-SNE was used to visualize the distribution of fault features before diagnosis and the distribution of fault features at different stages of model iteration.

[0078] Taking the T2 task as an example, such as Figure 6As shown in the figure, (a) represents the fault feature distribution before diagnosis, (b) represents the fault feature distribution after 30 iterations, (c) represents the fault feature distribution after 50 iterations, and (d) represents the fault feature distribution after 100 iterations. The figure demonstrates that the proposed method effectively eliminates the distribution differences between the source and target domains and possesses good diagnostic capabilities.

[0079] This invention relates to a deep transfer learning-based gearbox fault diagnosis method, focusing on gearbox fault diagnosis using unlabeled samples. By fusing time-domain and frequency-domain multimodal information as input, an optimized model is trained in a deep adversarial network using an adversarial strategy. Simultaneously, the joint distribution of source and target domain data is adaptively considered, enabling the model to better adaptively learn fault diagnosis knowledge.

[0080] The above description is only used to illustrate the technical solution of the present invention and is not intended to limit it. Any other modifications or equivalent substitutions made by those skilled in the art to the technical solution of the present invention, as long as they do not depart from the spirit and scope of the technical solution of the present invention, should be covered within the scope of the claims of the present invention.

Claims

1. A multimodal gearbox fault diagnosis method based on deep transfer learning, characterized in that: Includes the following steps: S1. Acquire raw vibration signals: The raw vibration signals of the gearbox under different working conditions were collected and set as source domain and target domain data, where the source domain is labeled data and the target domain is unlabeled data; S2, Multimodal Information Fusion: Extract the time-domain waveform sequence from the original vibration signal acquired and set in step S1 to obtain the time-domain vibration signal. Then, perform a Fast Fourier Transform (FFT) on the time-domain vibration signal to obtain the frequency-domain vibration signal. Finally, obtain the time-domain sample. and frequency domain samples Z-score standardization is performed, and data-level fusion of the time and frequency domains is carried out. The processed source and target domain data samples are divided into training and test sets, and the data sampling of the training and test sets does not overlap. S3. Construct a multimodal adversarial transfer network model: Multimodal adversarial transfer network models consist of feature extractors Category classifier Domain discriminant Composition, Feature Extractor A one-dimensional convolutional neural network is used to receive data samples from the source or target domain and extract fault information from them; a category classifier. The received features are classified into task faults using a fully connected layer, and then mapped to probability outputs using a Softmax function; a domain discriminator is also included. The model is constructed using a 1D-CNN network structure as the feature extractor, and the model training uses the Adam optimizer as the stochastic gradient descent optimization method to correct the deviation between the first-order momentum and the second-order momentum to update the network parameters. The objective loss function of a multimodal adversarial transfer network model The expression is: In the formula, The traditional cross-entropy loss for class classifiers, For hyperparameters, For the domain adversarial classification loss of the domain discriminator, , , These represent the network parameters of the feature extractor, domain discriminator, and category classifier, respectively. The expression is: , The expression is: ， In the above formula, Indicates the number of samples in the source domain. Indicates the number of samples in the target domain. Represents a category classifier. Indicates feature extractor, This represents the input to the feature extractor. For model parameter regularization, For data Domain discriminant loss; S4. Using the fused data samples from the source and target domains as input, a transfer diagnostic model for the target domain is trained. The processed training sets from the source and target domains in step S2 are input into the multimodal adversarial transfer network model constructed in step S3, based on the target loss function constructed in step S3. The network parameters were optimized through iterative adversarial training using the stochastic gradient descent algorithm. After training, the optimal network parameters were saved. , and This yields a trained transfer diagnostic model for the target domain. S5. Input the fused source and target domain samples to be tested into the trained transfer diagnostic model to obtain the fault diagnosis results: In step S4, the transfer diagnostic model trained with the input of the test set data of the source domain and the test set data of the target domain after processing in step S2 is used to detect faults and finally obtain the fault diagnosis results.

2. The multimodal gearbox fault diagnosis method based on deep transfer learning according to claim 1, characterized in that: In step S1, the tag data includes normal tag data and fault type tag data.

3. The multimodal gearbox fault diagnosis method based on deep transfer learning according to claim 1, characterized in that: In step S2, frequency domain samples The expression is: , Represented as each time-domain sample Perform a Fast Fourier Transform to the frequency domain, and take the first half of the frequency domain data result due to the symmetry of the spectrum. The formula for calculating the standardized Z-score is: In the formula, The Z score is the numerical value. Time-domain samples The average value, Time-domain samples The standard deviation.

4. The multimodal gearbox fault diagnosis method based on deep transfer learning according to claim 1, characterized in that: In step S3, due to the feature extractor Domain discriminant The network parameters are iteratively optimized using an adversarial mechanism. Therefore, a gradient inversion layer (GRL) is constructed between the fully connected layer and the domain discriminator to reduce the loss of the domain discriminator. Backpropagation to feature extractor Time-reversal gradients allows for different forward or backward propagation operations, enabling... and Combat training; During forward propagation, the gradient inversion layer acts as an identity mapping. , It is a gradient inversion layer. For the input data; During backpropagation, by multiplying by a negative unit identity matrix I Automatically invert the gradient direction , For hyperparameters, I It is an identity matrix.

5. The multimodal gearbox fault diagnosis method based on deep transfer learning according to claim 1, characterized in that: In step S4, the feature extractor is minimized through backpropagation during training. Loss and category classifier Loss optimization network parameters and The backpropagation minimization expression is: , During training, the domain discriminator is maximized through backpropagation. Loss optimization network parameters The backpropagation maximization expression is: .

Images