Bearing fault diagnosis method and system based on sample enhancement and domain adaptation

Abstract

The invention provides a bearing fault diagnosis method and system based on sample enhancement and domain self-adaption, and belongs to the field of industrial intelligent manufacturing, and the method comprises the steps: employing wavelet transform, converting an original vibration signal into time-frequency representation, taking the time-frequency representation as an image and a time sequence, extracting image features and time sequence features, and carrying out the recognition of the image features and the time sequence features; carrying out feature fusion on the image features and the time sequence features through a weight network; inputting the fused features into a classification network to obtain a classification result of fault diagnosis; constructing an enhanced sample as a new time-frequency representation according to the size of the first attention score; a plurality of bearing data sets disclosed in other fields are used as a source domain, collected original vibration signals of the bearing are used as a target domain, and further training is performed on an image encoder, a time sequence encoder and a gating mechanism based on the source domain and the target domain. According to the method, efficient and accurate bearing fault diagnosis is realized under the condition of limited samples.

Description

Technical Field

[0001] This invention belongs to the field of intelligent industrial manufacturing, specifically relating to a bearing fault diagnosis method and system based on sample augmentation and domain adaptation. Background Technology

[0002] In recent years, with the development of artificial intelligence technology and the introduction of the intelligent manufacturing concept, more and more researchers have been studying industrial intelligent manufacturing. Fault diagnosis is a crucial part of intelligent manufacturing. Timely and effective fault diagnosis of mechanical equipment enables timely monitoring of equipment status and prompt replacement of failed or soon-to-be-failed components. When a component fails, it often leads to a production standstill, and the faulty component may damage other working components, affecting production efficiency and causing significant economic losses. Furthermore, mechanical systems often have complex organizational structures and intricate relationships between components, making fault diagnosis a resource-intensive process. Therefore, timely and effective fault diagnosis of industrial machinery is of great significance to industrial production and is one of the key issues that need to be addressed in the process of intelligent manufacturing.

[0003] Bearings are among the most widely used components in industrial machinery, especially in electrical and electric drive fields. In actual industrial scenarios, bearings are used to support mechanical parts and reduce friction between them. During the operation of mechanical equipment, bearings are often subjected to high-speed, high-load, and high-heat industrial environments. Therefore, the harsh operating environment leads to frequent bearing failures. Furthermore, bearing failures are diverse, including fatigue, wear, damage, cracks, corrosion, overheating, etc. Bearings with different failures have different remaining lifespans, and the handling methods for failed bearings also differ. Due to the commonality of bearings in industry and the frequency of their failures, research on bearing fault diagnosis is of great significance. Timely and effective bearing fault diagnosis can extend the service life of equipment, improve equipment utilization, and thus improve economic efficiency. This is also one of the important research contents for realizing intelligent manufacturing in industry.

[0004] In fault diagnosis, sensors are widely used for data acquisition in industrial scenarios. Sensors can collect various signals, including vibration signals, acoustic emission signals, force, rotational speed, current signals, power signals, and temperature. With the emergence of machine learning, new solutions have been provided for bearing fault diagnosis. Machine learning methods analyze bearing vibrations collected by sensors. However, these machine learning-based methods require manual feature extraction, a process that demands expert and domain knowledge. Furthermore, feature extraction methods need to be designed according to different types of monitored objects and signal sources, especially for multi-sensor-based fault diagnosis tasks. The bearing vibration signals collected by sensors contain a large amount of noise and redundant information, making feature engineering more difficult and labor-intensive. In addition, bearing fault diagnosis methods based on classic machine learning methods such as support vector machines and artificial neural networks treat the feature extraction process and the decision-making process as two independent parts, without considering the relationship between them. Optimization typically employs asynchronous optimization methods, and they struggle to learn complex nonlinear relationships.

[0005] In recent years, deep learning-based fault diagnosis methods have been continuously proposed. Deep learning methods have narrowed the gap between feature extraction, feature selection, and classification decisions, and their powerful feature extraction capabilities have enabled high-precision bearing fault diagnosis. However, research on bearing fault diagnosis still faces significant challenges. Problems such as how to more effectively extract fault features from bearing vibration signals and how to reduce the dependence of deep learning-based bearing fault diagnosis methods on the number of training samples remain unresolved. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this application proposes a bearing fault diagnosis method and system based on sample augmentation and domain adaptation, aiming to reduce the dependence of deep learning-based bearing fault diagnosis methods on the number of training samples and improve the detection performance of bearing fault diagnosis methods in limited sample fields.

[0007] In a first aspect, the present invention provides a bearing fault diagnosis method based on sample enhancement and domain adaptation, comprising:

[0008] Step S1: Acquire the raw vibration signal of the bearing;

[0009] Step S2: Use the continuous wavelet transform of the wavelet function as the complex Morlet wavelet to convert the original vibration signal into a time-frequency representation, where the same type of fault signal has different time-frequency representations;

[0010] Step S3: Treat the time-frequency representation as an image, segment each time-frequency representation into multiple non-overlapping or partially overlapping image blocks, and use the first convolutional neural network to perform linear projection on each image block to obtain... The first eigenvector of dimension; taking the time-frequency representation as a time series, each time-frequency representation is divided into multiple non-overlapping or partially overlapping time series blocks, and a second convolutional neural network is used to perform linear projection on each time series block to obtain The second eigenvector of dimension;

[0011] Step S4: Input the first classification label and the first feature vector into the image encoder to extract image features, input the second classification label and the second feature vector into the time series encoder to extract time series features, and calculate the first attention score between the local features of the second feature vector;

[0012] Step S5: Employ a gating mechanism to fuse image features with time-series features through a weighted network;

[0013] Step S6: Input the fused features into the classification network to obtain the classification result of the fault diagnosis;

[0014] Step S7: Based on the magnitude of the first attention score, construct enhanced samples and input the enhanced samples into step S2 as a new time-frequency representation;

[0015] Step S8: Using multiple publicly available bearing datasets from other fields as the source domain and the original vibration signal of the bearing collected in step S1 as the target domain, further train the image encoder, time series encoder, and gating mechanism based on the source domain and the target domain. Assign a discriminator and a classifier to each of the multiple source domains. The discriminator is used to distinguish whether the feature comes from the source domain or the target domain, and the classifier is used to identify the fault type. Apply the further trained image encoder, time series encoder, and gating mechanism to steps S4 and S5.

[0016] The first and second classification labels are randomly generated Class Tokens and are d-dimensional features that can be self-learned during the training of the image encoder or time series encoder.

[0017] The step of inputting the first classification label and the first feature vector into an image encoder to extract image features, and inputting the first classification label and the first feature vector into a time series encoder to extract time series features, includes:

[0018] Step S4.1: Add the position code to the first feature vector element by element to obtain the first additive feature, and add the position code to the second feature vector element by element to obtain the second additive feature. The position code represents the position information of the local features of the image block or time series block in the global context.

[0019] Step S4.2: Input the first classification label and the first additive feature into the image encoder for multiple encodings to extract features and obtain image features;

[0020] Step S4.3: Input the second classification label and the second additive feature into the time series encoder for multiple encodings to extract features, and obtain time series features. The time series encoder also calculates the first attention score between each local feature in the second additive feature.

[0021] Step S4.4: In each encoding operation of the image encoder, the second attention score is used as the feature selection probability to dynamically select some features of the image features or time series features to participate in the subsequent linear layer transformation of the image encoder.

[0022] The time-series encoder calculates the first attention score between each local feature in the second additive feature, as follows:

[0023] ;

[0024] in, The first attention score matrix, d represents the second additive feature, d is the dimension, and softmax is the softmax function.

[0025] The second attention score is calculated as follows:

[0026] ;

[0027] in, The second attention score, i.e. and All The maximum attention score minus the average attention score. From or Randomly selected local features for The number of features, where d is the number of dimensions. for or Local features in for Local features in.

[0028] The method employs a gating mechanism to fuse image features with time-series features through a weighted network, including:

[0029] Two weighted networks consisting of linear layers are used to assign fusion weights to image features and time series features respectively. The two weight values ​​are normalized by Softmax and then fused by weighted summation.

[0030] The step of constructing enhanced samples based on the magnitude of the first attention score includes:

[0031] Arrange the first attention scores in descending order and select the highest first attention scores. The corresponding second summation features constitute the initial set of important timestamps. ;

[0032] From the initial set of important timestamps Random selection Each timestamp serves as a significant timestamp, forming a set of significant timestamps. ;

[0033] In time-frequency representation, for those not selected to form the initial important timestamp set The time node is randomly selected from a set of important timestamps. The elements in the sample are replaced to obtain the enhanced sample.

[0034] The further training of the image encoder, time series encoder, and gating mechanism based on the source and target domains includes:

[0035] In the forward propagation, each source domain is paired with a target domain to form a training pair. The image encoder, time series encoder, and gating mechanism extract features to obtain source domain features and target domain features. The source domain features and target domain features are then input into a discriminator for binary classification to obtain the classification result of whether the feature comes from the source domain or the target domain. Finally, the source domain features and target domain features are input into a classifier for multi-class classification to obtain the fault type.

[0036] Parameter optimization is performed during backpropagation. The parameters of the discriminator's classifier network and the feature extraction network of the classifier's fault diagnosis model are updated based on the classification loss of the discriminator and classifier. During training, the optimization objectives include minimizing the classification error of the source domain and minimizing the domain discrimination error between the two domains.

[0037] Secondly, the present invention provides a bearing fault diagnosis system based on sample enhancement and domain adaptation, comprising:

[0038] The signal acquisition module is used to acquire the raw vibration signal of the bearing;

[0039] The wavelet transform module is used to convert the original vibration signal into a time-frequency representation by using a continuous wavelet transform with the wavelet function being a complex Morlet wavelet. Among them, fault signals of the same type have different time-frequency representations.

[0040] The feature representation module is used to treat the time-frequency representation as an image, segment each time-frequency representation into multiple non-overlapping or partially overlapping image blocks, and use a first convolutional neural network to perform linear projection on each image block to obtain... The first eigenvector of dimension; taking the time-frequency representation as a time series, each time-frequency representation is divided into multiple non-overlapping or partially overlapping time series blocks, and a second convolutional neural network is used to perform linear projection on each time series block to obtain The second eigenvector of dimension;

[0041] The feature extraction module is used to input the first classification label and the first feature vector into the image encoder to extract image features, input the second classification label and the second feature vector into the time series encoder to extract time series features, and calculate the first attention score between the local features of the second feature vector.

[0042] The gating mechanism module is used to fuse image features and time series features through a weighted network using a gating mechanism.

[0043] The fault diagnosis module is used to input the fused features into the classification network to obtain the classification result of the fault diagnosis.

[0044] The enhanced sample construction module is used to construct enhanced samples based on the magnitude of the first attention score. The enhanced samples are then input into the wavelet transform module as a new time-frequency representation.

[0045] The domain adaptation module is used to take multiple publicly available bearing datasets from other fields as the source domain and the original vibration signals of the bearings acquired by the signal acquisition module as the target domain. Based on the source domain and the target domain, the image encoder, time series encoder, and gating mechanism are further trained. For each of the multiple source domains, a discriminator and a classifier are assigned. The discriminator is used to distinguish whether the features come from the source domain or the target domain, and the classifier is used to identify the fault type. The further trained image encoder, time series encoder, and gating mechanism are applied to the feature extraction module and the gating mechanism module.

[0046] Thirdly, this application proposes an electronic device comprising: one or more processors, and a memory for storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the aforementioned bearing fault diagnosis method based on sample augmentation and domain adaptation.

[0047] Fourthly, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the aforementioned bearing fault diagnosis method based on sample augmentation and domain adaptation.

[0048] Fifthly, this application proposes a computer program product, including a computer program or instructions that, when executed by a processor, implement the aforementioned bearing fault diagnosis method based on sample enhancement and domain adaptation.

[0049] Beneficial effects:

[0050] This application proposes a bearing fault diagnosis method and system based on sample augmentation and domain adaptation, which achieves efficient and accurate bearing fault diagnosis under limited sample conditions, and has the following significant advantages:

[0051] (1) Improve diagnostic performance with limited samples. Through the synergistic effect of sample diversity enhancement and domain adaptive transfer learning, the model can maintain high diagnostic accuracy even with only a small number of labeled samples. In actual industrial scenarios, it can effectively cope with the challenge of scarce fault samples and reduce the cost of data collection and labeling.

[0052] (2) Enhance feature representation capability. Feature extraction of different modalities can mine fault features from different dimensions, ensuring the comprehensiveness and representativeness of features. The random probability self-attention mechanism enhances the adaptability of feature selection, enabling the model to focus on the most discriminative features. Compared with traditional methods, the feature dimensions are richer and the discriminability between features is significantly improved.

[0053] (3) Reduce the risk of overfitting. The random probability self-attention mechanism increases the data diversity during the training process through dynamic feature selection. Attention enhancement training generates new enhanced samples based on the selection of part of the time frequency of attention score, which is used for the enhancement training of the model. This improves the stability and robustness of the model in the scenario of limited samples.

[0054] (4) Enhance the generalization ability of the model. By mapping bearing fault samples from different domains to the same feature space as the target domain samples, multi-source domain adaptive transfer learning is realized, which further improves the performance and generalization ability of the diagnostic model in the target domain.

[0055] (5) Achieve end-to-end intelligent diagnosis. It eliminates the reliance on expert knowledge and manual feature engineering, significantly lowers the technical threshold, and automates the entire diagnosis process. No manual intervention is required from data input to result output. It supports real-time or near real-time bearing condition monitoring and meets the timeliness requirements of industrial sites.

[0056] (6) It has engineering applicability. It solves the problem of fault diagnosis under limited sample conditions, provides reliable technical support for predictive maintenance of industrial equipment, helps to reduce equipment maintenance costs, and effectively avoids production interruptions caused by sudden equipment failures through timely and accurate fault diagnosis, extends equipment service life, improves equipment utilization, and creates significant economic benefits for enterprises.

[0057] In summary, this invention not only solves the technical challenge of bearing fault diagnosis under limited sample conditions, but also has significant engineering application value, providing strong technical support for promoting the development of intelligent manufacturing in industry. Attached Figure Description

[0058] Figure 1 A flowchart of a bearing fault diagnosis method based on sample augmentation and domain adaptation according to an embodiment of the present invention;

[0059] Figure 2 A schematic flowchart of a bearing fault diagnosis method based on sample augmentation and domain adaptation according to an embodiment of the present invention;

[0060] Figure 3 Time-frequency representation of different types of fault signals in embodiments of the present invention;

[0061] Figure 4 A schematic diagram of a bearing fault diagnosis system based on sample augmentation and domain adaptation according to an embodiment of the present invention. Detailed Implementation

[0062] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0063] Existing bearing fault diagnosis methods have the following main drawbacks.

[0064] (1) Existing machine learning-based bearing fault diagnosis methods require manual feature extraction, which requires expert knowledge and domain knowledge. Feature extraction methods need to be designed according to different types of signals. Furthermore, there is a lot of noise and redundant information in the bearing vibration signals collected by sensors, which makes feature engineering more difficult.

[0065] (2) Existing bearing fault diagnosis methods based on classic machine learning methods such as support vector machines and artificial neural networks treat the feature extraction process and the decision-making process as two independent parts, without considering the relationship between the two parts. In the optimization process, asynchronous optimization methods are generally used, and it is difficult to learn complex nonlinear relationships.

[0066] (3) In actual industrial scenarios, it is difficult to collect enough fault signals for training deep learning models, which often results in deep learning-based fault diagnosis methods failing to achieve the expected performance. Problems such as reducing the dependence of deep learning-based bearing fault diagnosis methods on the number of training samples have not yet been effectively solved.

[0067] Therefore, this invention proposes a bearing fault diagnosis method and system based on sample augmentation and domain adaptation to solve the above problems.

[0068] For scenarios with limited samples, this invention designs methods from two perspectives: enhancing sample diversity and domain-adaptive transfer learning. To improve sample diversity, the invention converts the one-dimensional bearing vibration signal into a two-dimensional time-frequency representation. This time-frequency representation is used for feature extraction in both the image modality and the time-series modality. A gating mechanism is used to fuse the features of the two modalities. During feature extraction, a stochastic probabilistic self-attention mechanism is proposed to dynamically select some features for forward propagation, improving the feature diversity of each sample. Furthermore, an attention-enhanced training method is designed, selecting some time-frequency data based on attention scores to generate new enhanced samples for model enhancement training. In addition, to further improve the model's fault diagnosis performance, this invention designs a domain-adaptive transfer learning method, jointly training the model on multiple public datasets from other domains as well as existing datasets. This training method enables the fault diagnosis model to learn fault-related knowledge from other bearing datasets, further improving diagnostic performance and generalization ability in scenarios with scarce samples.

[0069] The ultimate goal of this invention is to develop a deep learning-based bearing fault diagnosis method for finite sample scenarios, so as to improve the detection performance of bearing fault diagnosis methods in finite sample scenarios.

[0070] Example 1:

[0071] This embodiment provides a bearing fault diagnosis method based on sample augmentation and domain adaptation, such as... Figure 1 , Figure 2 As shown, Figure 2 This is the overall architecture diagram of the technical solution of this invention. Wavelet transform is used to convert the one-dimensional original signal of the bearing into a two-dimensional time-frequency representation for training a deep learning model. To improve sample diversity, the time-frequency representation is used for feature extraction in both image and time-series modes. In feature extraction, a random probability self-attention mechanism is designed to select some features for subsequent feature extraction, further improving the feature diversity of each sample. A gating mechanism is used to fuse the features of the two modes for fault diagnosis. Then, this embodiment designs an attention-enhanced training method, selecting some time-frequency data based on attention scores to generate new enhanced samples for enhanced model training. Furthermore, to fully utilize publicly available bearing datasets (source domains) in other fields, this invention pairs different source domains with existing datasets (target domains). Then, multiple discriminators are used for domain category classification, and multiple classifiers are used for fault category classification. The domain adaptive transfer learning method allows the fault diagnosis model to improve its diagnostic performance and generalization ability in scenarios with scarce samples by learning knowledge from other source domains.

[0072] A bearing fault diagnosis method based on sample augmentation and domain adaptation, such as Figure 1 As shown, it specifically includes:

[0073] Step S1: Acquire the raw vibration signal of the bearing;

[0074] In this embodiment, the original vibration signal of the bearing It is a univariate time series, where, The original vibration signal only contains information about the change in bearing vibration amplitude over time, which is the number of sampling points.

[0075] Step S2: Use the continuous wavelet transform of the wavelet function as the complex Morlet wavelet to convert the original vibration signal into a time-frequency representation, where the same type of fault signal has different time-frequency representations;

[0076] In this embodiment, to extract more fault-related information, a continuous wavelet transform of the complex Morlet wavelet is used to convert the original vibration signal into a time-frequency representation. ,in, The number of features in the time domain. The number of features in the frequency domain, such as Figure 3 As shown, different types of fault signals have different time-frequency representations. Through this transformation, the energy of the signal is expanded in both time and frequency dimensions. The transformed time-frequency representation is a multivariate time series that has both time domain and frequency domain characteristics.

[0077] Step S3: Treat the time-frequency representation as an image, segment each time-frequency representation into multiple non-overlapping or partially overlapping image blocks, and use the first convolutional neural network to perform linear projection on each image block to obtain... The first eigenvector of dimension; taking the time-frequency representation as a time series, each time-frequency representation is divided into multiple non-overlapping or partially overlapping time series blocks, and a second convolutional neural network is used to perform linear projection on each time series block to obtain The second eigenvector of dimension;

[0078] In this embodiment, the time-frequency representation is used for feature extraction. The time-frequency representation is used as the image and time series for feature extraction, respectively. Then, two weighted networks are used to adaptively assign weights to the image features and time series features for feature fusion, which is used for fault diagnosis. Specifically, it includes feature embedding, class token, position encoding, attention encoding, selective feature extraction, and gating mechanism.

[0079] Step S3 is the feature embedding step. For the image input, each time-frequency representation is segmented into several non-overlapping or partially overlapping image patches. Each image patch is linearly projected through a convolutional neural network and embedded into a feature embedding module. For time-series inputs, a convolutional neural network is also used to embed the features of each timestamp along the time dimension into a 3D feature vector, transforming it into a 3D feature vector. 3D eigenvectors.

[0080] Step S4: Input the first classification label and the first feature vector into the image encoder to extract image features, input the second classification label and the second feature vector into the time series encoder to extract time series features, and calculate the first attention score between the local features of the second feature vector;

[0081] The step of inputting the first classification label and the first feature vector into an image encoder to extract image features, and inputting the first classification label and the first feature vector into a time series encoder to extract time series features, includes:

[0082] Step S4.1: Add the position code to the first feature vector element by element to obtain the first additive feature, and add the position code to the second feature vector element by element to obtain the second additive feature. The position code represents the position information of the local features of the image block or time series block in the global context.

[0083] In this embodiment, since the self-attention mechanism itself lacks the ability to perceive sequence order, for each local feature in the image and time series, positional encoding is used to introduce the local feature's positional information in the global sequence. The positional encoding is a randomly generated vector that can be learned during model training. By adding it element-wise with the embedded features (i.e., the first feature vector or the second feature vector), the absolute or relative positional information of each local feature in the global sequence is injected into the model. The positional encoding step is described in... Figure 2 Not shown in the image.

[0084] Step S4.2: Input the first classification label and the first additive feature into the image encoder for multiple encodings to extract features and obtain image features;

[0085] In this embodiment, the first classification label and the second classification label are randomly generated Class Tokens and are d-dimensional features that can be self-learned during the training process of the image encoder or time series encoder.

[0086] The Class Token is randomly generated and can be learned during model training. The ClassToken is concatenated with the first additive feature and input into the subsequent feature encoder. During model training, the ClassToken aggregates the global context information of the entire input sequence through a self-attention mechanism, and finally serves as the global feature representation of the input sample for subsequent fault diagnosis.

[0087] Step S4.3: Input the second classification label and the second additive feature into the time series encoder for multiple encodings to extract features, and obtain time series features. The time series encoder also calculates the first attention score between each local feature in the second additive feature.

[0088] In this embodiment, it serves as the time-frequency representation of image data after feature embedding. Includes Local features and Class Tokens, and time-frequency representation after feature embedding as time-series data. Includes Each local feature is represented by a Class Token. The Vision Transformer (ViT) is used as the encoder, specifically ImageViT and TimeViT, to extract features from image data and time-series data, respectively. ViT calculates the first attention score between each local feature in the second summed feature set, encoding the input features. Taking time-series data as an example... The process of calculating the attention score between each local feature in the image is as follows:

[0089] ;

[0090] in, The first attention score matrix, d represents the second additive feature, d is the dimension, and softmax is the softmax function.

[0091] Based on the first attention score matrix Recoded as:

[0092] ;

[0093] in, This is the second additive feature after recoding.

[0094] Step S4.4: In each encoding operation of the image encoder, the second attention score is used as the feature selection probability to dynamically select some features of the image features or time series features to participate in the subsequent linear layer transformation of the image encoder.

[0095] In this embodiment, due to the limited number of training samples, to reduce the risk of model overfitting, this invention designs a random probability self-attention mechanism to selectively select some features for subsequent feature extraction. After each feature re-encoding based on the attention score, the attention score is used as the feature selection probability, and local features... The probability of being selected is denoted as the second attention score, which is calculated as follows:

[0096] ;

[0097] in, The second attention score, i.e. and All The maximum attention score minus the average attention score. From or Randomly selected local features for The number of features, where d is the number of dimensions. for or Local features in for Local features in.

[0098] In each forward propagation, some features are dynamically selected to participate in subsequent linear layer transformations, while unselected features are replaced with the mean vector of the current features. This probabilistic selection method ensures that the feature combinations are different in each feature extraction process, increasing... The diversity of characteristics combines to form new Then, two linear layers are used to encode the... Further feature extraction is performed, and after multiple encodings and linear layers, the encoded Class Token is used as the output of ImageViT and TimeViT.

[0099] Step S5: Employ a gating mechanism to fuse image features with time-series features through a weighted network;

[0100] The method employs a gating mechanism to fuse image features with time-series features through a weighted network, including:

[0101] Two weighted networks consisting of linear layers are used to assign fusion weights to image features and time series features respectively. The two weight values ​​are normalized by Softmax and then fused by weighted summation.

[0102] In this embodiment, in order to fuse the output image features and time series features, two weighted networks composed of linear layers are used to assign fusion weights to the image features and time series features respectively. These two weight values ​​are normalized by Softmax and represent the importance of the two perspective features to the current diagnostic task. The image features and time series features are fused by weighted summation.

[0103] Step S6: Input the fused features into the classification network to obtain the classification result of the fault diagnosis;

[0104] In this embodiment, for fused features, a classification network consisting of two linear layers is used for classification to obtain the classification result of fault diagnosis, namely, faulty and no fault. In model training, the cross-entropy loss between the model output and the true label is minimized, and the entire model is optimized through the backpropagation algorithm.

[0105] Step S7: Based on the magnitude of the first attention score, construct enhanced samples and input the enhanced samples into step S2 as a new time-frequency representation;

[0106] The step of constructing enhanced samples based on the magnitude of the first attention score includes:

[0107] Step S7.1: Arrange the first attention scores in descending order, and take the first one with the highest first attention score. The corresponding second summation features constitute the initial set of important timestamps. ;

[0108] Step S7.2: From the initial set of important timestamps Random selection Each timestamp serves as a significant timestamp, forming a set of significant timestamps. ;

[0109] Step S7.3: In the time-frequency representation, for those not selected to form the initial important timestamp set... The time node is randomly selected from a set of important timestamps. The elements in the sample are replaced to obtain the enhanced sample.

[0110] In this embodiment, the first attention score matrix in time series feature extraction is used to enhance the generation of samples, and the attention scores between different local features and Class Token are used as the selection criteria for time series, specifically including the construction of the initial important timestamp set and the construction of the important timestamp set.

[0111] (1) Construction of the initial set of important timestamps. For the input time series After one training iteration, a Class Token and each input timestamp can be obtained. ( Attention weights between ) Based on the attention score, select the top These timestamps serve as the initial important timestamps, forming the initial important timestamp set. ,in, The remaining timestamps are considered unimportant timestamps.

[0112] (2) Construction of the important timestamp set. Random selection Each timestamp serves as a significant timestamp, forming a set of significant timestamps. ,in, The remaining timestamps are treated as unimportant timestamps, and the collection is constructed through a two-step selection process. , The timestamps in the middle are more diverse, from Important timestamps are randomly selected and replaced with unimportant timestamps to generate new augmented samples, increasing sample diversity. These augmented samples are then used for enhanced training of the model.

[0113] Step S8: Using multiple publicly available bearing datasets from other fields as the source domain and the original vibration signal of the bearing collected in step S1 as the target domain, further train the image encoder, time series encoder, and gating mechanism based on the source domain and the target domain. Assign a discriminator and a classifier to each of the multiple source domains. The discriminator is used to distinguish whether the feature comes from the source domain or the target domain, and the classifier is used to identify the fault type. Apply the further trained image encoder, time series encoder, and gating mechanism to steps S4 and S5.

[0114] The further training of the image encoder, time series encoder, and gating mechanism based on the source and target domains includes:

[0115] Step S8.1: In the forward propagation, each source domain is paired with a target domain to form a training pair. The image encoder, time series encoder, and gating mechanism extract features to obtain source domain features and target domain features. The source domain features and target domain features are respectively input into the discriminator for binary classification to obtain the classification result of whether the feature comes from the source domain or the target domain. The source domain features and target domain features are respectively input into the classifier for multi-class classification to obtain the fault type.

[0116] In this embodiment, domain-adaptive transfer learning uses multiple publicly available bearing datasets (source domain) from other domains and existing datasets (target domain) to further train the model backbone (ImageViT, TimeViT, and gating mechanisms). For multiple source domains ( , … Each is assigned a discriminator. and a classifier The discriminator is used to distinguish whether features come from the source domain or the target domain, while the classifier is used to identify the fault type, specifically including forward propagation and model optimization.

[0117] Among these, forward propagation occurs. During training, each source domain is linked to the target domain. Pairing them up forms a training pair, which is then input into the model for feature extraction. The source domain features and target domain features are respectively input into the discriminator for binary classification and the classifier for multi-class classification.

[0118] Step S8.2: Perform parameter optimization during backpropagation. Update the parameters of the discriminator's classifier network and the feature extraction network of the classifier's fault diagnosis model based on the classification loss of the discriminator and classifier. During training, the optimization objectives include minimizing the classification error of the source domain and minimizing the domain discrimination error between the two domains.

[0119] In this embodiment, model parameters are optimized. The parameters of the feature extraction network, discriminator, and classifier networks of the fault diagnosis model are updated based on the classification losses of the discriminator and classifier. During training, the optimization objectives are to minimize the classification error in the source domain and to minimize the domain discrimination error between the two domains. Minimizing the classification error in the source domain improves the model's ability to learn features from supervised samples in the source domain, while minimizing the domain discrimination error between the two domains reduces the distribution difference between features in the source and target domains.

[0120] This embodiment proposes a bearing fault diagnosis method based on sample augmentation and domain adaptation, including:

[0121] (1) Signal time-frequency conversion. The one-dimensional bearing vibration signal is converted into a two-dimensional time-frequency representation using the continuous wavelet transform of the complex Morlet wavelet, as shown below. Figure 3 As shown, different types of fault signals have different time-frequency representations. The time-frequency representation realizes the dual feature expression of the signal in the time domain and frequency domain. The data after time-frequency conversion retains the complete information of the original signal, while enhancing the distinguishability of fault features. It provides a rich information foundation for subsequent feature extraction, overcomes the limited feature extraction of traditional one-dimensional vibration signals, and provides more discriminative input features for deep learning models.

[0122] (2) The time-frequency representation was used as the image modality and the time series modality for feature extraction, respectively. The time-frequency representation was used as the input of the image modality and the time series modality into the improved Vision Transformer (ViT) architecture for feature extraction. The image modality retained spatial structure information through image block segmentation and convolutional embedding, while the time series modality captured temporal dependencies through temporal convolutional embedding. This fully explored the complementary information of different modalities, improved the comprehensiveness and complementarity of fault features, and enhanced the richness of feature expression.

[0123] (3) Random Probabilistic Self-Attention Mechanism. To address the overfitting problem that is prone to occur in scenarios with limited samples, a feature selection method based on attention scores is designed. In each forward propagation process, some features are dynamically selected to participate in the forward propagation. By calculating the attention scores between features, the most representative feature subset is selected in a probabilistic manner. This mechanism effectively enhances the diversity of feature combinations, prevents model overfitting, and improves the discriminative ability of features.

[0124] (4) Gating mechanism. A weighted network composed of linear layers is designed to adaptively assign fusion weights to image features and time series features, enabling the model to autonomously adjust the importance of different features according to the specific diagnostic task, realizing the effective integration of multimodal features and giving full play to the advantages of image features and time series features.

[0125] (5) Enhanced sample generation based on attention score. An important timestamp set is constructed by using the attention score between the Class Token and the timestamp. A two-step selection mechanism (initial important timestamp set and important timestamp set) ensures the diversity of the selected timestamps. An enhanced sample is generated by using a timestamp replacement strategy, which significantly improves the diversity of training samples.

[0126] (6) Domain Adaptive Transfer Learning. A multi-source domain and target domain pairing training mechanism was constructed. Through adversarial training of multiple domain discriminators and classifiers, cross-domain feature distribution alignment was achieved. This framework includes dual optimization objectives of source domain classification loss and domain discrimination loss, which can not only ensure the effective transfer of source domain knowledge, but also reduce the distribution differences between domains and improve the diagnostic performance and generalization ability of the model in scenarios with scarce samples.

[0127] (7) End-to-end fault diagnosis model. A complete deep learning framework is constructed to realize end-to-end processing from the input of the original vibration signal to the output of the fault classification result. The model includes the entire process of feature extraction, feature fusion and classification decision-making, without the need for manual intervention and expert knowledge. The model is optimized by using the cross-entropy loss function and the backpropagation algorithm to ensure continuous improvement of diagnostic performance.

[0128] Example 2:

[0129] This embodiment provides a bearing fault diagnosis system based on sample augmentation and domain adaptation, such as... Figure 4As shown, it includes: a signal acquisition module, a wavelet transform module, a feature representation module, a feature extraction module, a gating mechanism module, a fault diagnosis module, an enhanced sample construction module, and a domain adaptation module. The signal acquisition module, wavelet transform module, feature representation module, feature extraction module, gating mechanism module, fault diagnosis module, enhanced sample construction module, and domain adaptation module are connected sequentially. The enhanced sample construction module is connected to the wavelet transform module, and the domain adaptation module is connected to the signal acquisition module, feature extraction module, and gating mechanism module respectively.

[0130] The signal acquisition module is used to acquire the raw vibration signal of the bearing;

[0131] The wavelet transform module is used to convert the original vibration signal into a time-frequency representation by using a continuous wavelet transform with the wavelet function being a complex Morlet wavelet. Among them, fault signals of the same type have different time-frequency representations.

[0132] The feature representation module is used to treat the time-frequency representation as an image, segment each time-frequency representation into multiple non-overlapping or partially overlapping image blocks, and use a first convolutional neural network to perform linear projection on each image block to obtain... The first eigenvector of dimension; taking the time-frequency representation as a time series, each time-frequency representation is divided into multiple non-overlapping or partially overlapping time series blocks, and a second convolutional neural network is used to perform linear projection on each time series block to obtain The second eigenvector of dimension;

[0133] The feature extraction module is used to input the first classification label and the first feature vector into the image encoder to extract image features, input the second classification label and the second feature vector into the time series encoder to extract time series features, and calculate the first attention score between the local features of the second feature vector.

[0134] The gating mechanism module is used to fuse image features and time series features through a weighted network using a gating mechanism.

[0135] The fault diagnosis module is used to input the fused features into the classification network to obtain the classification result of the fault diagnosis.

[0136] The enhanced sample construction module is used to construct enhanced samples based on the magnitude of the first attention score. The enhanced samples are then input into the wavelet transform module as a new time-frequency representation.

[0137] The domain adaptation module is used to take multiple publicly available bearing datasets from other fields as the source domain and the original vibration signals of the bearings acquired by the signal acquisition module as the target domain. Based on the source domain and the target domain, the image encoder, time series encoder, and gating mechanism are further trained. For each of the multiple source domains, a discriminator and a classifier are assigned. The discriminator is used to distinguish whether the features come from the source domain or the target domain, and the classifier is used to identify the fault type. The further trained image encoder, time series encoder, and gating mechanism are applied to the feature extraction module and the gating mechanism module.

[0138] Example 3:

[0139] This embodiment proposes an electronic device, including: one or more processors, and a memory for storing instructions, which, when executed by the one or more processors, cause the one or more processors to perform the aforementioned bearing fault diagnosis method based on sample augmentation and domain adaptation.

[0140] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements a bearing fault diagnosis method based on sample enhancement and domain adaptation as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0141] The processor is used to execute all or part of the steps in the sample-enhanced and domain-adaptive bearing fault diagnosis method described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0142] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the bearing fault diagnosis method based on sample enhancement and domain adaptation described in the above embodiments.

[0143] Example 4:

[0144] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0145] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the bearing fault diagnosis method based on sample enhancement and domain adaptation described in the various embodiments of this application.

[0146] The aforementioned storage media include: flash memory, hard disks, multimedia cards, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disks, optical discs, servers, APP (Application) application stores, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the aforementioned bearing fault diagnosis method based on sample enhancement and domain adaptation.

[0147] Example 5:

[0148] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned bearing fault diagnosis method based on sample enhancement and domain adaptation.

[0149] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0150] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0151] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of equivalent technology of this disclosure, then the intent of this disclosure also includes such modifications and variations.

Claims

1. A bearing fault diagnosis method based on sample enhancement and domain adaptation, characterized in that, The application relates to a bearing fault diagnosis method based on image and time sequence fusion, and belongs to the field of mechanical fault diagnosis. Step S1: collecting original vibration signals of a bearing; Step S2: converting the original vibration signals into time-frequency representations by using a continuous wavelet transform with a complex Morlet wavelet as a wavelet function, wherein different time-frequency representations are obtained for signals of the same type of fault; Step S3: taking the time-frequency representation as an image, dividing each time-frequency representation into a plurality of non-overlapping or partially overlapping image blocks, performing linear projection on each image block by using a first convolutional neural network to obtain a first feature vector of the first dimension; taking the time-frequency representation as a time series, dividing each time-frequency representation into a plurality of non-overlapping or partially overlapping time series blocks, performing linear projection on each time series block by using a second convolutional neural network to obtain a second feature vector of the second dimension; Step S4: inputting the first classification mark and the first feature vector into an image encoder to extract image features, inputting the second classification mark and the second feature vector into a time sequence encoder to extract time sequence features, and calculating first attention scores between local features of the second feature vector; Step S5: performing feature fusion of the image features and the time sequence features by using a gating mechanism and a weight network; Step S6: inputting the fused features into a classification network to obtain a classification result of fault diagnosis; Step S7: constructing an enhanced sample according to the size of the first attention scores, and inputting the enhanced sample into step S2 as new time-frequency representations; Step S8: taking multiple bearing data sets disclosed in other fields as source domains, taking original vibration signals of the bearing collected in step S1 as a target domain, further training the image encoder, the time sequence encoder and the gating mechanism based on the source domains and the target domain, respectively assigning a discriminator and a classifier to each of the multiple source domains, wherein the discriminator is used for distinguishing whether the features come from the source domains or the target domain, and the classifier is used for identifying a fault type, and applying the image encoder, the time sequence encoder and the gating mechanism after the further training to steps S4 and S5.

2. The bearing fault diagnosis method based on sample enhancement and domain adaptation according to claim 1, characterized in that, The first classification mark and the second classification mark are randomly generated Class Tokens and d-dimensional features that can be self-learned during the training of the image encoder or the time sequence encoder.

3. The bearing fault diagnosis method based on sample enhancement and domain adaptation according to claim 1, characterized in that, The inputting of the first classification mark and the first feature vector into the image encoder to extract image features and the inputting of the first classification mark and the first feature vector into the time sequence encoder to extract time sequence features comprise the following steps. Step S4.1: adding a position encoding and the first feature vector element by element to obtain first added features, and adding the position encoding and the second feature vector element by element to obtain second added features, wherein the position encoding represents position information of local features of an image block or a time sequence block in a global; Step S4.2: inputting the first classification mark and the first added features into the image encoder for multiple times of encoding to extract features and obtain image features; Step S4.3: inputting the second classification mark and the second added features into the time sequence encoder for multiple times of encoding to extract features and obtain time sequence features, and the time sequence encoder calculates first attention scores between each of the local features in the second added features; Step S4.4: in each encoding of the image encoder, a second attention score is used as a feature selection probability to dynamically select part of the image features or the time sequence features to participate in linear layer transformation of a subsequent image encoder.

4. The bearing fault diagnosis method based on sample enhancement and domain adaptation according to claim 3, characterized in that, The time sequence encoder calculates the first attention scores between each of the local features in the second added features according to the following formula: ； wherein, is a first attention score matrix, is a second added feature, d is a dimension number, and softmax is a softmax function.

5. The bearing fault diagnosis method based on sample enhancement and domain adaptation according to claim 1, characterized in that, The second attention score is calculated according to the following formula: ； wherein, is a second attention score, i.e. is a first attention score, is the maximum attention score among all the average value of the attention scores, is a partial local feature randomly selected from or is the number of features in d is the dimension number, is a local feature in or is a local feature in ​​​ 6. The bearing fault diagnosis method based on sample enhancement and domain adaptation according to claim 1, characterized in that, The feature fusion of the image features and the time sequence features by using the gating mechanism and the weight network comprises the following steps. Two weight networks composed of linear layers are used to assign fusion weights to image features and time series features respectively, the two weight values are normalized by Softmax, and the image features and time series features are fused by weighted summation.

7. The bearing fault diagnosis method based on sample enhancement and domain adaptation according to claim 1, characterized in that, The enhanced sample is constructed according to the size of the first attention score, including: arranging the first attention scores in descending order, taking the first attention scores of the top corresponding second addition features, to constitute an initial important time stamp set ​ selecting, from an initial set of important time stamps randomly one time stamp as an important time stamp, constituting a set of important time stamps ; In the time-frequency representation, for the time nodes that are not selected to constitute the initial important timestamp set , elements in the important timestamp set are randomly adopted to replace them to obtain enhanced samples.

8. The bearing fault diagnosis method based on sample enhancement and domain adaptation according to claim 1, characterized in that, The image encoder, the time series encoder and the gating mechanism are further trained based on the source domain and the target domain, including: In the forward propagation, each source domain is paired with the target domain to form a training pair, the image encoder, the time series encoder and the gating mechanism perform feature extraction to obtain source domain features and target domain features, the source domain features and the target domain features are input into the discriminator for binary classification to obtain the classification result of whether the features come from the source domain or the target domain, and the source domain features and the target domain features are input into the classifier for multi-classification to obtain the fault type; In the backward propagation, the parameters are optimized, and the feature extraction network of the discriminator network of the discriminator and the fault diagnosis model of the classifier are updated according to the classification loss of the discriminator and the classifier, and in the training process, the optimization target includes: minimizing the classification error of the source domain and minimizing the domain discrimination error between the two domains.

9. A bearing fault diagnosis system based on sample augmentation and domain adaptation, characterized in that, Including: The signal acquisition module is configured to acquire the original vibration signal of the bearing. The wavelet transform module is configured to convert the original vibration signal into a time-frequency representation by using a continuous wavelet transform of a wavelet function, wherein the same type of fault signal has different time-frequency representations. The feature representation module is configured to represent each time-frequency representation as an image, divide each time-frequency representation into a plurality of non-overlapping or partially overlapping image blocks, and perform linear projection on each image block by using a first convolutional neural network to obtain a first feature vector of a first dimension. The feature representation module is configured to represent each time-frequency representation as a time series, divide each time-frequency representation into a plurality of non-overlapping or partially overlapping time series blocks, and perform linear projection on each time series block by using a second convolutional neural network to obtain a second feature vector of a second dimension. The feature representation module is configured to represent each time-frequency representation as a time series, divide each time-frequency representation into a plurality of non-overlapping or partially overlapping time series blocks, and perform linear projection on each time series block by using a second convolutional neural network to obtain a second feature vector of a second dimension. The feature extraction module is configured to input the first classification label and the first feature vector into the image encoder to extract image features, input the second classification label and the second feature vector into the time series encoder to extract time series features, and calculate the first attention score between local features of the second feature vector. The gating mechanism module is configured to use a gating mechanism to fuse the image features and the time series features through a weight network. The fault diagnosis module is configured to input the fused features into a classification network to obtain a classification result of fault diagnosis. The enhanced sample construction module is configured to construct an enhanced sample according to the size of the first attention score, and input the enhanced sample into the wavelet transform module as a new time-frequency representation. The domain adaptation module is configured to use multiple bearing data sets disclosed by other domains as source domains, use the original vibration signal of the bearing acquired by the signal acquisition module as a target domain, further train the image encoder, the time series encoder and the gating mechanism based on the source domain and the target domain, assign a discriminator and a classifier to each source domain, the discriminator is used to distinguish whether the features come from the source domain or the target domain, and the classifier is used to identify the fault type, and apply the further trained image encoder, the time series encoder and the gating mechanism to the feature extraction module and the gating mechanism module.

Images