A motor fault diagnosis method for limited label samples and cross-device scenarios

Abstract

The application discloses a motor fault diagnosis method for limited label samples and cross-device scenes, and belongs to the field of motor fault diagnosis. The method comprises the following steps: obtaining a label-free pre-training sample set and a labeled fine-tuning sample set of a motor; constructing an MMCA motor fault diagnosis model; pre-training the MMCA motor fault diagnosis model by using the label-free pre-training sample set, so as to obtain a pre-training completed motor fault diagnosis model; fine-tuning the pre-training completed motor fault diagnosis model by using the labeled fine-tuning sample set, so as to obtain a final motor fault diagnosis model; and diagnosing a to-be-tested working condition by using the final motor fault diagnosis model, so as to obtain a motor fault diagnosis result. The application solves the problems that the existing method only relies on the time-frequency dual domains of vibration signals, and it is difficult to realize motor fault diagnosis with high accuracy, and the internal coupling relationship of the motor is complex, different modal signals have significant sensitivity differences, and it is difficult to realize multi-signal synchronous collection and feature alignment.

Description

Technical Field

[0001] This invention belongs to the field of motor fault diagnosis, and in particular relates to a method for motor fault diagnosis for limited tag samples and cross-device scenarios. Background Technology

[0002] In crucial sectors such as modern industry, manufacturing, and transportation, electric motors, as core equipment for converting electrical energy into mechanical energy, undertake the critical tasks of driving, transmitting, and controlling. They are not only a vital power source for achieving mechanized, automated, and intelligent production, but also a fundamental guarantee for the stable production and development of various industries. Their stability and reliability directly affect equipment operating efficiency, production safety, and economic benefits. A malfunction can lead to production interruptions, equipment damage, and economic losses, or even serious injuries, fatalities, and major safety accidents. Therefore, conducting motor fault diagnosis to accurately identify fault types and monitor their health status in real time is of paramount engineering practical significance.

[0003] Unlike common mechanical fault diagnosis objects such as bearings and gears, motors have the following characteristics: (1) Motors have complex structures, consisting of multiple components such as stators, rotors, and bearings, with compact structures and diverse levels. (2) Motors can be regarded as electro-magnetic-mechanical multi-field coupling systems, with more complex fault modes and more diverse fault types, namely electrical faults, mechanical faults, etc. (3) It is difficult to fully reflect the characteristics of different types of faults with only a single type of signal. For example, it may be difficult to detect certain electrical faults by only using vibration signals. For these reasons, motor fault diagnosis often relies more on multi-type detection signals and multi-modal fusion methods. Existing technologies have proposed a fault diagnosis method based on the fusion of four types of signal features: voltage, current, torque, and speed, using an improved convolutional neural network, and have carried out multi-signal fault diagnosis of induction motors. Existing technologies have proposed a fault diagnosis method based on a gated convolutional attention neural network, using different types of signal inputs, and realizing feature fusion and fault diagnosis at the end of the network. Existing technologies have proposed an improved AdaBoost motor fault diagnosis method driven by attention mechanisms and multi-sensor information, which improves the diagnostic accuracy by fusing vibration, electromagnetic, and current data. Existing technologies have proposed a novel framework for motor bearing fault diagnosis from the perspective of multi-transform domain and multi-source data fusion. Within this framework, feature extraction and fusion of vibration and current signals are achieved in the time, frequency, and time-frequency domains. These methods have all achieved accurate fault diagnosis of motors under the condition of multiple signal types. However, in real-world industrial environments, it is often difficult to obtain a sufficient number of high-quality labeled data samples. Insufficient labeling often leads to a limited number of usable samples, thus affecting the accuracy of the diagnosis.

[0004] The quantity and quality of data often have a significant impact on supervised learning models. Therefore, in real-world industrial conditions, high-quality samples are extremely scarce due to difficulties in data labeling and complex operating conditions, becoming a major bottleneck limiting the widespread adoption and implementation of deep learning-based fault diagnosis methods in real-world engineering scenarios. To address this issue, existing technologies embed multi-attribute information of motor fault samples into heterogeneous graphs and propose a semi-supervised meta-pathspace extended graph neural network for fault diagnosis. Existing technologies apply transfer learning strategies to hybrid convolutional and self-attention deep learning classification network models to achieve motor fault diagnosis. Existing technologies propose an adaptive fault diagnosis method based on improved generative adversarial networks and fine-tuning, performing fault diagnosis on asynchronous motors with limited labeled samples. Existing technologies use conditional variational autoencoders and generative adversarial networks to address the long-tail distribution problem in motor fault diagnosis. Most methods address the issue of insufficient labels and data in motor fault diagnosis from the perspective of data generation, but generative methods may suffer from drawbacks such as high computational cost, limited data quality, and insufficient adaptability to varying operating conditions.

[0005] In recent years, self-supervised learning has gradually become a research hotspot in the field of intelligent fault diagnosis due to its strong feature learning ability even under conditions of insufficient sample labeling and complex operating conditions. This method constructs pre-defined auxiliary tasks, enabling the model to automatically extract discriminative deep feature representations from large amounts of unlabeled data, thereby achieving effective knowledge learning and transfer. Among these studies, discriminative self-supervised methods based on contrastive learning have recently achieved breakthroughs, including Simplified, MoCo, and BYOL. Contrastive learning self-supervised methods typically include three key steps: first, constructing positive sample pairs from unlabeled data through data augmentation and forming negative sample pairs from different instances; then, extracting features from the sample pairs using shared or independent encoders; and finally, using a contrastive loss function to narrow the feature distance between positive samples and widen the feature distance between negative samples, thereby achieving self-supervised learning of highly discriminative feature representations. Existing technology proposes a self-supervised multi-view contrastive learning network (SMCLN) based on time-frequency analysis, which efficiently extracts valuable feature information under limited labeling and significantly improves classification accuracy. Existing technology also constructs a novel self-supervised domain adversarial generalization framework, using self-supervised methods to generate pseudo-labels. Existing technologies propose a pre-training method for mechanical fault diagnosis based on cross-sensor contrastive learning. This method captures complementary features between different sensors through cross-sensor comparison to enhance the ability to distinguish fault features. Existing technologies also improve upon BYOL by introducing a fuzzy neural network that can automatically extract fault features without requiring labeled information, resulting in AFFE-BYOL. Furthermore, existing technologies propose a time-frequency prediction self-supervised learning framework, TFPred, to extract latent fault representations from unlabeled fault data. Finally, existing technologies propose a novel self-supervised representation learning framework, TFAI, based on time-frequency alignment and interaction. Through these two self-supervised learning strategies, fault diagnosis is achieved in cross-device scenarios. It is evident that compared to supervised learning, self-supervised contrastive learning not only reduces the reliance on manual annotation but also significantly improves the model's generalization ability in varying operating conditions and cross-device scenarios.

[0006] However, current research on such methods mainly focuses on the time-frequency dual-domain analysis of vibration signals, and is mostly concentrated on fault diagnosis of single components such as bearings. It has not yet been fully extended to the field of motor system fault diagnosis that relies on the collaborative analysis of multiple modal signals such as current and vibration. This may be because: (1) Relying solely on the time-frequency dual-domain analysis of vibration signals may not be able to achieve high accuracy in motor fault diagnosis. (2) The internal coupling relationship of the motor is complex, and the sensitivity differences of different modal signals are significant, making it difficult to synchronously acquire multiple signals and align features. Therefore, research on current self-supervised fault diagnosis methods for motors is scarce. Summary of the Invention

[0007] To address the aforementioned shortcomings in existing technologies, this invention provides a motor fault diagnosis method for limited tag samples and cross-device scenarios. This method solves the problems that existing methods may struggle to achieve high accuracy in motor fault diagnosis by relying solely on the time-frequency dual domain of vibration signals, as well as the complex internal coupling relationships within the motor, significant differences in sensitivity between different modal signals, and difficulties in simultaneous acquisition of multiple signals and feature alignment.

[0008] To achieve the aforementioned objectives, the present invention employs the following technical solution: a method for diagnosing motor faults in scenarios with limited tag samples and across devices, comprising: Obtain the unlabeled pre-trained sample set and the labeled fine-tuned sample set of the motor; Construct an MMCA motor fault diagnosis model; the MMCA motor fault diagnosis model includes: A frequency-domain DWMR feature encoder is used to extract features from vibration frequency-domain signals to obtain frequency-domain features; A time-domain DWMR feature encoder is used to extract features from vibration time-domain signals to obtain time-domain features; The multi-channel fusion encoder (MCFE) is used to fuse frequency domain features and time domain features to obtain fused features. Multilayer perceptron (MLP) is used to perform nonlinear mapping on fused features to obtain mapped features; Fully connected (FC) layers are used to obtain classification results based on the mapped features. The MMCA motor fault diagnosis model was pre-trained using an unlabeled pre-training sample set to obtain a pre-trained motor fault diagnosis model. The pre-trained motor fault diagnosis model was fine-tuned using a labeled fine-tuning sample set to obtain the final motor fault diagnosis model. The final motor fault diagnosis model is used to diagnose the test condition and obtain the motor fault diagnosis results.

[0009] Furthermore, the DWMR feature encoder employs an improved ResNet-18 network. Specifically, the improved ResNet-18 network sets the kernel of the first convolutional layer to 1×7; sets the kernel of the max-pooling layer connected to the first convolutional layer to 1×3; and sets the structure of each residual block of the ResNet-18 network to two sequentially connected bottleneck layers. The bottleneck layer includes a first input, three parallel branches connected to the first input, a 1×1 Conv convolution, a dynamic scale weighting unit connected to each parallel branch, a concatenation operation connected to the dynamic scale weighting unit, a first element-wise addition operation connected to the 1×1 Conv convolution and the concatenation operation, and a first output connected to the first element-wise addition operation. Each parallel branch includes a sequentially connected convolutional layer, batch normalization, ReLU activation function, feature length alignment unit, and feature gating unit. The convolutional layers of each parallel branch... The kernels are 1×3, 1×5, and 1×7 respectively; the feature gating unit includes a second input terminal, a first adaptive average pooling connected to the second input terminal, a first 1×1 Conv convolution connected to the first adaptive average pooling, a second 1×1 Conv convolution connected to the first 1×1 Conv convolution, a sigmoid activation function connected to the second 1×1 Conv convolution, an element-wise multiplication connected to the sigmoid activation function and the second input terminal respectively, and a second output terminal connected to the element-wise multiplication; the dynamic scale weighting unit includes a third input terminal, a second adaptive average pooling connected to the third input terminal, a second element-wise addition operation connected to the second adaptive average pooling, a first fully connected layer connected to the second element-wise addition operation, a softmax activation function connected to the first fully connected layer, and a weight output terminal connected to the softmax activation function and the third input terminal; the feature length alignment unit is a clipping alignment operation.

[0010] Furthermore, the multi-channel fusion encoder MCFE includes 6 stacked convolutional fusion blocks; the expression for the data processing of each convolutional fusion block is as follows:

[0011]

[0012]

[0013] in, The temporal features output by the current convolutional fusion block; For convolution residual blocks; These are the temporal features output by the convolutional fusion block from the previous layer; The frequency domain features are those of the current convolutional fusion block after interpolation and alignment. The frequency domain features output by the current convolutional fusion block; This represents the frequency domain features output by the previous layer's convolutional fusion block; This is a linear interpolation operation; To perform mean-variance normalization for each channel; For feature size; The length of the feature vector.

[0014] Furthermore, the convolutional residual block includes a fourth input terminal, a first 1×3 Conv convolution connected to the fourth input terminal, a second 1×3 Conv convolution connected to the first 1×3 Conv convolution, a third element-wise addition operation connected to the fourth input terminal and the second 1×3 Conv convolution respectively, and a third output terminal connected to the third element-wise addition operation.

[0015] Furthermore, during pre-training, a current signal branch is introduced into the MMCA motor fault diagnosis model, and vibration frequency domain branch and vibration time domain branch are constructed based on the frequency domain DWMR feature encoder and the time domain DWMR feature encoder, respectively. The current signal branch is used to extract features from the three-phase current signal using a fusion module, a DWMR feature encoder, a first projection head, and a first prediction head connected in sequence to obtain current features; the fusion module is a 1×1 convolution, used to integrate the current signals of each phase based on the three-phase current signal. The vibration time-domain branch is used to extract features from the vibration time-domain signal using a sequentially connected time-domain DWMR feature encoder, a second projection head, and a second prediction head to obtain vibration time-domain features. The vibration frequency domain branch is used to extract features from the vibration frequency domain signal using a DWMR feature encoder, a third projection head, and a third prediction head connected in sequence, to obtain vibration frequency domain features.

[0016] Furthermore, pre-training incorporates a multimodal contrastive learning strategy (MMCL) and a multimodal fusion alignment strategy (MMFA), specifically: A momentum model is constructed; the momentum model has a three-branch structure; each branch corresponds one-to-one with the branch for removing the current signal from the first predictor head, the branch for removing the vibration time domain from the second predictor head, and the branch for removing the vibration frequency domain from the third predictor head; the parameters of the momentum model do not participate in backpropagation and gradient update, but are iteratively optimized through a momentum update strategy:

[0017] in, For the first The parameters of the momentum model after the next iteration; The momentum coefficient; For the first The parameters of the momentum model after -1 iteration; For the first The parameters of the MMCA motor fault diagnosis model during pre-training after the next iteration; Multimodal contrastive learning strategy (MMCL): The query vector is the feature output of the DWMR feature encoder corresponding to mode m of each pre-trained sample in the unlabeled pre-training sample set, and the key vector is the momentum representation output of the branch corresponding to mode n processed by the momentum model of each pre-trained sample. Based on the query vector and key vector, the similarity score of each pre-trained sample in mode m is calculated. The multimodal contrastive loss is calculated based on the similarity scores of each pre-trained sample in each mode. Mode m and mode n are both one of current, vibration time domain, and vibration frequency domain; mode m and mode n are not the same; the expression for the multimodal contrastive loss is:

[0018]

[0019]

[0020] in, For multimodal contrast loss; For modality The loss; Modal identifier; For vibration time-domain modes; These are vibration frequency domain modes; It is a current mode; This refers to the batch size of the pre-trained samples; For the first mode m, Similarity scores between positive sample pairs; For mode m, and Similarity score between negative sample pairs; This is a temperature coefficient used to adjust the steepness of the probability distribution; and All are pre-trained sample indexes; A similarity score; For modality The corresponding features output by the DWMR feature encoder; For modality Corresponding momentum characterization; Modal identifier; For transpose; Multimodal Fusion Alignment Strategy (MMFA): Based on the features and momentum representations output by the DWMR feature encoders corresponding to each mode, the attention weight matrices for vibration time-domain modes and vibration frequency-domain modes, as well as the attention weight matrices for vibration time-domain modes and current modes, are calculated. Based on these attention weight matrices, a cross-modal alignment set is constructed. , , and ;Will and Input the first diagnostic branch to obtain the cross-modal classification loss in the time and frequency domains; and Input the second diagnostic branch to obtain the cross-modal classification loss for time and current; add the cross-modal classification loss for time and frequency domains and the cross-modal classification loss for time and current to obtain the alignment loss; the expressions for the attention weight matrix of vibration time domain mode and vibration frequency domain mode, the attention weight matrix of vibration time domain mode and current mode, and the cross-modal alignment set are respectively:

[0021]

[0022] in, The attention weight matrix is ​​obtained by using the vibration time-domain features as the query vector and the vibration frequency-domain momentum representation as the key vector. For activation functions; The vibration time-domain characteristics; This represents the momentum in the frequency domain of vibration. The attention weight matrix is ​​obtained by using the vibration frequency domain features as the query vector and the vibration time domain momentum representation as the key vector. It represents the frequency domain characteristics of vibration; This represents the momentum in the time domain of vibration. The attention weight matrix is ​​obtained by using the vibration time-domain features as the query vector and the momentum representation of the current as the key vector. This is a characterization of the momentum of electric current. The attention weight matrix is ​​obtained by using current characteristics as the query vector and the momentum representation in the vibration time domain as the key vector. Current characteristics; It is a time-frequency time-domain triple; For positive samples in the time domain, specifically A list consisting of the data on the main diagonal; From The time-domain negative samples sampled in the middle are specifically: A list consisting of the minimum values ​​in each row; For time-frequency domain triples; For positive samples in the frequency domain, specifically A list consisting of the data on the main diagonal; From The time-domain negative samples sampled in the middle are specifically: A list consisting of the minimum values ​​in each row; For vibration current vibration triplet; From The sampled positive samples in the time domain are specifically: A list consisting of the data on the main diagonal; From The sampled time-domain negative samples are specifically: A list consisting of the minimum values ​​in each row; It is a ternary current oscillation current; For positive current samples, specifically A list consisting of the data on the main diagonal; From The negative samples of current in the middle are specifically: A list consisting of the smallest data in each row.

[0023] Furthermore, both the first diagnostic branch and the second diagnostic branch include a first multi-channel fusion encoder (MCFE), a first multilayer perceptron (MLP), and a second fully connected layer (FC) connected in sequence; the first multi-channel fusion encoder (MCFE) of the first diagnostic branch and the second diagnostic branch is used to perform feature fusion on the cross-modal alignment set of the two input modalities.

[0024] Furthermore, the pre-trained loss function is a weighted sum of the multimodal contrast loss and the alignment loss.

[0025] Furthermore, during fine-tuning, the DWMR feature encoders of the time-domain branch and the frequency-domain branch in the pre-trained motor fault diagnosis model are used to replace the corresponding DWMR feature encoders of the MMCA motor fault diagnosis model. The multi-channel fusion encoder MCFE, MLP multilayer perceptron, and FC fully connected layer in the MMCA motor fault diagnosis model are replaced with the first diagnostic branch to obtain the replaced MMCA motor fault diagnosis model. The replaced MMCA motor fault diagnosis model is then trained using a labeled fine-tuning sample set to obtain the final motor fault diagnosis model.

[0026] Furthermore, when training the replaced MMCA motor fault diagnosis model, a two-way symmetric KL divergence is introduced to constrain the vibration time-domain and vibration frequency-domain modes; the loss function of the final motor fault diagnosis model is:

[0027]

[0028] in, The loss function for the final motor fault diagnosis model; for The weights; For KL divergence loss; The classification loss is based on supervised labels; For distribution For approximation The relative entropy loss at that time; The probability distribution of the output of the time-domain branch; The probability distribution of the frequency domain branch output; For distribution For approximation The relative entropy loss at that time; Indexed by category; For distribution The Middle The probability values ​​of each category; For distribution The Middle The probability values ​​of each category.

[0029] The beneficial effects of this invention are as follows: This invention proposes a self-supervised motor fault diagnosis framework (MMCA) based on multimodal contrast and feature alignment to achieve accurate identification and reliable diagnosis of motor fault states under complex conditions such as scarce labeled samples, significant changes in operating conditions, and even cross-device migration. This framework is mainly based on a dynamic weighted multiscale residual encoder and a multimodal contrast learning and fusion alignment mechanism. It performs multimodal contrast learning using the vibration time-domain and frequency-domain data and current data of the tested equipment, and combines this with a multimodal feature fusion and alignment mechanism to improve the consistency of representations between different modes and fully mine complementary information, thereby obtaining fault feature representations with greater discriminative and generalizable capabilities. Since the data received by MMCA includes multiple modes with varying characteristics, a dynamic weighted multiscale residual encoder (DWMR) suitable for various modal data is proposed. This encoder includes a gating mechanism and three different scale convolutional kernels to automatically and selectively enhance effective information for different categories of data and adaptively capture feature representations of signals at different scales such as short-term, local, and long-term dependencies, achieving efficient and robust feature extraction. An improved multimodal contrastive learning strategy, MMCL, applicable to two or more data types, was proposed. Combined with a three-branch structure, it achieves efficient contrastive learning between vibration time-domain, vibration frequency-domain, and current signals, effectively capturing the correlation features between different modes and improving the discriminative power and robustness of multimodal feature representations. Furthermore, through the multimodal fusion and alignment strategy MMFA, deep semantic alignment and feature fusion between vibration time-domain, vibration frequency-domain, and current modes are further realized. Attached Figure Description

[0030] Figure 1 This is a flowchart of the method of the present invention.

[0031] Figure 2 This is a structural diagram of the Dynamic Weighted Multi-Scale Residual Encoder (DWMR) of the present invention.

[0032] Figure 3 This is a structural diagram of the multi-channel fusion encoder MCFE of the present invention.

[0033] Figure 4 This is a structural diagram of the MMCA motor fault diagnosis model in the pre-training stage of this invention.

[0034] Figure 5 This is a structural diagram of the multimodal fusion alignment strategy MMFA of the present invention.

[0035] Figure 6 This is a structural diagram of the MMCA motor fault diagnosis model in the fine-tuning stage of this invention.

[0036] Figure 7 This is an overall framework diagram of the MMCA of the present invention.

[0037] Figure 8 This is a schematic diagram showing the comparative experimental results of various methods under different working conditions in Example 1 of this invention.

[0038] Figure 9 The diagram shows the confusion matrix and t-SNE clustering analysis of each method in the embodiments of the present invention.

[0039] Figure 10 This is a schematic diagram showing the comparative experimental results of various methods under different working conditions in Example 2 of this invention.

[0040] Figure 11 This is the confusion matrix and t-SNE diagram of different methods in Example 2 of this invention with a 5% labeled sample size.

[0041] Figure 12 This is the confusion matrix and t-SNE diagram of different methods in Example 3 of this invention with a 5% labeled sample size.

[0042] Figure 13 This is a schematic diagram showing the results under different parameter conditions in the embodiments of the present invention. Detailed Implementation

[0043] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0044] like Figure 1 As shown, in one embodiment of the present invention, a method for diagnosing motor faults in a limited number of tagged samples and across device scenarios includes: Obtain the unlabeled pre-trained sample set and the labeled fine-tuned sample set of the motor; Construct an MMCA motor fault diagnosis model; the MMCA motor fault diagnosis model includes: A frequency-domain DWMR feature encoder is used to extract features from vibration frequency-domain signals to obtain frequency-domain features; A time-domain DWMR feature encoder is used to extract features from vibration time-domain signals to obtain time-domain features; The multi-channel fusion encoder (MCFE) is used to fuse frequency domain features and time domain features to obtain fused features. Multilayer perceptron (MLP) is used to perform nonlinear mapping on fused features to obtain mapped features; Fully connected (FC) layers are used to obtain classification results based on the mapped features. The MMCA motor fault diagnosis model was pre-trained using an unlabeled pre-training sample set to obtain a pre-trained motor fault diagnosis model. The pre-trained motor fault diagnosis model was fine-tuned using a labeled fine-tuning sample set to obtain the final motor fault diagnosis model. The final motor fault diagnosis model is used to diagnose the test condition and obtain the motor fault diagnosis results.

[0045] The DWMR feature encoder uses an improved ResNet-18 network. Specifically, the improved ResNet-18 network has a 1×7 kernel in the first convolutional layer and a 1×3 kernel in the max-pooling layer connected to the first convolutional layer. Each residual block in the ResNet-18 network is structured as a two-layer bottleneck layer. The bottleneck layer includes a first input, three parallel branches connected to the first input, a 1×1 Conv convolution, a dynamic scale weighting unit connected to each parallel branch, a concatenation operation connected to the dynamic scale weighting unit, a first element-wise addition operation connected to the 1×1 Conv convolution and the concatenation operation, and a first output connected to the first element-wise addition operation. Each parallel branch includes a convolutional layer, batch normalization, ReLU activation function, feature length alignment unit, and feature gating unit connected in sequence. The convolutional kernels of the convolutional layers in each parallel branch are... The values ​​are 1×3, 1×5, and 1×7, respectively. The feature gating unit includes a second input terminal, a first adaptive average pooling connected to the second input terminal, a first 1×1 Conv convolution connected to the first adaptive average pooling, a second 1×1 Conv convolution connected to the first 1×1 Conv convolution, a sigmoid activation function connected to the second 1×1 Conv convolution, an element-wise multiplication connected to the sigmoid activation function and the second input terminal, and a second output terminal connected to the element-wise multiplication. The dynamic scale weighting unit includes a third input terminal, a second adaptive average pooling connected to the third input terminal, a second element-wise addition operation connected to the second adaptive average pooling, a first fully connected layer connected to the second element-wise addition operation, a softmax activation function connected to the first fully connected layer, and a weight output terminal connected to the softmax activation function and the third input terminal. The feature length alignment unit is a clipping alignment operation.

[0046] In this embodiment, since the input data during pre-training includes three types of signals: vibration time-domain signals, vibration frequency-domain signals, and current signals, a Dynamic Weighted Multi-Scale Residual Encoder (DWMR) is designed to achieve more efficient and robust feature extraction under different signal modes. The DWMR is an improvement upon ResNet-18, as follows: Figure 2(a) Introducing a multi-scale dynamic feature fusion mechanism into its bottleneck layer, such as the bottleneck layer... Figure 2 (b)

[0047] Specifically, the bottleneck layer constructs three parallel branches, employing one-dimensional convolutions with kernel sizes of 1×3, 1×5, and 1×7 respectively, to capture feature representations of the signal at different scales, including short-term, local, and long-term dependencies. Each branch undergoes convolution, batch normalization, ReLU activation, and further incorporates a feature gating module, such as... Figure 2 (c) Fine-grained weighting of feature channels is achieved through adaptive signal intensity modulation, enabling the encoder to automatically and selectively enhance effective information for different inputs. Subsequently, weights are dynamically allocated at the branch level using a dynamic scale weighting unit. The result of the dynamic scale weighting unit is as follows: Figure 2 As shown in (d), global average pooling is mainly used to extract global features for each branch. The importance score of each branch is calculated through nonlinear mapping using a fully connected network consisting of two linear layers, thus achieving adaptive enhancement of important branches. The features of each branch are concatenated along the channel dimension after passing through gating and dynamic scale weighting units. A 1×1 convolutional layer is then used for channel integration and information compression to unify the output dimension and improve feature interactivity. Finally, a residual connection strategy is used to preserve input path information. That is, when the input and output dimensions do not match or the stride changes, shortcut projection is used to align the channels and lengths, ensuring that the main branch and residual signal can be effectively superimposed, thereby stabilizing gradient propagation and improving the continuity of feature representation.

[0048] In the main network, a hierarchical structure similar to ResNet-18 is adopted, gradually increasing the number of channels (64, 128, 256, and 512) and decreasing the feature resolution through four layers. Each layer consists of two bottleneck layers, each with the structure described above, used for feature extraction and nonlinear enhancement. Simultaneously, shortcut projection is implemented through 1×1 convolutions to maintain consistency between input and output dimensions when the dimensionality changes, thereby achieving stable multi-scale residual learning and deep feature fusion.

[0049] The multi-channel fusion encoder (MCFE) comprises six stacked convolutional fusion blocks; the expression for the data processing procedure of each convolutional fusion block is as follows:

[0050]

[0051]

[0052] in, The temporal features output by the current convolutional fusion block; For convolution residual blocks; These are the temporal features output by the convolutional fusion block from the previous layer; The frequency domain features are those of the current convolutional fusion block after interpolation and alignment. The frequency domain features output by the current convolutional fusion block; This represents the frequency domain features output by the previous layer's convolutional fusion block; This is a linear interpolation operation; To perform mean-variance normalization for each channel; For feature size; The length of the feature vector.

[0053] The convolutional residual block includes a fourth input terminal, a first 1×3 Conv convolution connected to the fourth input terminal, a second 1×3 Conv convolution connected to the first 1×3 Conv convolution, a third element-wise addition operation connected to the fourth input terminal and the second 1×3 Conv convolution, and a third output terminal connected to the third element-wise addition operation.

[0054] In this embodiment, a 6-layer multi-channel fusion encoder (MCFE) is designed to achieve the fusion and alignment of signals from different channels. Its structure is as follows: Figure 3 As shown, it mainly consists of 6 layers of convolutional fusion blocks, such as Figure 3 (a). Let the time-domain characteristics be... Frequency domain or current characteristics are ,in, For batch size, and These are the lengths of the feature vectors, The number of channels is given, and the input time-domain features of each layer are... The input frequency domain or current characteristics are Because different channels differ in sampling frequency, amplitude scale, and dynamic response characteristics, direct feature extraction can lead to inconsistent feature distributions between modes, weakening the fusion effect. To address this issue, the encoder first normalizes and aligns the signals of each channel with the time scale:

[0055] in, This indicates that mean-variance normalization is performed on each channel. This represents a linear interpolation operation that makes the length of the frequency domain or current feature consistent with the time domain feature. Subsequently, the encoder extracts and fuses the features through layer-by-layer convolutional residual blocks; the fusion process is as follows... Figure 3 (b) The structure of the convolutional residual block is as follows: Figure 3 (c). For the first layer:

[0056] in, This refers to the frequency domain or current characteristics of the next higher level. This represents the features after interpolation alignment. This represents the temporal features of the next layer. Through this layer-by-layer weighted fusion and residual learning, the temporal features gradually absorb local features from other channels, forming a unified high-dimensional representation. After processing by a 6-layer convolutional fusion encoder, the output features not only achieve alignment and information complementarity between different modalities, but also provide a highly consistent and discriminative input representation for subsequent joint feature extraction and discrimination.

[0057] During pre-training, a current signal branch is introduced into the MMCA motor fault diagnosis model, and vibration frequency domain branch and vibration time domain branch are constructed based on the frequency domain DWMR feature encoder and the time domain DWMR feature encoder, respectively. The current signal branch is used to extract features from the three-phase current signal using a fusion module, a DWMR feature encoder, a first projection head, and a first prediction head connected in sequence to obtain current features; the fusion module is a 1×1 convolution, used to integrate the current signals of each phase based on the three-phase current signal. The vibration time-domain branch is used to extract features from the vibration time-domain signal using a sequentially connected time-domain DWMR feature encoder, a second projection head, and a second prediction head to obtain vibration time-domain features. The vibration frequency domain branch is used to extract features from the vibration frequency domain signal using a DWMR feature encoder, a third projection head, and a third prediction head connected in sequence, to obtain vibration frequency domain features.

[0058] During the pre-training phase, the structure of the MMCA motor fault diagnosis model is as follows: Figure 4 As shown, the network mainly consists of three branches, which process vibration time-domain signals, vibration frequency-domain signals, and current signals, respectively, denoted as branch T, branch F, and branch C. Each branch contains an independent feature encoder, projection head, and prediction head. Since the input to branch C is three-phase current signal data, a fusion module is additionally introduced in this branch. After each batch of data is processed by the three branches, query vectors corresponding to the three modes can be obtained. .

[0059] Pre-training introduces a multimodal contrastive learning strategy (MMCL) and a multimodal fusion alignment strategy (MMFA), specifically: A momentum model is constructed; the momentum model has a three-branch structure; each branch corresponds one-to-one with the branch for removing the current signal from the first predictor head, the branch for removing the vibration time domain from the second predictor head, and the branch for removing the vibration frequency domain from the third predictor head; the parameters of the momentum model do not participate in backpropagation and gradient update, but are iteratively optimized through a momentum update strategy:

[0060] in, For the first The parameters of the momentum model after the next iteration; The momentum coefficient; For the first The parameters of the momentum model after -1 iteration; For the first The parameters of the MMCA motor fault diagnosis model during pre-training after the next iteration; To effectively align features from different modalities, a momentum contrast mechanism is introduced, and a momentum model similar to the three-branch structure is constructed. The momentum model has a structure largely consistent with the main network, but does not include a prediction head module. The parameters of this model do not participate in regular backpropagation and gradient updates; instead, they are iteratively optimized through a momentum update strategy.

[0061] Subsequently, the embedding sequence can be calculated using a momentum model to obtain the key vector. These key vectors, along with the query vectors, are then used to calculate the multimodal contrastive learning loss and modality alignment loss. By employing a moving average strategy to update the parameters of the momentum model, not only is the stability of the training process improved, but a relatively consistent key-value dictionary is also constructed, thereby enhancing the discriminative ability and contrast effect between different modal features.

[0062] Multimodal contrastive learning strategy (MMCL): The query vector is the feature output of the DWMR feature encoder corresponding to mode m of each pre-trained sample in the unlabeled pre-training sample set, and the key vector is the momentum representation output of the branch corresponding to mode n processed by the momentum model of each pre-trained sample. Based on the query vector and key vector, the similarity score of each pre-trained sample in mode m is calculated. The multimodal contrastive loss is calculated based on the similarity scores of each pre-trained sample in each mode. Mode m and mode n are both one of current, vibration time domain, and vibration frequency domain; mode m and mode n are not the same; the expression for the multimodal contrastive loss is:

[0063]

[0064]

[0065] in, For multimodal contrast loss; For modality The loss; Modal identifier; For vibration time-domain modes; These are vibration frequency domain modes; It is a current mode; This refers to the batch size of the pre-trained samples; For the first mode m, Similarity scores between positive sample pairs; For mode m, and Similarity score between negative sample pairs; This is a temperature coefficient used to adjust the steepness of the probability distribution; and All are pre-trained sample indexes; A similarity score; For modality The corresponding features output by the DWMR feature encoder; For modality Corresponding momentum characterization; Modal identifier; For transpose; To achieve multimodal contrastive learning, a multimodal contrastive learning strategy MMCL that can be extended to any number of modalities M is designed. This method can not only handle any number of modalities, but also reduce computational complexity through averaging strategies, while maintaining consistency and alignment information between different modalities.

[0066] Specifically, the query vector for each training sample in modality m is: The key vector generated by the momentum encoder is ,in, For batch size, Here, m represents the modal index, encompassing the vibration time domain, frequency domain, and current. First, for each mode as a baseline, a similarity matrix is ​​calculated. Then, the similarity matrices of other modes are averaged to obtain the comprehensive similarity score for each sample.

[0067] After scaling by the temperature parameter, the similarity matrix is ​​used as the input to the logits cross-entropy loss. The cross-entropy loss is used to distinguish between positive and negative samples. Specifically, the diagonal elements, which represent the representation of the same instance in different modalities, are used as the positive sample similarity, while the off-diagonal elements, which represent other instances, are used as the negative sample similarity. This encourages the different modal representations of the same instance to be close together, while maintaining distance between different instances.

[0068] Finally, the loss of all modal benchmarks is averaged to obtain the complete multimodal contrast loss:

[0069] The core advantage of this method lies in its efficient alignment of multimodal features by averaging the similarity between each of the three modalities as a benchmark and the other modalities. Simultaneously, it preserves symmetry, allowing each modality to fully utilize the mutual information of the other modalities. Furthermore, it explicitly distinguishes between positive and negative samples, ensuring that the representations of the same instance are as close as possible across different modalities, while maintaining separation between the representations of different instances, thereby improving the discriminative power and robustness of the multimodal features.

[0070] Multimodal Fusion Alignment Strategy (MMFA): Based on the features and momentum representations output by the DWMR feature encoders corresponding to each mode, the attention weight matrices for vibration time-domain modes and vibration frequency-domain modes, as well as the attention weight matrices for vibration time-domain modes and current modes, are calculated. Based on these attention weight matrices, a cross-modal alignment set is constructed. , , and ;Will and Input the first diagnostic branch to obtain the cross-modal classification loss in the time and frequency domains; and Input the second diagnostic branch to obtain the cross-modal classification loss for time and current; add the cross-modal classification loss for time and frequency domains and the cross-modal classification loss for time and current to obtain the alignment loss; the expressions for the attention weight matrix of vibration time domain mode and vibration frequency domain mode, the attention weight matrix of vibration time domain mode and current mode, and the cross-modal alignment set are respectively:

[0071]

[0072] in, The attention weight matrix is ​​obtained by using the vibration time-domain features as the query vector and the vibration frequency-domain momentum representation as the key vector. For activation functions; The vibration time-domain characteristics; This represents the momentum in the frequency domain of vibration. The attention weight matrix is ​​obtained by using the vibration frequency domain features as the query vector and the vibration time domain momentum representation as the key vector. It represents the frequency domain characteristics of vibration; This represents the momentum in the time domain of vibration. The attention weight matrix is ​​obtained by using the vibration time-domain features as the query vector and the momentum representation of the current as the key vector. This is a characterization of the momentum of electric current. The attention weight matrix is ​​obtained by using current characteristics as the query vector and the momentum representation in the vibration time domain as the key vector. Current characteristics; It is a time-frequency time-domain triple; For positive samples in the time domain, specifically A list consisting of the data on the main diagonal; From The time-domain negative samples sampled in the middle are specifically: A list consisting of the minimum values ​​in each row; For time-frequency domain triples; For positive samples in the frequency domain, specifically A list consisting of the data on the main diagonal; From The time-domain negative samples sampled in the middle are specifically: A list consisting of the minimum values ​​in each row; For vibration current vibration triplet; From The sampled positive samples in the time domain are specifically: A list consisting of the data on the main diagonal; From The sampled time-domain negative samples are specifically: A list consisting of the minimum values ​​in each row; It is a ternary current oscillation current; For positive current samples, specifically A list consisting of the data on the main diagonal; From The negative samples of current in the middle are specifically: A list consisting of the smallest data in each row.

[0073] Both the first diagnostic branch and the second diagnostic branch include a first multi-channel fusion encoder (MCFE), a first multilayer perceptron (MLP), and a second fully connected layer (FC) connected in sequence. The first multi-channel fusion encoder (MCFE) of the first diagnostic branch and the second diagnostic branch is used to perform feature fusion on the cross-modal alignment set of the two input modalities.

[0074] While contrastive learning can effectively enhance the discriminative power of features, relying solely on cross-modal contrastive learning is insufficient for effective alignment of multimodal features. Therefore, a multimodal fusion alignment strategy, MMFA, is further incorporated into the pre-training stage to achieve deep semantic alignment and feature fusion across time, frequency, and current modes, such as... Figure 5 As shown.

[0075] In this strategy, the similarity matrix between each mode is first calculated, and these similarity matrices are then normalized using softmax. This yields the probability distributions between the time-domain and frequency-domain modes, as well as between the time-domain and current-domain modes.

[0076] in, , , These are query vectors for time, frequency, and current mode, respectively. , , Let it be its corresponding momentum bond vector. This is the temperature coefficient.

[0077] To improve the discriminative ability in modality alignment, a negative sample sampling mechanism based on probability distribution is adopted. Specifically, for each sample, a negative sample sampling mechanism is used based on the probability distribution of another modality. A negative sample index is selected through polynomial sampling. A negative sample with a higher sampling probability indicates that it is closer to the benchmark in the embedding space. Semantically similar, they constitute hard negative samples. The sampled negative sample vectors are then labeled as follows: , and This forms a negative sample set for feature fusion alignment between branches T and F. , and the negative sample set for T and C. , Subsequently, the positive sample set and the hard negative sample set are concatenated to construct a mixed input to form an enhanced cross-modal alignment set. The alignments for TF and TC are as follows:

[0078] in, , These are respectively used as inputs for vibration time-domain and vibration frequency-domain features. , The vibration time-domain and current characteristics are used as inputs. Simultaneously, pseudo-labels are generated according to the positive and negative sample arrangement order, for example... and The positive sample pairs are labeled "1". and The value is "0" to indicate a match or mismatch between modalities. Subsequently, the outputs of the first and second diagnostic branches are matched with their corresponding pseudo-labels, and the cross-modal classification loss based on the cross-entropy function is calculated. and Finally, the two are summed to measure the feature consistency and discriminative ability between the TF and TC modalities, thus obtaining the alignment loss:

[0079] The multimodal fusion alignment learning strategy fully leverages the complementary information between multimodal data by jointly aligning features from TF and TC modalities. This strategy not only improves semantic consistency between different modalities but also provides more robust and discriminative feature representations for subsequent downstream tasks.

[0080] The pre-trained loss function is a weighted sum of the multimodal contrast loss and the alignment loss.

[0081] The learning strategy in the pre-training phase mainly relies on multimodal contrastive loss. Alignment loss with fusion The total loss of the joint training can be expressed as:

[0082] in, These are the weighting coefficients for the multimodal contrast loss and the fusion alignment loss.

[0083] During fine-tuning, the DWMR feature encoders of the time-domain branch and the frequency-domain branch in the pre-trained motor fault diagnosis model are used to replace the corresponding DWMR feature encoders of the MMCA motor fault diagnosis model. The multi-channel fusion encoder MCFE, MLP multilayer perceptron, and FC fully connected layer in the MMCA motor fault diagnosis model are replaced with the first diagnostic branch to obtain the replaced MMCA motor fault diagnosis model. The replaced MMCA motor fault diagnosis model is then trained using a labeled fine-tuning sample set to obtain the final motor fault diagnosis model.

[0084] When training the replaced MMCA motor fault diagnosis model, a two-way symmetric KL divergence is introduced to constrain the vibration time-domain and vibration frequency-domain modes; the loss function of the final motor fault diagnosis model is:

[0085]

[0086] in, The loss function for the final motor fault diagnosis model; for The weights; For KL divergence loss; The classification loss is based on supervised labels; For distribution For approximation The relative entropy loss at that time; The probability distribution of the output of the time-domain branch; The probability distribution of the frequency domain branch output; For distribution For approximation The relative entropy loss at that time; Indexed by category; For distribution The Middle The probability values ​​of each category; For distribution The Middle The probability values ​​of each category.

[0087] During the fine-tuning phase, branches T and F, as well as the fusion connecting them, are migrated to the encoder TF for the downstream fault diagnosis task. The fine-tuning phase includes... Figure 6 Structurally, the model employs a dual-branch architecture to reduce computational overhead, and replaces the original predictor head with a classifier to construct the target model. Regarding the learning strategy, considering the modal differences in the data samples and the potential bias in feature distribution, relying solely on classification loss could lead to semantic space separation between different modalities. Therefore, a cross-modal consistency loss based on KL divergence is introduced to co-optimize with the classification loss. This mechanism aims to measure and minimize the difference between the feature distributions of the two modalities, prompting the model to learn modality-independent semantic representations, thereby enhancing its robustness and generalization ability under varying operating conditions.

[0088] Specifically, let the feature embeddings of the time-domain branch and the frequency-domain branch be respectively... and After Softmax normalization, it can be regarded as a feature probability distribution. and To avoid the asymmetry of unidirectional KL divergence affecting training stability, a bidirectional symmetric KL divergence is used to constrain the two modes, as defined below:

[0089] in, Indicates when the distribution For approximation The relative entropy loss over time is used to characterize the degree of difference between the features of the two modalities in the semantic space. hour, This indicates that the features of the two modalities are fully aligned. By using symmetric constraints, the feature distribution of the time and frequency domain branches can be optimized simultaneously during gradient propagation, ensuring that both maintain consistent feature representations while sharing semantics.

[0090] In the specific optimization process, the consistency loss and cross-entropy classification loss are trained together to ensure the discrimination performance of downstream tasks while maintaining modality alignment. The final overall loss function is defined as:

[0091] in, This represents the classification loss based on supervised labels, with weights... This strategy is used to balance the relationship between cross-modal consistency constraints and classification task-driven requirements. Through this joint optimization strategy, the model can balance semantic consistency and task discriminability during the feature fusion stage, achieving stable identification and accurate classification of fault modes under complex operating conditions.

[0092] It is important to note that during the fine-tuning phase, a similar TFC three-branch structure to the pre-training phase can be used, or a TC two-branch structure can be chosen for feature extraction. However, the three-branch structure often fails to provide significant performance improvements in actual downstream tasks and significantly increases computational overhead. In contrast, while the TC two-branch structure utilizes fault information from the three-phase current data, its computational overhead is higher than that of the F branch, and its correlation and complementarity with the time-domain vibration signal are relatively weak, resulting in slightly lower discriminative ability after cross-modal feature fusion. Therefore, in order to fully utilize multimodal information and obtain the best diagnostic performance during the fine-tuning phase, only the TF two-branch structure was ultimately chosen as the backbone architecture for the fine-tuning phase.

[0093] The overall framework of MMCA is as follows Figure 7 It consists of four steps: Step 1: Data preprocessing. First, the original vibration signal and the three-phase current signal are segmented. The window length is set to 1024 and the step size is 512. Training set, fine-tuning set and test set are constructed respectively.

[0094] Step 2: Pre-training. The three-phase current signal is processed by the fusion module and then input into branch C; the vibration signal is subjected to Fast Fourier Transform to obtain frequency domain features, which are then input into branches T and F respectively. The three-branch features are subjected to self-supervised learning through a multi-modal comparison and alignment strategy, and robust multi-modal shared feature representations are obtained through a predefined pseudo-task.

[0095] Step 3: Fine-tuning. The pre-trained T and F branches are transferred to the downstream fault diagnosis task, and deep feature fusion is achieved through the MCFE module. The model is fine-tuned using a small number of labeled samples, the scale of which is set according to the specific task.

[0096] Step 4: Testing. The model performance is tested using 200 labeled samples per class, and the generalization ability and robustness of the model are further verified under different operating conditions and cross-device scenarios.

[0097] To verify the effectiveness of MMCA in motor fault diagnosis and its adaptability to multiple devices and operating conditions, three experimental cases with no similarity in equipment type, operating environment, and load characteristics were designed: Case 1 uses a compact, low-speed small motor (15–35 Hz) with stable operation and light load, mainly reflecting the signal characteristics under light load and low speed scenarios; Case 2 uses a traction motor (40–60 Hz) installed on a train bogie, which is large, high-power, and affected by lateral load and complex vibration environment, exhibiting significant differences in operating conditions; Case 3 is a cross-device verification experiment, that is, using the data from Case 1 for pre-training, and the data from Case 2 for fine-tuning and classification, and vice versa, to test the model's transferability between completely different devices and operating conditions. Subsequent experiments will be carried out around these three sets of experimental designs with no similarity in equipment structure, operating speed, and load characteristics, to fully demonstrate the robustness and wide adaptability of MMCA.

[0098] In the experiment, the number of fault samples for Case 1 and Case 2 were 14996 and 6245, respectively. During the pre-training phase, all samples were unlabeled, with quantities of 11396 and 4245, respectively. Of the remaining samples, 200 from each state class were used for fine-tuning, and 200 were used for testing. All signal samples had a uniform length of 1024, a step size of 512, and a batch size of 64. Specific sample parameter settings are shown in Table 1. The pre-training phase iterated for 50 rounds. The initial learning rate of the backbone network during the pre-training phase was 0.001, and the weight values ​​were... The optimizer is set to 0.5, AdamW is selected, and the weight decay factor is set to 0.001. During the fine-tuning phase, labeled fine-tuning samples are used, and 30 fine-tuning iterations are performed based on different sample sizes in different experimental settings. The weights are... The value was set to 0.1; finally, during the testing phase, the remaining 200 samples from each class were used for model performance testing. All experiments were conducted in a Python environment, using the PyTorch deep learning framework, and the experimental equipment was configured with a Core i7-13700KF CPU and an NVIDIA RTX 4080 GPU.

[0099] Table 1

[0100] In the comparative experiment, to fully verify the superiority of MMCA, four representative advanced self-supervised learning (SSL) methods were selected for comparison.

[0101] (1) BYOL: Learn feature consistency through dual network structure, without negative sample pairs, and realize stable self-supervised representation learning by using the momentum update of the target network.

[0102] (2) MOCO: A large-scale negative sample library is constructed using dynamic queues and momentum encoders, and the discriminative and representational quality is improved through a contrastive learning mechanism.

[0103] (3) TFpred: : Combining time-domain and frequency-domain signals into a self-supervised framework, it learns robust feature representations through cross-domain feature prediction tasks.

[0104] (4) TFAL: By aligning the time domain and frequency domain representations, it enhances cross-modal consistency and feature discrimination capabilities, and achieves efficient feature adaptive learning.

[0105] To verify the effectiveness of this invention on low-speed small induction motors, a fault test bench for low-speed small induction motors was built. This test bench includes a test motor, a motor controller, a data acquisition unit, and various sensors. The motor model is YB3-80M1-2, controlled by a motor controller model HLT-F0.7. This controller also integrates digital-to-analog conversion, converting data collected by vibration acceleration sensors and Hall current sensors and inputting it into a computer for recording. In the experiment, eight common faults of key motor components were simulated, and data on these faults were collected. These included mechanical faults such as bearing failure, rotor eccentricity, shaft bending, and shaft imbalance; electromechanical coupling faults such as broken rotor bars; and electrical faults such as short circuits, as shown in Table 2. The changes in fault characteristics under different speed conditions were explored using three different operating conditions: 15Hz, 25Hz, and 35Hz.

[0106] Table 2

[0107] This study focuses on analyzing the generalization ability of MMCA under varying operating conditions with different sample sizes. Based on the dataset, five representative labeled sample sizes were selected for fine-tuning: 6 (3%), 10 (5%), 20 (10%), 40 (20%), and 60 (30%). Simultaneously, six series of variable operating condition fault diagnosis tasks were designed accordingly. Table 3 below shows the results obtained by the proposed model in performing these tasks. It can be seen that in all 30 tests with different sample sizes and operating conditions, MMCA achieved an accuracy rate higher than 86%. When the sample size reached 10%, the accuracy rate was higher than 97%, with a mean of 96.15%. When the sample size reached 30%, all results approached 100%. This demonstrates the model's generalization ability; that is, it can still perform cross-domain fault diagnosis tasks even with a very small number of labeled samples. Furthermore, under the same labeled sample size, the accuracy rates of all variable operating condition tasks are similar, indicating that the model is robust to changes in operating conditions.

[0108] Table 3

[0109] MMCA and various comparative models were tested separately. To reflect the real-world scenario of scarce labeled samples, the size of all labeled samples used for fine-tuning was set to 5%. This means pre-training was performed using unlabeled samples from one domain, followed by fine-tuning using 10 labeled samples from each class in another domain. All methods employed the optimal parameters described in the paper, and the results are as follows. Figure 8 As shown, TFAI's results across the six tasks ranged only from 75.72% to 86.34%. This may be because the Transformer encoder is sensitive to sample size, requiring fine-tuning with a certain number of labeled samples under conditions of very few labels. In contrast, ResNet exhibits stronger local inductive bias and feature reuse capabilities, enabling stable extraction of key features under small sample conditions. Therefore, the TFPred model, using ResNet as its encoder, performs better. MMCE, through its encoder, extracts multi-level features and effectively fuses and aligns the potential representations of multi-modal signals such as vibration, spectrum, and current, achieving feature complementarity and enhanced consistency. Consequently, it achieved an accuracy of no less than 93% across all tasks, representing the best result among all methods, further demonstrating its stronger robustness and generalization ability.

[0110] To further compare the performance of MMCA with other models, a visualization analysis of the results was conducted using a 5% labeled sample size under the condition of greatest velocity change from 15 Hz to 35 Hz. Figure 9 The confusion matrix and t-SNE clustering analysis plots are shown. Globally, most misclassifications occur in the five fault types UB, BA, SE, DE, and RBB. This is due to the high similarity of their vibration signals. The remaining fault types, NO (normal mode), SC (electrical fault), and O and R (outer ring and roller faults of motor bearings respectively), exhibit significantly different vibration signals, thus almost all methods can distinguish them well. BYOL failed to distinguish SE, DE, and RBB well, possibly because these faults have globally similar vibration signal characteristics, and BYOL relies mainly on global consistency learning, lacking the ability to distinguish local modulation features. TFAI's misclassifications mainly occur in UB, BA, and SE, possibly because their time-frequency modes are highly similar and lack significant non-stationary features. TFAI's time-frequency alignment mechanism strengthens modal consistency in this case but weakens class discriminability, leading to inter-class aliasing in the latent space. In contrast, MMCA incorporates current signals for contrastive learning during the pre-training stage, thus utilizing differences in current signals to enhance feature discriminability and improve the ability to distinguish these three fault types.

[0111] To further verify the effectiveness of XMSSL in diagnosing motor faults, motor data from the BJTU-RAO dataset was used as Case 2 for testing. The subway train bogie transmission system fault simulation test bench in the BJTU-RAO dataset was scaled down and simplified to a 1:2 ratio based on a real subway bogie. A single power transmission chain in the test bench includes a motor, a reduction gearbox, and an axle box. The transmission chain is driven by a three-phase asynchronous AC motor, and the motor speed can be controlled by a frequency converter. The load is applied via a hydraulic system. The motor bearing model is SKF 6205-2RSH. Motor fault types include rotor bar breakage, shaft bending, short circuit, and bearing failure. The experiment was conducted under four different operating conditions: 40Hz / 0kN, 60Hz / 0kN, 40Hz / 10kN, and 60Hz / -10kN, to demonstrate the adaptability of MMCA to different operating conditions. Details are shown in Table 4.

[0112] Table 4

[0113] Since the dataset contains four operating conditions, testing under all combinations of these conditions would result in 12 variable operating condition tasks, which is excessive. Therefore, drawing on orthogonal experimental design principles, seven representative variable operating condition tasks were selected for testing to reduce testing complexity while ensuring experimental comprehensiveness. Five representative labeled sample sizes were also selected for fine-tuning, and the results are shown in Table 5. In the BJTU-RAO dataset, due to the significant differences in vibration signals of the motor under different fault states, all test tasks achieved high classification accuracy. Especially in the task where the operating condition change is most pronounced (40Hz / 10kN→60Hz / -10kN), even using only 3% of the labeled samples, the MMCA method still achieved an average accuracy of 97.64% across all tasks. Furthermore, the accuracy of MMCA in all experimental tasks was no less than 96.32%, further validating the method's excellent adaptability and generalization performance for variable operating condition tasks with limited labeled samples.

[0114] Table 5

[0115] Five self-supervised methods were tested using motor data from the BJTU-RAO dataset. Similar to the previous case, the size of all labeled samples used for fine-tuning was set to 5%, i.e., 10 labeled samples per class from another domain were used for fine-tuning. The results are as follows. Figure 10As shown. BYOL and MoCo maintained an accuracy of around 85% to 92% for this type of cross-domain task, indicating that their generalization ability is insufficient compared to the three latest methods. Similar to Case 1, TFpred, due to the addition of complex data augmentation methods, can still achieve good classification accuracy even when there are insufficient labeled samples. TFAI's performance in Case 2 is completely different from that in Case 1, achieving a classification accuracy of no less than 97.8%. This may be because (1) the sampling frequency in Case 2 is 60 kHz, the rotation speed range is 40–60 Hz, and a single sample contains 1024 data points, so each sample can completely cover one rotation cycle of the motor; (2) the differences between various faults are more significant in Case 2. This is also the reason for the performance improvement of all methods. MMCA performed relatively stably on 7 different cross-domain tasks, with an accuracy of 98.94 to 99.94%, which is the best among all methods, and it also proves its robustness and generalization ability.

[0116] With the proportion of labeled samples set at 5%, Figure 11 The results of five methods are visualized and analyzed in the cross-domain task with the greatest difference in operating conditions, from 40 / 10 to 60 / -10. BYOL and MoCo mainly misclassified three types of faults: BA, BF, and RBB. TFpred's misclassifications mainly came from RBB, SC, and NO. This may be related to TFpred's use of only time-domain samples during the fine-tuning phase. RBB and SC faults exhibit slight periodic fluctuations or amplitude perturbations in the time-domain waveform, especially in the early or minor fault stages, where their waveform morphology is similar to NO. Due to the lack of frequency-domain features, TFpred may have difficulty capturing these subtle differences, leading to insufficient discrimination. The results of TFAI and MMCA in this task clearly demonstrate good intra-class clustering and inter-class separability. However, MMCA performs better in terms of accuracy, further validating the aforementioned analysis of its performance advantages.

[0117] If the model can learn discriminative and robust signal representations during the pre-training phase, it can maintain good adaptability and generalization ability when transferred to fault diagnosis tasks in different domains. Based on this assumption, Case 3 and a series of cross-domain experiments were designed to further verify the domain transfer performance of the proposed MMCA. The experiments adopted an orthogonal experimental approach, setting up a total of 12 cross-domain fault diagnosis tasks. Six tasks used the data from Case 1 as the pre-training set and the data from Case 2 as the fine-tuning set, while the other six tasks did the opposite, to comprehensively evaluate the transfer effect of MMCA under different equipment and operating conditions.

[0118] Table 6 shows the performance of MMCA in 12 cross-device fault diagnosis tasks under different labeled sample proportions. It can be seen that when the labeled sample proportion is only 3%, although the accuracy varies between tasks, the overall accuracy exceeds 90%, with a mean of 93.76%. When the proportion increases to 5%, the accuracy of all tasks is above 95%, with a mean of 97.75%. When the labeled sample proportion reaches 10%, the fault classification accuracy of all tasks is almost 100%. This result demonstrates that the proposed MMCA exhibits excellent adaptability and generalization ability when dealing with tasks under different equipment and varying operating conditions.

[0119] Table 6

[0120] With a sample size of 5%, comparative experiments were conducted on 12 cross-device fault diagnosis tasks using five methods. The results are shown in Table 7 below. BYOL and MoCo maintained relatively stable accuracy across all tasks, ranging from 86% to 91%. TFpred, due to its integration of multiple data augmentation strategies during pre-training, effectively improved the model's robustness and generalization ability, thus exhibiting relatively stable performance across all tasks. TFAI performed exceptionally well in the first six tasks, but its accuracy dropped significantly in the latter six. Based on its results in Case 1, this can be attributed to the relatively stable signal features and small inter-class differences under low-speed conditions. TFAI's time-frequency alignment mechanism further strengthens modal consistency, leading to feature aliasing. Simultaneously, its Transformer encoder structure is sensitive to data size, making it prone to underfitting under small sample conditions. Furthermore, when the target domain is 15 Hz, the performance of all methods decreased because the sample length is insufficient to fully cover one rotation cycle of the motor, resulting in the loss of crucial information. It is worth noting that MMCA maintained excellent performance across all tasks, with an average accuracy of 97.75%. It achieved the best results in all but one task, indicating that it has stronger feature robustness and cross-domain generalization ability in dealing with different equipment types and varying operating conditions.

[0121] Table 7

[0122] With a 5% sample size, and using the cross-device task with the greatest operating condition difference (60Hz / -10kN→15Hz) as the test condition, a comparative analysis of five methods was conducted, and the resulting visualization analysis results are as follows. Figure 12When faced with cross-domain tasks under different equipment and varying operating conditions, most t-SNE results exhibit significant boundary confusion, with most methods showing marked inter-class boundary confusion in their t-SNE visualizations. This is primarily due to the substantial differences between the source and target domains in terms of rotational speed, load, and signal energy distribution, making it difficult for the model to align the feature distributions of the two domains in the latent space. MMCA introduces a dual alignment strategy during both the pre-training and fine-tuning stages, and integrates current signal information, thereby achieving consistency constraints on cross-modal features and full utilization of complementary features. Thanks to this design, MMCA significantly outperforms the other four methods in terms of classification accuracy for various faults and t-SNE visualization results, demonstrating clearer inter-class boundaries and higher feature discriminative power.

[0123] and These are the core weight parameters for the cross-domain comparison and alignment strategies in the pre-training phase, and the collaborative alignment strategy in the fine-tuning phase. They are used to balance the importance of each loss function, and their values ​​directly affect the model's learning process and the final fault diagnosis performance. To explore the impact of weight settings, experiments were conducted on the cross-domain task 60Hz / -10kN→15Hz in Case 3, and the sample size was set to 10% for detailed analysis.

[0124] During the pre-training phase, The weighting between contrast loss and alignment loss is determined by... Figure 13 (a) shows that values ​​between 0.3 and 0.8 do not have a significant impact on the results. However, when the value is maintained around 0.5, the final diagnostic accuracy is the highest, and the range of the five test results is also the smallest, indicating that the model's performance is more stable. This may be because when the value is too large, the model focuses on cross-domain comparison, leading to insufficient alignment; when it is too small, it weakens inter-domain constraints. When the two are balanced, the model has both discriminative power and domain-invariant feature representation ability.

[0125] Fine-tuning phase The weighting of classification loss and consistency loss is determined, such as Figure 13 (b) When its value is 0.1, the model achieves optimal stability and performance. However, the larger the value, the smaller the weight of the classification loss. Therefore, a significant drop in classification accuracy occurs after exceeding 0.8. This may be because the consistency loss based on KL divergence is mainly used to constrain the distribution similarity between time-frequency features. If its weight is too high, the model will excessively pursue the consistency of features between the two domains, neglecting discriminative learning with label information. This leads to a smoother feature distribution and blurred inter-class boundaries, thus weakening classification performance.

[0126] The ablation experiments aimed to verify the necessity of each module and loss function in MMCA and to evaluate their contribution to improving model performance. The objects covered included: Dynamic Weighted Multi-scale Residual Encoder, current channel data (replaced by vibration data from another channel if not used), MMFA, MMCL, classification loss, and consistency loss. Ablation experiments were conducted in Case 3's cross-domain task 60Hz / -10kN→15Hz and at a 10% labeled sample size to comprehensively evaluate whether the removal of these modules would impair MMCA's performance in dealing with limited samples, device type differences, and variable operating conditions.

[0127] The experimental results in Table 8 show that removing any item negatively impacts accuracy. Removing the consistency loss during the fine-tuning stage has the least impact, as its main function is to constrain the distribution similarity between time-frequency features. It plays a role in feature alignment and stability during the fine-tuning stage. The classification loss, however, is the main discriminative driver during the fine-tuning stage, crucial for guiding the model to learn discriminative features and achieving accurate classification; therefore, it cannot be removed. During the pre-training stage, removing MMFA and MMCL respectively resulted in 92.62% and 95.71% accuracy, a significant performance decrease, indicating their critical importance to the model's discriminative ability and cross-domain generalization performance. Replacing the current channel with vibration data from another channel reduced accuracy to 97.82%, demonstrating that in motor fault classification, the current signal provides complementary discriminative information in feature representation, and its fusion with the vibration signal can more fully characterize the fault mode. Encoder replacement caused a 1.57% decrease in model accuracy, further demonstrating that multi-scale residual learning has stronger feature extraction capabilities compared to ResNet-18.

[0128] Table 8

[0129] This invention proposes a novel self-supervised fault diagnosis framework based on multimodal comparison and alignment to address the challenges of fault diagnosis in real-world industrial environments, such as a lack of labeled data, variable operating conditions, and even cross-device migration. The framework employs a dynamically weighted multi-scale residual encoder to achieve adaptive feature extraction for different data types. It combines two different learning strategies, MMCL and MMFA, for deep feature comparison, interaction, and fusion alignment. It achieves highly robust and accurate fault classification with only a small number of labeled samples. In cross-device experiments, with only 3% labeled samples, an average fault diagnosis accuracy of 93.76% was achieved under equipment migration conditions. When the sample size reached 20%, the average accuracy for cross-device tasks reached 99.99%, demonstrating its effectiveness, generalizability, and domain adaptability.

Claims

1. A method for diagnosing motor faults in scenarios with limited tag samples and across devices, characterized in that, include: Obtain the unlabeled pre-trained sample set and the labeled fine-tuned sample set of the motor; Construct an MMCA motor fault diagnosis model; The MMCA motor fault diagnosis model includes: A frequency-domain DWMR feature encoder is used to extract features from vibration frequency-domain signals to obtain frequency-domain features; A time-domain DWMR feature encoder is used to extract features from vibration time-domain signals to obtain time-domain features; The multi-channel fusion encoder (MCFE) is used to fuse frequency domain features and time domain features to obtain fused features. Multilayer perceptron (MLP) is used to perform nonlinear mapping on fused features to obtain mapped features; Fully connected (FC) layers are used to obtain classification results based on the mapped features. The MMCA motor fault diagnosis model was pre-trained using an unlabeled pre-training sample set to obtain a pre-trained motor fault diagnosis model. The pre-trained motor fault diagnosis model was fine-tuned using a labeled fine-tuning sample set to obtain the final motor fault diagnosis model. The final motor fault diagnosis model is used to diagnose the test condition and obtain the motor fault diagnosis results.

2. The motor fault diagnosis method for limited tag samples and cross-device scenarios according to claim 1, characterized in that, The DWMR feature encoder uses an improved ResNet-18 network. Specifically, the improved ResNet-18 network has a 1×7 kernel in the first convolutional layer, a 1×3 kernel in the max-pooling layer connected to the first convolutional layer, and a two-layer bottleneck structure for each residual block. The bottleneck layer includes a first input, three parallel branches connected to the first input, a 1×1 Conv convolution, a dynamic scale weighting unit connected to each parallel branch, a concatenation operation connected to the dynamic scale weighting unit, a first element-wise addition operation connected to the 1×1 Conv convolution and the concatenation operation, and a first output connected to the first element-wise addition operation. Each parallel branch includes a convolutional layer, batch normalization, ReLU activation function, feature length alignment unit, and feature gating unit connected sequentially. The convolutional kernels of the convolutional layers in each parallel branch are 1×3, 1×5, and 1×7, respectively; the feature gating unit includes a second input, a first adaptive average pooling connected to the second input, a first 1×1 Conv convolution connected to the first adaptive average pooling, a second 1×1 Conv convolution connected to the first 1×1 Conv convolution, a sigmoid activation function connected to the second 1×1 Conv convolution, an element-wise multiplication connected to the sigmoid activation function and the second input, and a second output connected to the element-wise multiplication; the dynamic scale weighting unit includes a third input, a second adaptive average pooling connected to the third input, a second element-wise addition operation connected to the second adaptive average pooling, a first fully connected (FC) layer connected to the second element-wise addition operation, a softmax activation function connected to the first FC layer, and a weight output connected to the softmax activation function and the third input; the feature length alignment unit is a clipping alignment operation.

3. The motor fault diagnosis method for limited tag samples and cross-device scenarios according to claim 1, characterized in that, The multi-channel fusion encoder (MCFE) comprises six stacked convolutional fusion blocks; the expression for the data processing procedure of each convolutional fusion block is as follows: in, The temporal features output by the current convolutional fusion block; For convolution residual blocks; These are the temporal features output by the convolutional fusion block from the previous layer; The frequency domain features are those of the current convolutional fusion block after interpolation and alignment. The frequency domain features output by the current convolutional fusion block; This represents the frequency domain features output by the previous layer's convolutional fusion block; This is a linear interpolation operation; To perform mean-variance normalization for each channel; For feature size; The length of the feature vector.

4. The motor fault diagnosis method for limited tag samples and cross-device scenarios according to claim 3, characterized in that, The convolutional residual block includes a fourth input terminal, a first 1×3 Conv convolution connected to the fourth input terminal, a second 1×3 Conv convolution connected to the first 1×3 Conv convolution, a third element-wise addition operation connected to the fourth input terminal and the second 1×3 Conv convolution, and a third output terminal connected to the third element-wise addition operation.

5. The motor fault diagnosis method for limited tag samples and cross-device scenarios according to claim 1, characterized in that, During pre-training, a current signal branch is introduced into the MMCA motor fault diagnosis model, and vibration frequency domain branch and vibration time domain branch are constructed based on the frequency domain DWMR feature encoder and the time domain DWMR feature encoder, respectively. The current signal branch is used to extract features from the three-phase current signal using a fusion module, a DWMR feature encoder, a first projection head, and a first prediction head connected in sequence to obtain current features; the fusion module is a 1×1 convolution, used to integrate the current signals of each phase based on the three-phase current signal. The vibration time-domain branch is used to extract features from the vibration time-domain signal using a sequentially connected time-domain DWMR feature encoder, a second projection head, and a second prediction head to obtain vibration time-domain features. The vibration frequency domain branch is used to extract features from the vibration frequency domain signal using a DWMR feature encoder, a third projection head, and a third prediction head connected in sequence, to obtain vibration frequency domain features.

6. The motor fault diagnosis method for limited tag samples and cross-device scenarios according to claim 5, characterized in that, Pre-training introduces a multimodal contrastive learning strategy (MMCL) and a multimodal fusion alignment strategy (MMFA), specifically: A momentum model is constructed; the momentum model has a three-branch structure; each branch corresponds one-to-one with the branch for removing the current signal from the first predictor head, the branch for removing the vibration time domain from the second predictor head, and the branch for removing the vibration frequency domain from the third predictor head; the parameters of the momentum model do not participate in backpropagation and gradient update, but are iteratively optimized through a momentum update strategy: in, For the first The parameters of the momentum model after the next iteration; The momentum coefficient; For the first The parameters of the momentum model after -1 iteration; For the first The parameters of the MMCA motor fault diagnosis model during pre-training after the next iteration; Multimodal contrastive learning strategy (MMCL): The query vector is the feature output of the DWMR feature encoder corresponding to mode m of each pre-trained sample in the unlabeled pre-training sample set, and the key vector is the momentum representation output of the branch corresponding to mode n processed by the momentum model of each pre-trained sample. Based on the query vector and key vector, the similarity score of each pre-trained sample in mode m is calculated. The multimodal contrastive loss is calculated based on the similarity scores of each pre-trained sample in each mode. Mode m and mode n are both one of current, vibration time domain, and vibration frequency domain; mode m and mode n are not the same; the expression for the multimodal contrastive loss is: in, For multimodal contrast loss; For modality The loss; Modal identifier; For vibration time-domain modes; These are vibration frequency domain modes; It is a current mode; This refers to the batch size of the pre-trained samples; For the first mode m, Similarity scores between positive sample pairs; For mode m, and Similarity score between negative sample pairs; This is a temperature coefficient used to adjust the steepness of the probability distribution; and All are pre-trained sample indexes; A similarity score; For modality The corresponding features output by the DWMR feature encoder; For modality Corresponding momentum characterization; Modal identifier; For transpose; Multimodal Fusion Alignment Strategy (MMFA): Based on the features and momentum representations output by the DWMR feature encoders corresponding to each mode, the attention weight matrices for vibration time-domain modes and vibration frequency-domain modes, as well as the attention weight matrices for vibration time-domain modes and current modes, are calculated. Based on these attention weight matrices, a cross-modal alignment set is constructed. , , and ;Will and Input the first diagnostic branch to obtain the cross-modal classification loss in the time and frequency domains; and Input the second diagnostic branch to obtain the cross-modal classification loss for time and current; add the cross-modal classification loss for time and frequency domains and the cross-modal classification loss for time and current to obtain the alignment loss; the expressions for the attention weight matrix of vibration time domain mode and vibration frequency domain mode, the attention weight matrix of vibration time domain mode and current mode, and the cross-modal alignment set are respectively: in, The attention weight matrix is ​​obtained by using the vibration time-domain features as the query vector and the vibration frequency-domain momentum representation as the key vector. For activation functions; The vibration time-domain characteristics; This represents the momentum in the frequency domain of vibration. The attention weight matrix is ​​obtained by using the vibration frequency domain features as the query vector and the vibration time domain momentum representation as the key vector. It represents the frequency domain characteristics of vibration; This represents the momentum in the time domain of vibration. The attention weight matrix is ​​obtained by using the vibration time-domain features as the query vector and the momentum representation of the current as the key vector. This is a characterization of the momentum of electric current. The attention weight matrix is ​​obtained by using current characteristics as the query vector and the momentum representation in the vibration time domain as the key vector. Current characteristics; It is a time-frequency time-domain triple; For positive samples in the time domain, specifically A list consisting of the data on the main diagonal; From The time-domain negative samples sampled in the middle are specifically: A list consisting of the minimum values ​​in each row; For time-frequency domain triples; For positive samples in the frequency domain, specifically A list consisting of the data on the main diagonal; From The time-domain negative samples sampled in the middle are specifically: A list consisting of the minimum values ​​in each row; For vibration current vibration triplet; From The sampled positive samples in the time domain are specifically: A list consisting of the data on the main diagonal; From The sampled time-domain negative samples are specifically: A list consisting of the minimum values ​​in each row; It is a ternary current oscillation current; For positive current samples, specifically A list consisting of the data on the main diagonal; From The negative samples of current in the middle are specifically: A list consisting of the smallest data in each row.

7. The motor fault diagnosis method for limited tag samples and cross-device scenarios according to claim 6, characterized in that, Both the first diagnostic branch and the second diagnostic branch include a first multi-channel fusion encoder (MCFE), a first multilayer perceptron (MLP), and a second fully connected layer (FC) connected in sequence. The first multi-channel fusion encoder (MCFE) of the first diagnostic branch and the second diagnostic branch is used to perform feature fusion on the cross-modal alignment set of the two input modalities.

8. The motor fault diagnosis method for limited tag samples and cross-device scenarios according to claim 6, characterized in that, The pre-trained loss function is a weighted sum of the multimodal contrast loss and the alignment loss.

9. The motor fault diagnosis method for limited tag samples and cross-device scenarios according to claim 1, characterized in that, During fine-tuning, the DWMR feature encoders of the time-domain branch and the frequency-domain branch in the pre-trained motor fault diagnosis model are used to replace the corresponding DWMR feature encoders of the MMCA motor fault diagnosis model. The multi-channel fusion encoder MCFE, MLP multilayer perceptron, and FC fully connected layer in the MMCA motor fault diagnosis model are replaced with the first diagnostic branch to obtain the replaced MMCA motor fault diagnosis model. The replaced MMCA motor fault diagnosis model is then trained using a labeled fine-tuning sample set to obtain the final motor fault diagnosis model.

10. The method for motor fault diagnosis in limited tag samples and cross-device scenarios according to claim 9, characterized in that, When training the replaced MMCA motor fault diagnosis model, a two-way symmetric KL divergence is introduced to constrain the vibration time-domain and vibration frequency-domain modes; the loss function of the final motor fault diagnosis model is: in, The loss function for the final motor fault diagnosis model; for The weights; For KL divergence loss; The classification loss is based on supervised labels; For distribution For approximation The relative entropy loss at that time; The probability distribution of the output of the time-domain branch; The probability distribution of the frequency domain branch output; For distribution For approximation The relative entropy loss at that time; Indexed by category; For distribution The Middle The probability values ​​of each category; For distribution The Middle The probability values ​​of each category.

Images