Mechanical fault diagnosis method and system based on deep learning

Abstract

The invention relates to a mechanical fault diagnosis method and system based on deep learning. The method comprises the following steps: converting a multi-source time domain signal into a time frequency image through continuous wavelet transform, extracting features by using a primary feature encoder, and extracting cross-source common features through adversarial training of a shared feature discriminator; therefore, a gated multi-scale encoder is guided to enhance common feature expression, and deep fusion of multi-source features is realized through a cross multi-head attention network. Global average pooling and maximum pooling are synchronously carried out on the fused features to give consideration to overall and local information, and a comprehensive feature vector is formed; and finally, by means of a double-branch diagnosis network, the training loss of the multi-class network is dynamically weighted according to the prior probability output by the binary network, so that multi-source information is effectively fused under the condition of data imbalance, and the accuracy and robustness of fault classification are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of mechanical fault diagnosis, and in particular relates to a mechanical fault diagnosis method and system based on deep learning. Background Technology

[0002] In modern industrial systems, rotating machinery, as critical equipment, directly impacts production safety and efficiency through its operational status. Intelligent fault diagnosis technology for rotating machinery has always been a core research direction in industrial operations and maintenance. In recent years, deep learning-based fault diagnosis methods have made significant progress due to their powerful automatic feature extraction capabilities. However, in practical industrial applications, these methods consistently face two intertwined fundamental challenges.

[0003] First, there is the inherent limitation and severe imbalance of the data. Mechanical equipment is in a normal, healthy state most of the time, resulting in a small number of fault samples that can be collected, and the frequency of different fault types varies greatly. Using such an imbalanced dataset to train a model leads to severe "label bias" and "training bias." The model will overfit to the majority of normal samples, while its ability to identify the crucial minority fault samples is poor. In other words, although the overall accuracy of the model may not be low, the false negative rate for actual faults is very high, significantly reducing its generalization performance and practical value.

[0004] Secondly, the multi-source heterogeneous information from multiple sensor systems is difficult to fuse effectively. To comprehensively perceive equipment status, various sensors, such as vibration, current, and acoustic sensors, are deployed simultaneously. These signals from different physical sources exhibit significant differences in time-frequency characteristics, dimensions, and noise patterns. Most existing mainstream methods adopt relatively simple strategies, such as direct splicing at the feature level or voting and averaging at the decision level. These strategies fail to delve into the inherent, complex correlations and complementarities between different source signals; they are essentially just simple information superposition rather than deep fusion, resulting in low information utilization, failing to fully leverage the comprehensive advantages of multi-source monitoring, and having a relatively low upper limit for diagnostic performance.

[0005] In summary, current technologies have not yet been able to effectively address the two key challenges of imbalanced data samples and deep fusion of multi-source information within a unified framework. This hinders the crucial step of intelligent diagnostic models moving from laboratory performance to highly reliable and robust applications in industrial settings.

[0006] Therefore, there is an urgent need for an innovative technical solution that can guide the model to focus on scarce fault characteristics from a mechanism perspective and achieve deep complementarity and fusion of multi-source information at the representation level, thereby enabling accurate and reliable mechanical fault diagnosis under real industrial data conditions. Summary of the Invention

[0007] Therefore, it is necessary to provide a mechanical fault diagnosis method and system based on deep learning to address the aforementioned technical problems.

[0008] Firstly, this application provides a deep learning-based mechanical fault diagnosis method, including:

[0009] S1. Collect at least two different physical characteristics of runtime domain signals from the same mechanical equipment, perform continuous wavelet transform on each runtime domain signal to generate the corresponding time-frequency image; integrate the time-frequency images corresponding to all types of runtime domain signals to obtain a multi-source time-frequency image;

[0010] S2. Input the multi-source time-frequency images into the corresponding primary feature encoders for feature extraction to obtain multi-source primary feature maps; input the multi-source primary feature maps into a shared feature discriminator, extract the common features in the multi-source primary feature maps through adversarial training, and use the common features as feedback information.

[0011] S3. Based on feedback information, a gated multi-scale encoder is used to perform secondary feature encoding and weighted enhancement of common features on the multi-source primary feature map to obtain a multi-source enhanced feature map.

[0012] S4. Input the multi-source enhanced feature map into the feature fusion network based on the cross-multi-head attention mechanism. By calculating the cross attention between different source features, the multi-source enhanced feature map is fused to generate multi-source fused features.

[0013] S5. Perform global average pooling and global max pooling operations on the multi-source fusion features to obtain the pooling results; concatenate the pooling results to obtain the comprehensive diagnostic feature vector;

[0014] S6. Input the comprehensive diagnostic feature vector into the binary diagnostic network and the multi-class diagnostic network. Dynamically weight the training loss of the multi-class diagnostic network through the output of the binary diagnostic network, and use the output of the multi-class diagnostic network as the mechanical fault diagnosis result.

[0015] Secondly, this application also provides a deep learning-based mechanical fault diagnosis system for implementing the method described in the first aspect, the system comprising:

[0016] The multi-source time-frequency feature construction module is used to collect at least two different physical characteristics of runtime domain signals from the same mechanical equipment, perform continuous wavelet transform on each runtime domain signal to generate the corresponding time-frequency image, and integrate the time-frequency images corresponding to all types of runtime domain signals to obtain a multi-source time-frequency image.

[0017] The feature commonality extraction module is used to input the multi-source time-frequency images into the corresponding primary feature encoders for feature extraction, thereby obtaining multi-source primary feature maps; the multi-source primary feature maps are then input into a shared feature discriminator, which extracts common features from the multi-source primary feature maps through adversarial training, and uses the common features as feedback information.

[0018] The feature enhancement and optimization module is used to perform secondary feature encoding and weighted enhancement of common features on the multi-source primary feature map based on feedback information using a gated multi-scale encoder, so as to obtain a multi-source enhanced feature map;

[0019] The multi-source feature fusion module is used to input multi-source enhanced feature maps into a feature fusion network based on a cross-multi-head attention mechanism. By calculating the cross attention between different source features, the multi-source enhanced feature maps are fused to generate multi-source fused features.

[0020] The feature vectorization module performs global average pooling and global max pooling operations on the multi-source fusion features to obtain the pooling results; the pooling results are then concatenated to obtain the comprehensive diagnostic feature vector.

[0021] The fault diagnosis decision module is used to input the comprehensive diagnostic feature vector into the binary diagnostic network and the multi-class diagnostic network. The training loss of the multi-class diagnostic network is dynamically weighted by the output of the binary diagnostic network, and the output of the multi-class diagnostic network is used as the mechanical fault diagnosis result.

[0022] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a deep learning-based mechanical fault diagnosis method as described in the first aspect.

[0023] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a deep learning-based mechanical fault diagnosis method as described in the first aspect.

[0024] The aforementioned deep learning-based mechanical fault diagnosis method and system first acquires runtime signals with various physical characteristics and performs continuous wavelet transform to generate multi-source time-frequency images containing rich time-frequency features, providing comprehensive input for subsequent deep analysis. Then, a primary feature encoder is used to extract primary deep features from each source signal, and a shared feature discriminator is innovatively introduced. Through an adversarial training mechanism, the encoder is forced to discard source-specific information, thereby extracting common features that are relevant to the fault essence and consistent across sources. Using these common features as guidance, a gated multi-scale encoder adaptively strengthens the components related to common fault modes in each source feature, resulting in enhanced multi-source feature representations. Subsequently, a feature fusion network constructed based on a cross-multi-head attention mechanism allows for deep interactive querying and information aggregation between features from different sources, achieving... This approach achieves deep fusion, moving from simple feature coexistence to intrinsic semantic association. Global average pooling and global max pooling are applied to the fused features simultaneously to capture the overall feature distribution and the most salient local patterns, respectively. These two are then concatenated to form a comprehensive diagnostic feature vector, thus balancing global representativeness and local discriminativeness. Finally, a dual-branch diagnostic network is designed, utilizing the "abnormal" prior probability provided by the binary diagnostic network to dynamically adjust the training loss weights of the multi-class diagnostic networks. This mechanism effectively curbs model bias caused by the dominance of normal samples, forcing the model to give equal importance to scarce fault samples. Ultimately, under the realistic condition of extremely imbalanced data distribution, high-precision and robust identification and classification of multiple types of faults are achieved. Furthermore, the end-to-end design of the entire process ensures the synergistic optimization of feature learning and fault diagnosis, enhancing the practicality and generalization ability of the method. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A flowchart illustrating a deep learning-based mechanical fault diagnosis method provided by this invention;

[0027] Figure 2 This is a schematic diagram of the process for generating multi-source fusion features in one optional embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of a mechanical fault diagnosis system based on deep learning provided by the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] refer to Figure 1 The document presents a flowchart illustrating a deep learning-based mechanical fault diagnosis method provided in this application, which includes the following steps:

[0031] S1. Collect at least two different physical characteristics of runtime domain signals from the same mechanical equipment, perform continuous wavelet transform on each runtime domain signal to generate the corresponding time-frequency image; integrate the time-frequency images corresponding to all types of runtime domain signals to obtain a multi-source time-frequency image.

[0032] Specifically, the core objective of this step is to acquire multi-dimensional raw data that comprehensively reflects the operating status of mechanical equipment, and to convert time-domain signals into image forms that are easier for deep learning models to capture fault features through time-frequency transformation. First, at least two different physical characteristics of the same mechanical equipment's operating domain signals are collected. Signals with different physical characteristics are chosen because a single type of signal is insufficient to comprehensively characterize the equipment's fault state. For example, vibration signals can directly reflect mechanical fault characteristics such as impact and wear in the mechanical structure; current signals can indirectly reflect electrical anomalies such as changes in motor load and winding faults; and acoustic signals can capture abnormal sound radiation generated by friction on the equipment surface and collisions of components. The complementarity of multiple signal sources can significantly reduce the risk of missing faults due to a single signal. In the specific acquisition process, appropriate sensors are selected according to the signal type: vibration signals use an accelerometer with a range of ±50g and a frequency response of 1-10000Hz, installed in locations with high vibration transmission efficiency, such as the equipment bearing housing and motor end cover; current signals use a Hall current sensor connected in series in the equipment's power supply circuit to ensure accurate acquisition of operating current changes; and acoustic signals use a microphone sensor with a signal-to-noise ratio ≥60dB, placed close to key components such as the gearbox and bearing assembly. The sampling frequency of all sensors is uniformly set to 10kHz, and data is collected synchronously through data acquisition devices such as DH5902N to avoid feature misalignment caused by asynchronous sampling. The acquisition time of each group of signals is set to 10 seconds to ensure that the complete operating status cycle is included.

[0033] Continuous wavelet transform (WT) is performed on each acquired runtime domain signal to generate a time-frequency image. The Morlet mother wavelet is selected for the transform, as it exhibits good localization characteristics in both the time and frequency domains, effectively capturing the non-stationary characteristics of fault signals. Mechanical fault signals typically exhibit non-stationary features such as short-duration impulses and frequency abrupt changes. Traditional Fourier transform struggles to simultaneously achieve both time and frequency resolution, while continuous wavelet transform, through adjustment of the scaling parameter 'a' and the shift parameter 'b', can accurately capture different frequency components of the signal. The transform process follows the formula below: ,in This represents the input time-domain signal. Represents the Morlet mother wavelet and its complex conjugate wavelet Parameter 'a' adjusts the frequency resolution, ranging from 1 to 128. A smaller value for 'a' corresponds to higher resolution of high-frequency components, better capturing the instantaneous characteristics of fault impacts. Parameter 'b' adjusts the time resolution, ranging from 1 to 256, ensuring complete coverage of the signal's time dimension. The wavelet coefficients represent the amplitudes of the time-frequency images, with their values ​​directly corresponding to the pixel values ​​in the time-frequency image. Through this continuous wavelet transform, a one-dimensional time-domain signal is converted into a two-dimensional time-frequency image. The horizontal axis of the image corresponds to the time dimension, the vertical axis to the frequency dimension, and the pixel values ​​correspond to the amplitudes of the wavelet coefficients, visually presenting the energy distribution of the signal at different times and frequencies. Fault features will be highlighted in the time-frequency image as high-energy regions. Finally, the time-frequency images corresponding to all types of time-domain signals are integrated to form a multi-source time-frequency image dataset. For example, 1000 sets of time-domain data for vibration signals generate 1000 time-frequency images, and 1000 sets of current signals are simultaneously acquired to generate 1000 time-frequency images. After integration, a multi-source time-frequency image dataset containing 2000 images is obtained, providing a comprehensive and complementary data foundation for subsequent multi-source feature extraction.

[0034] S2. Input the multi-source time-frequency images into the corresponding primary feature encoders for feature extraction to obtain multi-source primary feature maps; input the multi-source primary feature maps into a shared feature discriminator, extract the common features in the multi-source primary feature maps through adversarial training, and use the common features as feedback information.

[0035] Specifically, this step aims to extract fundamental fault features from multi-source time-frequency images using a primary feature encoder, and then leverage a shared feature discriminator through adversarial training to uncover common information within these multi-source features, laying the foundation for subsequent feature fusion. The multi-source time-frequency images are input into their respective primary feature encoders for feature extraction. Each signal source is configured with an independent primary feature encoder, and all encoders have identical structures and parameters to ensure fairness and comparability of feature extraction from different sources. The primary feature encoder is based on a lightweight YOLOv10 network design. This network optimizes the convolutional block and bottleneck layer structure to reduce computational load while maintaining feature extraction capabilities, making it suitable for resource-constrained deployment environments in industrial settings. The encoder's specific structural design is as follows: The input is a 3×H×W (H=256, W=256) time-frequency image. First, it passes through a ConvBlock module (3×3 convolutional kernel, 8 output channels, equipped with a batch normalization (BN) layer and a SiLU activation function). The BN layer accelerates network convergence and suppresses overfitting, while the SiLU activation function effectively avoids the gradient vanishing problem compared to ReLU. Then, it passes through multiple alternating ConvBlock and C3Block modules. The C3Block module uses multi-path convolution to extract features in parallel and performs feature fusion through a bottleneck structure, enhancing the ability to capture multi-scale features. The convolutional kernel size is 3×3, and the number of output channels gradually increases to 16, 64, and 128. Finally, it connects to the SPPF (Spatial Pyramid Pooling Fast) module, which processes features of different sizes simultaneously through pooling operations at multiple scales, further improving the extraction accuracy of multi-scale features. If the input image size is 3×H×W, the corresponding output feature map size is 128×H / 16×W / 16. This process can be expressed by the following formula: , ,in and These represent the input time-frequency images from two different signal sources. This indicates a feature extraction module based on a lightweight YOLOv10 architecture. and These correspond to the multi-source primary feature maps extracted from two different signal sources. These feature maps have initially extracted basic fault-related features such as edges, textures, and high-energy regions in the time-frequency image.

[0036] The multi-source primary feature maps are input into a shared feature discriminator, which extracts common features and generates feedback information through adversarial training. The core design concept of the shared feature discriminator originates from domain adversarial learning, aiming to align the distribution differences of multi-source features and uncover common features that do not change with the signal source and are only related to the fault state. The discriminator's input is a multi-source primary feature map (128×H / 16×W / 16). First, a linear layer flattens the three-dimensional feature map into a one-dimensional vector, followed by a gradient inversion layer (GRL). This layer maintains the feature unchanged during forward propagation and inverts the gradient direction during backward propagation, making it a key component for adversarial training. Next, a fully connected layer (output channel 128), a batch normalization (BN) layer, and a dropout layer (dropout probability 0.3) are passed through it, followed by another fully connected layer (output channel 64), a BN layer, and a dropout layer (dropout probability 0.3). Finally, a fully connected layer outputs a two-dimensional vector, corresponding to the classification results of the two signal sources. The adversarial training process employs a minimax game strategy, with its loss function defined as follows: ,in This represents the adversarial training loss value. and These represent two primary feature encoders that process the first and second source signals, respectively. Indicates a shared feature discriminator, This indicates that the shared feature discriminator corresponds to the first source input. Classification branches, This indicates that the shared feature discriminator corresponds to the second source input. Classification branches, This represents a logarithmic function with the natural constant as its base. This represents the mathematical expectation operator. The goal of the shared feature discriminator is to maximize the classification accuracy of the feature source, that is, to accurately distinguish whether the input feature comes from a vibration signal or a current signal; while the goal of the primary feature encoder is to minimize the classification accuracy of the discriminator, that is, to generate features that the discriminator cannot distinguish the source. In this game, the primary feature encoder is forced to abandon the specific features related to the signal source and instead learn the common fault-related features shared by multiple source signals, while the shared feature discriminator continuously optimizes and improves its ability to distinguish feature sources, eventually reaching a Nash equilibrium. At this point, the features extracted by the discriminator to distinguish feature sources are the common features of the multiple source signals, and the corresponding backpropagation gradient signal is output as feedback information. This feedback information carries the distribution information of the common features and provides guidance for the weighted enhancement of subsequent secondary feature encoding.

[0037] S3. Based on the feedback information, a gated multi-scale encoder is used to perform secondary feature encoding and common feature weighting enhancement on the multi-source primary feature map to obtain the multi-source enhanced feature map.

[0038] Specifically, this step, based on the feedback information output by the shared feature discriminator, further deepens feature extraction through a gated multi-scale encoder, strengthens the common information in multi-source features, suppresses redundancy and noise, and obtains a multi-source enhanced feature map with stronger discriminative power. The gated multi-scale encoder consists of two parts: a multi-scale feature extraction module and an adaptive gating network. The multi-scale feature extraction module is responsible for mining multi-scale fault features from the primary features, while the adaptive gating network is responsible for weighted enhancement of common features using feedback information. First, the multi-source primary feature maps are input into a multi-scale feature extraction module. This module contains multiple parallel convolutional branches, each using convolutional kernels of different sizes (1×1, 3×3, 5×5, 7×7). Different kernel sizes correspond to different receptive fields: the 1×1 kernel has the smallest receptive field, primarily capturing local details in the feature maps, such as small high-energy regions of fault impact in time-frequency images; the 3×3 and 5×5 kernels have medium receptive fields, used to capture medium-scale feature combinations, such as continuous distribution areas of fault features; and the 7×7 kernel has the largest receptive field, used to capture global structural features, such as the overall frequency band shift caused by the fault. Each convolutional branch has a structure of "convolutional layer + BN layer + SiLU activation function" to ensure the stability and non-linear expressive power of feature extraction in each branch. The output feature dimensions of all branches are kept consistent (128×H / 16×W / 16) to facilitate subsequent splicing and weighted fusion.

[0039] The core function of an adaptive gating network is to dynamically adjust the weights of each convolutional branch based on feedback information, strengthening the outputs of branches related to common features and suppressing the influence of redundant branches. The feedback information is the gradient signal generated during the adversarial training of the shared feature discriminator. This signal contains the distribution characteristics of common information from multiple features, guiding the gating network to learn which scales of features are more relevant to fault diagnosis. The gating weight coefficients generated by the adaptive gating network can be formally represented as follows: ,in These are the weighting coefficients of the i-th convolutional branch. It is the concatenated result of the multi-scale features output by each convolutional branch. The sigmoid activation function restricts the weight coefficients to between 0 and 1. It is a trainable gating weight matrix used to map the concatenated feature vectors to the gating coefficients corresponding to each branch. During training, feedback information guides the gating weight matrix through gradient propagation. The learning process enables the adaptive gating network to dynamically adjust the contribution of each branch feature based on the common characteristics among multi-source data. Branches with high correlation to common features receive higher weight coefficients, and their output features dominate in subsequent fusion. Branches with low correlation to common features (such as branches containing source-specific noise) receive lower weight coefficients, and their output features are suppressed. Finally, the output features of each convolutional branch are multiplied element-wise with the corresponding gating weight coefficients to achieve weighted modulation of the features. Then, all weighted branch features are integrated through a concatenation operation to obtain a multi-source enhanced feature map. This feature map not only retains the rich information of multi-scale fault features but also highlights the common features of multi-source signals through weighted enhancement, effectively improving the discriminative power and robustness of the features.

[0040] S4. Input the multi-source enhanced feature map into the feature fusion network based on the cross-multi-head attention mechanism. By calculating the cross attention between different source features, the multi-source enhanced feature map is fused to generate multi-source fused features.

[0041] Specifically, this step utilizes a feature fusion network based on a cross-multi-head attention mechanism to fully explore the complex correlations between multi-source enhanced features, achieving deep fusion of multi-source features and generating multi-source fused features that comprehensively characterize the equipment fault state. The core advantage of the cross-multi-head attention mechanism lies in its ability to capture the dependencies between features across signal sources. Compared to traditional fusion methods such as splicing and weighted summation, it can more fully utilize the complementarity of multi-source information. First, the multi-source enhanced feature maps are preprocessed: the enhanced feature map (128×H / 16×W / 16) of each source is converted into a two-dimensional feature vector through a Reshape operation, with a dimension of ((H / 16)×(W / 16))×128, where (H / 16)×(W / 16) is the feature sequence length and 128 is the feature dimension. This conversion facilitates the attention mechanism in calculating the similarity between features. The conversion process can be represented as: , ,in and These represent the enhanced feature vectors of the two sources after the reshape operation. Subsequently, the two-dimensional feature vector of each source is linearly projected using three independent learnable weight matrices ( and The feature vectors are mapped to query vector Q, key vector K, and value vector V, respectively. The dimensions of Q, K, and V after projection are all ((H / 16)×(W / 16))×128, and the projection weight matrix of each source is initialized independently to ensure that it adapts to the distribution characteristics of each source feature. The mathematical expression of this process is as follows: , , ; , , ,in These represent the query vector, key vector, and value vector of the first source, respectively. These represent the query vector, key vector, and value vector of the second source, respectively.

[0042] To enhance the attention mechanism's ability to capture complex feature relationships, a multi-head attention structure is adopted. Q, K, and V are divided into eight parallel attention heads according to their feature dimensions. Each attention head has a feature dimension of 128 / 8 = 16. Multiple attention heads can independently capture the correlations between features from different representation subspaces. Cross-attention computation is the core of the fusion process: for the first source feature, its query vector is used... Key vector with the second source Performing a dot product operation yields the original attention weight matrix (dimension ((H / 16)×(W / 16))×((H / 16)×(W / 16))). Each element in this matrix represents the correlation strength between a position in the first source feature sequence and a position in the second source feature sequence. To avoid excessively large weight values ​​due to high feature dimensions, the original attention weight matrix is ​​divided by the square root of the ratio of feature dimension to head number (i.e., ...). The weight matrix is ​​scaled (where C represents the total dimension of the feature vector and h represents the number of attention heads); then, the scaled weight matrix is ​​normalized using the Softmax function so that the sum of the weights in each row is 1, resulting in the normalized first cross-attention weights; this weight matrix is ​​then compared with the value vector of the second source. Matrix multiplication is performed to obtain the first cross-attention feature (dimension ((H / 16)×(W / 16))×128), which incorporates key information related to the first source feature from the second source feature. Finally, the first cross-attention feature is combined with the original enhanced feature vector of the first source. By adding the residuals, the basic feature information of the first source is effectively preserved, while the complementary information of the second source is incorporated to obtain the intermediate fused features of the first source. The mathematical expression for this process is: .

[0043] The same logic is used to calculate the second source intermediate fusion feature. : Use the query vector from the second source Key vector with the first source Perform dot product, scaling, and normalization to obtain the second cross-attention weights; and compare them with the value vector of the first source. Multiplying them yields the second cross-attention feature; this is then combined with the original enhanced feature vector from the second source. The residuals are added together to obtain Its mathematical expression is: To further enhance the nonlinear expressive power of the fused features, layer normalization (LayerNorm) is applied to the intermediate fused features. Layer normalization normalizes the feature dimensions, reducing distribution differences between batches and accelerating gradient propagation. The normalized features are then input into an MLP (Multilayer Perceptron) for nonlinear transformation. The MLP contains two fully connected layers: the first layer has an output dimension of 256, and the second layer has an output dimension of 128, with a SiLU activation function in between. Finally, the output of the MLP is summed with the residuals of the original intermediate fused features to obtain the first source fused features. Second source fusion features Together, they constitute a multi-source fusion feature, the corresponding mathematical expression of which is: , ,in Presentation layer normalization operation, This represents a fully connected layer. This fusion feature uncovers the intrinsic correlations between multi-source enhancement features, integrates complementary information from different sources, and forms a unified feature representation that can comprehensively and accurately characterize the fault state of equipment.

[0044] S5. Perform global average pooling and global max pooling operations on the multi-source fusion features to obtain the pooling results; concatenate the pooling results to obtain the comprehensive diagnostic feature vector.

[0045] Specifically, this step uses two pooling strategies, Global Average Pooling (GAP) and Global Max Pooling (GMP), to extract key statistical information from multi-source fusion features. A comprehensive diagnostic feature vector is then constructed through concatenation, providing a concise and efficient feature input for subsequent fault diagnosis. The combined use of GAP and GMP complementarily captures feature information: GAP, by calculating the average value of each feature channel, reflects the overall distribution trend of features across the entire sequence, highlighting the global consistency of fault features; GMP, by calculating the maximum value of each feature channel, captures key peak information in the feature sequence, highlighting transient features such as fault impacts. The combination of both avoids information loss caused by a single pooling operation.

[0046] In the specific implementation process, the first source fusion feature in the multi-source fusion feature is first analyzed. (Perform global average pooling operation on dimension ((H / 16)×(W / 16))×128: ​​Take the arithmetic mean of the ((H / 16)×(W / 16)) feature values ​​for each feature channel (128 channels in total) to obtain the first source average pooled feature with dimension 1×128.) This process can be represented as: Simultaneously, a global max pooling operation is performed, taking the maximum value of ((H / 16)×(W / 16)) feature values ​​for each feature channel to obtain the first source max pooling feature with dimension 1×128. The corresponding expression is: The same operation was used to fuse the second source features. (The dimensions ((H / 16)×(W / 16))×128) are processed to obtain the second source average pooling feature with a dimension of 1×128. And the second source max pooling feature with dimension 1×128 Their expressions are as follows: , ,in and These represent global average pooling and global max pooling operations, respectively. Pooling operations not only extract key feature information but also significantly reduce the feature dimension from ((H / 16)×(W / 16))×128 to 1×128, which significantly reduces the computational cost of subsequent diagnostic networks and improves the inference efficiency of the model.

[0047] Then, a feature concatenation operation is performed, combining the first source average pooling features. First source max pooling feature Second source average pooling characteristics Second source max pooling characteristics The mathematical expression for the concatenation process along the feature dimension is as follows: ,in This indicates a splicing operation. The concatenated comprehensive diagnostic feature vector has a dimension of 1×(128+128+128+128)=1×512. This comprehensive diagnostic feature vector integrates the global distribution information and key peak information of multi-source fusion features. It retains the complementarity of multi-source information while achieving highly condensed features, providing accurate and efficient feature input for subsequent binary and multi-class diagnostic networks. This ensures that the diagnostic model can accurately identify fault states even with limited data.

[0048] S6. Input the comprehensive diagnostic feature vector into the binary diagnostic network and the multi-class diagnostic network. Dynamically weight the training loss of the multi-class diagnostic network through the output of the binary diagnostic network, and use the output of the multi-class diagnostic network as the mechanical fault diagnosis result.

[0049] Specifically, this step utilizes a dual-branch structure consisting of a binary diagnostic network and a multi-class diagnostic network, combined with a dynamic weighted loss strategy, to address the data imbalance problem while achieving accurate classification of equipment faults and outputting the final mechanical fault diagnosis result. The design concept of the dual-branch diagnostic network is to use the binary diagnostic network to capture global prior information of "normal / abnormal," guiding the multi-class diagnostic network to focus on fault category differentiation, effectively alleviating the problem of low accuracy in identifying minority-class faults under data imbalance. First, the comprehensive diagnostic feature vector (1×512) is simultaneously input into both the binary diagnostic network and the multi-class diagnostic network. Both networks employ a fully connected layer structure to ensure effective fitting of the high-dimensional comprehensive features. The structure of the binary diagnostic network is as follows: the input layer dimension is 512, followed by a fully connected layer (512×512), a BN layer, a SiLU activation function, a fully connected layer (512×128), a BN layer, and a SiLU activation function. Finally, a two-dimensional vector is output through a fully connected layer (512×2), which, after normalization by the Softmax function, yields a sample belonging to the normal operating condition (…). ) and fault conditions ( The probability distribution of the sample, which is the prior information of the sample's abnormal state, can reflect the confidence that the sample belongs to the fault state. Its output probability can be expressed as: ,in This represents the input comprehensive diagnostic feature vector. This indicates the operating condition category (0 for normal, 1 for fault). The loss function of the binary diagnostic network is defined as follows: ,in This represents the loss of the binary diagnostic network. This represents the number of training samples. and Representing the first The true probability and predicted probability of a sample belonging to normal or abnormal conditions.

[0050] The structure of the multi-class diagnostic network is consistent with that of the binary diagnostic network, except for the output layer dimension—the output layer dimension is the total number of fault categories C (e.g., C=4 corresponds to four fault types). After normalization by the Softmax function, the probability distribution of the sample belonging to each fault category is obtained, directly reflecting the fault category attribution of the sample. Its output probability can be expressed by the Softmax function as follows: ,in Represents the Lth layer of the network. Line number Column characteristics, Indicates the first Each sample in category The predicted probability is calculated based on the output of the binary diagnostic network. To address the data imbalance problem, the training loss of the multi-class diagnostic network is dynamically weighted by the output of the binary diagnostic network. This dynamic weighting strategy calculates dynamic weights by fusing the prior probability of the binary diagnostic network with the fault probability distribution of the multi-class diagnostic network. These weights are used to adjust the weight allocation of the multi-class diagnostic loss. The higher the fault condition probability (prior probability) output by the binary diagnostic network, the more likely the sample is to be in a fault state, and the larger the corresponding weight of the multi-class diagnostic loss is, prompting the model to pay more attention to the classification accuracy of fault samples during training. Conversely, the higher the normal condition probability, the smaller the weight of the multi-class diagnostic loss is, avoiding model bias caused by normal samples (with a high proportion) dominating the loss function.

[0051] During the model training phase, a total loss function is constructed to jointly optimize the entire network. The total loss function integrates the loss of the binary diagnostic network. Dynamically weighted multi-class diagnostic loss and the adversarial loss of the shared feature discriminator And through weighting factors and Balancing the contributions of each loss term, the expression for the total loss function is: ,in and These are hyperparameters, and their values ​​range from 0 to 1 (e.g., ...). , The training process employs the Adam optimizer with an initial learning rate of 1e-4. A learning rate decay strategy (decreasing to 0.9 every 10 epochs) is used to adjust the learning rate and prevent oscillations in the later stages of training. The batch size is set to 32, and the number of training epochs is 100. An early stopping strategy (training stops if the validation set accuracy shows no improvement for 10 consecutive epochs) is used to prevent overfitting. During backpropagation, the gradient of the total loss function is propagated to all trainable parameters of the primary feature encoder, shared feature discriminator, feature fusion network, binary diagnostic network, and multi-class diagnostic network via a chain rule, achieving collaborative optimization of each module and ensuring that the entire network architecture converges towards improving fault diagnosis accuracy. In the inference phase, dynamic weights are not calculated; the class with the highest probability in the probability distribution output by the multi-class diagnostic network is directly used as the final mechanical fault diagnosis result. This result fully utilizes the complementary information of multi-source fusion features and the prior guidance of the dual-branch network. In real-world engineering scenarios with limited and imbalanced data, it enables accurate and robust identification of mechanical faults, providing a reliable basis for equipment maintenance decisions.

[0052] After the final fault diagnosis results are output during the inference phase, in order to further improve the interpretability of the model's decision-making and help technicians understand the basis for the model's judgment on fault categories, the Grad-CAM++ interpretable module is introduced. By generating heatmaps, the key areas of concern in the model's decision-making process are visualized, and the feature locations corresponding to each type of fault are accurately located.

[0053] The specific implementation process of this module is as follows: First, clarify the calculation logic of the class prediction score in the deep convolutional neural network—the class confidence score output by the model. It is composed of the weighted sum of the feature maps of the last convolutional layer of the network, and its calculation formula is: ,in Representing the The spatial location of each convolutional feature map activation value, Is this feature map a category? The contribution weight of the feature map directly determines its importance in fault diagnosis.

[0054] To accurately capture the contribution of each location in the image to the class score, Grad-CAM++ employs a positive gradient weighting strategy, retaining only gradient information that has a positive impact on the class score. The weights are calculated as follows: ,in The weighting coefficients for each feature map location. The function ensures that only positive gradients participate in the calculation, avoiding interference from negative gradients in the class determination, thereby accurately measuring the contribution of each pixel position to the fault class and avoiding the loss of local information.

[0055] To further improve the accuracy of fault feature localization, Grad-CAM++ introduces higher-order derivatives (second and third derivatives) to capture the subtle influence of feature maps on class scores. The expression is updated to: By calculating the feature map With category score The first, second, and third derivatives between and Grad-CAM++ can capture the relationships between features more precisely, enabling more accurate feature localization in complex fault scenarios.

[0056] Finally, based on the aforementioned weighted gradient and higher-order derivative information, a category-specific heatmap is generated. The calculation formula is as follows: By multiplying the heatmap pixel-by-pixel with the original time-frequency image, the final visualization result is obtained. Areas with higher brightness in the heatmap represent a greater contribution to the fault category determination. Technicians can intuitively identify key fault feature areas upon which the model's decisions rely, verifying the rationality of the diagnostic results and providing a clear basis for fault cause analysis and equipment maintenance. With the introduction of the Grad-CAM++ module, this method significantly improves the interpretability and transparency of the model while ensuring the accuracy of fault diagnosis, meeting the reliability requirements of intelligent diagnostic technology in industrial scenarios.

[0057] The aforementioned deep learning-based mechanical fault diagnosis method first acquires runtime signals with various physical characteristics and performs continuous wavelet transform to generate multi-source time-frequency images containing rich time-frequency features, providing comprehensive input for subsequent deep analysis. Then, a primary feature encoder extracts primary deep features from each source signal, and innovatively introduces a shared feature discriminator. Through adversarial training, the encoder is forced to discard source-specific information, thereby extracting common features consistent across sources and related to the fault's essence. Using these common features as guidance, a gated multi-scale encoder adaptively strengthens components related to common fault modes in each source feature, resulting in enhanced multi-source feature representations. Subsequently, a feature fusion network built based on a cross-multi-head attention mechanism enables deep interactive querying and information aggregation between features from different sources, achieving... This approach achieves deep fusion from simple feature coexistence to the level of intrinsic semantic association. Global average pooling and global max pooling are simultaneously applied to the fused features to capture the overall distribution and most salient local patterns, respectively. These two are then concatenated to form a comprehensive diagnostic feature vector, thus balancing the global representativeness and local discriminativeness of the features. Finally, a dual-branch diagnostic network is designed, utilizing the "abnormal" prior probability provided by the binary diagnostic network to dynamically adjust the training loss weights of the multi-class diagnostic network. This mechanism effectively curbs model bias caused by the dominance of normal samples, forcing the model to give equal importance to scarce fault samples. Ultimately, under the realistic condition of extremely imbalanced data distribution, high-precision and robust identification and classification of multiple types of faults are achieved. Furthermore, the end-to-end design of the entire process ensures the synergistic optimization of feature learning and fault diagnosis, improving the practicality and generalization ability of the method.

[0058] In one optional embodiment, the multi-source primary feature maps are input into a shared feature discriminator, common features are extracted from the multi-source primary feature maps through adversarial training, and the common features are used as feedback information, including the following steps:

[0059] S11. By using a shared feature discriminator, source classification and discrimination are performed on the multi-source primary feature maps, and the multi-source discrimination results are output.

[0060] Specifically, the core function of the shared feature discriminator is to distinguish the signal sources of the input multi-source primary feature maps. The input consists of primary feature maps from different signal sources (such as vibration signals and current signals), with a uniform feature map dimension of 128×H / 16×W / 16 (H and W are the original height and width of the time-frequency image). The discriminator's network structure is specifically designed to adapt to the task of source discrimination of multi-source features: First, a linear layer flattens the three-dimensional feature map into a one-dimensional vector, ensuring that the features can be effectively processed by the fully connected layer; then, a gradient inversion layer (GRL) is applied, followed by a fully connected layer (output channel 128), a batch normalization (BN) layer, a dropout layer (dropout probability set to 0.3 to suppress overfitting), another fully connected layer (output channel 64), a BN layer, and a dropout layer (dropout probability 0.3). Finally, a fully connected layer outputs a two-dimensional vector, which corresponds to the classification probability distribution of the two signal sources. During the classification process, the shared feature discriminator learns the specific features related to the signal source in the primary feature map of the multi-source system, judges the source of each input feature map, and outputs the multi-source discrimination result, that is, the probability value of each feature map belonging to each signal source. For example, the output vector [0.92, 0.08] indicates that the probability of the feature map coming from the first signal source is 92% and the probability of coming from the second signal source is 8%.

[0061] S12. Based on the multi-source discrimination results, the gradient flowing to the primary feature encoder is reversed using the gradient inversion layer to obtain the backpropagation gradient signal.

[0062] Specifically, this step, based on the multi-source discrimination results output by S11, combines the gradient inversion layer to invert the gradient flowing towards the primary feature encoder, thus realizing the core logic of adversarial training. The unique feature of the gradient inversion layer is that it does not change the feature data during forward propagation, only playing a role in calculating the gradient during backpropagation: multiplying the gradient originally flowing towards the primary feature encoder by a fixed negative coefficient (usually set to -1), completely inverting the gradient direction. The essence of this design is to construct an adversarial relationship between the primary feature encoder and the shared feature discriminator: in forward propagation, the shared feature discriminator outputs discrimination results based on the specific features of the feature map; in backpropagation, the gradient inversion layer changes the gradient flow direction, making the training objective of the primary feature encoder opposite to that of the shared feature discriminator. The primary feature encoder no longer attempts to optimize "making the discriminator accurately distinguish the source," but instead adjusts its parameters in the direction of "making it difficult for the discriminator to distinguish the source." Through this gradient inversion processing, the backpropagation gradient signal is finally obtained, which carries the gradient information of the discriminator's discrimination of the feature source, providing a crucial basis for subsequent parameter updates and common feature extraction of the primary feature encoder.

[0063] S13. Based on the multi-source discrimination results and the true source labels corresponding to the multi-source primary feature maps, calculate the adversarial training loss value; the formula for calculating the adversarial training loss value is:

[0064]

[0065] in, This represents the adversarial training loss value. and These represent two primary feature encoders that process the first and second source signals, respectively. Indicates a shared feature discriminator, This indicates that the shared feature discriminator corresponds to the first source input. Classification branches, This indicates that the shared feature discriminator corresponds to the second source input. Classification branches, and These represent the two input multi-source primary feature maps, This represents a logarithmic function with the natural constant as its base. This represents the mathematical expectation operator.

[0066] Specifically, based on the multi-source discrimination results output by S11 and the true source labels corresponding to the multi-source primary feature maps, the adversarial training loss value is calculated using a specific loss function. This loss function is the mathematical core of the adversarial game, and its calculation formula is as follows: The specific meanings of each parameter are as follows: This represents the adversarial training loss value, which is a core indicator for measuring the effectiveness of adversarial games. and These represent two primary feature encoders that process the first source signal and the second source signal, respectively. They have identical structures and are responsible for extracting the primary features of the corresponding signal sources. This represents a shared feature discriminator, which consists of a linear layer, a batch normalization (BN) layer, a dropout layer, and a fully connected layer. and These represent the features in the shared feature discriminator corresponding to the first source input. Second source input The classification branches each focus on classifying and discriminating features from a single source; and These represent the two multi-source primary feature maps that are input, and are the core processing objects in adversarial training. This represents a logarithmic function with the natural constant as its base, used to convert classification probabilities into loss values; This represents the mathematical expectation operator, used to calculate the average value of the loss value in a batch of samples, ensuring the stability of the loss calculation.

[0067] This loss function constructs a "minimal-maximum" adversarial game relationship: and The optimization objective is to minimize That is, by adjusting its own parameters, it generates a discriminator that shares features. The inability to distinguish the source of the feature map forces it to abandon the extraction of source-specific features and instead focus on fault-related features common to multiple source signals; while the shared feature discriminator The optimization goal is to maximize That is, by optimizing its own parameters, it accurately identifies the signal source of the input feature map and distinguishes it as much as possible. and The specific differences. This adversarial game continues until both sides reach a Nash equilibrium, at which point... and The extracted features have eliminated source-specific information to the greatest extent possible, while retaining common features of multi-source signals.

[0068] S14. The parameters of the primary feature encoder are updated using the backpropagation gradient signal, which drives the primary feature encoder to generate features that increase the error rate of the shared feature discriminator, and the backpropagation gradient signal is used as feedback information to characterize the common features.

[0069] Specifically, this step utilizes the backpropagation gradient signal obtained in S12, and uses the backpropagation algorithm to process the two primary feature encoders. and The trainable parameters are updated. The parameter update process uses the Adam optimizer, with an initial learning rate set to 1e-4. Gradient descent is used to gradually adjust the weights and biases of components such as convolutional layers, batch normalization (BN) layers, and the C3Block module in the encoder. The core goal of the update is to drive the primary feature encoder to generate features that increase the error rate of the shared feature discriminator; that is, through continuous optimization, to... Output and Output As the feature distribution gradually converges, the shared feature discriminator is unable to distinguish the signal source by feature differences, thus forcing the primary feature encoder to actively discard specific features related to the signal source and enhance the ability to extract common features (such as common patterns such as energy anomalies and frequency shifts caused by faults) from multi-source signals.

[0070] During parameter updates, the backpropagation gradient signal is not only used to adjust the parameters of the primary feature encoder, but also serves as feedback information representing common features. This feedback information contains the distribution characteristics of common information among multi-source features, specifically including key information such as the dimensionality importance of common features, the range of feature value distribution, and the similarity between multi-source features. This information is directly input into the subsequent gated multi-scale encoder, providing precise guidance for the weighted enhancement of common features in the secondary feature encoding process. This ensures that the secondary feature encoder can specifically strengthen common features, suppress redundant information, and further improve the discriminative power and robustness of the features.

[0071] The entire process effectively aligned the distribution differences of multi-source features through adversarial training, extracting common features that do not change with the signal source and are only related to the fault state. This laid a solid foundation for the subsequent deep fusion and accurate diagnosis of multi-source features. At the same time, through the gradient signal feedback mechanism, a closed-loop linkage between feature extraction and feature optimization was achieved.

[0072] In one optional embodiment, based on feedback information, a gated multi-scale encoder is used to perform secondary feature encoding and common feature weighting enhancement on the multi-source primary feature map to obtain a multi-source enhanced feature map, including the following steps:

[0073] S21. Input the multi-source primary feature maps into the multi-scale feature extraction module composed of multiple parallel convolutional branches. Each convolutional branch uses a convolutional kernel of a different size to extract features under different receptive fields, thereby obtaining multiple sets of multi-scale features.

[0074] Specifically, the multi-scale feature extraction module is a core component of the gated multi-scale encoder. Its design goal is to capture multi-scale fault features from multi-source primary feature maps under different receptive fields, providing a rich feature foundation for subsequent weighted enhancement of common features. This module consists of multiple parallel convolutional branches with consistent structures but different kernel sizes. Each convolutional branch independently processes the input multi-source primary feature map (dimensions of 128×H / 16×W / 16, where H and W are the original height and width of the time-frequency image), ensuring that feature information can be captured comprehensively from local to global and from details to the whole.

[0075] Specifically, the number of parallel convolutional branches is set to 4, with kernel sizes of 1×1, 3×3, 5×5, and 7×7 for each branch. Different kernel sizes correspond to different receptive fields. The 1×1 kernel has the smallest receptive field and is mainly used to capture local detail features in the feature map, such as small high-energy regions corresponding to fault impacts in time-frequency images. It can also perform channel-dimensional feature fusion and dimensionality reduction without changing the feature space structure. The 3×3 and 5×5 kernels have medium receptive fields and are used to capture medium-scale feature combinations (such as continuous distribution areas of fault features and the correlation between adjacent high-energy regions) and feature patterns over a wider range (such as local frequency band shifts caused by faults), respectively. The 7×7 kernel has the largest receptive field and is mainly used to capture global structural features, such as the global energy distribution trend related to faults in the entire time-frequency image and cross-regional feature correlations.

[0076] The internal structure of each convolutional branch is uniformly designed as "convolutional layer + batch normalization (BN) layer + SiLU activation function". The convolutional layer employs padding to ensure consistent spatial dimensions (H / 16×W / 16) between the input and output feature maps, preventing feature loss. The output channel number of each branch's convolutional layer is set to 128 to ensure dimensionality matching during subsequent feature concatenation. The BN layer normalizes the convolutional output features, accelerating network training convergence, suppressing overfitting, and improving the stability of feature extraction. The SiLU activation function is expressed as SiLU(x) = x·σ(x) (σ is the Sigmoid function), which, compared to the traditional ReLU activation function, effectively avoids the gradient vanishing problem and enhances the non-linear expressive power of features. Through the synergistic effect of the parallel convolutional branches, the multi-scale feature extraction module processes the primary feature map of each source and outputs four sets of multi-scale features, each with dimensions of 128×H / 16×W / 16. Each set of features corresponds to a fault feature representation within a receptive field, providing a diverse feature base for the subsequent weight allocation of the adaptive gating network.

[0077] S22. Input the concatenated multi-scale features into an adaptive gating network. Using the adaptive gating network and feedback information, calculate the gating weight coefficients corresponding to each convolutional branch. The formula for calculating the gating weight coefficients is as follows:

[0078]

[0079] in, Indicates the first The gating weights of each convolutional branch, This represents the Sigmoid activation function. This represents a trainable gated weight matrix. This indicates a splicing operation. They represent the 1st to the 1st. Multi-scale features output by each convolutional branch; feedback information is used to guide the gating weight matrix during training. The learning process enables adaptive gating networks to dynamically adjust the contribution of each branch feature based on the common characteristics among multiple data sources.

[0080] Specifically, the core function of the adaptive gating network is to dynamically calculate the gating weight coefficients of each convolutional branch by combining the feedback information from the shared feature discriminator, thereby achieving selective enhancement of multi-scale features. The calculation formula is as follows: , where n is the number of parallel convolutional branches (n=4 in this embodiment). The specific meanings and functions of each parameter in this formula are as follows: This represents the gating weight coefficient corresponding to the i-th convolutional branch, whose value range is determined by the Sigmoid activation function. The weight coefficient is limited to between 0 and 1. The closer the weight coefficient is to 1, the stronger the correlation between the multi-scale features and common features of the corresponding branch, and it should be given a higher weight in subsequent fusion. Conversely, the closer the weight coefficient is to 0, the more redundant or noisy information the branch features contain, and it should be suppressed. This represents a trainable gated weight matrix, designed with dimensions of (128×n)×n (where 128 is the number of channels in a single branch's output feature, and n is the number of branches), ensuring that the concatenated multi-scale features can be mapped to n gated weight coefficients. The initial values ​​are set using the Xavier initialization strategy to ensure stable gradient propagation during training. This indicates a channel-dimensional concatenation operation on the multi-scale features output from n parallel convolutional branches. The concatenated feature dimension is (128×n)×H / 16×W / 16. This operation integrates feature information from all scales, providing a comprehensive feature basis for calculating the gating weight coefficients. These represent the multi-scale features output by the 1st to nth convolutional branches (each) The dimensions are all 128×H / 16×W / 16.

[0081] Feedback information plays a guiding role in this process. It consists of gradient signals related to common features extracted by the shared feature discriminator through adversarial training, containing the distribution patterns and key feature patterns of common features from multi-source data. During model training, this feedback information is passed to the adaptive gating network through the gradient backpropagation mechanism, guiding the process. The parameters are updated. When the multi-scale features of a branch highly match the common features in the feedback information, the gradient signal will prompt... The weight coefficient of the corresponding branch increases; conversely, when the branch characteristics have low correlation with common characteristics or contain specific noise, the gradient signal will promote... The weight coefficients of the corresponding branches are reduced. Through this dynamic learning mechanism, the adaptive gating network can accurately identify multi-scale features that are effective for fault diagnosis, achieve targeted reinforcement of common features, and suppress interference from redundant information and noise.

[0082] The specific structural design of the adaptive gating network is as follows: First, the concatenated multi-scale features are converted into a one-dimensional vector (dimension 128×n) through global average pooling. This operation can extract global statistical information of the features and reduce the interference of spatial dimension on weight calculation. Then, the one-dimensional vector is input into two fully connected layers (the first layer outputs a dimension of 256, and the second layer outputs a dimension of n), and a SiLU activation function is inserted in between to enhance the nonlinear expressive power. Finally, n gating weight coefficients are output through the Sigmoid activation function, ensuring that the value of each coefficient is between 0 and 1, which meets the requirements of weighted fusion.

[0083] S23. Based on the gating weight coefficients, the corresponding multi-scale features are weighted and fused to obtain a multi-source enhanced feature map.

[0084] Specifically, the weighted fusion process is as follows: for the multi-scale features output by each convolutional branch... (Dimensions 128×H / 16×W / 16), and compare them with the corresponding gating weight coefficients. Perform element-wise multiplication, that is, multiply the pixel value at each location in the feature map by... This enables weighted modulation of the characteristics of the branch. Because... The value range is from 0 to 1, when When the value is close to 1, the characteristics of the corresponding branch are fully preserved or even enhanced, and the common features it contains can play a full role; when When the value approaches 0, the features of the corresponding branch are significantly suppressed, and the impact of redundant information and noise contained therein on subsequent diagnosis is reduced. For example, if the value of a certain branch (such as a 3×3 convolution kernel branch) is close to 0, the features of the corresponding branch are significantly suppressed, and the impact of redundant information and noise on subsequent diagnosis is reduced. This indicates that the medium-scale features captured by this branch are highly correlated with common features, and the contribution of this feature is significantly increased after weighting; if another branch (such as the 7×7 convolution kernel branch) This indicates that the global features captured by this branch contain a lot of redundant information, and its influence is greatly weakened after weighting.

[0085] After weighted modulation of all branch features, a channel-dimensional concatenation operation is performed on all weighted feature maps. This involves concatenating n weighted feature maps with dimensions of 128×H / 16×W / 16 into a single fused feature map with dimensions of (128×n)×H / 16×W / 16. In this embodiment, n=4, therefore the fused feature map has a dimension of 512×H / 16×W / 16. This feature map integrates the multi-scale features after weighted modulation, preserving fault feature information under different receptive fields while highlighting key information related to common features through gating weight coefficients, effectively suppressing redundancy and noise interference.

[0086] Finally, the concatenated fused feature map is input into a 1×1 convolutional layer for channel-level fusion and dimensionality reduction. The output channel number of this 1×1 convolutional layer is set to 128 to ensure that the dimensionality of the output feature map is consistent with that of the multi-source primary feature map (128×H / 16×W / 16), facilitating subsequent processing by the feature fusion network. This 1×1 convolutional layer is also equipped with a Batch Normalization (BN) layer and a SiLU activation function to further enhance the non-linear expressive power and training stability of the features. After the above series of operations, a multi-source enhanced feature map is finally obtained. This feature map has the characteristics of multi-scale feature complementarity, prominent common features, and low noise interference, which can provide high-quality feature input for subsequent feature fusion based on the cross-multi-head attention mechanism, significantly improving the fault diagnosis performance of the model under limited and imbalanced data.

[0087] refer to Figure 2 In one optional embodiment, the multi-source enhanced feature map is input into a feature fusion network based on a cross-multi-head attention mechanism. By calculating the cross-attention between features from different sources, the multi-source enhanced feature map is fused to generate a multi-source fused feature, including the following steps:

[0088] S31. Perform linear projection processing on the multi-source enhanced feature map of each source to generate the query vector, key vector and value vector corresponding to each source.

[0089] Specifically, the multi-source augmented feature map for each source is first preprocessed and linearly projected to generate a query vector (Q), key vector (K), and value vector (V) for each source. This is the foundation for the feature interaction achieved by the cross-multi-head attention mechanism. The dimension of the multi-source augmented feature map is 128×H / 16×W / 16 (H and W are the original height and width of the time-frequency image). To adapt to the sequence-type input format of the attention mechanism, a reshape operation is first performed to convert the three-dimensional feature map into a two-dimensional feature vector. The dimension of the converted vector is ((H / 16)×(W / 16))×128, where (H / 16)×(W / 16) is the feature sequence length (denoted as L), and 128 is the feature dimension (denoted as C). This conversion process can be represented as: , ,in and These are multi-source enhanced feature maps of the first and second sources, respectively. and This is the transformed two-dimensional feature vector.

[0090] Subsequently, the two-dimensional feature vector of each source is subjected to independent linear projection processing, i.e., through three learnable weight matrices ( Corresponding to the first source, Corresponding to the second source), the feature vectors are mapped to Q, K, and V vectors respectively. All weight matrices are initialized using the Xavier initialization strategy to ensure stable gradient propagation during training and avoid gradient vanishing or exploding. The mathematical expression of the projection process is: , , ; , , .in, These represent the query vector, key vector, and value vector of the first source, respectively. Let H, W, and C represent the query vector, key vector, and value vector of the second source, respectively. All projected vectors maintain a dimension of L×C (i.e., ((H / 16)×(W / 16))×128) to ensure dimensionality matching for subsequent attention calculations. The projection weight matrix for each source is trained independently, which can adapt to the distribution characteristics of each source feature, accurately capture the semantic information of different source features, and provide high-quality input for cross-attention calculations.

[0091] S32. Based on the query vector of the first source and the key vector of the second source, the first cross-attention weight is calculated by scaling the dot product; the first cross-attention weight is used to perform a weighted summation on the value vector of the second source to obtain the first cross-attention feature.

[0092] Specifically, based on the query vector from the first source Key vector with the second source The first cross-attention weights are obtained by scaling the dot product, and then the value vector of the second source is weighted accordingly. The first cross-attention feature is obtained, and the second source feature complements and enhances the first source feature. The core logic of cross-attention is to calculate... and The similarity is used to determine the information in the second source feature that is valuable to the first source feature, and then integrate it into the first source feature.

[0093] The specific calculation process is as follows: First, for and Performing a dot product operation yields the original attention weight matrix, which has dimensions L×L. Each element in the matrix... This represents the correlation strength between the i-th position in the first source feature sequence and the j-th position in the second source feature sequence; a larger value indicates a stronger semantic association between the two. Since the numerical range of the dot product result expands with a larger feature dimension C, it may cause the output of the Softmax function to tend towards extremes (close to 0 or 1), affecting gradient propagation. Therefore, the original attention weight matrix needs to be scaled, with a scaling factor of [value missing]. Where h is the number of attention heads (h=8 in this implementation), this scaling operation can normalize the weight values ​​to a reasonable range, improving the stability of the attention mechanism. The scaled attention weight matrix is ​​normalized by the Softmax function so that the sum of the weights in each row is 1, resulting in the first cross-attention weight matrix. This matrix quantifies the contribution ratio of each feature at the second source to the corresponding feature at the first source.

[0094] Finally, the first cross-attention weight matrix and the value vector of the second source are combined. Matrix multiplication is performed to obtain the first cross-attention feature, which still has the dimension of L×C. This feature integrates complementary information from the second source that is highly correlated with the first source, thus achieving effective interaction between cross-source features.

[0095] S33. Add the residuals of the first cross-attention feature and the multi-source enhancement feature map of the first source to obtain the first source intermediate fusion feature; the expression of the first source intermediate fusion feature is:

[0096]

[0097] in, This indicates the first source intermediate fusion feature. Represents the normalized exponential function, This represents the query vector from the first source. This represents the transpose of the key vector of the second source. Represents the value vector of the second source. This represents the total dimension of the feature vector. Indicates the number of attention heads. This represents the multi-source enhanced feature map of the first source.

[0098] Specifically, the meanings of each parameter in this formula are as follows: The first source intermediate fusion feature has a dimension of L×C; This is a normalization exponential function used to normalize the scaled attention weight matrix; The query vector for the first source. The second source key vector transpose, Scaling factor These four elements together constitute the computational part of the first cross-attention feature, which is the value vector of the second source. This is the multi-source enhanced feature vector after reshaping the first source, representing the original feature information of the first source.

[0099] The core function of residual addition is to preserve the basic feature information of the first source, avoiding the loss of its own key fault features during the fusion process. Cross-attention features provide complementary information from the second source. It provides the core features of the first source, and the combination of the two makes It incorporates complementary cross-source information while retaining its own inherent characteristics, significantly improving feature richness and robustness. Simultaneously, the residual connection structure alleviates the vanishing gradient problem in deep networks, accelerates model training convergence, and ensures the stability of the feature fusion process.

[0100] S34. Based on the query vector of the second source and the key vector of the first source, the second cross-attention weight is calculated by scaling the dot product; the second cross-attention weight is used to perform a weighted summation on the value vector of the first source to obtain the second cross-attention feature.

[0101] Specifically, this step is logically symmetrical to S32, aiming to achieve complementary enhancement of the second source features by the first source features. Based on the query vector from the second source... Key vector with the first source The second cross-attention weights are obtained by scaling the dot product. First, the... and Perform a dot product operation to obtain the original attention weight matrix (dimension L×L); then use the same scaling factor. The values ​​are scaled to avoid training instability caused by excessively large numerical ranges; then normalized using the Softmax function to obtain the second cross-attention weight matrix, which quantifies the contribution ratio of each feature of the first source to the corresponding feature of the second source.

[0102] Compare the second cross-attention weight matrix with the value vector of the first source. Matrix multiplication is performed to obtain the second cross-attention feature (dimension L×C). This feature integrates complementary information from the first source that is highly correlated with the second source, thus completing the cross-source feature interaction in the second direction.

[0103] S35. Add the residuals of the second cross-attention feature and the multi-source enhancement feature map of the second source to obtain the second source intermediate fusion feature; the expression of the second source intermediate fusion feature is:

[0104]

[0105] in, This indicates the intermediate fusion feature of the second source. This represents the query vector from the second source. This represents the transpose of the key vector of the first source. Represents the value vector of the first source. This represents the multi-source enhanced feature map of the second source.

[0106] Specifically, The second source intermediate fusion feature has a dimension of L×C; This is the query vector for the second source. The first source key vector transpose, The first source's value vector, along with the scaling factor, constitutes the computational part of the second cross-attention feature; This is the multi-source enhanced feature vector after reshaping the second source, representing the original feature information of the second source. The residuals are summed... It also possesses both the core features of the second source and the complementary features of the first source, thus achieving cross-source enhancement of the second source features.

[0107] S36. Perform layer normalization and nonlinear transformation on the first source intermediate fusion feature and the second source intermediate fusion feature to obtain the first source fusion feature and the second source fusion feature respectively; construct multi-source fusion features based on the first source fusion feature and the second source fusion feature; the expressions for the first source fusion feature and the second source fusion feature are:

[0108]

[0109]

[0110] in, Indicates the first source fusion feature, Indicates the second source fusion feature; The representation layer is normalized, and MLP stands for fully connected layer.

[0111] Specifically, layer normalization is first performed, which normalizes the feature dimension (C dimension). For each position in the feature sequence, the mean and variance of that position are calculated across all batches of samples. Then, the feature values ​​are standardized to a distribution with a mean of 0 and a variance of 1. The calculation formula is as follows: , where μ is the mean, σ² is the variance, and ε is the minimum value (set to 1e-5) to avoid a denominator of 0. Layer normalization can reduce the difference in feature distribution between batches, reduce the heterogeneity of features of different samples, accelerate gradient propagation, and improve the stability and convergence speed of model training.

[0112] The normalized features are then input into an MLP (Multilayer Perceptron) for nonlinear transformation. The MLP is designed with two fully connected layers: the first fully connected layer has an input dimension of C(128) and an output dimension of 256, and introduces nonlinear expressive power through the SiLU activation function; the second fully connected layer has an input dimension of 256 and an output dimension of C(128), ensuring that the output feature dimension is consistent with the input. The core function of the nonlinear transformation is to uncover the complex nonlinear relationships in the fused features, enhance the discriminative ability of the features, and make the features more suitable for subsequent fault diagnosis tasks.

[0113] Finally, the output of the MLP is added to the original intermediate fused features using residual summation to obtain the final source fused features. The corresponding mathematical expression is: , .in, Indicates the first source fusion feature, The second source fusion feature is represented by the symbol L×C ((H / 16)×(W / 16)×128). This residual connection further preserves the effective information of the intermediate fusion features, avoiding information loss during the nonlinear transformation process.

[0114] First source fusion features Second source fusion features By concatenating features along their dimensional axes, a multi-source fusion feature is obtained, with dimensions L×(2C) ((H / 16)×(W / 16)×256). This multi-source fusion feature achieves deep bidirectional interaction between multi-source features through a cross-multi-head attention mechanism, integrating core features from each source with complementary cross-source features. Furthermore, layer normalization and nonlinear transformations enhance the discriminative power and robustness of the features, providing high-quality feature input for subsequent pooling operations and diagnostic networks, significantly improving the model's fault diagnosis accuracy under imbalanced data.

[0115] In one optional embodiment, the comprehensive diagnostic feature vector is input into a binary diagnostic network and a multi-class diagnostic network. The training loss of the multi-class diagnostic network is dynamically weighted by the output of the binary diagnostic network, and the output of the multi-class diagnostic network is used as the mechanical fault diagnosis result. This includes the following steps:

[0116] S41. Input the comprehensive diagnostic feature vector into the binary diagnostic network to obtain the prior probability representing that the sample belongs to an abnormal state. .

[0117] Specifically, the comprehensive diagnostic feature vector (dimension 1×512) obtained from S5 is input into the binary diagnostic network. The core function of this network is to determine whether the sample belongs to normal or abnormal operating conditions, and the output represents the prior probability that the sample belongs to an abnormal state. The binary diagnostic network is designed with a fully connected layer architecture, suitable for classification tasks with high-dimensional comprehensive features. The input layer dimension is 512, perfectly matching the dimension of the comprehensive diagnostic feature vector. Subsequently, a fully connected layer (512×512), a batch normalization (BN) layer, and a SiLU activation function are sequentially connected; then a fully connected layer (512×128), a BN layer, and a SiLU activation function are connected; finally, a fully connected layer (512×2) outputs a two-dimensional vector. This vector is normalized using the Softmax function to determine if the sample belongs to a normal working condition. ) and fault conditions ( The probability distribution of ). Among them, the corresponding fault conditions ( The probability value of ) is the prior probability. Its value ranges from 0 to 1. The closer the probability is to 1, the higher the confidence that the sample belongs to an abnormal state; the closer it is to 0, the higher the confidence that the sample belongs to a normal state. This prior probability provides a crucial state basis for the subsequent loss weighting of multi-class diagnostic networks, guiding the model to focus on high-confidence fault samples.

[0118] S42. Input the comprehensive diagnostic feature vector into the multi-class diagnostic network to obtain the predicted probability distribution of each fault category. From the predicted probability distribution Extract the probability components corresponding to all fault categories. .

[0119] Specifically, the comprehensive diagnostic feature vector is synchronously input into a multi-class diagnostic network. The core function of this network is to identify the specific fault type of abnormal samples and output the predicted probability distribution belonging to each fault category. And extract the probability components corresponding to all fault categories from them. The structure of the multi-class diagnostic network is consistent with that of the binary diagnostic network, except for the output layer dimension: the output layer dimension is set to the total number of fault categories. (For example Corresponding to four fault types, (Corresponding to six fault types), ensuring coverage of all fault categories to be identified. After processing by the fully connected layer mapping, BN layer normalization, and SiLU activation function, the output is... The predicted probability distribution is obtained by normalizing a 3D vector using the Softmax function. , Each element in the matrix corresponds to the probability that a sample belongs to a certain category (including the normal category), and the sum of all elements is 1. Since the binary diagnostic network has completed the initial "normal / abnormal" judgment, the core focus of the multi-class diagnostic network is the fault type of the abnormal samples. Therefore, from... After removing the probability elements corresponding to the normal category, extract the probability components corresponding to all remaining fault categories to form... , The dimension is 1× The sum of its elements is still 1, providing a probabilistic basis for subsequent cosine similarity calculations at the fault category level.

[0120] S43. Calculate the prior probability With probability part The cosine similarity between them is calculated and mapped to dynamic weights. Dynamic weights The calculation formula is:

[0121]

[0122] in, Indicates dynamic weights. This represents the function for calculating cosine similarity. Represents the natural constant.

[0123] Specifically, calculate the prior probability. With probability part The cosine similarity between them is calculated, and this similarity is converted into dynamic weights through a specific mapping function. This enables loss weighting adjustment based on sample anomaly confidence. The core function of cosine similarity is to measure... and semantic relevance, Reflects the overall anomaly confidence level of the sample. This reflects the probability distribution of samples across different fault categories. A higher correlation between the two indicates a better match between the sample's abnormal state and the fault category distribution, thus warranting higher training weights. Cosine similarity is calculated using the vector similarity formula. (Scalar) expansion to AND After finding vectors with the same dimension, calculate the cosine of the angle between them, with a value ranging from -1 to 1. The closer the value is to 1, the stronger the correlation; the closer it is to -1, the weaker the correlation.

[0124] The formula for calculating dynamic weights is: The specific meanings of each parameter are as follows: For dynamic weights, This is a cosine similarity calculation function, with an output value ranging from -1 to 1; The natural constant is approximately 2.718; the negative sign in the exponent part results in a higher cosine similarity, a smaller exponent value, and a smaller denominator, ultimately... The larger the value, the better. This formula is essentially a sigmoid-like mapping, mapping cosine similarity (from -1 to 1) to dynamic weights between 0 and 2: when the sample has high anomaly confidence and a high degree of matching with the fault category distribution (cosine similarity close to 1). A value close to 2 has the strongest weighting effect on the loss of multiple diagnostic categories; however, when the sample anomaly confidence is low or the match with the fault category distribution is low (cosine similarity close to -1), the effect is less pronounced. A weight close to 0 has the weakest weighting effect on the multi-class diagnostic loss. Through this dynamic weight allocation, the model can automatically focus on high-confidence fault samples during training, effectively alleviating the problem of normal samples dominating training due to data imbalance.

[0125] S44. Use dynamic weights to weight the cross-entropy loss of the multi-class diagnostic network to obtain the weighted multi-class loss; the formula for calculating the weighted multi-class loss is:

[0126]

[0127] in, Indicates weighted average loss. Represents the set of indices for the training samples. This indicates the total number of fault categories. Index representing the fault category, Indicates the first The dynamic weights corresponding to each sample Indicates the first Each sample in category The real labels on it The output of the multi-class diagnostic network represents the first... Each sample in category The predicted probability, This represents the natural logarithm function.

[0128] Specifically, the dynamic weights obtained using S43 (No. The dynamic weights corresponding to each sample are used to weight the cross-entropy loss of the multi-class diagnostic network, resulting in a weighted multi-class loss. The calculation formula is as follows: The specific meanings and functions of each parameter in this formula are as follows: Weighted multi-class loss is the core indicator for evaluating the classification performance of multi-class diagnostic networks; This represents the set of indices for the training samples, covering all samples that participated in the training. This indicates the total number of fault categories. Index representing the fault category (from 1 to ...) ), corresponding to different fault types; For the first The dynamic weights corresponding to each sample are calculated by S43 and are used to adjust the contribution of that sample in the loss function. Indicates the first Each sample in category The true label on the sample is encoded using one-hot encoding (e.g., when the sample belongs to type 2 fault). The remaining categories ); The output of the multi-class diagnostic network represents the first... Each sample in category The predicted probability; This represents the natural logarithm function, used to convert predicted probabilities into loss values. The closer the predicted probabilities are to the true labels, the smaller the loss value.

[0129] The core design goal of weighted multi-class loss is to address the data imbalance problem: in real-world engineering data, the proportion of normal samples is usually much higher than that of faulty samples. The loss of the minority of faulty samples is easily masked by the loss of the large number of normal samples, causing the model to be biased towards normal samples. This is addressed through dynamic weighting. The weighted loss of high-confidence fault samples is amplified. During training, the model will prioritize optimizing the classification error of such samples, thereby improving the recognition accuracy of minority class faults and avoiding normal samples from excessively dominating the loss function.

[0130] S45. Combining the loss of the binary diagnostic network, the weighted multi-class loss, and the adversarial loss of the shared feature discriminator, a total loss function is constructed. This total loss function is then used to jointly train the primary feature encoder, the shared feature discriminator, the feature fusion network, the binary diagnostic network, and the multi-class diagnostic network. The formula for calculating the loss of the binary diagnostic network is as follows:

[0131]

[0132] in, This represents the loss of the binary diagnostic network. This represents normal operating conditions. Represents a fault condition. This represents the number of training samples. and Representing the first The true probability of a sample belonging to normal or abnormal conditions.

[0133] Specifically, the loss in the binary diagnostic network The weighted multi-class loss obtained from S44 and the adversarial loss of the shared feature discriminator We construct a total loss function and use it to jointly train all trainable modules of the entire model, ensuring that each module is optimized in a coordinated manner and improving the overall diagnostic performance.

[0134] Loss of binary diagnostic networks The calculation formula is: The meanings of each parameter are as follows: Represents the number of training samples; This represents normal operating conditions. This represents a fault condition; and Representing the first The true probability and predicted probability of a sample belonging to normal or abnormal conditions; This represents a logarithmic function with the natural constant as the base. This loss function measures the classification error of the binary diagnostic network for "normal / abnormal," ensuring that the model can accurately distinguish the overall state of the samples.

[0135] The expression for the total loss function is: ,in and These are hyperparameters, all ranging from 0 to 1, used to balance the contribution weights of the three loss terms. Adjusting the ratio of binary diagnostic loss to weighted multi-class loss, Adjusting the degree of impact of counter-loss (e.g.) , This ensures that the total loss can simultaneously optimize the three core tasks of state judgment, fault classification, and multi-source feature alignment.

[0136] The joint training is implemented as follows: The Adam optimizer is selected as the parameter update tool, with an initial learning rate set to 1e-4. A learning rate decay strategy is adopted (decreasing to 0.9 every 10 training epochs) to avoid gradient oscillations in the later stages of training. The batch size is set to 32 to ensure training stability and efficient use of computational resources. The number of training epochs is set to 100, and an early stopping strategy is adopted: training is stopped when the validation set accuracy shows no improvement for 10 consecutive epochs to prevent model overfitting. During training, the gradient of the total loss function is propagated through the backpropagation algorithm along the path of "diagnostic network → feature fusion network → multi-level feature encoding module → shared feature discriminator," updating all trainable parameters of the primary feature encoder, shared feature discriminator, feature fusion network, binary diagnostic network, and multi-class diagnostic network. This achieves collaborative optimization of each module, ensuring that the entire model architecture converges towards improving fault diagnosis accuracy, robustness, and generalization ability.

[0137] S46. During the inference phase, the category with the highest probability output by the multi-class diagnostic network is taken as the mechanical fault diagnosis result.

[0138] Specifically, during the inference phase, there's no need to calculate dynamic weights and loss functions. The trained multi-class diagnostic network is directly used to predict the fault category of the input sample, and the category with the highest probability output by the multi-class diagnostic network is taken as the final mechanical fault diagnosis result. The specific process is as follows: the sample to be diagnosed undergoes processing steps S1-S5 to obtain a comprehensive diagnostic feature vector; this vector is then input into the trained multi-class diagnostic network, which outputs a 1×... Predicted probability distribution ; Traversal For all elements in the dataset, find the category index corresponding to the element with the highest probability value. The fault type corresponding to this index is the final diagnosis result. For example, if... ( If the probability is high, then the category with the highest probability is category 2, and the diagnosis result is category 2 fault. This decision-making method based on the highest probability can make full use of the classification capabilities of multi-class diagnostic networks, combined with the advantages of multi-source feature fusion and dynamic weighted training in the early stage, to achieve accurate and reliable identification of mechanical faults in real-world engineering scenarios with limited and unbalanced data, providing a direct basis for equipment maintenance decisions.

[0139] The aforementioned deep learning-based mechanical fault diagnosis method first acquires runtime signals with various physical characteristics and performs continuous wavelet transform to generate multi-source time-frequency images containing rich time-frequency features, providing comprehensive input for subsequent deep analysis. Then, a primary feature encoder extracts primary deep features from each source signal, and innovatively introduces a shared feature discriminator. Through adversarial training, the encoder is forced to discard source-specific information, thereby extracting common features consistent across sources and related to the fault's essence. Using these common features as guidance, a gated multi-scale encoder adaptively strengthens components related to common fault modes in each source feature, resulting in enhanced multi-source feature representations. Subsequently, a feature fusion network built based on a cross-multi-head attention mechanism enables deep interactive querying and information aggregation between features from different sources, achieving... This approach achieves deep fusion from simple feature coexistence to the level of intrinsic semantic association. Global average pooling and global max pooling are simultaneously applied to the fused features to capture the overall distribution and most salient local patterns, respectively. These two are then concatenated to form a comprehensive diagnostic feature vector, thus balancing the global representativeness and local discriminativeness of the features. Finally, a dual-branch diagnostic network is designed, utilizing the "abnormal" prior probability provided by the binary diagnostic network to dynamically adjust the training loss weights of the multi-class diagnostic network. This mechanism effectively curbs model bias caused by the dominance of normal samples, forcing the model to give equal importance to scarce fault samples. Ultimately, under the realistic condition of extremely imbalanced data distribution, high-precision and robust identification and classification of multiple types of faults are achieved. Furthermore, the end-to-end design of the entire process ensures the synergistic optimization of feature learning and fault diagnosis, improving the practicality and generalization ability of the method.

[0140] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0141] Based on the same inventive concept, this application also provides a system for implementing the deep learning-based mechanical fault diagnosis method described above. The solution provided by this system is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more deep learning-based mechanical fault diagnosis system embodiments provided below can be found in the limitations of the deep learning-based mechanical fault diagnosis method described above, and will not be repeated here.

[0142] In one exemplary embodiment, such as Figure 3 As shown, a deep learning-based mechanical fault diagnosis system 30 is provided to implement the methods in the above embodiments. The system includes:

[0143] The multi-source time-frequency feature construction module 31 is used to collect at least two different physical characteristics of runtime domain signals from the same mechanical equipment, perform continuous wavelet transform on each runtime domain signal to generate the corresponding time-frequency image, and integrate the time-frequency images corresponding to all types of runtime domain signals to obtain a multi-source time-frequency image.

[0144] The feature commonality extraction module 32 is used to input the multi-source time-frequency images into the corresponding primary feature encoders for feature extraction to obtain multi-source primary feature maps; input the multi-source primary feature maps into the shared feature discriminator, extract the common features in the multi-source primary feature maps through adversarial training, and use the common features as feedback information.

[0145] The feature enhancement and optimization module 33 is used to perform secondary feature encoding and common feature weighting enhancement on the multi-source primary feature map based on feedback information using a gated multi-scale encoder to obtain a multi-source enhanced feature map.

[0146] The multi-source feature fusion module 34 is used to input the multi-source enhanced feature map into the feature fusion network based on the cross-multi-head attention mechanism, and to fuse the multi-source enhanced feature map by calculating the cross attention between different source features to generate multi-source fused features.

[0147] The feature vectorization module 35 is used to perform global average pooling and global max pooling operations on the multi-source fusion features to obtain the pooling results; the pooling results are concatenated to obtain the comprehensive diagnostic feature vector.

[0148] The fault diagnosis decision module 36 is used to input the comprehensive diagnostic feature vector into the binary diagnostic network and the multi-class diagnostic network, dynamically weight the training loss of the multi-class diagnostic network through the output of the binary diagnostic network, and use the output of the multi-class diagnostic network as the mechanical fault diagnosis result.

[0149] Embodiments of this application also provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the aforementioned method embodiments.

[0150] Embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0151] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0152] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A mechanical fault diagnosis method based on deep learning, characterized in that, The method includes: S1. Acquire at least two runtime domain signals with different physical characteristics from the same mechanical equipment, perform continuous wavelet transform on each type of runtime domain signal to generate a corresponding time-frequency image; integrate the time-frequency images corresponding to all types of runtime domain signals to obtain a multi-source time-frequency image; S2. Input the multi-source time-frequency images into the corresponding primary feature encoders for feature extraction to obtain multi-source primary feature maps; input the multi-source primary feature maps into a shared feature discriminator, extract the common features in the multi-source primary feature maps through adversarial training, and use the common features as feedback information; S3. Based on the feedback information, a gated multi-scale encoder is used to perform secondary feature encoding and common feature weighting enhancement on the multi-source primary feature map to obtain a multi-source enhanced feature map; S4. Input the multi-source enhanced feature map into a feature fusion network based on a cross-multi-head attention mechanism, and fuse the multi-source enhanced feature map by calculating the cross attention between different source features to generate multi-source fused features; S5. Perform global average pooling and global max pooling operations on the multi-source fusion features to obtain pooling results; concatenate the pooling results to obtain a comprehensive diagnostic feature vector; S6. Input the comprehensive diagnostic feature vector into the binary diagnostic network and the multi-class diagnostic network, dynamically weight the training loss of the multi-class diagnostic network through the output of the binary diagnostic network, and use the output of the multi-class diagnostic network as the mechanical fault diagnosis result.

2. The method according to claim 1, characterized in that, The step of inputting the multi-source primary feature map into a shared feature discriminator, extracting common features from the multi-source primary feature map through adversarial training, and using the common features as feedback information includes: S11. The shared feature discriminator is used to classify and determine the source of the multi-source primary feature map, and the multi-source discrimination result is output. S12. Based on the multi-source discrimination result, the gradient flowing to the primary feature encoder is reversed using the gradient inversion layer to obtain the backpropagation gradient signal. S13. Based on the multi-source discrimination result and the true source label corresponding to the multi-source primary feature map, calculate the adversarial training loss value; the formula for calculating the adversarial training loss value is: in, This represents the adversarial training loss value. and These represent the two primary feature encoders that process the first source signal and the second source signal, respectively. This refers to the shared feature discriminator. This indicates that the shared feature discriminator corresponds to the first source input. Classification branches, This indicates that the shared feature discriminator corresponds to the second source input. Classification branches, and These represent the two input multi-source primary feature maps, This represents a logarithmic function with the natural constant as its base. Represents the mathematical expectation operator; S14. The parameters of the primary feature encoder are updated using the backpropagation gradient signal, which drives the primary feature encoder to generate features that increase the error rate of the shared feature discriminator, and the backpropagation gradient signal is used as the feedback information characterizing the common features.

3. The method according to claim 1, characterized in that, Based on the feedback information, a gated multi-scale encoder is used to perform secondary feature encoding and common feature weighting enhancement on the multi-source primary feature map to obtain a multi-source enhanced feature map, including: S21. Input the multi-source primary feature maps into a multi-scale feature extraction module composed of multiple parallel convolutional branches. Each convolutional branch uses a convolutional kernel of a different size to extract features under different receptive fields, thereby obtaining multiple sets of multi-scale features. S22. Input the concatenated result of the multi-scale features into an adaptive gating network, and use the adaptive gating network and the feedback information to calculate the gating weight coefficients corresponding to each convolutional branch; the formula for calculating the gating weight coefficients is: in, Indicates the first The gating weight coefficients of the convolutional branches, This represents the Sigmoid activation function. This represents a trainable gated weight matrix. This indicates a splicing operation. They represent the 1st to the 1st. The multi-scale features output by the convolutional branches; the feedback information is used to guide the gating weight matrix during training. The learning enables the adaptive gating network to dynamically adjust the contribution of each branch feature based on the common characteristics among multi-source data. S23. Based on the gating weight coefficients, the corresponding multi-scale features are weighted and fused to obtain the multi-source enhanced feature map.

4. The method according to claim 1, characterized in that, The step of inputting the multi-source enhanced feature map into a feature fusion network based on a cross-multi-head attention mechanism, and fusing the multi-source enhanced feature map by calculating the cross-attention between different source features to generate multi-source fused features includes: S31. Perform linear projection processing on the multi-source enhanced feature map of each source to generate the query vector, key vector and value vector corresponding to each source; S32. Based on the query vector of the first source and the key vector of the second source, the first cross-attention weight is calculated by scaling the dot product; the first cross-attention weight is used to perform a weighted summation on the value vector of the second source to obtain the first cross-attention feature. S33. The first cross-attention feature and the multi-source enhanced feature map of the first source are added together by residual processing to obtain the first source intermediate fusion feature; the expression of the first source intermediate fusion feature is: in, This represents the first source intermediate fusion feature. Represents the normalized exponential function, The query vector representing the first source, This represents the transpose of the key vector of the second source. The value vector representing the second source, This represents the total dimension of the feature vector. Indicates the number of attention heads. The multi-source enhanced feature map representing the first source; S34. Based on the query vector of the second source and the key vector of the first source, the second cross-attention weight is calculated by scaling the dot product; the second cross-attention weight is used to perform a weighted summation on the value vector of the first source to obtain the second cross-attention feature. S35. The second cross-attention feature and the multi-source enhanced feature map of the second source are added together by residual processing to obtain the second source intermediate fusion feature; the expression of the second source intermediate fusion feature is: in, This indicates the second source intermediate fusion feature. The query vector representing the second source. This represents the transpose of the key vector of the first source. The value vector representing the first source, The multi-source enhanced feature map representing the second source; S36. Perform layer normalization and nonlinear transformation on the first source intermediate fusion feature and the second source intermediate fusion feature to obtain the first source fusion feature and the second source fusion feature respectively; construct the multi-source fusion feature based on the first source fusion feature and the second source fusion feature; the expressions of the first source fusion feature and the second source fusion feature are: in, This represents the first source fusion feature. This represents the second source fusion feature; The representation layer is normalized, and MLP stands for fully connected layer.

5. The method according to any one of claims 1 to 4, characterized in that, The step of inputting the comprehensive diagnostic feature vector into a binary diagnostic network and a multi-class diagnostic network, dynamically weighting the training loss of the multi-class diagnostic network by the output of the binary diagnostic network, and using the output of the multi-class diagnostic network as the mechanical fault diagnosis result includes: S41. Input the comprehensive diagnostic feature vector into the binary diagnostic network to obtain the prior probability representing that the sample belongs to an abnormal state. ; S42. Input the comprehensive diagnostic feature vector into the multi-class diagnostic network to obtain the predicted probability distribution belonging to each fault category. From the predicted probability distribution Extract the probability components corresponding to all fault categories. ; S43. Calculate the prior probability. With the probability part The cosine similarity between them is calculated, and the cosine similarity is mapped to dynamic weights. The dynamic weights The calculation formula is: in, This represents the dynamic weight. This represents the function for calculating cosine similarity. Represents the natural constant; S44. The cross-entropy loss of the multi-class diagnostic network is weighted using the dynamic weights to obtain a weighted multi-class loss; the formula for calculating the weighted multi-class loss is: in, This represents the weighted multi-class loss. Represents the set of indices for the training samples. This indicates the total number of fault categories. Index representing the fault category, Indicates the first The dynamic weights corresponding to each sample Indicates the first Each sample in category The real labels on The output of the multi-class diagnostic network represents the first... Each sample in category The predicted probability, Represent the natural logarithm function; S45. Combining the loss of the binary diagnostic network, the weighted multi-class loss, and the adversarial loss of the shared feature discriminator, a total loss function is constructed; the primary feature encoder, the shared feature discriminator, the feature fusion network, the binary diagnostic network, and the multi-class diagnostic network are jointly trained using the total loss function; wherein, the formula for calculating the loss of the binary diagnostic network is: in, This represents the loss of the binary diagnostic network. This represents normal operating conditions. Represents a fault condition. This represents the number of training samples. and Representing the first The true probability of a sample belonging to normal or abnormal conditions; S46. During the inference phase, the category with the highest probability output by the multi-class diagnostic network is taken as the mechanical fault diagnosis result.

6. A deep learning-based mechanical fault diagnosis system, used to implement the method according to any one of claims 1 to 5, characterized in that, The system includes: A multi-source time-frequency feature construction module is used to acquire at least two different physical characteristics of runtime domain signals from the same mechanical equipment, perform continuous wavelet transform on each of the runtime domain signals to generate a corresponding time-frequency image, and integrate the time-frequency images corresponding to all types of runtime domain signals to obtain a multi-source time-frequency image. The feature commonality extraction module is used to input the multi-source time-frequency images into the corresponding primary feature encoders for feature extraction to obtain multi-source primary feature maps; input the multi-source primary feature maps into a shared feature discriminator, extract the common features in the multi-source primary feature maps through adversarial training, and use the common features as feedback information; The feature enhancement and optimization module is used to perform secondary feature encoding and common feature weighting enhancement on the multi-source primary feature map based on the feedback information using a gated multi-scale encoder to obtain a multi-source enhanced feature map. The multi-source feature fusion module is used to input the multi-source enhanced feature map into a feature fusion network based on a cross-multi-head attention mechanism, and to fuse the multi-source enhanced feature map by calculating the cross attention between different source features to generate multi-source fused features; The feature vectorization module is used to perform global average pooling and global max pooling operations on the multi-source fusion features to obtain pooling results; the pooling results are then concatenated to obtain a comprehensive diagnostic feature vector. The fault diagnosis decision module is used to input the comprehensive diagnostic feature vector into a binary diagnostic network and a multi-class diagnostic network, dynamically weight the training loss of the multi-class diagnostic network by the output of the binary diagnostic network, and use the output of the multi-class diagnostic network as the mechanical fault diagnosis result.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

Images