A Gear Fault Diagnosis Method and System Based on Multimodal Fusion and STIM

By optimizing FMD parameters using PGA and combining MTF conversion with the STIM model, the problems of signal decomposition accuracy and timing information preservation in gear fault diagnosis are solved, achieving efficient fault feature extraction and accurate fault diagnosis, which is suitable for gear fault diagnosis under complex working conditions.

CN123115941APending Publication Date: 2026-11-17BEIHUA UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611275132.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-21
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing gear fault diagnosis methods struggle to effectively preserve the temporal dependencies and state evolution information of signals in noisy environments, and their decomposition accuracy is limited. Traditional methods rely on manual parameter tuning and have high computational complexity, making it difficult to meet real-time requirements.

Method used

The phototropic growth optimization algorithm (PGA) is used to optimize the eigenmode decomposition (FMD) parameters. The signal is converted into a two-dimensional time-frequency image by combining the Markov transform field (MTF). A dual-branch deep learning STIM model, including the Swing Transformer and Informer models, is constructed to achieve adaptive fusion of multimodal features.

Benefits of technology

It achieves high-precision decomposition and feature extraction of gear fault signals in high-noise environments, improving the accuracy and robustness of fault diagnosis. It has excellent fitting ability and generalization potential, and is suitable for gear fault diagnosis under complex working conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115941A_ABST
    Figure CN123115941A_ABST
Patent Text Reader

Abstract

This invention discloses a gear fault diagnosis method and system based on multimodal data fusion and the STIM model, belonging to the field of gearbox fault diagnosis technology. The method first acquires the original vibration signal of the gearbox, uses envelope entropy as the fitness function, and employs a light-growing optimization algorithm to adaptively find the optimal parameter combination for feature mode decomposition. After performing FMD decomposition using the optimal parameters, sensitive modal components are automatically screened and reconstructed through a time-frequency domain comprehensive evaluation factor, effectively eliminating noise interference. Subsequently, a Markov transform field is used to convert the reconstructed one-dimensional signal into a two-dimensional time-frequency image, preserving complete temporal dependency information. Finally, a STIM dual-branch multimodal fusion model is constructed: the Swing Transformer branch extracts local-global visual features of the MTF image, and the Informer branch captures long-range temporal features of the one-dimensional signal. The outputs of the two branches are adaptively fused through a multi-head attention mechanism and input into a classifier to output the fault identification result. This invention has the technical advantages of high diagnostic accuracy and strong generalization ability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of gearbox fault diagnosis technology, specifically to a gear fault diagnosis method and system based on multimodal fusion and STIM. Background Technology

[0002] Gear fault diagnosis is a key technology to ensure the safe operation of mechanical equipment. Existing diagnostic methods are mainly divided into two categories: signal processing and machine learning.

[0003] In signal processing, adaptive decomposition methods such as Empirical Mode Decomposition (EMD), Ensemble Empirical Mode Decomposition (EEMD), and Variational Mode Decomposition (VMD) are widely used. However, EMD and EEMD suffer from inherent defects such as mode aliasing and endpoint effects, and their decomposition accuracy is limited by extremum interpolation and envelope fitting errors. Although VMD has a solid mathematical theoretical foundation, its decomposition performance heavily relies on the manual pre-setting of key parameters such as the number of modes and the penalty factor, lacks an adaptive parameter tuning mechanism, and does not fully consider the impulsive and periodic characteristics of gear fault signals, resulting in insufficient fault feature extraction capabilities in high-noise environments.

[0004] In machine learning, convolutional neural networks (CNNs), due to their local receptive field structure, struggle to capture the long-range global temporal dependencies of vibration signals. While the standard Transformer model possesses global modeling capabilities, its self-attention mechanism exhibits quadratic computational complexity when processing long signal sequences, making it difficult to meet the real-time requirements of practical industrial scenarios. Furthermore, existing methods often directly input one-dimensional vibration signals into the model or simply convert them into two-dimensional images such as Gram angle fields, failing to effectively preserve the temporal dependencies and state evolution information of the signals. Key fault features are easily obscured under complex noise interference. Therefore, there is an urgent need for a gear fault diagnosis method that can adaptively optimize decomposition parameters, effectively preserve temporal information, and possess efficient feature fusion capabilities.

[0005] Therefore, there is an urgent need to develop gear fault diagnosis methods and systems based on multimodal fusion and STIM to solve the above problems. Summary of the Invention

[0006] This invention proposes a gear fault diagnosis method and system based on multimodal fusion and STIM. It adopts a two-stage hierarchical matching architecture and an unconfirmed trajectory dual-wheel fault tolerance and life-saving mechanism, and designs an adaptive distance constraint based on the target diagonal and an exponential saturation gradual scale penalty, which effectively solves the above-mentioned technical problems.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a gear fault diagnosis method and system based on multimodal fusion and STIM, comprising the following steps:

[0008] S1: Collect the original vibration signal of the gearbox and use the photo-growing optimization algorithm PGA to optimize the key parameters of the eigenmode decomposition (FMD) to obtain the optimal parameter combination;

[0009] S2: The original vibration signal is decomposed by FMD using the optimal parameter combination to obtain modal components. The sensitive modal component discrimination algorithm based on the time-frequency domain comprehensive evaluation factor is applied to automatically filter the components containing the main fault characteristics to obtain the noise-reduced and enhanced fault signal.

[0010] S3: The above fault signal is converted into a two-dimensional time-frequency image using Markov Transformed Field MTF;

[0011] S4: Construct a two-branch deep learning STIM model, including:

[0012] The first branch: The visual feature extraction branch adopts the shifted window Swin Transformer model, which includes a hierarchical architecture, a shifted window self-attention mechanism and cross-scale relative position bias, and is used to extract local and global multi-scale visual features of MTF images.

[0013] The second branch: a branch for extracting temporal features using a long-sequence temporal model, which includes a sparse self-attention mechanism and a generative decoder, is used to capture the long-range temporal dependence features of vibration signals;

[0014] The visual features and temporal features extracted from the two branches above are input into the multi-head attention mechanism (MHA) module. By calculating the feature correlation of different subspaces, adaptive fusion of multimodal features is achieved.

[0015] S5: Input the features fused by the MHA module into the fully connected layer and the Softmax classifier, and finally output the gear fault type identification result.

[0016] Preferably, in step S1, the envelope entropy of the original vibration signal is used as the fitness function, and the PGA algorithm is used to globally optimize the number of modes n and the filter length L of the FMD to obtain the optimal parameter combination.

[0017] Preferably, in step S2, the sensitive modal component discrimination algorithm based on the time-frequency domain comprehensive evaluation factor comprehensively calculates the time-domain kurtosis value of each component and the frequency-domain cross-correlation coefficient with the original signal, constructs a weighted comprehensive evaluation factor, and automatically determines the sensitive components through the principle of maximum difference, and accumulates and reconstructs the selected sensitive components.

[0018] Preferably, the principle of maximizing the difference in step S2 is as follows: arrange each modal component in descending order of the comprehensive evaluation factor, calculate the difference between adjacent comprehensive evaluation factors, take the position of the maximum difference as the boundary point between the sensitive modal component and the interference modal component, and select all modal components before the boundary point as the sensitive modal components.

[0019] Preferably, in step S3, the fault signal is converted into a two-dimensional time-frequency image, and the gear vibration signal is decomposed and reconstructed using PGA-FMD. Quantile division and state transition probability calculation are performed to map the temporal dependency of the signal into the pixel spatial information of the image, thereby generating an MTF image with high fault discrimination.

[0020] Preferably, the Swin Transformer model in S4 adopts a hierarchical coding architecture, which progressively downsamples the input image through a multi-stage shift window self-attention module and extracts local texture features and global semantic features of the image at different scales; the cross-scale relative position bias is used to align and fuse position information at different scales between each stage.

[0021] Preferably, the ProbSparse sparse self-attention mechanism of the Informer model in S4 calculates the sparsity metric of each query, selects several active queries with the highest sparsity scores, and performs dot product attention calculation only on the active queries to reduce the computational complexity of long sequence modeling.

[0022] Preferably, the fault types in S5 include: healthy state, tooth surface wear, tooth root crack, broken tooth, and tooth surface pitting.

[0023] Preferably, a gear fault diagnosis system based on multimodal data fusion and STIM model is proposed, comprising:

[0024] The signal acquisition module is used to acquire the raw vibration signals of the gearbox;

[0025] The signal preprocessing module is used to perform the signal decomposition, sensitive mode screening and reconstruction operations described in S1 to S2;

[0026] The image conversion module is used to perform the Markov transformation field image conversion operation described in S3;

[0027] The multimodal fusion diagnostic module is used for STIM model feature extraction, fusion, and fault classification operations described in S4 to S5.

[0028] This invention provides a gear fault diagnosis method and system based on multimodal fusion and STIM, which has the following advantages compared with existing technologies:

[0029] This invention uses envelope entropy as the fitness function and employs the phototropic growth optimization algorithm (PGA) to globally optimize the key parameters of Eigenmode Decomposition (FMD), fundamentally overcoming the limitations of traditional methods that rely on manual parameter tuning based on experience. The PGA algorithm converges to the optimal solution in the fourth iteration, with a convergence speed significantly better than the gray wolf optimization and whale optimization algorithms. It effectively suppresses the mode aliasing problem, concentrating the decomposed modal components near the center frequency without frequency divergence, thus providing a high-purity signal foundation for subsequent feature extraction.

[0030] The automatic selection algorithm for time-frequency domain comprehensive evaluation factors proposed in this invention weightedly fuses time-domain kurtosis and frequency-domain cross-correlation coefficients, and adaptively determines the boundaries of sensitive mode components through the principle of maximizing the difference, avoiding the subjectivity of manually setting thresholds. Experiments show that after reconstruction of sensitive components, random noise and high-frequency interference in the original signal are effectively eliminated, fault impact characteristics are significantly enhanced, and the time-domain waveforms of signals such as wear faults are clearly distinguishable.

[0031] The STIM dual-branch multimodal fusion model constructed in this invention extracts local-global multi-scale visual features from MTF images through the hierarchical shift window mechanism of the Swing Transformer. Simultaneously, it utilizes the ProbSparse sparse self-attention mechanism of the Informer to efficiently capture the long-range temporal dependence of vibration signals, and achieves adaptive deep fusion of features through a multi-head attention mechanism. Experimental data shows that the model converges after approximately 40 iterations, achieving 100% accuracy in diagnosing single gear faults, validating the model's excellent fitting ability and generalization potential. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0033] Figure 1 This is an overall flowchart of the gear fault diagnosis method based on multimodal data fusion and STIM of the present invention;

[0034] Figure 2 This is a diagram showing the internal structure of the gearbox of the present invention;

[0035] Figure 3 This is a flowchart illustrating the optimization of key parameters of Feature Mode Decomposition (FMD) using the Photogrowing Optimization Algorithm (PGA) of this invention.

[0036] Figure 4This is a comparison chart of the fitness iteration curves of the PGA, GWO, and WOA algorithms of this invention;

[0037] Figure 5 The time-domain waveform and spectrum of the wear fault signal decomposed by PGA-FMD in this invention are shown.

[0038] Figure 6 This is a flowchart illustrating how the Markov Transformed Field (MTF) of this invention converts a one-dimensional signal into a two-dimensional image.

[0039] Figure 7 The MTF diagrams for different gear states in this invention are shown (a is the reconstructed signal after transformation, and b is the original data after transformation).

[0040] Figure 8 This is a schematic diagram of the sliding window mechanism of the Swing Transformer core module in the STIM model of this invention;

[0041] Figure 9 This is a schematic diagram of the Informer model in the STIM model of this invention;

[0042] Figure 10 This is a schematic diagram illustrating the feature fusion process of the multi-head attention mechanism (MHA) of the present invention.

[0043] Figure 11 This is a graph showing the changes in loss and accuracy between the training and validation sets in the PGA-FMD reconstruction ablation study of this invention.

[0044] Figure 12 This is a visualization of the dimensionality reduction of the fusion features extracted by the STIM model using the T-SNE algorithm in this invention.

[0045] Figure 13 This is a classification confusion matrix diagram on the STIM model test set of this invention;

[0046] Figure 14 This is a comparison chart of the diagnostic accuracy of PGA-FMD, FMD, VMD, and EEMD on an industrial gearbox dataset.

[0047] Figure 15 This is a comparison chart of the diagnostic accuracy of MTF, Gram angle field, recursive graph, and fast Fourier transform on an industrial gearbox dataset under the PGA-FMD+STIM model architecture of this invention.

[0048] Figure 16 The figures show the results of seven ablation experiments of this invention.

[0049] Figure 17 This is a schematic diagram of the gearbox test bench of the present invention;

[0050] Figure 18 Features are visualized using t-SNE. Detailed Implementation

[0051] To make the technical problems, technical solutions, and beneficial effects of the present invention clearer, the present invention will be further described in detail with reference to the embodiments and accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention. The technical solutions of the present invention will be described in detail below with reference to the embodiments and accompanying drawings, but the scope of protection is not limited thereto.

[0052] Please see Figure 1-18 The present invention provides a technical solution: The STIM dual-branch multimodal fusion model constructed by the present invention extracts local-global multi-scale visual features of MTF images through the hierarchical shift window mechanism of Swing Transformer, while using the ProbSparse sparse self-attention mechanism of Informer to efficiently capture the long-range temporal dependence of vibration signals, and achieves feature adaptive deep fusion through a multi-head attention mechanism.

[0053] The gear fault diagnosis method and system based on multimodal fusion and STIM provided by this invention includes the following steps:

[0054] S1: Collect the original vibration signal of the gearbox and use the photo-growing optimization algorithm PGA to optimize the key parameters of the eigenmode decomposition (FMD) to obtain the optimal parameter combination;

[0055] S2: The original vibration signal is decomposed by FMD using the optimal parameter combination to obtain modal components. The sensitive modal component discrimination algorithm based on the time-frequency domain comprehensive evaluation factor is applied to automatically filter the components containing the main fault characteristics to obtain the noise-reduced and enhanced fault signal.

[0056] S3: The above fault signal is converted into a two-dimensional time-frequency image using Markov Transformed Field MTF;

[0057] S4: Construct a two-branch deep learning STIM model, including:

[0058] The first branch: The visual feature extraction branch adopts the shifted window Swin Transformer model, which includes a hierarchical architecture, a shifted window self-attention mechanism and cross-scale relative position bias, and is used to extract local and global multi-scale visual features of MTF images.

[0059] The second branch: a branch for extracting temporal features using a long-sequence temporal model, which includes a sparse self-attention mechanism and a generative decoder, is used to capture the long-range temporal dependence features of vibration signals;

[0060] The visual features and temporal features extracted from the two branches above are input into the multi-head attention mechanism (MHA) module. By calculating the feature correlation of different subspaces, adaptive fusion of multimodal features is achieved.

[0061] S5: Input the features fused by the MHA module into the fully connected layer and the Softmax classifier, and finally output the gear fault type identification result.

[0062] In some embodiments of the present invention, fault simulation employs, for example... Figure 17 The experimental platform shown mainly consists of a drive motor, a planetary gearbox, a parallel shaft reduction gearbox, a load, and a load controller. During signal acquisition, the accelerometer is mounted on the bearing housing of the reduction gearbox. The data acquisition card is an ADA16-8 / 2 (LPCI) type, with a sampling frequency of 5120 Hz, a motor speed of 1470 r / min, a sampling duration of 10 s, and a total of 5 samplings. 200 samples are randomly selected for each state, each sample containing 2048 sampling points, for a total of 1000 samples.

[0063] The internal transmission structure of the gearbox of the test bench is as follows: Figure 2 As shown, it mainly consists of two parts: a planetary gearbox and a reduction gearbox. The reduction gearbox contains four spur gears with 32, 72, 80, and 40 teeth respectively, all made of S45C, with a gear module of 2.5 mm.

[0064] In terms of fault simulation, five gear operating states were set: healthy state, tooth surface wear, tooth root crack, broken tooth, and tooth surface pitting. The labels and descriptions for each state are shown in Table 1. The fault processing methods are as follows: ① Tooth surface wear: Manual grinding and filing of a single tooth surface on the 40-tooth gear, with a wear amount of 15% of the tooth thickness; ② Broken tooth: Manual breakage of a single tooth on the 72-tooth gear near the pitch circle; ③ Pitting: Forming pits on the tooth surface of the 72-tooth gear through spot welding, with a pitting area ratio of approximately 2%; ④ Tooth root crack: Pre-fabricated cracks with a depth of 0.5 mm were machined at the tooth root of the 40-tooth gear through wire cutting. Gear state descriptions are shown in Table 1.

[0065] Table 1 Gear Condition Description

[0066] In subsequent experiments, we performed signal preprocessing, using data labeled 0, 1, 2, 3, and 4. Taking the wear fault data labeled 1 as an example, we randomly selected a signal for analysis, and its time-domain waveform is shown below. Figure 1 The signal processing section is shown in the diagram.

[0067] Since the signal acquisition system does not have a noise filtering device, the acquired signal is affected by background signals and noise, making it impossible to directly and effectively extract signal features. Based on this, it is difficult to accurately diagnose the working state and fault type of the gear. Therefore, it is necessary to perform noise reduction processing on the acquired vibration signal to highlight the signal features.

[0068] Using envelope entropy as the fitness function, the PGA (photogenerative optimization algorithm) is employed to... Figure 3 The FMD decomposition parameters of the wear fault signal were optimized to obtain the optimal parameter combination [n_opt, L_opt]. The Grey Wolf Optimization Algorithm (GWO) and the Whale Optimization Algorithm (WOA) were compared. All three algorithms had a population size of 30 and a maximum number of iterations of 10. The parameter optimization range was n∈[2, 7], L∈[10, 70]. The fitness iteration curves of the three algorithms are shown below. Figure 4 As shown.

[0069] from Figure 4 It can be seen that the fitness values ​​of GWO and WOA are better in the initial stage of iteration. The PGA algorithm converges to the optimal solution in the fourth iteration, and the final fitness value is significantly lower than that of GWO and WOA algorithms. This proves that the PGA algorithm has a faster convergence speed and better global optimization ability, and is more suitable for parameter optimization of FMD.

[0070] Using the optimal parameters obtained through PGA optimization, the wear fault signal is decomposed using FMD to obtain the time-domain waveforms and spectra of each modal component, as shown below. Figure 5 As shown.

[0071] Depend on Figure 5 As can be seen, the FMD decomposition is relatively reasonable, with each modal component mainly concentrated near the center frequency, and no obvious frequency divergence phenomenon is observed. This indicates that the proposed parameter selection method can effectively suppress the modal aliasing problem generated during the decomposition process, reduce information leakage between modal components, and provide a guarantee for subsequent signal feature extraction. This also demonstrates the effectiveness of the PGA algorithm in optimizing key FMD parameters. The optimal PGA-FMD parameter combinations corresponding to the five gear states are shown in Table 2.

[0072] Table 2 Optimal Parameters for Single-Fault PGA-FMD

[0073] When the frequency components of a signal change rapidly, time-domain analysis may fail to accurately capture the signal's characteristics, while frequency-domain analysis can more comprehensively reveal the frequency components contained in the signal. False mode components (MFs) typically possess unique frequency-domain characteristics, such as sharp peaks or anomalous frequency components. Therefore, frequency-domain information is more crucial than time-domain information when identifying false MF components. Accordingly, in equation (13), β is set to 0.4 and γ to 0.6. Calculation Figure 5 The time-frequency domain comprehensive evaluation factors of each MF component shown are arranged into a descending sequence {δ}.

[0074] The maximum difference is between M2 and M3, where M3 is inclusive. Figure 8 The sensitive MF component of the wear fault characteristics is shown, while M1, M2, and M4 are interference components containing noise and background signals. The M3 component is selected for reconstruction, and the time-domain waveform of the resulting reconstructed signal is shown below. Figure 1 The signal processing section is shown in the diagram.

[0075] In summary, after PGA-FMD decomposition and sensitive component reconstruction, random noise and high-frequency interference in the original signal are effectively eliminated, and the fault impact characteristics are significantly enhanced, laying a reliable foundation for subsequent accurate feature extraction and fault classification.

[0076] Subsequently, a Markov Transition Field (MTF) was used to convert the one-dimensional signal into a two-dimensional image. The data was subjected to MFT time-frequency image conversion. The transition probabilities between each state were statistically analyzed along the time axis to construct a Q×Q state transition matrix. Each element in the state transition matrix was mapped to the corresponding pixel value in the image, generating a Q×Q two-dimensional time-frequency image. That is, following the previous signal processing steps and methods, a randomly selected group of five-state gear vibration signals was decomposed and reconstructed using PGA-FMD. The reconstructed signal was then converted into a 224 × 224 resolution two-dimensional time-frequency image (quantile Q = 64) using MFT. The conversion result is shown below. Figure 7 As shown in (a). For comparison, the result of directly converting the original signal via MTF is shown in (a). Figure 7 As shown in (b).

[0077] In practical applications, the quantile Q can range from 32 to 128, and the image resolution can range from 128×128 to 512×512. The specific value can be adaptively adjusted according to hardware computing resources and diagnostic accuracy requirements. The larger the Q value, the richer the temporal details contained in the image, but the computational cost also increases accordingly. The higher the image resolution, the more sufficient the input information for subsequent deep learning models, but the number of model parameters and computational overhead also increase.

[0078] Depend on Figure 7It can be seen that the MTF images of different gear fault states exhibit significant visual differences: healthy images have uniform and orderly textures; pitting faults show a scattered distribution; broken tooth faults show periodic banded patterns; tooth root cracks show localized mesh distortion; and wear faults show blurred longitudinal stripes. (Comparison) Figure 3 and Figure 7 As can be seen, after PGA-FMD decomposition and sensitive component reconstruction, random noise and high-frequency interference in the original signal are effectively removed, fault impact characteristics are significantly enhanced, noise interference is significantly reduced, and the visual discrimination of fault characteristics is much higher than that of the original signal, proving the effectiveness of the preprocessing method of the present invention.

[0079] This invention employs an Informer model, which includes a ProbSparse sparse self-attention mechanism and a generative decoder. The core parameters of the STIM model, constructed to capture long-range temporal-dependent features of vibration signals, are shown in Table 3. This model is a dual-branch, multimodal fusion deep learning model. The first branch receives the MTF image (224×224×3) generated in step S3 and uses an improved Swin Transformer model to extract local and global multi-scale visual features. The second branch receives the one-dimensional fault reconstruction signal obtained in step S2. The MHA module calculates the correlation between visual features and temporal features in different subspaces using multiple parallel attention heads. The visual feature vector output from the first branch and the temporal feature vector output from the second branch are input into the multi-head attention mechanism (MHA) module for adaptive deep fusion. The calculation process of each attention head is as follows: the visual features and temporal features are mapped to the query, key, and value spaces respectively through linear transformation; the dot product attention weight between the query and the key is calculated; the attention weight is multiplied by the value to obtain the output of the attention head; the outputs of all attention heads are concatenated and then subjected to a linear transformation to obtain the final fused features.

[0080] This invention uses the STIM model for pattern testing. The experimental dataset is randomly divided into a training set (700 samples), a validation set (200 samples), and a test set (100 samples) in a 7:2:1 ratio. During model training, the optimizer used is AdamW, with an initial learning rate of 0.0001, 100 iterations, and 8 MHA attention heads. Adaptive deep fusion of multimodal features is achieved by calculating the feature correlations in different subspaces, and the cross-entropy loss function is used.

[0081] Table 3 Ablation Experiment Plan

[0082] The experimental hardware environment consisted of an AMD 5700×3D CPU, an NVIDIA RTX 4060 8G GPU, and 32GB of RAM; the software environment consisted of a Windows 11 operating system, Python 3.9, and the PyTorch 2.0 deep learning framework.

[0083] During model training, the loss and accuracy curves for the training and validation sets are as follows: Figure 11 As shown, the model basically converges after about 40 iterations, with the accuracy on both the training and validation sets reaching 100% and no obvious overfitting, proving that the model has good fitting ability and generalization potential.

[0084] The T-SNE algorithm was used to perform dimensionality reduction and visualization of the fusion features extracted by the model. The results are as follows: Figure 12 As shown, the features of the five gear states are completely separated, the clustering of similar fault features is extremely high, and there is no overlap between different fault features, proving that the method of the present invention can effectively extract fault features with strong discriminative capabilities.

[0085] The model's classification confusion matrix on the test set is as follows: Figure 13 As shown. From Figure 13 As can be seen, the method of the present invention achieves 100% accuracy in classifying the five gear states, with no misclassified samples, thus realizing accurate diagnosis of single gear faults.

[0086] This invention compares the performance of different signal decomposition methods. To verify the advantages of the PGA-FMD method of this invention, under the same MTF+STIM model architecture, PGA-FMD is compared with three mainstream signal decomposition methods: the original FMD, VMD, and EEMD. The diagnostic accuracy of different methods is as follows: Figure 14 As shown.

[0087] Depend on Figure 14 As can be seen, the PGA-FMD method of this invention achieves 100% accuracy in diagnosing all five gear conditions, with an average accuracy significantly better than the original FMD (96.44%), VMD (89.24%), and EEMD (86.66%). Its advantages are particularly evident in the diagnosis of severe faults such as broken teeth and cracks, proving that the PGA-optimized FMD method can effectively improve decomposition accuracy and enhance the ability to extract fault features.

[0088] To assess the performance of different time-series-image conversion methods, this invention compares MTF with three mainstream time-series-image conversion methods—Gramian Angular Field, Recurrence Plot, and Fast Fourier Transform (FFT)—under the same PGA-FMD+STIM model architecture. The diagnostic accuracy of different methods is shown below. Figure 15 As shown. The GAF parameters are set as follows: after polar coordinate transformation, a cosine function is used to generate GASF and GADF, and the image size is uniformly scaled to 224×224; the RP parameters are set as follows: the distance threshold is 0.1 times the standard deviation of the time series, the embedding dimension is 10, and the time delay is 1; the FFT parameters are set as follows: the sampling frequency is consistent with the original signal, the window function is a Hamming window, the window length is 256, and the overlap length is 128.

[0089] Depend on Figure 15 The results show that the MTF method achieves an average diagnostic accuracy of 100%, significantly outperforming GAF (96.2%), RP (95.7%), and FFT (96.0%), especially in diagnosing severe faults such as breakage and missing teeth. This demonstrates that MTF can more effectively preserve the temporal characteristics and state evolution information of fault signals, and the generated images possess higher fault discrimination capabilities.

[0090] To clarify the contribution of each module of the present invention to the overall performance, eight ablation experiments were designed. All experiments were conducted under the same training settings (training set, test set, and validation set ratio of 7:2:1, 100 training rounds, and initial learning rate of 0.0001). The experimental scheme and results are shown in Table 4. Figure 16 As shown.

[0091] Table 4 Ablation Experiment Plan

[0092] The results of the ablation experiment show that:

[0093] (1) The complete model (Solution A) achieved 100% diagnostic accuracy. When FMD with fixed parameters was used instead of PGA optimization (Solution B), the accuracy decreased by 7.5%, indicating that PGA adaptive parameter optimization has the greatest impact on model performance and is the core link to improve the accuracy of fault feature extraction.

[0094] (2) Removing the sensitive MF selection step (scheme C) results in a 4.9% decrease in accuracy, which proves that eliminating noise components can effectively improve feature purity and enhance the model's fault detection capability.

[0095] (3) Directly using the original signal (Scheme D) results in a significant decrease in accuracy of 12.7%, which highlights the necessity of the proposed signal preprocessing method under strong noise conditions.

[0096] (4) Replacing Swin Transformer + CSARPB with standard ViT (patch size 16×16, 12 layers, hidden layer dimension 768, other components are the same as Scheme A) (Scheme E) reduces the accuracy by 6.0% compared with Scheme A; and reduces the accuracy by 1.4% compared with the original Swin Transformer (Scheme F). This proves that the hierarchical architecture and shift window mechanism of Swin Transformer are more suitable for multi-scale feature extraction from fault time-frequency images.

[0097] (5) Removing the cross-scale relative position bias (Scheme F, with the same parameters as Scheme A) results in a 4.6% decrease in accuracy, which indicates that the proposed improved Swing Transformer + CSARPB model outperforms the original Swing Transformer in multi-scale feature extraction from fault time-frequency images.

[0098] (6) Replacing the Informer with a standard LSTM (hidden layer dimension 256, output dimension 5, other components are the same as in scheme A) (Scheme G) resulted in a 3.3% decrease in accuracy, which proves that the Informer can more effectively capture the long-range time dependence in vibration signals, thereby supplementing the global information that visual features fail to cover.

[0099] (7) Removing the MHA feature fusion module (replacing it with feature concatenation followed by a fully connected layer) (Scheme H) will result in a 1.9% decrease in accuracy. This proves that MHA can perform adaptive deep fusion of multimodal features, make full use of the complementary information between different features, and further improve the diagnostic performance of the model.

[0100] To evaluate the generalization ability of the method of the present invention under different working conditions and different fault types, the present invention uses the Tsinghua University gearbox fault dataset and the QPZZ-II rotating machinery fault simulation test bench dataset for cross-domain verification. The Tsinghua University dataset contains single fault data, while the QPZZ-II dataset contains gear compound fault data.

[0101] A total of 1200 samples were selected from the Tsinghua University dataset, representing six gear states (health, pitting, wear, missing teeth, damage, and cracks), with 2048 sampling points per sample. Similarly, 1200 samples were selected from the QPZZ-II dataset, representing six gear states (health, broken teeth, wear, pitting, combined broken teeth and wear faults, and combined pitting and wear faults), with 1024 sampling points per sample. All samples were preprocessed and used for model training using the method described in this invention. The datasets were divided in a 7:2:1 ratio, and the model parameters remained consistent with those described previously.

[0102] The diagnostic results of the two public datasets are shown in Table 5. As can be seen from Table 5, the average diagnostic accuracy of the method of the present invention reaches 99.75% on the Tsinghua University single fault dataset and 99.25% on the QPZZ-II composite fault dataset, which is only slightly lower than that of the actual test dataset of the present invention. This proves that the method of the present invention has extremely strong robustness and generalization ability for faults of different sources, different operating conditions, and single / composite faults.

[0103] Table 5. Results of generalization experiments on public datasets

[0104] In summary, to address the challenges of extracting gear fault features under strong noise environments and the insufficient accuracy and generalization of traditional diagnostic models, this invention proposes a gear fault diagnosis method based on PGA-FMD-MTF and STIM multimodal fusion. The method has been validated using measured data and publicly available datasets. The PGA-FMD method of this invention, which uses envelope entropy as the fitness function, can achieve adaptive global optimization of key FMD parameters, effectively solving the inherent defects of traditional manual parameter tuning, significantly suppressing the modal aliasing problem, and realizing high-precision decomposition and noise removal of gear fault vibration signals.

[0105] Meanwhile, the time-frequency domain comprehensive evaluation factor algorithm proposed in this invention can adaptively select MF components that are sensitive to fault information, while taking into account the time-domain impact characteristics and frequency-domain correlation of the components, avoiding the subjective influence of human threshold setting, and effectively enhancing the characterization ability of fault features; the MTF time-series-image conversion method can completely preserve the time-series dependence and state evolution information of the signal, and the generated time-frequency image has extremely high fault visual discrimination, strong anti-noise interference ability and high computational efficiency, laying a reliable foundation for subsequent visual feature extraction;

[0106] The STIM dual-branch fusion model constructed in this invention takes into account both the local visual features of the fault and the global long-term temporal dependencies. It achieves adaptive deep fusion of multimodal features through the MHA mechanism, achieving 100% diagnostic accuracy for single gear faults. The average accuracy on composite fault datasets reaches 99.25%, which has significant performance advantages over traditional methods, and also has excellent generalization ability and robustness.

[0107] The method proposed in this invention has achieved excellent diagnostic results in gear fault diagnosis. Future research will address the resource constraints of edge devices in industrial settings by employing knowledge distillation, model pruning, and quantization techniques to achieve lightweight model design, thereby promoting the edge deployment and engineering application of the method. The method will also be extended to gear fault diagnosis under complex operating conditions such as variable speed and variable load, improving its adaptability under non-stationary conditions.

[0108] In some embodiments, the present invention proposes a gear fault diagnosis system based on multimodal data fusion and STIM model, comprising:

[0109] The signal acquisition module is used to acquire the raw vibration signals of the gearbox;

[0110] The signal preprocessing module is used to perform signal decomposition, sensitive mode screening, and reconstruction operations from S1 to S2.

[0111] The image conversion module is used to perform the Markov transform field image conversion operation in step S3;

[0112] The multimodal fusion diagnostic module is used to perform STIM model feature extraction, fusion, and fault classification operations from S4 to S5.

[0113] This invention is applicable to fault diagnosis of various gearbox equipment, including but not limited to: wind turbine gearboxes, rail transit vehicle gearboxes, marine propulsion system gearboxes, mining machinery gearboxes, and metallurgical equipment gearboxes. The method and system of this invention can be integrated into equipment condition monitoring systems or portable fault diagnosis instruments to achieve online monitoring and intelligent fault diagnosis of key gearbox components.

[0114] The above description is a further detailed explanation of the present invention in conjunction with specific preferred embodiments. For those skilled in the art, several simple deductions or substitutions can be made without departing from the present invention, and all such deductions or substitutions should be considered as falling within the scope of patent protection determined by the submitted claims.

Claims

1. A gear fault diagnosis method based on multimodal fusion and STIM, comprising the following steps: S1: Collect the original vibration signal of the gearbox and use the photo-growing optimization algorithm PGA to optimize the key parameters of the eigenmode decomposition (FMD) to obtain the optimal parameter combination; S2: The original vibration signal is decomposed by FMD using the optimal parameter combination to obtain modal components. The sensitive modal component discrimination algorithm based on the time-frequency domain comprehensive evaluation factor is applied to automatically filter the components containing the main fault characteristics to obtain the noise-reduced and enhanced fault signal. S3: The above fault signal is converted into a two-dimensional time-frequency image using Markov Transformed Field MTF; S4: Construct a two-branch deep learning STIM model, including: The first branch: The visual feature extraction branch adopts the shifted window Swin Transformer model, which includes a hierarchical architecture, a shifted window self-attention mechanism and cross-scale relative position bias, and is used to extract local and global multi-scale visual features of MTF images. The second branch: a branch for extracting temporal features using a long-sequence temporal model, which includes a sparse self-attention mechanism and a generative decoder, is used to capture the long-range temporal dependence features of vibration signals; The visual features and temporal features extracted from the two branches above are input into the multi-head attention mechanism (MHA) module. By calculating the feature correlation of different subspaces, adaptive fusion of multimodal features is achieved. S5: Input the features fused by the MHA module into the fully connected layer and the Softmax classifier, and finally output the gear fault type identification result.

2. The gear fault diagnosis method based on multimodal fusion and STIM according to claim 1, characterized in that: In step S1, the envelope entropy of the original vibration signal is used as the fitness function, and the PGA algorithm is used to globally optimize the number of modes n and the filter length L of the FMD to obtain the optimal parameter combination.

3. The gear fault diagnosis method based on multimodal fusion and STIM according to claim 2, characterized in that: In step S2, the sensitive modal component discrimination algorithm based on the time-frequency domain comprehensive evaluation factor comprehensively calculates the time-domain kurtosis value of each component and the frequency-domain cross-correlation coefficient with the original signal, constructs a weighted comprehensive evaluation factor, and automatically determines the sensitive components through the principle of maximum difference, and accumulates and reconstructs the selected sensitive components.

4. The gear fault diagnosis method based on multimodal fusion and STIM according to claim 2, characterized in that: The principle of maximizing the difference in step S2 is as follows: arrange each modal component in descending order of the comprehensive evaluation factor, calculate the difference between adjacent comprehensive evaluation factors, take the position of the maximum difference as the boundary point between the sensitive modal component and the interference modal component, and select all modal components before the boundary point as the sensitive modal components.

5. The gear fault diagnosis method based on multimodal fusion and STIM according to claim 3, characterized in that: In step S3, the fault signal is converted into a two-dimensional time-frequency image. The gear vibration signal is then decomposed and reconstructed using PGA-FMD, and quantile division and state transition probability calculation are performed. The temporal dependency of the signal is mapped to the pixel spatial information of the image, generating an MTF image with high fault discrimination.

6. The gear fault diagnosis method based on multimodal fusion and STIM according to claim 1, characterized in that: The Swin Transformer model described in S4 adopts a hierarchical coding architecture, which progressively downsamples the input image through a multi-stage shift window self-attention module and extracts local texture features and global semantic features of the image at different scales; the cross-scale relative position bias is used to align and fuse position information at different scales between each stage.

7. The gear fault diagnosis method based on multimodal fusion and STIM according to claim 1, characterized in that: The ProbSparse sparse self-attention mechanism of the Informer model in S4 calculates the sparsity metric of each query, selects several active queries with the highest sparsity scores, and performs dot product attention calculation only on the active queries to reduce the computational complexity of long sequence modeling.

8. The gear fault diagnosis method based on multimodal fusion and STIM according to claim 1, characterized in that: The fault types in S5 include: healthy condition, tooth surface wear, tooth root crack, broken tooth, and tooth surface pitting.

9. A gear fault diagnosis system based on multimodal data fusion and STIM model, characterized in that: include: The signal acquisition module is used to acquire the raw vibration signals of the gearbox; The signal preprocessing module is used to perform the signal decomposition, sensitive mode screening and reconstruction operations described in S1 to S2; The image conversion module is used to perform the Markov transformation field image conversion operation described in S3; The multimodal fusion diagnostic module is used to perform the STIM model feature extraction, fusion, and fault classification operations described in S4 to S5.