A rv reducer fault diagnosis method based on wgan-rag and scvit multi-modal information fusion

Abstract

This invention discloses a fault diagnosis method for RV reducers based on WGAN-RAG and SCViT multimodal information fusion, belonging to the field of mechanical fault diagnosis technology. The method includes: acquiring vibration and current dual-mode signals of the RV reducer under multiple operating conditions; proposing a small-sample data augmentation algorithm based on WGAN-RAG; converting the augmented data into a time-frequency image using short-time Fourier transform; and proposing a dual-branch fault diagnosis model based on SCViT. This model performs preliminary feature extraction through parameter-sharing convolutional layers, then feeds the extracted vibration features into a self-calibrating convolutional branch to extract the global state features of the RV reducer, and feeds the current features into a ViT branch to extract the local detail features of the reducer. A dynamic weight allocation module is designed to fuse the dual-mode features and generate the final fault category probability. Compared with other methods, this invention maintains high fault diagnosis accuracy even with a small number of samples and under different operating conditions.

Description

Technical Field

[0001] This invention relates to the field of mechanical fault diagnosis technology, specifically to a fault diagnosis method for RV reducers based on the fusion of WGAN-RAG and SCViT multimodal information. Background Technology

[0002] With the rapid development of automation technology, industrial robots have become the core of intelligent manufacturing, and their application in the manufacturing industry is becoming increasingly widespread. As a core component, the RV reducer of a robot joint directly affects the performance and safety of the entire machine, and is a crucial factor influencing the long-term stable operation of industrial robots. Therefore, research on fault diagnosis of RV reducers is of great significance in the engineering field. Traditional mechanical fault diagnosis methods typically rely on a single data source, such as vibration signals, temperature data, or current data. However, a single data source often cannot comprehensively and accurately reflect the fault characteristics, easily leading to misdiagnosis or missed diagnosis. Therefore, methods based on multimodal data fusion have gradually become a research hotspot. By comprehensively considering multiple data sources and examining the equipment's operating status from different perspectives, the accuracy and reliability of diagnosis can be effectively improved. Therefore, timely acquisition of comprehensive joint status information has significant theoretical and practical value.

[0003] With the increasing complexity and intelligence of industrial equipment, traditional fault diagnosis methods can no longer meet the demands for high precision and efficiency. Data-driven fault diagnosis methods, as an emerging technology, have received widespread attention in recent years. This method does not rely on precise mechanistic models or complete expert knowledge, and has made significant progress in fields such as vibration signal analysis, acoustic emission monitoring, and thermal imaging. It is also widely used in high-demand fields such as industrial robotics and aerospace.

[0004] Compared to single data sources, multi-source data fusion has significant advantages in diagnosing complex joint faults in industrial robots. It not only improves the accuracy and reliability of fault diagnosis but also promotes a deeper understanding of the fault mechanisms, further enhancing the accuracy and reliability of fault diagnosis. With the application of emerging technologies such as deep learning, data fusion capabilities are increasingly enhanced, providing new impetus and direction for the development of complex fault diagnosis technology. Therefore, this invention proposes a fault diagnosis method for RV reducers based on the fusion of WGAN-RAG and SCViT multimodal information. Summary of the Invention

[0005] Technical problems to be solved: To address the shortcomings of existing technologies, this invention provides a fault diagnosis method for RV reducers based on the fusion of WGAN-RAG and SCViT multimodal information.

[0006] Technical solution: To achieve the above technical solution, the following steps are included: S1. Collect periodic vibration signals and periodic current signals of the RV reducer of the industrial robot under different faults and working conditions to obtain a dataset of vibration signals and current signals of RV reducer faults. S2. Create an original small-sample imbalanced dataset and input the created dataset into the trained WGAN-RAG (Wasserstein Generative Adversarial Network with Residual and Attention Guidance) model to perform data augmentation. To address the common problem of imbalanced small samples in industrial fault diagnosis, an original imbalanced small sample dataset was constructed to simulate the challenges of data sparsity and class imbalance in real-world applications. An original imbalanced small sample dataset was artificially created and input into the trained WGAN-RAG model. The WGAN-RAG network specifically includes improvements to the generator and discriminator. Regarding the generator, improvements include: Introducing two layers of upsampling to increase feature length helps capture spatial hierarchy and local patterns in the data, resulting in more coherent and realistic output; By incorporating a two-layer residual network to alleviate the gradient vanishing problem in deep networks, and further promoting gradient flow and stable training through residual blocks, the generator's ability to learn complex features is improved. The generator integrates a multi-head self-attention mechanism to enable the model to capture the correlations and overall structural information between different temporal positions and feature dimensions in the data; The random noise input to the generator passes through a fully connected layer and then enters a section consisting of a one-dimensional convolutional layer (Conv 1d), batch normalization (BN), ReLU, and an upsampling module. This stage includes two upsampling layers. The upsampled data then passes through a two-layer residual network, which consists of two cascaded residual network units, each containing skip connections to aid in deep feature extraction and address gradient minimization. Finally, a multi-head attention mechanism is introduced, and the data is passed through a fully connected layer and a linear activation function to form the output. For the single-head attention (which serves as the basis for multi-head attention) in the generator improvement, the calculation process is as follows: In the formula, Indicates original features; , and They represent queries respectively. ,key Sum The learnable weight matrix; It is the scaling factor, where, The dimension of the key vector; the output is a weighted value vector. The weights are determined by the attention score; For multi-head self-attention mechanisms, the results of multiple single-head attention are concatenated and linearly transformed, as expressed by: In the formula, This represents a learnable linear projective weight matrix; ,in, , , , , and These represent the learnable projection matrix corresponding to each head; Ultimately, by combining the output of multi-head attention with the original features Weighted mixing is performed to obtain the output features. The expression is as follows: In the formula, The mixing coefficient between attention and original features; Regarding the discriminator, improvements include: Introduce a dropout mechanism to prevent overfitting; The discriminator takes input from the real signal distribution and the generated signal distribution, and consists of multiple linear layers. Overfitting is prevented by Dropout, and the output is a score of the realism of the input signal. WGAN-RAG uses Wasserstein distance as the loss function, which has better training stability and convergence compared to the JS divergence of traditional GANs; Meanwhile, a gradient penalty term is introduced to force the gradient norm of the discriminator to be close to 1, in order to satisfy the Lipschitz continuity condition required by WGAN, further improving the stability of training and the quality of generated data; the generator and discriminator compete against each other during training and optimize together.

[0007] S3. Based on S2, the one-dimensional data expanded by the WGAN-RAG network is converted into a two-dimensional time-frequency image fault dataset through short-time Fourier transform (STFT). The Short Time Fourier Transform (STFT) maps the local spectral characteristics of a time series to a two-dimensional time-frequency graph, intuitively presenting the joint time-frequency features of the original signal; for a continuous-time signal... The basic formula for calculating its continuous short-time Fourier transform can be expressed as: In the formula, Represents the original signal; Indicates time; Represents the integral variable; Indicates frequency; Indicates a The window function centered at [center] has a length that affects both time and frequency resolution. A longer window increases frequency resolution but decreases time resolution. The choice of window length must consider signal characteristics for accurate analysis. The formula for calculating time resolution is as follows: In the formula, Indicates the length of the signal to be processed; This represents the overlap length during the translation process of the window function; Indicates the length of the window function; The formula for calculating frequency resolution is as follows: In the formula, even represents an even number; odd represents an odd number.

[0008] S4. Divide the two-dimensional time-frequency image fault dataset obtained in S3 into a training set and a test set according to a preset ratio, and build a dual-branch network model (A Convolution-Vision Transformer Fusion Network for Dual-Modality Classification, SCViT). Use the training set as training samples to train the model. The two-dimensional time-frequency image fault dataset obtained by S3 includes: a two-dimensional time-frequency image set of vibration signals and current signals of the RV reducer; The two-branch network model includes two branches: SCConv and ViT. The method for building a two-branch network model is as follows: The vibration time-domain image is used as input, and a convolutional neural network (CNN) is used as the initial feature extraction layer. A self-calibrated convolutional module (SCConv) is used for global feature extraction. The current time-domain graph is used as input, and a convolutional neural network (CNN) is used as the initial feature extraction layer. The Vision Transformer (ViT) module is used for local feature extraction. In the dual-branch network model, the convolutional neural network (CNN) is a 4-layer CNN convolutional neural network with shared parameters; During training, the learning rate and dropout parameters are dynamically adjusted to prevent overfitting.

[0009] S5. Based on S4, the dual-branch network model SCViT performs a feature merging operation on the features processed independently by the SCConv branch and the ViT branch to obtain the final diagnostic result. The feature merging method is as follows: add the dimension of the SCConv branch to the dimension of the ViT branch, perform the first dimensionality reduction operation through a linear layer after adding the dimension, and then perform the second dimensionality reduction operation through a linear layer after passing the activation function and the preset value dropout, until the number of dimensions is reduced to the same as the fault category. The final diagnostic result is obtained by mapping the merged features to a 7-dimensional fault category space through two fully connected layers, and then outputting a normalized probability distribution using the Softmax function, as shown in the following expression: In the formula, Indicates the input signal Fault category The normalized probability; This represents the feature vector after merging the two branches; This represents the weight parameters of the fully connected layer; Indicates the fault category, where, An index representing the fault category.

[0010] Compared with existing technologies, this invention provides a fault diagnosis method for RV reducers based on the fusion of WGAN-RAG and SCViT multimodal information, which has the following advantages: 1. The purpose of this invention is to address the problems of scarce fault samples and complex coupling of multi-source signals in RV reducers in industrial scenarios. By constructing a collaborative framework of generative adversarial networks and multi-scale feature extraction, it overcomes the limitations of traditional single-mode diagnosis. This method achieves cross-domain feature alignment and collaborative enhancement of vibration-current signals, providing high-quality data for RV reducer condition assessment under complex operating conditions, and simultaneously constructing an interpretable decision-diagnostic model for predictive maintenance of industrial equipment.

[0011] 2. This invention constructs a WGAN-RAG model for data augmentation, aiming to expand the original dataset with high quality and effectively preserve the temporal characteristics of the signal.

[0012] 3. This invention designs an SCViT dual-branch diagnostic network. This network first converts WGAN-RAG-enhanced vibration and current data into time-frequency images using a short-time Fourier transform (STFT). These images are then processed by a parameter-shared convolutional neural network (CNN) module for preliminary feature extraction. Subsequently, the self-calibrating convolution (SCConv) module in the vibration signal processing branch effectively captures the global state features of the RV reducer; while the VisionTransformer (ViT) module in the current signal processing branch utilizes a self-attention mechanism to analyze the local state features of the reducer. Finally, the fused features are input into a classifier to achieve fault diagnosis. This invention integrates the high-quality data generation capability of WGAN-RAG, the global feature capture advantage of SCConv, and the local detail perception expertise of ViT, forming a multimodal fusion space that combines global structure analysis and local detail perception. Compared with existing state-of-the-art diagnostic models, this effectively improves the fault characterization capability of RV reducer fault features under complex operating conditions. Attached Figure Description

[0013] Figure 1 This is a flowchart of the steps of the present invention; Figure 2 This is a schematic diagram of the current signal according to an embodiment of the present invention. Figure 2 (a) is a schematic diagram of the overall current signal. Figure 2 (b) is a schematic diagram of the current signal section; Figure 3 This is a schematic diagram of vibration signals according to an embodiment of the present invention. Figure 3 (a) is a schematic diagram of the overall vibration signal. Figure 3 (b) is a schematic diagram of the vibration signal portion; Figure 4 This is a distribution diagram of the probability density function of each vibration signal in an embodiment of the present invention; Figure 5 This is a distribution diagram of the probability density function of each current signal in an embodiment of the present invention; Figure 6 This is a time-domain signal diagram of the short-time Fourier transform according to an embodiment of the present invention. Figure 6 (a) is the time-domain signal diagram of the short-time Fourier transform of the vibration signal. Figure 6 (b) is a time-domain signal diagram of the short-time Fourier transform of the current signal; Figure 7 The graph shows the diagnostic accuracy of this invention under different operating conditions. Figure 7 (a) is a curve showing the diagnostic accuracy for operating condition 1. Figure 7 (b) A curve graph showing the diagnostic accuracy for operating condition 2. Figure 7 (c) A curve graph showing the diagnostic accuracy for operating condition 3; Figure 8 This is the t-SNE diagram for the vibration-current fusion diagnosis of this invention; Figure 9 This is the t-SNE diagram of the vibration signal of this invention; Figure 10 This is the t-SNE diagram of the current signal of this invention; Figure 11 This is a comparison chart of the diagnostic results of the SCViT diagnostic model of the present invention with other existing diagnostic models under different operating conditions. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] This invention addresses key issues in RV reducer fault diagnosis, such as significant heterogeneity of multi-source data, insufficient feature representation ability under small sample conditions, and weak cross-modal information complementarity. It proposes an RV reducer fault diagnosis method that integrates a WGAN-RAG (Wasserstein Generative Adversarial Network with Residual and Attention Guidance) data augmentation model and a SCViT (A Convolution-Vision Transformer Fusion Network for Dual-Modality Classification) dual-branch diagnostic network. The SCViT network innovatively integrates a parameter-sharing convolutional neural network (CNN) as the initial feature extraction layer, and further combines a self-calibrated convolutional module (SCConv) for vibration signal feature extraction, and a Vision Transformer (ViT) module for current signal feature extraction.

[0016] Example: refer to Figures 1-11 A fault diagnosis method for RV reducers based on WGAN-RAG and SCViT multimodal information fusion is proposed, which includes the following steps: S1. Collect periodic vibration signals and periodic current signals of the RV reducer of the industrial robot under different faults and working conditions to obtain a dataset of vibration signals and current signals of RV reducer faults. In this embodiment, artificial defects were created on the sun gear and planetary gears on a self-made RV reducer test bench using an electro-thermal process. These defects included three damage sizes (0.1mm / 0.3mm / 0.5mm) and two fault types: crack-type and wear-type. This resulted in six typical fault modes: sun gear root crack, planetary gear root crack, single-tooth surface wear of the sun gear, multi-tooth surface wear of the sun gear, single-tooth surface wear of the planetary gear, and multi-tooth surface wear of the planetary gear. These, combined with the normal state, constituted seven operating states of the RV reducer. The reducer operated under three conditions during the experiment: Condition 1: The reducer output shaft rotated continuously in one direction for a full revolution, with no additional load; Condition 2: The reducer output shaft maintained a 90° reciprocating oscillation, with an additional load of 8kg; Condition 3: The reducer output shaft maintained a 90° reciprocating oscillation, with no additional load. As a further aspect of the present invention, in S1: This embodiment takes the RV reducer of the joint of the swing arm robot as the research object, and its specific parameters are shown in Table 1. Time-domain plots of cyclic vibration and current signals during joint failure in a swing-arm robot, as shown below. Figure 2 middle, Figure 2 (a) and Figure 2 (b) Figure 3 middle, Figure 3 (a) and Figure 3 (b) Figure 4 and Figure 5 As shown, the amplitude of the vibration acceleration of the RV reducer changes unevenly, indicating that the RV reducer has malfunctioned. Table 1: Test Platform Parameters S2. Create an original small-sample imbalanced dataset and input the created dataset into the trained WGAN-RAG (Wasserstein Generative Adversarial Network with Residual and Attention Guidance) model to perform data augmentation. To verify the effectiveness of the algorithm, an original small-sample imbalanced dataset was artificially created and input into the trained WGAN-RAG model. The WGAN-RAG network specifically includes improvements to the generator and discriminator. Regarding the generator, improvements include: Introducing two layers of upsampling to increase feature length helps capture spatial hierarchy and local patterns in the data, resulting in more coherent and realistic output; By incorporating a two-layer residual network to alleviate the gradient vanishing problem in deep networks, and further promoting gradient flow and stable training through residual blocks, the generator's ability to learn complex features is improved. The generator integrates a multi-head self-attention mechanism to enable the model to capture long-term dependencies and global context; The random noise input to the generator passes through a fully connected layer and then enters a section consisting of a one-dimensional convolutional layer (Conv 1d), batch normalization (BN), ReLU, and an upsampling module. This stage includes two upsampling layers. The upsampled data then passes through a two-layer residual network, which consists of two cascaded residual network units, each containing skip connections to aid in deep feature extraction and address gradient minimization. Finally, a multi-head attention mechanism is introduced, and the data is passed through a fully connected layer and a linear activation function to form the output. For the single-head attention (which serves as the basis for multi-head attention) in the generator improvement, the calculation process is as follows: In the formula, Indicates original features; , and They represent queries respectively. ,key Sum The learnable weight matrix; It is the scaling factor, where, The dimension of the key vector; the output is a weighted value vector. The weights are determined by the attention score; For multi-head self-attention mechanisms, the results of multiple single-head attention are concatenated and linearly transformed, as expressed by: In the formula, This represents a learnable linear projective weight matrix; ,in, , , , , and These represent the learnable projection matrix corresponding to each head; Ultimately, by combining the output of multi-head attention with the original features Weighted mixing is performed to obtain the output features. The expression is as follows: In the formula, The mixing coefficient between attention and original features; Regarding the discriminator, improvements include: Introduce a dropout mechanism to prevent overfitting; The discriminator takes input from the real signal distribution and the generated signal distribution, and consists of multiple linear layers. Overfitting is prevented by Dropout, and the output is a score of the realism of the input signal. WGAN-RAG uses Wasserstein distance as the loss function, which has better training stability and convergence compared to the JS divergence of traditional GANs; Meanwhile, a gradient penalty term is introduced to force the gradient norm of the discriminator to be close to 1, in order to satisfy the Lipschitz continuity condition required by WGAN, further improving the stability of training and the quality of generated data; the generator and discriminator compete against each other during training and optimize together.

[0017] S3. Based on S2, the one-dimensional data expanded by the WGAN-RAG network is converted into a two-dimensional time-frequency image fault dataset through short-time Fourier transform (STFT). like Figure 6 middle Figure 6 (a) and Figure 6 As shown in (b), the Short Time Fourier Transform (STFT) maps the local spectral characteristics of a time series into a two-dimensional time-frequency diagram, intuitively presenting the joint time-frequency characteristics of the original signal; for a continuous-time signal The basic formula for calculating its continuous short-time Fourier transform can be expressed as: In the formula, Represents the original signal; Indicates time; Represents the integral variable; Indicates frequency; Indicates a The window function centered at [center] has a length that affects both time and frequency resolution. A longer window increases frequency resolution but decreases time resolution. The choice of window length must consider signal characteristics for accurate analysis. The formula for calculating time resolution is as follows: In the formula, Indicates the length of the signal to be processed; This represents the overlap length during the translation process of the window function; Indicates the length of the window function; The formula for calculating frequency resolution is as follows: In the formula, even represents an even number; odd represents an odd number.

[0018] S4. Divide the two-dimensional time-frequency image fault dataset obtained in S3 into a training set and a test set according to a preset ratio, and build a dual-branch network model (A Convolution-Vision Transformer Fusion Network for Dual-Modality Classification, SCViT). Use the training set as training samples to train the model. The two-dimensional time-frequency image fault dataset obtained by S3 includes: a two-dimensional time-frequency image set of vibration signals and current signals of the RV reducer; In this embodiment, the preset ratio is 75% training set and 25% test set; The two-branch network model includes two branches: SCConv and ViT. The method for building a two-branch network model is as follows: The vibration time-domain image is used as input, and a convolutional neural network (CNN) is used as the initial feature extraction layer. A self-calibrated convolutional module (SCConv) is used for global feature extraction. The self-calibrating convolution module takes the input feature map Split into and , is represented as: The self-calibration pathway performs multi-scale context extraction and feature calibration operations; the spatial context pathway preserves the original spatial information. Convolution kernel in the self-calibrating convolution module It was split into: , , and Four parts; among them, Used for multi-scale context extraction; Used for feature calibration; Used to preserve the original spatial information in standard convolution; Features used for residual connection fusion calibration; The self-calibrating method in the self-calibrating convolution module is as follows: the self-calibrating convolution performs feature transformation between the original scale space and the downsampled latent space (with a larger receptive field). The latter serves as a reference to guide the feature transformation process in the original feature space. The steps include: multi-scale space transformation operation, adaptive feature calibration operation, feature construction operation, and original feature integration operation, where: Multi-scale spatial transformation operation: First, the self-calibration path adjusts the input... Downsampling is performed, and the corresponding self-calibrating convolutional part is labeled. The operation expands the receptive field, followed by convolution. Feature extraction is performed, followed by upsampling. The system restores the signal to its original scale. This multi-scale spatial transformation is crucial for capturing overall time-frequency correlation patterns in vibration signals. For example, for distributed faults such as multi-tooth wear on the sun gear, the effects span multiple meshing cycles. By expanding the receptive field, SCConv can effectively integrate these cross-time and cross-frequency patterns.

[0019] Adaptive feature calibration operation: Adaptive feature calibration is the core of self-calibrating convolution, which involves adjusting the feature model from the model itself. The multi-scale time-frequency information extracted is locally weighted, and The output after multi-scale spatial transformation and the output after convolution operation Element-wise addition is performed, then a calibration weight map is generated using the sigmoid activation function. Finally, this weight map is convolved with the weight map obtained from the convolution operation. Processing The calibrated features are obtained by performing element-wise multiplication. : in, This indicates element-wise operations, where σ is the Sigmoid activation function.

[0020] Adaptive feature calibration uses the global time-frequency information extracted from path 1 as a reference to adaptively weight and correct the original scale features retained in path 2, which enables the model to highlight the features in the vibration signal that are more relevant to the fault.

[0021] Finally, a dual-path fusion operation is performed, and path 1, after self-calibration, is output. and after convolution operation The original spatial information is retained and the output path 2 is used. By concatenating and merging the expressions, the final output expression is as follows: SCConv, through its calibration operation, can adaptively integrate information from the surrounding and broader time-frequency regions and model inter-channel dependencies. This allows it to effectively expand the receptive field, encode more precise discriminative regions, and effectively avoid interference from irrelevant regions.

[0022] The current time-domain graph is used as input, and a convolutional neural network (CNN) is used as the initial feature extraction layer. The Vision Transformer (ViT) module is used for local feature extraction. In the Vision Transformer branch, ViT divides the current time-frequency map into non-overlapping local blocks, each of which is independently encoded as a feature vector, directly corresponding to the transient characteristics of a specific time window and frequency band in the signal; the introduction of position encoding completely preserves the spatiotemporal position information of the features, ensuring that the time and frequency band location of the fault can be accurately located; the self-attention mechanism dynamically establishes cross-regional correlations, enabling local transient features to activate relevant harmonic components and construct a complete fault feature representation network; In the dual-branch network model, the convolutional neural network (CNN) is a 4-layer CNN convolutional neural network with shared parameters; During training, the learning rate and dropout parameters are dynamically adjusted to prevent overfitting.

[0023] S5. Based on S4, the dual-branch network model SCViT performs a feature merging operation on the features processed independently by the SCConv branch and the ViT branch to obtain the final diagnostic result. The feature merging method is as follows: add the dimension of the SCConv branch to the dimension of the ViT branch, perform the first dimensionality reduction operation through a linear layer after adding the dimension, and then perform the second dimensionality reduction operation through a linear layer after passing the activation function and the preset value dropout, until the number of dimensions is reduced to the same as the fault category. In this embodiment, the SCConv branch has a dimension of 912600; the ViT branch has a dimension of 7; the dimension after the first dimensionality reduction operation is 265; the ReLU activation function and a dropout rate of 0.5 are used as the exit mechanism; the dimension obtained by the second dimensionality reduction operation is 7. The final diagnostic result is obtained by mapping the merged features to a 7-dimensional fault category space through two fully connected layers, and then outputting a normalized probability distribution using the Softmax function, as shown in the following expression: In the formula, Indicates the input signal Fault category The normalized probability; This represents the feature vector after merging the two branches; This represents the weight parameters of the fully connected layer; Indicates the fault category, where, An index representing the fault category.

[0024] To compare the advantages of source data fusion with single-type diagnostic data, current data and vibration data were used for diagnosis separately, and the results were compared with those of multi-source data fusion diagnosis. Figure 7 middle Figure 7 (a) Figure 7 (b) and Figure 7 (c) Diagnostic curves for each method under different operating conditions. In operating condition 1, the accuracy of the blue vibration single-mode curve reached 94.57%, the accuracy of the green current single-mode curve stabilized at 91.56%, while the accuracy of the red dual-mode curve stabilized rapidly at 97.4% after a short iteration cycle; the vibration single-mode curves in operating conditions 2 and 3 showed oscillations, while the accuracy of the red dual-mode curve stabilized at 97.83% and 96.54%, respectively.

[0025] The T-SNE method is used to extract model features and reduce dimensionality, such as... Figure 8 , Figure 9 and Figure 10 As shown, when using multi-source data fusion for diagnosis, the clustering and separation of fault features are obvious, with clear boundaries, demonstrating good feature extraction performance. The boundaries between fault features are relatively far apart. Conversely, when using single-type data for diagnosis, there is some overlap between features, leading to lower diagnostic accuracy. A comparison with different state-of-the-art models for diagnosing equipment defects under different operating conditions is provided. Figure 11 As shown in Table 2: Table 2: Final Fusion Results Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A fault diagnosis method for RV reducers based on WGAN-RAG and SCViT multimodal information fusion, characterized in that: Includes the following steps: S1. Collect periodic vibration signals and periodic current signals of the RV reducer of the industrial robot under different faults and working conditions to obtain the RV reducer fault mixed mode dataset. S2. Create an original small-sample imbalanced dataset and input the created dataset into the trained WGAN-RAG model to perform data augmentation. The WGAN-RAG network includes an improved generator and an improved discriminator; Improvements to the generator include: the introduction of two-layer upsampling operations, the addition of a two-layer residual network, and the use of a multi-head self-attention mechanism; The discriminator is improved by introducing an exit mechanism; S3. Based on S2, the one-dimensional data expanded by the WGAN-RAG network is converted into a two-dimensional time-frequency image fault dataset through short-time Fourier transform. S4. Divide the two-dimensional time-frequency image fault dataset obtained in S3 into a training set and a test set according to a preset ratio, and build a dual-branch network model. Use the training set as training samples to train the model. The dual-branch network includes: a convolutional neural network with an SCConv branch and a convolutional neural network with a ViT branch; The convolutional neural network is a 4-layer CNN network with shared parameters; The method for building a two-branch network model is as follows: The vibration time-domain image is used as input, and it is passed through a convolutional neural network (CNN) as the initial feature extraction layer, and then a self-calibrating convolutional module is used for global feature extraction. The current time-domain graph is used as input, and a convolutional neural network is used as the initial feature extraction layer, and the ViT module is used for local feature extraction. S5. Based on S4, the dual-branch network model SCViT performs a feature merging operation on the features processed independently by the SCConv branch and the ViT branch to obtain the final diagnostic result.

2. The RV reducer fault diagnosis method based on WGAN-RAG and SCViT multimodal information fusion as described in claim 1, characterized in that: Based on S2, the one-dimensional data augmented by the WGAN-RAG network is converted into a two-dimensional time-frequency image fault dataset using short-time Fourier transform. Short-time Fourier transform maps the local spectral characteristics of the time series to a two-dimensional time-frequency image, intuitively presenting the joint time-frequency characteristics of the original signal. For a continuous-time signal, the basic calculation formula for its continuous short-time Fourier transform is expressed as: In the formula, Represents the original signal; Indicates time; Represents the integral variable; Indicates frequency; Indicates a The window function centered on; It represents the imaginary unit.

3. The RV reducer fault diagnosis method based on WGAN-RAG and SCViT multimodal information fusion as described in claim 1, characterized in that: The two-dimensional time-frequency image fault dataset obtained in S3 is divided into training set and test set according to a preset ratio, and a dual-branch network model is built. The training set is used as training samples for model training. The two-dimensional time-frequency image fault dataset obtained in S3 includes: two-dimensional time-frequency image set of vibration signal and current signal of RV reducer.

4. The RV reducer fault diagnosis method based on WGAN-RAG and SCViT multimodal information fusion as described in claim 1, characterized in that: Based on S4, the dual-branch network model SCViT performs a feature merging operation on the features processed independently by the SCConv branch and the ViT branch to obtain the final diagnostic result. The feature merging method is as follows: the dimension of the SCConv branch is added to the dimension of the ViT branch, the added dimension is passed through a linear layer to perform the first dimensionality reduction operation, and after passing through the activation function and the preset value dropout, the second dimensionality reduction operation is performed through a linear layer to reduce the number of dimensions to the same as the fault category. The final diagnostic result is obtained by mapping the merged features to a fault category space of a preset number of dimensions through two fully connected layers, and outputting a normalized probability distribution through the Softmax function.

Images