A UAV motor fault diagnosis method based on multimodal data fusion

Abstract

The present invention relates to the field of fault diagnosis technology, and in particular to a method for diagnosing motor faults of unmanned aerial vehicles (UAVs) based on multimodal data fusion. The method comprises extracting deep modal features from vibration and current signals using a parallel five-layer one-dimensional CNN, wherein the first layer uses a 64×16 convolution kernel, and the subsequent layers are adjusted to a 3×1 convolution kernel and pooled; integrating features layer by layer through a multi-layer feature fusion module, generating a weight map through 3×3 convolution and Sigmoid, and splicing and fusion after dynamic feature modulation; adjusting the fusion feature dimension to adapt to the Transformer, introducing position encoding, and using a multi-head self-attention mechanism to model global dependencies; and finally classifying nine types of faults through a fully connected layer of a deep neural network. The present invention aims to solve the problems of incomplete acquisition of traditional single-modal fault diagnosis features, neglect of modal differences in early fusion in multimodal fusion, lack of inter-modal correlation information mining in late fusion, and insufficient long-distance dependency modeling capability of CNN.

Description

Technical Field

[0001] The present invention belongs to the technical field of fault diagnosis and relates to a method for diagnosing motor faults of unmanned aerial vehicles (UAVs) based on multimodal data fusion. Background Art

[0002] Electric drones, thanks to their high efficiency, flexibility, and environmental friendliness, are widely used in agriculture, logistics, 3D mapping, and other fields. As a core power component, the operating status of their motors directly impacts the safety and reliability of drones. Motor failures primarily manifest as abnormalities in components such as bearings, rotors, and stators, such as bearing outer race failures, rotor demagnetization, and stator interturn short circuits. With the advancement of sensor and data analysis technologies, fault diagnosis methods based on multimodal data fusion have become a research hotspot. By integrating complementary information from multiple sources, such as vibration and current, they can improve the comprehensiveness of fault signatures and diagnostic accuracy.

[0003] Traditional single-modal fault diagnosis methods such as those that rely solely on vibration signals or current signals are difficult to obtain comprehensive fault characteristics. For example, although vibration signals can reflect mechanical fault characteristics, they are not sensitive enough to electrical faults; current signals are easily disturbed by motor operating conditions, and the accuracy of single-modal diagnosis is limited. In the field of multimodal fusion, early fusion strategies such as directly cascading multi-sensor data ignore the distribution differences and timing feature differences of different modalities, resulting in reduced effectiveness of features after fusion. Late fusion strategies such as independent feature extraction and subsequent splicing fail to fully explore the inherent correlation between modalities, and the feature fusion depth is insufficient. In addition, traditional convolutional neural networks CNN When processing long sequence signals, it is difficult to model long-distance dependencies due to the limitations of local feature extraction, resulting in information loss in complex fault modes.

[0004] Existing research attempts to solve the above problems by improving fusion strategies and network structures. For example, patent CN117591915A is titled "UAV motor small sample fault diagnosis method driven by current signal data", which uses a wide range learning system to BLS and CNN Although the combination reduces the dependence on time series features, it only uses a single current mode and does not solve the problem of multimodal fusion. In terms of multimodal fusion, early fusion methods such as Xia proposed by others CNN The model directly integrates multiple vibration signals as channel input without considering modal differences; late fusion methods such as Wang et al. 1DCNN-VAF The algorithm processes vibration and acoustic signals in parallel, but lacks interaction between modalities. The intermediate fusion strategy improves the fusion effect through multi-layer feature interaction, such as Wan Multi-sensor information coupling network MICN The introduction of attention mechanism enhances feature association, but there is still the problem of insufficient utilization of modality-specific features.CNN Long-distance dependency modeling flaws, Transformer The architecture realizes global feature modeling through the self-attention mechanism. FTDCNN Model Combination CNN and Transformer , in the multi-layer feature fusion module MLF The introduction of attention mechanism in the algorithm effectively improves the global information capture capability. In existing methods, single modality diagnosis has one-sided information, and early and late fusion strategies are difficult to balance modality differences and association mining. CNN The architecture is not capable of processing long sequences. Therefore, a multimodal fusion model combining an intermediate fusion strategy with an attention mechanism is adopted. FTDCNN ,By preserving modal specific features, enhancing cross-modal correlation modeling and global dependency analysis, the complex ,problems in UAV motor fault diagnosis can be solved more ,effectively. Summary of the Invention

[0005] The present invention provides a UAV motor fault diagnosis method based on multimodal data fusion, which solves the problems of incomplete acquisition of traditional single-modal fault diagnosis features, early fusion ignoring modal differences in multimodal fusion, and lack of inter-modal correlation information mining in late fusion. CNN The problem of insufficient ability to model long-distance dependencies.

[0006] In order to solve the above problems, the technical solution adopted by the invention is:

[0007] A method for diagnosing motor faults in a UAV based on multimodal data fusion includes the following steps:

[0008] S01 Multimodal Parallel Feature Extraction: Converting the one-dimensional time series signal of the drone motor into , m=1 is the vibration signal, m=2 For the current signal, input the parallel five-layer one-dimensional convolutional neural network CNN , extracting deep modality-specific features through layer-by-layer convolution and pooling;

[0009] S02 Cross-modal aggregation of multi-layer feature fusion module: The multi-layer feature fusion module is used to integrate deep modality-specific features layer by layer in the intermediate stage, and then convoluted again after cascading with the output of the previous layer to form multi-level feature transfer;

[0010] S03 Transformer Global modeling of the attention mechanism: adjust the dimension of the fused feature F, , to adapt Transformer Input format, introducing position encoding PE Injection timing information:

[0011]

[0012] in is the feature dimension, is the sequence position, ∈[0, ] is the channel index, Represents the fused feature tensor after dimension adjustment; Represents a tensor dimension rearrangement operation, Represent the position encoding function, which is used to inject timing information, is the sequence position, is the channel index, and the query is calculated through the multi-head self-attention mechanism ( Q ),key( K ),value( V )matrix:

[0013]

[0014]

[0015] And calculate the attention score by dot product and normalization:

[0016]

[0017] in 、 、 Query （Q） ,key( K ),value( V ) matrix weight parameters, Represents the dimensions of the bond matrix

[0018] S04 Deep Neural Network Classification: Input the high-level features processed by the attention mechanism into the deep neural network, and classify nine types of faults through the fully connected layer. The loss function uses cross entropy loss; the nine types of faults include: normal operation C1 , bearing outer ring failure C2 , bearing retainer failure C3 , bearing rolling element failure C4 , bearing inner ring failure C5 , rotor demagnetization C6 , rotor shaft bending C7 , stator phase short circuit C8 , stator turn short circuit C9。

[0019] The principles and advantages of this solution are:

[0020] First, multimodal parallel feature extraction utilizes two independent five-layer one-dimensional CNN Process vibration signals separatelym=1 With current signal m=2 Through layer-by-layer convolution, the first layer uses a 64×16 convolution kernel to capture low-level temporal features. The number of filters in subsequent layers is increased to 64, and the convolution kernel is adjusted to 3×3 and 2×2 maximum pooling to extract deep specific features of each mode, such as mechanical impact features in vibration signals and electromagnetic anomaly features in current signals, while retaining the uniqueness of the mode. Secondly, the multi-layer feature fusion module MLF Achieve layer-by-layer aggregation of cross-modal features in the middle layer: through cross-enhancement strategy 3×3 convolution + Sigmoid Generate a weight graph to learn the correlation between modalities, use the cross-modal gating mechanism to dynamically modulate feature weights, strengthen complementary information and suppress redundancy; then through feature splicing and 3×3 convolution, realize the multi-level transmission of cross-modal shared features, and solve the problem that early fusion ignores modal differences and late fusion lacks association mining. Finally, Transformer The attention mechanism globally models the fusion features: injecting temporal information through dimension adjustment and position encoding PE, and using the multi-head self-attention mechanism to calculate the query ( Q ),key( K ), value (V) matrix, through dot product and normalization to capture long-range dependencies, such as the cross-modal correlation between periodic shocks in vibration signals and harmonic components in current signals, to make up for CNN The shortcomings of local modeling are finally solved through deep neural networks DNN Implement fault classification.

[0021] Compared with existing technologies, early fusion, such as directly concatenating vibration and current signals, ignores the differences in modal distribution. For example, the high-frequency impact of the vibration signal is incompatible with the low-frequency fluctuation characteristics of the current signal, resulting in feature confusion; late fusion, such as independently extracting features through a parallel network and then splicing the dynamic correlation between unmodeled modes, such as the coordinated changes of the increase in vibration amplitude and the increase in current harmonic components during bearing failure, results in low fusion efficiency.

[0022] MLF Through layer-by-layer cross-enhancement and gating mechanisms, cross-modal feature weights are dynamically adjusted. For example, in the case of bearing outer ring faults, C2 During diagnosis, the weight of the impact feature of the vibration signal is significantly increased, while the weight of the low-frequency feature caused by load fluctuations in the current signal is suppressed, avoiding the "feature conflict" problem of early fusion.

[0023] Through cross-layer cascading, shallow features such as the original signal’s time domain waveform and deep features such as the fault feature vector are gradually integrated to improve the richness of feature expression. C6 During diagnosis, MLF It can capture the cross-layer correlation between the decrease of the fundamental frequency component of the current signal and the high-frequency vibration caused by the abnormal electromagnetic force in the vibration signal, while the late fusion model FTDCNN-L Due to the lack of intermediate interaction,C6 The probability of diagnosing a bearing fault is as high as 15.46%, but this solution reduces the misdiagnosis rate to 0.02%.

[0024] Tradition CNN Relying on local convolution kernels makes it difficult to capture global dependencies in long-term time series signals, such as the cross-modal correlation between the weak vibration signal at the beginning of bearing wear and the gradually increasing noise in the current signal after a motor has been running for a long time. This solution uses multi-head attention to parallelly calculate the feature correlations in different subspaces to effectively model long-range dependencies. For example, in the case of a stator inter-turn short circuit, C9 In the early stage of the process, the tiny harmonic distortion in the current signal and the periodic low-frequency vibration caused by the magnetic field imbalance in the vibration signal can be associated through the attention mechanism. 1dCNN Due to local modeling limitations, the missed detection rate of such early faults is as high as 25.61%. This solution achieves an average accuracy of 99.96%.

[0025] By injecting timing information through sine / cosine position encoding, the model can distinguish the previous and next dependencies in the signal and avoid feature misjudgment caused by timing confusion. Ablation experiments show that after removing the attention mechanism, FTDCNN-WT , the inter-class overlap rate between bearing retainer fault C3 and rolling element fault C4 increases by 40%, verifying the necessity of attention mechanism for global modeling.

[0026] Compared with existing methods, this solution achieves comprehensive superiority in accuracy, stability, and generalization ability:

[0027] Accuracy: The average accuracy is 99.96%, which is better than the second best comparison method. FusionDBN , an increase of 1.09% to 98.87%, especially in the case of easily confused faults such as C3 and C4, the misjudgment rate was reduced by more than 90%.

[0028] Stability: The standard deviation is only 0.01%, which is significantly lower than other methods such as DRN The standard deviation is 4.26%, indicating that the model can maintain reliable performance under different working conditions.

[0029] Generalization capability: In a complex scenario with nine types of faults, the recognition accuracy of a small sample fault such as rotor shaft bending C7 is 100% with a sample size of only 10%, while the single-modal model FTDCNN-I The misjudgment rate for C7 is as high as 16.28%.

[0030] Furthermore, the vibration signal is transmitted through SAEC00X5 The piezoelectric acceleration sensor is used to collect the current signal, and the current signal is collected by the WCS1800 Hall current sensor. HK_USB6202_S The data acquisition card performs analog-to-digital conversion with a sampling frequency of 100kHz and 16-bit accuracy. The vibration signal is transmitted through SAEC00X5The piezoelectric acceleration sensor is used for data acquisition. Its high precision, high temperature stability and wide frequency response characteristics can accurately capture the weak fault characteristics and high-frequency impact signals in the motor vibration. The current signal is collected through WCS1800 The Hall effect and closed-loop technology can realize high-precision, high-resolution and high-bandwidth measurement, accurately reflecting the transient changes and electromagnetic anomalies of the motor current. HK_USB6202_S The data acquisition card performs analog-to-digital conversion with a sampling frequency of 100kHz and 16-bit accuracy, ensuring high-fidelity synchronous acquisition of multi-modal signals, avoiding feature aliasing caused by insufficient sampling frequency or insufficient quantization accuracy to mask weak signals, providing a basis for subsequent FTDCNN The model extracts modal-specific features and cross-modal complementary information, providing a high-quality data foundation and significantly improving the accuracy and reliability of fault diagnosis.

[0031] Furthermore, the data preprocessing includes: segmenting the original signal into 10,000 non-overlapping slices, extracting sub-slices containing 2,048 sampling points with a step size of 1, and normalizing the data distribution to generate a balanced sample set for training. The original signal's 10,000 sampling points are segmented into 10,000 non-overlapping slices to avoid feature aliasing between adjacent slices and ensure that each slice independently contains complete fault cycle information, such as the characteristic frequency period of a bearing fault. For example, for a bearing inner race fault with a characteristic frequency of 200 Hz and a period of 5 ms, a 10,000-point slice with a sampling frequency of 100 kHz would contain 20 complete cycles in a 100 ms duration, facilitating the capture of periodic fault characteristics.

[0032] Furthermore, the parameter configuration of the five-layer one-dimensional convolutional neural network is as follows: the first layer convolution kernel is 64×16 and the number of channels is 16; the subsequent layers use 3×1 convolution kernels, and the number of channels increases to 32, 64, 64, and 64 respectively. Each layer of convolution is followed by 2×2 maximum pooling to achieve progressive extraction from low-level temporal features to high-level complex patterns. The first layer uses a 64×16 large-size convolution kernel with a kernel length of 16 and a number of channels of 16 to capture low-level temporal features in the signal with a larger receptive field, such as periodic fluctuations or The fundamental frequency trend of the current signal is detected. The subsequent four layers sequentially use small 3×1 convolution kernels, with the number of channels increasing to 32, 64, 64, and 64. Dense convolutions mine fine-grained signal features, such as vibration and shock waveform details and current harmonic components, while enhancing the richness of feature representation through channel expansion. Each convolution layer is followed by 2×2 max pooling, which reduces the dimension of the feature map and computational complexity while suppressing noise interference and preserving dominant features, achieving progressive extraction from raw signals to abstract fault modes. This configuration enables the network to capture global signal temporal regularities while refining local feature associations layer by layer, adapting to the differentiated characteristics of high-frequency vibration shocks and low-frequency current trend signals, providing hierarchical and highly discriminative input for subsequent cross-modal feature fusion.

[0033] Furthermore, the multi-layer feature fusion module MLF Contains at least three layers MLF0 、 MLF1 、 MLF2 , bottom-level modules MLF0 3×3 convolution is used to enhance shallow semantic feature extraction and high-level modules MLF2 The computational complexity is reduced by 1×1 convolution, and the parameter configuration of each layer is differentiated to adapt to the requirements of feature fusion at different levels. Multi-layer feature fusion module MLF Through hierarchical parameter configuration and differentiated operations, efficient aggregation and optimization of multimodal features are achieved: underlying module MLF0 Using 3×3 convolution, with a larger receptive field, it enhances the shallow semantic feature extraction capability and can capture the basic correlation between vibration and current signals at a low level, such as the synchronization between vibration impact and current fluctuation; high-level modules MLF2 The introduction of 1×1 convolution reduces the feature dimension through cross-channel interaction, while reducing the computational complexity, retaining cross-modal core correlation information such as common fault patterns in high-level abstract features; the configuration of each layer parameter is differentiated, such as the convolution kernel size and the number of channels, so that MLF It can adapt to the fusion needs of features at different levels. The bottom layer focuses on retaining the detailed differences of the original signal, the middle layer strengthens the dynamic interaction of complementary features between modalities, and the high layer focuses on the abstract integration of global features, forming a progressive fusion path from detail association-cross-layer interaction-global aggregation, avoiding the lack of semantics of shallow features or redundancy of high-level features caused by a single parameter configuration, thereby improving the adequacy and efficiency of multimodal feature fusion.

[0034] Furthermore, the dimension adjustment operation will fuse the feature tensors F The shape from B,C,L Convert to L,B,C, in B is the batch size, C is the number of channels, L is the sequence length, so that the features meet Transformer The time series modeling input format of , which puts the sequence length L in the first dimension, conforming to Transformer When processing time series data, the "sequence priority" modeling logic makes it easier for the position encoding module PE to inject time series information in chronological order, ensuring that the model can accurately capture the order dependency of each sampling point in the signal; at the same time, the batch size B and number of channels C Position adjustment B Center, C At the end, the multi-head self-attention mechanism Multi-HeadAttention It can process cross-channel feature correlations of different batches of samples in parallel, such as the interaction of various dimensional features of vibration and current signals, to improve global modeling efficiency. Through this conversion, TransformerCan be based on long sequence input L Dimensional modeling of long-distance dependencies of multimodal features, such as the cross-time correlation between multiple impact cycles in vibration signals and harmonic components in current signals, to compensate for CNN The shortcomings of local convolution are overcome, thereby enhancing the model's ability to capture global features of complex failure modes (such as progressive bearing wear).

[0035] Furthermore, the multi-head self-attention mechanism includes 8 attention heads, and the key dimension of each head is is 64, value dimension The multi-head self-attention mechanism significantly improves the richness and modeling capability of multimodal feature representation through the parallel calculation of 8 attention heads and the interaction with subspace features: each head independently adopts the parameter configuration of key dimension 64 and value dimension 64, which can capture the differentiated correlation of multimodal features in different subspaces, such as the high-frequency impact subspace of vibration signals and the low-frequency harmonic subspace of current signals, avoiding the omission of feature correlation due to the limited perspective of a single head; the parallel calculation mechanism enables the model to pay attention to the local details of the signal at the same time, such as the correlation between a single vibration impact and the current pulse at the corresponding moment, and the global structure such as the change law of the current trend in multiple vibration cycles, thereby enhancing the diversity of feature representation; finally, the output of each head is projected back to the original number of channels through a linear layer, while retaining the cross-subspace interaction information and maintaining the consistency of feature dimensions, ensuring that the subsequent classifier can effectively integrate multimodal global dependency features. This design enables the model to handle bearing retainer failures. C3 Rolling element failure C4 When subtle fault differences occur, different attention heads can be used to capture the impact interval pattern of the vibration signal and the torque fluctuation pattern of the current signal, thereby expanding the inter-class distance by more than 35%.

[0036] Furthermore, the specific method of multi-level feature transfer in S02 includes:

[0037] Cross-enhancement strategy: vibration characteristics and current characteristics Perform 3×3 convolution respectively, through Sigmoid Function to generate weight graph:

[0038] in 、 A weight map representing vibration characteristics and current characteristics, 、 Respectively The vibration characteristics and current characteristics of the layer, It is a 3×3 convolution operation;

[0039] Cross-modal gating mechanism: Dynamically modulate features through weight maps, the formula is:

[0040]

[0041] in is element-wise multiplication;

[0042] Cross-modal feature concatenation and fusion: The modulated features are concatenated and then subjected to 3×3 convolution. The formula is:

[0043] ,in 、 Respectively represent the characteristics after vibration mode enhancement and current mode enhancement, Indicates the number of channels after fusion, It represents the output after cross-modal feature concatenation and convolution, and is convolved again after being cascaded with the output of the previous layer to form a multi-level feature transfer.

[0044] Furthermore, the L-th convolution operation in S01 is defined as: in Indicates the The feature map dimensions or pooling parameters of the layer convolution operation, , for the Tier m The output characteristics of the modality, is the maximum pooling operation, The first layer uses a 64×16 convolution kernel to capture low-level temporal features. Subsequent layers gradually increase the number of filters to 64 and adjust the convolution kernel size to 3×1. Each convolution layer is followed by a 2×2 maximum pooling. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the process of the present invention;

[0046] Figure 2 shows the change curves of the training set and the validation set, where Figure 2(a) is the loss curve and Figure 2(b) is the accuracy curve;

[0047] Figure 3 Visualization of the FTDCNN-V model using only vibration data. Different colors represent nine types of faults.

[0048] Figure 4 Visualization of the FTDCNN-I model using only current data;

[0049] Figure 5 Visualization of the FTDCNN model that fuses vibration and current signals;

[0050] Figure 6(a) Confusion matrix of FTDCNN-V based on numerical values;

[0051] Figure 6 (a') Corresponding results of the percentage of confusion matrix of FTDCNN-V based on numerical values;

[0052] 7(b) shows the confusion matrix of the numerical-based FTDCNN-I;

[0053] 7(b') shows the corresponding results of the percentage of confusion matrix of FTDCNN-I based on the numerical value;

[0054] Figure 8(c) shows the confusion matrix of FTDCNN based on numerical values;

[0055] 8(c’) shows the corresponding results of the percentage of confusion matrix of FTDCNN-I based on the numerical value. DETAILED DESCRIPTION

[0056] Example 1, basically as Figure 1 As shown, a UAV motor fault diagnosis method based on multimodal data fusion is characterized by comprising the following steps:

[0057] S01 Multimodal Parallel Feature Extraction: Converting the one-dimensional time series signal of the drone motor into , m=1 is the vibration signal, m=2 is the current signal, and the input is a parallel five-layer one-dimensional convolutional neural network CNN, which extracts deep modal specific features through layer-by-layer convolution and pooling

[0058] S02 Cross-modal aggregation of multi-layer feature fusion module: The multi-layer feature fusion module is used to integrate deep modality-specific features layer by layer in the intermediate stage, and then convoluted again after cascading with the output of the previous layer to form multi-level feature transfer;

[0059] S03 Transformer Global modeling of attention mechanism: the fused features To adjust the dimensions, , to adapt Transformer The input format introduces position encoding PE to inject timing information:

[0060]

[0061] in is the feature dimension, is the sequence position, ∈[0, ] is the channel index, Represents the fused feature tensor after dimension adjustment; Represents a tensor dimension rearrangement operation, Represent the position encoding function, which is used to inject timing information, is the sequence position, is the channel index, PE represents the position encoding, and the query is calculated through the multi-head self-attention mechanism ( Q ),key( K ),value( V )matrix:

[0062]

[0063]

[0064] And calculate the attention score by dot product and normalization:

[0065]

[0066] in 、 、 Query （Q） ,key( K ),value( V ) matrix weight parameters, represents the dimension of the bond matrix, It is the transpose of Q and K and performs matrix multiplication to calculate the similarity score of each position of Q and K;

[0067] S04 Deep Neural Network Classification: Input high-level features processed by the attention mechanism into the deep neural network DNN , the nine types of faults are classified by the fully connected layer, and the loss function adopts the cross entropy loss; the nine types of faults include: normal operation C1 , bearing outer ring failure C2 , bearing retainer failure C3 , bearing rolling element failure C4 , bearing inner ring failure C5 , rotor demagnetization C6 , rotor shaft bending C7 , stator phase short circuit C8 , stator turn short circuit C9 .

[0068] First, multimodal parallel feature extraction utilizes two independent five-layer one-dimensional CNN Process vibration signals separately m=1 With current signal m=2Through layer-by-layer convolution, the first layer uses a 64×16 convolution kernel to capture low-level temporal features. The number of filters in the subsequent layers is increased to 64, and the convolution kernel is adjusted to 3×3 and 2×2 maximum pooling to extract deep-level specific features of each modality, such as mechanical impact features in vibration signals and electromagnetic anomaly features in current signals, while retaining the uniqueness of the modality. Secondly, the multi-layer feature fusion module MLF realizes layer-by-layer aggregation of cross-modal features in the middle layer: through the cross-enhancement strategy 3×3 convolution + Sigmoid Generate a weight graph to learn the correlation between modalities, use the cross-modal gating mechanism to dynamically modulate feature weights, strengthen complementary information and suppress redundancy; then through feature splicing and 3×3 convolution, realize the multi-level transmission of cross-modal shared features, and solve the problem that early fusion ignores modal differences and late fusion lacks association mining. Finally, Transformer The attention mechanism globally models the fusion features: injecting temporal information through dimension adjustment and position encoding PE, and using the multi-head self-attention mechanism to calculate the query ( Q ),key( K ),value( V ) matrix, captures long-range dependencies through dot product and normalization, such as the cross-modal correlation between periodic impulses in vibration signals and harmonic components in current signals, to compensate for CNN The shortcomings of local modeling are finally solved through deep neural networks DNN Implement fault classification.

[0069] Early fusion, such as directly concatenating vibration and current signals, ignores the differences in modal distribution, such as the incompatibility between the high-frequency impact of the vibration signal and the low-frequency fluctuation characteristics of the current signal, resulting in feature confusion; late fusion, such as independently extracting features through parallel networks and then splicing the dynamic correlation between unmodeled modes, such as the coordinated changes in the increase in vibration amplitude and the increase in current harmonic components during bearing failure, results in low fusion efficiency.

[0070] MLF Through layer-by-layer cross-enhancement and gating mechanisms, the cross-modal feature weights are dynamically adjusted. For example, in the diagnosis of bearing outer ring fault C2, the impact feature weight of the vibration signal is significantly increased, while the low-frequency feature weight of the current signal due to load fluctuation is suppressed, avoiding the "feature conflict" problem of early fusion. Experiments show that compared with the early fusion model, FTDCNN- E , the inter-class interval of this scheme is expanded by more than 30%.

[0071] Through cross-layer cascading, shallow features such as the original signal’s time domain waveform and deep features such as the fault feature vector are gradually integrated to improve the richness of feature expression. For example, in the rotor demagnetization C6 diagnosis, MLF It can capture the cross-layer correlation between the decrease of the fundamental frequency component of the current signal and the high-frequency vibration caused by the abnormal electromagnetic force in the vibration signal, while the late fusion model FTDCNN-LDue to the lack of intermediate interaction, the probability of misdiagnosing C6 as a bearing fault is as high as 15.46%. This solution reduces the misdiagnosis rate to 0.02%.

[0072] Tradition CNN Relying on local convolution kernels makes it difficult to capture global dependencies in long-term time series signals. For example, after a motor has been running for a long time, the cross-modal correlation between the weak vibration signal at the beginning of bearing wear and the gradually increasing noise in the current signal. This solution uses multi-head attention to parallelly calculate the feature correlations in different subspaces to effectively model long-range dependencies. For example, in the early stage of stator inter-turn short circuit C9, the tiny harmonic distortion in the current signal and the periodic low-frequency vibration in the vibration signal caused by magnetic field imbalance can be associated through the attention mechanism. 1dCNN Due to local modeling limitations, the missed detection rate of such early faults is as high as 25.61%. This solution achieves an average accuracy of 99.96%.

[0073] By injecting timing information through sine / cosine position encoding, the model can distinguish the previous and next dependencies in the signal and avoid feature misjudgment caused by timing confusion. Ablation experiments show that after removing the attention mechanism, FTDCNN-WT , the inter-class overlap rate between bearing retainer fault C3 and rolling element fault C4 increases by 40%, verifying the necessity of attention mechanism for global modeling.

[0074] Compared with existing methods such as DRN, FusionDBN Compared with other approaches, this solution surpasses them in terms of accuracy, stability, and generalization ability:

[0075] Accuracy: The average accuracy is 99.96%, which is better than the second best comparison method. FusionDBN , 98.87% increased by 1.09%, especially in the case of easily confused faults such as C3 and C4 The misjudgment rate is reduced by more than 90%.

[0076] Stability: The standard deviation is only 0.01%, which is significantly lower than other methods such as DRN The standard deviation is 4.26%, indicating that the model can maintain reliable performance under different working conditions.

[0077] Generalization capability: In a complex scenario with nine types of faults, the recognition accuracy of a small sample fault such as rotor shaft bending C7 is 100% with a sample size of only 10%, while the single-modal model FTDCNN-I The misjudgment rate for C7 is as high as 16.28%.

[0078] The vibration signal is transmitted through SAEC00X5 The piezoelectric acceleration sensor collects the current signal through WCS1800 The Hall current sensor collects HK_USB6202_SThe data acquisition card performs analog-to-digital conversion with a sampling frequency of 100kHz and 16-bit accuracy. The vibration signal is transmitted through SAEC00X5 The piezoelectric acceleration sensor is used for data acquisition. Its high precision, high temperature stability and wide frequency response characteristics can accurately capture the weak fault characteristics and high-frequency impact signals in the motor vibration. The current signal is collected through WCS1800 The Hall effect and closed-loop technology can realize high-precision, high-resolution and high-bandwidth measurement, accurately reflecting the transient changes and electromagnetic anomalies of the motor current. HK_USB6202_S The data acquisition card performs analog-to-digital conversion at a 100kHz sampling frequency and 16-bit accuracy, ensuring high-fidelity synchronous acquisition of multimodal signals. This avoids feature aliasing due to insufficient sampling frequency or masking of weak signals due to insufficient quantization accuracy, providing a high-quality data foundation for the subsequent FTDCNN model to extract modal-specific features and cross-modal complementary information, significantly improving the accuracy and reliability of fault diagnosis.

[0079] The data preprocessing includes: segmenting the original signal into 10,000 non-overlapping slices, extracting sub-slices containing 2,048 sampling points with a step size of 1, and normalizing the data distribution to generate a balanced sample set for training. The original signal's 10,000 sampling points are segmented into 10,000 non-overlapping slices to avoid feature aliasing between adjacent slices and ensure that each slice independently contains complete fault cycle information, such as the characteristic frequency period of a bearing fault. For example, for a bearing inner race fault with a characteristic frequency of 200 Hz and a period of 5 ms, a 10,000-point slice with a sampling frequency of 100 kHz would contain 20 complete cycles in a 100 ms duration, making it easier to capture periodic fault characteristics.

[0080] The parameter configuration of the five-layer one-dimensional convolutional neural network is as follows: the first layer convolution kernel is 64×16 and the number of channels is 16; the subsequent layers use 3×1 convolution kernels, and the number of channels increases to 32, 64, 64, and 64 respectively. Each layer of convolution is followed by 2×2 maximum pooling to achieve progressive extraction from low-level temporal features to high-level complex patterns. The first layer uses a large-size 64×16 convolution kernel with a kernel length of 16 and a number of channels of 16 to capture low-level temporal features in the signal with a larger receptive field, such as periodic fluctuations of vibration signals or current. The fundamental frequency trend of the signal; the subsequent four layers use small 3×1 convolution kernels, with the number of channels increasing to 32, 64, 64, and 64 layer by layer. Dense convolutions mine fine-grained signal features, such as vibration impact waveform details and current harmonic components, and enhance the richness of feature expression through channel expansion. Each convolution layer is followed by 2×2 max pooling, which reduces the dimension of the feature map and computational complexity while suppressing noise interference and preserving dominant features, achieving progressive extraction from raw signals to abstract fault modes. This configuration enables the network to capture the global temporal regularity of the signal while refining local feature associations layer by layer, adapting to the differentiated characteristics of high-frequency vibration impact and low-frequency current trend signals, providing hierarchical and highly discriminative input for subsequent cross-modal feature fusion.

[0081] The multi-layer feature fusion module MLF Contains at least three layers MLF0 、 MLF1 、 MLF2 , bottom-level modules MLF0 3×3 convolution is used to enhance shallow semantic feature extraction and high-level modules MLF2 The computational complexity is reduced by 1×1 convolution, and the parameter configuration of each layer is differentiated to adapt to the requirements of feature fusion at different levels. Multi-layer feature fusion module MLF Through hierarchical parameter configuration and differentiated operations, efficient aggregation and optimization of multimodal features are achieved: underlying module MLF0 Using 3×3 convolution, with a larger receptive field, it enhances the shallow semantic feature extraction capability and can capture the basic correlation between vibration and current signals at a low level, such as the synchronization between vibration impact and current fluctuation; high-level modules MLF2 The introduction of 1×1 convolution reduces the feature dimension through cross-channel interaction, while reducing the computational complexity, retaining cross-modal core correlation information such as common fault patterns in high-level abstract features; the configuration of each layer parameter is differentiated, such as the convolution kernel size and the number of channels, so that MLF It can adapt to the fusion needs of features at different levels. The bottom layer focuses on retaining the detailed differences of the original signal, the middle layer strengthens the dynamic interaction of complementary features between modalities, and the high layer focuses on the abstract integration of global features, forming a progressive fusion path from detail association-cross-layer interaction-global aggregation, avoiding the lack of semantics of shallow features or redundancy of high-level features caused by a single parameter configuration, thereby improving the adequacy and efficiency of multimodal feature fusion.

[0082] The dimension resizing operation will fuse the feature tensors F The shape from B,C,L Convert to L,B,C ,in B is the batch size, C is the number of channels, L is the sequence length, so that the features meet Transformer The time series modeling input format of , which puts the sequence length L in the first dimension, conforming to Transformer The "sequence-first" modeling logic when processing time series data facilitates the position encoding module PE Inject timing information in chronological order to ensure that the model can accurately capture the order dependency of each sampling point in the signal; at the same time, the position of batch size B and number of channels C is adjusted so that B is in the middle and C is at the back, so that the multi-head self-attention mechanism Multi- HeadAttention It can process cross-channel feature correlations of different batches of samples in parallel, such as the interaction of various dimensional features of vibration and current signals, to improve global modeling efficiency. Through this conversion, / / There seems to be a hyphen missing in the original, should it be "Multi-Head Attention"? The long-distance dependency of multimodal features can be modeled based on the long sequence input L dimension, such as the cross-time correlation between multiple impact cycles in the vibration signal and the harmonic components in the current signal, to make up for the Transformer The shortcomings of local convolution are overcome, thereby enhancing the model's ability to capture global features of complex fault modes such as progressive bearing wear.

[0083] The multi-head self-attention mechanism contains 8 attention heads, and the key dimension of each head is is 64, value dimension The multi-head self-attention mechanism significantly improves the richness and modeling capability of multimodal feature representation through the parallel computation of 8 attention heads and the interaction with subspace features: each head independently adopts a parameter configuration of 64 key dimensions and 64 value dimensions, which can capture the differentiated correlation of multimodal features in different subspaces, such as the high-frequency impact subspace of the vibration signal and the low-frequency harmonic subspace of the current signal, avoiding the omission of feature correlation due to the limited perspective of a single head; the parallel computation mechanism enables the model to pay attention to the local details of the signal at the same time, such as the correlation between a single vibration impact and the current pulse at the corresponding moment, and the global structure such as the changing law of the current trend in multiple vibration cycles, thereby enhancing the diversity of feature representation; finally, the output of each head is projected back to the original number of channels through a linear layer, while retaining the cross-subspace interaction information and maintaining the consistency of the feature dimension, ensuring that the subsequent classifier can effectively integrate the multimodal global dependency features. This design enables the model to capture the impact interval pattern of the vibration signal and the torque fluctuation pattern of the current signal through different attention heads when dealing with subtle fault differences such as bearing retainer fault C3 and rolling element fault C4, thereby expanding the inter-class distance by more than 35%.

[0084] The specific method of multi-level feature transfer in S02 includes:

[0085] Cross-enhancement strategy: vibration characteristics and current characteristics Perform 3×3 convolution respectively, through CNN Function to generate weight graph: ,in 、 A weight map representing vibration characteristics and current characteristics, 、 Respectively The vibration characteristics and current characteristics of the layer, It is a 3×3 convolution operation

[0086] Cross-modal gating mechanism: Dynamically modulate features through weight maps, the formula is:

[0087]

[0088] in Element-wise multiplication

[0089] Cross-modal feature concatenation and fusion: The modulated features are concatenated and then subjected to 3×3 convolution. The formula is:

[0090]

[0091] in 、 Respectively represent the characteristics after vibration mode enhancement and current mode enhancement, Indicates the number of channels after fusion, It represents the output after cross-modal feature concatenation and convolution, and is convolved again after being cascaded with the output of the previous layer to form a multi-level feature transfer.

[0092] The L-th convolution operation in S01 is defined as: Among them, Indicates the The feature map dimensions or pooling parameters of the layer convolution operation, , for the Tier The output characteristics of the modality, is the maximum pooling operation, It is a linear rectification function. The first layer uses a 64×16 convolution kernel to capture low-level temporal features. The subsequent layers gradually increase the number of filters to 64 and adjust the convolution kernel size to 3×1. Each convolution layer is followed by a 2×2 maximum pooling.

[0093] In actual use, the signal acquisition system is built

[0094] Sensor deployment:

[0095] Vibration signal: Pass Sigmoid The piezoelectric accelerometer has a measurement range of ±100g, a frequency response of 0.5Hz-20kHz, and a temperature drift of ≤0.05% / °C. It is fixed to the three orthogonal X / Y / Z axes of the motor housing to collect three-dimensional vibration acceleration signals, covering the mechanical vibration characteristics of components such as bearings and rotors.

[0096] Current signal: through SAEC00X5 Hall current sensor accuracy ±0.2%FS, bandwidth WCS1800 , isolation voltage 2.5kV is connected in series with the motor three-phase power supply circuit, collecting [[ID= The phase current time domain waveform has a resolution of 0.1mA and reflects the abnormalities of the stator winding and rotor magnetic field in real time.

[0097] use ​ Data acquisition card, 16-bit ​ , 8-channel synchronous sampling, with a maximum single-channel rate of 250kS / s, synchronously collecting 3-channel vibration and 3-channel current signals at a sampling frequency of 100kHz, ensuring that the multi-modal data timestamp error is less than 1μs, and the original data is stored in binary format. .dat file.

[0098] Data preprocessing process

[0099] Segmented Slicing:

[0100] The original signal is cut into 10,000 non-overlapping slices along the time axis, corresponding to a duration of 100ms. 100kHz sampling contains 10,000 sampling points to ensure that each slice contains a complete fault cycle. For example, when the characteristic frequency of the bearing inner race fault is 200Hz, each slice contains 20 cycles.

[0101] For each slice, a sub-slice of 2048 points is cut with a step size of 1. A single slice can generate 7953 sub-samples of 2048 points × 7953, which solves the data scarcity problem of small sample faults such as rotor shaft bending C7. The total sample size reaches 27 categories × 7953 ≈ 214,731.

[0102] Standardization:

[0103] The vibration and current signals are analyzed separately ​ Normalization eliminates dimensional differences, unifies data distribution to the [-1, 1] range, and improves model convergence efficiency.

[0104] Sample label assignment:

[0105] Label each sub-slice with 9 types of labels according to the motor operating status ​ , the training set, validation set, and test set are divided into 7:2:1 ratios to ensure that the samples of each category are balanced.

[0106] 2. Multimodal Parallel Feature Extraction

[0107] 2.1 Five-layer one-dimensional ​ Network Architecture

[0108] Input layer: 3 channels of vibration signal and 3 channels of current signal are used as two parallel ​ The input shape is (2048,3) (sequence length × number of channels).

[0109] First convolution layer ( L=1 ):

[0110] Convolution kernel configuration: 64 16×3 convolution kernels (kernel length 16, spanning 3 channels), stride 16, padding 15, capturing low-level timing features such as vibration fundamental frequency and current DC offset.

[0111] Output: Feature map shape (128, 64), after 2×2 maximum pooling, the sequence length is compressed to 2048 / 16=128.

[0112] Intermediate layer convolution ( L=2-4 ):

[0113] Convolution kernel configuration: 32, 64, and 64 3×1 convolution kernels are used in sequence, with a kernel length of 3, single-channel processing, a stride of 1, and no padding to capture fine-grained features such as the rising edge slope of the vibration shock waveform and the phase difference of the current harmonics.

[0114] Pooling layer: Each layer is connected to 2×2 maximum pooling, and the sequence length is halved ( ​ ).

[0115] The last convolution layer ( L=5 ):

[0116] Convolution kernel configuration: 64 3×1 convolution kernels, stride 1, output feature map shape (8,64), retaining high-level abstract features such as envelope entropy of vibration signals and kurtosis of current signals.

[0117] Output:

[0118] Vibration characteristics and current characteristics , characterizing the modal specific information of mechanical and electromagnetic faults respectively.

[0119] Multi-layer feature fusion module ​ Cross-modal aggregation

[0120] Three-layer ​ Module Architecture ​

[0121] Low-level modules ​ Shallow fusion:

[0122] Cross-enhancement: and Apply 3×3 convolution 64→32 channels respectively, through ​ Generating cross-modal weight maps , :

[0123] Gated Modulation:

[0124] Dynamic weighting of features: , m′ is another mode), strengthen the complementary characteristics (such as the current spike at the corresponding moment of vibration impact), represents the representation of the original features, represents the learnable weight vector / matrix of the main modality, represents the learnable weight of another auxiliary modality, Is the main mode The output after cross-modal gating weighted enhancement.

[0125] Feature stitching:

[0126] Splicing and is (8,64), after 3×3 convolution ( 64→64 After the channel is cascaded with the input, the output , Indicates the enhanced characteristics of another modality after the same / similar gating modulation; Represents the final fusion feature

[0127] Middle-layer module MLF1 (middle-layer fusion):

[0128] Repeat the MLF0 operation and introduce residual connections ​ To avoid gradient disappearance, output , focusing on the dynamic interaction of cross-modal features, such as the correlation between vibration impact frequency and current harmonic order.

[0129] High-level modules ​ :

[0130] Dimensionality reduction optimization:

[0131] 1×1 convolution (128→64 channels) is used to reduce computational complexity and retain core correlation features, such as the "vibration shock + current low-frequency fluctuation" pattern of bearing faults.

[0132] After concatenation with MLF1 output, convolution is performed again to output fusion features. , completing feature aggregation from shallow details to high-level abstractions.

[0133] ​ Global Modeling of Attention Mechanism

[0134] Feature dimension adaptation

[0135] Dimension conversion: , B is the batch size, adjusted to ,satisfy ​ Input format, sequence length L=8 comes first.

[0136] Positional encoding:

[0137] Generate position encoding matrix , inject timing information through sine / cosine functions:

[0138]

[0139] in, is the sequence position, It is the channel index to ensure that the model distinguishes the time sequence.

[0140] Multi-head self-attention calculation

[0141] Parameter configuration:

[0142] There are 8 attention heads, each with a key / value dimension of 64, and the query dimension is the same as the input (64), so the total computation dimension is 8×64=512.

[0143] Parallel attention computation:

[0144] ​ generate:

[0145] Attention score calculation:

[0146] in, , , For the h The query / key / value matrix of the header.

[0147] Feature stitching and projection:

[0148] Splicing 8 heads output , projected back to 64 channels through a linear layer: Output global modeling features , capturing the long-range dependence of multimodal features, such as the change of current trend over multiple consecutive vibration cycles, It represents a linear transformation layer that compresses the dimension of the input feature from 512 to 64.

[0149] Deep Neural Network Classification

[0150] Feature dimension restoration and pooling

[0151] Dimension conversion: Convert from (8,B,64) to (B,64,8) to facilitate global average pooling GAP operation.

[0152] Global average pooling: compresses the sequence dimension to (B, 64) and extracts the global feature vector.

[0153] Fully connected network classification

[0154] Three fully connected layers:

[0155] First layer: 128 neurons, ​ Activation, capturing nonlinear correlations between features;

[0156] Second layer: 64 neurons, ​ Activation, further dimensionality reduction;

[0157] Output layer: 9 neurons, ​ Activate and output the fault category probability distribution C1-C9.

[0158] Training parameters:

[0159] Optimizer: ​ , learning rate 1e-4, weight decay 1e-5;

[0160] Loss function: cross entropy loss ​ ;

[0161] Batch size: 32, training epochs: 100, early stopping mechanism, termination if validation set loss does not decrease for 5 consecutive epochs.

[0162] System integration and real-time diagnosis

[0163] Hardware deployment

[0164] Edge computing nodes: equipped with ​ , ARM architecture, 4-core CPU + 128-core GPU, integrated sensor interface and ​ Data acquisition card, supporting real-time data processing.

[0165] Communication module: through ​ The diagnostic results (such as fault type and confidence level) are transmitted to the cloud server or drone ground station to trigger alarms (such as sound and light alarms, motor shutdown commands).

[0166] Real-time diagnostic process

[0167] Data acquisition: Vibration and current signals are continuously collected at a frequency of 100kHz, generating a data slice every 100ms.

[0168] Online preprocessing: 2048-point subsamples are taken from the slice with a step size of 1, and then normalized in real time before input ​ Model;

[0169] Fault identification: The model's forward propagation time is less than 10ms, and it outputs nine types of fault probabilities. When the threshold is greater than 0.95, it is determined to be a corresponding fault.

[0170] Normal state C1: continue monitoring;

[0171] Fault status C2-C9: Pass ​ The bus sends a shutdown command and uploads the fault type and timestamp to the cloud.

[0172] Experimental Example 1

[0173] The stator, rotor and bearing fault simulation experiments were designed. According to the operating characteristics of the UAV motor, nine operating states were set, including normal operation, four bearing faults, two rotor faults and two stator faults. The motor fault locations of the corresponding modules are as follows: ​As shown in the figure, the motor was then operated at speeds of 5krpm, 5.5krpm, and 6krpm in each of these nine states, while synchronously collecting motor current and vibration signals at a sampling frequency of 100kHz. A total of 27 data sets were obtained, each containing 10,000K samples. During data preprocessing, each data set was first segmented into non-overlapping slices of 10,000K. Subslices containing 2,048 samples were then extracted from these slices with a step size of 1 sample point. These subslices were then normalized to obtain the sample sets used for model training and testing.

[0174] Table 1 Motor working status corresponding code

[0175] Using Shiao ​ Piezoelectric accelerometer is used to collect vibration signals. The sensor can measure The acceleration is characterized by high precision, high temperature stability and wide frequency response. The current signal is collected by the WCS1800 Hall effect current sensor, which is based on the Hall effect and closed loop technology and can provide high precision, high resolution and high bandwidth measurement results. ​ The data acquisition card can be connected to a computer via a USB interface, supports 16-bit accuracy and a sampling rate of up to 250KS / s, meeting the needs of high-precision and high-speed measurement and monitoring.

[0176] As shown in Figure 2, ​ The model uses a deep learning framework ​ Construct, the hyperparameters are set as follows: the optimizer is ​ , the learning rate is 0.0001; the loss function is cross entropy loss; the batch size is 32, the number of epochs is 100, and the learning rate scheduler is used ​ The loss curves of the training set and validation set are shown in Figure a, and the accuracy curves are shown in Figure b. As shown in Figure a, the training set loss and validation set loss converge to the minimum value around the 15th epoch, proving that the model has a fast convergence speed; as shown in Figure b, after 100 epochs of training, the validation set accuracy finally converges to 100%. ​ When evaluating, the average accuracy of the model can reach 99.962%, indicating that the model has excellent generalization ability on unseen data. ​ Able to accurately diagnose drone motor failures.

[0177] like ​ As shown in the figure, in order to more intuitively evaluate the impact of multimodal fusion on the ability to distinguish fault features, this study evenly extracts several samples from the test set and performs high-dimensional features of the model output under different input modes. ​ edStochasticNeighborEmbedding ( T-SNE) visualization. T-SNE High-dimensional features can be mapped to a two-dimensional plane, making it easier for samples of the same type to cluster together and easier for samples of different types to disperse, thereby intuitively presenting the distribution of the feature space. Figure 3 Three models are shown T-SNE distribution, including two control baselines: using only vibration data FTDCNN-V As attached Figure 3 Compared with using only current data FTDCNN-I Attachment Figure 4 , and a complete model that integrates vibration and current signals FTDCNN Attachment Figure 5 Among them, nine fault types C1–C9 are distinguished by different colors. When the distribution of points within a class is more compact and the distance between classes is larger, it indicates that the model has a stronger ability to distinguish different fault categories. Specifically, FTDCNN-V There is a certain degree of separability in most fault categories, but there is still significant overlap between categories such as C5 and C7, resulting in vague classification boundaries. FTDCNN-I Similar confusion phenomena also appear in some areas of fault categories C6 and C7. FTDCNN exist T-SNE The distribution shows more compact intra-class clustering and larger inter-class intervals, significantly reducing the confusion between different fault categories. The above results fully verify the effectiveness of multimodal fusion in improving the ability to distinguish fault features.

[0178] Experimental Example 2

[0179] As shown in Figure 6-8, FTDNN, FTDCNN-V and FTDCNN-I The test results are visualized in confusion matrices. The test set contains 4,392 samples. By comparing the confusion matrix distributions of different models, we can more intuitively analyze the specific improvements brought by multimodal fusion compared to single-modal methods in various fault categories, especially the significant improvement in recognition accuracy in category 1 (bearing outer race fault), category 3 (bearing rolling element fault), and category 4 (rotor demagnetization).

[0180] The control baseline, FTDCNN-V, exhibited some misclassification in these key categories. For example, 7 samples in category 1 were misclassified as category 6, accounting for approximately 0.16% of the total test samples. In category 3, 10 samples were misclassified as category 0, 5 samples were misclassified as category 1, and 14 samples were misclassified as category 2, for a total of 29 misclassifications, accounting for approximately 0.66% of the total test samples. In category 4, 21 samples were misclassified as category 5, accounting for approximately 0.48%. While these misclassifications represent a relatively low proportion of the overall sample size, they nonetheless demonstrate the limitations of a single vibration signal in distinguishing certain similar fault types. In contrast, the control baseline, FTDCNN-I, exhibited significantly higher misclassification rates in these categories, particularly in categories 1 and 3. Specifically, 666 samples in category 1 were misclassified as category 3, 1 sample as category 2, and 12 samples as category 4, for a total of 679 misclassifications, accounting for approximately 15.46% of the total test samples. In category 3, 713 samples were misclassified as category 1, and 2 samples were misclassified as category 4, for a total of 717 misclassifications, accounting for approximately 16.28%. In category 4, 1087 samples were misclassified as category 0, and 28 samples were misclassified as category 3, for a total of 1115 misclassifications, accounting for approximately 25.61%. These high misclassification rates demonstrate the limitations of single current signals in complex fault identification, particularly their weak ability to distinguish between different fault types. Notably, among single-modal signals, the vibration signal's misclassification rates for categories 1, 3, and 4 were 0.16%, 0.66%, and 0.48%, respectively, significantly exceeding the current signal's 15.46%, 16.28%, and 25.61%. However, the FTDCNN model, which integrates multimodal signals, performs nearly perfectly across all these categories, with only minor misclassifications in rare cases. For example, 20 examples in category 3 were misclassified as category 0, accounting for approximately 0.46% of the total test examples; and only one example in category 4 was misclassified as category 1, accounting for approximately 0.02%. These misclassification rates are significantly lower than those of the single-modal models, particularly for categories 1 and 3, where the misclassification rates dropped from 0.16% for FTDCNN-V and 15.46% and 16.28% for FTDCNN-I, respectively, to nearly 0% for FTDCNN. The misclassification rate for category 4 also dropped from 25.61% for FTDCNN-I to 0.02% for FTDCNN.

[0181] The above analysis demonstrates that multimodal fusion significantly improves the model's fault identification capabilities, particularly for Category 1 bearing outer race faults and Category 3 bearing rolling element faults, which have high misclassification rates. Furthermore, the vibration signal in the single-modal model demonstrates higher fault identification accuracy than the current signal, further strengthening the effectiveness of the multimodal fusion approach. This demonstrates the effectiveness of multimodal fusion in improving the accuracy of UAV motor fault diagnosis.

[0182] The above are only embodiments of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Those of ordinary skill in the art are aware of all common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can obtain all existing technologies in the field, and have the ability to apply conventional experimental means before that date. Those of ordinary skill in the art can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for those of ordinary skill in the art to implement this application. It should be pointed out that for those skilled in the art, several variations and improvements can be made without departing from the structure of the present invention. These should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A UAV motor fault diagnosis method based on multimodal data fusion, characterized in that: The following steps are involved: S01 Multimodal Parallel Feature Extraction: Converting the one-dimensional time series signal of the drone motor into , m=1 is the vibration signal, m= 2 For the current signal, input the parallel five-layer one-dimensional convolutional neural network CNN , extracting deep modality-specific features through layer-by-layer convolution and pooling; S02 Cross-modal aggregation of multi-layer feature fusion module: The multi-layer feature fusion module is used to integrate deep modality-specific features layer by layer in the intermediate stage, and then convoluted again after cascading with the output of the previous layer to form multi-level feature transfer; The specific method of multi-level feature transfer in S02 includes: Cross-enhancement strategy: vibration characteristics and current characteristics Perform 3×3 convolution respectively, through Sigmoid Function to generate weight graph: ,in 、 A weight map representing vibration characteristics and current characteristics, 、 Respectively The vibration characteristics and current characteristics of the layer, It is a 3×3 convolution operation Cross-modal gating mechanism: Dynamically modulate features through weight maps, the formula is: in Element-wise multiplication Cross-modal feature concatenation and fusion: The modulated features are concatenated and then subjected to 3×3 convolution. The formula is: in 、 Respectively represent the characteristics after vibration mode enhancement and current mode enhancement, Indicates the number of channels after fusion, Represents the output after cross-modal feature concatenation and convolution, and is convolved again after cascading with the output of the previous layer to form a multi-level feature transfer; S03 Transformer Global modeling of the attention mechanism: adjust the dimension of the fused feature F, , to adapt Transformer Input format, introducing position encoding PE Inject timing information: in is the feature dimension, is the sequence position, ∈[0, ] is the channel index, Represents the fused feature tensor after dimension adjustment; Represents a tensor dimension rearrangement operation, Represent the position encoding function, which is used to inject timing information, is the sequence position, is the channel index, and the query is calculated through the multi-head self-attention mechanism （Q） ,key( K ),value( V )matrix: Among them, Q is the query matrix, K is the key matrix, and V is the value matrix. Represents the position encoding feature tensor and calculates the attention score through dot product and normalization: in 、 、 Query （Q） ,key( K ),value( V ) matrix weight parameters, represents the dimension of the bond matrix; S04 Deep Neural Network Classification: Input high-level features processed by the attention mechanism into the deep neural network DNN , the nine types of faults are classified by the fully connected layer, and the loss function adopts the cross entropy loss; the nine types of faults include: normal operation C1 , bearing outer ring failure C2 , bearing retainer failure C3 , bearing rolling element failure C4 , bearing inner ring failure C5 , rotor demagnetization C6 , rotor shaft bending C7 , stator phase short circuit C8 , stator inter-turn short circuit C9.

2. The method for diagnosing motor faults of unmanned aerial vehicles based on multimodal data fusion according to claim 1, characterized in that: The vibration signal is collected by a SAEC00X5 piezoelectric acceleration sensor, and the current signal is collected by a WCS1800 Hall current sensor. The analog-to-digital conversion is performed by a HK_USB6202_S data acquisition card with a sampling frequency of 100 kHz and 16-bit accuracy.

3. The method for diagnosing motor faults of unmanned aerial vehicles based on multimodal data fusion according to claim 1, characterized in that: The data preprocessing includes: dividing the original signal into 10,000 non-overlapping slices, intercepting sub-slices containing 2,048 sampling points with a step size of 1, and standardizing the data distribution through normalization to generate a balanced sample set for training.

4. The method for diagnosing motor faults of unmanned aerial vehicles based on multimodal data fusion according to claim 1, characterized in that: The parameter configuration of the five-layer one-dimensional convolutional neural network is as follows: the first layer has a convolution kernel of 64×16 and 16 channels; subsequent layers use 3×1 convolution kernels, and the number of channels increases to 32, 64, 64, and 64 respectively. Each layer of convolution is followed by 2×2 maximum pooling to achieve progressive extraction from low-level temporal features to high-level complex patterns.

5. The method for diagnosing motor faults of unmanned aerial vehicles based on multimodal data fusion according to claim 1, characterized in that: The multi-layer feature fusion module MLF includes at least three layers: MLF0, MLF1, and MLF2. The bottom-layer module MLF0 uses 3×3 convolution to enhance shallow semantic feature extraction, and the high-layer module MLF2 uses 1×1 convolution to reduce computational complexity. The parameter configurations of each layer are differentiated to adapt to the feature fusion requirements of different levels.

6. The method for diagnosing motor faults of unmanned aerial vehicles based on multimodal data fusion according to claim 1, characterized in that: The dimension adjustment operation transforms the shape of the fused feature tensor F from B, C, L to L, B, C, where B is the batch size, C is the number of channels, and L is the sequence length, so that the features conform to the Transformer's temporal modeling input format.

7. The method for diagnosing motor faults of unmanned aerial vehicles based on multimodal data fusion according to claim 1, characterized in that: The multi-head self-attention mechanism contains 8 attention heads, and the key dimension of each head is is 64, value dimension The number of channels is 64, and the diversity of feature representation is enhanced by parallel subspace calculation. The final output is projected back to the original number of channels through a linear layer.

Images