A motor intelligent diagnostic system based on multimodal fusion

Abstract

This invention provides a multimodal fusion-based intelligent diagnostic system for motors. It comprises six modules: multimodal data acquisition, adaptive preprocessing, cross-domain feature fusion, lightweight inference deployment, diagnostic result output, and closed-loop control. The system synchronously acquires data from multiple sensors (sound, vibration, electrical, and temperature). Standardized feature vectors are generated through adaptive Kalman filtering, improved 3σ outlier processing, and wavelet packet transform. Fault features are extracted using a hybrid architecture of convolutional attention networks and gated recurrent units. Low-latency inference at the edge is achieved through knowledge distillation, channel sparsity, and INT8 quantization. This invention achieves both diagnostic accuracy and real-time performance, featuring high fault classification accuracy, early warning, strong cross-condition adaptability, and low edge inference latency, making it suitable for the entire lifecycle health management of motors.

Description

Technical Field

[0001] This invention relates to the field of intelligent motor diagnostic technology, specifically to an intelligent motor diagnostic system based on multimodal fusion. Background Technology

[0002] Electric motors are core power components in industrial manufacturing, rail transportation, wind power generation, and other equipment, and their stable operation directly affects production efficiency and system safety. Industry data shows that unplanned downtime caused by sudden motor failures accounts for more than 40% of total equipment downtime, and latent faults such as early bearing wear and winding insulation deterioration are the main causes of serious accidents. Therefore, achieving accurate early fault identification and low-latency real-time on-site diagnosis of motors has become a key requirement for predictive maintenance.

[0003] Current motor fault diagnosis mainly falls into two technical categories. One is the traditional method based on single signals such as vibration and current, relying on techniques like spectrum analysis and harmonic detection to extract features. This type of method has limited information dimensions, making it difficult to comprehensively characterize fault mechanisms under complex operating conditions. It is insensitive to weak early-stage faults, and its diagnostic accuracy significantly decreases under varying speed and load conditions. The other category is intelligent diagnostic methods based on deep learning. While these methods can improve recognition accuracy, they suffer from large model sizes, high computational costs, and heavy reliance on cloud computing power. Data round-trip transmission often results in delays exceeding one second, making it difficult to meet the requirements of real-time protection and rapid alarm in industrial settings.

[0004] To balance accuracy and real-time performance, existing technologies attempt to deploy lightweight models at the edge. However, common methods such as pruning and quantization weaken the ability to represent fault features, increasing the rate of missed early fault detection. Furthermore, most solutions employ static models, lacking adaptability to operating conditions and closed-loop verification mechanisms. When diagnostic confidence is insufficient or multi-source data is inconsistent, it is difficult to proactively trigger re-inspections and parameter adjustments, easily leading to misjudgments and missed diagnoses.

[0005] In summary, existing technologies suffer from a core contradiction: large models offer high accuracy but suffer from high latency, while lightweight models offer fast response but weak early fault identification capabilities. How to simultaneously achieve high-accuracy early fault diagnosis, low-latency edge inference, and adaptive closed-loop control in complex industrial scenarios with varying loads and speeds is a critical technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a motor intelligent diagnostic system based on multimodal fusion, which solves the technical problems of "high latency of cloud inference for large models and sacrifice of early fault identification capability for lightweight models, making it difficult to meet the low-latency real-time diagnosis in industrial fields".

[0007] To achieve the above objectives, the present invention is implemented using the following technical solution: This invention provides a motor intelligent diagnostic system based on multimodal fusion, comprising: a multimodal data acquisition module, which synchronously acquires raw data of the motor in acoustic, vibration, electrical, and temperature parameters through multiple sensors; The adaptive preprocessing module sequentially performs adaptive Kalman filtering, improved 3σ outlier processing, and wavelet packet transform on the raw data output by the multimodal data acquisition module to generate standardized feature vectors. The adaptive Kalman filtering performs time-domain filtering on the raw signal, and its output denoised signal serves as the input for the improved 3σ outlier processing. The improved 3σ outlier processing uses a fixed window length for sliding outlier detection, and performs cubic spline local reconstruction on outlier points using only a few neighboring points before and after their time sequence. The cross-domain feature fusion module first uses a convolutional attention network to enhance the channel and spatial dimensions of the standardized feature vector, then uses a gated recurrent unit to perform temporal modeling on the enhanced feature sequence, and finally performs adaptive weighted fusion of the temporal features of the four modes of acoustics, vibration, electrical and temperature to output the fused fault features. The lightweight inference deployment module compresses the model through knowledge distillation, channel sparsity and parameter quantization, and outputs the fault type, severity and remaining lifetime prediction values. The diagnostic results output module is used to display the diagnostic results on a visual terminal and issue early warnings based on a three-level alarm mechanism. The closed-loop control module is used to evaluate the confidence level and verify the consistency of multimodal data of the diagnostic results output by the cross-domain feature fusion module. When the verification fails or the confidence level does not meet the preset conditions, a secondary detection is triggered. The module is also used to adaptively adjust the sampling strategy according to the real-time monitored changes in the working conditions.

[0008] Specifically, the adaptive Kalman filtering is used to remove noise interference under varying operating conditions based on the state equation and the observation equation to obtain the true amplitude estimates of motor vibration, acoustic, current and other signals.

[0009] Specifically, the improved 3σ outlier processing uses a fixed time window to dynamically identify outliers in continuously acquired time-series data. Only when a data point deviates from the statistical center of the window by more than three times the statistical dispersion is it identified as a transient outlier. Then, a cubic spline basis function constrained by the neighboring observations before and after the outlier is called to perform local smoothing reconstruction. The reconstructed data is directly used as the input for time-frequency transformation.

[0010] Specifically, the wavelet packet transform divides the preprocessed signal into multi-resolution frequency bands based on a preset orthogonal wavelet basis function. The cumulative square amplitude of the wavelet coefficients in each sub-band is used as the energy characterization value of that frequency band. A fixed number of sub-band energy values ​​are extracted in order from low to high frequency and combined with the time-domain waveform feature parameters extracted from the original signal to form a standardized multi-source feature tensor.

[0011] Specifically, the adaptive weighted fusion is as follows: Let the acoustic, vibration, electrical, and temperature feature vectors be respectively... The fusion weight is (satisfy The weights are updated using a reinforcement learning algorithm, and the features are fused. .

[0012] Specifically, the channel attention branch of the convolutional attention network sequentially performs global pooling, multilayer perceptron mapping, and sigmoid activation on the input feature tensor to generate channel weights; its spatial attention branch sequentially performs channel-dimensional pooling, convolution, and sigmoid activation on the input feature tensor to generate spatial weights; then the channel weights and spatial weights are fused with the original features to output enhanced features.

[0013] Specifically, the gated loop unit includes a reset gate and an update gate. The reset gate is used to adjust the influence of the previous hidden state on the candidate hidden state, and the update gate is used to balance the weights of the previous hidden state and the current candidate state in the output hidden state. In this way, the input enhanced feature sequence is time-series modeled, and the output hidden state vector containing fault time-series information is generated.

[0014] Specifically, the lightweight inference deployment module performs multi-level compression on the model, including: Knowledge distillation: A pre-trained ResNet-101 fusion architecture with gated recurrent units is used as the teacher model, and a lightweight ResNet-18 variant is used as the student model. The student model is guided to mimic the output of the teacher model through the distillation loss function. Channel filtering: Apply L1 regularization sparsity constraints to the convolutional channels of the knowledge-distilled student model, and remove redundant channels with weight values ​​below a preset threshold after training; Parameter quantization: An asymmetric quantization scheme is adopted, which uses the quantization formula and the dequantization formula to map the 32-bit floating-point parameter to the integer range of 0 to 255, thereby reducing the parameter size.

[0015] Specifically, the closed-loop control module monitors the fault diagnosis confidence level output by the cross-domain feature fusion module in real time. When the system's confidence level for the current diagnosis result is lower than a preset threshold, or when the consistency verification between multimodal data fails, the system determines that the current diagnosis result is not credible enough and automatically triggers a secondary detection process. The system re-acquires the original multimodal data of the motor within one sampling period before and after the current moment, performs preprocessing and feature fusion, and performs inference again. The two diagnosis results are cross-validated, and the one with higher confidence is taken as the final output. If the confidence level is still lower than the threshold after the secondary detection, a "diagnosis uncertain" warning is issued through the diagnosis result output module, prompting manual review.

[0016] Compared with the prior art, the beneficial effects achieved by the present invention are: (1) This invention significantly improves the ability to identify latent faults such as early bearing wear and winding insulation aging by simultaneously collecting and fusing cross-domain features through acoustic, vibration, electrical and temperature multimodal sensors, combined with a hybrid architecture of convolutional attention network and gated recurrent unit, improves the accuracy of fault classification, and advances the warning time for early fault identification.

[0017] (2) In the adaptive preprocessing module, the present invention adopts an improved 3σ outlier processing method, which uses cubic spline interpolation to replace outliers instead of directly removing them, thus maintaining the continuity and integrity of time series data. No additional filling of missing points is required, avoiding repeated sampling and secondary processing due to missing data, thereby shortening the overall time consumption of the preprocessing stage. This provides a continuous and complete data foundation for subsequent wavelet packet transformation and feature fusion, further ensuring the accuracy of the diagnostic results.

[0018] (3) This invention uses knowledge distillation, channel sparsification and INT8 quantization technology to compress the deep learning model into a lightweight form and deploy it on an industrial-grade edge computing unit. The inference latency is no more than 0.2 seconds. While maintaining high diagnostic accuracy, it realizes low-latency real-time diagnosis in industrial sites. It overcomes the shortcomings of existing large models that rely on cloud computing power and have an inference latency of more than 1 second. It shortens the downtime caused by sudden motor failures. The diagnostic process is fully automated and does not require on-site supervision by professional personnel. The visual interface and hierarchical alarm mechanism reduce the operating threshold and adapt to the needs of users at different levels. Attached Figure Description

[0019] Figure 1 This is a flowchart of the intelligent motor diagnostic system of the present invention.

[0020] Figure 2 This is a flowchart of the cross-domain feature fusion module of the present invention. Detailed Implementation

[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1; This embodiment describes in detail the hardware configuration, software deployment environment, and overall workflow of the intelligent motor diagnostic system.

[0023] The hardware configuration of the intelligent motor diagnostic system of the present invention includes: sensor components, edge computing unit, cloud server, and execution and display components.

[0024] The sensor assembly includes an acoustic sensor, a triaxial vibration acceleration sensor, a stator current sensor, a rotor voltage sensor, and an infrared temperature sensor.

[0025] The edge computing unit uses an industrial-grade edge gateway equipped with an NVIDIA Jetson Xavier NX module to support GPU hardware acceleration.

[0026] The cloud server is equipped with an NVIDIA A100 GPU and a 1TB solid-state drive storage array.

[0027] The execution and display components are a servo motor-driven conveyor belt and a large visual touch screen.

[0028] The software deployment environment edge of this invention includes: a terminal operating system, a cloud operating system, a deep learning framework and data processing library, a visualization development tool, and a communication protocol.

[0029] The edge operating system uses Ubuntu 20.04 LTS to run the system kernel of the edge computing unit, manage hardware drivers, and deploy lightweight inference services.

[0030] The cloud operating system uses CentOS Stream 9 to provide a stable server operating environment, supporting large-scale data storage, deep learning model training task scheduling, and concurrent access by multiple users.

[0031] The deep learning framework uses TensorFlow 2.15.0 to build a teacher model (a fusion architecture of ResNet-101 and GRU) and a student model (a lightweight ResNet-18 variant), and performs knowledge distillation training and INT8 quantization compression of the models.

[0032] The data processing library consists of SciPy and PyTorchLightning libraries. SciPy is used to implement the adaptive Kalman filter algorithm, wavelet packet transform, and cubic spline interpolation calculation in the improved 3σ outlier handling. PyTorchLightning is used to simplify the management of cloud training processes, including model checkpoint saving, distributed training scheduling, and automatic hyperparameter tuning.

[0033] The visualization development tools used are ECharts and Vue.js, which are used to build a multi-dimensional visualization terminal in the diagnostic result output module to realize real-time dynamic rendering and front-end interaction of multimodal data time series curves, fault type pie charts, remaining life countdown and health status scores (0-100 points).

[0034] The communication protocol supports OPCUA, MQTT and HTTP / 2.

[0035] Reference Appendix Figure 1 The overall system workflow is as follows: It adopts closed-loop control logic and executes six stages in sequence: system initialization, data acquisition, preprocessing, model inference, result judgment and cyclic diagnosis.

[0036] System initialization phase: Complete sensor calibration, configure operating parameters, load lightweight model to edge computing unit, and perform system self-test.

[0037] Data acquisition phase: The closed-loop control module controls the positioning motor of the conveyor device. The multimodal data acquisition module synchronously acquires sound, vibration, electrical, and temperature data according to a dynamic sampling strategy, and the raw data is transmitted to the preprocessing module.

[0038] Preprocessing stage: The adaptive preprocessing module sequentially employs adaptive Kalman filtering for noise reduction, improved 3σ rule combined with cubic spline interpolation for outlier processing, and wavelet packet transform to extract time-frequency domain features, generating a 15-dimensional standardized feature vector.

[0039] Model inference stage: The edge computing unit obtains the preprocessed feature vector, runs the lightweight model, and outputs the fault type, severity, and remaining lifetime prediction.

[0040] Result Determination Phase: The diagnostic results are displayed on the visual terminal, and a warning is issued according to the three-level alarm mechanism. Level 1 alarm triggers shutdown, Level 2 alarm triggers voice prompt, and Level 3 alarm only displays an indicator. The results are fed back to the closed-loop control module. If the diagnostic confidence is below 97% or the multimodal data consistency check fails, a secondary test is triggered; if the speed fluctuation is ≥10% or the load fluctuation is ≥15%, the sampling frequency is automatically adjusted.

[0041] Cyclic Phase: After completing all the above steps, the system will store the diagnostic data in the historical database and enter the next motor diagnostic cycle.

[0042] Example 2; This embodiment specifically describes a motor intelligent diagnostic system, which includes: a multimodal data acquisition module, an adaptive preprocessing module, a cross-domain feature fusion module, a lightweight inference deployment module, a diagnostic result output module, and a closed-loop control module.

[0043] The modal data acquisition module serves as the system's data entry point. By setting up acoustic, triaxial vibration, stator current, rotor voltage, and infrared temperature sensors, it synchronously acquires motor operating data in four dimensions: acoustic, vibration, electrical, and temperature. Among them, the acoustic sensor captures high-frequency acoustic emission signals from bearing rolling elements and gear meshing faults; the vibration sensor detects mechanical faults such as rotor imbalance and foundation loosening; the current and voltage sensors analyze harmonic distortion and voltage imbalance to identify electrical faults such as rotor bar breakage and inter-turn short circuits; and the infrared temperature sensor monitors the thermal load status of the windings and bearings for overload and insulation aging early warning. The data collected by each sensor are timestamped to obtain multimodal raw data.

[0044] The adaptive preprocessing module receives the raw data output by the multimodal data acquisition module and performs signal denoising, anomaly removal, and feature extraction. Through three steps—adaptive filtering, outlier handling, and time-frequency domain transformation—the raw signal is converted into a standardized feature vector, providing high-quality input for the cross-domain feature fusion module.

[0045] Specifically, this module executes the following steps in sequence: S1, Adaptive Kalman Filter An adaptive Kalman filter is used to remove noise interference under varying operating conditions. The system's state vector X at time k is defined. k Its components are the estimated true amplitude values ​​of motor vibration, acoustic, current and other signals after Kalman filtering.

[0046] The state equation is X k =AX k 1+Bu k +w k The observation equation is Z k =HX k +v k Where A is the state transition matrix, B is the control matrix, and u k The control input vector is H, the observation matrix is ​​w k For process noise and v k To monitor noise, this module uses a recursive least squares method to dynamically update the filter parameters, enabling the filter to adapt to changes in operating conditions and maintain stable noise reduction under varying speeds and loads.

[0047] S2, Improved 3σ outlier handling Even after filtering, the signal may still contain anomalous transition points caused by sensor interruptions, electromagnetic interference, etc. Let the time-series data sequence within the sliding window be {x1, x2, ..., x...}. n}, where x i This represents the sampled value at time i within the window. This module uses a sliding window mechanism to dynamically detect time series data. The sliding window size N=50, which can ensure the integrity of local time series features while avoiding outliers caused by an excessively large window and statistical biases caused by an excessively small window.

[0048] Calculate the mean μ and standard deviation σ of the data within the window: , .

[0049] when At that time, determine the current data point x j For outliers, instead of directly removing them, cubic spline interpolation is used for replacement. The values ​​of the two neighboring points before and after the outlier are then used for smooth reconstruction, and the correction value x is calculated. j ′.

[0050] , where x j ' is to replace x j The subsequent correction value, These are cubic spline basis functions.

[0051] S3. Wavelet Packet Transform and Feature Extraction To extract effective fault features from non-stationary signals, this module uses the db4 wavelet basis to perform 4-level wavelet packet decomposition on the processed signal. After 4-level decomposition, 16 frequency band subspaces with equal bandwidth are obtained. Considering that the motor fault features are mainly concentrated in the first 8 low-frequency and high-frequency bands, this module extracts the energy features of the first 8 frequency bands.

[0052] Let the wavelet packet coefficient sequence in the m-th frequency band subspace be {x}. m,1 ,x m,2 ,...,x m,M}, where m is the frequency band number m = 1, 2, ..., 8), where M is the total number of coefficients within this frequency band. The energy characteristic E of this frequency band... m The calculation formula is: , m=1,2,...,8); Where, x m,n This represents the amplitude of the coefficient of the nth wavelet packet in the mth frequency band. The energy contribution of this coefficient is summed to obtain the energy value for the entire frequency band. Changes in the energy distribution across different frequency bands directly reflect the increase or decrease in energy in the characteristic frequency bands for different fault types of the motor.

[0053] This module extracts three features from the time domain: peak value, kurtosis, and waveform factor; and two features from the electrical signal: current harmonic distortion rate and voltage imbalance. Combining the temperature rise rate and absolute temperature value collected by the temperature sensor, a standardized vector F=[F1,F2,...,F] containing 15 features is finally constructed. 15 The output is sent to the cross-domain feature fusion module.

[0054] refer to Figure 2 The cross-domain feature fusion module receives a 15-dimensional standardized feature vector output by the adaptive preprocessing module, strengthens key fault features through a convolutional attention network (CBAM), captures temporal dependencies through a gated recurrent unit (GRU), and optimizes multimodal fusion weights to provide discriminative feature representations for subsequent fault classification and remaining lifetime prediction.

[0055] Specifically, this module performs the following operations in sequence: S1, Convolutional Attention Network Attention Enhancement The channel attention branch is calculated as follows: ,in, and These represent global average pooling and global max pooling, respectively, used to aggregate spatial information; For multilayer perceptrons with shared weights; This is the Sigmoid activation function.

[0056] The spatial attention branch is calculated as follows: ,in, It is a convolutional layer with a kernel size of 7×7.

[0057] The final attention feature is: F′ not only retains the main information of the original feature F, but also strengthens the fault-related areas in the channel and spatial dimensions, and suppresses operating noise.

[0058] S2, Gated Cyclic Unit Timing Modeling The enhanced feature sequence {F1′,F2′,...,F} output by CBAM is used to... T The input is '} (T is the time series length) into the GRU network. GRU controls the flow of information through reset and update gates, avoiding the gradient vanishing problem of traditional RNNs. In this embodiment, the number of hidden units in GRU is set to 1024, which can fully integrate the temporal features of multimodal faults, while ensuring that the inference speed and storage usage of the model at the edge are within a reasonable range.

[0059] Let the input feature at time t be x. t (i.e. F) t (′), the hidden state in the previous time step was h t 1, then reset gate r t Update Gate Z t Candidate hidden state and the current hidden state h t The calculation formulas are as follows: ; ; ; ; , , This is the weight matrix. , , For bias terms, It is the Sigmoid activation function. The hidden state from the previous time step is used to reset the forgetting level of the gate control's previous information and update the fusion ratio between the current candidate state and the historical state of the gate control. The hidden state at the final time step aggregates the fault evolution information within the entire time window and serves as the input for subsequent fully connected layers.

[0060] S3, Multimodal Feature Adaptive Fusion Let the acoustic, vibration, electrical, and temperature eigenvectors be respectively... The fusion weight is (satisfy The weights are updated using a reinforcement learning algorithm, and the features are fused. .

[0061] The lightweight inference deployment module compresses the large-scale, high-precision model trained in the cloud into a lightweight model suitable for real-time operation at the edge. It compresses the model sequentially through three techniques: knowledge distillation, channel sparsity, and INT8 quantization, thereby achieving low-latency inference at the edge.

[0062] Knowledge distillation involves using a complex teacher model to guide a lightweight student model in learning. In this module, the teacher model employs a pre-trained ResNet-101 and GRU fusion architecture, which boasts a large number of parameters and high accuracy; the student model is a lightweight ResNet-18 variant, with approximately 1 / 5 the number of parameters of the teacher model.

[0063] The distillation loss function is: L hard For cross-entropy loss, L softTo soften the cross-entropy loss, the knowledge distillation loss weight λ=0.3 is an optimal empirical value that balances hard label supervision and soft label distribution learning, enabling the student model to better learn the fault classification decision logic of the teacher model.

[0064] Calculate the KL divergence between the student model output and the softened output of the teacher model to enable the student model to achieve fault classification accuracy comparable to that of the teacher model.

[0065] Even after knowledge distillation, the student model still contains some redundant convolutional channels. This module uses L1 regularization to impose sparsity constraints on the channel weights. A regularization term is added to the training loss function: , Among them, L task For the original task loss; W c Here, represents the weight vector of the c-th convolutional channel; η is the regularization coefficient, controlling the sparsity intensity, which is taken as 1 × 10 in this embodiment. 4 C represents the total number of channels. After training, channels with weights close to zero are considered redundant and undergo structural pruning to remove approximately 40% of the redundant channels, further reducing the model size.

[0066] The channel-sparsed model still uses 32-bit floating-point parameters. To further reduce inference latency, this module employs an asymmetric INT8 quantization scheme, mapping floating-point parameters to an integer range of 0-255.

[0067] The quantification formula is:

[0068] The inverse quantization formula is:

[0069] Where x is the original floating-point value, x q x is the quantized 8-bit integer. min and x max These are the minimum and maximum values ​​of the tensor to be quantized, respectively. `round` is the rounding function. Quantization reduces the model parameter size to about 1 / 4 of its original size, and integer operations are faster than floating-point operations.

[0070] After the above three-stage optimization, the model's inference latency on the NVIDIA Jetson Xavier NX edge gateway is ≤0.2 seconds, and the inference speed reaches 150 frames / second, meeting the requirements of real-time response in industrial settings.

[0071] The diagnostic results output module displays multimodal data time-series curves, fault type pie charts, remaining life countdown, and health status scores (0-100 points) in real time on a large screen.

[0072] Level-based alarms: Level 1 alarm (serious fault) triggers audible and visual alarms and shutdown signals; Level 2 alarm (general fault) triggers voice prompts; Level 3 alarm (early warning) only displays a warning indicator.

[0073] Data traceability: Supports querying historical data by motor number, diagnosis time, and fault type, with a storage period of ≥3 years, and supports fault source analysis and model iterative optimization.

[0074] The closed-loop control module has the functions of diagnostic result verification and adaptive adjustment of the acquisition strategy. Its specific working logic is as follows: Diagnostic confidence verification and secondary detection triggering mechanism: The closed-loop control module monitors the fault diagnosis confidence output by the cross-domain feature fusion module in real time. When the system's confidence in the current diagnosis result is lower than the threshold of 97% in this embodiment, or when the consistency verification between multimodal data fails, the system determines that the current diagnosis result is not credible enough and automatically triggers the secondary detection process. The system re-acquires the original multimodal data of the motor in one sampling period before and after the current moment, performs preprocessing and feature fusion, and performs inference again. The two diagnosis results are cross-validated, and the one with higher confidence is taken as the final output. If the confidence is still lower than the threshold after the secondary detection, the diagnosis result output module issues a "diagnosis uncertain" warning, prompting manual review.

[0075] The adaptive sampling and adjustment mechanism for operating condition awareness: The closed-loop control module monitors the motor's speed and load changes in real time. When a speed fluctuation reaches or exceeds 10% of the rated speed, or a load fluctuation reaches or exceeds 15% of the rated load, it is determined that a significant change in the current operating condition has occurred. At this time, the system automatically increases the sampling frequency of the multimodal data acquisition module from the current value to twice the original sampling frequency to capture transient fault characteristics during the sudden change in operating condition. Simultaneously, the adaptive preprocessing module adjusts the frequency band division strategy of the wavelet packet transform, dynamically shifting the originally fixed frequency band boundary according to the current speed fundamental frequency to ensure that feature extraction matches the changes in operating condition. After the speed and load return to a stable state and remain so for 30 seconds, the system automatically reverts to the reference sampling frequency and the default feature extraction strategy to balance diagnostic accuracy and edge computing resource consumption.

[0076] System test results To verify the technical effectiveness of the present invention, systematic tests were conducted on both publicly available datasets and in industrial field environments.

[0077] Dataset Validation: Offline validation was performed using the XJTU-SY motor fault dataset and the IEEE PHM 2012 challenge dataset. The system described in this invention was deployed on a cloud server configured with an NVIDIA A100 GPU. After training the model using the training set, it was evaluated on the test set. The results show that the fault classification accuracy reaches 99.37%. For latent faults such as early bearing wear and winding insulation aging, the system can issue an early warning at least 72 hours before the fault develops to the failure threshold, with an early fault identification lead time of ≥72 hours.

[0078] Industrial Pilot Test: Thirty asynchronous motors were selected at an automotive parts factory for a continuous 168-hour field test. The system edge was deployed on an industrial-grade edge gateway to collect and process vibration, acoustic, current, voltage, and temperature data in real time. A total of 47 faults and abnormal events occurred during the test. Statistical results showed that the system's false positive rate was less than 0.3%, the false negative rate was less than 0.15%, and the relative error between the predicted and actual remaining lifespan did not exceed ±3%.

[0079] Real-time performance testing: In the aforementioned industrial pilot environment, continuous sampling tests were conducted on the edge inference latency, with 1000 samples taken. Statistical results show that the average latency of a single inference operation is 0.18 seconds, the maximum is 0.21 seconds, the minimum is 0.15 seconds, the standard deviation is 0.02 seconds, and the inference operation latency does not exceed 0.2 seconds.

[0080] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No markings in the claims should be construed as limiting the scope of the claims.

Claims

1. A motor intelligent diagnostic system based on multimodal fusion, characterized in that, The system includes: The multimodal data acquisition module synchronously collects raw data on the motor's acoustics, vibration, electrical properties, and temperature through multiple sensors. The adaptive preprocessing module sequentially performs adaptive Kalman filtering, improved 3σ outlier processing, and wavelet packet transform on the raw data output by the multimodal data acquisition module to generate standardized feature vectors. The adaptive Kalman filtering performs time-domain filtering on the raw signal, and its output denoised signal serves as the input for the improved 3σ outlier processing. The improved 3σ outlier processing uses a fixed window length for sliding outlier detection, and performs cubic spline local reconstruction on outlier points using only a few neighboring points before and after their time sequence. The cross-domain feature fusion module first uses a convolutional attention network to enhance the channel and spatial dimensions of the standardized feature vector, then uses a gated recurrent unit to perform temporal modeling on the enhanced feature sequence, and finally performs adaptive weighted fusion of the temporal features of the four modes of acoustics, vibration, electrical and temperature to output the fused fault features. The lightweight inference deployment module compresses the model through knowledge distillation, channel sparsity and parameter quantization, and outputs the fault type, severity and remaining lifetime prediction values. The diagnostic results output module is used to display the diagnostic results on a visual terminal and issue early warnings based on a three-level alarm mechanism. The closed-loop control module is used to evaluate the confidence level and verify the consistency of multimodal data of the diagnostic results output by the cross-domain feature fusion module. When the verification fails or the confidence level does not meet the preset conditions, a secondary detection is triggered. The module is also used to adaptively adjust the sampling strategy according to the real-time monitored changes in the working conditions.

2. The system according to claim 1, characterized in that, The adaptive Kalman filter is used to remove noise interference under varying operating conditions based on the state equation and observation equation to obtain the true amplitude estimates of motor vibration, acoustic, current and other signals.

3. The system according to claim 1, characterized in that, The improved 3σ outlier processing uses a fixed time window to dynamically identify outliers in continuously acquired time-series data. Only when a data point deviates from the statistical center of the window by more than three times the statistical dispersion is it identified as a transient outlier. Then, a cubic spline basis function constrained by the neighboring observations before and after the outlier is called to perform local smoothing reconstruction. The reconstructed data is directly used as the input for time-frequency transformation.

4. The system according to claim 1, characterized in that, The wavelet packet transform divides the preprocessed signal into multi-resolution frequency bands based on a preset orthogonal wavelet basis function. The cumulative square amplitude of the wavelet coefficients in each sub-band is used as the energy characterization value of that frequency band. A fixed number of sub-band energy values ​​are extracted in order from low to high frequency and combined with the time-domain waveform feature parameters extracted from the original signal to form a standardized multi-source feature tensor.

5. The system according to claim 1, characterized in that, The adaptive weighted fusion is as follows: Let the acoustic, vibration, electrical, and temperature feature vectors be respectively... The fusion weights are respectively ,satisfy The weights are updated using a reinforcement learning algorithm, and the features are fused into... .

6. The system according to claim 1, characterized in that, The channel attention branch of the convolutional attention network sequentially performs global pooling, multilayer perceptron mapping, and sigmoid activation on the input feature tensor to generate channel weights. Its spatial attention branch sequentially performs channel-dimensional pooling, convolution, and sigmoid activation on the input feature tensor to generate spatial weights; then the channel weights and spatial weights are fused with the original features to output enhanced features.

7. The system according to claim 1, characterized in that, The gated loop unit includes a reset gate and an update gate. The reset gate is used to adjust the influence of the previous hidden state on the candidate hidden state, and the update gate is used to balance the weights of the previous hidden state and the current candidate state in the output hidden state. In this way, the input enhanced feature sequence is time-series modeled, and the output hidden state vector contains fault time-series information.

8. The system according to claim 1, characterized in that, The lightweight inference deployment module performs multi-level compression on the model, including: Knowledge distillation: A pre-trained ResNet-101 fusion architecture with gated recurrent units is used as the teacher model, and a lightweight ResNet-18 variant is used as the student model. The student model is guided to mimic the output of the teacher model through the distillation loss function. Channel filtering: Apply L1 regularization sparsity constraints to the convolutional channels of the knowledge-distilled student model, and remove redundant channels with weight values ​​below a preset threshold after training; Parameter quantization: An asymmetric quantization scheme is adopted, which uses the quantization formula and the dequantization formula to map the 32-bit floating-point parameter to the integer range of 0 to 255, thereby reducing the parameter size.

9. The system according to claim 1, characterized in that, The closed-loop control module monitors the fault diagnosis confidence level output by the cross-domain feature fusion module in real time. When the system's confidence level for the current diagnosis result is lower than a preset threshold, or when the consistency check between multimodal data fails, the system determines that the current diagnosis result is not credible enough and automatically triggers a secondary detection process. The system reacquires the original multimodal data of the motor within one sampling period before and after the current moment, performs preprocessing and feature fusion, and performs inference again. The two diagnosis results are cross-validated, and the one with higher confidence is taken as the final output. If the confidence level is still lower than the threshold after the secondary detection, the diagnosis result output module issues a "diagnosis uncertain" warning, prompting manual review.

Images