Motor state monitoring method and device based on multi-mode sensor

Abstract

The invention discloses a motor state monitoring method and device based on a multi-modal sensor, and the method comprises the steps: firstly, carrying out the comprehensive analysis of the current and rotating speed data of a motor, and explicitly constructing a working condition vector which can accurately represent the current load and rotating speed state of the motor; then, self-adaptive adjustment and reconstruction are carried out on original vibration frequency spectrum features and current frequency spectrum features through modes such as an attention mechanism by taking a working condition vector as guidance, so that feature components introduced by working condition changes are stripped, and pure features which are highly related to a fault state and basically irrelevant to a working condition state are extracted; and furthermore, the decoupled pure features are fused to effectively excavate the complementary relationship between vibration and current information, and finally, accurate fault diagnosis is realized. In this way, the accuracy and reliability of motor state monitoring are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent monitoring, and more specifically, to a method and apparatus for monitoring the condition of a motor based on a multimodal sensor. Background Technology

[0002] As an indispensable core power device in modern industrial production and daily life, electric motors are used in many key fields such as intelligent manufacturing, rail transportation, and new energy vehicles. The operating status of the motor directly affects the stability and production efficiency of the entire system. Once an unexpected motor failure occurs, it may not only cause production line shutdowns and huge economic losses, but also, in some high-risk scenarios, even lead to serious safety accidents. Therefore, in order to ensure the continuity of production and improve the reliability and safety of equipment operation, developing a solution that can monitor the health status of motors in real time and provide early warnings of potential faults has extremely important practical significance and application value.

[0003] To achieve the above objectives, existing technologies typically employ sensor signal analysis methods for motor condition monitoring. Vibration signal analysis and motor current signal analysis are two of the most widely used approaches. Traditional solutions rely on signal processing techniques combined with expert experience to set fault thresholds. However, with the development of artificial intelligence, automatically learning fault characteristics from massive amounts of vibration or current data using machine learning or deep learning models has become a mainstream research direction. However, these existing solutions still face a major challenge in practical applications: the operating conditions of motors in real industrial scenarios (such as load and speed) are dynamic, not static. These changes in operating conditions cause severe and complex non-fault-related fluctuations in the vibration and current signals across the spectrum. The characteristic components generated by these fluctuations are often intertwined and coupled with actual fault characteristic components. Traditional monitoring models struggle to effectively distinguish whether signal changes are caused by changes in operating conditions or by motor faults, resulting in poor model generalization ability and a high rate of false alarms and false negatives when facing variable operating conditions, severely limiting their reliability in practical deployments.

[0004] Therefore, an optimized motor condition monitoring scheme based on multimodal sensors is needed. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a motor condition monitoring method and apparatus based on multimodal sensors. First, by comprehensively analyzing the motor's current and speed data, a condition vector is explicitly constructed that accurately characterizes the motor's current load and speed state. Next, guided by the condition vector, the original vibration and current spectrum features are adaptively adjusted and reconstructed using attention mechanisms and other methods to remove feature components introduced by changes in operating conditions, thereby extracting pure features that are highly correlated with the fault state and largely unrelated to the operating condition state. Furthermore, these decoupled pure features are fused to effectively uncover the complementary relationship between vibration and current information, ultimately achieving accurate fault diagnosis. This approach improves the accuracy and reliability of motor condition monitoring.

[0006] According to one aspect of this application, a method for monitoring the condition of a motor based on a multimodal sensor is provided, comprising: Obtain the original vibration current, original current, and original rotational speed current; Vibration spectrum vectors and current spectrum vectors are extracted from the original vibration flow and original current flow based on Fast Fourier Transform; Extract the operating condition vector from the raw current flow and raw speed flow; Based on the operating condition vector, the vibration spectrum vector and the current spectrum vector are decoupled in a condition-guided manner to obtain the decoupled vibration spectrum vector and the decoupled current spectrum vector. Deep feature encoding is performed on the decoupled vibration spectrum vector and the decoupled current spectrum vector to obtain the decoupled vibration feature vector and the decoupled current feature vector. Multimodal feature fusion is performed on the decoupled vibration feature vector and the decoupled current feature vector to obtain the multimodal fused feature vector of the motor state; Based on the multimodal fusion feature vector of motor status, motor status monitoring results are generated, which include fault categories and their confidence levels.

[0007] According to another aspect of this application, a motor condition monitoring device based on a multimodal sensor is provided, comprising: The raw data acquisition module is used to acquire the raw vibration flow, raw current flow, and raw rotational speed flow. The spectrum feature extraction module is used to extract vibration spectrum vectors and current spectrum vectors from the original vibration flow and the original current flow based on the fast Fourier transform. The operating condition feature extraction module is used to extract operating condition vectors from the raw current flow and the raw speed flow. The feature decoupling module is used to perform condition-guided feature decoupling of vibration spectrum vector and current spectrum vector based on the working condition vector to obtain decoupled vibration spectrum vector and decoupled current spectrum vector. The deep feature encoding module is used to perform deep feature encoding on the decoupled vibration spectrum vector and the decoupled current spectrum vector to obtain the decoupled vibration feature vector and the decoupled current feature vector. The motor state multimodal fusion module is used to perform multimodal feature fusion on the decoupled vibration feature vector and the decoupled current feature vector to obtain the motor state multimodal fusion feature vector; The motor condition monitoring module is used to generate motor condition monitoring results based on the multimodal fusion feature vector of motor condition. The motor condition monitoring results include fault categories and their confidence levels.

[0008] Compared with existing technologies, this application provides a motor condition monitoring method and device based on multimodal sensors. First, by comprehensively analyzing the motor's current and speed data, a condition vector is explicitly constructed to accurately characterize the motor's current load and speed state. Then, guided by the condition vector, the original vibration and current spectrum features are adaptively adjusted and reconstructed using attention mechanisms to remove feature components introduced by changes in operating conditions, thereby extracting pure features that are highly correlated with the fault state and largely independent of the operating condition. These decoupled pure features are further fused to effectively uncover the complementary relationship between vibration and current information, ultimately achieving accurate fault diagnosis. This approach improves the accuracy and reliability of motor condition monitoring. Attached Figure Description

[0009] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 This is a flowchart of a motor condition monitoring method based on a multimodal sensor according to an embodiment of this application; Figure 2 This is a schematic diagram of the data flow of a motor condition monitoring method based on a multimodal sensor according to an embodiment of this application; Figure 3 This is a block diagram of a motor condition monitoring device based on a multimodal sensor according to an embodiment of this application. Detailed Implementation

[0011] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0012] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0013] While this application makes various references to certain modules of the systems according to embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The modules described are merely illustrative, and different aspects of the systems and methods may use different modules.

[0014] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0015] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0016] The technical solution of this application proposes a method for monitoring motor condition based on multimodal sensors. Figure 1 This is a flowchart of a motor condition monitoring method based on a multimodal sensor according to an embodiment of this application. Figure 2 This is a schematic diagram of the data flow in a motor condition monitoring method based on a multimodal sensor according to an embodiment of this application. Figure 1 and Figure 2As shown, the motor condition monitoring method based on multimodal sensors according to an embodiment of this application includes the following steps: S1, acquiring the original vibration flow, original current flow, and original rotational speed flow; S2, extracting vibration spectrum vectors and current spectrum vectors from the original vibration flow and original current flow based on Fast Fourier Transform; S3, extracting operating condition vectors from the original current flow and original rotational speed flow; S4, performing operating condition-guided feature decoupling on the vibration spectrum vector and current spectrum vector based on the operating condition vector to obtain decoupled vibration spectrum vectors and decoupled current spectrum vectors; S5, performing deep feature encoding on the decoupled vibration spectrum vector and decoupled current spectrum vector to obtain decoupled vibration feature vectors and decoupled current feature vectors; S6, performing multimodal feature fusion on the decoupled vibration feature vector and decoupled current feature vector to obtain a multimodal fused feature vector of motor condition; S7, generating motor condition monitoring results based on the multimodal fused feature vector of motor condition, wherein the motor condition monitoring results include fault categories and their confidence levels.

[0017] Specifically, S1 involves acquiring the raw vibration current, raw current, and raw speed data. It should be understood that data from a single type of sensor is often insufficient for accurately and robustly diagnosing complex motor faults. The operating state of a motor is the result of the combined effects of its electrical characteristics, mechanical characteristics, and operating conditions. Therefore, by simultaneously acquiring data from three different modes—vibration, current, and speed—a more complete and comprehensive state profile can be constructed. Specifically, vibration signals primarily reflect the mechanical health of the motor, such as bearing wear and rotor imbalance; current signals not only reveal the condition of the power supply system but also indirectly reflect changes in the motor load and internal electromagnetic field, and are related to various faults; while speed directly defines the core operating conditions of the motor. Here, the data provided by different sensors can complement each other, thereby effectively improving the accuracy and reliability of subsequent fault detection.

[0018] In a specific example of this application, vibration raw flow, current raw flow, and speed raw flow can be acquired by deploying a multimodal sensor system working collaboratively on the target motor. Specifically, firstly, to acquire the vibration raw flow, a suitable vibration sensor (such as an accelerometer) needs to be selected and installed at a key location that can most effectively capture the vibration of the motor structure, such as the motor housing. The sensor will continuously measure the acceleration change at that point and convert it into an electrical signal, forming time-series data. Secondly, to acquire the current raw flow, a current sensor (such as a current clamp) needs to be used and installed on one or more phases of the motor's power supply line to monitor the amount of current flowing through the motor in real time in a non-invasive manner. To ensure data time alignment, the acquisition of the current signal usually needs to be synchronized with the acquisition of the vibration signal. Finally, to acquire the speed raw flow, a speed sensor (such as a photoelectric or Hall effect sensor) needs to be deployed near the motor shaft. This sensor calculates the motor's rotational speed in real time by detecting the number of times a mark is passed on the shaft and outputs the speed reading at a certain frequency. These three data streams together constitute the multimodal input required for subsequent analysis.

[0019] Specifically, S2 extracts the vibration spectrum vector and current spectrum vector from the original vibration and current flows based on the Fast Fourier Transform. It should be understood that the original vibration and current signals exist as time series, directly showing the change in signal amplitude over time. However, in this representation, many periodic characteristic frequency components related to specific faults (such as bearing damage, rotor imbalance, gear wear, etc.) are hidden and not intuitive. Through the Fourier Transform, the signal can be decomposed into its constituent sinusoidal frequency components, making these characteristics appear clearly as peaks in the spectrum, thus transforming a time-domain problem that is difficult to analyze directly into a frequency-domain problem that is easier to identify patterns in.

[0020] In a specific example of this application, the vibration spectrum vector and the current spectrum vector can be extracted from the original vibration flow and the original current flow through the following steps: First, the original vibration flow and the original current flow are processed based on the Hanning window to obtain the vibration Hanning window signal and the current Hanning window signal; then, the vibration Hanning window signal and the current Hanning window signal are subjected to a fast Fourier transform to obtain the vibration spectrum vector and the current spectrum vector.

[0021] The Hanning window is a window function whose shape smoothly approaches zero at both ends. It is applied to truncated data segments before the Fourier transform to reduce spectral leakage caused by signal truncation. This leakage causes the true frequency components to diffuse to neighboring frequencies, interfering with the accuracy of the analysis. The Fast Fourier Transform (FFT) is an efficient computational algorithm for the Discrete Fourier Transform (DFT), significantly reducing computational complexity and making spectral analysis of long data sequences possible on a computer.

[0022] In practice, the system first extracts a fixed-length data segment from the continuous raw vibration and current flows, for example, a data window containing N sampling points. Then, the Hanning window function is multiplied point-by-point with this data window. This process is performed independently for the vibration and current data segments to obtain windowed vibration and current Hanning window signals with smoothed and suppressed amplitudes at both ends. Next, the vibration Hanning window signal generated in the previous step is used as input, and a Fast Fourier Transform (FFT) algorithm is applied to calculate its corresponding complex spectrum. Typically, the modulus of these complex values ​​is taken to obtain a sequence of real numbers representing the amplitudes of each frequency component; this sequence constitutes the final vibration spectrum vector. Similarly, the exact same FFT and amplitude calculation process is performed on the current Hanning window signal to obtain the current spectrum vector.

[0023] Specifically, S3 extracts the operating condition vector from the raw current flow and raw speed flow. It should be understood that the vibration and current spectrum characteristics of a motor are not static; they change significantly with the motor's operating conditions (mainly speed and load). For example, a minor defect in the same bearing will produce drastically different vibration signal characteristic frequencies and amplitudes under high-speed heavy load and low-speed light load conditions. Without considering the influence of operating conditions, normal changes in operating conditions can easily be misjudged as faults, or fault signals can be buried in the background noise of these changes. Therefore, in the technical solution of this application, the operating condition vector is extracted to remove normal features strongly correlated with the operating conditions, thereby highlighting the abnormal features truly caused by faults. The operating condition vector is a compact, low-dimensional numerical vector used to quantitatively summarize the main operating parameters of the motor during the analysis period.

[0024] In a specific example of this application, the operating condition vector can be extracted from the raw current flow and the raw speed flow through the following steps: First, the mean speed value in the raw speed flow is calculated to obtain the average speed; then, the root mean square of the current value in the raw current flow is calculated to obtain the current RMS value; furthermore, the average speed and the current RMS value are vectorized to obtain the operating condition vector. That is, the two scalars, the calculated average speed and the current RMS value, are combined into a two-dimensional vector, i.e., the operating condition vector, in a predetermined order (e.g., [average speed, current RMS value]).

[0025] Among them, the average speed is a direct reflection of the motor's operating speed, while the current RMS (root mean square) value is widely used as an effective indicator to measure the motor's load level, because the motor's output torque and power are closely related to its current consumption.

[0026] Specifically, in step S4, based on the operating condition vector, the vibration spectrum vector and current spectrum vector are decoupled according to the operating condition to obtain the decoupled vibration spectrum vector and decoupled current spectrum vector. It should be understood that the normal vibration and current spectra of a motor will exhibit significant differences under different operating conditions (such as different speeds and different loads). If these spectral features caused by changes in operating conditions are mixed with abnormal features caused by faults, they will greatly interfere with the judgment of the diagnostic model, leading to a high false alarm rate or false negative rate. Therefore, in the technical solution of this application, the operating condition vector is used to extract features strongly correlated with the operating condition from the vibration and current spectrum vectors, allowing subsequent analysis to focus more on fault-related features. That is, based on the current real-time operating condition, those normal spectral components highly correlated with that operating condition are intelligently identified and suppressed. In this way, the monitoring system can have operating condition adaptive capabilities, improving its diagnostic robustness and accuracy in complex and variable operating environments.

[0027] In a specific example of this application, the vibration spectrum vector and the current spectrum vector can be decoupled under operating conditions through the following steps: First, the operating condition vector is input into the operating condition encoder to obtain the attention weight vector. Then, the element-wise multiplication of the attention weight vector with the vibration spectrum vector is calculated to obtain the decoupled vibration spectrum vector; similarly, the element-wise multiplication of the attention weight vector with the current spectrum vector is calculated to obtain the decoupled current spectrum vector.

[0028] The operating condition encoder is a small-scale feedforward neural network. The activation function of this small-scale feedforward neural network is the sigmoid activation function, which maps the low-dimensional operating condition vector into a high-dimensional attention weight vector. The dimension of this vector is exactly the same as the spectrum vector, and the value of each element is between 0 and 1. Each weight value in this vector quantitatively represents the degree of correlation between the feature of the corresponding frequency point and the current operating condition. The higher the correlation, the closer the weight value is to 0; conversely, the lower the correlation, the more likely the frequency feature is caused by factors unrelated to the operating condition (such as a fault), and the closer its weight value is to 1.

[0029] Specifically, in step S5, deep feature encoding is performed on the decoupled vibration spectrum vector and the decoupled current spectrum vector to obtain the decoupled vibration feature vector and the decoupled current feature vector. It should be understood that although the decoupled spectrum vector highlights frequency components more relevant to the fault, it is still essentially a relatively primitive and high-dimensional representation. The complex combinations of different frequency points, harmonic structures, and weak but crucial patterns in the vector are difficult to capture effectively using traditional methods. Therefore, in the technical solution of this application, deep feature encoding is performed on the decoupled vibration spectrum vector and the decoupled current spectrum vector. That is, a deep neural network (a neural network with multiple hidden layers) is used to map the input data to a new feature space. In this way, through a deep learning model, the system can automatically learn and discover these deep, discriminative patterns hidden in the spectrum data, thereby extracting more advanced and robust fault features, laying a solid foundation for the final accurate diagnosis.

[0030] In a specific example of this application, one-dimensional convolutional encoding is performed on the decoupled vibration spectrum vector and the decoupled current spectrum vector to obtain the decoupled vibration feature vector and the decoupled current feature vector, respectively. In this process, a one-dimensional convolutional neural network (1D-CNN) captures local patterns by sliding one-dimensional convolutional kernels across the sequence and learns hierarchical features from local to global by stacking multiple convolutional layers. Specifically, this process is implemented in software as two parallel, structurally similar but parameter-independent encoding channels: for vibration modes, the system takes the decoupled vibration spectrum vector as input and feeds it into a one-dimensional convolutional neural network specifically pre-trained for vibration signal feature learning. Inside the one-dimensional convolutional neural network, multiple one-dimensional convolutional kernels perform sliding convolution operations along the dimension of the spectrum vector, with each kernel responsible for identifying a specific local spectral pattern (e.g., the shape of a frequency peak, the relative relationship of a set of harmonics, etc.). The output of the convolution operation is processed by a nonlinear activation function (such as ReLU) and then typically downsampled through pooling layers (such as MaxPooling) to enhance the translation invariance of vibration features and reduce data dimensionality. By stacking several such "convolution-activation-pooling" layers, the network can progressively construct a hierarchical representation of vibration features, ranging from simple local patterns to complex global patterns. Finally, a flattened one-dimensional vector, the decoupled vibration feature vector, is output at the top of the one-dimensional convolutional neural network. For current modes, the system employs the exact same workflow, inputting the decoupled current spectrum vector into another one-dimensional convolutional neural network pre-trained specifically for current signal feature learning, ultimately generating the decoupled current feature vector.

[0031] Specifically, in step S6, multimodal feature fusion is performed on the decoupled vibration feature vector and the decoupled current feature vector to obtain a multimodal fused feature vector of the motor state. It should be understood that after decoupling guided by operating conditions, the vibration feature vector can effectively isolate load or speed variation interference and focus on the physical anomalies of mechanical components (such as bearings and rotors), while the current feature vector is better at capturing hidden fault signs in the electrical system (such as winding insulation and magnetic circuit imbalance). However, motor faults are essentially complex phenomena of electromechanical coupling. Relying solely on single-mode information can easily overlook the weak collaborative signs of early faults in multiple physical fields, especially under complex operating conditions of varying speeds and loads. This fragmented analysis will lead to the loss or misjudgment of key fault clues. Therefore, in the technical solution of this application, by utilizing a multimodal fusion mechanism to deeply mine the potential causal relationships and complementary enhancement relationships between the decoupled vibration and current feature vectors, the model gains fault tracing capabilities beyond those of a single mode. This approach significantly improves the confidence level of detecting minor faults, providing robust and interpretable feature support for predictive maintenance decisions.

[0032] Specifically, firstly, the fine-grained global response matrix of the vibration eigenvector relative to the decoupled current eigenvector is calculated. It should be understood that although condition-guided characteristic decoupling has eliminated the interference of load speed variations on single-mode characteristics, the inherent correlation between mechanical vibration and electromagnetic current during fault occurrence remains implicit in the deep characteristics of both modes. If only the decoupled single-mode characteristics are analyzed independently, the system will overlook the symbiosis and causal chain of this cross-domain fault characteristic, making it difficult to form a systematic understanding of the fault mechanism. Therefore, in the technical solution of this application, the fine-grained global response matrix of vibration characteristics to current characteristics is calculated to systematically scan all potential correlation paths between the two modes in a high-dimensional feature space. This ensures that no possible fault coupling channels are overlooked, while the fine-grained characteristics enable the system to distinguish between covariant characteristics caused by the same fault source and pseudo-correlation caused by random noise, thus allowing the system to accurately reconstruct the complete fault propagation path at the feature level.

[0033] In a specific example of this application, the vibration-current global fine-grained response matrix relative to the decoupled current eigenvector is calculated using the following formula: [Formula omitted for brevity]

[0034] in, This represents the global fine-grained asymmetric interaction response matrix between vibration and current. This is a fine-grained element-level gated interaction response matrix for the global vibration-current domain. express function, It is a dynamic weight matrix. Let be the projection matrix. It is the decoupled vibration characteristic vector. It is the transpose of the decoupled current eigenvector. This represents vector multiplication. This indicates dot product by position. This indicates addition by position. express function, It is the global fine-grained response matrix of the vibration-current.

[0035] Next, the global fine-grained response matrix of vibration and current is decomposed into column vectors to obtain the sequence distribution of local response encoding vectors of vibration and current. Although the global fine-grained response matrix fully records the complex relationship between the two modal characteristics, this global static representation is difficult to capture the evolution logic of fault characteristics during transient operating conditions. Therefore, in the technical solution of this application, through column decomposition, each local response encoding vector represents the microscopic interaction state of vibration and current within a specific frequency band or time window (such as "the phase-locked relationship between current harmonics and shell vibration in the 2kHz frequency band at a certain speed"). This structured representation allows subsequent models to gradually track the continuous changes of characteristic coupling along the frequency axis or time axis, thereby distinguishing the essential difference between "transient operating condition disturbances" and "persistent fault symptoms".

[0036] In a specific example of this application, the vibration-current global fine-grained response matrix is ​​decomposed into column vectors using the following formula to obtain the sequence distribution of the vibration-current local response encoding vector; wherein, the formula is:

[0037] in, This represents the matrix decomposition operation. and These are the 1st, 2nd, and 3rd elements in the sequence distribution of the vibration-current local response encoding vector. The and the first A vibration-current local response encoding vector.

[0038] Furthermore, the sequence distribution of the vibration-current local response encoding vector is input into a response inference generation engine based on a forward LSTM model to obtain multimodal feature fusion, resulting in a multimodal fused feature vector of the motor state. It should be understood that the propagation delay of fault features on the time axis is strongly correlated with the instantaneous impact of operating condition disturbances. Analyzing only a single local response vector cannot capture the causal logic between modes. Although the sequence distribution of the vibration-current local response encoding vector preserves local fragments of spectral interaction, it lacks the ability to model the temporal logic of feature evolution. Therefore, in the technical solution of this application, through the unique gating mechanism of LSTM (forget gate to filter instantaneous operating condition disturbances, input gate to enhance persistent fault features, and output gate to control state transmission), the model can gradually analyze the interactive evolution law of vibration and current features along the sequence distribution. This improves the system's early warning capability for progressive faults and its robustness against transient disturbances.

[0039] In a specific example of this application, the sequence distribution of the vibration-current local response encoding vector is input into a response inference generation engine based on a forward LSTM model using the following formula to obtain multimodal feature fusion and thus a multimodal fused feature vector of the motor state; wherein, the formula is:

[0040] in, Represents temporal response reasoning, It is the multimodal fusion feature vector of the motor state.

[0041] Specifically, in step S7, a motor condition monitoring result is generated based on the multimodal fusion feature vector of the motor condition. This motor condition monitoring result includes the fault category and its confidence level. In the technical solution of this application, the multimodal fusion feature vector of the motor condition is input into a classifier-based motor condition monitor to obtain the motor condition monitoring result, which includes the fault category and its confidence level. It should be understood that the multimodal fusion feature vector of the motor condition is essentially a set of high-dimensional numerical values, which cannot directly indicate the specific fault condition of the motor. Therefore, in the technical solution of this application, a complex mapping relationship between the fault category and its confidence level from the multimodal fusion feature vector of the motor condition is learned and established by a classifier to generate the motor condition monitoring result.

[0042] The classifier is a pre-trained machine learning or deep learning model, which is essentially a trained supervised learning model, such as a support vector machine (SVM), decision tree, random forest, or deep neural network (DNN). The training process of this model is usually carried out on a dataset containing a large number of labeled motors.

[0043] Here, fault categories refer to a predefined set of various possible operating states of the motor. The number of fault categories the system can identify depends on the scope of its training dataset. Confidence level refers to a probability value associated with each fault category, ranging from 0 to 1 (or represented as 0% to 100%). It quantifies the degree of certainty the model has in its judgment. For example, a confidence level of 95% for a fault category means that the model is very confident that the motor is in that fault state. The confidence levels of all categories together form a probability distribution, providing rich information for evaluation and decision-making.

[0044] Taking the scheme of this application as an example, after inputting the multimodal fusion feature vector of the motor state into the classifier-based motor state monitor, the monitor performs forward propagation calculation, and finally the Softmax layer outputs a probability distribution. A possible output result is: "Fault A, 30%; Fault B, 15%; Fault C, 45%; Fault D, 10%". Then the final classifier output is: Fault C, 45%.

[0045] In summary, the motor condition monitoring method based on multimodal sensors according to the embodiments of this application is explained. First, by comprehensively analyzing the motor's current and speed data, a condition vector is explicitly constructed that accurately characterizes the motor's current load and speed state. Then, guided by the condition vector, the original vibration and current spectrum features are adaptively adjusted and reconstructed using attention mechanisms and other methods to remove feature components introduced by changes in condition, thereby extracting pure features that are highly correlated with the fault state and largely unrelated to the condition state. Furthermore, these decoupled pure features are fused to effectively mine the complementary relationship between vibration and current information, ultimately achieving accurate fault diagnosis. This approach improves the accuracy and reliability of motor condition monitoring.

[0046] Furthermore, a motor condition monitoring device based on a multimodal sensor is also provided.

[0047] Figure 3 This is a block diagram of a motor condition monitoring device based on a multimodal sensor according to an embodiment of this application. Figure 3As shown, the motor condition monitoring device 300 based on a multimodal sensor according to an embodiment of this application includes: a raw data acquisition module 310 for acquiring raw vibration flow, raw current flow, and raw speed flow; a spectrum feature extraction module 320 for extracting vibration spectrum vectors and current spectrum vectors from the raw vibration flow and raw current flow based on fast Fourier transform; a working condition feature extraction module 330 for extracting working condition vectors from the raw current flow and raw speed flow; and a feature decoupling module 340 for performing working condition-guided feature decoupling on the vibration spectrum vector and current spectrum vector based on the working condition vector to obtain decoupled vibration. The system includes a spectral vector and a decoupled current spectral vector; a deep feature encoding module 350, used to perform deep feature encoding on the decoupled vibration spectral vector and the decoupled current spectral vector to obtain a decoupled vibration feature vector and a decoupled current feature vector; a motor state multimodal fusion module 360, used to perform multimodal feature fusion on the decoupled vibration feature vector and the decoupled current feature vector to obtain a motor state multimodal fusion feature vector; and a motor state monitoring module 370, used to generate motor state monitoring results based on the motor state multimodal fusion feature vector, wherein the motor state monitoring results include fault categories and their confidence levels.

[0048] As described above, the multimodal sensor-based motor condition monitoring device 300 according to the embodiments of this application can be implemented in various wireless terminals, such as servers with multimodal sensor-based motor condition monitoring algorithms. In one possible implementation, the multimodal sensor-based motor condition monitoring device 300 according to the embodiments of this application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the multimodal sensor-based motor condition monitoring device 300 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the multimodal sensor-based motor condition monitoring device 300 can also be one of many hardware modules of the wireless terminal.

[0049] Alternatively, in another example, the multimodal sensor-based motor condition monitoring device 300 and the wireless terminal can also be separate devices, and the multimodal sensor-based motor condition monitoring device 300 can be connected to the wireless terminal via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0050] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for monitoring the condition of a motor based on a multimodal sensor, characterized in that, include: Obtain the original vibration current, original current, and original rotational speed current; Vibration spectrum vectors and current spectrum vectors are extracted from the original vibration flow and original current flow based on Fast Fourier Transform; Extract the operating condition vector from the raw current flow and raw speed flow; Based on the operating condition vector, the vibration spectrum vector and the current spectrum vector are decoupled in a condition-guided manner to obtain the decoupled vibration spectrum vector and the decoupled current spectrum vector. Deep feature encoding is performed on the decoupled vibration spectrum vector and the decoupled current spectrum vector to obtain the decoupled vibration feature vector and the decoupled current feature vector. Multimodal feature fusion is performed on the decoupled vibration feature vector and the decoupled current feature vector to obtain the multimodal fused feature vector of the motor state; Based on the multimodal fusion feature vector of motor status, motor status monitoring results are generated, which include fault categories and their confidence levels.

2. The motor condition monitoring method based on multimodal sensors according to claim 1, characterized in that, Based on Fast Fourier Transform, vibration spectrum vectors and current spectrum vectors are extracted from the original vibration flow and original current flow, including: The vibration raw current and current raw current are processed based on the Hanning window to obtain the vibration Hanning window signal and the current Hanning window signal. Fast Fourier transform is performed on the vibration Hanning window signal and the current Hanning window signal to obtain the vibration spectrum vector and the current spectrum vector.

3. The motor condition monitoring method based on multimodal sensors according to claim 1, characterized in that, The operating condition vector is extracted from the raw current and raw speed flow, including: The mean value of the rotational speed in the original flow is calculated to obtain the average rotational speed. Calculate the root mean square of the original current value to obtain the current RMS value; The average speed and current RMS value are vectorized to obtain the operating condition vector.

4. The motor condition monitoring method based on multimodal sensors according to claim 3, characterized in that, Based on the operating condition vector, operating condition-guided feature decoupling is performed on the vibration spectrum vector and the current spectrum vector to obtain the decoupled vibration spectrum vector and the decoupled current spectrum vector, including: Input the condition vector into the condition encoder to obtain the attention weight vector; The element-wise multiplication of the attention weight vector and the vibration spectrum vector is calculated to obtain the decoupled vibration spectrum vector; The attention weight vector is calculated by multiplying the current spectrum vector element by element to obtain the decoupled current spectrum vector.

5. The motor condition monitoring method based on multimodal sensors according to claim 4, characterized in that, The operating condition encoder is a small feedforward neural network, and the activation function of the small feedforward neural network is the Sigmoid activation function.

6. The motor condition monitoring method based on multimodal sensors according to claim 1, characterized in that, Deep feature encoding is performed on the decoupled vibration spectrum vector and the decoupled current spectrum vector to obtain the decoupled vibration feature vector and the decoupled current feature vector, including: One-dimensional convolutional encoding is performed on the decoupled vibration spectrum vector and the decoupled current spectrum vector respectively to obtain the decoupled vibration feature vector and the decoupled current feature vector.

7. The motor condition monitoring method based on multimodal sensors according to claim 1, characterized in that, Multimodal feature fusion is performed on the decoupled vibration feature vector and the decoupled current feature vector to obtain the multimodal fused feature vector of the motor state, including: Calculate the vibration-current global fine-grained response matrix relative to the decoupled vibration eigenvector and the decoupled current eigenvector. The vibration-current global fine-grained response matrix is ​​decomposed into column vectors to obtain the sequence distribution of the vibration-current local response encoding vectors; The sequence distribution of the vibration-current local response encoding vector is input into the response inference generation engine based on the forward LSTM model to obtain multimodal feature fusion to obtain the multimodal fused feature vector of the motor state.

8. The motor condition monitoring method based on multimodal sensors according to claim 1, characterized in that, Based on the multimodal fusion feature vector of motor condition, motor condition monitoring results are generated. These results include fault categories and their confidence levels, including: The multimodal fusion feature vector of motor status is input into a classifier-based motor status monitor to obtain motor status monitoring results, which include fault categories and their confidence levels.

9. A motor condition monitoring device based on multimodal sensors, characterized in that, include: The raw data acquisition module is used to acquire the raw vibration flow, raw current flow, and raw rotational speed flow. The spectrum feature extraction module is used to extract vibration spectrum vectors and current spectrum vectors from the original vibration flow and the original current flow based on the fast Fourier transform. The operating condition feature extraction module is used to extract operating condition vectors from the raw current flow and the raw speed flow. The feature decoupling module is used to perform condition-guided feature decoupling of vibration spectrum vector and current spectrum vector based on the working condition vector to obtain decoupled vibration spectrum vector and decoupled current spectrum vector. The deep feature encoding module is used to perform deep feature encoding on the decoupled vibration spectrum vector and the decoupled current spectrum vector to obtain the decoupled vibration feature vector and the decoupled current feature vector. The motor state multimodal fusion module is used to perform multimodal feature fusion on the decoupled vibration feature vector and the decoupled current feature vector to obtain the motor state multimodal fusion feature vector; The motor condition monitoring module is used to generate motor condition monitoring results based on the multimodal fusion feature vector of motor condition. The motor condition monitoring results include fault categories and their confidence levels.

Images