Fault detection method and device for industrial equipment, medium and electronic equipment

Abstract

The invention relates to a fault detection method and device for industrial equipment, a medium and electronic equipment, and relates to the technical field of data processing, and the method comprises the steps: collecting multi-modal data, and carrying out the preprocessing of the multi-modal data, the multi-modal data comprises at least one modal data of vibration data, current data, temperature data and sound data; mutual information of the multi-modal data is acquired, and weight information of each modal data is determined based on the mutual information; performing feature fusion on the multi-modal data obtained by preprocessing according to the weight information to obtain a target feature; and inputting the target feature into a hybrid neural network to obtain a fault prediction result, and executing an early warning operation according to the fault prediction result. According to the invention, the weight information of different modalities can be dynamically acquired by using the mutual information, so that the accuracy of fault detection can be improved to a certain extent.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more specifically, to a fault detection method, apparatus, medium, and electronic equipment for industrial equipment. Background Technology

[0002] Currently, modal fusion technology has been gradually applied to the field of industrial equipment fault early warning, providing important support for ensuring continuous equipment operation and reducing the risk of unplanned downtime. Fault early warning mainly improves the accuracy of early warning through multi-sensor data acquisition and analysis to compensate for the limitations of single-dimensional monitoring. However, most multi-modal data fusion uses fixed weights, resulting in low accuracy of fault detection. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a fault detection method, apparatus, medium and electronic equipment for industrial equipment.

[0004] In a first aspect, this disclosure provides a fault detection method for industrial equipment, the method comprising: Collect multimodal data and preprocess the multimodal data, wherein the multimodal data includes at least one modal data selected from vibration data, current data, temperature data and sound data; The mutual information of the multimodal data is obtained, and the weight information of each modality is determined based on the mutual information; Based on the weight information, feature fusion is performed on the preprocessed multimodal data to obtain target features; The target features are input into a hybrid neural network to obtain fault prediction results, and an early warning operation is performed based on the fault prediction results.

[0005] Optionally, obtaining the mutual information of the multimodal data includes: Acquire the first mutual information between the vibration data and the current data, and acquire the second mutual information between the temperature data and the sound data; The first weight of the vibration data and the second weight of the current data are determined based on the first mutual information and the second mutual information. The third weight of the temperature data and the fourth weight of the sound data are determined based on the first mutual information and the second mutual information.

[0006] Optionally, determining the first weight of the vibration data and the second weight of the current data based on the first mutual information and the second mutual information includes: The first basic weight of the vibration data is determined based on the first mutual information and the second mutual information, and the second basic weight of the current data is determined based on the first basic weight. The operating state of the industrial equipment is obtained, and the first state coefficient corresponding to the operating state is determined. The first weight is determined based on the first state coefficient and the first basic weight, and the second weight is determined based on the first state coefficient and the second basic weight.

[0007] Optionally, determining the third weight of the temperature data and the fourth weight of the sound data based on the first mutual information and the second mutual information includes: The third basic weight of the temperature data is determined based on the first mutual information and the second mutual information, and the fourth basic weight of the sound data is determined based on the third basic weight. The operating status of the industrial equipment is obtained, and the second state coefficient corresponding to the operating status is determined. The third weight is determined based on the second state coefficient and the third basic weight, and the fourth weight is determined based on the second state coefficient and the fourth basic weight.

[0008] Optionally, the hybrid neural network includes temporal branches and static branches, and the step of inputting the target features into the hybrid neural network to obtain the fault prediction result includes: The preprocessed vibration data and current data are input into the time-series branch, and a time-series feature vector is obtained based on the one-dimensional convolutional layer and recurrent neural network layer of the time-series branch. The preprocessed temperature data and sound data are input into the static branch, and a static feature vector is obtained based on the two-dimensional convolutional layer of the static branch; The time-series feature vector and the static feature vector are fused to obtain the target feature, and the fault prediction result is obtained based on the target feature.

[0009] Optionally, the step of performing the early warning operation based on the fault prediction result includes: When the fault prediction result is greater than the first threshold and less than the second threshold, a mild warning operation is performed, which is used to trigger the local LED indicator. When the fault prediction result is greater than the second threshold and less than the third threshold, a moderate early warning operation is performed. The moderate early warning operation is used to send early warning information to the workshop manufacturing execution system. When the fault prediction result is greater than the third threshold, a severe warning operation is executed. The severe warning operation is used to link the programmable logic controller of the industrial equipment to shut down the industrial equipment.

[0010] Secondly, this disclosure provides a fault detection device for industrial equipment, the device comprising: The acquisition module is configured to acquire multimodal data and preprocess the multimodal data, wherein the multimodal data includes at least one modal data selected from vibration data, current data, temperature data, and sound data. The acquisition module is configured to acquire the mutual information of the multimodal data and determine the weight information of each modality based on the mutual information; The fusion module is configured to perform feature fusion on the preprocessed multimodal data according to the weight information to obtain target features; The early warning module is configured to input the target features into a hybrid neural network to obtain a fault prediction result, and to perform an early warning operation based on the fault prediction result.

[0011] Thirdly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0012] Fourthly, this disclosure provides an electronic device, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect.

[0013] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0014] Through the above technical solution, after collecting multimodal data, the multimodal data is preprocessed, and mutual information is calculated based on the preprocessed multimodal data. The mutual information can accurately obtain the weight information of each modality data. Then, the multimodal data is fused based on the weight information, and the fused features are input into a lightweight hybrid neural network to obtain the fault detection result. Based on the fault detection result, the corresponding early warning operation is executed, which can ensure the accuracy and flexibility of the early warning to a certain extent.

[0015] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a fault detection method for industrial equipment according to an exemplary embodiment.

[0017] Figure 2 This is an example diagram illustrating the architecture of a lightweight hybrid neural network in a fault detection method for industrial equipment, according to an exemplary embodiment.

[0018] Figure 3 This is a block diagram of a fault detection device for industrial equipment according to an embodiment of the present disclosure.

[0019] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0020] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0021] In the description of this disclosure, it should be noted that the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0022] In the description of this disclosure, it should also be noted that, unless otherwise expressly specified and limited, the terms "set up," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure based on the specific circumstances.

[0023] The technical solutions in this embodiment will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0024] In the process of fault early warning, related technologies can collect vibration, temperature, and current data, perform simple feature fusion on these modal data, and implement alarm functions using basic thresholds. For example, in the monitoring of critical equipment such as motors and fans, vibration sensors can collect mechanical vibration signals, infrared sensors can monitor component temperature, and current transformers can capture electrical parameters. Anomaly detection can then be achieved through fixed rules or conventional algorithms. For instance, fault rules could be vibration > 4g and temperature > 80℃, and conventional algorithms could include LSTM (Long Short-Term Memory) networks and support vector machines. Optionally, these technologies can also interface with controllers via industrial Ethernet to output basic early warning signals.

[0025] However, the aforementioned fault prediction solutions often employ fixed weights or simple data concatenation during data fusion, failing to dynamically adjust modal priorities based on different operating states such as equipment startup and steady-state. Furthermore, model deployment largely relies on industrial servers, lacking lightweight designs adapted to edge controllers, resulting in high real-time inference latency. In addition, these early warning solutions typically lack environmental robustness, failing to adequately adapt to electromagnetic interference and data loss due to temporary sensor offlineness in industrial settings, leading to false alarms and missed alarms. Moreover, in terms of functional completeness, most early warning solutions only achieve a partial data acquisition-anomaly alarm process, unable to achieve a complete closed loop of multimodal deep fusion, intelligent inference, and hierarchical execution, particularly lacking the ability to identify early, subtle faults based on cross-modal correlation analysis, thus failing to meet enterprises' core requirements for early detection and handling of equipment faults.

[0026] To address the aforementioned issues, this disclosure proposes a fault detection method, apparatus, medium, and electronic device for industrial equipment. This fault detection method for industrial equipment achieves more accurate fault warnings by constructing a four-level architecture of multimodal perception, adaptive fusion, intelligent reasoning, and early warning execution.

[0027] Figure 1 This is an exemplary embodiment illustrating a fault detection method for industrial equipment, which can be applied to a fault detection system, such as... Figure 1 As shown, the method may include: In step S110, multimodal data is collected and preprocessed.

[0028] This disclosure can be applied to a fault detection system, which may include a multimodal perception layer, an adaptive fusion layer, an intelligent inference layer, and an early warning execution layer. The multimodal perception layer can be used to collect multimodal data from industrial equipment. Here, the multimodal data may include at least one modal data selected from vibration data, current data, temperature data, and sound data.

[0029] For example, vibration data can be acquired using a vibration sensor, which can be mounted on the bearing housing of the equipment, i.e., on the bearing housing of industrial equipment. The vibration sensor can acquire the X / Y / Z triaxial vibration acceleration of the industrial equipment. When industrial equipment malfunctions, its vibration characteristics often become abnormal. Therefore, by acquiring triaxial vibration acceleration, it is possible to capture vibration changes in different directions, thus avoiding blind spots in single-dimensional monitoring.

[0030] Optionally, current data can be acquired through a current transformer, which can be used to acquire the operating current of industrial equipment. Load fluctuations and electrical component failures in industrial equipment can cause abnormal fluctuations in the operating current. Therefore, by acquiring current signals, the operating status of the electrical system of industrial equipment can be monitored.

[0031] Optionally, temperature data can be acquired using an infrared temperature sensor, which can be used to collect temperature changes in heat-prone components such as motor stators and bearing outer rings. Electrical faults or increased mechanical friction in industrial equipment can cause abnormal increases in localized temperatures. A non-contact design avoids interference from traditional contact sensors while accurately capturing temperature trends at heat-generating points.

[0032] Optionally, sound data can be acquired using a microphone array, which can be used to detect acoustic signals during the operation of the acquisition equipment. Industrial equipment often produces characteristic abnormal noises due to structural anomalies, such as high-frequency screams or periodic impact sounds. The microphone array can capture wide-band acoustic signals, and abnormal acoustic signature features can be extracted through spectral analysis.

[0033] It should be noted that the fault detection system may also include a data acquisition card, which can be used for synchronous aggregation of sensor data, signal preprocessing and high-speed transmission, etc. That is, when any of the above sensors acquires modal data, it can be sent to the data acquisition card, which can aggregate, transmit and process multimodal data.

[0034] As an optional approach, after acquiring multimodal data, embodiments of this disclosure can preprocess the modal data. For example, the adaptive fusion layer may include a data preprocessing module that can preprocess multimodal data acquired from different sensors separately.

[0035] Specifically, for vibration signals (vibration data), the embodiments of this disclosure can perform wavelet denoising processing to extract time-domain features such as kurtosis and peak factor. For example, in the process of wavelet denoising processing, the db4 wavelet basis can be selected and decomposed into 5 layers.

[0036] For example, in the noise reduction process, this embodiment of the disclosure can perform a 5-level wavelet decomposition on the vibration signal x(t) to obtain the low-frequency approximation coefficient a5, and obtain the high-frequency detail coefficients d1, d2, d3, d4 and d5 of each level. The specific calculation formula is as follows: ; Where h(t) is the low-pass filter of the db4 wavelet, and g(t) is the high-pass filter. This represents the convolution operation.

[0037] Furthermore, embodiments of this disclosure can apply soft thresholding to the high-frequency detail coefficients di(t) to obtain d'i(t), and finally reconstruct the denoised signal using the coefficients after soft thresholding. (t), the specific formula is shown below: ; Among them, W -1 This is the inverse wavelet transform.

[0038] After performing noise reduction processing, embodiments of this disclosure can perform time-domain feature extraction operations. Kurtosis is used to describe the peak characteristics of the signal and is sensitive to impulsive faults, as detailed below: ; Where μ is the mean of the noise-reduced signal and σ is the standard deviation.

[0039] The peak factor can be used to reflect the ratio of the signal's peak value to its RMS value, as detailed below: ; Optionally, for temperature and current data (temperature and current data), embodiments of this disclosure can perform sliding window smoothing to calculate mean and volatility characteristics. Here, sliding window smoothing can be achieved using the following formula: ; in, , Here are the smoothed temperature and current values, where T(t) is the temperature signal and I(t) is the current signal. The window size can be 5 seconds, the sampling frequency is fs, and the window can contain N values. w =5×fs data points.

[0040] Furthermore, embodiments of this disclosure can obtain the average of the smoothed values ​​within the window based on the following formula: ; Where M represents the total number of windows within the analysis period.

[0041] Furthermore, embodiments of this disclosure can calculate the volatility of the signal based on the following formula: ; This volatility is used to describe the intensity of signal fluctuations.

[0042] Optionally, a Short-Time Fourier Transform (STFT) is performed on the sound data (sound signal) in this embodiment of the disclosure to extract Mel-frequency spectral features.

[0043] For example, in this embodiment of the disclosure, a Hanning window can be applied to the sound signal s(t), where the window length is L and the overlap rate is 50%, to calculate the Fourier transform of each window, as shown in the following formula: ; Where n is the window index, f is the frequency, and w(m) is the Hanning window function.

[0044] Based on this, the embodiments of this disclosure can convert the linear frequency spectrum of the STFT into a Mel spectrum to simulate the characteristics of human hearing, as shown in the following formula: ; Among them, H k (f) is the transfer function of the k-th Mel filter bank, which satisfies the following formula: ; Among them, f k This represents the frequency corresponding to the Mel scale.

[0045] By performing targeted preprocessing on multimodal data, the characteristics corresponding to each mode can be obtained. For example, preprocessing vibration signals can yield time-domain characteristics such as kurtosis and peak factor; preprocessing temperature and current data can yield mean and volatility characteristics; and preprocessing sound signals can yield Mel-spectrum characteristics. Based on these preprocessed characteristics, subsequent fault detection can be performed more accurately.

[0046] In step S120, the mutual information of the multimodal data is obtained, and the weight information of each modal data is determined based on the mutual information.

[0047] As an optional approach, after preprocessing the multimodal data, this embodiment of the disclosure can obtain the mutual information (MI) of the preprocessed multimodal data. Mutual information is a statistical index used to quantify the correlation strength between two random variables (such as vibration and current signals). The value of mutual information can range from [0, +∞), with a larger value indicating a higher correlation between the two variables. Therefore, mutual information can effectively measure the redundancy and complementarity of multimodal data. This embodiment of the disclosure calculates cross-modal mutual information (such as vibration-current MI, temperature-sound MI) to dynamically allocate the weights of each mode, thereby solving the problem that traditional fixed weights cannot adapt to different operating states of the equipment.

[0048] For example, attention mechanisms can be used to calculate modal weights to compute mutual information (MI) for cross-modal data such as vibration, current, temperature and sound. The weights can be dynamically adjusted according to the operating status of industrial equipment to output a fused feature vector, which can be 64-dimensional.

[0049] As an example, embodiments of this disclosure can calculate mutual information between vibration (V) and current (I), and temperature (T) and sound (S), with specific calculation formulas as follows: ; Wherein, P is a probability distribution, which can be predicted statistically using histograms.

[0050] Subsequently, the embodiments of this disclosure can perform a dynamic weight adjustment operation, in which the basic weight of the vibration data and the vibration weight of the current data can be obtained by mutual information normalization.

[0051] ; in, For the fundamental weights of vibration modes, This represents the basic weights for the current modes.

[0052] Based on this, embodiments of this disclosure can obtain the weights of vibration modes according to the basic weights of vibration data, and obtain the weights of current modes according to the basic weights of current data, as specifically calculated as follows: ; Here, α can be determined based on the operating state of the industrial equipment. Different operating states of industrial equipment result in different corresponding state coefficients. For example, when the industrial equipment is in a steady-state stage, the state coefficient corresponding to the vibration data is 0.3, meaning that the vibration weight can be increased to 0.3 in the steady-state stage. As another example, when the industrial equipment is in the startup stage, the state coefficient corresponding to the current data is 0.7, meaning that the current weight can be increased to 0.7 in the startup stage.

[0053] Optionally, embodiments of this disclosure may perform a dynamic weight adjustment operation, in which the basic weight of the temperature data and the vibration weight of the sound data may be obtained by mutual information normalization.

[0054] ; in, For the basic weights of temperature modes, These are the basic weights for the sound modalities.

[0055] Based on this, embodiments of this disclosure can obtain the weight of the temperature mode according to the basic weight of the temperature data, and obtain the weight of the sound mode according to the basic weight of the sound data, as specifically calculated as follows: ; Here, β can be determined based on the operating state of the industrial equipment. Different operating states of the industrial equipment will result in different state coefficients. For example, when the industrial equipment is in a steady-state state, the state coefficient corresponding to temperature data is 0.8, meaning that the weight of temperature can be increased to 0.8 in the steady-state state. Similarly, when the industrial equipment is in the startup phase, the state coefficient corresponding to sound data is 0.2, meaning that the weight of sound can be increased to 0.2 in the startup phase.

[0056] In summary, during the acquisition of mutual information from multimodal data, the embodiments of this disclosure can acquire first mutual information between vibration data and current data, and second mutual information between temperature data and sound data. Based on this, a first weight for vibration data and a second weight for current data are determined according to the first and second mutual information, and a third weight for temperature data and a fourth weight for sound data are determined according to the first and second mutual information.

[0057] Specifically, determining the first weight of vibration data and the second weight of current data based on the first mutual information and the second mutual information includes: determining the first basic weight of vibration data based on the first mutual information and the second mutual information, and determining the second basic weight of current data based on the first basic weight; acquiring the working state of industrial equipment and determining the first state coefficient corresponding to the working state; determining the first weight based on the first state coefficient and the first basic weight, and determining the second weight based on the first state coefficient and the second basic weight.

[0058] Optionally, determining the third weight of the temperature data and the fourth weight of the sound data based on the first mutual information and the second mutual information includes: determining the third basic weight of the temperature data based on the first mutual information and the second mutual information, and determining the fourth basic weight of the sound data based on the third basic weight; acquiring the working state of the industrial equipment and determining the second state coefficient corresponding to the working state; determining the third weight based on the second state coefficient and the third basic weight, and determining the fourth weight based on the second state coefficient and the fourth basic weight.

[0059] It should be noted that the embodiments of this disclosure can obtain mutual information between vibration data and current data, as well as mutual information between temperature data and sound data, or mutual information between vibration data and temperature data, as well as mutual information between current data and sound data, or mutual information between vibration data and sound data, as well as mutual information between temperature data and current data. The embodiments of this disclosure do not explicitly limit which two modes' mutual information is calculated.

[0060] Preferably, in this embodiment of the present disclosure, the aforementioned multiple mutual information values ​​can be calculated separately. Based on this, these mutual information values ​​are compared, and the case with the largest mutual information value is used as the basis for subsequent weight information calculation. For example, if the first mutual information value between vibration data and current data, and the second mutual information value between temperature data and sound data are determined to be greater than the mutual information values ​​in other cases, then the weights of vibration, current, temperature, and sound can be calculated based on these two mutual information values.

[0061] Since the weights of the different modes are flexibly determined based on mutual information, the mutual information of the modal data at different times is different, so the corresponding weights will also change dynamically. That is, the weight information of each mode in this embodiment is dynamically changing, and the weight information of the industrial equipment at different stages is very likely to be different.

[0062] In step S130, feature fusion is performed on the preprocessed multimodal data according to the weight information to obtain the target features.

[0063] As an alternative approach, after obtaining the weight information of different modalities, embodiments of this disclosure can perform feature analysis on the preprocessed multimodal data based on this weight information to obtain target features.

[0064] For example, in this embodiment of the disclosure, the features of vibration (16-dimensional), temperature (8-dimensional), current (8-dimensional), and sound (32-dimensional) can be fused according to the weights obtained above, and the specific fusion formula is as follows: ; The total dimension of the different modalities is 16+8+8+32=64 dimensions, and the semicolon above indicates vector concatenation.

[0065] The process of feature fusion can also be called multimodal data fusion. Multimodal data fusion refers to the technology of performing correlation analysis and feature integration on different types (modal) data such as vibration, temperature, current, and acoustics collected during the operation of industrial equipment through specific algorithms. The embodiments of this disclosure break through the information limitations of single-modal data and can explore the coupling relationship between different physical quantities (such as the causal relationship between abnormal vibration and temperature rise) to form a more comprehensive equipment status characterization. That is, the embodiments of this disclosure can be achieved through "mutual information calculation of dynamic weights + gating mechanism feature fusion", which is fundamentally different from the simple weighting or splicing fusion methods of related technologies.

[0066] The embodiments disclosed herein calculate the cross-modal correlation strength of vibration-current and temperature-sound through mutual information (MI), and dynamically allocate modal weights by combining the device operating state coefficients, thereby achieving accurate adaptation to different operating scenarios.

[0067] In step S140, the target features are input into the hybrid neural network to obtain the fault prediction result, and an early warning operation is performed based on the fault prediction result.

[0068] In some implementations, after performing the feature fusion operation, the present disclosure may input the target features obtained by fusion into a hybrid neural network, which may be a lightweight hybrid neural network that may include temporal branches and static branches.

[0069] To better illustrate the structure of hybrid neural networks, embodiments of this disclosure are provided as follows: Figure 2 The example diagram shown is based on... Figure 2 As can be seen, the temporal branch can include 1D convolutional layers (one-dimensional convolutional layers) and GRU layers (Gated Recurrent Units). The 1D convolutional layer can include 32 3×3 kernels, and the GRU can be a recurrent neural network layer, which can be a hidden unit. Here, the temporal branch is used to obtain 32-dimensional temporal features of vibration and current.

[0070] In other words, the embodiments of this disclosure can input preprocessed vibration data and current data into a time-series branch, and the time-series feature vector can be obtained based on the one-dimensional convolutional layer and recurrent neural network layer of the time-series branch.

[0071] Optionally, the static branch may include a 2D convolutional layer, which may include 16 3×3 kernels. Here, the static branch is used to process the spectral features of temperature and sound to obtain a 32-dimensional static feature vector.

[0072] In other words, the embodiments of this disclosure can input preprocessed temperature data and sound data into a static branch, and obtain a static feature vector based on the two-dimensional convolutional layer of the static branch.

[0073] Based on this, embodiments of this disclosure can fuse temporal feature vectors and static feature vectors to obtain target features, and obtain fault prediction results based on the target features. For example, embodiments of this disclosure can fuse features of temporal branches and static branches through a gating fusion mechanism, and output a fault probability (0-1) through a fully connected layer, which is the fault prediction result.

[0074] In this embodiment, the lightweight hybrid neural network refers to a compact deep learning model designed for resource-constrained scenarios in industrial edge devices, combining the advantages of multiple network structures. This embodiment can use "1D convolution + GRU temporal branch" to process time-series data such as vibration and current (capturing dynamic trends), and "2D convolution static branch" to process temperature and sound spectrum data (extracting frequency domain features), compressing parameter scale through knowledge distillation. Compared to a single LSTM or fully connected network, while ensuring inference accuracy, this embodiment can control edge inference latency to within 200ms, adapting to embedded PLC deployments.

[0075] By constructing a lightweight hybrid neural network, the embodiments of this disclosure can extract dynamic trend features using 1D convolutional layers + GRU layers for vibration and current time series data; and extract frequency domain features using 2D convolutional layers for temperature and sound spectrum data; and then establish cross-modal correlations through a gating mechanism to deeply mine early microscopic changes in faults.

[0076] As an optional approach, after obtaining the fault prediction result, this embodiment of the disclosure can perform an early warning operation based on the fault prediction result; that is, if the fault prediction results are different, the corresponding early warning operation may also be different.

[0077] For example, when the fault prediction result is greater than a first threshold and less than a second threshold, this embodiment of the present disclosure can perform a mild warning operation, which is used to trigger a local light-emitting diode (LED) indicator, wherein the first threshold can be 0.3 and the second threshold can be 0.5; when the fault prediction result is greater than the second threshold and less than a third threshold, this embodiment of the present disclosure can perform a moderate warning operation, which is used to send a warning message to the Manufacturing Execution System (MES) system, wherein the third threshold can be 0.8; when the fault prediction result is greater than the third threshold, this embodiment of the present disclosure can perform a severe warning operation, which is used to link the programmable logic controller (PLC) of the industrial equipment to shut down the industrial equipment.

[0078] As can be seen, the embodiments of this disclosure can set three levels of warning thresholds, the range of which can be mild (0.3-0.5), moderate (0.5-0.8), and severe (>0.8). Specifically, a mild warning can be used to trigger a local LED indicator; a moderate warning can be pushed to the workshop MES system; and a severe warning can be used to shut down the PLC of the linked equipment.

[0079] In summary, the embodiments of this disclosure can first perform multimodal data acquisition during the fault detection of industrial equipment. Specifically, the sensing layer can synchronously acquire modal data such as vibration (1kHz), temperature (1Hz), current (50Hz), and sound (10Hz) at preset frequencies to generate a timestamped raw dataset. Based on this, feature extraction and fusion operations are performed. This operation can use a preprocessing module to reduce noise and normalize the raw data, such as using Z-score standardization: feature value = (x-μ) / σ, where μ and σ are the mean and standard deviation of the equipment under normal conditions. Then, a dynamic association module can calculate cross-modal mutual information to generate a weighted fusion feature vector.

[0080] Finally, the fused feature vector obtained above can be input into the inference layer to load the lightweight model, i.e., input the fused features and output the fault probability. Optionally, if the probability exceeds the corresponding threshold for N consecutive sampling periods, a corresponding level of warning can be triggered. For example, if the probability exceeds the first threshold but is less than the second threshold for 3 consecutive periods (each period is 10s), a mild warning can be triggered.

[0081] Here, the warning information can include the fault type (such as "bearing inner ring wear"), confidence level (such as 95%), and recommended actions. For example, the fault type can be bearing inner ring wear with a confidence level of 95%, and the recommended action is to stop the machine and check the lubricating grease.

[0082] This embodiment of the disclosure achieves this advantage through a three-level early warning threshold + linkage execution mechanism. It can set three-level thresholds of mild, moderate and severe according to the fault probability (fault prediction result) and trigger different execution actions accordingly. At the same time, it can output fault type, confidence level and handling suggestions.

[0083] The embodiments disclosed herein utilize an innovative technical framework of dynamic multimodal fusion, lightweight intelligent reasoning, and hierarchical early warning execution to address the shortcomings of existing industrial equipment fault early warning solutions in terms of adaptability, sensitivity, and practicality.

[0084] This disclosure employs a lightweight hybrid neural network design, consisting of 1D convolutions, a GRU temporal branch, and a 2D convolutional static branch. By compressing parameters through knowledge distillation and adapting to edge device deployment, it can control inference latency within a preset time period, such as less than 200ms. Furthermore, by introducing a noise-robust loss function and a data missing compensation mechanism, this disclosure maintains over 85% accuracy even under electromagnetic interference and 30% data missing scenarios, thereby improving system environmental adaptability. Moreover, by dynamically adjusting the feature weights of the temporal and static branches using a gating fusion mechanism to establish cross-modal associations, it can improve the identification accuracy of complex faults.

[0085] Figure 3 This is a fault detection device for industrial equipment according to an exemplary embodiment, such as... Figure 3 As shown, the fault detection device 200 for industrial equipment may include: a data acquisition module 210, an acquisition module 220, a fusion module 230, and an early warning module 240.

[0086] The acquisition module 210 is configured to acquire multimodal data and preprocess the multimodal data, wherein the multimodal data includes at least one modal data selected from vibration data, current data, temperature data, and sound data. The acquisition module 220 is configured to acquire the mutual information of the multimodal data and determine the weight information of each modality based on the mutual information; The fusion module 230 is configured to perform feature fusion on the preprocessed multimodal data according to the weight information to obtain target features; The early warning module 240 is configured to input the target features into a hybrid neural network to obtain a fault prediction result, and to perform an early warning operation based on the fault prediction result.

[0087] In some implementations, the acquisition module 220 may include: The mutual information acquisition submodule is configured to acquire the first mutual information between the vibration data and the current data, and to acquire the second mutual information between the temperature data and the sound data; The weight determination submodule is configured to determine a first weight of the vibration data and a second weight of the current data based on the first mutual information and the second mutual information; and to determine a third weight of the temperature data and a fourth weight of the sound data based on the first mutual information and the second mutual information.

[0088] In some implementations, the weight determination submodule may also be configured to determine a first basic weight of the vibration data based on the first mutual information and the second mutual information, and determine a second basic weight of the current data based on the first basic weight; acquire the operating state of the industrial equipment and determine a first state coefficient corresponding to the operating state; determine the first weight based on the first state coefficient and the first basic weight, and determine the second weight based on the first state coefficient and the second basic weight.

[0089] In some implementations, the weight determination submodule may also be configured to determine a third basic weight of the temperature data based on the first mutual information and the second mutual information, and to determine a fourth basic weight of the sound data based on the third basic weight; to obtain the working state of the industrial equipment and to determine a second state coefficient corresponding to the working state; to determine the third weight based on the second state coefficient and the third basic weight, and to determine the fourth weight based on the second state coefficient and the fourth basic weight.

[0090] In some embodiments, the hybrid neural network includes a temporal branch and a static branch. The early warning module 240 can also be configured to input the preprocessed vibration data and the current data into the temporal branch, and obtain a temporal feature vector based on the one-dimensional convolutional layer and recurrent neural network layer of the temporal branch; input the preprocessed temperature data and the sound data into the static branch, and obtain a static feature vector based on the two-dimensional convolutional layer of the static branch; fuse the temporal feature vector and the static feature vector to obtain a target feature, and obtain the fault prediction result based on the target feature.

[0091] In some embodiments, the early warning module 240 may also be configured to perform a mild early warning operation when the fault prediction result is greater than a first threshold and less than a second threshold, the mild early warning operation being used to trigger a local LED indicator; perform a moderate early warning operation when the fault prediction result is greater than the second threshold and less than a third threshold, the moderate early warning operation being used to send early warning information to the workshop manufacturing execution system; and perform a severe early warning operation when the fault prediction result is greater than the third threshold, the severe early warning operation being used to link the programmable logic controller of the industrial equipment to shut down the industrial equipment.

[0092] In this embodiment, after collecting multimodal data, the multimodal data is preprocessed, and mutual information is calculated based on the preprocessed multimodal data. The mutual information can accurately obtain the weight information of each modality data. Then, the multimodal data is fused based on the weight information, and the fused features are input into a lightweight hybrid neural network to obtain fault detection results. Based on the fault detection results, corresponding early warning operations are performed, which can ensure the accuracy and flexibility of early warning to a certain extent.

[0093] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0094] Figure 4 This is a block diagram illustrating an electronic device 700 according to an exemplary embodiment. Figure 4 As shown, the electronic device 700 may include a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0095] The processor 701 controls the overall operation of the electronic device 700 to complete all or part of the steps in the aforementioned industrial equipment fault detection method. The memory 702 stores various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 703 may include a screen and audio components. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 702 or transmitted via communication component 705. The audio component also includes at least one speaker for outputting audio signals. I / O interface 704 provides an interface between processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0096] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described fault detection method for industrial equipment.

[0097] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the above-described industrial equipment fault detection method. For example, the computer-readable storage medium may be the memory 702 including program instructions, which may be executed by the processor 701 of the electronic device 700 to complete the above-described industrial equipment fault detection method.

[0098] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the above-described fault detection method for industrial equipment.

[0099] In another exemplary embodiment, a computer program product is also provided, comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described fault detection method for industrial equipment when executed by the programmable device.

[0100] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0101] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0102] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A failure detection method of an industrial device, characterized by, The method comprises: Collecting multi-modal data, and pre-processing the multi-modal data, wherein the multi-modal data comprises at least one modal data among vibration data, current data, temperature data and sound data; Obtaining mutual information of the multi-modal data, and determining weight information of each modal data based on the mutual information; According to the weight information, the multi-modal data obtained by pre-processing is fused to obtain target features; Inputting the target features into a hybrid neural network to obtain a fault prediction result, and performing a warning operation according to the fault prediction result.

2. The failure detection method of an industrial device according to claim 1, characterized by, The mutual information of the multi-modal data comprises: Obtaining first mutual information of the vibration data and the current data, and obtaining second mutual information of the temperature data and the sound data; According to the first mutual information and the second mutual information, determining a first weight of the vibration data and a second weight of the current data; According to the first mutual information and the second mutual information, determining a third weight of the temperature data and a fourth weight of the sound data.

3. The failure detection method of an industrial device according to claim 2, characterized by, According to the first mutual information and the second mutual information, the first weight of the vibration data and the second weight of the current data are determined, comprising: According to the first mutual information and the second mutual information, a first basic weight of the vibration data is determined, and a second basic weight of the current data is determined according to the first basic weight; Obtaining the working state of the industrial equipment, and determining the first state coefficient corresponding to the working state; Based on the first state coefficient and the first basic weight, the first weight is determined, and based on the first state coefficient and the second basic weight, the second weight is determined.

4. The failure detection method of an industrial device according to claim 2, characterized by, According to the first mutual information and the second mutual information, the third weight of the temperature data and the fourth weight of the sound data are determined, comprising: According to the first mutual information and the second mutual information, a third basic weight of the temperature data is determined, and a fourth basic weight of the sound data is determined according to the third basic weight; Obtaining the working state of the industrial equipment, and determining the second state coefficient corresponding to the working state; Based on the second state coefficient and the third basic weight, the third weight is determined, and based on the second state coefficient and the fourth basic weight, the fourth weight is determined.

5. The failure detection method of an industrial device according to any one of claims 1 to 4, characterized by, The hybrid neural network comprises a time sequence branch and a static branch, and the target features are input into the hybrid neural network to obtain a fault prediction result, comprising: The pre-processed vibration data and current data are input into the time sequence branch, and based on the one-dimensional convolution layer and the recurrent neural network layer of the time sequence branch, a time sequence feature vector is obtained; The pre-processed temperature data and sound data are input into the static branch, and based on the two-dimensional convolution layer of the static branch, a static feature vector is obtained; The time sequence feature vector and the static feature vector are fused to obtain target features, and the fault prediction result is obtained based on the target features.

6. The failure detection method of an industrial device according to any one of claims 1 to 4, characterized by, According to the fault prediction result, the warning operation comprises: When the failure prediction result is greater than a first threshold value and less than a second threshold value, a mild warning operation is performed, the mild warning operation being configured to trigger a local light-emitting diode indicator; When the failure prediction result is greater than the second threshold value and less than a third threshold value, a moderate warning operation is performed, the moderate warning operation being configured to send warning information to a plant manufacturing execution system; When the failure prediction result is greater than the third threshold value, a severe warning operation is performed, the severe warning operation being configured to link a programmable logic controller of the industrial equipment to shut down the industrial equipment.

7. A failure detection device of an industrial device, characterized by, The device comprises: a collection module configured to collect multi-modal data and pre-process the multi-modal data, the multi-modal data comprising at least one modal data of vibration data, current data, temperature data and sound data; an acquisition module configured to acquire mutual information of the multi-modal data and determine weight information of each modal data based on the mutual information; a fusion module configured to perform feature fusion on the pre-processed multi-modal data according to the weight information to obtain target features; a warning module configured to input the target features into a hybrid neural network to obtain a failure prediction result and perform a warning operation according to the failure prediction result.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the method of any one of claims 1-6.

9. An electronic device, comprising: comprise: a memory having a computer program stored thereon; a processor configured to execute the computer program in the memory to implement the steps of the method of any one of claims 1-6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the method of any one of claims 1-6.

Images