Equipment fault mode identification and diagnosis method based on deep learning

Abstract

The invention relates to an equipment fault mode identification and diagnosis method based on deep learning, and aims to improve the accuracy and generalization ability of equipment fault diagnosis. A vibration signal, a temperature signal, an acoustic signal, a current signal and image data of equipment are synchronously acquired through a multi-modal data acquisition system, and weighted fusion is performed on different modal data by adopting a self-adaptive multi-head attention mechanism. And then, performing time sequence modeling by using a bidirectional long-short term memory network (BiLSTM), and finally outputting a fault type and a fault saliency map to help operation and maintenance personnel to position a fault area. And through a transfer learning technology, the adaptability and diagnosis precision of the model under different equipment and working conditions are further improved. The method can be widely applied to fault diagnosis and intelligent operation and maintenance of various devices, and the operation reliability and the maintenance efficiency of the devices are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment fault diagnosis, and in particular to an equipment fault pattern recognition and diagnosis method based on deep learning. Background Art

[0002] At present, with the continuous improvement of automation, the operation status monitoring and fault diagnosis of various equipment have become the key to ensure equipment reliability and production efficiency. Equipment is usually composed of multiple subsystems and operates in a complex production environment. It is easily affected by various factors, such as equipment aging, improper operation, and changes in the external environment. These factors may cause different types of equipment failures and affect the normal production. Therefore, timely and accurate identification of equipment failures and effective diagnosis are of great significance to ensure production safety, reduce downtime, and reduce maintenance costs.

[0003] At present, equipment fault diagnosis technology mainly relies on sensor data collection, signal processing and machine learning methods. Traditional fault diagnosis methods usually rely on a single type of data, such as vibration signals, temperature signals, etc., and use fixed rules or algorithms for analysis. However, these methods have certain limitations when processing data under complex working conditions. First, a single data source may not be able to fully reflect the status of the equipment, resulting in limited ability to identify fault modes. Second, traditional methods lack a flexible fusion mechanism when processing multimodal data and cannot effectively deal with the asymmetry of information between different modalities. Finally, the generalization ability of machine learning models is also challenged, especially when the equipment types are different or the working conditions vary greatly, the diagnostic accuracy of the model may decrease.

[0004] In view of the above problems, the present invention aims to solve the problems of insufficient multimodal data fusion and poor model generalization ability in the prior art, thereby improving the accuracy and reliability of equipment fault diagnosis. Summary of the invention

[0005] In order to solve the problems of the prior art, an embodiment of the present invention provides a method for identifying and diagnosing equipment failure patterns based on deep learning. The technical solution is as follows:

[0006] On the one hand, a method for equipment failure mode recognition and diagnosis based on deep learning is provided, comprising the following steps:

[0007] The vibration signal, temperature signal, acoustic signal, current signal and camera image data of the equipment are collected simultaneously by a multimodal sensor system, wherein the sensor includes a vibration sensor, a temperature sensor, an acoustic sensor, a current sensor and a camera, and the collected data of the sensor is synchronously transmitted to a central processing system through a data collection system;

[0008] Preprocess the data of each sensor, including noise removal, data standardization, alignment and segmentation;

[0009] The signals of various sensors are subjected to feature extraction. The vibration signal and current signal are processed by one-dimensional convolutional neural network. The acoustic signal is converted into a time-frequency image by short-time Fourier transform and then features are extracted by convolutional neural network. The image data is subjected to feature extraction by convolutional neural network.

[0010] The features of each modality are integrated, and the multi-head attention mechanism is used to perform adaptive weighted integration of the features of different modalities;

[0011] The fused features are input into the Bidirectional Long Short-Term Memory Network (BiLSTM) for time series modeling;

[0012] Finally, the fault type is output through the fully connected layer, and a fault saliency map is generated to help locate the specific area where the fault occurs.

[0013] Furthermore, the multi-head attention mechanism adopts a query matrix, a key matrix and a value matrix, adaptively calculates weights and performs weighted fusion on features of different modalities to improve the accuracy of fault diagnosis.

[0014] Furthermore, the data preprocessing step includes removing noise from the sensor signal, denoising by wavelet transform, processing the data by a standardization method, and segmenting the data by a sliding window.

[0015] Furthermore, the vibration signal and the current signal are used to extract local features through a one-dimensional convolutional neural network, and the network uses multiple convolutional layers and pooling layers for feature learning.

[0016] Furthermore, the acoustic signal is converted into a time-frequency image through short-time Fourier transform, and the frequency pattern and time-varying characteristics of the acoustic signal are extracted through a convolutional neural network.

[0017] Furthermore, after the image data passes through multiple convolutional layers and pooling layers of the convolutional neural network, the appearance features of the device are extracted to identify external damage or deformation of the device.

[0018] Furthermore, when performing time series modeling, the BiLSTM network uses forward and reverse LSTM layers to process data respectively to extract contextual information during the operation of the device.

[0019] Furthermore, the fault saliency map is calculated through the output of the attention mechanism, which helps operation and maintenance personnel identify key areas where equipment failures occur and provide decision support for subsequent maintenance.

[0020] Furthermore, the model is trained using a transfer learning approach, which first pre-trains on a general device dataset and then fine-tunes on target device data, thereby enhancing the generalization capability of the model.

[0021] Furthermore, during the training process of the deep learning model, data enhancement technology is used to expand the original data to improve the robustness of the model under complex working conditions. Data enhancement includes adding noise to the time series signal and rotating and scaling the image data.

[0022] The technical solution provided by the embodiment of the present invention has the following beneficial effects:

[0023] The present invention significantly improves the accuracy and robustness of equipment fault diagnosis by introducing multimodal data fusion technology based on deep learning. Through the adaptive multi-head attention mechanism, the weights of different modal data can be dynamically adjusted, thereby overcoming the limitations of fixed weighted fusion in traditional methods and achieving the adaptability of data fusion. This innovative design not only enhances the ability to identify fault modes, but also enables the diagnostic system to maintain a high degree of accuracy under a variety of working conditions. In addition, the use of transfer learning methods to share knowledge between different devices improves the generalization ability of the model, making the technology widely applicable to a variety of devices, and effectively reducing the demand for training data, reducing the time and cost of equipment fault diagnosis. Overall, the present invention optimizes the equipment operation and maintenance efficiency while improving the accuracy of fault diagnosis, and provides reliable technical support for the intelligent operation and maintenance of equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 This is an architecture diagram of a deep learning-based equipment fault diagnosis system in an embodiment of the present invention.

[0026] Figure 2 4 is a flowchart of multimodal data processing in an embodiment of the present invention.

[0027] Figure 3 This is a fault significance map in an embodiment of the present invention, showing the location information of the fault area.

[0028] Figure 4 This is a workflow diagram of the multi-head attention mechanism in an embodiment of the present invention.

[0029] Figure 54 is a structural diagram of a bidirectional long short-term memory network (BiLSTM) in an embodiment of the present invention.

[0030] Figure 6 This is a diagram of the fault significance map output and location process in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0032] The present invention provides a method for equipment fault pattern recognition and diagnosis based on deep learning, which mainly focuses on the fusion of multimodal data, the design of deep learning models, and the application of transfer learning in different devices, solving the problems of poor data fusion capability and insufficient model generalization capability in the prior art. By using multimodal sensors, combined with adaptive deep learning models and transfer learning technology, the present invention can accurately identify and diagnose equipment faults in complex environments. The specific implementation methods of the technical solution will be introduced in detail below, including data acquisition and preprocessing, model design and optimization, and final fault diagnosis and result output.

[0033] Data collection and preprocessing

[0034] In order to achieve efficient equipment fault diagnosis, the present invention first collects various operating data of the equipment through a multimodal sensor system, including vibration signals, acoustic signals, temperature signals, current signals and camera image data. Figure 1 As shown in the figure, the system architecture diagram shows the complete process from multimodal sensors to data processing and diagnosis. Specifically, the vibration sensor is mainly used to collect the vibration characteristics of the equipment, which is suitable for monitoring mechanical faults such as bearing damage or gear wear; the acoustic sensor is used to collect the sound wave signals emitted by the equipment to help identify abnormal sounds during equipment operation; the temperature sensor is used to monitor the temperature changes of the equipment and capture possible overheating faults; the current sensor is used to detect the current fluctuations of the motor or electrical equipment to identify abnormalities in the electrical system; the camera is used to capture the appearance image of the equipment to assist in analyzing the physical damage on the outside of the equipment such as cracks or corrosion.

[0035] All sensor data is synchronized through a unified data acquisition system and transmitted to the central processing system in real time through edge computing devices. In the data preprocessing stage, noise removal is first performed, especially for vibration signals and acoustic signals, wavelet transform is used for denoising to eliminate high-frequency noise and retain the effective signal of the equipment. Then, all collected data is standardized to eliminate the impact of differences in the magnitude of different sensor data. Standardization converts all data into a standard normal distribution with zero mean and unit variance by subtracting the mean and dividing by the standard deviation, thereby avoiding the adverse effects of data deviation on model training.

[0036] Since the data sampling frequencies of different modes may be different, the linear interpolation method is used to align the data of each mode to the same time step in the data alignment stage to ensure that the data of all modes can be matched at the same time point. After that, the original data is segmented by the sliding window method, with each segment length of 2 seconds and a step length of 0.5 seconds. Each segment of data is labeled as the corresponding fault category or normal state, forming a multimodal data set for subsequent deep learning model training.

[0037] Deep learning model design

[0038] The core technical solution of the present invention is based on a deep learning model, which adopts the strategies of adaptive multimodal data fusion, time series modeling and fault mode classification. The design of the model mainly includes four modules: feature extraction, fusion, time series modeling and classification.

[0039] In the feature extraction stage, different neural network architectures are used to process different types of data. Time series data such as vibration signals and current signals are extracted using a one-dimensional convolutional neural network (1D-CNN). Figure 2 As shown in the figure, the feature extraction process of 1D-CNN includes extracting periodic fluctuations, sharp impact signals and other features from the signal after the input data is processed by the convolution layer, which is then used for fault identification. 1D-CNN can effectively capture the local features and timing changes of the signal through convolution operations. Specifically, the calculation formula of 1D convolution is:

[0040]

[0041] Among them, f ij is the convolution output, w mn is the weight of the convolution kernel, x (i+m)(j+n) is the value of the input signal, b is the bias term, and σ is the activation function. Through this method, 1D-CNN can effectively extract features such as periodic fluctuations and sharp impact signals in vibration signals, which can then be used for fault identification.

[0042] For device appearance image data, convolutional neural network (CNN) is used for feature extraction. Figure 3 As shown in the figure, the working principle diagram of CNN shows how to gradually extract deep features of the image, such as cracks, deformations, etc., through convolutional layers and pooling layers. For acoustic signals, the signal is first converted into a time-frequency image through short-time Fourier transform (STFT), and then the frequency pattern and time-varying features in the acoustic signal are extracted through CNN.

[0043] In the multimodal feature fusion stage, the present invention uses a multi-head attention mechanism to dynamically adjust the weights of different modal data to achieve adaptive fusion. Figure 4 As shown in the figure, the workflow of the multi-head attention mechanism is demonstrated. After the features of different modal data are specifically encoded, they are weighted averaged through the multi-head attention mechanism to automatically determine the importance of each modality in fault diagnosis. The specific fusion formula is as follows:

[0044]

[0045] Among them, Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key, and Softmax is used to normalize the weights. Through this mechanism, the model can automatically assign different weights to each modal data during the diagnosis process, thereby effectively improving the accuracy of fault diagnosis.

[0046] In the time series modeling stage, the fused multimodal features are input into the bidirectional long short-term memory network (BiLSTM) for modeling. Figure 5 As shown in the figure, the structure of BiLSTM shows how to process time series data through bidirectional information. BiLSTM can capture the contextual information of sequence data and model the long-term and short-term state changes of the device, which is particularly suitable for processing dynamic changes in time series data. The state update formula of BiLSTM is:

[0047] h t =σ(W h ·h t-1 +W x ·x t +b)

[0048] Among them, h t is the hidden state at the current moment, h t-1 is the hidden state of the previous moment, x t is the current input, W h and W x is the weight matrix, b is the bias term, and σ is the activation function.

[0049] In the classification stage, the output of BiLSTM is mapped into a high-dimensional feature vector through the fully connected layer, and finally the fault type is classified through the Softmax classifier. The Softmax function outputs the probability value of each fault type, and selects the category with the largest probability value as the diagnosis result.

[0050] Model training and optimization

[0051] During the model training process, data augmentation technology is used to expand the training set and improve the robustness of the model. Data augmentation includes adding noise to time series data, rotating and scaling image data, and other operations to simulate different device states and environmental changes. The model is pre-trained using a common device data set in the early stage to learn the common features of device operation. Then, using the strategy of transfer learning, the model is fine-tuned on the data of the target device to enhance its adaptability on specific devices. Transfer learning can significantly improve the generalization ability of the model and avoid overfitting problems caused by insufficient data.

[0052] During the optimization process, the Adam optimization algorithm is used for gradient descent to minimize the loss function of the model. The specific loss function is the cross entropy loss function:

[0053]

[0054] Among them, y i is the true label, is the predicted probability output by the model, and N is the number of samples. Through the back propagation algorithm, the weights and biases of the model are optimized, and finally the loss function is minimized, thereby improving the performance of the model.

[0055] Fault diagnosis and result output

[0056] In the fault diagnosis stage, the model infers the multimodal data collected in real time and outputs the fault type and fault significance of the equipment. The fault type includes whether the equipment has a fault and the specific type of fault (such as bearing wear, gear damage, overload, etc.). At the same time, through the attention mechanism, the model also outputs the fault significance map of the equipment, showing the key areas where the fault occurs. Figure 6 As shown in the figure, the fault significance map shows the abnormal characteristics of key parts of the equipment. This graph can help operation and maintenance personnel quickly identify the specific location of equipment failures, thereby guiding subsequent maintenance work.

[0057] In summary, the present invention proposes a comprehensive solution for equipment fault pattern recognition and diagnosis by combining multimodal data acquisition, adaptive deep learning model design and transfer learning technology. The present invention has made innovations in multimodal data fusion, adaptive weight allocation and time series modeling, effectively improving the accuracy of fault identification and the generalization ability of the model under complex working conditions. At the same time, the fault saliency map generated based on the attention mechanism intuitively displays the key fault areas, providing efficient decision support for operation and maintenance personnel.

[0058] Through a complete data preprocessing process, an optimized deep learning model, and transfer learning applications on target devices, the present invention achieves full-process technology integration from data collection to fault diagnosis. Experiments and practice have shown that the present invention is not only efficient and accurate, but also highly adaptable and can be widely used in various types of equipment scenarios, providing reliable technical support for intelligent equipment operation and maintenance.

[0059] Application Examples

[0060] In order to further illustrate the specific application effect of the present invention, taking a fan device as an example, the specific application process of the multi-modal fault pattern recognition and diagnosis method based on the present invention is introduced.

[0061] 1. Equipment background and operating environment

[0062] The fan equipment model is FJ-2000, which is used in the air circulation system of the chemical plant. The operating power is 200kW and the speed is 3000rpm. Common faults that may occur during operation include bearing wear, blade deformation, motor overload, and mechanical looseness. Since the equipment is in a high temperature and high humidity environment for a long time, the failure will have a serious impact on production.

[0063] 2. Data Collection and Preprocessing

[0064] According to the operating status of the wind turbine equipment, a multi-modal sensor system is installed to collect data:

[0065] Vibration signal: The vibration sensor is installed at the connection between the fan's bearing seat and the motor, with a sampling frequency of 25kHz, to capture the vibration characteristics of mechanical failures.

[0066] Temperature signal: The temperature sensor is installed near the bearing with a sampling frequency of 1Hz and is used to monitor the bearing temperature rise.

[0067] Acoustic signal: The acoustic sensor is arranged outside the fan housing with a sampling frequency of 48kHz to capture abnormal noise.

[0068] Current signal: The current sensor is installed in the motor power supply line with a sampling frequency of 10kHz to monitor motor load fluctuations.

[0069] Image data: The camera is installed at the wind turbine inlet to capture images of the blades in real time with a resolution of 1920×1080 and 10 frames per second.

[0070] The collected multimodal data is preliminarily processed by the edge computing device and synchronously transmitted to the central processing system. The data preprocessing steps include:

[0071] 1. Noise removal: Wavelet transform is used to denoise vibration signals and acoustic signals.

[0072] 2. Data alignment: All modal data are aligned to a time step of 1 ms by linear interpolation method.

[0073] 3. Segmentation: Each segment of data lasts for 2 seconds, with a step length of 0.5 seconds, forming segmented data for model input.

[0074] 3. Feature extraction and fusion

[0075] Vibration signal: The vibration features of bearings and mechanical parts are extracted through 1D-CNN, with a convolution kernel size of 5 and a pooling layer window of 2 to extract local periodic features.

[0076] Acoustic signal: After being converted into a time-frequency image by STFT, ResNet18 is used to extract frequency features.

[0077] Current signal: 1D-CNN is used to capture the current fluctuation pattern and analyze the load changes of the motor operation.

[0078] Image data: The appearance features of wind turbine blades are extracted through the VGG16 network to identify possible cracks or wear.

[0079] Temperature signal: Extract the mean value and gradient change rate of temperature change as features.

[0080] The extracted features are fused through a multi-head attention mechanism, and each modal data is assigned a specific weight to generate a fused feature vector.

[0081] 4. Fault diagnosis and saliency map generation

[0082] The fused features are input into BiLSTM for time series modeling, and finally the fault type is output through the Softmax classifier. In an actual test, the system diagnosis results are as follows:

[0083] Fault type: Bearing wear

[0084] Fault significance diagram: It shows that the high-frequency characteristics of the vibration signal make a significant contribution, the current signal fluctuates abnormally, and no obvious blade damage is found in the image.

[0085] The significance map intuitively shows that the high-frequency vibration signal near the bearing is the main cause of the fault, helping operation and maintenance personnel to locate the problem.

[0086] 5. Model optimization and migration

[0087] In order to adapt to another type of fan equipment in the chemical plant, transfer learning is used for model optimization:

[0088] 1. The weights of the 1D-CNN and ResNet layers are fixed, and only the BiLSTM layer is adjusted.

[0089] 2. Use data enhancement to expand the sample size, such as adding white noise to the vibration signal, performing random spectral shearing on the acoustic signal, etc.

[0090] 3. Fine-tune the model weights using data from the target device, ultimately increasing the diagnostic accuracy from 85% to 95%.

[0091] 6. Fault repair and effect verification

[0092] According to the diagnosis results, the operation and maintenance team disassembled the bearing and found that the ball surface in the bearing was severely worn and the lubricant was deteriorated. After replacing the bearing and adding new lubricant, the fan resumed normal operation. No fault characteristics were found in the re-test of the repaired equipment.

[0093] 7. Application results

[0094] Through the method of the present invention, the factory achieves the following goals:

[0095] 1. The fault diagnosis accuracy rate is increased to 95%, and the diagnosis time is shortened to 5 seconds.

[0096] 2. The fault significance map can directly locate the problem area, reducing maintenance time by 30%.

[0097] 3. The system has strong scalability and has been applied to fault monitoring of other similar equipment.

[0098] The invention solves the problems of poor data fusion capability and insufficient model generalization performance in the prior art, and provides an innovative and practical technical method for the field of equipment intelligent diagnosis.

[0099] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for equipment failure pattern recognition and diagnosis based on deep learning, characterized in that: The following steps are involved: The vibration signal, temperature signal, acoustic signal, current signal and camera image data of the equipment are collected simultaneously by a multimodal sensor system, wherein the sensor includes a vibration sensor, a temperature sensor, an acoustic sensor, a current sensor and a camera, and the collected data of the sensor is synchronously transmitted to a central processing system through a data collection system; Preprocess the data of each sensor, including noise removal, data standardization, alignment and segmentation; The signals of various sensors are subjected to feature extraction. The vibration signal and current signal are processed by one-dimensional convolutional neural network. The acoustic signal is converted into a time-frequency image by short-time Fourier transform and then features are extracted by convolutional neural network. The image data is subjected to feature extraction by convolutional neural network. The features of each modality are integrated, and the multi-head attention mechanism is used to perform adaptive weighted integration of the features of different modalities; The fused features are input into the Bidirectional Long Short-Term Memory Network (BiLSTM) for time series modeling; Finally, the fault type is output through the fully connected layer, and a fault saliency map is generated to help locate the specific area where the fault occurs.

2. The method according to claim 1, characterized in that The multi-head attention mechanism adopts a query matrix, a key matrix and a value matrix, adaptively calculates weights and performs weighted fusion on features of different modalities to improve the accuracy of fault diagnosis.

3. The method according to claim 1, characterized in that The data preprocessing step includes removing noise from the sensor signal, denoising with wavelet transform, processing the data with a standardization method, and segmenting the data with a sliding window.

4. The method according to claim 1, characterized in that: The vibration signal and the current signal are subjected to local feature extraction by a one-dimensional convolutional neural network, and the network adopts a plurality of convolutional layers and pooling layers for feature learning.

5. The method according to claim 1, characterized in that The acoustic signal is converted into a time-frequency image through short-time Fourier transform, and the frequency pattern and time-varying characteristics of the acoustic signal are extracted through a convolutional neural network.

6. The method according to claim 1, characterized in that After the image data passes through multiple convolutional layers and pooling layers of the convolutional neural network, the appearance features of the device are extracted to identify external damage or deformation of the device.

7. The method according to claim 1, characterized in that When performing time series modeling, the BiLSTM network uses forward and reverse LSTM layers to process data to extract contextual information during device operation.

8. The method according to claim 1, characterized in that The fault saliency map is calculated through the output of the attention mechanism, which helps operation and maintenance personnel identify the key areas where equipment failures occur and provide decision support for subsequent maintenance.

9. The method according to claim 1, characterized in that: The model is trained using a transfer learning approach, which first pre-trains on a general device dataset and then fine-tunes on target device data to enhance the generalization capability of the model.

10. The method according to claim 1, characterized in that During the training process of the deep learning model, data enhancement technology is used to expand the original data to improve the robustness of the model under complex working conditions. Data enhancement includes adding noise to the time series signal and rotating and scaling the image data.

Images