A Smart Fault Diagnosis Method for Coal Mine Equipment Based on Multi-Source Heterogeneous Data

Abstract

This invention discloses an intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data, belonging to the field of coal mine equipment fault diagnosis technology. The method first collects and preprocesses multi-source heterogeneous data from coal mine equipment. It then constructs fault-information correlations through multimodal feature extraction and fusion. Subsequently, it builds a deep diagnostic model with subdivided fault identification branches based on the fused features. An adaptive hybrid supervised loss function is used for training to obtain optimal parameters. The trained lightweight model is then used to accurately identify major and subdivided fault categories. Combined with physical mechanism analysis, the root causes of faults are uncovered. Finally, personalized maintenance guidance is generated based on the diagnostic results, and closed-loop optimization continuously improves system performance. This invention can fully exploit the complementary information of multi-source data, improve the diagnostic accuracy of complex coupled faults and subdivided faults, provide strong support for efficient maintenance of coal mine equipment, and ensure the safety and stability of coal mine production.

Description

Technical Field

[0001] This invention belongs to the field of coal mine equipment fault diagnosis technology, specifically relating to an intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data. Background Technology

[0002] In coal mine production, there are numerous types of equipment and complex operating environments, making it difficult for traditional fault diagnosis methods to comprehensively and accurately capture changes in equipment operating status. With the rapid development of technologies such as the Internet of Things and big data, multi-source heterogeneous data has become an important foundation for intelligent fault diagnosis of coal mine equipment. This data comes from multiple sources, including different sensors, control systems, log files, and video monitoring systems, and is characterized by diversity, heterogeneity, and real-time requirements.

[0003] In existing technologies, some intelligent diagnostic methods have begun to utilize multi-source data, but several shortcomings remain: First, the fusion of multi-source heterogeneous data is insufficient, failing to effectively uncover complementary relationships between different types of data and making it difficult to establish accurate information-fault mapping relationships. Second, fault diagnosis often remains at the level of broad category identification, unable to accurately distinguish sub-faults within a single fault category. For example, for broad categories of mechanical faults, it is impossible to accurately determine whether the cause is bearing wear, gear meshing problems, or loose connections, affecting maintenance efficiency. Third, parameter optimization during model training is insufficiently targeted, failing to fully consider the characteristic differences of sub-faults, resulting in weak model generalization ability and difficulty in meeting the needs of rapid maintenance of coal mine equipment. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data, comprising the following steps:

[0006] S1. Multi-source heterogeneous data acquisition and preprocessing: Collect multi-source heterogeneous data during the operation of coal mine equipment, and clean, fill missing values, standardize, time align and dimensionality reduce the collected multi-source heterogeneous data to obtain different types of standardized data. The multi-source heterogeneous data includes physical sensor data, image data, audio data and text data.

[0007] S2. Multimodal feature extraction and fusion: Feature extraction is performed on different types of standardized data to obtain single features of each modality. Then, the single features of each modality are dynamically weighted and fused through a multi-level attention fusion network combined with a cross-modal interaction mechanism to form a comprehensive feature vector and construct the correlation between multi-source information and equipment faults.

[0008] S3. Fault Diagnosis Model Construction and Training: Construct a deep diagnostic model containing a main branch for fault category identification and a sub-branch for detailed fault identification. The sub-branch for detailed fault identification targets a single fault category and identifies detailed faults under that fault category by analyzing the fault frequency characteristics. Then, a hybrid supervised loss function is used to train the model, and the model parameters are iteratively optimized until convergence to obtain the optimal diagnostic model.

[0009] S4. Fault Identification and Maintenance Guidance Generation: Input the multi-source heterogeneous data processed in real time into the optimal diagnostic model. First, identify the major fault categories through the main branch, then analyze and subdivide the fault characteristics through the corresponding sub-branches, output accurate fault diagnosis results, and then generate targeted maintenance guidance based on the diagnosis results in conjunction with historical fault logs and maintenance case libraries.

[0010] Preferably, in S1, the physical sensing data includes vibration signals, temperature, pressure, current, and voltage; the image data includes infrared thermal imaging images and visible light monitoring images; the audio data includes equipment operating noise audio signals; and the text data includes equipment operation logs, maintenance records, and historical fault ledgers, and the text data is synchronously obtained from an enterprise resource planning system or a manufacturing execution system.

[0011] Preferably, the missing value imputation uses a generative adversarial network or interpolation, the standardization process maps the data of each dimension to a unified numerical range of [0, 1], and the dimensionality reduction process uses PCA or an autoencoder to compress the high-dimensional data.

[0012] Preferably, in S2, the feature extraction specifically includes: extracting time-domain and frequency-domain features from the time-series signals of physical sensing data using a one-dimensional convolutional neural network or LSTM;

[0013] Spatial features were extracted from infrared and visible light images using a two-stream convolutional neural network, respectively.

[0014] After converting the audio signal into a Mel spectrogram, it is input into a ResNet-like structure to extract acoustic features.

[0015] Semantic features are extracted from text data using word embeddings combined with convolutional neural networks;

[0016] The multi-level attention fusion network can adaptively learn the importance weights of different modal data for the current fault diagnosis task based on the real-time features of the input data. The cross-modal interaction layer allows the feature vectors of different modalities to interact and map in both directions, capturing the complex dependencies between modalities.

[0017] Preferably, in S3, the hybrid supervised loss function is defined as: L=L0+αL1+βL2+γL3+δL4, where L0 is the main classification loss of the fault category based on the comprehensive feature vector, L1, L2 and L3 are the auxiliary classification losses based on visible light image features, infrared image features and sensor signal features, respectively, L4 is the subdivision fault identification loss, which adopts an improved cross-entropy loss function to optimize the loss calculation by strengthening the mapping relationship between fault frequency features and subdivision fault categories, and α, β, γ and δ are hyperparameters used to balance the contribution of each branch loss. The hyperparameters are dynamically adjusted according to the performance feedback of the model on the validation set through an online learning mechanism, or automatically optimized and allocated through an independent task-related weight learning module.

[0018] The loss functions for L0, L1, L2, L3, and L4 are similar, but the predicted values ​​will differ. The loss function for L0 is L0 = ΣLc(c, c*) + ΣLr(b, b*), where Lc represents the cross-entropy loss function, Lr represents the smooth L1 loss function, c is the predicted class, b is the predicted location, and c* and b* represent the labeled samples.

[0019] Preferably, the major fault categories include mechanical faults, electrical faults, and hydraulic faults, and the sub-faults are specific fault types under each major fault category. Among them, the major fault categories under mechanical faults include bearing wear type faults, gear meshing failure type faults, and connecting component loosening type faults. Sub-identification is achieved by extracting the frequency amplitude, spectrum distribution, and harmonic features of the major fault categories.

[0020] Preferably, in S3, the sub-branch for fault identification has a built-in frequency feature analysis unit that converts the fault signal to the frequency domain through fast Fourier transform, extracts key indicators such as characteristic frequency, sideband spacing and spectral energy distribution, and establishes a mapping relationship library between indicators and sub-fault types.

[0021] Preferably, in S3, the model training process also includes model lightweighting, specifically using a combination of knowledge distillation, random pruning and optimization of Siamese networks, and quantization-aware training to convert the floating-point model into an INT8 low-precision model and encapsulate it in ONNX or TensorRT format to adapt to edge deployment.

[0022] Preferably, in S4, the maintenance guidance generation includes fault location, fault cause analysis, maintenance process recommendation, spare parts model matching, and push of historical successful maintenance cases. The maintenance guidance is displayed through the equipment monitoring platform interface or pushed to the responsible person via SMS / WeChat, and a maintenance work order is automatically generated.

[0023] Preferably, it also includes closed-loop optimization: collecting feedback data from maintenance personnel on diagnostic results and maintenance guidance, performing incremental learning based on the feedback data, and regularly updating model parameters and maintenance case library.

[0024] Compared with the prior art, the beneficial effects of the present invention are:

[0025] 1. This invention collects multi-source heterogeneous data and performs deep fusion to fully explore the complementary information of different types of data such as physical sensing, images, audio, and text, and constructs a precise information-fault correlation relationship. This effectively solves the problem that a single data type is difficult to fully characterize complex coupled faults and improves the accuracy of fault diagnosis.

[0026] 2. This invention extracts key indicators in the frequency domain of faults through a frequency feature analysis unit, achieving dual accurate identification of fault categories and sub-categories. For example, the sub-diagnosis of major fault categories provides a clear basis for precise maintenance.

[0027] 3. This invention employs a hybrid supervised loss function for model training, simultaneously focusing on fault category identification, single-modal feature discrimination, and detailed fault identification performance. By balancing the contributions of each branch through hyperparameters, the robustness and generalization ability of the model are improved. Combined with various lightweight techniques, this ensures the model can run efficiently on underground coal mine edge equipment, meeting real-time diagnostic requirements. Attached Figure Description

[0028] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Example 1

[0031] Please see Figure 1 This invention provides an intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data, comprising the following steps:

[0032] S1. Multi-source heterogeneous data acquisition and preprocessing: Multi-source heterogeneous data is collected during the operation of coal mine equipment. This data is then cleaned, imputed, standardized, time-aligned, and dimensionality reduced to obtain different types of standardized data. The multi-source heterogeneous data includes physical sensor data, image data, audio data, and text data. Physical sensor data includes vibration signals, temperature, pressure, current, and voltage. Image data includes infrared thermal imaging images and visible light monitoring images. Audio data includes equipment operating noise audio signals. Text data includes equipment operation logs, maintenance records, and historical fault ledgers. The text data is synchronously obtained from the Enterprise Resource Planning (ERP) system or Manufacturing Execution System (MES). Missing value imputation uses generative adversarial networks (GANs) or interpolation. Standardization maps data across dimensions to a unified numerical range of [0, 1]. Dimensionality reduction uses PCA or an autoencoder to compress high-dimensional data.

[0033] Furthermore, data acquisition involves the simultaneous collection of multi-source heterogeneous data through various sensors, monitoring systems, and information management systems deployed on coal mine equipment. Specifically, this includes: physical sensor data (vibration signals, temperature, pressure, current, and voltage, etc.), image data (infrared thermal imaging images and visible light monitoring images), audio data (equipment operating noise audio signals), and text data (equipment operation logs, maintenance records, and historical fault ledgers, etc.). Among these, text data can be obtained from Enterprise Resource Planning (ERP) systems or Manufacturing Execution System (MES) systems.

[0034] Data preprocessing: Outliers in vibration signals and temperature data are removed using the 3σ criterion. Generative Adversarial Networks (GANs) are used to supplement missing data from some sensors due to communication interruptions. All physical sensor data and image feature data are standardized to the [0, 1] interval. Time alignment of vibration signals, image frames, and audio clips is achieved through timestamp matching. PCA is used to reduce the dimensionality of visible light and infrared images, thereby lowering computational complexity.

[0035] S2. Multimodal Feature Extraction and Fusion: Feature extraction is performed on standardized data of different types to obtain single features for each modality. These single features are then dynamically weighted and fused using a multi-level attention fusion network combined with a cross-modal interaction mechanism to form a comprehensive feature vector, constructing the correlation between multi-source information and equipment faults. Feature extraction specifically includes: extracting time-domain and frequency-domain features from time-series signals of physical sensor data using a one-dimensional convolutional neural network or LSTM; extracting spatial features from infrared and visible light images using a dual-stream convolutional neural network; extracting acoustic features from audio signals after converting them to Mel spectrograms and inputting them into a ResNet-like structure; and extracting semantic features from text data using word embeddings combined with a convolutional neural network. The multi-level attention fusion network can adaptively learn the importance weights of different modal data for the current fault diagnosis task based on the real-time features of the input data. The cross-modal interaction layer allows bidirectional interaction and mapping learning of feature vectors from different modalities, capturing complex dependencies between modalities.

[0036] Furthermore, single-modal feature extraction: For different types of standardized data, specialized algorithms are used to extract features. For time-series signals such as vibration and current, one-dimensional convolutional neural networks (1D-CNN) or LSTM are used to extract temporal features (such as peak values ​​and RMS values) and frequency domain features (such as characteristic frequencies). For infrared and visible light images, dual-stream convolutional neural networks are used to extract spatial features such as temperature distribution and texture features, respectively. For audio signals, they are first converted to Mel spectrograms, and then input into a ResNet-like structure to extract acoustic features. For text data, word embedding techniques combined with convolutional neural networks are used to extract semantic features and capture fault-related descriptive information.

[0037] Multi-feature fusion: A multi-level attention fusion network is constructed, inputting single features extracted from each modality into the network. The network adaptively learns the importance weights of different modal data for the current fault diagnosis task based on the real-time features of the input data. Simultaneously, a cross-modal interaction layer is added during the fusion process, allowing bidirectional interaction and mapping learning between feature vectors of different modalities. This captures complex dependencies between modalities, mapping features of different dimensions to embedding vectors of the same dimension before weighted fusion to form a comprehensive feature vector, thus constructing a precise correlation between multi-source information and equipment faults. In other words, based on single-modal feature extraction, a multi-level attention fusion network is used for feature fusion to fully utilize the complementary information between features of different modalities. This network first dynamically evaluates the importance of different modal features in the current fault diagnosis task through an attention mechanism, assigning adaptive weights to each modal feature. Subsequently, the cross-modal interaction layer promotes information interaction between different modal features through a bidirectional mapping mechanism, mining the potential correlation between temperature distribution and vibration time series, and the implicit patterns of texture features and acoustic features. Finally, a comprehensive feature vector integrating multi-dimensional information is generated, providing a more discriminative feature representation for subsequent fault classification.

[0038] S3. Fault Diagnosis Model Construction and Training: A deep diagnostic model is constructed, consisting of a main branch for fault category identification and a sub-branch for detailed fault identification. The detailed fault identification sub-branch targets a single fault category and identifies sub-faults within that category by analyzing fault frequency characteristics. The model is then trained using a hybrid supervised loss function, iteratively optimizing the model parameters until convergence to obtain the optimal diagnostic model. The hybrid supervised loss function is defined as: L = L0 + αL1 + βL2 + γL3 + δL4, where L0 is the main classification loss for fault categories based on comprehensive feature vectors; L1, L2, and L3 are auxiliary classification losses based on visible light image features, infrared image features, and sensor signal features, respectively; L4 is the detailed fault identification loss, employing an improved cross-entropy loss function. This loss calculation is optimized by strengthening the mapping relationship between fault frequency characteristics and detailed fault categories. α, β, γ, and δ are hyperparameters used to balance the contributions of each branch's loss. These hyperparameters are dynamically adjusted based on the model's performance feedback on the validation set through an online learning mechanism, or automatically optimized and allocated through an independent task-related weight learning module. The loss functions for L0, L1, L2, L3, and L4 are similar, but the predicted values ​​will differ. The loss function for L0 is L0 = ΣLc(c, c*) + ΣLr(b, b*), where Lc represents the cross-entropy loss function, Lr represents the smooth L1 loss function, c is the predicted class, b is the predicted location, and c* and b* represent the labeled samples.

[0039] The major fault categories include mechanical faults, electrical faults, and hydraulic faults. The faults are further subdivided into specific fault types under each major fault category. Among them, the major category of mechanical faults includes bearing wear faults, gear meshing faults, and loose connection faults. The subdivision and identification are achieved by extracting the frequency amplitude, spectrum distribution, and harmonic features of the major fault categories.

[0040] The sub-branch for detailed fault identification has a built-in frequency feature analysis unit that converts the fault signal to the frequency domain through fast Fourier transform, extracts key indicators such as characteristic frequency, sideband spacing, and spectral energy distribution, and establishes a mapping relationship library between indicators and detailed fault types.

[0041] The model training process also includes lightweight model processing, which uses a combination of knowledge distillation, random pruning and optimization of Siamese networks, and quantization-aware training to convert the floating-point model into an INT8 low-precision model and encapsulate it in ONNX or TensorRT format to adapt to edge deployment.

[0042] Furthermore, the model structure is designed as follows: a deep diagnostic model with two branches is constructed, including a main branch for identifying major fault categories and a sub-branch for identifying detailed faults. The main branch identifies major fault categories such as mechanical faults, electrical faults, and hydraulic faults based on comprehensive feature vectors. For each major fault category, the sub-branch incorporates a frequency feature analysis unit, which converts the fault signal to the frequency domain using Fast Fourier Transform (FFT) to extract key indicators such as characteristic frequencies, sideband spacing, and spectral energy distribution. A mapping relationship library between these indicators and detailed fault types is established. For example, for major mechanical fault categories, by analyzing frequency amplitude differences, spectral distribution patterns, and harmonic characteristics, detailed faults such as bearing wear, gear meshing defects, or loose connections can be distinguished.

[0043] Adaptive Hybrid Supervised Loss Function Design: An adaptive hybrid supervised loss function is used to optimize model training. α, β, γ, and δ are hyperparameters that are dynamically adjusted through an online learning mechanism based on the model's performance feedback on the validation set (such as the accuracy of identifying specific faults), or automatically calculated using an independent task-related weight learning module after analyzing the current task objective to balance the loss contributions of each branch.

[0044] Model Training and Lightweighting: The preprocessed fused feature data is divided into training, validation, and test sets. The model is iteratively trained using the training set, and its performance is monitored using the validation set. Parameters are adjusted until the model converges. After training, a lightweighting process is initiated: a well-trained complex model is selected as the teacher model. Knowledge distillation strategies such as hard labeling, soft labeling, or noise injection are used to transfer its feature representations and decision logic to a simpler student model. FP16 mixed-precision computation is used during training to reduce computational load and GPU memory usage. During the inference phase, Quantization-Aware Training (QAT) is applied to quantize the model parameters from FP32 or FP16 to INT8, significantly reducing the model size and accelerating the inference process. Finally, the lightweight model is packaged into ONNX or TensorRT formats to adapt to the resource-constrained deployment conditions of edge devices in underground coal mines.

[0045] Specifically, during the model training phase, a multi-round, multi-sample training strategy was adopted to ensure the accuracy and robustness of the diagnostic model. First, a large-scale dataset covering various faults in coal mine equipment was collected. This dataset not only includes operational data under normal conditions but also records detailed data on sub-fault types under different major fault categories, such as mechanical, electrical, and hydraulic faults. Examples include vibration signals from bearing wear faults at different wear levels and noise signals from gear meshing failure faults at different speeds. The dataset underwent rigorous annotation to ensure that each sample accurately corresponds to the relevant fault category and sub-fault type.

[0046] During training, powerful computing resources are used for parallel computation to accelerate model convergence. Simultaneously, data augmentation techniques are introduced, such as rotating, flipping, and scaling image data, and adding noise and filtering signal data, to increase data diversity and generalization, preventing overfitting. After each training round, model performance is evaluated on an independent validation set. Based on the evaluation results, the values ​​of hyperparameters α, β, γ, and δ are adjusted, or automatically optimized through an independent task-related weight learning module, to balance the contribution of each branch's loss, ensuring the model achieves good recognition results across different fault types.

[0047] After multiple rounds of iterative optimization, when the model's performance metrics (such as accuracy, recall, and F1 score) on the validation set reach a preset threshold and no longer significantly improve, the model parameters are considered to have converged, yielding the optimal diagnostic model. At this point, the model is lightweighted by employing knowledge distillation techniques to transfer knowledge from the large, complex model to a smaller, lightweight model. Simultaneously, combined with Siamese network random pruning and optimization, and quantized perceptual training methods, the number of model parameters and computational load are further reduced. The floating-point model is converted to an INT8 low-precision model, reducing the model's computational resource requirements and improving inference speed. Finally, the lightweight model is packaged into ONNX or TensorRT format for efficient deployment on edge devices, enabling real-time intelligent diagnosis of coal mine equipment faults.

[0048] S4. Fault Identification and Maintenance Guidance Generation: Real-time processed multi-source heterogeneous data is input into the optimal diagnostic model. First, the main branch identifies the major fault categories, then the corresponding sub-branches analyze and refine the fault characteristics, outputting accurate fault diagnosis results. Based on these results, historical fault logs and maintenance case libraries are linked to generate targeted maintenance guidance. Maintenance guidance generation includes fault location, fault cause analysis, maintenance process recommendations, spare parts model matching, and push notifications of historical successful maintenance cases. Maintenance guidance is displayed through the equipment monitoring platform interface or pushed to the responsible personnel via SMS / WeChat, and maintenance work orders are automatically generated.

[0049] Furthermore, real-time fault identification: The trained optimal diagnostic model is deployed on a mining edge computing gateway or explosion-proof AI box and integrated into the coal mine equipment monitoring platform. It receives data streams from the equipment in real time, and after preprocessing, feature extraction and fusion, it is input into the model. First, the main branch identifies the major fault categories, and then the corresponding sub-branch is started to analyze and subdivide the fault features, outputting accurate diagnostic results such as fault categories, subdivided types and fault locations.

[0050] Fault root cause analysis: Construct a physical model of the equipment (such as a dynamic model or thermodynamic model), combine it with a data-driven model, and guide data learning through physical constraints. Simultaneously, based on a knowledge graph or rule base of fault characteristics and known physical fault mechanisms, automatically analyze and recommend possible fault causes and their corresponding physical principles, uncovering the root causes of faults and providing a basis for thorough maintenance and preventative measures.

[0051] Personalized Maintenance Guidance Generation: A reinforcement learning-based maintenance strategy optimization module is introduced. The reinforcement learning agent learns the optimal maintenance guidance strategy through interaction with simulated or real maintenance environments. When generating maintenance guidance, not only real-time diagnostic results and root cause analysis are considered, but also real-time equipment status parameters (such as temperature and vibration) and historical maintenance records (such as maintenance frequency and replaced parts) are integrated to automatically generate personalized maintenance guidance, including fault location, root cause explanation, step-by-step maintenance process, spare parts model matching, historical successful maintenance case recommendations, and preventative measures suggestions. Maintenance guidance is displayed in real-time through platform interface pop-ups, audible and visual alarms, and is simultaneously pushed to the responsible person via SMS / WeChat, and maintenance work orders are automatically generated, achieving seamless integration of diagnosis and maintenance, and supporting integration with enterprise ERP / MES systems.

[0052] It also includes closed-loop optimization: collecting feedback data from maintenance personnel on diagnostic results and maintenance guidance, performing incremental learning based on the feedback data, and regularly updating model parameters and maintenance case library.

[0053] Furthermore, the monitoring system includes a feedback interface, allowing maintenance personnel to annotate the accuracy of diagnostic results, the rationality of root cause analysis, and the practicality of maintenance guidance. Feedback data is collected to construct an incremental learning dataset. The model parameters are periodically updated using incremental learning algorithms, simultaneously optimizing the fault feature mapping database, fault mechanism association database, and maintenance case database. This forms a closed-loop iterative mechanism of "diagnosis-root cause analysis-maintenance guidance-feedback-optimization," continuously improving the model's diagnostic accuracy, root cause analysis accuracy, and the relevance of maintenance guidance.

[0054] Example 2

[0055] Taking the fault diagnosis of an underground belt conveyor in a coal mine as an example, the specific steps of applying the intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data described in this invention are as follows:

[0056] Multi-source heterogeneous data acquisition and preprocessing:

[0057] Data Acquisition: Vibration signals, bearing temperature, and motor current are collected using vibration sensors, temperature sensors, and current sensors deployed on the belt conveyor. Infrared thermal images and visible light monitoring images of key parts of the equipment are acquired using infrared thermal imagers and high-definition cameras. Noise audio signals from the equipment during operation are collected using audio acquisition equipment. Text data such as operation logs, historical maintenance records, and fault ledgers for the belt conveyor are obtained from the coal mine enterprise's ERP and MES systems.

[0058] Data preprocessing: Outliers in vibration signals and temperature data were removed using the 3σ criterion. GAN was used to supplement missing data from some sensors due to communication interruptions. All physical sensor data and image feature data were standardized to the [0, 1] interval. Time alignment of vibration signals, image frames, and audio clips was achieved through timestamp matching. PCA was used to reduce the dimensionality of visible light and infrared images, retaining 85% of the effective information.

[0059] Multimodal feature extraction and fusion:

[0060] Single-modal feature extraction: For vibration signals, a 1D-CNN was used to extract time-domain peak values, RMS values, and frequency-domain feature frequencies. For infrared and visible light images, a dual-stream CNN was used to extract temperature gradient features and surface texture features, respectively. After converting audio signals to Mel spectrograms, acoustic features were extracted using ResNet50. For text data, Word2Vec was used for word embedding, combined with a CNN to extract semantic features such as "bearing wear" and "gear noise."

[0061] Multi-feature fusion: The above single-modal features are input into a multi-level attention fusion network. The network adaptively learns the weights of each modal feature: the frequency domain feature weight of vibration signal is 0.35, the temperature gradient feature weight of infrared image is 0.25, the acoustic feature weight of audio is 0.2, and the semantic feature weight of text is 0.2. At the same time, bidirectional interaction of each modal feature is realized through cross-modal interaction layer, and finally weighted fusion is obtained to obtain comprehensive feature vector.

[0062] Fault diagnosis model construction and training:

[0063] Model Structure Design: The main branch adopts a CNN and fully connected layer structure to identify three major categories of faults: mechanical faults, electrical faults, and hydraulic faults. The sub-branch for fault identification focuses on mechanical faults and incorporates an FFT frequency analysis unit to extract indicators such as the characteristic frequency and sideband spacing of the fault signal. It establishes a mapping relationship between these indicators and major categories of faults, including bearing wear, gear meshing problems, and loose connections.

[0064] Adaptive hybrid supervised loss function settings: Total loss function L = L0 + 0.2L1 + 0.2L2 + 0.15L3 + 0.3L4, where L0 adopts the cross-entropy loss function and L4 adopts the improved cross-entropy loss function; During training, the recognition accuracy of each fault type on the validation set is monitored in real time through an online learning mechanism. When the recognition accuracy of the gear meshing failure category is lower than 90%, the δ value is automatically adjusted from 0.3 to 0.4 to strengthen the weight of the subdivided fault recognition loss.

[0065] Model Training and Lightweighting: The model was trained using 1000 sets of historical fault data (including 500 sets of normal data, 200 sets of mechanical fault data, 200 sets of electrical fault data, and 100 sets of hydraulic fault data, with the mechanical fault data containing 80 sets of major fault data covering 3 sub-types). The model converged after 100 iterations. This trained model was selected as the teacher model, and knowledge was transferred to a simplified student model using a soft-label knowledge distillation strategy. FP16 mixed-precision training was employed to reduce computational resource consumption, and then quantization-aware training was used to convert the model to an INT8 low-precision model, which was then packaged in TensorRT format. The model size was reduced by 75% compared to the original model, and the inference speed was increased by 3 times.

[0066] Fault identification and root cause analysis:

[0067] Real-time fault identification: The model deployed on the edge computing gateway is connected to the real-time data stream of the belt conveyor. At a certain moment, the model detects that the comprehensive feature vector matches the characteristics of the mechanical fault category. Further analysis of the sub-branches of the fault reveals that the characteristic frequency of the vibration signal is 120Hz and the sideband interval is 15Hz, which is consistent with the frequency characteristics of the gear meshing failure category. The diagnostic result is output as: Mechanical fault - Fault category - Gear meshing failure.

[0068] Fault Root Cause Analysis: By calling the dynamic physical model of the belt conveyor and combining it with the fault mechanism correlation analysis module, based on the fault characteristics of poor gear meshing, the corresponding physical mechanism was matched: long-term operation of the gears leads to wear on the tooth surface, increased meshing clearance, and periodic impact vibration during operation, forming a large type of abnormal noise. The root cause of the fault was determined to be wear on the gear tooth surface and excessive meshing clearance.

[0069] Maintenance instructions generated:

[0070] The reinforcement learning maintenance strategy optimization module combines current fault diagnosis results, root cause analysis, real-time bearing temperature of the belt conveyor, and recent maintenance records (gearbox lubricant was replaced 3 months ago) to generate personalized maintenance guidance: ① Fault location: Belt conveyor drive roller gearbox. ② Root cause: Gear tooth surface wear leading to excessive meshing clearance. ③ Maintenance procedure: First, shut down the machine and disconnect the power. Then, remove the gearbox end cover, check the gear wear and meshing clearance, replace the worn gear (model: M120), adjust the meshing clearance to 0.2-0.3mm, add special gear oil (model: GL-585W / 90), reassemble, then perform a no-load test run, and finally conduct a load test. ④ Recommended spare parts: 1 M120 gear, 2L GL-585W / 90 gear oil. ⑤ Preventive measures: Check gear meshing clearance every month and change gear oil every 6 months. ⑥ Push three successful repair cases of similar gear meshing failures to the mobile devices of maintenance personnel, and automatically generate repair work orders to connect to the enterprise MES system.

[0071] Closed-loop optimization:

[0072] After completing the repairs according to the maintenance instructions, the maintenance personnel reported in the system that the diagnostic results were accurate, the root cause analysis was reasonable, the maintenance process was clear, and the spare parts recommendations were precise. The equipment operated normally after the repair. The system added the fault data, diagnostic results, root cause analysis, maintenance records, and feedback information to the incremental learning dataset. At the end of the month, the system updated the model parameters using an incremental learning algorithm, optimized the mapping relationship between gear wear and vibration frequency in the fault mechanism association library, and added this repair case to the maintenance case library to provide a reference for subsequent similar fault diagnosis and repair.

[0073] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent fault diagnosis of coal mine equipment based on multi-source heterogeneous data, characterized in that, Includes the following steps: S1. Multi-source heterogeneous data acquisition and preprocessing: Collect multi-source heterogeneous data during the operation of coal mine equipment, and clean, fill missing values, standardize, time align and dimensionality reduce the collected multi-source heterogeneous data to obtain different types of standardized data. The multi-source heterogeneous data includes physical sensor data, image data, audio data and text data. S2. Multimodal feature extraction and fusion: Feature extraction is performed on different types of standardized data to obtain single features of each modality. Then, the single features of each modality are dynamically weighted and fused through a multi-level attention fusion network combined with a cross-modal interaction mechanism to form a comprehensive feature vector and construct the correlation between multi-source information and equipment faults. S3. Fault Diagnosis Model Construction and Training: Construct a deep diagnostic model containing a main branch for fault category identification and a sub-branch for detailed fault identification. The sub-branch for detailed fault identification targets a single fault category and identifies detailed faults under that fault category by analyzing the fault frequency characteristics. Then, a hybrid supervised loss function is used to train the model, and the model parameters are iteratively optimized until convergence to obtain the optimal diagnostic model. S4. Fault Identification and Maintenance Guidance Generation: Input the multi-source heterogeneous data processed in real time into the optimal diagnostic model. First, identify the major fault categories through the main branch, then analyze and subdivide the fault characteristics through the corresponding sub-branches, output accurate fault diagnosis results, and then generate targeted maintenance guidance based on the diagnosis results in conjunction with historical fault logs and maintenance case libraries.

2. The intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data according to claim 1, characterized in that, In S1, the physical sensing data includes vibration signals, temperature, pressure, current, and voltage; the image data includes infrared thermal imaging images and visible light monitoring images; the audio data includes equipment operating noise audio signals; and the text data includes equipment operation logs, maintenance records, and historical fault ledgers. The text data is synchronously obtained from the enterprise resource planning system or manufacturing execution system.

3. The intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data according to claim 1, characterized in that, The missing value imputation uses generative adversarial networks or interpolation methods, the standardization process maps the data of each dimension to a unified numerical range of [0, 1], and the dimensionality reduction process uses PCA or autoencoders to compress the high-dimensional data.

4. The intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data according to claim 1, characterized in that, In S2, the feature extraction specifically includes: extracting time-domain and frequency-domain features from the time-series signals of physical sensing data using a one-dimensional convolutional neural network or LSTM; Spatial features were extracted from infrared and visible light images using a two-stream convolutional neural network, respectively. After converting the audio signal into a Mel spectrogram, it is input into a ResNet-like structure to extract acoustic features. Semantic features are extracted from text data using word embeddings combined with convolutional neural networks; The multi-level attention fusion network can adaptively learn the importance weights of different modal data for the current fault diagnosis task based on the real-time features of the input data. The cross-modal interaction layer allows the feature vectors of different modalities to interact and map in both directions, capturing the complex dependencies between modalities.

5. The intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data according to claim 1, characterized in that, In S3, the hybrid supervised loss function is defined as: L=L0+αL1+βL2+γL3+δL4, where L0 is the main classification loss of the fault category based on the comprehensive feature vector, L1, L2 and L3 are the auxiliary classification losses based on visible light image features, infrared image features and sensor signal features, respectively, and L4 is the subdivision fault identification loss. An improved cross-entropy loss function is adopted, which optimizes the loss calculation by strengthening the mapping relationship between fault frequency features and subdivision fault categories. α, β, γ and δ are hyperparameters used to balance the contribution of each branch loss. The hyperparameters are dynamically adjusted according to the performance feedback of the model on the validation set through an online learning mechanism, or automatically optimized and allocated through an independent task-related weight learning module. The loss functions for L0, L1, L2, L3, and L4 are similar, but the predicted values ​​will differ. The loss function for L0 is L0 = ΣLc(c, c*) + ΣLr(b, b*), where Lc represents the cross-entropy loss function, Lr represents the smooth L1 loss function, c is the predicted class, b is the predicted location, and c* and b* represent the labeled samples.

6. The intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data according to claim 1, characterized in that, The major fault categories include mechanical faults, electrical faults, and hydraulic faults. The sub-faults are the specific fault types under each major fault category. Among them, the major fault categories under mechanical faults include bearing wear faults, gear meshing failure faults, and loose connection faults. Sub-fault identification is achieved by extracting the frequency amplitude, spectrum distribution, and harmonic features of the major fault categories.

7. The intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data according to claim 1, characterized in that, In S3, the sub-branch for detailed fault identification has a built-in frequency feature analysis unit. It converts the fault signal to the frequency domain through fast Fourier transform, extracts key indicators such as characteristic frequency, sideband spacing and spectral energy distribution, and establishes a mapping relationship library between indicators and detailed fault types.

8. The intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data according to claim 1, characterized in that, In S3, the model training process also includes lightweight model processing, which uses a combination of knowledge distillation, random pruning and optimization of Siamese networks, and quantization-aware training to convert floating-point models into low-precision INT8 models and encapsulate them in ONNX or TensorRT formats to adapt to edge deployment.

9. The intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data according to claim 1, characterized in that, In S4, maintenance guidance is generated, including fault location, fault cause analysis, maintenance process recommendation, spare parts model matching, and push of historical successful maintenance cases. The maintenance guidance is displayed through the equipment monitoring platform interface or pushed to the responsible person via SMS / WeChat, and maintenance work orders are automatically generated.

10. The intelligent fault diagnosis method for coal mine equipment based on multi-source heterogeneous data according to claim 1, characterized in that, It also includes closed-loop optimization: collecting feedback data from maintenance personnel on diagnostic results and maintenance guidance, performing incremental learning based on the feedback data, and regularly updating model parameters and maintenance case library.

Images