A method, system, equipment and medium for safety monitoring of underground coal mine operations

Abstract

This invention discloses a method, system, equipment, and medium for safety monitoring of underground coal mine operations, relating to the field of coal mine safety monitoring technology. This invention integrates a dynamic quantum convolutional layer, an electromagnetic field compensation channel, a dual-band feature alignment layer, and a vibration temporal alignment layer into a MAF-YOLO network. During underground coal mine operation safety monitoring, the dynamic quantum convolutional layer extracts target detail features and target contour features from the image, eliminating the fogging phenomenon caused by dust interference in the coal mine. The electromagnetic field compensation channel compensates for electromagnetic offset in the image, eliminating the ripple phenomenon caused by electromagnetic interference in the coal mine. The dual-band feature alignment layer and vibration temporal alignment layer align the target detail features and target contour features, eliminating the ghosting phenomenon caused by vibration in the coal mine. Overall, this improves the accuracy of underground coal mine operation safety monitoring.

Description

Technical Field

[0001] This invention relates to the field of coal mine safety monitoring technology, and in particular to a method, system, equipment and medium for monitoring safety during underground coal mine operations. Background Technology

[0002] Underground safety monitoring in coal mines is a crucial component of modern mine safety management. By integrating sensor technology, data communication, automated control, and intelligent analysis, it enables real-time monitoring, risk warning, and intelligent decision-making regarding the mine's production environment. Monitoring primarily includes gas monitoring, dust monitoring, temperature and humidity monitoring, hydrological monitoring, electromechanical equipment monitoring, power supply system monitoring, geological disaster monitoring, and personnel safety monitoring. Among these, personnel safety monitoring is currently the most critical aspect, typically achieved by deploying high-definition cameras underground to cover key areas, while simultaneously using AI algorithms to identify personnel violations and safety conditions.

[0003] Currently, the YOLOv5 open-source network model is commonly used for automated monitoring of personnel safety operations. However, during long-term monitoring, the image blurring rate caused by visible light and infrared radiation underground, equipment vibration, and other factors leads to an increase in target positioning errors over time. This is mainly manifested in the fogging phenomenon caused by dust interference in coal mines, the ripples caused by electromagnetic interference, and the ghosting caused by vibrations. Consequently, the currently used YOLOv5 open-source network model struggles to clearly identify various features in the image, resulting in low accuracy in monitoring underground coal mine operations and difficulty in predicting dangerous situations. Summary of the Invention

[0004] This invention provides a method, system, equipment, and medium for monitoring safety in underground coal mine operations. It can solve the problems in the prior art where dust interference in underground coal mines causes fogging in images, electromagnetic interference in underground coal mines causes ripples in images, and vibration in underground coal mines causes ghost images in images, resulting in low accuracy in monitoring safety in underground coal mine operations.

[0005] This invention provides a method for safety monitoring in underground coal mine operations, comprising the following steps:

[0006] Acquire multiple construction images of underground coal mine workers in the visible light band during the construction process within a preset time period, as well as data sequences of dust concentration, magnetic field strength, and vibration in the underground coal mine environment.

[0007] When the current dust concentration in the underground coal mine is below a threshold, multiple surface temperature distribution images of the target are extracted by using infrared waves to penetrate the dust.

[0008] Multiple construction images, multiple surface temperature distribution images, as well as data sequences of environmental dust concentration, environmental magnetic field strength, and environmental vibration are input into a pre-trained work safety monitoring model. For each construction image, during feature extraction, the dynamic quantum convolutional layer dynamically extracts target detail features and target contour features based on the dynamic magnitude of the current environmental dust concentration; for each surface temperature distribution image, target temperature morphology features are extracted.

[0009] The electromagnetic field compensation channel uses the ambient magnetic field strength data to correct the offset of the target detail features and target contour features; the dual-band feature alignment layer aligns and fuses the corrected target detail features extracted in the visible light band with the target temperature and morphology features extracted in the infrared band to obtain the target detail morphology features; the vibration time-series alignment layer uses the ambient vibration data to time-series align and fuse the target detail morphology features with the corrected target contour features to obtain the feature map corresponding to each image output by the operation safety monitoring model.

[0010] Among them, the target detail features are the detailed texture features of the target extracted from the corresponding construction image by the dynamic quantum convolutional layer when the current dust concentration is below the threshold; the target contour features are the edge contour features of the target extracted from the corresponding construction image by the dynamic quantum convolutional layer when the current dust concentration is above the threshold.

[0011] Identify the construction process within the feature map corresponding to each image and generate operation safety monitoring results.

[0012] Preferably, the acquisition of the operation safety monitoring model includes:

[0013] The improved MAF-YOLO network was trained using multiple construction images to obtain a work safety monitoring model;

[0014] The improved MAF-YOLO network incorporates a dynamic quantum convolutional layer and an electromagnetic field compensation channel into the feature extraction module of the MAF-YOLO network, and incorporates a dual-band feature alignment layer and a vibration time sequence alignment layer into the feature fusion module of the MAF-YOLO network.

[0015] Preferably, when the dynamic quantum convolutional layer extracts features from the construction image, the dynamic quantum convolutional layer DQConv adjusts the weights of the convolutional kernels of the feature extraction module according to the dust concentration in the coal mine, as expressed as:

[0016] ;

[0017] Where: C d Indicates dust concentration; α=0.25; β=0.03; W dq W represents the dynamically adjusted convolutional kernel weights. base Indicates the weights of the basic convolutional kernel;

[0018] The dynamic quantum convolutional layer DQConv monitors the dust concentration in coal mines by dynamically amplifying the edge response weights. When the dust concentration is below a threshold, the edge response weights are enhanced during feature extraction to extract target detail features in the construction image. When the dust concentration is above a threshold, the edge contour feature extraction is enhanced during feature extraction to extract target contour features in the construction image.

[0019] Preferably, when the electromagnetic field compensation channel corrects the offset of the target detail features and target contour features, it corrects the feature offset of the target detail features and target contour features in the image using a magnetometer, as expressed as:

[0020] ;

[0021] Among them: B rms Indicates electromagnetic field strength; B max =50μT; F corr F represents the corrected eigenvalues; raw γ represents the original eigenvalue; γ represents the compensation coefficient.

[0022] Preferably, the vibration timing alignment layer uses environmental vibration data to temporally align the target's detailed morphological features with the corrected target contour features, including:

[0023] The vibration timing alignment layer measures micro-vibration data in the current environment using a triaxial micro-vibration sensor. Based on the micro-vibration data and the images captured by the time-synchronized camera, the acquired data is synchronized with the distortion caused by image jitter. By calibrating the camera clock signal synchronously, the detailed morphological features of the target are time-aligned with the corrected target contour features.

[0024] Preferably, after the feature map corresponding to each image is obtained, the features of the target in the feature map are corrected by the Tri-Head triaxial stress correction head and the multi-task loss function, including:

[0025] The Tri-Head, a triaxial stress correction head, dynamically corrects the target confidence in the feature map by combining XYZ axial stress data. It uses pressure data, micro-vibration sensors, and infrared thermal imagers to sense pressure changes in the roof, coal wall, and floor of the coal mine in real time and dynamically adjusts the detection area.

[0026] The multi-task loss function identifies missing features in the feature map by simulating a crack scanner, a metal fatigue monitor, and a dual-camera calibration system.

[0027] This invention also provides a coal mine underground operation safety monitoring system, comprising:

[0028] The data acquisition module is used to acquire multiple construction images of coal mine workers in the visible light band during the construction process within a preset time period, as well as data sequences of dust concentration, magnetic field strength, and vibration in the coal mine environment.

[0029] When the current dust concentration in the underground coal mine is below a threshold, multiple surface temperature distribution images of the target are extracted by using infrared waves to penetrate the dust.

[0030] The detection module is used to input multiple construction images, multiple surface temperature distribution images, as well as data sequences of environmental dust concentration, environmental magnetic field strength, and environmental vibration into a pre-trained work safety monitoring model. For each construction image, during feature extraction, the dynamic quantum convolutional layer dynamically extracts target detail features and target contour features based on the dynamic magnitude of the current environmental dust concentration; for each surface temperature distribution image, it extracts target temperature morphology features.

[0031] The electromagnetic field compensation channel uses the ambient magnetic field strength data to correct the offset of the target detail features and target contour features; the dual-band feature alignment layer aligns and fuses the corrected target detail features extracted in the visible light band with the target temperature and morphology features extracted in the infrared band to obtain the target detail morphology features; the vibration time-series alignment layer uses the ambient vibration data to time-series align and fuse the target detail morphology features with the corrected target contour features to obtain the feature map corresponding to each image output by the operation safety monitoring model.

[0032] Among them, the target detail features are the detailed texture features of the target extracted from the corresponding construction image by the dynamic quantum convolutional layer when the current dust concentration is below the threshold; the target contour features are the edge contour features of the target extracted from the corresponding construction image by the dynamic quantum convolutional layer when the current dust concentration is above the threshold.

[0033] The recognition module is used to identify the construction process within the feature map corresponding to each image and generate work safety monitoring results.

[0034] This invention also provides an electronic device, including a memory and a processor;

[0035] The memory is used to store computer programs;

[0036] When the processor executes the computer program stored in the memory, it implements the steps of the coal mine underground operation safety monitoring method as described above.

[0037] This invention also provides a computer-readable storage medium, characterized in that it is used to store a computer program, which, when executed by a processor, implements the steps of a coal mine underground operation safety monitoring method as described above.

[0038] This invention provides a method, system, equipment, and medium for safety monitoring in underground coal mine operations. Compared with existing technologies, its advantages are as follows:

[0039] This invention integrates a dynamic quantum convolutional layer and an electromagnetic field compensation channel into the feature extraction module of the MAF-YOLO network, and a dual-band feature alignment layer and a vibration temporal alignment layer into the feature fusion module of the MAF-YOLO network. During safety monitoring of underground coal mine operations, in the feature extraction stage, the dynamic quantum convolutional layer adjusts the emphasis of the feature extraction module on detail features and target contour features based on the current environmental dust concentration. When the dust concentration is low, target detail features are extracted; when the dust concentration is high, target contour features are extracted. This eliminates the fogging phenomenon in the image caused by dust interference in underground coal mines. Simultaneously, the electromagnetic field compensation channel corrects the target detail features based on the current environmental magnetic field strength. The offset of segment features and target contour features eliminates the phenomenon of ripples in images caused by electromagnetic interference in underground coal mines. When fusing target detail features and target contour features, the dual-band feature alignment layer captures target shape details through smoke using short-wave infrared and integrates these details into the overall feature set. Simultaneously, the vibration timing alignment layer aligns the target detail features and target contour features temporally based on vibration offsets collected by microseismic sensors, eliminating the phenomenon of ghosting in images caused by vibrations in underground coal mines. Finally, the feature map obtained by fusing target detail features and target contour features is used to identify safety in underground coal mine operations, thus improving the overall accuracy of safety monitoring in underground coal mine operations. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the overall process of a safety monitoring method for underground coal mine operations provided in an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the monitoring process for a safety monitoring method for underground coal mine operations provided in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the system architecture of a coal mine underground operation safety monitoring method provided in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram showing the real-time monitoring results of a coal mine underground operation safety monitoring method provided in an embodiment of the present invention. Detailed Implementation

[0044] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0045] See Figure 1 This invention provides a method for safety monitoring in underground coal mine operations, which specifically includes the following steps:

[0046] Step S1: Collection of violations and dangerous situations: Use self-developed intrinsically safe mining cameras or networks to collect on-site images of violations and dangerous situations of personnel underground, such as not wearing safety helmets, smoking, hot work, and mine collapse, as these images change over time.

[0047] Step S2: Dataset Upload: The collected images are transmitted to the server's Python backend using the Django framework; the frontend uploads the image data to a specific folder on the server via the network, and the server backend program automatically starts the YOLOv5 network model after receiving the images.

[0048] Step S3: Dataset Training and Labeling: Based on the acquired mine images, generate a mining-specific AI recognition training set. Use the labelimg software to select violations and dangerous situations to generate corresponding total training and test training sets; specifically including:

[0049] Step S31: Improve the quality of the dataset by flipping, cropping, and adjusting the hue, brightness, and resolution of the sample images.

[0050] Step S32: Use the YOLO format of the labelimg annotation software to annotate the processed image photos, classifying them according to violation type and danger type.

[0051] Step S33: Randomly select 80% of the total number of training images from the labeled training set, and use the remaining 20% ​​as the test dataset for testing the model algorithm after training.

[0052] Step S34: Obtain the average precision (mAP) of the overall sample detection, the current average precision (P), and the average recall (R). The specific formulas are as follows:

[0053] .

[0054] .

[0055] .

[0056] Wherein: TP refers to the number of correctly identified violations and dangerous situations; FN refers to the number of unidentified violations and dangerous situations; FP refers to the number of incorrectly identified violations and dangerous situations; and N represents the number of categories of violations and dangerous situations.

[0057] Step S4: MAF-YOLOv5 network model algorithm optimization: Based on an innovative architecture of quantum computing, multispectral fusion, and joint decision-making of physical fields, high-precision real-time detection in complex underground coal mine environments is achieved, specifically including:

[0058] Step S41: Quantum-environment coupling Backbone performance verification, improving recognition stability and anti-interference capability at the Backbone level through dynamic quantum convolutional layers and electromagnetic field compensation channels.

[0059] Dynamic quantum convolutional layer (DQConv) adjusts the kernel weights in real time based on the downhole dust concentration, using the following formula: C d Where the dust concentration is (unit: mg / m³), α=0.25, β=0.03, W dq W represents the dynamically adjusted convolutional kernel weights. base This represents the weights of the basic convolutional kernel. Under dust cover, traditional convolutional kernels, with their fixed weights, are prone to losing details. Dynamic quantum convolutional layers, however, dynamically amplify edge response weights and monitor dust concentration in real time (e.g., dust concentration gradient in coal mines). When the dust concentration exceeds a safe threshold (e.g., >500mg / m³ in coal mines), a cross-network layer feature remediation scheme is automatically triggered. The front-end network focuses on enhancing image edge sharpening, while the back-end network focuses on enhancing contour feature channels, allowing the system to automatically allocate priorities for different features. In low dust conditions, edge response weights are enhanced; in high dust conditions, contour feature extraction is emphasized, enabling recognition problems such as (not wearing a safety helmet, broken belt, etc.) to still be identified in blurred images. When the dust concentration is too high, the stability of recognition is improved by enhancing the edge or contour features of the image in the dusty environment.

[0060] In quantum mechanics, a qubit can exist simultaneously in a superposition of |0〉 and |1〉 states, described by probability amplitudes. Traditional convolution kernels have fixed weights and can be viewed as a "deterministic feature extractor." DQConv, however, treats the weights of the convolution kernel as a dynamic probability distribution. It dynamically adjusts the probability distribution of the basic weights based on environmental input, thereby changing the tendency of the convolution kernel to "observe" and "extract" features. DQConv directly couples this to the physical environment, mimicking how external fields (such as electromagnetic fields) couple with the system in quantum systems, changing its wave function shape and energy levels, thus affecting the system's observable properties.

[0061] Furthermore, DQConv introduces continuous and smooth dynamic adjustment. Its weights are not simply "on" or "off", but change optimally according to the environment on a continuous spectrum. This is similar to the probabilistic nature of quantum states, which transcends classical binary thinking.

[0062] The electromagnetic compensation channel is as follows: characteristic offset is corrected using magnetometer data, and the formula is: B rms B is the electromagnetic field strength. max =50μT, verified under normal ventilation (dust <50mg / m³); F corr F represents the corrected eigenvalues; raw γ represents the original eigenvalue; γ represents the compensation coefficient; MAF-YOLO's mAP@0.5 is 96.8%, a 3.3% improvement over YOLOv8-X. In high dust conditions after blasting (500mg / m³): MAF-YOLO's mAP@0.5 reaches 87.4%, with a 45.3% improvement in eigenvalue stability. Regarding electromagnetic interference (30μT): MAF-YOLO's mAP@0.5 is 89.2%, with a 21.9% improvement in anti-interference capability. It uses magnetometer data to cancel electromagnetic interference from downhole equipment (such as large motors), similar to noise-canceling headphones filtering out noise. Strong electromagnetic fields in industrial environments (such as power lines and motors) can interfere with the electronic signals of image sensors, causing ripples, noise, or color distortion in the image. Indeed, magnetic field fluctuations can cause abnormal charge transfer in sensor chips, resulting in motion target blurring (such as blurred edges of coal blocks when monitoring conveyor belts in coal mines). By using Hall sensors (such as the SS495A) to capture the strength and direction of the current environmental magnetic field and identify interference characteristics such as power frequency harmonics, a three-axis electromagnetic coil array generates a compensating magnetic field that is opposite in phase and matches the strength of the interfering magnetic field. This directly neutralizes electromagnetic noise sources around the sensor. After compensation in electromagnetic interference environments, image noise is reduced by 60%, eliminating ripple noise caused by power lines and color distortion caused by motors. Under normal conditions, it can identify more minor defects such as belt cracks and loose safety helmet buckles than traditional algorithms, improving the stability and clarity of images captured by cameras in the face of underground electromagnetic interference.

[0063] Step S42: Verify the effectiveness of the multispectral spatiotemporal fusion Neck by using dual-band feature alignment and vibrational timing alignment to reduce the fusion error of the identification.

[0064] Dual-band feature alignment: This method fuses visible light (400-700nm) and short-wave infrared (900-1700nm) features. Alignment error is dynamically controlled using a temperature coefficient T=0.6 and a Sigmoid function. Visible light captures object shape details, while short-wave infrared penetrates dust to capture temperature characteristics. The thermal imaging properties of short-wave infrared are used to capture the surface temperature distribution of equipment, for example, identifying tiny hot spots in overheated (>80℃) conveyor belt bearings. The feature weights are dynamically adjusted using a temperature coefficient T=0.6 and a Sigmoid function, and the alignment error is controlled using the following formula: Where ΔT is the dual-band temperature difference, when the temperature difference ΔT>10℃, the weight of short-wave infrared is automatically increased to more than 70%, suppressing misjudgment caused by smoke obstruction of visible light. The image of the infrared camera is overlaid on the image of the visible light camera, and cross-modal detail alignment and misjudgment suppression are achieved through T=0.6 and the Sigmoid function to solve the problem of not being able to accurately detect the heat point when smoke covers the area under conditions such as open flames in mines.

[0065] Vibration Timing Alignment: Utilizing micro-vibration sensors to synchronize feature map timing, the alignment error is ≤1.8 pixels. Verified results show that in conveyor belt crack detection, the fusion error decreased from 6.7 pixels to 1.2 pixels; in hydraulic support tilt detection, the fusion error decreased from 8.9 pixels to 1.8 pixels; and in personnel safety helmet recognition, the fusion error decreased from 5.1 pixels to 0.7 pixels. Given the frequent vibrations underground, which can cause image distortion and loss of recognizability, a three-axis micro-vibration sensor synchronizes the data with the time-synchronized images captured by a camera. This data is then synchronized with the distortion caused by image jitter. By synchronously calibrating the camera clock signal, sub-pixel alignment of vibration data and image frames is achieved. Texture reconstruction is performed on blurred images, improving the recognition of details such as safety helmet straps and hydraulic valve cracks, and restoring the image, thus enhancing the stability of underground image recognition.

[0066] Step S43: Stress-vision joint head robustness test, using a triaxial stress correction head (Tri-Head) and optimized multi-task loss function to reduce the false negative rate.

[0067] The Tri-Head: Combines XYZ axial stress data to dynamically correct the target confidence level, with a weight allocation of w=(0.4, 0.4, 0.2). By linking pressure data with micro-seismic sensors and infrared thermal imagers, it dynamically adjusts the key detection areas by sensing pressure changes in the coal mine roof, coal wall, and floor in real time. When the pressure data is abnormal but the image does not change, a secondary scan is triggered (such as infrared detection of rock mass temperature rise), avoiding misjudgment by a single sensor and solving the problem of traditional cameras "not being able to see physical hazards".

[0068] Multi-task loss function: Jointly optimize detection and stress early warning tasks, with a weighting ratio of 0.7L. det +0.2L stress +0.1L sync Verified results show that under pressure from the roof (Z-axis), the false negative rate decreased from 23.5% to 2.1%; under coal wall spalling (X-axis), the false negative rate decreased from 17.8% to 1.7%; and under floor bulging (Y-axis), the false negative rate decreased from 19.4% to 2.3%. This multi-task function is like installing three "alarms" for mine safety, working in tandem. The crack scanner (main task) focuses 70% of its energy on identifying equipment cracks (such as belt breakage marks). Through repeated comparative training, the AI ​​becomes better at detecting minute cracks. The metal fatigue monitor uses 20% of its resources to detect metal deformation or signs of fatigue (e.g., alarming when material stretching exceeds 0.2 mm / m), essentially equipping the equipment with "fatigue sensing patches." The dual-camera calibration system uses 10% of its capabilities to automatically align the temperature anomalies (e.g., a sudden temperature rise of more than 8°C) between ordinary camera images and thermal imaging images, preventing the omission of dangers due to image mismatch. After separating tasks, MAF-YOLO can effectively reduce the false negative rate and improve safety.

[0069] Step S44: Full architecture test. Using the dedicated NVIDIA Jetson Orin NX chip (200TOPS computing power), the processing performance is faster. It is equipped with the innovative MAF-YOLO framework and uses a self-built mine safety dataset (MSD-2025) containing 12 key targets (anchor bolt breakage, conveyor belt misalignment, etc.). The extreme working condition samples account for 35% (dust concentration >200mg / m³, illuminance mutation >10^5lux). The test results are shown in Table 1.

[0070] Table 1 Test Results

[0071]

[0072] Even in extreme environments with dust concentrations exceeding 200 mg / m³, the recognition accuracy remains at 89.4% (compared to only 63.1% for ordinary systems), with a false alarm rate reduced by 81%.

[0073] Step S5: Model Training and Recognition: Use the training set to train the YOLOv5 network model for underground AI image recognition to generate the underground operation recognition model and algorithm required by the video monitoring platform. This recognition model is a new type of underground algorithm model generated by algorithm optimization based on a unique dataset. The safe operation recognition algorithm identifies and provides the types of violations and dangerous situations, as well as the corresponding information related to violations and dangerous situations.

[0074] The YOLOv5 network model includes: creating a brand-new dataset of underground safety operation violations and hazardous situations. The dataset includes identification categories from common mine worker violations, hazardous situations, and equipment status detection. Violations include not wearing a safety helmet correctly, open flames underground, employees leaving their posts, employees sleeping on duty, and employees smoking; hazardous situations include open flames underground, smoke detection, open flame detection, and mine collapse detection; equipment status detection includes belt breakage detection, large coal lump detection, and coal flow quality detection.

[0075] By training a training set of violations and dangerous situations using YOLOv5 with optimized algorithms, an underground safety operation model was generated. The trained YOLOv5 safety operation model was then used to monitor and test the video stream transmitted from an intrinsically safe mining camera developed in-house, obtaining information such as the type of violation, prediction box, violation location, violation image screenshot, and violation time.

[0076] Step S6: Video recognition processing: Import the trained downhole safety operation recognition algorithm model into the video monitoring platform developed based on Java to activate the early warning and display it on the host computer page developed based on Vue. Record the recognition results in the database for easy querying later.

[0077] The server backend program generates information such as the violation name, violation image, violation location, and violation time, and returns the results to the platform page in JSON format; the platform page displays the identification results and alarm data.

[0078] Step S7: The backend algorithm recognition results are returned to the platform page. The system will store the recognition results, also known as "historical records," to facilitate the retrieval of past recognition records and the analysis and processing of violations. Based on the training model, various underground mining recognition and processing application scenarios have been added, including dynamic monitoring of water and coal moisture content through texture feature recognition and multimodal data fusion; detection of roadway wall support structure displacement combined with point cloud data to construct a three-dimensional safety model for monitoring underground roadway deformation; and microscopic image analysis technology to identify the visual features of different ore types to achieve rapid identification and sorting of ore components.

[0079] Step S71: Dynamically monitor the water content of coal through texture feature recognition and multimodal data fusion. Texture feature recognition: Based on the YOLOv5 architecture, construct a texture difference detection model between wet and dry coal blocks, extract features, and use visible light images to analyze the surface roughness of coal blocks (gray-level co-occurrence matrix energy value <0.2 indicates wetness) and color saturation (HSV saturation is reduced by 15-30% in wet areas). Transfer learning: Use a pre-trained ResNet-50 backbone network, freeze the parameters of shallow convolutional layers, and fine-tune the fully connected layers for small sample data (<1000 labeled images) to improve the model's generalization ability. The training loss function is: L oss =λ cls ·L cls +λ box ·L box +λ texture ·L texture , of which L texture To mitigate the texture feature contrast loss, cosine similarity is used to quantify the difference between wet and dry regions. By learning the texture patterns and surface roughness variations of a large number of coal block images, dry coal exhibits higher roughness, while water-rich coal is smoother with higher moisture content. Multimodal data fusion is employed: visible light images (400-700nm) and infrared thermal imaging (8-14μm) data are fused to establish a coal pile surface temperature-moisture content correlation model. Infrared thermal imaging: based on the principle of water evaporation and heat absorption, the surface temperature of dry coal blocks (moisture content <5%) is 8-12℃ higher than that of wet areas (moisture content >15%). High-humidity areas are located using a temperature gradient map, with the formula: ΔT = k·(W dry -W wet (k=0.75), where ΔT is the temperature difference, W dry W wet The moisture content of dry and wet coal blocks is determined separately. Infrared cameras are used to locate the wet areas by identifying the temperature difference in the coal pile, while visible light cameras are used to observe surface color changes. Multimodal data fusion is used to combine visible light images and infrared thermal imaging data to indirectly determine the moisture content by analyzing the temperature distribution differences on the coal pile surface. This can effectively identify low-temperature areas on the conveyor belt surface caused by water seepage.

[0080] Step S72: Detect the displacement of the roadway wall support structure and construct a 3D safety model using point cloud data to monitor underground roadway deformation. Detect the displacement of the roadway wall support structure: Based on a binocular depth camera (such as an explosion-proof model), real-time acquisition of 3D point cloud data of the roadway wall is performed. Using 3D point cloud slicing technology, transverse cross-sectional slices are generated at 0.5m intervals along the roadway axis. Cracks are located through curvature calculation and normal vector analysis. Crack quantification formula: Where C is the local curvature difference value, ni is the normal vector of the i-th point in the point cloud, and navg is the average normal vector of the region. When C > 0.35, it is determined to be a crack region. The ICP (Iterative Closest Point) algorithm is used to compare the historical point cloud with the real-time point cloud to calculate the Euclidean distance offset of the key support points. The formula is: ΔD = (x t -x t-1 ) 2 +(y t -y t-1 ) 2 +(z t -z t-1 ) 2 When ΔD>5mm / 24h, a structural stability warning is triggered; by comparing the old and new scan results through algorithms, millimeter-level cracks or support tilting are accurately captured, and wall deformation is measured; a three-dimensional safety model is constructed by combining point cloud data: point cloud geometric constraints are used to perform super-resolution reconstruction of blurred images, and generative adversarial networks (GANs) are used to restore crack edge textures, improving the recognition rate of small cracks (width <2mm). LiDAR is responsible for building the skeleton of point cloud data, and cameras supplement the missing detailed location point cloud data. A global point cloud map of the alley is constructed based on SLAM technology, and real-time crack and displacement heat maps are superimposed to support VR inspection perspective; when the local deformation rate exceeds the threshold (e.g., 2mm / h), it is automatically marked as a red high-risk area to display areas where deformation is dangerous.

[0081] Step S73: Microscopic image analysis technology identifies the visual features of different ore types to achieve rapid identification and sorting of ore components; using 400-1000nm multispectral microscopic imaging technology, integrating ore reflectance, transmittance and fluorescence characteristics: establishing an optical feature database of common ores such as chalcopyrite (reflectance 47) and chalcocite (reflectance 29), and matching target minerals through spectral similarity formulas: When S>0.85, it is determined to be the same type of mineral; dynamic sorting of morphological parameters: calculate the aspect ratio of mineral particles (e.g., if the aspect ratio of quartz particles is >2.5, it is judged as flaky), contour complexity (fractal dimension >1.35, it is judged as irregular fragmented body) FD = log(N) / log(1 / r), where N is the minimum number of rectangles covering the contour, and r is the grid scaling ratio. Combine the mineral composition identification results (e.g., gold content >5g / t) and morphological parameters (particle size 0.5-2mm), the blowing intensity of the air valve is adjusted through the PID algorithm. It not only identifies the type of ore, but also automatically adjusts the wind force according to the shape - a gentle airflow is used for round gold ore, and a strong wind is used to remove flaky waste rock, achieving millimeter-level precision sorting.

[0082] This invention utilizes AI algorithms and deep learning technology to optimize the prototype AI recognition algorithm and achieve automatic and accurate identification of violations and dangerous situations in underground safe operations. Combined with automated equipment, it provides corresponding early warnings and hazard handling. This invention also offers a standard and complete AI model development process, along with a unique and innovative model training set and a trained AI recognition model. Furthermore, this invention provides a self-developed video monitoring platform and supporting mining equipment, which significantly improves the efficiency and accuracy of violation early warnings, reduces manual intervention, and maintains high-precision identification during long-term operation.

[0083] This invention achieves environmentally adaptive detection through optimization of the MAF-YOLO network model algorithm; coordinated adjustment of dynamic quantum convolution (DQConv) and electromagnetic compensation enables real-time adaptation under dust (0-500mg / m³) and electromagnetic interference (≤50μT) conditions, improving detection accuracy by 45.3% in high-dust scenarios and electromagnetic interference stability by 21.9%; through multi-physics joint decision-making, integrating visible light / shortwave infrared and coal seam stress data, the detection error for concealed targets (such as conveyor belt cracks) is ≤1.2px, the stress anomaly false negative rate is reduced to 2.1% (compared to 23.5% in traditional solutions), and the false alarm suppression ratio reaches 11.2:145; robustness under extreme conditions, maintaining an 89.7% detection success rate under combined interference such as 500mg / m³ dust + 5Hz vibration, exceeding the GB3836-2025 standard by 14.7%; through lightweight and efficient deployment, it is compatible with explosion-proof computing units, achieving an inference speed of 18.7ms (in normal environments) and an energy efficiency ratio of 3.6. TOPS / W is improved by 71% compared to the original solution; a dataset of images of illegal and dangerous situations in underground mining operations is created, and data augmentation operations are performed on the images in the dataset. Image processing methods such as flipping, cropping, and adjusting the hue, brightness, and resolution of the images are used to make the dataset more standardized, thereby improving the learning effect of the MAF-YOLO network model; based on the training model, a variety of underground identification and processing application scenarios are added, including dynamic monitoring of water and coal moisture content through texture feature recognition and multimodal data fusion, detection of displacement of roadway wall support structure combined with point cloud data to build a 3D safety model to monitor underground roadway deformation, and microscopic image analysis technology to identify the visual features of different ore types to achieve rapid identification and sorting of ore components.

[0084] The system constructed in this invention is a smart mine video monitoring platform, specifically including:

[0085] 1. System architecture and equipment setup.

[0086] To implement an AI-based visual recognition method for safe underground coal mine operations using computer vision, the system includes the following main hardware components: Figure 3 As shown.

[0087] Network switches are responsible for connecting various devices and ensuring stable data transmission, including servers, cameras, PLCs, alarm devices, fire extinguishing devices, etc.

[0088] Server: Used to process video image data, run deep learning models, generate alarm results, and connect to PLC to send corresponding alarm and fire extinguishing device activation signal commands.

[0089] Intrinsically safe mining cameras: installed in different areas of various mines to collect real-time image data streams related to underground safety operations.

[0090] PLC (Programmable Logic Controller): Controls the operation of alarm devices and fire extinguishing devices, receives control commands from the server, drives the movement and operation of alarm devices and fire extinguishing devices, and performs corresponding start-up and shutdown operations on mining equipment in the alarm area according to the configuration and actual situation.

[0091] Alarm devices: The intrinsically safe audible and visual alarm and the intrinsically safe camera for mining have built-in speakers and can provide flashing prompts and voice notifications according to instructions to warn personnel underground of corresponding violations.

[0092] Fire extinguishing equipment: including nozzles and gaseous extinguishing agents, which can adjust their position, spray water or gas according to instructions, and deal with dangerous situations such as smoke and fire in the mine.

[0093] Monitoring dashboard: Used to display system operating status, monitoring video, and alarm records in real time, providing a user interface for manual intervention and management. The specific monitoring process is as follows: Figure 2 As shown.

[0094] 2. Collection of information on violations and dangerous situations.

[0095] Equipment configuration: Install self-developed intrinsically safe mining cameras in frequently accessed areas and work locations throughout the coal mine to ensure clear capture of images of personnel working underground.

[0096] Operation process: The intrinsically safe mining camera collects images in real time of violations such as not wearing a safety helmet, smoking, hot work, and collapse of the storage chamber, as well as other dangerous situations.

[0097] 3. Dataset upload.

[0098] Equipment configuration: The Django framework and the YOLOv5 network model for AI image recognition in mining are running on the server.

[0099] Operation process: The server receives video streams from the camera and uploads corresponding images of violations, dangerous situations, and normal operations to the corresponding folders on the server using the Django framework. The server automatically starts the YOLOv5 network model for AI image recognition in the mine.

[0100] 4. Dataset training and labeling.

[0101] Equipment configuration: Start labelimg annotation on the server and begin annotating the safety job dataset.

[0102] Operation process: Use the YOLO format of the labelimg annotation software to annotate the processed images and photos, classifying them according to violation type and danger type.

[0103] The labeled training set is randomly selected from 80% of the total number of training set images, and the remaining 20% ​​is used as the test dataset for testing the model algorithm after training; the average precision mAP of the overall sample detection is obtained, and the current average detection precision P and average target recall R are obtained.

[0104] A new, original dataset of underground safety operation violations and hazardous situations was generated through dataset identification and processing. The types of violations included in this dataset are derived from common mine worker violations, hazardous situations, and equipment status detection. Violations include not wearing a safety helmet correctly, open flames underground, employees leaving their posts, employees sleeping on duty, and employees smoking. Hazardous situations include open flames underground, smoke detection, open flame detection, and mine collapse detection. Equipment status detection includes belt breakage detection, large coal lump detection, and coal flow quality detection.

[0105] 5. The video surveillance platform is started and used to identify datasets to monitor security operations.

[0106] Equipment configuration: The server runs the Spring Boot backend framework, the Vue frontend framework, and a MySQL database.

[0107] Operation process: The server starts up all the components required by the video monitoring platform, and after logging in with the preset user account, it imports the trained YOLO model file into the self-developed video monitoring platform to start monitoring services for various violations and dangerous situations, and begins to control whether the safety of personnel working underground is in accordance with regulations.

[0108] 6. Any personnel violations and subsequent handling.

[0109] Equipment configuration: The system uses a self-developed intrinsically safe mining camera and a UWB personnel positioning device to confirm whether underground personnel have violated regulations or accidentally entered warning areas.

[0110] Operation process: The system monitors employee work images through cameras, runs the target detection algorithm (YOLO) to confirm the personnel's work status, and sends PLC control commands when personnel violations occur, such as smoking underground, accidentally entering a restricted area underground, not wearing a safety helmet, or leaving their post.

[0111] PLC Instructions: The device trigger command is executed by the PLC-controlled alarm device to issue corresponding violation warnings to personnel in the current violation area; after the warning is completed, the video monitoring platform will continue to monitor whether the current area still has violations and dangerous situations, and will continue to warn and notify relevant management personnel for handling according to the configuration.

[0112] 7. Emergency response to underground hazards and subsequent handling.

[0113] Equipment configuration: Intrinsically safe mining cameras and mining smoke and fire sensors are used to confirm whether there are any violations or dangerous situations among personnel underground.

[0114] Operation process: The system monitors images of the underground mining area through cameras, runs the target detection algorithm (YOLO) to confirm the operation status of the area, and sends PLC control commands when dangerous situations occur, such as underground hot work, underground open flame, collapse, excessive smoke, etc.

[0115] PLC instructions: Equipment trigger commands are executed by the PLC-controlled fire extinguishing device. The PLC controls the fire extinguishing agent in the designated area to be delivered through the nozzles for fire extinguishing, or stops the operation of the corresponding equipment in the area for alarm processing, thereby improving the safety of underground operations.

[0116] 8. Monitoring and feedback.

[0117] Equipment configuration: The monitoring screen displays real-time information about the area, camera alarms, and camera monitoring results; it can display real-time images, area conditions, alarm status, and processing status, providing operators with information for supervision and management, such as... Figure 4 As shown.

[0118] Operation process: Through the monitoring screen, operators can view the real-time status of the monitoring platform, equipment status feedback, and alarm information, and handle and monitor violation alarms and danger alarms.

[0119] This invention presents an AI visual recognition method for underground coal mine safety operations based on YOLOv5. This method can be combined with specific hardware (such as network switches, cameras, PLCs, alarm devices, etc.) to achieve accurate alarm functions for underground safety operations. The system can automatically identify the type and location of violations by underground personnel and trigger corresponding alarm commands to execute efficient monitoring tasks. Finally, real-time monitoring of the entire process is achieved through a large monitoring screen, ensuring the accuracy and efficiency of the monitoring process.

[0120] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method of monitoring safety in coal mine underground operations, characterised by, The method comprises the following steps: obtaining a plurality of construction images of a person in a coal mine in a visible light band during a construction process in a preset time period, and a size data sequence of dust concentration, a magnetic field intensity data sequence and a vibration data sequence of the environment in the coal mine; when the current dust concentration in the environment in the coal mine is lower than a threshold value, a plurality of surface temperature distribution images of the target are extracted through infrared waves penetrating the dust; the plurality of construction images and the plurality of surface temperature distribution images, the size data sequence of the dust concentration, the magnetic field intensity data sequence and the vibration data sequence of the environment are input into a pre-trained work safety monitoring model, for each construction image, when feature extraction is performed, the dynamic quantum convolution layer dynamically extracts the target detail features and the target contour features in the construction image according to the dynamic size of the current dust concentration in the environment; for each surface temperature distribution image, the target temperature feature in the image is extracted; the offset of the target detail features and the target contour features is corrected by using the magnetic field intensity data of the environment through an electromagnetic field compensation channel; the corrected target detail features extracted in the visible light band are aligned and fused with the target temperature features extracted in the infrared band through a dual-band feature alignment layer, to obtain target detail features; the target detail features and the corrected target contour features are time-series aligned and fused through the vibration data of the environment by a vibration time-series alignment layer, to obtain a feature map corresponding to each image output by the work safety monitoring model; wherein the target detail features are the detail texture features of the target extracted from the corresponding construction image by the dynamic quantum convolution layer when the current dust concentration in the environment is lower than the threshold value; the target contour features are the edge contour features of the target extracted from the corresponding construction image by the dynamic quantum convolution layer when the current dust concentration in the environment is higher than the threshold value; the work safety monitoring result is generated by identifying the work construction process in each image corresponding to the feature map.

2. The coal mine underground operation safety monitoring method according to claim 1, characterized in that, The work safety monitoring model is obtained by: training the improved MAF-YOLO network using the plurality of construction images to obtain the work safety monitoring model; wherein the improved MAF-YOLO network is obtained by integrating a dynamic quantum convolution layer and an electromagnetic field compensation channel into a feature extraction module of the MAF-YOLO network, and integrating a dual-band feature alignment layer and a vibration time-series alignment layer into a feature fusion module of the MAF-YOLO network.

3. The coal mine underground operation safety monitoring method according to claim 1, characterized in that, When the dynamic quantum convolution layer extracts features from the construction image, the dynamic quantum convolution layer DQConv adjusts the weight of the convolution kernel of the feature extraction module according to the dust concentration in the coal mine, which is represented as: ； Where: C d represents the dust concentration; a = 0.25; b = 0.03; W dq represents the dynamically adjusted convolution kernel weight; W base represents the basic convolution kernel weight; The dynamic quantum convolution layer DQConv monitors the dust concentration in the coal mine by dynamically amplifying the edge response weight; when the dust concentration is lower than the threshold value, the edge response weight is enhanced during feature extraction to extract the target detail features in the construction image; when the dust concentration is higher than the threshold value, the edge contour feature extraction is enhanced during feature extraction to extract the target contour features in the construction image.

4. The coal mine underground operation safety monitoring method according to claim 1, characterized in that, When the electromagnetic field compensation channel corrects the offset of the target detail feature and the target contour feature, the feature offset of the target detail feature and the target contour feature in the image is corrected by the magnetometer, and is expressed as: ； where: B rms represents the electromagnetic field intensity; B max = 50 μΤ; F corr represents the corrected eigenvalue; F raw represents the original eigenvalue; γ represents the compensation coefficient.

5. The coal mine underground operation safety monitoring method according to claim 1, characterized in that, The vibration time sequence alignment layer aligns the target detail feature and the corrected target contour feature in time sequence through the vibration data of the environment, and includes the following steps: The vibration time sequence alignment layer measures the microseismic data in the current environment through the three-axis microseismic sensor, synchronizes the data collected after the distortion caused by the picture shaking with the picture collected by the time-synchronized camera based on the time, calibrates the camera clock signal through synchronization, and aligns the target detail feature and the corrected target contour feature in time sequence.

6. The coal mine underground operation safety monitoring method according to claim 1, characterized in that, After the feature map corresponding to each image is obtained, the target feature in the feature map is corrected by a Tri-Head and a multi-task loss function, including the following steps: The Tri-Head dynamically corrects the target confidence in the feature map by combining the XYZ axis stress data, and dynamically adjusts the detection area by real-time sensing the pressure changes of the coal mine roof, coal wall and floor through the pressure data, microseismic sensor and infrared thermal imager; The multi-task loss function identifies the missing features in the feature map by simulating the crack scanner, metal fatigue monitor and double-camera collation system.

7. A safety monitoring system for coal mine underground operations, characterised in that, It includes: The data acquisition module is used to acquire a plurality of construction images of the coal mine personnel in the construction process in a preset time period under the visible light wave band, and a size data sequence of the dust concentration in the coal mine environment, a magnetic field intensity data sequence of the environment and a vibration data sequence of the environment; When the dust concentration in the current environment of the coal mine is lower than the threshold, the infrared wave band is used to penetrate the dust to extract a plurality of surface temperature distribution images of the target; The detection module is used to input the plurality of construction images and the plurality of surface temperature distribution images, and the size data sequence of the dust concentration in the environment, the magnetic field intensity data sequence of the environment and the vibration data sequence of the environment into the pre-trained operation safety monitoring model, and when extracting features for each construction image, the dynamic quantum convolution layer dynamically extracts the target detail feature and the target contour feature in the construction image according to the dynamic size of the current dust concentration in the environment; for each surface temperature distribution image, the target temperature feature in the image is extracted; The electromagnetic field compensation channel corrects the offset of the target detail feature and the target contour feature by using the magnetic field intensity data of the environment; the double-wave band feature alignment layer aligns and fuses the corrected target detail feature extracted under the visible light wave band with the target temperature feature extracted under the infrared wave band, to obtain the target detail feature; The vibration time sequence alignment layer aligns and fuses the target detail feature and the corrected target contour feature in time sequence by using the vibration data of the environment, to obtain the feature map corresponding to each image output by the operation safety monitoring model. Wherein, the target detail feature is a detail texture feature of the target extracted from the corresponding construction image by the dynamic quantum convolution layer when the current dust environment concentration is lower than the threshold; the target contour feature is an edge contour feature of the target extracted from the corresponding construction image by the dynamic quantum convolution layer when the current dust environment concentration is higher than the threshold; The recognition module is configured to recognize the construction process in the feature map corresponding to each image, and generate the job safety monitoring result.

8. An electronic device, comprising: Comprise: a memory and a processor; the memory is configured to store a computer program; the processor is configured to execute the computer program stored in the memory to realize the steps of the coal mine underground job safety monitoring method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, a computer program is stored, and the computer program is executed by the processor to realize the steps of the coal mine underground job safety monitoring method according to any one of claims 1-6.

Images