Roadway surrounding rock roof separation polling and early warning system based on multi-source information fusion

Abstract

The invention, which belongs to the technical field of mine safety monitoring, provides a multi-source information fusion-based roadway surrounding rock roof separation tour-inspection early-warning system comprising a data acquisition module, an edge calculation module, a cloud analysis module and an early-warning feedback module. The data acquisition module acquires multi-source data of surrounding rock of a coal mine tunnel in an automatic mode; the edge calculation module performs spatio-temporal data synchronization on the collected multi-source data and extracts data features; the cloud analysis module performs feature weighted fusion based on the multi-source data to construct a risk early warning model; and the early warning feedback module triggers graded early warning according to prediction data of the risk early warning model, and provides visual feedback, so that a set of automatic, full-coverage and multi-source data fused roadway roof separation monitoring system is constructed.

Description

Technical Field

[0001] This invention relates to the field of mine safety monitoring technology, specifically to a roadway surrounding rock roof delamination inspection and early warning system and method based on multi-source information fusion. Background Technology

[0002] Roof delamination and collapse in coal mine roadways are phenomena caused by the separation and fracturing of roof strata or between roof strata and coal seams, leading to roof structural instability and subsequent collapse. The hazards are multidimensional, affecting personnel safety, production activities, and the social environment. Minor consequences include equipment damage and production interruption, while severe consequences include casualties. Currently, monitoring of roof delamination in roadways mainly relies on the following technologies:

[0003] (1) Mechanical delamination meter: The scale data is read manually at regular intervals. The method is simple and can only monitor a certain point or line of the surrounding rock. The manual reading has large errors, low efficiency and no real-time early warning.

[0004] (2) Fixed electronic sensors: such as fiber optic grating sensors, can only monitor fixed points and cannot cover the entire tunnel;

[0005] (3) Handheld laser rangefinder: requires manual operation, poses safety risks and has discrete data.

[0006] The aforementioned technologies have significant drawbacks. Fixed sensors monitor only a single point, resulting in insufficient coverage and an inability to capture the overall deformation trend of the roof. Manual inspections are time-consuming and lack real-time performance, making it difficult to detect sudden delamination in a timely manner. Data from different sensors is not fused and analyzed, resulting in isolated data and difficulty in accurately assessing risk levels. Furthermore, the level of intelligence is low, lacking the ability to predict the development trend of delamination. Later, technologies using radar waves to detect surrounding rock structures emerged, such as the Chinese invention application No. 202211156727.X, "A Real-Time Online Monitoring System for Dynamic Deformation of Roadway Surrounding Rock Based on Millimeter-Wave Radar." This system can achieve real-time monitoring, analysis, and early warning of the specific deformation location and amount of roadway surrounding rock. However, it only uses millimeter-wave radar, resulting in a single detection method and a lack of multi-source data fusion. Additionally, the fixed observation point setup leads to low coverage, easily causing detection omissions and potential safety hazards. Summary of the Invention

[0007] The technical problem to be solved by this invention is how to construct an automated, comprehensive, multi-source data fusion roadway roof delamination monitoring system.

[0008] The present invention solves the above-mentioned technical problems through the following technical means:

[0009] This invention provides a multi-source information fusion-based inspection and early warning system for roof delamination in roadways, comprising a data acquisition module, an edge computing module, a cloud analysis module, and an early warning feedback module. The data acquisition module automatically collects multi-source data on the surrounding rock of coal mine roadways. The edge computing module performs spatiotemporal data synchronization on the collected multi-source data and extracts data features. The cloud analysis module performs feature weighted fusion based on the multi-source data to construct a risk early warning model. The early warning feedback module triggers tiered early warnings based on the predicted data from the risk early warning model and provides visual feedback.

[0010] Furthermore, the data acquisition module includes a monorail system (5) and an inspection robot (1) slidably connected thereon; the inspection robot (1) can move along the track on the monorail system (5); the inspection robot (1) is equipped with a ground-penetrating radar (2), a lidar (3) and a multi-mode camera (4).

[0011] Furthermore, the ground-penetrating radar (2) is used to detect the delamination structure of the surrounding rock in coal mine roadways. The ground-penetrating radar (2) emits high-frequency electromagnetic pulses into the surrounding rock. The electromagnetic waves are reflected at the rock stratum interface. The receiving antenna captures the reflected waves, records the reflection time, amplitude, and waveform characteristics, and generates a B-scan image. Based on the propagation speed of the emitted electromagnetic waves in the rock stratum, the reflection time is converted into the delamination depth h, as shown in the following formula:

[0012]

[0013] Where c is the speed of light, ε r is the relative permittivity, and t is the reflection time recorded by the ground-penetrating radar.

[0014] Furthermore, the lidar (3) is used for surface deformation scanning of the surrounding rock in coal mine roadways to generate a three-dimensional point cloud; a laser beam is emitted at a frequency of 10Hz, and the three-dimensional coordinates of the roadway roof and sidewall surfaces are obtained through the emission time ranging principle; the current point cloud is registered with historical data through the iterative nearest point algorithm to calculate the surface settlement, as shown in the following formula:

[0015]

[0016] Where, x i y i z i x represents the coordinates of the current point cloud in the system's three-dimensional coordinate system. j y j z j These are the coordinates of the historical point cloud in the system's three-dimensional coordinate system.

[0017] Furthermore, the multi-mode camera (4) is used to detect cracks in the surrounding rock of coal mine roadways. The camera captures high-definition images of the surrounding rock surface, capturing the length, width, and direction of crack expansion.

[0018] Furthermore, the edge computing module performs spatiotemporal data synchronization on the collected multi-source data, including:

[0019] (1) Add a unified timestamp to the data collected by ground penetrating radar (2), lidar (3) and multi-mode camera (4), and use Kalman filtering to eliminate timestamp deviation;

[0020] (2) Using the pose data output by SLAM, the delamination position detected by the ground penetrating radar (2) is mapped to the surface three-dimensional point cloud coordinate system obtained by the lidar (3), as shown in the following formula:

[0021] P GPR =T SLAM ·P local

[0022] Among them, P GPR For the global coordinates of the ground penetrating radar, P local T represents the local coordinates of the ground-penetrating radar. SLAM This is the pose transformation matrix output by SLAM.

[0023] Furthermore, the edge computing module extracts data features, including:

[0024] (1) Extract the maximum amplitude of reflected waves from the B-scan image, identify the rock layer interface, and determine the delamination thickness by the time window threshold method;

[0025] (2) Use the RANSAC algorithm to segment the planar region in the 3D point cloud and calculate the regional settlement.

[0026] (3) Use the U-Net model to perform semantic segmentation on crack images and quantify crack length and propagation rate.

[0027] Furthermore, the cloud-based analytics module constructs a risk warning model, including:

[0028] (1) Construct a risk index, which is the internal delamination amount D. internal Weight α = 0.5; Surface settlement D surface Weight β = 0.3; Crack propagation rate C crack The weight γ = 0.2;

[0029] (2) Deep learning prediction: Input historical data into the Long Short-Term Memory (LSTM) network model to predict the amount of delamination, surface subsidence and crack propagation rate in the next 6 hours.

[0030] (3) Construct a risk early warning model, as shown in the following formula:

[0031]

[0032] in, This represents the typical maximum value of the delamination amount; This represents the typical maximum value of surface settlement. This represents the typical maximum crack propagation rate.

[0033] Furthermore, the early warning feedback module triggers tiered early warnings, including:

[0034] (1) Level 1 warning: When R < 0.4, the platform records and generates an inspection report;

[0035] (2) Level II early warning: When 0.4≤R<0.7, the audible and visual alarm in the roadway is activated and the information is pushed to the management personnel terminal;

[0036] (3) Level 3 warning: When R≥0.7, an emergency shutdown command is automatically triggered to shut down equipment in the dangerous area and activate the personnel evacuation plan.

[0037] Furthermore, the early warning feedback module provides visual feedback, specifically:

[0038] The cloud platform generates a digital twin model of the surrounding rock of the tunnel, which is overlaid and displayed in the form of a heat map, supporting real-time viewing of historical data curves and predicted trends on multiple terminals.

[0039] The advantages of this invention are:

[0040] (1) It has full-dimensional detection capability and collaborative perception of the interior and surface of the surrounding rock. By using ground penetrating radar to detect the delamination inside the surrounding rock (depth 0.5-5 meters), and combining lidar and multispectral camera to monitor surface deformation and cracks, it can achieve full-dimensional coverage detection of the surrounding rock "from the inside to the outside". By combining inspection robots (dynamic coverage) and fixed sensors (static high precision), it solves the blind spot problem of traditional single-dimensional monitoring technology.

[0041] (2) Improved data fusion accuracy: The spatiotemporal alignment technology of cross-modal data (electromagnetic wave reflection, laser point cloud, image) enables the matching and correction of underground delamination location and surface deformation area, and the comprehensive detection accuracy is better than the detection results of a single sensor.

[0042] (3) High precision and real-time performance: millimeter-level detection, dynamic weight adjustment, automatic optimization of risk model weights based on roadway lithology (such as mudstone and sandstone) to reduce false alarm rate.

[0043] (4) Automation replaces manual labor, with a high degree of mechanization and informatization, reducing operation and maintenance costs, improving efficiency of single full-lane inspection, and saving labor costs. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the structure of the roadway surrounding rock roof delamination inspection and early warning system based on multi-source information fusion according to an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of the data acquisition module structure in an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] This embodiment provides a multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system, such as... Figure 1 As shown, it includes a data acquisition module, an edge computing module, a cloud analysis module, and an early warning feedback module. The data acquisition module automatically collects multi-source data of the surrounding rock in coal mine roadways. The edge computing module performs spatiotemporal data synchronization on the collected multi-source data and extracts data features. The cloud analysis module performs feature weighted fusion based on the multi-source data to construct a risk early warning model. The early warning feedback module triggers tiered early warnings based on the predicted data from the risk early warning model and provides visual feedback.

[0048] like Figure 2 As shown, the data acquisition module includes a monorail system 5 and an inspection robot 1 slidably connected to it; the inspection robot 1 can move along the track on the monorail system 5; the inspection robot 1 is equipped with a ground penetrating radar 2, a lidar 3 and a multi-mode camera 4.

[0049] Ground-penetrating radar 2 is installed at the bottom of the chassis, using a pulse antenna with a center frequency of 1 GHz, a detection depth of 0.5–5 meters, and a vertical resolution of ≤1 cm. Ground-penetrating radar 2 is used to detect the delamination structure of surrounding rock in coal mine roadways. It emits high-frequency electromagnetic pulses into the surrounding rock; the electromagnetic waves are reflected at the rock interface, and the receiving antenna captures the reflected waves, recording the reflection time, amplitude, and waveform characteristics to generate a B-scan image. Based on the propagation speed of the emitted electromagnetic waves in the rock strata, the reflection time is converted into the delamination depth h, as shown in the following formula:

[0050]

[0051] Where c is the speed of light, ε r is the relative permittivity, and t is the reflection time recorded by the ground-penetrating radar.

[0052] The lidar uses a 16-line LiDAR (model Velodyne VLP-16) with a scanning frequency of 10Hz, a horizontal field of view of 360°, and a ranging accuracy of ±1mm. LiDAR 3 is used for scanning the deformation of the surrounding rock surface in coal mine roadways, generating a 3D point cloud. It emits a laser beam at a frequency of 10Hz and obtains the 3D coordinates of the roadway roof and sidewall surfaces using the emission time ranging principle. The current point cloud is registered with historical data using an iterative nearest-point algorithm to calculate the surface settlement, as shown in the following formula:

[0053]

[0054] Where, x i y i z i x represents the coordinates of the current point cloud in the system's three-dimensional coordinate system. j y j z j These are the coordinates of the historical point cloud in the system's three-dimensional coordinate system.

[0055] The number of iterations is ≤100, and the convergence threshold is 0.001m.

[0056] The multi-mode camera integrates a 20-megapixel visible light camera (SONY IMX586) and a 640×480 resolution infrared thermal imager (FLIR Lepton 3.5), operating at 30fps. Multi-mode camera 4 is used to detect cracks in the surrounding rock of coal mine roadways, capturing high-resolution images of the rock surface and determining the length, width, and direction of crack propagation. The infrared thermal imager detects the temperature distribution on the rock surface and identifies areas of water seepage (temperature anomalies with a temperature difference greater than 2℃).

[0057] The edge computing module performs spatiotemporal data synchronization on the collected multi-source data, including:

[0058] (1) Add a unified timestamp (accuracy ±10ms) to the data collected by ground penetrating radar 2, lidar 3 and multi-mode camera 4, and use Kalman filtering to eliminate timestamp deviation, with synchronization error <10ms;

[0059] (2) Using the pose data output by SLAM, the delamination position detected by the ground penetrating radar 2 is mapped to the surface three-dimensional point cloud coordinate system obtained by the lidar 3 (error less than 1cm), as shown in the following formula:

[0060] P GPR =T SLAM ·P local

[0061] Among them, P GPR For the global coordinates of the ground penetrating radar, P local T represents the local coordinates of the ground-penetrating radar. SLAMThis is the pose transformation matrix output by SLAM.

[0062] The edge computing module extracts data features from the collected multi-source data, including:

[0063] (1) Extract the maximum amplitude of reflected waves from the B-scan image, identify the rock layer interface, and determine the delamination thickness by the time window threshold method; if the thickness corresponding to the time difference of adjacent reflected waves is greater than 50 mm, it is determined to be a risky delamination and defined as abnormal data.

[0064] (2) Use the RANSAC algorithm to segment the planar region in the three-dimensional point cloud and calculate the regional settlement. If the settlement per hour is greater than 10 mm / h, it is judged as high-risk settlement and defined as abnormal data.

[0065] (3) The U-Net model is used to perform semantic segmentation on crack images to quantify crack length and propagation rate. Infrared images can also be used to locate temperature anomaly areas and assess the risk of water seepage.

[0066] The cloud-based analytics module performs feature-weighted fusion of multi-source data to construct a risk warning model, including:

[0067] (1) Construct risk indices, namely, internal delamination amount Dinternal, weight α=0.5; surface settlement amount Dsurface, weight β=0.3; crack propagation rate Ccrack, weight γ=0.2;

[0068] (2) Deep learning prediction: Input historical data into the Long Short-Term Memory (LSTM) network model to predict the amount of delamination, surface subsidence and crack propagation rate in the next 6 hours.

[0069] (3) Construct a risk early warning model, as shown in the following formula:

[0070]

[0071] in, This represents the typical maximum value of the delamination amount; This represents the typical maximum value of surface settlement. This represents the typical maximum crack propagation rate.

[0072] Based on practical experience, the typical reference range for the above parameters is shown in the table below:

[0073] symbol Physical meaning unit Typical values <![CDATA[D internal ]]> Delamination within the surrounding rock mm 0~50mm <![CDATA[D surface ]]> Surface settlement of surrounding rock mm 0~10mm <![CDATA[C crack ]]> Crack propagation rate mm / h 0~5mm / h

[0074] The early warning feedback module triggers tiered early warnings based on the prediction data from the risk early warning model, including:

[0075] (1) Level 1 warning (low risk): When R < 0.4, the platform records and generates an inspection report;

[0076] (2) Level II early warning (medium risk): When 0.4≤R<0.7, activate the audible and visual alarm in the roadway and push the information to the management personnel terminal;

[0077] (3) Level 3 warning (high risk): When R≥0.7, an emergency shutdown command is automatically triggered to shut down equipment in the dangerous area and activate the personnel evacuation plan.

[0078] The early warning feedback module can also provide visual feedback, specifically:

[0079] The cloud platform generates a digital twin model of the surrounding rock of the tunnel, which is overlaid in the form of a heat map and supports real-time viewing of historical data curves and predicted trends on multiple terminals (PC, mobile).

[0080] Based on the above system, the working principle of the system in this embodiment is as follows:

[0081] Step 1: Start the inspection robot and move it along the tunnel at a speed of 0.5m / s, while simultaneously activating the ground penetrating radar, lidar, and multi-mode camera;

[0082] Step 2: Collect data in real time and transmit it to the edge computing module to perform noise filtering (e.g., wavelet denoising for ground penetrating radar data);

[0083] Step 3: The edge computing module extracts features (delamination amount, settlement amount, crack propagation rate). If the feature data is determined to be abnormal (such as the thickness corresponding to the time difference between adjacent reflected waves is greater than 50 mm, settlement amount is greater than 10 mm / h), it is immediately uploaded to the cloud analysis module.

[0084] Step 4: The cloud-based analysis module integrates multi-source data to generate a risk index and constructs a risk early warning model. If the index is greater than 0.7 (an empirical value based on historical data statistical analysis of the Huainan mining area), a level-three early warning is triggered and the emergency response system is activated.

[0085] The technical advantages of the multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system provided in this embodiment compared with existing technologies are shown in the following table:

[0086]

[0087] As can be seen from the table above, the multi-source information fusion roadway surrounding rock roof delamination inspection and early warning system provided in this embodiment has the following advantages:

[0088] (1) Full-dimensional detection capability, co-sensing of the interior and surface of the surrounding rock: By using ground-penetrating radar (GPR) to detect the delamination inside the surrounding rock (depth 0.5 to 5 meters), and combining lidar (LiDAR) and multispectral camera to monitor surface deformation and cracks, the full-dimensional coverage detection of the surrounding rock "from the inside to the outside" is realized, solving the blind spot problem of traditional single-dimensional monitoring technology.

[0089] (2) Improved data fusion accuracy: The spatiotemporal alignment technology of cross-modal data (electromagnetic wave reflection, laser point cloud, image) enables the matching and correction of underground delamination location and surface deformation area, and the comprehensive detection accuracy is better than the detection results of a single sensor.

[0090] (3) High precision and real-time performance: millimeter-level detection, dynamic weight adjustment, automatic optimization of risk model weights based on roadway lithology (such as mudstone and sandstone) to reduce false alarm rate.

[0091] (4) Automation replaces manual labor, with a high degree of mechanization and informatization, reducing operation and maintenance costs, improving efficiency of single full-lane inspection, and saving labor costs.

[0092] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system, characterized in that: It includes a data acquisition module, an edge computing module, a cloud analysis module, and an early warning feedback module; The data acquisition module uses an automated method to collect multi-source data on the surrounding rock of coal mine roadways; the edge computing module performs spatiotemporal data synchronization on the collected multi-source data and extracts data features; The cloud-based analytics module performs feature-weighted fusion based on multi-source data to build a risk warning model; The early warning feedback module triggers tiered early warnings based on the prediction data from the risk early warning model and provides visual feedback.

2. The multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system according to claim 1, characterized in that: The data acquisition module includes a monorail system (5) and an inspection robot (1) that slides on it; the inspection robot (1) can move along the track on the monorail system (5); the inspection robot (1) is equipped with a ground-penetrating radar (2), a lidar (3) and a multi-mode camera (4).

3. The multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system according to claim 2, characterized in that: Ground-penetrating radar (2) is used to detect the delamination structure of surrounding rock in coal mine roadways. The ground-penetrating radar (2) emits high-frequency electromagnetic pulses into the surrounding rock. The electromagnetic waves are reflected at the rock interface. The receiving antenna captures the reflected waves, records the reflection time, amplitude, and waveform characteristics, and generates a B-scan image. Based on the propagation speed of the emitted electromagnetic waves in the rock layer, the reflection time is converted into the delamination depth h, as shown in the following formula: Where c is the speed of light, ε r is the relative permittivity, and t is the reflection time recorded by the ground-penetrating radar.

4. The multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system according to claim 3, characterized in that: The lidar (3) is used to scan the deformation of the surrounding rock surface in coal mine roadways and generate a three-dimensional point cloud; it emits a laser beam at a frequency of 10Hz and obtains the three-dimensional coordinates of the roadway roof and sidewall surfaces through the emission time ranging principle; it registers the current point cloud with historical data through the iterative nearest point algorithm and calculates the surface settlement, as shown in the following formula: Where, x i y i z i x represents the coordinates of the current point cloud in the system's three-dimensional coordinate system. j y j z j These are the coordinates of the historical point cloud in the system's three-dimensional coordinate system.

5. The multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system according to claim 4, characterized in that: The multi-mode camera (4) is used to detect cracks in the surrounding rock of coal mine roadways. It captures high-definition images of the surrounding rock surface and measures the length, width, and direction of crack expansion.

6. The multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system according to claim 5, characterized in that: The edge computing module performs spatiotemporal data synchronization on the collected multi-source data, including: (1) Add a unified timestamp to the data collected by ground penetrating radar (2), lidar (3) and multi-mode camera (4), and use Kalman filtering to eliminate timestamp deviation; (2) Using the pose data output by SLAM, the delamination position detected by the ground penetrating radar (2) is mapped to the surface three-dimensional point cloud coordinate system obtained by the lidar (3), as shown in the following formula: P GPR =T SLAM ·P local Among them, P GPR For the global coordinates of the ground penetrating radar, P local T represents the local coordinates of the ground-penetrating radar. SLAM This is the pose transformation matrix output by SLAM.

7. The multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system according to claim 6, characterized in that: Extracting data features, including: (1) Extract the maximum amplitude of reflected waves from the B-scan image, identify the rock layer interface, and determine the delamination thickness by the time window threshold method; (2) Use the RANSAC algorithm to segment the planar region in the 3D point cloud and calculate the regional settlement. (3) Use the U-Net model to perform semantic segmentation on crack images and quantify crack length and propagation rate.

8. The multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system according to claim 1, characterized in that: Construct a risk early warning model, including: (1) Construct a risk index, which is the internal delamination amount D. internal Weight α = 0.5; Surface settlement D surface Weight β = 0.3; Crack propagation rate C crack The weight γ = 0.2; (2) Deep learning prediction: Input historical data into the Long Short-Term Memory (LSTM) network model to predict the delamination amount, surface subsidence amount and crack propagation rate in the next 6 hours; (3) Construct a risk early warning model, as shown in the following formula: in, This represents the typical maximum value of the delamination amount; This represents the typical maximum value of surface settlement. This represents the typical maximum crack propagation rate.

9. The multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system according to claim 8, characterized in that: Triggering tiered alerts includes: (1) Level 1 warning: When R < 0.4, the platform records and generates an inspection report; (2) Level II early warning: When 0.4≤R<0.7, the audible and visual alarm in the roadway is activated and the information is pushed to the management personnel terminal; (3) Level 3 warning: When R≥0.7, an emergency shutdown command is automatically triggered to shut down equipment in the dangerous area and activate the personnel evacuation plan.

10. The multi-source information fusion-based roadway surrounding rock roof delamination inspection and early warning system according to claim 1, characterized in that: Provide visual feedback, specifically: The cloud platform generates a digital twin model of the surrounding rock of the tunnel, which is overlaid and displayed in the form of a heat map, supporting real-time viewing of historical data curves and predicted trends on multiple terminals.

Images