Multi-modal scheduling method and system for goaf grouting resources

Abstract

The invention relates to the technical field of mine safety and intelligent management, and provides a goaf grouting resource multi-modal scheduling method and system, which realize multi-dimensional perception of goaf fire through multi-modal data fusion, and improve comprehensiveness and accuracy of anomaly recognition. A Transform model and an improved evidence theory are introduced, so that the correlation analysis capability of cross-modal data is enhanced, and the false alarm rate is effectively reduced; dynamic deduction is combined with a physical model and deep reinforcement learning, so that the dynamic adaptability of fire development prediction is improved; path planning is fused with a heuristic algorithm and an AR visualization technology, and intelligent dynamic optimization of an escape path is achieved; a four-level response mechanism and an intelligent resource scheduling strategy are adopted, so that the emergency disposal period is shortened, and the grouting material matching and equipment scheduling efficiency is improved; the anti-interference capability and the self-diagnosis function are enhanced by the hardware design of the system, and the stability of data acquisition in a complex environment is guaranteed; the lightweight model and parallel computing technology improve the efficiency of real-time analysis and multi-fire-zone deduction.

Description

Technical Field

[0001] This invention relates to the field of mine safety and intelligent management technology, specifically to a multimodal scheduling method and system for grouting resources in goaf areas. Background Technology

[0002] In the field of mine goaf safety, existing technologies suffer from several bottlenecks, including insufficient multi-dimensional monitoring capabilities, low accuracy of data fusion analysis, lagging fire dynamic simulation, and poor emergency response efficiency. Traditional single-parameter monitoring (such as relying solely on temperature or gas concentration) cannot comprehensively capture the characteristics of fire evolution. The lack of cross-modal data correlation analysis leads to a high false alarm rate in anomaly identification, making it difficult to accurately determine the fire stage. Physical models are not dynamically matched with real-time operating conditions, lacking a data-driven intelligent decision-making mechanism. Grouting equipment scheduling and ventilation control rely on manual experience, resulting in slow response and insufficient path optimization. Emergency escape systems do not integrate real-time fire spread and personnel location data, path planning is statically lagging, and communication networks in complex tunnel environments suffer from high transmission delays and weak anti-interference capabilities, failing to meet the real-time and reliability requirements of emergency response.

[0003] Therefore, a multimodal scheduling method and system for grouting resources in goaf areas is proposed. By deploying a multimodal sensing network of infrared thermal imaging, gas sensors, and microseismic monitoring, the limitations of single monitoring are addressed, enabling simultaneous acquisition of multidimensional data such as temperature field, gas diffusion, and rock fracture. Cross-modal data fusion is performed using the Transformer model and improved DS evidence theory to improve the accuracy of early fire identification. Boundary conditions and fire source parameters are dynamically updated through coupled deduction of physical models and deep reinforcement learning (DRL), enabling real-time fire prediction and coordinated control of grouting and ventilation. Dynamic escape paths are generated based on dynamic graph models and AR visualization technology to solve the problem of lagging traditional escape guidance. Summary of the Invention

[0004] Technical problems to be solved

[0005] To address the shortcomings of existing technologies, this invention provides a multimodal scheduling method and system for grouting resources in goaf areas.

[0006] Technical solution

[0007] To achieve the above-mentioned objective, the present invention provides the following technical solution: a multi-modal scheduling method for grouting resources in goaf areas, comprising the following steps:

[0008] S1 Multimodal Data Acquisition and Preprocessing: An infrared thermal imaging monitoring network was deployed, employing distributed, uncooled infrared thermal imagers; a multi-parameter gas sensor network was constructed to detect CO, CH4, and O2 concentrations in real time; a microseismic monitoring network was deployed, using a 24-channel microseismic monitoring system, with sensors anchored at 50-meter intervals to the sides and roof of the tunnel, forming a triangular positioning array; non-uniformity correction was applied to the infrared thermal imaging data; gas concentration data was dynamically smoothed using an adaptive Kalman filter; and rock fracture feature vectors in the 8-16Hz frequency band were extracted from the microseismic signals through db4 wavelet decomposition.

[0009] S2 Multimodal Data Fusion and Feature Extraction: Feature-level fusion is performed based on the Transformer model. The input consists of a normalized temperature sequence, a normalized gas concentration sequence, and microseismic waveforms padded to 512 points. Cross-modal correlations are captured through a 6-layer encoder structure, and a 128-dimensional feature vector is output. Decision-level fusion is performed using an improved DS evidence theory. Three evidence sources are defined: temperature anomaly, gas anomaly, and microseismic anomaly. Conflicting evidence is processed through a discount factor. The fire determination threshold after fusion is 95%.

[0010] S3 Fire Dynamic Simulation and Path Planning: Couples the Navier-Stokes equation physical model with a deep reinforcement learning agent to dynamically adjust the opening of ventilation valves and the flow rate of grouting pumps; generates three-dimensional escape paths based on dynamic environment models, uses an improved Dijkstra algorithm to prioritize short-distance upwind paths, and visualizes them in real time through an AR terminal.

[0011] S4 Multimodal Chain Response and Resource Scheduling: Triggers a four-level response mechanism; intelligent matching of grouting materials, selecting materials based on fire zone characteristics; and uses an ant colony algorithm with a time window to plan the grouting vehicle path.

[0012] Preferably, the infrared thermal imager is arranged every 20 meters on the roof of the goaf, equipped with a pan-tilt unit to achieve 0-360° rotation monitoring, and has a built-in blackbody calibration module for daily automatic calibration; the infrared thermal imager has a resolution of 640×480, a temperature measurement accuracy of ±0.5℃, a response wavelength of 7.5-13μm, and achieves a noise standard deviation of ≤0.3℃ through a non-uniformity correction algorithm of the reference radiation source.

[0013] Preferably, the gas sensor network has a grid layout with a horizontal spacing of 30 meters, and sensor nodes are deployed in layers every 10 meters in the vertical direction at the air inlet, air outlet, and central part of the goaf area, forming a 6×6×5 three-dimensional monitoring array. Gas concentration filtering adopts the state equation X. k =FX k-1 +W k-1 With observation equation Z k =HX k +V kAn adaptive Kalman filter is used, where the state vector includes concentration and rate of change, and the observation noise covariance is dynamically updated.

[0014] Preferably, in the improved DS evidence theory, the basic probability assignment maps the eigenvector components through the Softmax function, conflict evidence processing employs the Yager improvement method, and inter-evidence distance is introduced. As a discount factor.

[0015] Preferably, the reward function of the DRL agent is R = -C CO ×0.6-T max ×0.3-t×0.1, where the weight of CO exceeding the standard area is 0.6, the weight of the highest temperature in the fire zone is 0.3, and the weight of the treatment time is 0.1. The PPO algorithm is used for training, and the capacity of the experience playback pool is 1e6.

[0016] A multimodal scheduling system for grouting resources in goaf areas is used to implement the aforementioned multimodal scheduling method for grouting resources in goaf areas. The system includes a multimodal perception module, an intelligent analysis module, a dynamic deduction and visualization module, and a resource scheduling and execution module.

[0017] Multimodal sensing module: includes an infrared thermal imager, a gas sensor and a micro-vibration monitoring node, and adopts an IP68 explosion-proof shell, temperature compensation circuit and dual power supply redundancy design;

[0018] Intelligent analysis module: It adopts an edge-cloud collaborative architecture, with NVIDIA Jetson AGX Orin nodes deployed at the edge layer and a GPU server cluster configured at the cloud layer, supporting lightweight Transformer model and parallel acceleration of the inference engine;

[0019] Dynamic simulation and visualization module: Based on MPI, multi-physics coupling simulation is realized, the command center large screen renders a three-dimensional situation map in real time, and the miner terminal supports voice navigation and gesture interaction;

[0020] Resource scheduling and execution module: Siemens S7-1500 PLC is used to control the grouting pump flow and ventilation valves. After PID parameter tuning, the adjustment time is <15s, the overshoot is <3%, and a four-level response mechanism is designed.

[0021] Preferably, the communication network of the multimodal sensing module adopts a hybrid networking of industrial Ethernet and LoRaWAN, and the data transmission is encrypted with AES-256, with the key automatically updated every 24 hours.

[0022] Preferably, the AR terminal of the visualization module is a HoloLens2 device, which constructs a three-dimensional map using SLAM technology and dynamically marks high-temperature zones, smoke diffusion fronts, and escape routes.

[0023] Preferably, the grouting material matching is based on a support vector machine classifier, and the input features include fire zone temperature, crack development degree, and CO concentration gradient.

[0024] Preferably, the triggering conditions for the four-level response mechanism are as follows: Monitoring level: temperature > 35℃ or CO concentration > 50ppm; Early warning level: multimodal fusion probability ≥ 80% for 5 minutes; Disposal level: fusion probability ≥ 95% or open flame detected; Review level: activated within 2 hours after disposal, generating an optimization report containing 300 feature parameters.

[0025] Beneficial effects

[0026] Compared with the prior art, the present invention provides a multimodal scheduling method and system for grouting resources in goaf areas, which has the following beneficial effects:

[0027] 1. In this solution, multi-modal data fusion is used to achieve multi-dimensional perception of fires in goaf areas, breaking through the limitations of single-parameter monitoring and significantly improving the comprehensiveness and accuracy of anomaly identification; the introduction of the Transformer model and improved evidence theory enhances the correlation analysis capability of cross-modal data and effectively reduces the false alarm rate; dynamic inference combined with physical models and deep reinforcement learning improves the dynamic adaptability of fire development prediction; path planning integrates heuristic algorithms and AR visualization technology to achieve intelligent dynamic optimization of escape routes; a four-level response mechanism and intelligent resource scheduling strategy shorten the emergency response cycle and improve the efficiency of grouting material matching and equipment scheduling.

[0028] 2. In this solution, the system hardware design strengthens anti-interference capabilities and self-diagnostic functions to ensure the stability of data acquisition in complex environments; the hybrid communication network and edge-cloud collaborative computing architecture reduce data transmission load and latency, adapting to low-bandwidth scenarios in mines; lightweight models and parallel computing technology improve the efficiency of real-time analysis and multi-fire zone simulation; 3D visualization and intelligent interactive terminals (such as AR navigation and voice warnings) enhance the convenience of command decision-making and on-site personnel response; intelligent control units and multi-level linkage mechanisms enable precise collaborative control of equipment such as grouting and ventilation, improving emergency response speed and reliability. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the method steps of the present invention;

[0030] Figure 2 This is a schematic diagram of the system framework of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Please see Figures 1-2 This invention proposes a multimodal scheduling method and system for grouting resources in goaf areas, including the following:

[0033] I. Multimodal Scheduling Method for Grouting in Goaf Areas

[0034] (I) Multimodal data acquisition and preprocessing

[0035] 1. Data source deployment and technical parameters

[0036] Infrared thermal imaging monitoring employs a distributed deployment scheme, with one uncooled infrared thermal imager (model FLIRA615, resolution 640×480, temperature measurement accuracy ±0.5℃, response wavelength 7.5-13μm) installed every 20 meters on the roof of the goaf. Equipped with a pan-tilt unit, it achieves 0-360° rotation monitoring, covering the entire temperature field of the goaf. The thermal imager has a built-in blackbody calibration module, automatically calibrating daily to eliminate temperature drift errors.

[0037] A multi-parameter gas sensor network constructs a three-dimensional monitoring array: sensor nodes (model MOTICOMMGS8, supporting simultaneous detection of CO / CH4 / O2, resolution 0.1ppm, response time <15s) are deployed in layers every 10 meters vertically at the air inlet, air outlet, and central part of the goaf. A grid layout with a horizontal spacing of 30 meters forms a 6×6×5 (length×width×height) three-dimensional monitoring network to capture gas diffusion dynamics in real time.

[0038] The microseismic monitoring network deploys a 24-channel microseismic monitoring system (model ESGIN-33, sampling rate 1kHz, dynamic range 120dB, frequency response 1-1000Hz). The sensors are fixed to the sides and roof of the roadway with anchor bolts at 50-meter intervals to form a triangular positioning array, which can locate vibration sources with an error of <5 meters.

[0039] 2. Data Preprocessing

[0040] Infrared thermal imaging correction employs a non-uniformity correction algorithm based on a reference radiation source. By acquiring blackbody radiation images at known temperatures, a response correction coefficient matrix for each pixel is established, and the noise standard deviation is reduced from 1.2℃ to 0.3℃ after correction.

[0041] Design an adaptive Kalman filter for gas concentration filtering, with the state equation as: X k=FX k-1 +W k-1 The observation equation is: Z k =HX k +V k , where X k Let c be the state vector at time k, containing the gas concentration c. k and concentration change rate F is the state transition matrix, describing the change of state over time; W k-1 Z represents process noise, characterizing unknown disturbances in the system. k Let V be the observed value at time k (e.g., gas concentration detection value); H is the observation matrix, which maps the state vector to the observation space; V k To observe noise and reflect the error of the detection equipment, this model achieves dynamic smoothing of gas concentration data by recursively updating the state estimate and noise covariance, thereby reducing the impact of environmental interference.

[0042] Microseismic signal feature extraction employs db4 wavelet decomposition with 5 layers. The detail coefficients of the 3rd layer (corresponding to frequencies of 8-16Hz) are extracted as rock stratum fracture feature vectors. The signal abrupt change point is detected by the modulus maxima method to locate the initiation time of the microseismic event with an accuracy of 0.1ms.

[0043] (II) Multimodal Data Fusion and Feature Extraction

[0044] 1. Feature-level fusion: Transformer model architecture

[0045] Input layer processing temperature matrix: normalize the thermal imager data to [-1,1] and convert it into a 16×16 grid temperature sequence (time window 5 minutes, step size 1 minute); gas concentration sequence: standardize the concentrations of the three gases and construct a 3×30 time series matrix (data from the first 30 minutes); microseismic waveform: extract the waveform 0.5 seconds before and after the event, pad with zeros to 512 points, and form a 1×512 feature vector.

[0046] The encoding layer parameters employ a 6-layer encoder, with each layer containing a 12-head self-attention mechanism. The hidden layer dimension is 768, and the feedforward neural network dimension is 3072. Residual connections are added after layer normalization. The multi-head attention mechanism can capture cross-modal correlations such as temperature-gas concentration (Spearman correlation coefficient improved by 0.28) and microseismic-temperature (mutual information gain of 0.15 bits) in parallel.

[0047] The output layer is reduced to 128 dimensions through a fully connected layer, and after L2 regularization, it outputs a feature vector. The similarity of the inner product of the vectors can characterize the stage of fire development (early / middle / late stage).

[0048] 2. Decision-level fusion: Improving the DS evidence theory

[0049] The Basic Probability Assignment (BPA) constructs a definition of three sources of evidence: temperature anomaly (E... T ), gas anomaly (E) G Microseismic anomalies (E) M The Softmax function maps the feature vector components to BPA values, for example: Where m T (A i () indicates the source of evidence for temperature anomalies in fire event A. i The basic probability allocation value; w represents the temperature-related component in the eigenvector. T The weight vector is obtained through training on historical data; the formula maps feature components to probability values ​​using the Softmax function, reflecting the degree to which temperature data supports fire risk. The improved algorithm introduces a discount factor to handle evidence conflicts and reduce the false alarm rate.

[0050] Discount factor introduced in conflict evidence handling (d i The distance between pieces of evidence is used to weight highly conflicting evidence. The fusion rule adopts the Yager improvement method to avoid the veto problem. After fusion, the fire probability determination threshold is set at 95%. After 2000 simulation tests, the false alarm rate decreased from 12% in the traditional DS theory to 5.8%.

[0051] (III) Fire Dynamic Simulation and Route Planning

[0052] 1. Coupling Mechanism of the Inference Engine

[0053] Physical Model: The Navier-Stokes equations are solved using the finite volume method, with the solution domain divided into a 10m × 10m × 5m grid (approximately 20,000 control volumes). Boundary conditions are set as follows: inlet velocity: 0.5–2 m / s (adjusted in real-time according to the ventilation system); heat release rate from the fire source: dynamically loaded using eigenvector predictions (range 50–500 kW / m²). 2 The PISO algorithm is used to achieve pressure-velocity coupling with a time step of 0.1s, and the flue gas temperature and CO concentration distribution results are output every 10 seconds.

[0054] Data-driven model: DRL-Agent training state space: contains temperature, gas concentration, and wind speed vectors (303 dimensions in total) from 100 monitoring points; Action space: adjusting ventilation valve opening (5 levels) and grouting pump flow rate (10-50 L / min); Reward function: R = -C CO ×0.6-T max ×0.3-t×0.1, where R is the reward value after the agent performs the action, and the smaller the value, the better the fire control effect; C COT represents the area of ​​the region where CO concentration exceeds the standard, with a weight of 0.6 indicating the priority of CO hazards; max The highest temperature in the fire zone, with a weight of 0.3, reflects the impact of temperature on fire development; t represents the response time, with a weight of 0.1, emphasizing the importance of rapid response. This function guides the agent to prioritize reducing CO diffusion and lowering the temperature in high-temperature areas, while shortening the response time. The PPO algorithm is used for training, with an experience playback pool capacity of 1e6 and an update frequency of once every 500 steps. After convergence, the fire prediction error is reduced by 22% compared to the pure physical model.

[0055] The coupling process initially generates a reference field from the physical model. Then, every 10 seconds, the boundary conditions are updated based on the control actions output by the DRL-Agent. At the same time, the model parameters (such as the offset of the fire source position) are corrected using new monitoring data, forming a closed loop of "simulation-measurement-correction".

[0056] 2. Generation of 3D escape paths

[0057] The dynamic environment modeling constructs a dynamic graph model that includes the tunnel topology (number of nodes > 5000), real-time fire situation (diffusion rate of high temperature zone and smoke zone 0.5-3m / s), and personnel location (positioning via UWB with an accuracy of 0.3m). The graph weights are updated every 2 seconds (weight of high temperature zone × 10, weight of smoke zone × 5).

[0058] The path optimization algorithm is an improvement on Dijkstra's algorithm, introducing a heuristic function h(n) = 0.8 × straight-line distance + 0.2 × ventilation speed, prioritizing short-distance, upwind paths. When a person's movement speed is detected to be <0.5 m / s, an emergency refuge point recommendation (based on a database of tunnel safety chamber locations) is automatically triggered.

[0059] The AR visualization uses the HoloLens2 terminal and constructs a 3D map of the goaf using SLAM technology, overlaying fire simulation results in real time: high-temperature zones are represented by red semi-transparent bodies, the smoke diffusion front is marked by yellow streamlines, and escape routes are dynamically guided by green arrows, supporting gesture interaction to switch perspectives.

[0060] (iv) Multimodal chain response and resource scheduling

[0061] 1. Thresholds and Actions of the Four-Level Response Mechanism

[0062] The system is configured with a four-level response mechanism. The triggering conditions and execution actions for each level are as follows:

[0063] Monitoring level: When a single parameter exceeds the warning threshold (such as temperature above 35℃ or CO concentration above 50ppm), the central control room will issue an audible and visual alarm and start the grouting pump lubricating oil preheating to raise the oil temperature to 40℃. The entire response process is completed within 10 seconds.

[0064] Warning level: When the probability of a fire in the multimodal fusion reaches 80% and lasts for 5 minutes, the system will push escape routes to miners within a 500-meter range and switch the ventilation system to reverse ventilation mode, increasing the air volume to 120% of the normal state, with the response time controlled within 30 seconds.

[0065] Disposal Level: Triggered when the fusion probability is ≥95% or an open flame is detected, automatically starting 3 grouting pumps and controlling the flow rate according to the distance to the fire source (50 liters / minute for near the fire source area, 30 liters / minute for medium distance, and 10 liters / minute for far distance). At the same time, the fire door in the fire source area is closed. The entire disposal action is completed within 60 seconds.

[0066] Review Level: Started within 2 hours after the treatment is completed, the system extracts more than 300 feature parameters (such as maximum temperature rise rate, CO peak concentration, etc.), updates the hyperparameters of the deep reinforcement learning (DRL) model through the Bayesian optimization algorithm, and generates a report including treatment efficiency score. The entire review process does not exceed 120 minutes.

[0067] 2. Resource scheduling optimization strategy

[0068] Intelligent matching of grouting materials: The system establishes a knowledge base for matching materials with fire conditions, and automatically selects grouting materials based on the characteristics of the fire zone: High-temperature areas (>600℃) preferentially use polymer gels, with a setting time of 1-3 minutes and a thermal conductivity as low as 0.05 W / (m·K), enabling rapid cooling and insulation; areas with developed cracks use cement-water glass dual-liquid grout, with an initial setting time of 20-30 seconds and a stone-forming rate of 95%, effectively sealing cracks; conventional fire zones use fly ash-based grouting materials, reducing costs by 40% compared to traditional materials. The system predicts fire zone type using a support vector machine (SVM) classifier, achieving a material matching accuracy of 92%.

[0069] Grouting truck path planning: An improved Time-Window Ant Colony Algorithm (TW-ACO) is used to plan the grouting truck's travel path. The algorithm finds the optimal path by simulating ant foraging behavior. Parameters include 50 ants, a pheromone evaporation rate of 0.1, a heuristic factor of 2, and an expectation heuristic factor of 1.5. The objective function is to minimize the sum of travel time and timeout penalty. Mine roadway simulation tests show that the average response time is reduced from 12 minutes for the traditional algorithm to 7.2 minutes.

[0070] II. Multimodal Scheduling System for Grouting in Goaf Areas

[0071] (I) Multimodal Sensing Module

[0072] 1. Hardware Enhancement Design

[0073] Anti-interference measures: The infrared thermal imager uses a metal explosion-proof shell (IP68 protection level) and has a built-in electromagnetic shielding layer (attenuation > 60dB); the gas sensor array is equipped with a temperature compensation circuit (compensation coefficient 0.02% / ℃); and the micro-vibration monitoring node adopts a dual power supply redundancy design (lithium battery + intrinsically safe power supply).

[0074] The self-diagnostic function integrates a health status monitoring module for each sensor, which automatically detects sampling channel noise every hour (thresholds: thermal imager noise > 0.8℃, gas sensor drift > 5%FS, micro-vibration node packet loss rate > 3%). When an abnormality is detected, the device triggers a self-test and reports a fault code.

[0075] 2. Communication Network Architecture

[0076] The hybrid networking solution uses industrial Ethernet (IEEE802.3af, transmission rate 100Mbps) for the backbone network, and deploys PoE switches at the corners of the alleys (100 meters apart); branch nodes (such as micro-vibration sensors) are connected to the gateway (model RAK7249, sensitivity -148dBm, coverage radius 800 meters) via the LoRaWAN protocol, and the gateway is aggregated to the central control room via optical fiber.

[0077] Data encryption transmission uses the AES-256 algorithm to encrypt monitoring data end-to-end, and the key is automatically updated every 24 hours.

[0078] (II) Intelligent Analysis Module

[0079] 1. Edge-Cloud Collaborative Computing Architecture

[0080] Three NVIDIA Jetson AGX Orin edge computing nodes (64-core ARM CPU, 200 TOPS computing power) are deployed at the edge layer. Each node is responsible for processing data from 8-10 sensor subnets, enabling 90% of real-time analysis tasks to be processed locally, with less than 10% of data uploaded to the cloud, thus reducing network load.

[0081] The cloud layer uses a Kubernetes cluster to manage 8 GPU servers (NVIDIA A100×4 / unit), supporting model training (single training data volume > 1TB), historical data backtracking (storage period of 3 years) and multi-mine collaborative analysis (federated learning mode).

[0082] 2. Algorithm Optimization Details

[0083] The Transformer model lightweighting employs knowledge distillation technology, compressing the original 12-layer encoder into 6 layers, reducing model parameters by 67%, and lowering inference latency from 280ms to 110ms, thus adapting to the computing capabilities of edge nodes.

[0084] The inference engine uses OpenMP to achieve multi-core parallel computing for the physical model (speedup ratio 4.2×), and the DRL model is deployed as a microservice using TensorFlowServing, supporting 100 concurrent request processing, with an overall inference latency of <10 seconds.

[0085] (III) Dynamic Deduction and Visualization Module

[0086] 1. Improved accuracy of fire simulation

[0087] The multiphysics-coupled integrated heat transfer (Fourier's law) and combustion (Arrhenius equation) modules simulate parameters including:

[0088] Coal spontaneous combustion characteristic parameters: activation energy 80 kJ / mol, pre-exponential factor 1e8s -1 ;

[0089] Flue gas transport parameters: Prandtl number 0.7, Schmidt number 0.75; verified by laboratory fire simulation, the temperature field prediction error is <12%, and the CO concentration error is <15%.

[0090] Parallel computing optimization is based on MPI to achieve cross-server distributed computing, supporting parallel simulation of up to 16 nodes. The computing speed is increased by 12 times under a grid scale of 100,000, meeting the needs of simultaneous simulation of multiple fire zones.

[0091] 2. Visual Interaction Design

[0092] The command center's large screen uses a 4x5 splicing method (resolution 11520×3840), displaying the following content:

[0093] 3D situation map: Real-time rendering of fire spread animation, with pressure distribution of grouting pipeline marked (color-coded);

[0094] Data dashboard: Displays key indicators (such as total grouting volume and CO exceeding the limit area change rate), and supports historical curve comparison (time span 1-24 hours);

[0095] Resource scheduling dashboard: dynamically displays the location of grouting trucks, material inventory (RFID real-time inventory) and personnel distribution (UWB location data).

[0096] The miner terminal interaction is based on a dedicated Android app, which supports:

[0097] Voice navigation: Employs offline speech synthesis technology (speech speed 120 words / minute, recognition accuracy 98%), updating escape instructions every 10 seconds;

[0098] Gesture controls: Swipe with three fingers to switch map views, long press to mark danger points and automatically sync to the command center;

[0099] Environmental alert: When personnel enter an area with CO > 24 ppm, the terminal will vibrate and flash red light to trigger an alarm (response time < 2 seconds).

[0100] (iv) Resource Scheduling and Execution Module

[0101] 1. Control Unit Technical Parameters

[0102] The PLC control system uses a Siemens S7-1500 PLC, equipped with an ET200SP distributed I / O module, and supports:

[0103] Analog control: Grouting pump flow rate adjustment (4-20mA signal, 12-bit resolution);

[0104] Digital control: ventilation valve switching (response time < 50ms), alarm device triggering; the control program is written in structured text (ST) and supports online debugging and fault self-diagnosis.

[0105] The grouting volume control uses an incremental PID algorithm with parameter K. p =1.2, K i =0.05, K d =0.2 After step response test, the settling time is <15s, the overshoot is <3%, and the flow fluctuation range is ±1.5L / min.

[0106] 2. System linkage test data

[0107] In system testing, the response metrics for different scenarios all met the design requirements:

[0108] The alarm delay for monitoring-level early warning is 8.2 seconds, which is better than the standard requirement of 10 seconds;

[0109] The delay in sending escape routes for early warning-level responses is 22 seconds, and should be kept within 30 seconds.

[0110] The grouting pump start-up time for treatment-level actions is 55 seconds, which meets the standard of <60 seconds;

[0111] The generation time for a debriefing report is 89 minutes, which is less than the 120-minute requirement.

[0112] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multimodal scheduling method for grouting resources in goaf areas, characterized in that: Includes the following steps: S1 Multimodal Data Acquisition and Preprocessing: An infrared thermal imaging monitoring network was deployed, employing distributed, uncooled infrared thermal imagers; a multi-parameter gas sensor network was constructed to detect CO, CH4, and O2 concentrations in real time; a microseismic monitoring network was deployed, using a 24-channel microseismic monitoring system, with sensors anchored at 50-meter intervals to the sides and roof of the tunnel, forming a triangular positioning array; non-uniformity correction was applied to the infrared thermal imaging data; gas concentration data was dynamically smoothed using an adaptive Kalman filter; and rock fracture feature vectors in the 8-16Hz frequency band were extracted from the microseismic signals through db4 wavelet decomposition. S2 Multimodal Data Fusion and Feature Extraction: Feature-level fusion is performed based on the Transformer model. The input consists of a normalized temperature sequence, a normalized gas concentration sequence, and microseismic waveforms padded to 512 points. Cross-modal correlations are captured through a 6-layer encoder structure, and a 128-dimensional feature vector is output. Decision-level fusion is performed using an improved DS evidence theory. Three evidence sources are defined: temperature anomaly, gas anomaly, and microseismic anomaly. Conflicting evidence is processed through a discount factor. The fire determination threshold after fusion is 95%. S3 Fire Dynamic Simulation and Path Planning: Couples the Navier-Stokes equation physical model with a deep reinforcement learning agent to dynamically adjust the opening of ventilation valves and the flow rate of grouting pumps; generates three-dimensional escape paths based on dynamic environment models, uses an improved Dijkstra algorithm to prioritize short-distance upwind paths, and visualizes them in real time through an AR terminal. S4 Multimodal Chain Response and Resource Scheduling: Triggers a four-level response mechanism; intelligent matching of grouting materials, selecting materials based on fire zone characteristics; and uses an ant colony algorithm with a time window to plan the grouting vehicle path.

2. The multimodal scheduling method for grouting resources in goaf areas according to claim 1, characterized in that: The infrared thermal imager is deployed every 20 meters on the roof of the goaf area, equipped with a pan-tilt unit to achieve 0-360° rotation monitoring, and has a built-in blackbody calibration module for daily automatic calibration; the infrared thermal imager has a resolution of 640×480, a temperature measurement accuracy of ±0.5℃, a response wavelength of 7.5-13μm, and achieves a noise standard deviation of ≤0.3℃ through a non-uniformity correction algorithm of the reference radiation source.

3. The multimodal scheduling method for grouting resources in goaf areas according to claim 1, characterized in that: The gas sensor network has a grid layout with a horizontal spacing of 30 meters. Sensor nodes are deployed in layers every 10 meters along the vertical direction at the air inlet, air outlet, and central part of the goaf, forming a 6×6×5 three-dimensional monitoring array. Gas concentration filtering is performed using the state equation X. k =FX k-1 +W k-1 With observation equation Z k =HX k +V k An adaptive Kalman filter is used, where the state vector includes concentration and rate of change, and the observation noise covariance is dynamically updated.

4. The multimodal scheduling method for grouting resources in goaf areas according to claim 1, characterized in that: In the improved DS evidence theory, the basic probability assignment maps the eigenvector components through the Softmax function, conflict evidence processing employs the Yager improvement method, and inter-evidence distance is introduced. As a discount factor.

5. The multimodal scheduling method for grouting resources in goaf areas according to claim 1, characterized in that: The reward function of the DRL agent is R = -C CO ×0.6-T max ×0.3-t×0.1, where the weight of CO exceeding the standard area is 0.6, the weight of the highest temperature in the fire zone is 0.3, and the weight of the treatment time is 0.

1. The PPO algorithm is used for training, and the capacity of the experience playback pool is 1e6.

6. A multimodal scheduling system for grouting resources in goaf areas, used to implement the multimodal scheduling method for grouting resources in goaf areas as described in any one of claims 1 to 5, characterized in that: The system includes a multimodal perception module, an intelligent analysis module, a dynamic deduction and visualization module, and a resource scheduling and execution module; Multimodal sensing module: includes an infrared thermal imager, a gas sensor and a micro-vibration monitoring node, and adopts an IP68 explosion-proof shell, temperature compensation circuit and dual power supply redundancy design; Intelligent analysis module: It adopts an edge-cloud collaborative architecture, with NVIDIA Jetson AGX Orin nodes deployed at the edge layer and a GPU server cluster configured at the cloud layer, supporting lightweight Transformer model and parallel acceleration of the inference engine; Dynamic simulation and visualization module: Based on MPI, multi-physics coupling simulation is realized, the command center large screen renders a three-dimensional situation map in real time, and the miner terminal supports voice navigation and gesture interaction; Resource scheduling and execution module: Siemens S7-1500 PLC is used to control the grouting pump flow and ventilation valves. After PID parameter tuning, the adjustment time is <15s, the overshoot is <3%, and a four-level response mechanism is designed.

7. The multimodal scheduling system for grouting resources in goaf areas according to claim 6, characterized in that: The communication network of the multimodal sensing module adopts a hybrid networking of industrial Ethernet and LoRaWAN, and the data transmission is encrypted with AES-256, with the key automatically updated every 24 hours.

8. The multimodal scheduling system for grouting resources in goaf areas according to claim 6, characterized in that: The AR terminal of the visualization module is a HoloLens2 device, which uses SLAM technology to build a three-dimensional map and dynamically mark high-temperature areas, smoke diffusion fronts and escape routes.

9. The multimodal scheduling system for grouting resources in goaf areas according to claim 6, characterized in that: The grouting material matching is based on a support vector machine classifier, and the input features include fire zone temperature, fracture development degree, and CO concentration gradient.

10. The multimodal scheduling system for grouting resources in goaf areas according to claim 6, characterized in that: The triggering conditions for the four-level response mechanism are as follows: Monitoring level: temperature > 35℃ or CO concentration > 50ppm; Early warning level: multimodal fusion probability ≥ 80% for 5 minutes; Disposal level: fusion probability ≥ 95% or open flame detected; Review level: activated within 2 hours after disposal, generating an optimization report containing 300 characteristic parameters.

Images