Coal mine fire and gas disaster risk prediction method and system

Abstract

The invention belongs to the technical field of coal mine disaster early warning, and discloses a coal mine fire and gas disaster risk prediction method and system, and the method comprises the steps: collecting multi-source heterogeneous data collected at a mine site, and carrying out the data preprocessing and conversion; for the preprocessed image and video data, a Video Swin Transform model is adopted to extract spatial and temporal features, and a cross-modal association fusion Transform model is adopted to realize multi-modal feature unified fusion; mapping the fused multi-modal feature representation to a disaster-causing factor index space, calculating a risk assessment index containing a secondary interaction item, and dividing risk levels; and generating a disaster early warning strategy according to risk level analysis, issuing an early warning notice, performing regular maintenance and updating, and outputting a decision report. According to the method, unified processing and fusion representation of multi-source heterogeneous data can be realized, complex association among disaster-causing factors is accurately captured, the risk index with physical significance and engineering reference value is calculated, and high-quality decision support is provided for prevention and control of coal mine fire and gas disasters.

Description

Technical Field

[0001] The present invention relates to the technical field of coal mine safety monitoring, disaster prediction and early warning, and relates to but is not limited to a method and system for predicting the risk of coal mine fire and gas disasters. Background Art

[0002] Coal mine fires and gas explosions have always been the most important and most serious types of disasters in coal mine production. They are sudden and destructive, and there are problems such as difficulty in identifying precursors and inaccurate predictions. Traditional risk assessment methods based on single sensors or simple data processing methods do not fully utilize the massive and multi-source heterogeneous data resources in modern coal mines, and are unable to deeply capture the highly nonlinear interaction mechanism between fire and gas disaster factors.

[0003] With the improvement of coal mine safety monitoring technology and information level, various types of sensors are installed in mines to monitor key indicators such as gas concentration, oxygen concentration, temperature, coal dust concentration, ventilation parameters, pressure, geological stress, etc. in real time; at the same time, visual information inside the mine can be obtained through cameras and thermal imaging equipment, and descriptive information of management, workers and inspectors can be obtained through text reports and records. In addition, static prior data such as mine geological structure parameters, coal seam thickness and spontaneous combustion tendency index can also provide important references for risk assessment. However, multimodal data have significant differences in data type, scale, spatiotemporal resolution, expression form, etc. How to effectively integrate these data, extract high-quality multimodal disaster factor characteristics and establish an accurate risk prediction model is a technical difficulty that needs to be solved urgently. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and system for predicting the risk of coal mine fire and gas disasters, aiming to propose a complete multimodal data processing process for coal mine fire and gas disasters, covering the full chain of steps of data collection, preprocessing, feature extraction, modal fusion, risk assessment, early warning decision-making and maintenance management.

[0005] The technical solutions of the embodiments of the present invention are as follows: In a first aspect, an embodiment of the present invention provides a method for predicting coal mine fire and gas disaster risks, comprising: The multi-source heterogeneous data collected at the mine site are collected, and the data are preprocessed and converted; wherein the multi-source heterogeneous data include numerical sensor data, image and video data, text data and static prior data, and the static prior data include at least coal spontaneous combustion tendency index, geological structure parameters and coal seam thickness; the Video Swin Transformer model is used to extract spatiotemporal features from the preprocessed image and video data, and the cross-modal association fusion Transformer model is used to realize the unified fusion of multimodal features; the fused multimodal feature representation is mapped to the disaster factor indicator space, the risk assessment index including the quadratic interaction term is calculated and the risk level is divided; wherein the interaction coefficient of the quadratic interaction term is set according to expert experience and historical data and fine-tuned in model training; the disaster warning strategy is generated based on the risk level analysis and the warning notice is issued, and regular maintenance and update are performed at the same time, and a decision report is output.

[0006] Furthermore, the multi-source heterogeneous data collected at the mine site is collected, and data preprocessing and conversion are performed, including: determining and eliminating outliers in numerical sensor data, filling in small-range missing data with interpolation, and processing through minimum-maximum normalization and Z-score standardization methods to make the dimensions of each feature consistent; performing word segmentation and embedding representation on text data, converting the text into a vector representation of a fixed dimension, and performing dimensionality reduction and feature extraction; extracting key frames from image and video data for image enhancement, and normalizing and cropping the enhanced key frames to a uniform size; aligning different modal data according to timestamps to ensure that each modal data corresponds at the same time step.

[0007] Furthermore, the Video SwinTransformer model is used to extract spatiotemporal features from the preprocessed image and video data, including: slicing the original acquired video data according to a certain time step to obtain a series of video frame sets with a time series relationship; dividing each video frame sequence into three dimensions in space and time, and performing feature embedding mapping on each block sequence through a linear layer; at the initial layer, the Video Swin Transformer model divides the video features into local windows and calculates multi-head self-attention within the window to capture local spatiotemporal relationships, and then gradually expands the feature receptive field by merging or sliding windows at a higher level, and fuses local features into a global representation; after multi-layer feature extraction and fusion, a high-dimensional spatiotemporal feature vector is output, and finally the spatiotemporal feature vector is mapped to a unified embedding dimension through a fully connected layer and a normalization layer, and an embedding identifier for identifying the video modality is added.

[0008] Furthermore, the cross-modal association fusion Transformer model is used to achieve unified fusion of multimodal features, including: unifying the multi-source heterogeneous data into the same feature dimension and embedding space, and assigning a corresponding modal identifier to each modality, and finally obtaining an aligned multimodal feature sequence at the same time step; using a multi-head self-attention mechanism, focusing on different subspaces and modal interactions of the multimodal feature sequence through multiple attention heads in parallel; through the superposition of multiple Transformer blocks, the cross-modal association fusion Transformer model will deeply fuse the multimodal feature sequence, capture the temporal dependency and the coupling relationship between modalities, and obtain the fused multimodal feature representation at the output layer. ; where M is the number of modes, H m is the feature representation of the mth mode, λ m represents the weight factor of the mth mode, satisfying , as a learnable parameter that is automatically adjusted during model training.

[0009] Furthermore, the method also includes: introducing a correlation bias matrix B in the multi-head self-attention mechanism to incorporate prior knowledge of disaster-causing factors, and calculating the attention score by the following formula: ; In the formula, is the attention score, is a normalization function, Q, K, and V are query, key, and value matrices, respectively, which are linear transformations of the input features. is the transpose of K; d k is the dimension of the key vector; B is obtained by constructing the association matrix between disaster-causing factors based on the knowledge graph of coal mine fire and gas disaster factors.

[0010] Furthermore, the multi-source heterogeneous data are unified into the same feature dimension and embedding space, and a corresponding modality identifier is assigned to each modality, including: normalizing the numerical data so that all numerical features are in the same dimension, mapping the normalized numerical features to the same embedding space dimension dm as the video features through one or more layers of fully connected networks, adding a nonlinear activation function during the mapping process to improve the representation ability of the features, and finally obtaining a numerical feature embedding vector with a dimension of dm; encoding the text data using a self-supervised pre-trained language model to obtain a sentence-level text embedding vector, mapping it to the same embedding space dimension dm as the video features and numerical features through a fully connected layer, and adding a text modality identifier; linearly transforming the static prior data and mapping it to the dm dimension and adding a priori modality identifier.

[0011] Further, mapping the fused multi-modal feature representation to the disaster-causing factor index space, calculating the risk assessment index including quadratic interaction terms, and dividing the risk levels, includes: obtaining multiple disaster-causing factor vectors by dimensionality reduction and linear mapping of the multi-modal feature representation, and each factor reflects the relative change rate or relative deviation degree after normalization processing; wherein, the multiple disaster-causing factor vectors include: gas concentration factor, temperature rise factor, oxygen concentration factor, coal dust concentration factor, ventilation parameter factor, pressure change factor, geological stress factor, coal spontaneous combustion tendency index factor, coal seam thickness factor, and geological structure complexity factor; setting the weight of each disaster-causing factor vector, and setting the interaction coefficient between any two disaster-causing factors according to expert experience and historical data; performing weighted summation on the multiple disaster-causing factor vectors based on the weight to obtain the basic risk index; based on the basic risk index, combining the interaction coefficient and the historical disaster frequency, finally determining the risk assessment index R; comparing and analyzing the risk assessment index R with the risk level thresholds T1, T2, and T3, and dividing into four different risk levels: low risk when R < T1, medium risk when T1 ≤ R < T2, high risk when T2 ≤ R < T3, and extremely high risk when R ≥ T3.

[0012] Further, generating a disaster warning strategy based on the risk level analysis and issuing a warning notice, includes: for the low risk level, in terms of monitoring, regularly monitoring and continuously collecting data according to a predetermined plan for real-time calibration of the model, in terms of prevention, regularly inspecting and maintaining sensors and monitoring equipment, carrying out prevention knowledge training, organizing patrols to timely discover and eliminate potential fire sources and gas accumulation points, and in terms of notification, regularly notifying the risk assessment results and safety status to the mine management and relevant departments; For the medium risk level, in terms of monitoring, strengthening patrols, focusing on inspecting areas where the disaster-causing factors are close to the warning value, focusing on monitoring high-risk disaster-causing factors identified by the model, and transmitting data in real time for in-depth analysis and identification of abnormal trends, in terms of prevention, strengthening ventilation, controlling temperature, rectifying hidden dangers, and strengthening equipment inspection, and in terms of notification, sending warning information to relevant departments for the person in charge to discuss countermeasures; For the high risk level, in terms of monitoring, activating the emergency plan, increasing the detection frequency, preparing emergency measures, monitoring key disaster-causing factors 24 hours a day, and predicting risks through the model; in terms of prevention, activating the response mechanism, strengthening the gas drainage intensity, taking measures such as injecting nitrogen and inert gas to inhibit spontaneous combustion, and at the same time prohibiting personnel from entering and being on standby for emergencies; in terms of notification, issuing high-risk warning information to all mine personnel, emphasizing precautions and prohibited behaviors; For the extremely high risk level, in terms of monitoring, continuing to monitor the disaster-causing factors to obtain real-time data; in terms of prevention, shutting down equipment, sealing off the area, evacuating all personnel and taking a roll call; in terms of notification, issuing an emergency evacuation order through all available channels, reporting to the department, requesting support, issuing an announcement, and controlling public opinion.

[0013] In a second aspect, an embodiment of the present invention provides a coal mine fire and gas disaster risk prediction system, characterized in that it includes: A data acquisition and preprocessing module is used to collect multi-source heterogeneous data collected at the mine site, and perform data preprocessing and conversion; wherein the multi-source heterogeneous data includes numerical sensor data, image and video data, text data and static prior data, and the static prior data at least includes coal spontaneous combustion tendency index, geological structure parameters, and coal seam thickness; A feature extraction and fusion module, which is used to extract spatiotemporal features from the preprocessed image and video data using a Video SwinTransformer model, and to achieve unified fusion of multimodal features using a cross-modal association fusion Transformer model; The risk assessment and prediction analysis module is used to map the fused multimodal feature representation to the disaster-causing factor index space, calculate the risk assessment index including the quadratic interaction term, and classify the risk level; wherein the interaction coefficient of the quadratic interaction term is set according to expert experience and historical data and fine-tuned during model training; The early warning notification and maintenance management module is used to generate disaster early warning strategies and issue early warning notifications based on risk level analysis, while performing regular maintenance and updates and outputting decision reports.

[0014] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: In the embodiment of the present invention, firstly, an advanced multimodal fusion algorithm is used to uniformly model text, image video, sensor numerical data, and prior information such as geological structure and coal seam characteristics, focusing on solving the difficult problems of heterogeneous data fusion and spatiotemporal feature extraction; secondly, a cross-modal association fusion Transformer model is introduced, including modal embedding, modal weight learnable parameters and prior correlation bias matrix, so that the model can automatically identify the complex relationship between key disaster-causing factors, thereby improving the accuracy of risk prediction; then, based on the fused multimodal features, a risk assessment formula containing quadratic interaction terms is formulated to quantify the weight of disaster-causing factors and their interaction effects in the overall risk, and combined with historical disaster frequencies and expert experience, to provide accurate risk assessment indexes and warning level classification standards; finally, based on the risk index and classification results, personalized warning strategies and decision support are provided for coal mining enterprises, including monitoring, prevention, emergency response and post-maintenance management. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which: Figure 1 A schematic diagram of a flow chart of a method for predicting coal mine fire and gas disaster risks provided by an embodiment of the present invention; Figure 2 A schematic diagram of the video feature extraction process of the Video Swin Transformer model provided in an embodiment of the present invention; Figure 3 A schematic diagram of risk level classification and corresponding disaster warning strategies provided by an embodiment of the present invention; Figure 4 A schematic diagram of the framework structure of a coal mine fire and gas disaster risk prediction system provided by an embodiment of the present invention; Figure 5 A schematic diagram of the maintenance management and model update process framework provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0017] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0018] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present invention are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present invention described here can be implemented in an order other than that illustrated or described here.

[0019] Those skilled in the art will appreciate that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the field to which the embodiments of the present invention belong. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0020] In 2024, a variety of deep learning algorithms for multimodal spatiotemporal data have emerged, including a new spatiotemporal Transformer structure, a cross-modal graph attention mechanism, and an improved self-attention model that introduces a priori knowledge bias matrix. These technologies provide a new technical path to solve the problem of multimodal data fusion and feature extraction for coal mine fire and gas disasters. However, this field still lacks a mature, complete technical solution for multimodal fusion, feature extraction, risk assessment, and early warning strategy formulation.

[0021] Based on this, the embodiment of the present invention proposes an innovative method for extracting and fusing the features of coal mine fire and gas disaster factors by using a multimodal fusion and feature extraction algorithm (cross-modal correlation fusion Transformer model) and an improved spatiotemporal convolution model. At the same time, a risk index calculation formula based on quadratic interaction terms is introduced, and historical data and expert experience are combined to quantitatively evaluate and classify the prediction results, providing a scientific basis for coal mine safety decision-making.

[0022] Figure 1 A flow chart of a method for predicting coal mine fire and gas disaster risks provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method comprises at least the following steps: Step S110, collecting multi-source heterogeneous data collected at the mine site, and performing data preprocessing and conversion.

[0023] Here, the multi-source heterogeneous data includes numerical sensor data, image and video data, text data and static prior data, and the static prior data includes at least coal spontaneous combustion tendency index, geological structure parameters, and coal seam thickness. The numerical sensor data includes gas concentration G, temperature T, oxygen O, coal dust D, ventilation parameter V, pressure P, geological stress S, coal seam thickness Th, coal spontaneous combustion tendency Sp, and geological structure complexity Gs; image and video data are obtained through cameras and infrared thermal imaging equipment installed at key positions in the mine, which can be used to detect potential disaster-causing features such as flames, smoke, and dust accumulation; text data includes underground staff inspection records, expert reports, equipment maintenance logs, etc., which exist in the form of natural language; static prior data such as coal spontaneous combustion tendency index, geological structure parameters, coal seam thickness, etc. can be updated regularly, but the frequency of changes is relatively low.

[0024] In the implementation, multiple types of sensors are deployed in the mine, such as gas sensors (monitoring gas concentration), temperature sensors, oxygen sensors, coal dust sensors, ventilation parameter sensors, pressure sensors, geological stress sensors, image and video recording units, etc. The coal spontaneous combustion tendency index, coal seam thickness and geological structure parameters are obtained as static prior data. The above multi-source heterogeneous data are cleaned, missing values ​​are filled, outliers are removed, normalized and time-series aligned, and natural language processing is performed on the text information to convert the text into computable numerical embedding.

[0025] Step S120, using the Video Swin Transformer model to extract spatiotemporal features from the preprocessed image and video data, and using the cross-modal association fusion Transformer model to achieve unified fusion of multi-modal features.

[0026] Here, the goal of this step is to build a unified multimodal feature representation for multi-source heterogeneous data of coal mine fire and gas disasters (numerical sensor data, text description data, image / video visual data, geological structure and coal seam characteristic parameters, etc.). Specifically, Video Swin Transformer is used to extract spatiotemporal features from image and video data, and a cross-modal association fusion Transformer model is used to realize the unified fusion of multimodal features and the association modeling of key disaster-causing factors. Modal embedding, modal identification and modal weight tuning mechanism are introduced into the cross-modal association fusion Transformer model, as well as a correlation bias matrix based on prior knowledge, to efficiently integrate the information of each modality, providing a solid data foundation and feature representation for subsequent risk assessment and early warning strategies.

[0027] Video Swin Transformer is a model structure based on Transformer and specifically for video spatiotemporal feature extraction. Its core idea is to convert long-span video data into multi-scale feature representations that can be efficiently processed by the model through hierarchical and local window-based self-attention mechanisms. The embodiment of the present invention uses the Video Swin Transformer model to extract spatiotemporal features of image / video data with the following characteristics: 1) 3D Patch Segmentation and Embedding: Segment the continuous video frame sequence at a certain time interval (such as T frames as a group), and divide each frame image in the spatial dimension to form a 3D patch (space × time three-dimensional small block). These patches are converted into fixed-dimensional feature vector embedding representations through linear mapping (linear layer). After such processing, the original video data is transformed from a high-dimensional pixel sequence into a set of compact feature vectors, which is ready for subsequent self-attention calculations.

[0028] 2) Hierarchical structure and multi-scale spatiotemporal modeling: Video Swin Transformer adopts a multi-level, hierarchical feature extraction architecture. At the initial level, the model processes patch features with higher resolution but smaller receptive fields, focusing on capturing local, short-term dynamic changes (such as sparks or smoke that appear in an instant). As the level deepens, the model reduces the spatial resolution and expands the receptive field layer by layer through downsampling (Pooling or specific strategies), aggregating more time steps and larger spatial range information at higher levels to capture the characteristics of long-term slow changes (such as gradual increase in temperature and slow accumulation of coal dust).

[0029] 3) Local Window Self-Attention (Swin Attention) Strategy: Unlike the traditional Transformer that performs self-attention on global features, the Video Swin Transformer performs local window division (Window-based Attention) on features at each layer. Calculating self-attention within each local window can effectively reduce the computational complexity and fully extract local spatiotemporal features. After that, by merging or sliding windows at a higher level, local information is gradually expanded to the global range to achieve gradual fusion of multi-scale features. For multi-scale distributed features in coal mine scenarios (such as local hot spots, local smoke clouds, and slow heating in a certain area), this local-global progressive fusion strategy helps to accurately capture and characterize.

[0030] 3) Multi-layer superposition and final feature output: After the multi-layer Swin Attention is superimposed with the hierarchical structure, the model can obtain a more abstract multi-scale spatiotemporal feature representation at the high level. The final output feature vector (such as 256 dimensions or 512 dimensions) condenses the complex spatiotemporal information of the video in the current time window.

[0031] The Transformer model has achieved remarkable results in natural language processing, computer vision, time series prediction and other fields with its self-attention mechanism. However, traditional Transformers are mostly used for a single modality (such as text sequences). In multimodal scenarios, the feature distribution, semantic space and data scale of different modalities are different. The cross-modal association fusion Transformer model proposed in the present invention adds the encoding of modal information, the learnable adjustment of modal weights, and the embedding of prior knowledge into the self-attention calculation in the form of a correlation bias matrix on the basis of the traditional Transformer.

[0032] Step S130, mapping the fused multimodal feature representation to the disaster factor index space, calculating the risk assessment index including the quadratic interaction term and dividing the risk level.

[0033] Here, the interaction coefficient of the quadratic interaction term is set according to expert experience and historical data and fine-tuned during model training. When calculating the risk assessment index, the embodiment of the present invention comprehensively considers the interaction between disaster-causing factors i and j, such as high gas concentration and temperature rise may increase the risk of gas explosion; reduced oxygen concentration and temperature rise may indicate an intensified combustion process; when ventilation is insufficient, coal dust concentration increases, increasing the risk of explosion; changes in pressure and geological stress may cause rock formation damage and induce disasters; coal seams with high spontaneous combustion tendency are more likely to spontaneously combust when the temperature rises. Therefore, it is necessary to design corresponding interaction coefficients for the interaction between gas concentration and temperature rise, the interaction between oxygen concentration and temperature rise, the interaction between coal dust concentration and ventilation parameters, the interaction between pressure change and geological stress, and the interaction between coal seam spontaneous combustion tendency and temperature rise.

[0034] After obtaining the fused multimodal feature representation H, the embodiment of the present invention obtains multiple disaster-causing factor vectors through dimensionality reduction and linear mapping of the multimodal feature representation H, and then sets the weight coefficient of each disaster-causing factor and the interaction coefficient of the quadratic interaction term based on expert experience and historical data, and performs weighted sum calculation to obtain the risk assessment index.

[0035] Step S140, generate a disaster warning strategy based on risk level analysis and issue a warning notice, perform regular maintenance and updates, and output a decision report.

[0036] Here, after completing the multimodal feature extraction, fusion and risk index calculation, the present invention can predict and analyze the risk situation in real time. Based on the risk assessment index R finally calculated, combined with the risk level thresholds (T1, T2, T3) set by the mine historical data and expert experience, the risk is divided into four levels: low, medium, high, and very high. By adopting corresponding response strategies for different levels, it is ensured that coal mine operations can detect potential dangers in a dynamic environment and respond quickly, thereby minimizing the probability of disasters and losses.

[0037] After predicting the risk level, it is necessary not only to respond and report in a timely manner, but also to continuously iterate, optimize and adjust throughout the life cycle of disaster risk control. By regularly maintaining and updating model parameters and risk level thresholds, the disaster warning strategy can be dynamically adjusted and a decision report can be output.

[0038] The embodiments of the present invention can realize the unified processing and fusion representation of multi-source heterogeneous data (including sensor numerical data, text reports, image and video information, geological structure parameters and coal seam characteristics, etc.), accurately capture the complex correlation between disaster-causing factors, and calculate the risk index with physical significance and engineering reference value, so as to provide high-quality decision support for the prevention and control of coal mine fire and gas disasters.

[0039] In some embodiments, the above step S110 of "collecting multi-source heterogeneous data collected at the mine site, and performing data preprocessing and conversion" is implemented by the following steps: S111, determining and eliminating abnormal values ​​in the numerical sensor data, filling in the small range of missing data by interpolation, and processing by minimum-maximum normalization and Z-score standardization methods to make the dimensions of each feature consistent; S112, performing word segmentation and embedding representation on the text data, converting the text into a vector representation of a fixed dimension, and performing dimensionality reduction and feature extraction; S113, extracting key frames from the image and video data to perform image enhancement processing, and normalizing and cropping the enhanced key frames to a uniform size; S114, aligning different modal data according to the timestamps to ensure that the modal data correspond to each other at the same time step.

[0040] It should be noted that the execution order of the above steps S111 to S114 is not limited, and S114 may be executed first to perform data alignment and then perform preprocessing operations on each modal data. The preprocessing operations of steps S111 to S113 may also be executed simultaneously.

[0041] Here, missing value filling, outlier processing, normalization and standardization of sensor numerical data can improve detection accuracy and reduce the impact of scale differences on model training. Time alignment and data synchronization ensure that each modality data corresponds at the same time step, thereby facilitating the correct multimodal fusion. In addition, image enhancement methods such as histogram equalization and contrast enhancement are used in image and video preprocessing to improve picture clarity, and the enhanced images are normalized and cropped to a uniform size to adapt to the subsequent Video Swin Transformer model input.

[0042] In some embodiments, in the above step S120, "using the Video Swin Transformer model to extract spatiotemporal features from the pre-processed image and video data" specifically includes the following process steps: S121, slicing the originally collected video data according to a certain time step to obtain a series of video frame sets with a time series relationship; S122, dividing each video frame sequence into three-dimensional blocks in space and time, and performing feature embedding mapping on each block sequence through a linear layer; S123, in the initial layer, the Video Swin Transformer model divides the video features into local windows and calculates multi-head self-attention within the window to capture local spatiotemporal relationships. Then, by merging or sliding windows at higher levels, the feature receptive field is gradually expanded to fuse local features into a global representation. S124, after multi-layer feature extraction and fusion, outputs a high-dimensional spatiotemporal feature vector, and finally maps the spatiotemporal feature vector to a unified embedding dimension through a fully connected layer and a normalization layer, and adds an embedding identifier for identifying the video modality.

[0043] Here, if Figure 2The figure shows the process flow of video feature extraction of Video Swin Transformer. First, the original video is sliced ​​according to a certain time step to obtain a series of video frames {F1, F2, …, FT} with a time series relationship. Each frame image is normalized, color corrected, enhanced and resized to improve the picture quality and the uniformity of the model input. Then, each video frame sequence is divided into blocks in space and time, so that each block sequence (Patch) is represented as an embedded vector of a certain dimension to adapt to the input requirements of the subsequent Transformer structure. At the initial layer, the model divides the video features into local windows and calculates multi-head self-attention within the window to capture the local spatiotemporal relationship. Subsequently, by merging or sliding windows at higher levels, the feature receptive field is gradually expanded, and the local features are fused into a global representation. The hierarchical structure with multiple layers of superposition enables the model to pay attention to rapid changes such as short-term sparks, and also to identify long-term trends such as slow accumulation of coal dust concentration or gradual increase in temperature.

[0044] After multi-layer feature extraction and fusion, Video Swin Transformer outputs a high-dimensional spatiotemporal feature vector (such as 256 or 512 dimensions), which integrates the multi-scale spatiotemporal information of the video sequence in the current time window. To adapt to subsequent multimodal fusion, this spatiotemporal feature vector is mapped to a unified embedding dimension through a fully connected layer and a normalization layer, and an embedding identifier (Modal Embedding) for identifying the video modality is added so that it can be processed in the same fusion framework with numerical sensor features, text features, and geological parameter features.

[0045] In some embodiments, in the above step S120, “using a cross-modal association fusion Transformer model to achieve unified fusion of multimodal features” is implemented through the following process steps: S121, unifying the multi-source heterogeneous data into the same feature dimension and embedding space, and assigning a corresponding modality identifier to each modality, and finally obtaining an aligned multi-modal feature sequence at the same time step.

[0046] Here, H_video extracted by Video Swin Transformer is uniformly processed with numerical sensor features, text features, and prior features, and then synchronously input into the cross-modal association fusion Transformer model.

[0047] At the same time step, the following set of modal eigenvectors {H video ,H sensor ,H text ,H prior}, where H video is the video feature vector, H sensoris the characteristic vector of the numerical sensor, H text is the text feature vector, H prior is a static prior feature vector, and each modal feature vector is a dm vector (or vector sequence). In addition to the conventional time position encoding (Positional Encoding), an independent modal identification vector is added for each mode. The four corresponding modal identification vectors are E video ,E sensor ,E text ,E prior , when the input is video features, H video +E video As Transformer input; the numerical sensor feature is H sensor +E sensor ; The text feature is H text +E text ; The static prior feature is H prior +E prior , thus forming a multimodal feature sequence {H video +E video ,H sensor +E sensor ,H text +E text ,H prior +E prior}, laying the foundation for the subsequent XModal-Transformer input. According to the actual situation, these can be expanded to time series data to form a [T×d] time series feature matrix. If it is an online prediction, the data of the most recent T time steps can be processed in the sliding window.

[0048] In some embodiments, for numerical sensor data, such as gas concentration G, temperature T, oxygen O, coal dust D, ventilation parameter V, pressure P, geological stress S, coal seam thickness Th, coal spontaneous combustion tendency Sp, geological structure complexity Gs, etc., the numerical data is first normalized so that all numerical features are in the same dimension, and the normalized numerical features are mapped to the same embedding space dimension dm as the video features through one or more layers of fully connected networks. During the mapping process, a nonlinear activation function is added to improve the representation ability of the features, and finally a numerical feature embedding vector with a dimension of dm is obtained.

[0049] In some embodiments, for text data such as expert reports and inspection logs (which often contain descriptions of mine environment and equipment status), a self-supervised pre-trained language model is used to encode the text to obtain a sentence-level text embedding vector, which is mapped to the same embedding space dimension dm as the video features and numerical features through a fully connected layer, and a text modality identifier is attached.

[0050] In some embodiments, for static prior data, such as geological structure parameters, coal seam spontaneous combustion tendency index and coal seam thickness, which have a direct impact on disaster-causing factors, the static prior data is linearly transformed and mapped to the dm dimension and a prior modal identifier is attached, so that the model knows that the feature comes from static prior information during multimodal fusion.

[0051] The purpose of this step is to map multi-source heterogeneous data into a unified multi-dimensional feature space, so that subsequent time series modeling and self-attention calculation can fully utilize the semantic and statistical characteristics of each modal information.

[0052] S122, adopting a multi-head self-attention mechanism, focusing on different subspaces and modal interactions of the multimodal feature sequence in parallel through multiple attention heads.

[0053] Here, in order to enhance the expressiveness of the model, a multi-head self-attention mechanism is used to focus on different subspaces and modal interactions of feature sequences through multiple attention heads in parallel. For each layer of TransformerBlock, layer normalization and residual connection strategies are still used to ensure training stability and smooth gradient propagation of deep networks.

[0054] S123, through the superposition of multiple Transformer blocks, the cross-modal correlation fusion Transformer model will deeply fuse the multimodal feature sequence, capture the temporal dependency and inter-modal coupling relationship, and obtain the fused multimodal feature representation at the output layer ; where M is the number of modes, H m is the feature representation of the mth mode, λ m represents the weight factor of the mth mode, satisfying , as a learnable parameter that is automatically adjusted during model training.

[0055] Here, a set of trainable parameters λ is defined m Represents the weight factor of each modality. When multimodal features are fused, the features of different modalities can be weighted summed or jointly modeled. Through continuous iterations in training, the model will adaptively adjust λ m For example, when the video in the mine is clear and the fire source characteristics can be clearly observed, the weight of the video feature may increase; when the text report quality is low, the weight of the text feature may automatically decrease. Thus, through modality embedding and learnable weight tuning, the contribution of each modality to the final feature representation is dynamically adjustable.

[0056] After XModal-Transformer multi-level attention calculation and fusion to capture the temporal dependency and inter-modal coupling relationship, the fused multimodal feature representation H is obtained. This fused feature representation includes: the dynamic evolution information of safety parameters (gas, temperature, oxygen, etc.) from the numerical sensor modality in the time and space dimensions; the implicit semantics of empirical descriptions and expert advice from the text modality; the dynamic changes of scenes and spatial key area features from the video modality; and the long-term geological attributes and coal spontaneous combustion tendency information from the prior modality.

[0057] The analysis of attention weights can explain the decision logic of the model: if the attention weight is concentrated on the characteristics of high-gas and high-temperature areas during high-risk prediction, it means that the model believes that gas-temperature coupling is the main disaster-causing factor; if the ventilation parameter weight is large, it means that the model believes that insufficient ventilation is the key reason for the increase in risk in the current situation; if the text modal feature is highly weighted, it means that the description in the mine inspection report may reveal potential hidden dangers (such as "XX tunnel temperature rises" and "local odor"). This multimodal feature representation provides input features for subsequent risk assessment.

[0058] In some embodiments, the method further includes: introducing a correlation bias matrix B in the multi-head self-attention mechanism to incorporate prior knowledge of disaster-causing factors, and calculating the attention score by the following formula: ; In the formula, is the attention score, is a normalization function, Q, K, and V are query, key, and value matrices, respectively, which are linear transformations of the input features. is the transpose of K; d k is the dimension of the key vector; B is obtained by constructing the association matrix between disaster-causing factors based on the knowledge graph of coal mine fire and gas disaster factors.

[0059] For example, if it is known from expert knowledge or historical data that when gas concentration increases and temperature rises at the same time, the risk of explosion increases significantly, then when constructing B, the element B(Gas, Temp) corresponding to the time series position pair of the two features of gas and temperature is set to a positive value, thereby increasing the self-attention to such associations. B can be initialized as an association matrix constructed based on the knowledge graph of disaster-causing factors and further adjusted during training. The model no longer "discovers" associations from scratch, but learns feature relationships more efficiently under the guidance of prior knowledge.

[0060] In this way, the correlation bias matrix B is added during the self-attention calculation. The size of the correlation bias matrix is ​​(N ，N), N is the total number of disaster-causing factors. Any element in the matrix represents the strength of the prior correlation between two disaster-causing factors, which can be dynamically adjusted during the model training process to embed the prior correlation knowledge into the model, so that the model can capture the nonlinear correlation between key disaster-causing factors faster and more accurately.

[0061] In some embodiments, the above step S130 "mapping the fused multimodal feature representation to the disaster factor index space, calculating the risk assessment index including the quadratic interaction term and dividing the risk level" is implemented by the following steps: S131, the multimodal feature representation is used to obtain multiple disaster factor vectors through dimension reduction and linear mapping, and each factor is normalized to reflect the relative change rate or relative deviation.

[0062] Here, the fused multimodal feature representation H is obtained by dimensionality reduction and linear mapping to obtain the disaster factor vector {f i}, i is a natural number from 1 to N (indicating the total number of disaster-causing factors), assuming N is 10, multiple disaster-causing factor vectors f1, f2, ..., f 10 Including: Gas concentration factor G c , Temperature rise factor T r , oxygen concentration factor O c , coal dust concentration factor D c , ventilation parameter factor V p , pressure change factor P c , geological stress factor S c , Coal spontaneous combustion tendency index factor S p , coal seam thickness factor T h and geological structure complexity factor G s .

[0063] Each factor reflects the relative rate of change or relative deviation after normalization. For example: ; In the formula, G c It is the gas concentration factor, which reflects the change of gas concentration in the mine, and its unit is %, G current Indicates the current gas concentration measurement value, G baseline Indicates the baseline value of gas concentration. Other factors are defined in a similar way to ensure that the parameters input into the risk calculation formula have clear meanings and uniform dimensions.

[0064] S132, setting the weight of each disaster factor vector, and setting the interaction coefficient between any two disaster factors based on expert experience and historical data.

[0065] Here, the weight of each disaster factor is set and ensure In addition, there are often non-linear couplings among disaster-causing factors. For example, when high gas concentration and rising temperature coexist, the explosion risk increases sharply; the temperature rise in a low-oxygen environment may exacerbate the spontaneous combustion of coal. Therefore, the present invention introduces quadratic interaction terms into the risk index. The interaction coefficients of the quadratic interaction terms can be determined by backtracking historical data and expert quantitative evaluation to obtain initial values, and fine-tuning is performed during model training.

[0066] S133, weighted summing the multiple disaster-causing factor vectors based on the weights to obtain a basic risk index, that is, the basic risk index 。

[0067] S134, finally determining the risk assessment index R based on the basic risk index, in combination with the interaction coefficient and the historical disaster frequency.

[0068] Here, the risk assessment index R is specifically calculated by the following formula: ;

[0069] In the formula, R represents the risk assessment index, and the higher the value, the greater the risk; N is the number of disaster-causing factors; f i is the i-th disaster-causing factor vector; β ij is the interaction coefficient between disaster-causing factors i and j, reflecting the gain or weakening effect of the combined action of disaster-causing factors i and j on the risk, and the initial value can be determined by backtracking historical data and expert quantitative evaluation, and fine-tuning is performed during model training; is the influence coefficient of the historical disaster frequency, which is appropriately adjusted according to the statistics of mine accidents in recent years to improve the practical reference value of the risk assessment index.

[0070] S135, comparing and analyzing the risk assessment index R with the risk level thresholds T1, T2, and T3 to divide into four different risk levels: low risk when R < T1, medium risk when T1 ≤ R < T2, high risk when T2 ≤ R < T3, and extremely high risk when R ≥ T3.

[0071] Here, based on the finally calculated risk index R, in combination with the risk level thresholds (T1, T2, T3) set according to the mine historical data and expert experience, the risk is divided into four levels: low, medium, high, and extremely high. It should be noted that different mines can adjust the thresholds according to their own conditions, the distribution of historical disaster data, and expert suggestions. The present invention provides a flexible framework, enabling mine managers to continuously optimize the risk level division criteria according to the actual situation. The risk level thresholds T1, T2, and T3 can be flexibly adjusted according to the mine characteristics, historical data distribution, regional characteristics, and expert suggestions, making the early warning system more in line with the actual situation.

[0072] In some embodiments, the above step S140 of "generating a disaster warning strategy based on risk level analysis and issuing a warning notification" is implemented through the following process: For low-risk levels, monitoring is carried out regularly according to the predetermined plan, and data is continuously collected for real-time calibration of the model. For prevention, sensors and monitoring equipment are regularly inspected and maintained, prevention and control knowledge training is carried out, and inspections are organized to promptly discover and eliminate potential fire sources and gas accumulation points. For notification, risk assessment results and safety conditions are regularly reported to mine management and relevant departments; For the medium-risk level, in terms of monitoring, we will strengthen inspections, focus on areas where disaster-causing factors are close to warning values, focus on monitoring high-risk disaster-causing factors identified by the model, and transmit data in real time to conduct in-depth analysis and identify abnormal trends. In terms of prevention, we will strengthen ventilation, control temperature, rectify hidden dangers, and strengthen equipment inspections. In terms of notification, we will send early warning information to relevant departments for the responsible persons to discuss response measures; For high-risk levels, we activated emergency plans, increased testing frequency, prepared emergency measures, monitored key disaster-causing factors 24 hours a day, and predicted risks through models. We activated response mechanisms in prevention, strengthened gas extraction, and adopted nitrogen and inert gas injection to suppress spontaneous combustion. At the same time, we prohibited personnel from entering and emergency standby. In terms of notification, all mine personnel issued high-risk warning information, emphasizing precautions and prohibited behaviors. For extremely high risk levels, in terms of monitoring, we will continue to monitor disaster-causing factors and obtain real-time data; in terms of prevention, we will shut down equipment, close areas, evacuate all personnel and count the number of people; in terms of notification, we will issue emergency evacuation orders, report to departments, request support, issue announcements and control public opinion through all available channels.

[0073] Here, under the low risk level, the indicators of disaster-causing factors are within the safe range, and all parameters are stable without obvious abnormal changes. At this time, mine production can proceed as planned, and only conventional monitoring and inspection measures need to be maintained. Under the medium risk level, some indicators of disaster-causing factors are close to the warning value, and slight abnormalities appear but are not significant. It is necessary to focus on and strengthen prevention to prevent the accumulation and escalation of risks. Under the high risk level, the indicators of disaster-causing factors obviously exceed the warning value, the risk index increases significantly, and there is potential danger. It is necessary to take strict prevention and control measures and keep emergency forces ready. Under the extremely high risk, the indicators of disaster-causing factors are seriously exceeded, and the risk index is at a high level. Serious disasters are about to occur or are very likely to occur. In this case, evacuation and rescue measures should be organized as soon as possible to minimize the loss of personnel and property. By adopting corresponding response strategies for different levels, it is ensured that coal mine operations can detect potential dangers in a dynamic environment and respond quickly, so as to minimize the probability and loss of disasters.

[0074] The embodiment of the present invention formulates corresponding monitoring, prevention and notification strategies for different risk levels to ensure that in the process of risk evolution from low to high, various response measures are tightened and upgraded step by step, so as to achieve refined management and dynamic response. Figure 3 As shown: 1) Low risk level: The main task at this risk level is to maintain safe operations, ensure data quality, and provide a reliable data source for subsequent model prediction calibration and optimization.

[0075] Monitoring measures: Carry out routine monitoring and regular data updates on key disaster-causing factors such as gas concentration (Gc), temperature (Tr), oxygen concentration (Oc), coal dust concentration (Dc), ventilation parameters (Vp), etc. according to the predetermined plan. Ensure that the deployed sensors, video equipment and text recording systems are in continuous operation and data transmission is unimpeded.

[0076] Preventive measures: Regularly check and maintain sensors, ventilation and extraction equipment to ensure they are in normal working condition. Carry out training on coal mine fire and gas disaster prevention and control to improve employees' safety awareness and response capabilities. Organize daily inspections to promptly eliminate potential hazards such as possible fire sources and gas accumulation points.

[0077] Notification measures: Risk assessment results and safety status will be regularly reported to mine management and relevant departments so that managers can maintain a comprehensive grasp of underground conditions.

[0078] 2) Medium risk level: When the risk index enters the medium risk range, it means that some disaster-causing indicators are close to critical values, and it is necessary to appropriately strengthen monitoring and prevention measures to prevent the risk from further increasing.

[0079] Monitoring measures: Improve inspection frequency and accuracy, and focus on areas with high gas or high temperature, areas with low oxygen concentration, or areas with rapid increase in coal dust concentration. Conduct targeted monitoring of potential risk factors identified in the Video Swin Transformer and XModal-Transformer models, and use high-precision sensors and real-time data transmission systems to conduct more intensive sampling and analysis of abnormal phenomena to quickly capture abnormal trends.

[0080] Preventive measures: Adjust the ventilation system configuration according to the monitoring results, increase the air volume or optimize the air path to reduce the concentration of gas and coal dust and control the temperature rise. For the hidden dangers found (such as local dust accumulation and air leakage points), immediately arrange cleaning or blocking work. Strengthen the inspection and maintenance of electrical and mechanical equipment, strictly control the sparks and overheating during the operation process, and prevent the formation of initial fire or explosion conditions. If necessary, add preventive measures such as water curtain or inert gas injection.

[0081] Notification measures: In addition to regular reports, relevant managers, safety officers and technicians are notified through text messages, mine communication systems and apps, so that the heads of relevant departments can understand the medium-risk status in advance and urge them to make preparations for response. Organize safety coordination meetings and propose targeted prevention and control plans for current risk points.

[0082] 3) High risk level: When the risk index reaches a high risk level, it means that the possibility of a disaster is greatly increased, and more stringent measures need to be taken and emergency plan preparation needs to be entered.

[0083] Monitoring measures: Start the special monitoring mode required by the emergency plan, increase the monitoring density and frequency, and implement 24-hour continuous real-time monitoring of key disaster-causing factors (gas, temperature, oxygen concentration, coal dust concentration). At the same time, introduce a model encryption prediction mechanism to predict the risk development trend in advance by shortening the data refresh interval and increasing the model operation frequency, providing data support for decision-making and emergency action preparation.

[0084] Preventive measures: According to the emergency plan for coal mine fire and gas disasters, activate the corresponding response mechanism. Strengthen gas extraction and reduce the accumulation of gas concentration. For coal seam areas with high tendency of spontaneous combustion, take measures such as nitrogen injection, inert gas injection or pressure control to curb the further maturity of spontaneous combustion conditions. Limit non-essential personnel from entering high-risk areas, and only allow professional technicians and safety monitoring personnel to enter necessary places under strict protection. Mine rescue teams, ventilation workers and professional technical forces are on standby and assemble necessary equipment and supplies in advance.

[0085] Notification measures: Issue high-risk warning information to all mine personnel, strengthen safety inspections and discipline. In accordance with the requirements of mine safety regulations, promptly report risk situations, measures taken and subsequent response plans to local safety supervision departments and relevant superior units, ensure information is open and transparent, and seek external professional support and guidance.

[0086] 4) Extremely high risk: When the risk index reaches an extremely high risk level, a disaster is very likely to occur or is already in a critical state. At this time, emergency evacuation and comprehensive protection measures should be taken without hesitation.

[0087] Monitoring measures: Although the main task at this time is personnel safety and property protection, it is still necessary to continuously monitor the disaster-causing factors to provide real-time data support for rescue decisions. For example, through wireless sensor networks, remote cameras and automated information transmission equipment, continuous attention can be paid to parameter changes at key points underground.

[0088] Preventive measures: Immediately shut down non-essential electrical equipment to avoid secondary disasters caused by electrical sparks. Close and isolate high-risk areas, and set up prominent signs and isolation devices at wellheads and tunnel turns to prevent people from entering by mistake. Under the guidance of the emergency plan, urgently and orderly organize all personnel to evacuate to a safe place. After evacuation, the number of personnel should be counted and confirmed in time to ensure that no one is left out. If conditions permit, emergency disposal measures can be carried out after evacuation (such as injecting inert gas, isolating damaged areas, and building protective walls).

[0089] Notification measures: Rapidly issue emergency evacuation orders through various available channels (including but not limited to: broadcasting, sirens, mine communications, mobile phone text messages, walkie-talkies). Urgently report the situation to the superior management department, local government and emergency management department, and request emergency support. If necessary, make announcements to the media and the public to avoid the spread of rumors or false information and maintain social stability.

[0090] Based on the same inventive concept, an embodiment of the present invention provides a coal mine fire and gas disaster risk prediction system, the system comprising: A data acquisition and preprocessing module is used to collect multi-source heterogeneous data collected at the mine site, and perform data preprocessing and conversion; wherein the multi-source heterogeneous data includes numerical sensor data, image and video data, text data and static prior data, and the static prior data at least includes coal spontaneous combustion tendency index, geological structure parameters, and coal seam thickness; A feature extraction and fusion module, which is used to extract spatiotemporal features from the preprocessed image and video data using a Video SwinTransformer model, and to achieve unified fusion of multimodal features using a cross-modal association fusion Transformer model; The risk assessment and prediction analysis module is used to map the fused multimodal feature representation to the disaster-causing factor index space, calculate the risk assessment index including the quadratic interaction term, and classify the risk level; wherein the interaction coefficient of the quadratic interaction term is set according to expert experience and historical data and fine-tuned during model training; The early warning notification and maintenance management module is used to generate disaster early warning strategies and issue early warning notifications based on risk level analysis, while performing regular maintenance and updates and outputting decision reports.

[0091] Figure 4 The overall framework diagram of the coal mine fire and gas disaster risk prediction system provided by the embodiment of the present invention is as follows: Figure 4As shown in the figure, the data acquisition and preprocessing module obtains the coal spontaneous combustion tendency index, coal seam thickness and geological structure parameters as static prior data by deploying multiple types of sensors in the mine, such as gas sensors (monitoring gas concentration), temperature sensors, oxygen sensors, coal dust sensors, ventilation parameter sensors, pressure sensors, geological stress sensors, image and video recording units, etc. Then, the multi-source heterogeneous data including sensor numerical data, text reports, image and video information, geological structure parameters and coal seam characteristics are cleaned, missing values ​​are filled, outliers are removed, normalized and time-series aligned, and natural language processing is performed on the text information to convert the text into computable numerical embedding.

[0092] The feature extraction and fusion module first uses the Video Swin Transformer to extract spatiotemporal features from image and video data, and then adopts the cross-modal association fusion Transformer model to realize the unified fusion of multimodal features. This module introduces modal embedding, modal identification and modal weight tuning mechanisms, as well as a correlation bias matrix based on prior knowledge, to achieve accurate extraction and association modeling of key disaster-causing factors.

[0093] The risk assessment and prediction analysis module maps the fused multimodal feature representation to the disaster factor indicator space, calculates the risk assessment index R according to the weight and the quadratic interaction coefficient, and divides the risk level based on R.

[0094] The early warning notification and maintenance management module dynamically adjusts the early warning strategy according to the risk level, issues alarm information, indicates emergency measures, regularly maintains and updates model parameters and thresholds, and outputs decision reports. The early warning notification and maintenance management module not only needs to respond and notify in a timely manner, but also needs to continuously iterate, optimize and adjust throughout the life cycle of disaster risk control. Figure 5 shown.

[0095] Model parameter and threshold optimization: modal weights in risk assessment models, interaction coefficients between disaster-causing factors ( β ij ), risk thresholds (T1, T2, T3) can be periodically updated and optimized based on new data. The system can automatically record the actual probability of disasters, the number of false alarms and missed alarms, the magnitude of deviations and their time series characteristics in the background. Based on this feedback information, the model can be retrained and parameters can be fine-tuned to ensure that subsequent predictions are more accurate.

[0096] Data traceability and causal analysis: Using the improved multimodal model and risk index calculation formula, the disaster-causing factors of existing or potential accidents can be traced and analyzed. For example, when it is found that the gas concentration rises sharply in a certain period of time, the system can query the corresponding temperature, oxygen and coal dust data change trends, associate the abnormal patterns identified in the text report and video, analyze the potential causes and propose prevention strategies.

[0097] Generate regular risk assessment reports: The system can generate risk assessment reports regularly (weekly, monthly, quarterly), including the latest risk index, risk level distribution, trend chart of key indicators of disaster-causing factors, statistics on hazard source identification, analysis of the causes of recent risk fluctuations, and summary of the implementation of corresponding response strategies. The report can be submitted to mine management, technical teams, expert groups, and regulatory authorities to provide data-driven references for mid- and long-term safety decisions, equipment upgrades, and policy adjustments.

[0098] The embodiment of the present invention first proposes a method for extracting and fusing features of coal mine fire and gas disaster factors that integrates multimodal data, including uniformly mapping numerical data, text information, image video features, and geological prior parameters to a multimodal fusion feature space. Secondly, by using the Video Swin Transformer and cross-modal association fusion Transformer model structures, modal embedding, modal weight learnable parameters, and prior correlation bias matrix B are introduced into the multi-head self-attention mechanism to accurately capture and characterize the complex associations of key disaster-causing factors. Then, based on the extracted multimodal fusion features, a risk index formula containing quadratic interaction terms is constructed. By considering the nonlinear coupling between disaster-causing factors and the frequency of historical disasters, the risk is quantitatively evaluated and graded for early warning. Finally, the corresponding early warning and emergency strategies are automatically selected and executed according to the risk level, providing a scientific decision-making basis for coal mine safety management and accident prevention and control.

[0099] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0100] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0101] In the several embodiments provided by the present invention, it should be understood that the disclosed methods can be implemented in other ways. The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments. The features disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0102] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for predicting coal mine fire and gas disaster risk, characterized in that: include: Collect multi-source heterogeneous data collected at the mine site, and perform data preprocessing and conversion; wherein the multi-source heterogeneous data includes numerical sensor data, image and video data, text data and static prior data, and the static prior data at least includes coal spontaneous combustion tendency index, geological structure parameters, and coal seam thickness; The Video Swin Transformer model is used to extract spatiotemporal features from the preprocessed image and video data, and the cross-modal association fusion Transformer model is used to achieve unified fusion of multimodal features; The fused multimodal feature representation is mapped to the disaster-causing factor index space, the risk assessment index including the quadratic interaction term is calculated, and the risk level is divided; wherein the interaction coefficient of the quadratic interaction term is set according to expert experience and historical data and fine-tuned during model training; Generate disaster warning strategies and issue warning notices based on risk level analysis, perform regular maintenance and updates, and output decision reports.

2. The method according to claim 1, characterized in that The multi-source heterogeneous data collected at the mine site is collected and preprocessed and converted, including: The outliers in the numerical sensor data are determined and removed, and the missing data in a small range are filled by interpolation. The data are processed by minimum-maximum normalization and Z-score standardization methods to make the dimensions of each feature consistent. Perform word segmentation and embedding on text data, convert the text into a vector representation of fixed dimension, and perform dimensionality reduction and feature extraction; Extract key frames from image and video data for image enhancement, and normalize and crop the enhanced key frames to a uniform size; Align different modal data according to timestamps to ensure that each modal data corresponds to the same time step.

3. The method according to claim 1, characterized in that: The video swin transformer model is used to extract spatiotemporal features from the preprocessed image and video data, including: The original collected video data is sliced ​​according to a certain time step to obtain a series of video frames with a time series relationship; Each video frame sequence is divided into three-dimensional blocks in space and time, and feature embedding mapping is performed on each block sequence through a linear layer; In the initial layer, the Video Swin Transformer model divides the video features into local windows and calculates multi-head self-attention within the window to capture local spatiotemporal relationships. It then gradually expands the feature receptive field by merging or sliding windows at higher levels to fuse local features into a global representation. After multi-layer feature extraction and fusion, a high-dimensional spatiotemporal feature vector is output. Finally, the spatiotemporal feature vector is mapped to a unified embedding dimension through a fully connected layer and a normalization layer, and an embedding identifier for identifying the video modality is added.

4. The method according to claim 3, characterized in that The cross-modal association fusion Transformer model is used to realize the unified fusion of multi-modal features, including: Unifying the multi-source heterogeneous data into the same feature dimension and embedding space, and assigning a corresponding modality identifier to each modality, and finally obtaining an aligned multi-modal feature sequence at the same time step; Adopt the multi-head self-attention mechanism, and use multiple attention heads to concurrently focus on different subspaces of the multi-modal feature sequence and modal interactions; By stacking multiple Transformer blocks, the cross-modal correlation fusion Transformer model will deeply fuse the multimodal feature sequence, capture the temporal dependency and inter-modal coupling relationship, and obtain the fused multimodal feature representation at the output layer. ; where M is the number of modes, H m is the feature representation of the mth mode, λ m represents the weight factor of the mth mode, satisfying , as a learnable parameter that is automatically adjusted during model training.

5. The method according to claim 4, characterized in that The method further includes: In the multi-head self-attention mechanism, introduce the relevance bias matrix B to incorporate the prior knowledge of disaster-causing factors, and calculate the attention scores through the following formula: ； In the formula, is the attention score, is a normalization function, Q, K, and V are query, key, and value matrices, respectively, which are linear transformations of the input features. is the transpose of K; d k is the dimension of the key vector; B is obtained by constructing the association matrix between disaster-causing factors based on the knowledge graph of coal mine fire and gas disaster factors.

6. The method according to claim 4, characterized in that The unifying of the multi-source heterogeneous data into the same feature dimension and embedding space, and assigning corresponding modal identifiers to each modality includes: Perform normalization processing on the numerical data to make all numerical features in the same dimension. Map the normalized numerical features to the same embedding space dimension dm as the video features through one or more fully connected networks. During the mapping process, add a non-linear activation function to improve the feature representation ability, and finally obtain a numerical feature embedding vector with a dimension of dm; For text data, use a self-supervised pre-trained language model to encode the text to obtain a sentence-level text embedding vector, and map it to the same embedding space dimension dm as the video features and numerical features through a fully connected layer, while attaching a text modality identifier; Perform a linear transformation on the static prior data and map it to the dm dimension, while attaching a prior modality identifier.

7. The method according to any one of claims 1 to 6, characterized in that: The mapping of the fused multi-modal feature representation to the disaster-causing factor index space, calculating the risk assessment index including quadratic interaction terms and dividing the risk levels includes: Obtain multiple disaster-causing factor vectors by dimensionality reduction and linear mapping of the multi-modal feature representation, and each factor reflects the relative change rate or relative deviation degree after normalization; among them, the multiple disaster-causing factor vectors include: gas concentration factor, temperature rise factor, oxygen concentration factor, coal dust concentration factor, ventilation parameter factor, pressure change factor, geological stress factor, coal spontaneous combustion tendency index factor, coal seam thickness factor, and geological structure complexity factor; Set the weight of each disaster-causing factor vector, and set the interaction coefficient between any two disaster-causing factors according to expert experience and historical data; Perform a weighted sum of the multiple disaster-causing factor vectors based on the weights to obtain a basic risk index; Based on the basic risk index, combine the interaction coefficient and the historical disaster frequency to finally determine the risk assessment index R; Compare and analyze the risk assessment index R with the risk level thresholds T1, T2, and T3 to divide into four different risk levels: low risk when R < T1, medium risk when T1 ≤ R < T2, high risk when T2 ≤ R < T3, and extremely high risk when R ≥ T3.

8. The method according to claim 7, characterized in that The generating of a disaster warning strategy based on the risk level analysis and issuing a warning notice includes: For the low risk level, in terms of monitoring, conduct regular monitoring and continuous collection according to a predetermined plan for real-time calibration of the model. In terms of prevention, regularly inspect and maintain sensors and monitoring equipment, carry out prevention and control knowledge training, organize patrols to timely detect and eliminate potential fire sources and gas accumulation points. In terms of notification, regularly notify the risk assessment results and safety status to the mine management and relevant departments; For the medium-risk level, in terms of monitoring, we will strengthen inspections, focus on areas where disaster-causing factors are close to warning values, focus on monitoring high-risk disaster-causing factors identified by the model, and transmit data in real time to conduct in-depth analysis and identify abnormal trends. In terms of prevention, we will strengthen ventilation, control temperature, rectify hidden dangers, and strengthen equipment inspections. In terms of notification, we will send early warning information to relevant departments for the responsible persons to discuss response measures; For high-risk levels, we activated emergency plans, increased testing frequency, prepared emergency measures, monitored key disaster-causing factors 24 hours a day, and predicted risks through models; in terms of prevention, we activated response mechanisms, strengthened gas extraction, and adopted nitrogen and inert gas injection to suppress spontaneous combustion, while prohibiting personnel from entering and emergency standby; in terms of notification, all mine personnel issued high-risk warning information, emphasizing precautions and prohibited behaviors; For extremely high risk levels, in terms of monitoring, we will continue to monitor disaster-causing factors and obtain real-time data; in terms of prevention, we will shut down equipment, close areas, evacuate all personnel and count the number of people; in terms of notification, we will issue emergency evacuation orders, report to departments, request support, issue announcements and control public opinion through all available channels.

9. A coal mine fire and gas disaster risk prediction system, characterized in that: include: A data acquisition and preprocessing module is used to collect multi-source heterogeneous data collected at the mine site, and perform data preprocessing and conversion; wherein the multi-source heterogeneous data includes numerical sensor data, image and video data, text data and static prior data, and the static prior data at least includes coal spontaneous combustion tendency index, geological structure parameters, and coal seam thickness; A feature extraction and fusion module, which is used to extract spatiotemporal features from the preprocessed image and video data using a Video SwinTransformer model, and to achieve unified fusion of multimodal features using a cross-modal association fusion Transformer model; The risk assessment and prediction analysis module is used to map the fused multimodal feature representation to the disaster-causing factor index space, calculate the risk assessment index including the quadratic interaction term, and classify the risk level; wherein the interaction coefficient of the quadratic interaction term is set according to expert experience and historical data and fine-tuned during model training; The early warning notification and maintenance management module is used to generate disaster early warning strategies and issue early warning notifications based on risk level analysis, while performing regular maintenance and updates and outputting decision reports.

Images