Coal mine fire hazard identification method based on size model fusion

Abstract

The application provides a coal mine fire hazard identification method based on size model fusion, comprising: acquiring underground multi-source monitoring data; performing feature detection on the multi-source monitoring data through a pre-trained lightweight target detection network, and extracting a local data slice and sending the local data slice to an edge computing gateway when an abnormal target feature is identified; wherein the lightweight target detection network is deployed on an underground end-side monitoring device; performing cross-modal cross-validation on the local data slice through a pre-trained lightweight large model, and if the validation confirms that the abnormal target feature has a fire hazard attribute, generating hazard confirmation information and sending the hazard confirmation information to a cloud data center, so that the cloud data center combines global monitoring feature data to perform deep correlation reasoning on the hazard confirmation information and outputs a comprehensive early warning report; wherein the lightweight large model is deployed on the edge computing gateway. The scheme of the application can improve the real-time performance and accuracy of underground fire hazard monitoring.

Description

Technical Field

[0001] This application relates to the field of coal mine disaster monitoring technology, and in particular to a method, device, equipment and storage medium for identifying coal mine fire hazards based on the fusion of large and small models. Background Technology

[0002] Coal mine fires are highly concealed and spread rapidly. The complex environment underground, including low light, high dust levels, high humidity, and electromagnetic interference, makes early identification of fire hazards extremely difficult. Currently, underground fire monitoring at relevant technical centers primarily relies on fixed sensor alarms, bundled tube monitoring, and manual inspections. These methods generally suffer from slow response times and are susceptible to subjective influences. Summary of the Invention

[0003] This application aims to at least partially address one of the technical problems in the related art.

[0004] Firstly, this application proposes a coal mine fire hazard identification method based on large-scale model fusion. The method includes: acquiring multi-source monitoring data underground; performing feature detection on the multi-source monitoring data using a pre-trained lightweight target detection network, and extracting local data slices when abnormal target features are identified, and sending the local data slices to an edge computing gateway; wherein the lightweight target detection network is deployed on underground end-side monitoring equipment; performing cross-modal cross-validation on the local data slices using a pre-trained lightweight large-scale model, and if the verification confirms that the abnormal target features have fire hazard attributes, generating hazard confirmation information and sending the hazard confirmation information to a cloud data center, so that the cloud data center can combine global monitoring feature data to perform deep correlation reasoning on the hazard confirmation information and output a comprehensive early warning report; wherein the lightweight large-scale model is deployed on the edge computing gateway.

[0005] In one implementation, the multi-source monitoring data includes a continuous video frame sequence and sensor sampling values. Before performing feature detection on the multi-source monitoring data through a pre-trained lightweight target detection network, the method further includes: extracting sensor sampling values ​​within a preset time sliding window corresponding to the current video frame in the continuous video frame sequence, calculating the arithmetic mean, and generating a synchronization value bound to the current video frame; if no new sensor sampling values ​​are obtained within the preset time sliding window, a zero-order hold strategy is adopted to use historical values ​​from the previous effective period for state filling.

[0006] In one implementation, the lightweight target detection network includes a feature extraction backbone network and a convolutional block attention mechanism module connected in series. The feature extraction backbone network includes some convolutional layers, and the convolutional block attention mechanism module includes a channel attention submodule and a spatial attention submodule. The step of performing feature detection on the multi-source monitoring data using the pre-trained lightweight target detection network includes: inputting the fused tensor data into the feature extraction backbone network and extracting spatial features using the some convolutional layers; inputting the spatial features into the channel attention submodule to perform feature enhancement, obtaining an intermediate feature map; inputting the intermediate feature map into the spatial attention submodule to perform feature enhancement again, obtaining an enhanced feature map; performing prediction based on the enhanced feature map, and outputting a prediction result including a confidence score and a classification label; comparing the confidence score with a preset visual judgment threshold; when the confidence scores of a consecutive preset number of video frames all exceed the preset visual judgment threshold, and the unit-time change rate of the corresponding synchronization value exceeds the limit, the abnormal target feature is identified, and an abnormal trigger signal is generated.

[0007] In one implementation, the step of performing cross-modal cross-validation on the local data slice using a pre-trained lightweight large model includes: parsing the local data slice, extracting the feature jump types of the numerical sequences of multi-source monitoring data, and converting the feature jump types into text prompts using an environment rule engine; extracting the time-window video slices and multi-scale visual feature maps of the time-window video slices in the local data slices, and alternately performing intra-modal self-attention interaction and cross-modal cross-attention interaction between the visual features of the time-window video slices and the text prompts using a cross-modal feature enhancement layer to obtain a fused feature stream; selecting candidate target queries from the multi-scale visual feature maps using the feature vectors of the text prompts, performing decoding prediction in combination with the fused feature stream, and outputting the image-text matching confidence score for the abnormal target features.

[0008] In one optional implementation, the step of generating hazard confirmation information if the verified abnormal target feature has fire hazard attributes includes: comparing the image-text matching confidence level with a preset review and judgment threshold; if the image-text matching confidence level is lower than the preset review and judgment threshold, determining that the abnormal target feature belongs to physical environmental interference conditions, generating a false alarm interception command and blocking the data transmission link; if the image-text matching confidence level is greater than or equal to the preset review and judgment threshold, determining that the abnormal target feature has the fire hazard attributes, extracting the classification label from the local data slice as the hazard type, extracting the physical location coordinates, and attaching the verified label to generate the hazard confirmation information.

[0009] In one implementation, the step of performing deep correlation reasoning on the hazard confirmation information by combining global monitoring feature data includes: using the physical location coordinates in the hazard confirmation information as spatial anchor points, obtaining global monitoring feature data that have topological connectivity with the spatial anchor points within the entire mine area; using a graph convolutional network to perform aggregation and transmission of feature information between nodes, and extracting spatial diffusion features; converting the spatial diffusion features and the hazard types in the hazard confirmation information into structured query statements, performing vector retrieval in a pre-built coal mine disaster mechanism graph knowledge base, and matching historical disaster evolution paths as prior references.

[0010] In one optional implementation, the output of the comprehensive early warning report includes: extracting extreme values ​​of gas concentration at adjacent nodes and ventilation fan operating status parameters from global monitoring feature data; calculating the spatial distance attenuation of the extreme values ​​of gas concentration at adjacent nodes in conjunction with the spatial anchor points; and converting the ventilation fan operating status parameters into a spread penalty gain; combining the image-text matching confidence level, the spatial distance attenuation, and the spread penalty gain into a multi-source real-time risk factor; performing a weighted calculation on the prior reference and the multi-source real-time risk factor to output a comprehensive risk rating index; and inputting the comprehensive risk rating index, the hazard type, and the global monitoring feature data into a preset causal language model for prediction and generation, and outputting the comprehensive early warning report.

[0011] In a second aspect, this application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0012] Thirdly, this application proposes a storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect.

[0013] Fourthly, this application proposes a program product comprising at least one of a program and instructions, wherein when the program and instructions are executed by an electronic device, they implement the steps of the method described in the first aspect.

[0014] This application provides a coal mine fire hazard identification method, equipment, and storage medium based on large and small model fusion. It adopts a cloud-edge-device hierarchical collaborative underground fire monitoring architecture. First, it acquires multi-source monitoring data from underground. A lightweight target detection network deployed on the underground edge monitoring equipment performs preliminary feature detection. When abnormal target features are identified, local data slices are extracted and uploaded to the edge computing gateway. Then, a lightweight large model deployed on the edge gateway performs cross-modal cross-validation on the local data slices. After confirming that the anomaly possesses fire hazard attributes, hazard confirmation information is generated and sent to the cloud data center. Finally, the cloud combines global monitoring feature data to perform deep correlation inference and output a comprehensive early warning report. This approach ensures real-time monitoring response through edge-side initial screening, significantly reducing the bandwidth load of uploading full monitoring data. Furthermore, the cross-modal validation of the lightweight large model on the edge effectively weakens environmental interference such as underground dust, water mist, and mine lamp reflections, reducing the false alarm rate. Simultaneously, the cloud-based global deep inference improves the comprehensiveness and accuracy of the early warning, achieving a balance between real-time monitoring and identification accuracy for underground fire hazard hazards.

[0015] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0016] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0017] Figure 1 This is a schematic diagram of the first embodiment provided in this application; Figure 2 This is a schematic diagram of the second embodiment provided in this application; Figure 3 This is a schematic diagram of the structure of a coal mine fire hazard identification system based on size model fusion provided in this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0019] The following describes, with reference to the accompanying drawings, a method and apparatus for identifying coal mine fire hazards based on size model fusion, according to embodiments of this application.

[0020] Figure 1This is a flowchart illustrating a method for identifying coal mine fire hazards based on size model fusion, as provided in an embodiment of this application. Figure 1 As shown, the method may include, but is not limited to, the following steps: S100: Acquire downhole multi-source monitoring data.

[0021] For example, downhole monitoring equipment can be used to acquire real-time data on carbon monoxide and ethylene concentrations in the well, while simultaneously recording on-site video images and equipment motor operating parameters as multi-source monitoring data.

[0022] In some embodiments, before performing the candidate steps, the acquired multi-source monitoring data may also be standardized through the following steps. A1: Construct a baseline rule mapping relationship for the spontaneous combustion characteristics of coal.

[0023] For example, to achieve early structured qualitative assessment of fire hazards, the acquired fire gas data is compared with preset critical thresholds to help determine the degree of spontaneous combustion in coal. The critical threshold can be set based on the characteristic gas reports of spontaneous combustion in specific coal seams and can be dynamically configured.

[0024] For example, when the CO concentration reaches 40 ppm, a marker for the slow oxidation stage of the coal is generated; when C2H4 gas is detected in the monitoring data, a marker for the accelerated oxidation stage of the coal is generated; and when C2H2 gas is detected in the monitoring data, a marker for the combustion stage of the coal is generated. These baseline rule mapping relationships are converted into structured text and stored as prior knowledge, serving as a reference for subsequent logical reasoning in the cloud data center.

[0025] A2: Perform numerical standardization and time-series alignment preprocessing on multi-source monitoring data.

[0026] For example, due to differences in physical principles, multi-source sensors exhibit inconsistent output dimensions and sampling frequencies. To establish a basic input format suitable for direct reading by neural network models, a hard boundary truncation is first performed based on the hardware range of the physical sensors. Subsequently, the Laida criterion (3σ principle) or interquartile range (IQR) algorithm based on normal distribution is used to further eliminate non-physical spikes and outliers in the measurement sequence caused by sudden factors such as electromagnetic interference. Furthermore, deviation standardization processing logic is employed to normalize the cleaned sensor sequence.

[0027] Based on the completion of spatial domain numerical unification, a time sliding window can be established to achieve temporal alignment of multi-source heterogeneous data, taking into account the differences in hardware sampling frequencies between video units and various gas sensors in downhole end-side monitoring equipment.

[0028] For example, since the video sampling rate is much higher than the response frequency of the gas detection device, the arithmetic mean of all sensor sampling values ​​within a preset time sliding window is extracted, starting from the time reference of the continuous video frames received by the device's internal processing chip. If new valid sensor data is not captured within this time window due to communication delays, a zero-order hold strategy is adopted, using historical values ​​from the previous valid period to fill in the state, thereby avoiding computational interruptions caused by data loss. This average value serves as a synchronization value bound to the current video frame time.

[0029] A3: Perform denoising and image enhancement processing on the raw image frames acquired by the video device.

[0030] For example, by reconstructing the grayscale distribution and contrast relationship of pixel regions, pixel blurring interference caused by suspended dust particles and insufficient light in the environment can be reduced. For the specific algorithm implementation of video frame denoising and contrast enhancement, those skilled in the art can use conventional image processing techniques such as median filtering and contrast-limited adaptive histogram equalization. The specific computational process is well-known in the field and will not be elaborated here. The video frame sequence, after standardization and temporal alignment, is synchronized with sensor values ​​and finally combined into a fused tensor data with a multi-dimensional feature matrix format.

[0031] S200: Performs feature detection on multi-source monitoring data through a pre-trained lightweight target detection network, and extracts local data slices when abnormal target features are identified and sends the local data slices to the edge computing gateway.

[0032] The lightweight target detection network is deployed on the downhole end-side monitoring equipment. The lightweight target detection network includes a feature extraction backbone network and a convolutional block attention mechanism module connected in series. The feature extraction backbone network contains some convolutional layers, and the convolutional block attention mechanism module includes a channel attention submodule and a spatial attention submodule.

[0033] In one implementation, a pre-trained lightweight target detection network performs feature detection on multi-source monitoring data, including: inputting fused tensor data into a feature extraction backbone network and extracting spatial features using some convolutional layers; inputting the spatial features into a channel attention submodule to perform feature enhancement, obtaining an intermediate feature map; inputting the intermediate feature map into a spatial attention submodule to perform feature enhancement again, obtaining an enhanced feature map; performing prediction based on the enhanced feature map, and outputting a prediction result containing a confidence score and a classification label; comparing the confidence score with a preset visual judgment threshold; when the confidence scores of a preset number of consecutive video frames all exceed the preset visual judgment threshold, and the unit time change rate of the corresponding synchronization value exceeds the limit, an abnormal target feature is identified, and an abnormal trigger signal is generated.

[0034] For example, in the feature extraction backbone network, the original C2f module is modified by introducing some convolutional layers to replace the original standard convolutional layers, reconstructing it into a C2f-PConv module. During computation, these convolutional layers only perform spatial feature extraction on a subset of the input feature map channels, retaining the feature information of the remaining channels without participating in the current convolution calculation. After spatial feature extraction, the two feature subsets are concatenated along the channel dimension. This network structure can reduce the floating-point computation and memory access frequency of the backbone network while extracting fire textures. Building on this, a lightweight hybrid convolution (GSConv) is introduced in the feature fusion neck network to replace conventional convolutions, processing the concatenated feature map and further compressing the model parameter volume while maintaining channel information fusion interaction. To address the image feature blurring caused by high dust suspension and low-light infrared illumination in underground mines, this lightweight network connects a convolutional block attention mechanism module in series between the backbone network and the detection head.

[0035] The module comprises a channel attention submodule and a spatial attention submodule, with network data flow exhibiting a sequential processing logic. Channel attention aims to assign higher weights to feature channels containing fire color and brightness, while spatial attention focuses on the core pixel region where the fire occurs. After the input feature map enters the channel attention submodule, it undergoes global average pooling and global max pooling operations to compress spatial dimension information. Subsequently, these two compressed features are input into a multilayer perceptron network with shared weights to extract deep channel interaction relationships. The two outputs of the multilayer perceptron are summed element-wise, and the result is mapped to a channel weight matrix in the zero-to-one range using an activation function. This weight matrix is ​​then multiplied element-wise with the original feature map to obtain an intermediate feature map with enhanced channel features. Based on the generated channel weights, the intermediate feature map enters the spatial attention submodule. Average pooling and max pooling operations are performed again along the channel dimension, and the outputs of the two submodules are concatenated along the channel dimension.

[0036] Next, a 7×7 conventional convolutional kernel is used to perform dimensionality reduction and spatial feature aggregation on the concatenated feature map, which is then processed by an activation function to generate a spatial weight matrix. Finally, this spatial weight matrix is ​​multiplied by the intermediate feature map to obtain the final enhanced feature map, which is then fed into the detection head for regression prediction.

[0037] The model's detection head outputs the two-dimensional coordinates of the predicted bounding box, the classification label of the target category, and the corresponding confidence score. The classification labels cover open flame, smoke, and overheating of mechanical equipment surfaces. The model's output confidence score is compared in real-time with a preset visual judgment threshold, typically set between 0.65 and 0.85, dynamically adjusted based on the specific lighting conditions in different mines. To avoid false alarms in a single frame caused by flying insects or sudden bursts of strong light underground, a time-based anti-shake mechanism can be introduced, confirming a visual anomaly only when the confidence score of 3 to 5 consecutive video frames exceeds this threshold.

[0038] Simultaneously, by comparing the sensor's synchronized values ​​with the set environmental safety parameter baseline, an anomaly trigger signal is immediately generated when the rate of change of CO concentration per unit time or the temperature value exceeds the limit. Regarding the initial construction and training process of this lightweight network, historical images containing real underground fires and simulated fire experiments were used as the sample set. The sample labels consisted of real bounding boxes of flames and smoke manually labeled by operational personnel. For training configuration, the optimizer adopted the AdamW algorithm, with an initial learning rate set to 0.001. The loss function during training combined the perfect intersection-union (CIoU) bounding box regression loss to measure location deviation, the target confidence loss based on binary cross-entropy (BCE), and the classification loss. After offline training and structured pruning were completed in the cloud data center, the network weights of the edge model were converted into an inference engine format file adapted to the edge hardware architecture and deployed.

[0039] S300: Performs cross-modal cross-validation on local data slices using a pre-trained lightweight large model. If the verification confirms that the abnormal target features have fire hazard attributes, it generates hazard confirmation information and sends the hazard confirmation information to the cloud data center, so that the cloud data center can combine global monitoring feature data to perform deep correlation reasoning on the hazard confirmation information and output a comprehensive early warning report.

[0040] The lightweight large model is deployed on the edge computing gateway.

[0041] In one implementation, please refer to Figure 2 , Figure 2 This is a schematic diagram of the second embodiment provided in this application. Figure 2 As shown, step S300 may include the following steps: S301: Parse local data slices, extract the characteristic jump types of numerical sequences of multi-source monitoring data, and convert the characteristic jump types into text prompts through the environmental rule engine.

[0042] For example, the edge computing gateway first performs structured depacketization on the received local data slices, separating the numerical sequence of multi-source monitoring data and the video slice of the time window aligned with the abnormal time window; then it analyzes the numerical change trend of the detection data within the time window, identifies the time node where the numerical change occurs, and then maps the purely numerical jump features into structured text prompts according to the preset coal industry disaster judgment rules through the built-in environmental rule engine. For example, a sharp increase in carbon monoxide concentration corresponds to the semantics of "coal oxidation, abnormal smoke", and the ambient temperature crossing the safety baseline corresponds to the semantics of "equipment overheating, open flame hazard", thus completing the semantic conversion of physical measurement data to the natural language dimension.

[0043] S302: Extract the multi-scale visual feature maps of time window video slices from the local data slices, and use the cross-modal feature enhancement layer to alternately perform intra-modal self-attention interaction and cross-modal cross-attention interaction between the visual features of the time window video slices and the text prompts to obtain the fused feature stream.

[0044] For example, video slices within a time window are input into an image backbone network for feature extraction, outputting a multi-scale visual feature map containing spatial structure information. Simultaneously, the generated text prompts are input into a text backbone network for encoding, yielding text lexical feature vectors carrying sequence contextual relationships. Both types of features are then input into a cross-modal feature enhancement layer, where self-attention interaction and cross-modal cross-attention interaction are performed, outputting a fused feature stream that completes deep modality alignment.

[0045] S303: Select candidate target queries from multi-scale visual feature maps using the feature vectors of text prompts, combine them with the fused feature stream for decoding and prediction, and output the image-text matching confidence score for abnormal target features.

[0046] For example, guided by the feature vector of the text prompt, the pixel region with the highest response value is selected from the multi-scale visual feature map as the candidate target query. The candidate target query and the fused feature stream are then input into the cross-modal decoder. Through the iterative update of the multi-layer neural network, the position and category matching degree of the candidate target are gradually refined. Finally, the predicted bounding box for the abnormal target feature and the corresponding image-text matching confidence score are output.

[0047] In one optional implementation, if the verified abnormal target features are confirmed to have fire hazard attributes, hazard confirmation information is generated, including: comparing the image-text matching confidence level with a preset review and judgment threshold; if the image-text matching confidence level is lower than the preset review and judgment threshold, the abnormal target features are determined to be subject to physical environmental interference conditions, a false alarm interception command is generated, and the data transmission link is blocked; if the image-text matching confidence level is greater than or equal to the preset review and judgment threshold, the abnormal target features are determined to have fire hazard attributes, the classification label in the local data slice is extracted as the hazard type, the physical location coordinates are extracted, and the verified label is attached to generate hazard confirmation information.

[0048] In one implementation, combining global monitoring feature data to perform deep correlation reasoning on hazard confirmation information may include the following steps: B1: Using the physical location coordinates in the hazard confirmation information as spatial anchor points, obtain global monitoring feature data that have topological connectivity with the spatial anchor points within the entire mine area.

[0049] For example, the physical location coordinates in the hazard confirmation information are used as spatial anchor points. Based on the topological relationship between mine roadway connectivity and airflow, global monitoring feature data of all monitoring nodes that are topologically connected to the anchor point within the entire mine are retrieved.

[0050] B2: Utilize graph convolutional networks to aggregate and transmit feature information between nodes, extracting spatial diffusion features; convert the spatial diffusion features and hazard types in hazard confirmation information into structured query statements, perform vector retrieval in a pre-built coal mine disaster mechanism graph knowledge base, and match historical disaster evolution paths as prior references.

[0051] For example, a mine spatial topology graph is constructed with monitoring points as graph nodes and roadway connections as graph edges. A graph convolutional network is used to aggregate and transmit feature information between nodes along the roadway connection path to obtain the diffusion pattern of hazard features along the spatial path and extract spatial diffusion features that characterize the spread trend of hazard. Finally, the spatial diffusion features and the hazard types in the hazard confirmation information are converted into a structured query statement. Vector similarity retrieval is performed in a pre-constructed coal mine disaster mechanism graph knowledge base to match the historical disaster evolution path and typical treatment cases that have the highest fit with the current working conditions, which serve as prior reference data for subsequent comprehensive risk assessment.

[0052] In one optional implementation, a comprehensive early warning report is output, including: The system extracts extreme values ​​of gas concentration at adjacent nodes and ventilation fan operating status parameters from global monitoring feature data. It then calculates the spatial distance attenuation of extreme gas concentration at adjacent nodes using spatial anchor points and converts the ventilation fan operating status parameters into a spread penalty gain. The system combines the confidence level of image-text matching, spatial distance attenuation, and spread penalty gain into a multi-source real-time risk factor. It performs a weighted calculation on the prior reference and the multi-source real-time risk factor to output a comprehensive risk rating index. Finally, it inputs the comprehensive risk rating index, hazard type, and global monitoring feature data into a pre-set causal language model for prediction and generation, and outputs a comprehensive early warning report.

[0053] For example, firstly, the extreme values ​​of gas concentration and the operating status parameters of ventilation fans are extracted from the global monitoring feature data of nodes topologically adjacent to the spatial anchor point of the hidden danger. Combined with the coordinates of the spatial anchor point, the attenuation of the extreme values ​​of gas concentration of adjacent nodes with the spatial distance of the roadway is calculated to quantify the dilution effect of physical space on abnormal gases. At the same time, the wind speed, wind direction and operating condition parameters of the ventilation fan are converted into spread penalty gain to quantify the boosting or inhibiting effect of airflow dynamics on the spread of fire hazards and toxic gases. Then, the confidence of image-text matching, spatial distance attenuation and spread penalty gain output from the edge side are combined to form a multi-source real-time risk factor covering local verification intensity, spatial diffusion trend and environmental dynamic influence. Then, the historical disaster evolution prior reference obtained above and the multi-source real-time risk factor are weighted and fused according to the preset weights to output a quantifiable and gradeable comprehensive risk rating index. Finally, the comprehensive risk rating index, the type of hidden danger and the global monitoring feature data of all dimensions are input into a preset causal language model. The model is combined with the logic of coal mine disaster mechanism to generate a comprehensive early warning report that includes analysis of the cause of hidden danger, prediction of spread trend, judgment of risk level and targeted treatment suggestions.

[0054] The coal mine fire hazard identification method based on large and small model fusion, as described in this application, can acquire multi-source monitoring data from underground. A lightweight target detection network deployed on underground end-side monitoring equipment performs preliminary feature detection. When abnormal target features are identified, local data slices are extracted and uploaded to an edge computing gateway. Then, a lightweight large model deployed on the edge gateway performs cross-modal cross-validation on the local data slices. After confirming that the anomaly possesses fire hazard attributes, hazard confirmation information is generated and sent to the cloud data center. Finally, the cloud combines global monitoring feature data to perform deep correlation reasoning and output a comprehensive early warning report. This method ensures real-time monitoring response through initial screening at the end-side, significantly reducing the bandwidth load of uploading full monitoring data. Furthermore, the cross-modal validation using the lightweight large model at the edge effectively weakens environmental interference such as underground dust, water mist, and mine lamp reflections, reducing the false alarm rate. Simultaneously, the cloud-based global deep reasoning enhances the comprehensiveness and accuracy of the early warning, achieving a balance between real-time monitoring and identification accuracy for underground fire hazard hazards.

[0055] The present application further provides a coal mine fire hazard identification system based on large and small model fusion. As an example, please refer to Figure 3 , Figure 3 which is a schematic structural diagram of the coal mine fire hazard identification system based on large and small model fusion provided by the present application. As shown in Figure 3 , the system may include: an underground end-side monitoring device, an edge computing gateway, a cloud data center, and a linkage disposal system.

[0056] The identification system adopts an end-edge-cloud collaborative distributed hardware architecture. The underground end-side monitoring device is deployed in underground environment monitoring areas such as goafs and roadway production areas of coal mines. This device not only integrates video monitoring equipment, a fire gas monitoring system, ambient temperature sensors and equipment operation state monitoring units for multi-source data perception, but also has a built-in mine-used explosion-proof intelligent analysis terminal with edge computing capability. The terminal is equipped with a built-in low-power artificial intelligence hardware acceleration chip, which is specially used to run a small coal mine fire monitoring model with a relatively low parameter quantity.

[0057] The edge computing gateway is deployed in a graphics processing unit server at a mine edge computing network node or a wellhead machine room. The edge computing gateway is located at an intermediate network node between the underground end-side monitoring device and the cloud data center, and is configured to receive local data slices uploaded by the underground end-side monitoring device through a local area network. The gateway runs a lightweight vertical large model for the coal industry, uses computing power resources to perform cross-modal data cross-comparison calculation tasks, and reduces bandwidth occupation when the system transmits data to the backbone network.

[0058] The cloud data center is deployed in a distributed computing power cluster configured in a ground scheduling center. This computing power cluster has massive data storage capability and parallel computing capability, and runs a vertical large model for the coal industry with a parameter quantity at the level of tens of billions. The cluster is connected to the mine industrial ring network, and is responsible for receiving structured data aggregated by upstream nodes, and performing tasks including multi-modal feature joint extraction, disaster logic reasoning, and iterative update of global model parameters. The linkage disposal system is in communication connection with the cloud data center, and is responsible for receiving early warning reports and control instructions, and driving mine physical alarm devices and control units such as ventilation units to perform on-site safety linkage actions.

[0059] It should be noted that the foregoing explanations and descriptions of the embodiments of the coal mine fire hazard identification method based on large and small model fusion are also applicable to the coal mine fire hazard identification system based on large and small model fusion in this embodiment, and will not be repeated here.

[0060] To implement the foregoing embodiments, the present application further proposes an electronic device. Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of the electronic device provided in the embodiment of the present application. As shown in Figure 4As shown, the electronic device 400 includes: a processor 401 and a memory 402 communicatively connected to the processor 401; the memory 402 stores computer-executable instructions; the processor 401 executes the computer-executable instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0061] To implement the above embodiments, this application also proposes a storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the methods provided in the foregoing embodiments.

[0062] To implement the above embodiments, this application also proposes a program product, including at least one of a program and instructions, wherein when the program and instructions are executed by an electronic device, they implement the steps of the method provided in the foregoing embodiments.

[0063] It should be noted that the acquisition, transmission, storage, use, and processing of data in this application comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.

[0064] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0065] It is worth noting that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0066] In the description of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.

[0067] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0068] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0069] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0070] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0071] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0072] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0073] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0074] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for identifying coal mine fire hazards based on size model fusion, characterized in that, include: Acquire downhole multi-source monitoring data; Feature detection is performed on the multi-source monitoring data using a pre-trained lightweight target detection network. When abnormal target features are identified, local data slices are extracted and sent to the edge computing gateway. The lightweight target detection network is deployed on the downhole end-side monitoring equipment. Cross-modal cross-validation is performed on the local data slices using a pre-trained lightweight large model. If the verification confirms that the abnormal target features have fire hazard attributes, hazard confirmation information is generated and sent to the cloud data center. The cloud data center then combines global monitoring feature data to perform deep correlation reasoning on the hazard confirmation information and outputs a comprehensive early warning report. The lightweight large model is deployed on the edge computing gateway.

2. The method according to claim 1, characterized in that, The multi-source monitoring data includes a sequence of consecutive video frames and sensor sampling values. Before performing feature detection on the multi-source monitoring data using a pre-trained lightweight target detection network, the method further includes: Extract sensor sampling values ​​within a preset time sliding window corresponding to the current video frame in the continuous video frame sequence, calculate the arithmetic mean, and generate a synchronization value bound to the current video frame; If no new sensor sampling value is obtained within the preset time sliding window, the zero-order hold strategy is adopted to fill the state using the historical value of the previous effective period.

3. The method according to claim 1, characterized in that, The lightweight target detection network includes a feature extraction backbone network and a cascaded convolutional block attention mechanism module. The feature extraction backbone network contains some convolutional layers, and the convolutional block attention mechanism module includes a channel attention submodule and a spatial attention submodule. The feature detection performed on the multi-source monitoring data using a pre-trained lightweight target detection network includes: The fused tensor data is input into the feature extraction backbone network and spatial features are extracted using the partial convolutional layers. The spatial features are input into the channel attention submodule to perform feature enhancement, resulting in an intermediate feature map. The intermediate feature map is then input into the spatial attention submodule to perform feature enhancement again, resulting in an enhanced feature map. Based on the enhanced feature map, a prediction is made, and a prediction result containing confidence score and classification label is output. The confidence score is compared with a preset visual judgment threshold. When the confidence score of a preset number of consecutive video frames exceeds the preset visual judgment threshold, and the rate of change of the corresponding synchronization value per unit time exceeds the limit, the abnormal target feature is identified and an abnormal trigger signal is generated.

4. The method according to claim 1, characterized in that, The step of performing cross-modal cross-validation on the local data slice using a pre-trained lightweight large model includes: The local data slices are analyzed to extract the characteristic jump types of the numerical sequences of multi-source monitoring data, and the characteristic jump types are converted into text prompts through the environmental rule engine; Extract the time window video slice and the multi-scale visual feature map of the time window video slice from the local data slice, and use the cross-modal feature enhancement layer to alternately perform the intramodal self-attention interaction and cross-modal cross-attention interaction between the visual features of the time window video slice and the text prompt to obtain the fused feature stream; Candidate targets are selected from the multi-scale visual feature map using the feature vectors of the text prompts, and then decoded and predicted using the fused feature stream to output the image-text matching confidence score for the abnormal target features.

5. The method according to claim 4, characterized in that, If the abnormal target features are verified to have fire hazard attributes, hazard confirmation information is generated, including: Compare the image-text matching confidence level with the preset review and judgment threshold; If the confidence level of the image-text matching is lower than the preset review and judgment threshold, the abnormal target feature is determined to be a physical environment interference condition, a false alarm interception command is generated and the data transmission link is blocked. If the confidence level of the image-text matching is greater than or equal to the preset review and judgment threshold, the abnormal target feature is determined to have the fire hazard attribute. The classification label in the local data slice is extracted as the hazard type, and the physical location coordinates are extracted. The verified label is then attached to generate the hazard confirmation information.

6. The method according to claim 1, characterized in that, The step of combining global monitoring feature data to perform deep correlation reasoning on the hazard confirmation information includes: Using the physical location coordinates in the hazard confirmation information as spatial anchor points, global monitoring feature data with topological connectivity to the spatial anchor points within the entire mine area are obtained; Graph convolutional networks are used to aggregate and transfer feature information between nodes to extract spatial diffusion features. The spatial diffusion characteristics and the hazard types in the hazard confirmation information are converted into structured query statements, and vector retrieval is performed in a pre-built coal mine disaster mechanism map knowledge base to match historical disaster evolution paths as prior references.

7. The method according to claim 6, characterized in that, The output comprehensive early warning report includes: Extract extreme values ​​of gas concentration at neighboring nodes and ventilator operating status parameters from global monitoring feature data, calculate the spatial distance attenuation of extreme values ​​of gas concentration at neighboring nodes using the spatial anchor point, and convert the ventilator operating status parameters into a spread penalty gain. The image-text matching confidence, the spatial distance attenuation, and the propagation penalty gain are combined to form a multi-source real-time risk factor. A weighted calculation is performed on the prior reference and the multi-source real-time risk factors to output a comprehensive risk rating index; The comprehensive risk rating index, the type of hidden danger, and the global monitoring feature data are input into a preset causal language model for prediction and generation, and the comprehensive early warning report is output.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 7.

9. A storage medium storing instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method of any one of claims 1 to 7.

10. A program product comprising at least one of a program and instructions, characterized in that, When at least one of the program or instructions is executed by an electronic device, it implements the steps of the method according to any one of claims 1 to 7.

Images