Mine fire infrared visual feature screening and feature fusion monitoring method

Abstract

The invention discloses a mine fire infrared visual feature screening and feature fusion monitoring method. The method mainly comprises the steps that a thermal imager is installed in a to-be-monitored area of a mine; an improved LCM model is adopted to segment a fire area or an interference heat source area, a data set is constructed, and key features of the fire area are screened out. And in combination with the constructed data set, establishing a key feature data set of the mine fire, training a classifier model, and storing the trained classifier model. According to the mine fire monitoring system, an improved LCM model is adopted to segment a suspected fire area appearing in a video stream of an infrared thermal imager, whether a mine fire exists or not is detected in real time through a trained classifier model, and when the mine fire is detected, fire alarm is started. According to the mine fire infrared monitoring method, the defects of an existing monitoring method can be overcome, the recognition speed, accuracy and reliability of the early fire can be improved on the basis of achieving underground large-range monitoring, and then the actual requirements of coal mine safety production are met.

Description

Technical Field

[0001] The invention relates to a mine fire infrared visual feature screening and feature fusion monitoring method, in particular to a vision-based target detection technology, an image segmentation technology, a visual feature extraction technology, and a mine exogenous fire monitoring technology. Background Art

[0002] Mine fire is one of the five major disasters in coal mine accidents. At present, relevant researchers have insufficient understanding of the disaster mechanism and disaster relief risks of mine fires, resulting in the lack of effective technical methods in the perception and identification of mine fires, causing major accidents of mine fires to occur from time to time, causing huge economic losses, and in serious cases, causing casualties in the mine. When a fire occurs, flames, smoke, toxic and harmful gases are usually produced. At the same time, accident investigations show that in mine fire accidents, deaths caused by trauma and burns account for less than 20%, and deaths caused by carbon monoxide poisoning and suffocation account for more than 80%. The higher the concentration of carbon monoxide and the longer the duration, the more serious the damage to the human body, until death. Therefore, in coal mines, it is crucial for coal production safety to promptly discover high-temperature heat sources or early fires, quickly locate the location of disaster sources, accurately identify disaster types, and launch emergency plans and emergency rescue in a targeted manner.

[0003] The existing mine fire sensing and alarm technology involves the influence of many factors such as coal geological conditions, mining methods and processes, sensor technology, etc., resulting in a high rate of missed alarms and false alarms in the existing mine fire alarm system. At the same time, the existing monitoring technology cannot achieve early stage and long-distance monitoring of fires, nor can it locate and track the source of disasters, making it difficult to meet the needs of coal mine safety production. In addition, in the harsh environment underground, the sensors that the disaster sensing devices rely on are easily interfered by environmental factors, and these sensors can only monitor a small area at or around the installation location, making it difficult to achieve large-scale monitoring of disasters in mines.

[0004] In the visual monitoring of external fires in mines, the existing external fire monitoring methods based on visible light and near-infrared bands are easily affected by various interfering light sources underground, such as tunnel lights, mining lamps, car lights and other interfering light sources. Thermal infrared visual monitoring not only has the advantages of large monitoring range, strong anti-interference ability and strong real-time performance, but also has temperature field characteristics that cannot be expressed by visible light and near-infrared monitoring, and it is easy to find obscured fire source targets. For this reason, fire monitoring based on thermal infrared visual features has become one of the important research directions for external fire monitoring in mines. However, the current thermal infrared fire visual monitoring is only applicable to specific scenarios of out-of-control fire sources or close distances, and cannot eliminate interfering heat sources underground, resulting in poor detection of early fire source targets, and cannot achieve long-distance and large-scale fire monitoring underground.

[0005] Aiming at the problems existing in thermal infrared fire visual monitoring, combined with the special environment of coal mine underground, the present invention uses the visual features in the process of exogenous fire spreading in the mine to realize a mine exogenous fire monitoring based on infrared visual feature fusion, and uses explosion-proof cameras and other external devices as video image acquisition devices to collect video images of the monitoring area in real time. Only a small number of cameras are deployed in key areas underground to achieve large-scale monitoring of coal mines; at the same time, the video images collected by the monitoring equipment are analyzed by computer vision and image processing technology. Aiming at the problem of many types of dynamic and static features in the fire spreading stage, the fire feature screening method is used to select key visual features, and the selected key visual features are combined to perform feature fusion identification on the suspected fire targets in the video image, so as to achieve rapid, accurate and reliable identification of high-temperature heat sources and mine fires in the monitoring area. Compared with the existing monitoring and early warning methods, the mine exogenous fire identification method based on the present invention will be faster, more accurate and more reliable. In addition, in the process of mine emergency rescue, fire monitoring based on infrared vision can also assist rescue personnel in making predictions on the disaster situation on the scene and formulating corresponding emergency plans. Summary of the invention

[0006] The technical problem to be solved by the present invention is to overcome the problems of high false alarm rate and missed alarm rate in mine fire identification by existing thermal infrared visual monitoring technology, and to improve the identification speed, accuracy and reliability of early underground fires on the basis of realizing large-area underground monitoring, thereby meeting the actual needs of coal mine safety production.

[0007] The present invention specifically adopts the following technical solutions to solve the above technical problems:

[0008] A mine fire infrared visual feature screening and feature fusion monitoring method, characterized in that the method comprises the following steps:

[0009] Step 1: Based on the spatial characteristics, geological characteristics, tunnel wall conditions and mining impact of the mine area to be monitored, install an infrared thermal imager at the optimal location of the area to be monitored;

[0010] Step 2: Collect fire infrared video and perform equal-interval sampling to obtain infrared images of the t-1th frame and tth frame in the video in turn; use the improved LCM model to segment the fire area in the t-1th frame and tth frame;

[0011] Step 3: Calculate the static features of the fire area in the t-1th frame and the tth frame in sequence, wherein the static features include at least circularity, rectangularity, temperature, texture, and eccentricity; use the similarity coefficient to locate the same fire area in the t-1th frame and the tth frame, and calculate the dynamic features of the same fire area, wherein the dynamic features include at least temperature change rate, area change rate, centroid movement, and perimeter growth rate; construct a sample feature vector of the fire area in the tth frame image based on the static features and dynamic features;

[0012] Step 4: Repeat steps 2 and 3 to obtain sample feature vectors of all fire areas in the fire infrared video, and establish a sample feature set of mine fires through the sample feature vectors of all fire areas;

[0013] Step 5: Collect the infrared video of the mine containing the interference heat source, and use the process of step 2 to obtain the interference heat source area in the t-1 frame and the t frame; use the process of step 3 to obtain the sample feature vector of the interference heat source area in the t frame image; repeat the processes of steps 2 and 3 to obtain the sample feature vectors of all the interference heat source areas in the infrared video of the mine containing the interference heat source, and establish the sample feature set of the mine interference heat source through the sample feature vectors of all the interference heat source areas;

[0014] Step 6: Establish a dataset for infrared visual feature screening of mine fires through the sample feature set of mine fires and the sample feature set of mine interference heat sources, and perform infrared visual key feature screening of mine fires;

[0015] Step 7: combining the selected infrared vision key features of mine fires and the data set to construct a key feature data set of mine fires; randomly extracting a portion of sample data from the key feature data set to form a training data set for a classifier for feature fusion monitoring, and inputting the training data set into the classifier for feature fusion monitoring for model training to obtain a trained classifier model for feature fusion monitoring;

[0016] Step 8: Use the improved LCM model to segment the suspected fire area appearing in the infrared thermal imager video stream, and calculate the infrared visual key features of the mine fire in the current frame based on the same suspected fire area in the current frame and the previous frame; construct the corresponding key feature vector based on the obtained infrared visual key features of the mine fire, and input the key feature vector into the trained feature fusion monitoring classifier model to identify whether there is an external mine fire in the current frame.

[0017] Furthermore, the mine fire infrared visual feature screening process includes the following steps:

[0018] Step 6A: Randomly extract a part of the sample feature vectors in the data set, and form a single feature training set F1, F2, ..., F with only the same feature according to the feature category in the sample feature vector.n , n is the total number of features;

[0019] Step 6B: Select a single feature training set F τ , input the feature screening classifier for model training, and after the training is completed, output the screening classifier model S corresponding to the single feature τ , 1≤τ≤n;

[0020] Step 6C: Repeat step 6B to obtain single feature training sets F1, F2, ..., F n The corresponding screening classifier models S1, S2, ..., S n ;

[0021] Step 6D: The feature vectors of another part of the sample in the data set are sequentially used to form a single feature test set f1, f2, ..., f with only the same feature according to the feature categories in the sample feature vectors. n ;

[0022] Step 6E: Set the single-feature test set f1, f2, ..., f n Input the corresponding filter classifier model S1, S2, ..., S n Prediction is performed in the algorithm, and the predicted value corresponding to each single feature test set is output; the index threshold method is used to analyze the predicted value and the true value, and the key infrared visual features of mine fire are screened out.

[0023] Furthermore, the improved LCM model includes changing the local area maximum value strategy in the original LCM model to calculating the local area mean, and introducing a multi-scale strategy to improve the saliency of the target in the mine infrared image; the saliency is given by the formula Calculate; calculate the saliency map at different scales through the saliency formula, take the pixel maximum value operation of the saliency map at multiple scales to obtain the final saliency map, adopt the target segmentation strategy of the original LCM model to extract the fire area, interference heat source area or suspected fire area in the final saliency map; where C j is the significant value of the central area at scale j; L j is the maximum grayscale value of the central area under scale j; c i,j is the contrast between the central area and the i-th neighborhood at scale j, c i =L / m i , i takes 4 or 8 neighbors, m i is the average grayscale value of the i-th neighborhood.

[0024] Furthermore, the texture feature is obtained by calculating the image entropy of the fire area, the interference heat source area or the suspected fire area. Where P q is the probability of grayscale q appearing in the suspected fire area; N is the grayscale level of the mine infrared image.

[0025] Furthermore, the temperature characteristics are obtained by calculating the grayscale average value of the fire area, the interference heat source area or the suspected fire area. Where M is the total number of pixels in the fire area, interfering heat source area or suspected fire area; I k is the grayscale of the kth pixel in the fire area, interfering heat source area or suspected fire area.

[0026] Furthermore, the temperature change rate D T =|R T,t -R T,t-1 | / R T,t , where R T,t and R T,t-1 are the grayscale average values ​​R of the same fire area, the same interference heat source area, or the same suspected fire area in the t-th frame and the t-1 frame respectively. T .

[0027] Furthermore, the similarity coefficient is obtained by calculating the inverse of the mean square relative error, which is used to measure whether the segmented areas of adjacent frames in the infrared video are the same fire area, the same interference heat source area, or the same suspected fire area; the mean square relative error Where μ t,τ , μ t-1,τ are the τth static eigenvalues ​​of the same fire area, the same interference heat source area, or the same suspected fire area in the tth frame and t-1 frame, respectively.

[0028] Furthermore, the feature categories are composed of static features and dynamic features, the total number of features is equal to the sum of the number of feature categories, and the indicator threshold method is to calculate the corresponding indicator value based on the predicted value and the true value, and compare the indicator value with the set threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of a mine fire monitoring system of the present invention;

[0030] Figure 2 A flow chart of constructing a mine fire feature screening data set according to the present invention;

[0031] Figure 3 The flow chart of the screening of key characteristics of mine fire of the present invention;

[0032] Figure 4 The mine fire visual feature fusion monitoring flow chart of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail and completely below in conjunction with the accompanying drawings and specific implementation methods. The embodiments should not be construed as limiting the scope of use of the present invention.

[0034] like Figure 1 As shown in the figure, the mine fire monitoring system is divided into an above-ground part and an underground part, which is used to monitor external fires in mines. Its main components include:

[0035] 1. Information processing server (101): responsible for storing the mine video images collected by the thermal imager (105), and analyzing whether the collected video contains suspected fire areas. If there are suspected fire areas, the visual features of the suspected fire areas are extracted and input into the trained SVM, BP, KNN and other classifier models. The predicted value is used to determine whether the fire is caused by external factors in the mine. If it is caused by external factors in the mine, the alarm module (107) sends out sound, light, vibration and other alarm signals through the communication network, and the monitoring server (102) sends out alarm prompts and human-computer interaction on the monitoring screen when the alarm module sends out the alarm information.

[0036] 2. Monitoring server (102): responsible for displaying monitoring data of the coal mine monitoring area. At the same time, the monitoring server (102) and the information processing server (101) are connected to each other in communication, displaying real-time images of the monitoring area, and providing alarm prompts and human-computer interaction after the alarm module (107) issues an alarm message. Production management personnel can retrieve and query historical data stored in the information processing server (101) through the monitoring server (102), and the monitoring server (102) is connected to the core switch (103) through a communication line to access the mine communication network.

[0037] 3. Core switch (103): The core management and switching device of the mine communication network, responsible for the management and data exchange of all devices connected to the mine communication network, with routing function, connected to the external Internet through a firewall.

[0038] 4. Ring network switch (104): underground switching equipment of the mine communication network, installed underground, with multiple ring network switches connected in a ring network manner.

[0039] 5. Thermal imager (105): a video acquisition device installed in key monitoring areas in a mine, responsible for acquiring video images of monitoring areas prone to fire, such as transport tunnels, ventilation tunnels, communication tunnels, heading tunnels, cut holes, coal mining faces, heading faces, electromechanical chambers, underground substations, etc. in the mine. The video images may be grayscale images or pseudo-color images. The thermal imager (105) is a mining explosion-proof camera.

[0040] 6. Communication substation (106): one end is connected to the thermal imager (105) for communication, and the other end is connected to the ring network switch (104) for communication. It can be connected to the devices at both ends via a wireless communication network or a wired communication network. In this example, wired communication is used for illustration.

[0041] 7. Alarm module (107): uses sound, light, vibration and other alarm methods, directly communicates with the monitoring server (102) through the existing communication network, and communicates with the information processing server (101) through the core switch (103); when the alarm module (107) receives the alarm signal sent by the information processing server (101), it uses one or more alarm methods such as sound, light, and vibration to alarm, prompting the staff to deal with the scene and activate the emergency plan.

[0042] like Figure 2 The process of constructing the mine fire feature screening dataset shown includes:

[0043] 1. Initialization (201): According to the spatial characteristics, geological properties, tunnel conditions and mining impact of the mine, a thermal imager is installed at the optimal position of the underground monitored area; the monitored area includes underground transport tunnels, ventilation tunnels, communication tunnels, heading tunnels, coal mining faces, heading faces, electromechanical chambers, underground substations and other areas prone to disaster hazards; the optimal position is that the closest distance to the monitored target area shall not be less than 5 meters, and the farthest distance shall not be more than 50 meters. The installation location must be in an open monitoring area, avoid obstructions as much as possible, and be installed on a solid roof or tunnel side wall without affecting normal construction operations.

[0044] 2. Fire area segmentation (202): sampling the fire infrared video at equal intervals, obtaining infrared images of the t-1th frame and the tth frame, and segmenting the fire area in the t-1th frame and the tth frame using an improved LCM model; the improved LCM model includes: changing the local area maximum value strategy in the original LCM model to calculating the mean of the local area, and introducing a multi-scale strategy to improve the saliency of the target in the mine infrared image; the saliency is given by the formula Calculate; calculate the saliency map at different scales through the saliency formula, take the pixel maximum value operation of the saliency map at multiple scales to obtain the final saliency map, adopt the target segmentation strategy of the original LCM model to extract the fire area, interference heat source area or suspected fire area in the final saliency map; where C j is the significant value of the central area at scale j; L j is the maximum grayscale value of the central area under scale j; c i,j is the contrast between the central area and the i-th neighborhood at scale j, c i =L / m i , i takes 4 or 8 neighbors, m iis the average grayscale value of the i-th neighborhood;

[0045] From the saliency calculation formula, we can see that if the central area is the target area, then L ≥ m i , and c i ≥1; if the central area is the background area, then L≤m i , and c i ≤1. Usually, it is advisable to select a sliding window of 9×9 image blocks, and i takes 8 neighborhoods, then the size of each region is 3×3. At the same time, combining the saliency calculation formula and multi-scale transformation strategy can obtain the saliency map of the mine infrared image, and realize the enhancement of weak fire source targets in infrared images and the suppression of background light and noise.

[0046] Since the saliency map has the characteristics of discontinuity between the target and the adjacent area, and consistency between the background and the adjacent area, an adaptive threshold-based target segmentation strategy is used to extract the fire area, and its adaptive threshold T = μ + kσ; where T is the adaptive segmentation threshold, μ and σ are the mean and standard deviation of the saliency map, respectively, and k is an arbitrary constant coefficient, k∈[3,5].

[0047] 3. Calculating visual features of the fire area (203): sequentially calculating static features of the fire area in the t-1th frame and the tth frame, the visual features including static features and dynamic features; the static features at least including circularity, rectangularity, temperature, texture, and eccentricity; using a similarity coefficient to locate the same fire area in the t-1th frame and the tth frame, and calculating dynamic features of the same fire area, the dynamic features at least including temperature change rate, area change rate, centroid movement, and perimeter growth rate; fusing the obtained static features and dynamic features to construct a sample feature vector of the fire area in the tth frame image;

[0048] 4. Establishing a sample feature set of mine fire (204): Repeating steps 202 and 203 to obtain sample feature vectors of all fire areas in the mine fire infrared video, and establishing a sample feature set of mine fire through the sample feature vectors of all fire areas;

[0049] 5. Establish a sample feature set of interference heat sources in a mine (205): Collect an infrared video of the mine containing the interference heat source, use the process of step 202 to perform equal-interval sampling, obtain infrared images of the t-1th frame and the tth frame in the video in turn, and use the improved LCM model to segment the interference heat source areas in the t-1th frame and the tth frame;

[0050] Using the process of step 203, the static features of the interference heat source area in the t-1th frame and the tth frame are calculated in sequence, the same interference heat source area in the t-1th frame and the tth frame is located using the similarity coefficient, and the dynamic features of the same interference heat source area are calculated, and according to the static features and dynamic features, a sample feature vector of the interference heat source area in the tth frame image is constructed;

[0051] Similarly, repeat steps 202 and 203 to obtain sample feature vectors of all interference heat source areas in the mine infrared video containing the interference heat source, and establish a sample feature set of the mine interference heat source through the sample feature vectors of all interference heat source areas;

[0052] The similarity coefficient is used to measure whether the segmented areas of adjacent frames in the infrared video are the same fire area, the same interference heat source area, or the same suspected fire area; the similarity coefficient is obtained by calculating the inverse of the mean square relative error. When the calculated similarity coefficient is greater than the set threshold, it is determined to be the same fire area or the same interference heat source area or the same suspected fire area; t,τ , μ t-1,τ are the τth static eigenvalue (or the value of the static feature τ) of the same fire area, the same interference heat source area, or the same suspected fire area in the tth frame and the t-1th frame, respectively.

[0053] The texture features are obtained by calculating the image entropy of the fire area, the interference heat source area or the suspected fire area. Where P q is the probability of grayscale q appearing in the suspected fire area; N is the grayscale level of the mine infrared image.

[0054] The temperature characteristics are obtained by calculating the grayscale average value of the fire area, the interference heat source area or the suspected fire area. Where M is the total number of pixels in the fire area, interfering heat source area or suspected fire area; I k is the grayscale of the kth pixel in the fire area, interfering heat source area or suspected fire area.

[0055] The temperature change rate D T =|R T,t -R T,t-1 | / R T,t , where R T,t and R T,t-1 are the grayscale average values ​​R of the same fire area, the same interference heat source area, or the same suspected fire area in the t-th frame and the t-1 frame respectively. T .

[0056] The centroid movement is obtained by calculating the centroid displacement distance of the same fire area or the same interference heat source area or the same suspected fire area in the t-1 frame and the t frame; the centroid displacement distance Where x t-1 ,y t-1 are the horizontal and vertical coordinates of the centroid of the fire area, interference heat source area, or suspected fire area in the t-1th frame; t ,y t are the horizontal and vertical coordinates of the centroid of the fire area, interference heat source area or suspected fire area in the tth frame respectively.

[0057] 6. Establish a feature screening data set (206): Based on the obtained sample feature set of mine fires and the sample feature set of mine interference heat sources, establish a data set for infrared visual feature screening of mine fires, which is used for infrared visual key feature screening of mine fires and training of classifier models based on key feature fusion monitoring.

[0058] like Figure 3 The key characteristics screening process for mine fires shown includes:

[0059] 1. Key feature screening (301): Randomly extract a portion of sample feature vectors from the feature screening data set, and sequentially construct single feature training sets F1, F2, ..., F with only the same feature according to the feature categories in the sample feature vectors. n , n is the total number of features; the feature categories are composed of static features and dynamic features, and the total number of features n is equal to the sum of the number of feature categories.

[0060] 2. Training feature screening classifier (302): Select a single feature training set F τ , input the feature screening classifier for model training, and after the training is completed, output the screening classifier model S corresponding to the single feature τ , 1≤τ≤n; repeat this process to obtain the single feature training sets F1, F2, ..., F n The corresponding screening classifier models S1, S2, ..., S n .

[0061] 3. Test the screening classifier model (303): The feature vectors of another part of the feature-screened data set are sequentially used to form a single-feature test set f1, f2, ..., f with only the same feature according to the feature categories in the sample feature vectors. n ; Single feature test set f1, f2, ..., f n Input the corresponding filter classifier model S1, S2, ..., S n Make predictions in the , and output the predicted values ​​corresponding to each single-feature test set.

[0062] 4. Screening infrared vision key features (304): The index threshold method is used to analyze the predicted value and the true value to screen out the infrared vision key features of the mine fire. The index threshold method is to calculate the corresponding index value based on the predicted value and the true value, and compare the index value with the set threshold value.

[0063] The indicators are accuracy (ACC), detection rate (TPR) and false positive rate (FPR). The larger the ACC and TPR, and the smaller the FPR, the better the recognition performance of the screening classifier model. In the formula, TP is the positive sample predicted to be positive; TN is the negative sample predicted to be negative; FP is the negative sample predicted to be positive; FN is the positive sample predicted to be negative.

[0064] The index value is the value of any one of the three indexes or the calculated value of a combination of two or more indexes. If the index value corresponding to the single feature exceeds the set discrimination threshold, the single feature is selected as the infrared visual key feature of the mine fire.

[0065] like Figure 4 The mine fire visual feature fusion monitoring process shown includes:

[0066] 1. Establish key feature data set (401): Combine Figure 2 and Figure 3 The infrared visual key features of mine fire screened out and the data set, extract the key features in the sample feature vector in the sample feature set, and form a new sample key feature vector, repeat the process, sequentially screen out the key features in each sample feature vector in the sample feature set, and construct a key feature data set of mine fire;

[0067] 2. Training a classifier for feature fusion monitoring (402): randomly extracting a portion of sample data from the key feature data set to form a training data set for the classifier for feature fusion monitoring, and inputting the training data set into the classifier for feature fusion monitoring for model training to obtain a trained classifier model for feature fusion monitoring;

[0068] 3. Mine fire monitoring based on feature fusion (403): The mine fire monitoring system collects video images in the monitoring area in real time, uses the improved LCM model to segment the suspected fire area appearing in the infrared thermal imager video stream, and calculates the mine fire infrared visual key features of the current frame based on the same suspected fire area in the current frame and the previous frame; constructs the corresponding key feature vector based on the obtained mine fire infrared visual key features, and inputs the key feature vector into the trained feature fusion monitoring classifier model to identify whether there is a mine fire in the current frame.

[0069] 4. Whether there is a mine fire (404): When the classifier model of the feature fusion monitoring recognizes that there is a mine fire in the video image, the mine fire monitoring system executes sequentially (405), otherwise it returns to execute (403).

[0070] 5. Mine fire warning (405): The information processing server sends an alarm signal to the alarm module. After receiving the alarm signal, the alarm module uses one or more of the following alarm methods: sound, light, and vibration to alert the staff to handle the scene and initiate the emergency plan.

Claims

1. A mine fire infrared visual feature screening and feature fusion monitoring method, characterized in that: The method comprises the following steps: Step 1: Based on the spatial characteristics, geological characteristics, tunnel wall conditions and mining impact of the mine area to be monitored, install an infrared thermal imager at the optimal location of the area to be monitored; Step 2: Collect fire infrared video and perform equal-interval sampling to obtain infrared images of the t-1th frame and tth frame in the video in turn; use the improved LCM model to segment the fire area in the t-1th frame and tth frame; Step 3: Calculate the static features of the fire area in the t-1th frame and the tth frame in sequence, wherein the static features include at least circularity, rectangularity, temperature, texture, and eccentricity; use the similarity coefficient to locate the same fire area in the t-1th frame and the tth frame, and calculate the dynamic features of the same fire area, wherein the dynamic features include at least temperature change rate, area change rate, centroid movement, and perimeter growth rate; construct a sample feature vector of the fire area in the tth frame image based on the static features and dynamic features; Step 4: Repeat steps 2 and 3 to obtain sample feature vectors of all fire areas in the fire infrared video, and establish a sample feature set of mine fires through the sample feature vectors of all fire areas; Step 5: Collect the infrared video of the mine containing the interference heat source, and use the process of step 2 to obtain the interference heat source area in the t-1 frame and the t frame; use the process of step 3 to obtain the sample feature vector of the interference heat source area in the t frame image; repeat the processes of steps 2 and 3 to obtain the sample feature vectors of all the interference heat source areas in the infrared video of the mine containing the interference heat source, and establish the sample feature set of the mine interference heat source through the sample feature vectors of all the interference heat source areas; Step 6: Establish a dataset for infrared visual feature screening of mine fires through the sample feature set of mine fires and the sample feature set of mine interference heat sources, and perform infrared visual key feature screening of mine fires; Step 7: combining the selected infrared vision key features of mine fires and the data set to construct a key feature data set of mine fires; randomly extracting a portion of sample data from the key feature data set to form a training data set for a classifier for feature fusion monitoring, and inputting the training data set into the classifier for feature fusion monitoring for model training to obtain a trained classifier model for feature fusion monitoring; Step 8: Use the improved LCM model to segment the suspected fire area appearing in the infrared thermal imager video stream, and calculate the infrared visual key features of the mine fire in the current frame based on the same suspected fire area in the current frame and the previous frame; construct the corresponding key feature vector based on the obtained infrared visual key features of the mine fire, and input the key feature vector into the trained feature fusion monitoring classifier model to identify whether there is an external mine fire in the current frame.

2. A mine fire infrared visual feature screening and feature fusion monitoring method according to claim 1, characterized in that: The mine fire infrared visual feature screening process comprises the following steps: Step 6A: Randomly extract a part of the sample feature vectors in the data set, and form a single feature training set F1, F2, ..., F with only the same feature according to the feature category in the sample feature vector. n , n is the total number of features; Step 6B: Select a single feature training set F τ , input the feature screening classifier for model training, and after the training is completed, output the screening classifier model S corresponding to the single feature τ , 1≤τ≤n; Step 6C: Repeat step 6B to obtain single feature training sets F1, F2, ..., F n The corresponding screening classifier models S1, S2, ..., S n ; Step 6D: The feature vectors of another part of the sample in the data set are sequentially used to form a single feature test set f1, f2, ..., f with only the same feature according to the feature categories in the sample feature vectors. n ; Step 6E: Set the single-feature test set f1, f2, ..., f n Input the corresponding filter classifier model S1, S2, ..., S n Prediction is performed in the algorithm, and the predicted value corresponding to each single feature test set is output; the index threshold method is used to analyze the predicted value and the true value, and the key infrared visual features of mine fire are screened out.

3. A mine fire infrared visual feature screening and feature fusion monitoring method according to claim 1, characterized in that: The improved LCM model includes changing the local area maximum value strategy in the original LCM model to calculating the local area mean, and introducing a multi-scale strategy to improve the saliency of the target in the mine infrared image; the saliency is given by the formula Calculate; calculate the saliency map at different scales through the saliency formula, take the pixel maximum value operation of the saliency map at multiple scales to obtain the final saliency map, adopt the target segmentation strategy of the original LCM model to extract the fire area, interference heat source area or suspected fire area in the final saliency map; where C j is the significant value of the central area at scale j; L j is the maximum grayscale value of the central area under scale j; c i,j is the contrast between the central area and the i-th neighborhood at scale j, c i =L / m i , i takes 4 or 8 neighbors, m i is the average grayscale value of the i-th neighborhood.

4. The method for monitoring external causes of mine fires by integrating visible light visual features according to claim 1 is characterized in that: The texture features are obtained by calculating the image entropy of the fire area, the interference heat source area or the suspected fire area. Where P q is the probability of grayscale q appearing in the suspected fire area; N is the grayscale level of the mine infrared image.

5. A mine fire infrared visual feature screening and feature fusion monitoring method according to claim 1, characterized in that: The temperature characteristics are obtained by calculating the grayscale average value of the fire area, the interference heat source area or the suspected fire area. Where M is the total number of pixels in the fire area, interfering heat source area or suspected fire area; I k is the grayscale of the kth pixel in the fire area, interfering heat source area or suspected fire area.

6. A mine fire infrared visual feature screening and feature fusion monitoring method according to claim 1, characterized in that: The temperature change rate D T =|R T,t -R T,t-1 | / R T,t , where R T,t and R T,t-1 are the grayscale average values ​​R of the same fire area, the same interference heat source area, or the same suspected fire area in the t-th frame and the t-1 frame respectively. T .

7. A mine fire infrared visual feature screening and feature fusion monitoring method according to claim 1, characterized in that: The similarity coefficient is obtained by calculating the inverse of the mean square relative error, which is used to measure whether the segmented areas of adjacent frames in the infrared video are the same fire area, the same interference heat source area, or the same suspected fire area; the mean square relative error Where μ t,τ , μ t-1,τ are the τth static eigenvalues ​​of the same fire area, the same interference heat source area, or the same suspected fire area in the tth frame and t-1 frame, respectively.

8. A mine fire infrared visual feature screening and feature fusion monitoring method according to claim 2, characterized in that: The feature categories are composed of static features and dynamic features, and the total number of features is equal to the sum of the number of feature categories; the indicator threshold method is to calculate the corresponding indicator value based on the predicted value and the true value, and compare the indicator value with the set threshold.

Images