A coal fire area multi-source remote sensing identification method and system based on three-layer ensemble learning

Abstract

The application discloses a coal fire area multi-source remote sensing identification method and system based on three-layer integrated learning, which comprises the following steps: based on multi-modal remote sensing data, the multi-element information of the fire area is extracted, the multi-modal remote sensing data comprises thermal infrared images, multispectral images, SAR images and night light images; based on the multi-element information of the fire area, the characteristic samples of the fire area are demarcated; the integrated learning model is constructed based on the characteristic samples of the fire area, and the range of the fire area is identified based on the integrated learning model. Compared with the traditional fire area identification method, the application only studies a single small range fire area, the corresponding parameters are repeatedly re-adjusted with the change of the study area, and the limitation caused by the decrease of the sensitivity of some parameters cannot be estimated. The application directly takes higher confidence for the sample, guarantees the high reliability of the sample, and can be applied to large-scale fire area monitoring through the collaborative learning of various parameters, and has strong generalization ability.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of remote sensing identification of coal fire area, and particularly relates to a coal fire area multi-source remote sensing identification method and system based on three-layer ensemble learning. BACKGROUND

[0002] Coal field fire area is formed by multiple factors: internal geological action makes part of the coal seam horizontally fold or fracture, and vertically uplift with the earth's crust; external geological action weathering and denudation of the earth's surface causes rock fissure development, and internal and external forces jointly make oxygen gradually contact with the coal seam along the fissure, under the influence of solar radiation, coal is easy to self-ignite due to low specific heat capacity. With the increasing demand for energy by human beings, coal as an economic resource is mined in large quantities, and disordered mining further aggravates the oxidation and spontaneous combustion of coal, forming a large area of coal field fire area.

[0003] Coal fire area is widely distributed in countries and regions rich in coal resources, and has become a common concern worldwide. Coal fire combustion not only seriously damages coal resources and hinders mining, but also causes unpredictable impact on the surrounding ecological system, water resources and soil, atmosphere and other environmental issues. Therefore, how to quickly and accurately identify and delineate the location of coal fire area has important practical significance.

[0004] A large amount of coal combustion at high temperature underground changes the physical and chemical field of the fire area, so geophysical, geochemical, thermal and drilling methods such as surface temperature measurement, natural potential, transient electromagnetic, isotope radon and sulfur measurement, and dual-element tracer gas are proposed. With the rapid development of remote sensing technology in recent years, the spatial and temporal resolution and performance of satellite sensors have been greatly improved, and it is feasible to use Lansat-8 TIRS data to retrieve fire area surface temperature and use differential interferometric synthetic aperture radar (D-InSAR) technology to accurately measure the vertical ground subsidence caused by coal fire. On this basis, subsequent scholars have also used time series InSAR technologies such as Persistent Scatterer InSAR (PSI), Small Baseline Subset InSAR (SBAS), and Distributed Scatterer InSAR (DSI) to weaken the influence of factors such as time-space decorrelation and atmospheric delay in coal field fire area, making the delineation result more accurate.

[0005] The above coal fire monitoring methods each have their advantages, and have achieved certain results in specific practice, but still have deficiencies: the field detection method is low in efficiency, narrow in application range and low in spatial resolution; single satellite-borne thermal infrared detection is not easy to define the segmentation threshold of coal fire pixel and background pixel; single (time series) differential interference is easily affected by mining area opening, natural ground deformation and other interference factors; at the present stage, most of the multi-source remote sensing collaborative detection technologies adopt multi-level filtering by constructing filters, but such methods are high in subjectivity of threshold setting, depend on prior knowledge, and are poor in identification effect of short-time burning coal fire area, the methods cannot well associate multi-source data, and are poor in generalization ability.

[0006] At present, machine learning is widely applied in various fields, but there is still a lack of relevant exploration in coal fire area identification. Integrated learning has significant advantages in model generalization by homogenous and heterogeneous integration of multiple single learning models. At the present stage, most applications only adopt single integrated learning method, and there is also a lack of comparison between different integrated methods. SUMMARY

[0007] The present application aims to solve the deficiencies of the prior art, and proposes a coal fire area multi-source remote sensing identification method and system based on three-layer integrated learning. In order to better mine the potential relationship between multi-source remote sensing data of the fire area, take into account the advantages of different integrated methods, and improve the robustness, generalization and accuracy of the identification result of the model, the present application combines thermal infrared, optical, radar and night light data, takes the multi-modal remote sensing data extracted under strong constraints as a data set, and constructs a three-layer improved integrated learning coal fire area identification model based on the integrated learning idea. While taking into account the efficiency, the influence of subjective factors is greatly reduced, and the method is more suitable for multi-dimensional data processing. Through multi-element collaborative learning, the problem of poor sensitivity of one or several elements can be alleviated.

[0008] To achieve the above object, the present application provides the following scheme:

[0009] A coal fire area multi-source remote sensing identification method based on three-layer integrated learning, comprising the following steps:

[0010] Based on multi-modal remote sensing data, fire area multi-element information is extracted, and the multi-modal remote sensing data includes thermal infrared image, multispectral image, SAR image and night light image.

[0011] Based on the fire area multi-element information, a fire area feature sample is delineated.

[0012] Based on the fire area feature sample, an integrated learning model is constructed, and the fire area range is identified based on the integrated learning model.

[0013] Preferably, the fire area multi-element information includes thermal anomaly area, vegetation rich area, sedimentation anomaly area and high light area.

[0014] Preferably, the extraction method of the thermal anomaly area comprises:

[0015] Inversion is performed on the surface temperature of the fire area of the coal fire area at multiple time phases to obtain an inversion result;

[0016] A first threshold value is set, and the inversion result is divided based on the first threshold value to obtain an initial thermal anomaly area;

[0017] Each of the annual winter phase images of the initial thermal anomaly area is subjected to intersection processing to obtain a first processed image;

[0018] The first processed image is superimposed in units of years to obtain the thermal anomaly area.

[0019] Preferably, the extraction method of the vegetation-rich area comprises:

[0020] The normalized vegetation index of the fire area at multiple time phases is calculated based on the thermal infrared image;

[0021] A second threshold value is set, and the thermal anomaly area is divided based on the second threshold value to obtain the vegetation-rich area.

[0022] Preferably, the extraction method of the settlement anomaly area comprises:

[0023] The SAR image of the coal fire area is obtained, and a small baseline interferogram is screened from the SAR image according to a space-time baseline threshold value;

[0024] The small baseline interferogram is subjected to registration, multi-observation, removal of terrain phase and orbit phase to obtain a differential interferogram;

[0025] Linear deformation and height error of high-coherence points in the differential interferogram are solved, and atmospheric delay phase correction is performed to obtain a surface deformation time sequence;

[0026] A third threshold value is set, and the surface deformation time sequence is divided based on the third threshold value to obtain the settlement anomaly area.

[0027] Preferably, the extraction method of the highlight area comprises:

[0028] Radiation calibration and comparison are performed on the night light image to obtain the brightness difference of the coal fire area and other ground objects;

[0029] Based on the brightness difference, the bright areas of different night light images are subjected to intersection processing to obtain an intersection region;

[0030] The intersection region is divided into blocks, the brightness average of different block areas in different night light images is calculated, and the fire area and the mining area are segmented based on the thermal anomaly area and the settlement anomaly area.

[0031] A fourth threshold value is set, and a location of the open fire is determined based on the fourth threshold value, i.e., the highlighted area.

[0032] Preferably, the method for delineating the fire area feature sample comprises:

[0033] A standard deviation ellipse is constructed based on the intersection of the thermal anomaly area and the subsidence anomaly area, to obtain a strongly constrained hidden fire area sample;

[0034] The highlighted area is superimposed on the strongly constrained hidden fire area sample, to obtain a strongly constrained open fire area sample;

[0035] A buffer area is established outside the standard deviation ellipse, and independent thermal anomaly areas, independent subsidence anomaly areas and non-anomaly feature points in the image of the buffer area are extracted, to obtain a non-fire area sample.

[0036] Preferably, the method for constructing the ensemble learning model comprises:

[0037] The strongly constrained hidden fire area sample, the strongly constrained open fire area sample and the non-fire area sample are oversampled, and the obtained oversampled data is shuffled to obtain a training set and a test set according to a preset ratio;

[0038] The training set and the test set are trained to obtain a first output result;

[0039] A secondary layer is trained based on the first output result, to obtain a second output result;

[0040] A final layer prediction model is established based on the second output result, to obtain the ensemble learning model.

[0041] The application also provides a coal fire area multi-source remote sensing identification system based on three-layer ensemble learning, comprising an information extraction module, a sample delineation module and an identification module.

[0042] The information extraction module is configured to extract fire area multi-element information based on multi-modal remote sensing data, wherein the multi-modal remote sensing data comprises thermal infrared images, multispectral images, SAR images and night light images.

[0043] The sample delineation module is configured to delineate fire area feature samples based on the fire area multi-element information.

[0044] The identification module is configured to construct an ensemble learning model based on the fire area feature samples, and identify the range of the fire area based on the ensemble learning model.

[0045] Compared with the prior art, the application has the following beneficial effects:

[0046] (1) The application cooperatively detects the fire area by combining multi-modal remote sensing data, jointly represents the abnormal phenomena caused by coal fire combustion, introduces night light images, analyzes the brightness values of different ground object pixels, and performs intersection, segmentation and hierarchical processing on the brightness area to assist in segmenting the fire area and the mining area.

[0047] (2) The three-layer ensemble learning of the application can solve the problem that statistical models cannot be established between multi-source remote sensing data, and because it integrates three different ensemble ideas, it reduces the influence of variance, bias and noise of the training results. The method can extract the pixel values of the region of interest once by masking multi-source data, massive data can improve the model learning ability and ensure the model precision, and the model itself can be evaluated for internal consistency precision;

[0048] (3) Compared with the traditional fire area recognition method which is limited to the cutoff time of the current research, the application can achieve long-time sequence continuous monitoring for the same fire area, and further improve the classification accuracy of the model by continuously increasing the region of interest;

[0049] (4) Compared with the traditional fire area recognition method which only studies a single small range of fire area, the corresponding parameters are repeatedly re-adjusted with the change of the study area, which cannot estimate the limitation caused by the decrease of the sensitivity of some parameters, the application directly takes higher confidence level for the sample to ensure the high reliability of the sample, and through the collaborative learning of multiple parameters, it can be applied to large-scale fire area monitoring and has strong generalization ability. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the application, the following briefly introduces the drawings needed to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0051] Figure 1 The method flowchart of the embodiment of the application;

[0052] Figure 2 The method block diagram of the embodiment of the application;

[0053] Figure 3 The integrated learning model schematic diagram of the embodiment of the application;

[0054] Figure 4 The system structure schematic diagram of the embodiment of the application. DETAILED DESCRIPTION

[0055] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0056] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0057] Embodiment one

[0058] In this embodiment, taking the general gobi and general temple test area as an example, as shown in Figure 1 、 2 , a coal fire area multi-source remote sensing identification method based on three-layer ensemble learning includes the following steps:

[0059] S1. Based on multi-modal remote sensing data, the multi-element information of the fire area is extracted. The multi-modal remote sensing data includes thermal infrared image, multi-spectral image, SAR image and night light image. The multi-element information of the fire area includes thermal anomaly area, vegetation rich area, sedimentation anomaly area and high light area.

[0060] The extraction method of the thermal anomaly area includes: inverting the ground surface temperature of the fire area of the fire area at multiple time phases to obtain the inversion result; setting a first threshold, dividing the inversion result based on the first threshold to obtain an initial thermal anomaly area; performing intersection processing on the adjacent images of each winter of the initial thermal anomaly area to obtain a first processing image; stacking the first processing image in units of years to obtain the thermal anomaly area.

[0061] In this embodiment, the atmospheric correction algorithm based on the thermal radiation transfer equation and the single window algorithm are used to invert the ground surface temperature of the general gobi and general temple test area combined with 6 thermal infrared images (2 adjacent images in each winter, a total of 3 years). The histogram of the ground surface temperature inverted by the two algorithms is counted, and the average value and the standard deviation are calculated. The result obtained by the algorithm with more significant normal distribution, smaller standard deviation and lower discrete degree is selected. The artificial threshold method is used to take the 95% confidence level, and the ground surface temperature mean value plus 2 times the standard deviation is taken as the first threshold. The part higher than the threshold is the initial thermal anomaly area. On the basis of the first threshold, the adjacent images of each winter are processed by intersection, the influence of the thermal anomaly area caused by direct sunlight is weakened, and then the stacking is performed in units of years, the influence of climate, season and other factors on the thermal anomaly area is weakened, the probability of coal fire being extinguished due to extreme weather is reduced, and the final thermal anomaly area is obtained.

[0062] The extraction method of the vegetation-rich area comprises: calculating normalized vegetation indexes of the fire area at multiple time phases based on the thermal infrared image; setting a second threshold value, dividing the thermal anomaly area based on the second threshold value, and obtaining the vegetation-rich area.

[0063] The normalized vegetation index NDVI of the fire area at multiple time phases is calculated by using the red wave band and the near-infrared wave band of the multispectral sensor, the second threshold value is determined by adjusting the multiple relationship of the mean value and the standard deviation and combining the actual NDVI value on the basis of the final thermal anomaly area, and finally, the area greater than the threshold value is taken as the vegetation-rich area.

[0064] In the embodiment, the three scenes of multispectral images corresponding to summer in different years are uniformly subjected to radiation calibration, atmospheric correction and terrain correction, and the NDVI is calculated, the average of the three scenes of NDVI calculation results is taken, the average value and the standard deviation of the NDVI in the range of the final thermal anomaly area are extracted and counted, the mean value plus one times of the standard deviation is taken as the second threshold value, most of the final thermal anomaly area is retained, the high-NDVI area is also screened, these areas are filtered out from the thermal anomaly area in subsequent processing, and finally, the area greater than the second threshold value is taken as the vegetation-rich area.

[0065] The extraction method of the subsidence anomaly area comprises: acquiring a SAR image of a coal fire area, screening out a small-baseline interferogram from the SAR image according to a time-space baseline threshold value; performing registration and multi-view on the small-baseline interferogram, removing terrain phases and orbit phases, and obtaining a differential interferogram; solving linear deformation and height error of high-coherence points in the differential interferogram, and performing atmospheric delay phase correction, to obtain a surface deformation time sequence; setting a third threshold value, dividing the surface deformation time sequence based on the third threshold value, and obtaining the subsidence anomaly area.

[0066] First, the small-baseline differential interferogram is generated: the small-baseline differential interferogram is generated by using the small-baseline interferogram and the SAR image of the coal fire area. n) the same area N scene SAR images acquired at different time, according to the space-time baseline threshold, a small baseline set below the threshold is selected to form M interferograms, then N / 2≤M≤[N(N-1)] / 2; through the use of precise orbit POD and reference digital elevation model DEM data, the interferograms are registered, multi-looked, and the flat ground and terrain phases are removed to obtain the differential interferograms. Secondly, the deformation phase estimation, the linear model is used to estimate the deformation phase of N images, and the singular value decomposition method is used to solve the least square solution of the unknown parameters in the minimum norm sense for each small baseline set sub-baseline, to obtain the linear deformation and height error. Again, the residual phase estimation, on the basis of the linear model, the space-time filtering is performed on the residual phase to further separate the atmospheric phase, the nonlinear deformation phase and part of the residual error and noise, and the final deformation variable is the superposition of the linear deformation and the nonlinear deformation. Finally, the external atmospheric correction data is used to correct the atmospheric delay phase, and the Delaunay grid composed of irregular triangles is used for 3D unwrapping to improve the unwrapping accuracy in low coherence area, and then the final average deformation rate is obtained; the confidence is set to 95%, and the third threshold is set to the mean surface deformation minus 2 times the standard deviation, and the part less than the third threshold is regarded as the subsidence anomaly area.

[0067] In the embodiment, Sarscape software is used for data processing of 36 SAR images, and the images are cropped in the test area. In order to consider the space-time coherence and processing efficiency, the maximum space baseline threshold is set to 2% of the critical baseline, the maximum time baseline threshold is set to 120 days, and the minimum space-time baseline threshold is set to 0, and 114 pairs of small baseline connection diagrams are generated. The images are registered, 4:1 multi-looked in the distance direction and the azimuth direction, and the 30m resolution SRTM DEM data and the precise orbit data POD are used to remove the terrain phase and the orbit phase in the interference phase respectively, the Delaunay minimum cost flow (Delaunay MCF) is used for phase unwrapping, the differential interference phase is edited, and the phase with poor unwrapping quality is discarded. The generic atmospheric correction online service data (GACOS) for InSAR is used to correct the atmospheric delay phase, estimate the average deformation rate and the residual phase component, and obtain the time series of the final surface deformation, and the third threshold is set to the mean surface deformation minus 2 times the standard deviation, and the part less than the third threshold is regarded as the subsidence anomaly area.

[0068] The extraction method of the highlight area comprises: performing radiation calibration and comparison on the luminescent image to obtain the brightness difference of the coal fire area and other ground objects; performing intersection processing on the bright areas of different luminescent images based on the brightness difference to obtain an intersection area; dividing the intersection area into blocks, calculating the brightness average of different blocks in different luminescent images, and segmenting the fire area and the mining area based on the thermal anomaly area and the precipitation anomaly area; and setting a fourth threshold value, and determining the location of the open fire point based on the fourth threshold value, that is, the highlight area.

[0069] The luminescent image is radiation calibrated, the bright areas of two luminescent images in the same month are subjected to intersection processing according to the brightness difference between the coal fire area and other ground objects, and sporadic scattered points caused by the sensor, part of the surface high-radiation unit and atmospheric scattering are filtered out; the intersection area is divided into blocks, the brightness average of each block in the two images is counted, and the brightness value is equally divided into 8 parts from the minimum value to the maximum value, 8 brightness intervals are obtained, the fire area and the mining area are segmented by comprehensively considering the temperature anomaly area and the precipitation anomaly area; the brightness average and the standard deviation of the open fire point range are counted, the threshold value is set as the brightness average plus one times the standard deviation for density segmentation to reduce the light overflow effect, and the location of the open fire point is further determined, and the part higher than the threshold value is regarded as the highlight area.

[0070] In the embodiment, first, the radiation calibration is performed on two scenes of LuoJia-1 luminescent images in the same month, and the formula is expressed as

[0071] L = DN 3 / 2 ·10 -10

[0072] In the formula, L is the absolute radiation corrected radiation brightness value (W / (m 2 ·sr·μm)), and DN is the image gray value.

[0073] The pixel brightness value of the whole luminescent image after calibration is counted, and the area with all values of 0 is subtracted, which is defined as a "bright area". The bright area contains the fire area, the mining area and a large number of sporadic scattered points. In order to segment the ground object types, the bright area is further processed through the following steps: the bright areas of two scenes of luminescent images in the same month are subjected to intersection processing to filter out sporadic scattered points caused by the sensor, part of the surface high-radiation unit and atmospheric scattering; the intersection area is divided into two blocks according to the brightness distribution, the brightness average of each block in the two images is counted, and the brightness value is equally divided into 8 parts from the minimum value to the maximum value, 8 brightness intervals are obtained, and the fire area and the mining area are segmented by comprehensively considering the temperature anomaly area and the precipitation anomaly area; the brightness average and the standard deviation σ of the open fire point range are counted, since the resolution of the luminescent image is low, the light overflow effect causes the possible scattered open fire points to be counted into one pixel, therefore, the density segmentation is performed by taking as the threshold value to further determine the location of the open fire point, and the part higher than the threshold value is regarded as the highlight area.

[0074] S2. Delimit a fire area characteristic sample based on the fire area multi-element information.

[0075] The delimiting method of the fire area characteristic sample comprises: constructing a standard deviation ellipse with the intersection of the thermal anomaly area and the settlement anomaly area to obtain a strongly constrained hidden fire area sample; superimposing a highlight area on the strongly constrained hidden fire area sample to obtain a strongly constrained open fire area sample; and establishing a buffer area outside the standard deviation ellipse to extract independent thermal anomaly areas, independent settlement anomaly areas and non-anomalous feature points in the image of the buffer area to obtain a non-fire area sample.

[0076] In this embodiment, a 1-time standard deviation ellipse is constructed with the intersection of the thermal anomaly area and the settlement anomaly area as the range of the strongly constrained hidden fire area sample, wherein the average center of the ellipse represents the spatial distribution center of gravity of the fire area, the long semi-axis represents the direction of the distribution of the fire area, and the short semi-axis represents the approximate range of the distribution of the fire area; the highlight area is superimposed on the basis of the strongly constrained hidden fire area as the strongly constrained open fire area. The delimiting of the non-fire area sample considers two aspects: ① independent temperature and settlement anomaly areas are extracted, and the category is delimit as non-fire area to improve the ability of the learning device to distinguish that the anomaly areas represented by the two are not coal fire areas and improve the sensitivity of the learning device; ② non-anomalous feature points are extracted, which are near the mean value in the four data sources and are a supplement to the non-fire area sample. Specifically, a 100m buffer area is established outside the standard deviation ellipse based on the temperature anomaly and the settlement anomaly area, the temperature anomaly and the settlement anomaly threshold are reduced to one time standard deviation, and independent temperature anomaly and settlement anomaly data (areas without intersection) in the image are extracted; the data in the buffer area and the single temperature and settlement anomaly data are masked, and non-anomalous feature points are randomly extracted in the remaining area, the number of points is determined according to the image resolution and the image area, and the test area is required to be covered.

[0077] S3. Constructing an integrated learning model based on the fire area characteristic sample, and identifying the fire area range based on the integrated learning model.

[0078] The method for constructing the integrated learning model comprises: oversampling the strongly constrained hidden fire area sample, the strongly constrained open fire area sample and the non-fire area sample, and shuffling the obtained oversampled data; dividing the shuffled data into a training set and a test set according to a preset proportion; training an initial layer based on the training set and the test set to obtain a first output result; training a secondary layer based on the first output result to obtain a second output result; and establishing a final layer identification model based on the second output result to obtain the integrated learning model.

[0079] In this embodiment, the extracted strongly constrained hidden fire area, strongly constrained open fire area and non-fire area sample are oversampled by ADASYN to balance the sample data, 70% of the shuffled data set is taken as the training data set, and 30% is taken as the test data set, wherein the training data set is denoted as Tr and the test data set is denoted as Te.

[0080] As Figure 3As shown, the initial layer includes five base learners: The Classification and Regression Tree (CART), K-Nearest Neighbor (KNN), Back Propagation Neural Network (BPNN), Support Vector Machine (SVM), and Naive Bayes (NB). The training data set is input into the five base learners, and the base learners are trained using Tr. The five trained models are used to respectively predict Tr and Te. Five-fold cross-validation is used to reduce the risk of overfitting. After 10 independent runs, Tr_tree_pred, Tr_knn_pred, Tr_bp_pred, Tr_svm_pred, Tr_nb_pred, and Te_tree_pred, Te_knn_pred, Te_bp_pred, Te_svm_pred, Te_nb_pred are obtained, i.e., the first output result. The 10 output results of the five models predicting Tr are averaged to obtain AV_Pred_tree, AV_Pred_knn, AV_Pred_bp, AV_Pred_svm, and AV_Pred_nb. The accuracy, recall, precision, F1 value, Kappa coefficient, and confusion matrix heat map are used to evaluate the quality of the classification results in multiple dimensions. The BPNN model with the best comprehensive performance is used as a meta-learner (StBP) to participate in the calculation of the secondary learner.

[0081] The secondary layer includes Random Forest (RF) based on Bagging ensemble, Adaptive Boosting (AdaBoost) based on Boosting ensemble, and Stacking-based stacked BP neural network (Stacking_BP, StBP) selected from the base learners.

[0082] Let Tr_Pred_Data = (Tr_tree_pred, Tr_knn_pred, Tr_bp_pred, Tr_svm_pred, Tr_nb_pred), and Te_Pred_Data = (Te_tree_pred, Te_knn_pred, Te_bp_pred, Te_svm_pred, Te_nb_pred). The three meta-learners AdaBoost, RF, and StBP of the secondary layer are trained using Tr_Pred_Data, and the trained models are used to predict Te_Pred_Data to obtain the meta-learner prediction results, i.e., the second output result.

[0083] The three meta-learners of the secondary layer are weighted and averaged to establish the final layer AdaBoost_RF_StBP coal fire area identification model. The weight is determined by the Kappa coefficient. The greater the Kappa coefficient, the greater the weight, and vice versa. The weight formula is:

[0084]

[0085] In the formula, Kappa 1，2，3 is the Kappa value of a model, and Kappa1, Kappa2 and Kappa3 are the Kappa values of the models.

[0086] The weight of each model is multiplied by the prediction probability of the three classification labels naked_fire (nf), concealed_fire (cf) and unfire (uf) of each model, and a weighted average is taken to obtain the final classification label prediction probability. The calculation formula is:

[0087]

[0088]

[0089]

[0090] In the formula, P nf , P cf and P uf represent the probabilities of the naked fire area, the concealed fire area and the non-fire area after the weighted average of the three models, respectively. P n , P c and P u represent the probabilities of the naked fire area, the concealed fire area and the non-fire area predicted by each model, respectively. W 1，2，3 is the weight of a model, and W1, W2 and W3 represent the weights of the three models, respectively.

[0091] The general gobi and the general temple are subjected to secondary masking, and are input into the AdaBoost_RF_StBP model to predict the final fire area range.

[0092] Example Two

[0093] In this embodiment, as shown in Figure 4 , a coal fire area multi-source remote sensing identification system based on three-layer ensemble learning, characterized in that it comprises an information extraction module, a sample demarcation module and an identification module.

[0094] The information extraction module is configured to extract fire area multi-element information based on the multi-modal remote sensing data. The information extraction module comprises a thermal anomaly area extraction unit, a vegetation-rich area extraction unit, a sedimentation anomaly area extraction unit, and a high-brightness area extraction unit. The multi-modal remote sensing data comprises thermal infrared images, multispectral images, SAR images, and night light images. The fire area multi-element information comprises a thermal anomaly area, a vegetation-rich area, a sedimentation anomaly area, and a high-brightness area.

[0095] The workflow of the thermal anomaly area extraction unit comprises: inverting the ground surface temperature of the fire area at multiple time phases of the coal fire area to obtain an inversion result; setting a first threshold value, dividing the inversion result based on the first threshold value to obtain an initial thermal anomaly area; performing intersection processing on the adjacent images of each winter of the initial thermal anomaly area to obtain a first processed image; and stacking the first processed image in units of years to obtain the thermal anomaly area.

[0096] In the present embodiment, the atmospheric correction algorithm based on the thermal radiation transfer equation and the single-window algorithm are used to perform ground surface temperature inversion on the Jiangjun Gobi and Jiangjun Temple test areas by combining 6 thermal infrared images (2 adjacent images in each winter, a total of 3 years). The histograms of the ground surface temperatures obtained by the two algorithms are counted, and the mean value and standard deviation are calculated. The result obtained by the algorithm with more significant normal distribution, smaller standard deviation, and lower discrete degree is selected. The artificial threshold method is used to take the 95% confidence level, and the mean value of the ground surface temperature plus 2 times the standard deviation is taken as the first threshold value. The part higher than the threshold value is the initial thermal anomaly area. On the basis of the first threshold value, the adjacent images of each winter are processed by intersection to weaken the influence of the thermal anomaly area caused by direct sunlight. Then, the images are stacked in units of years to weaken the influence of climate, season, and other factors on the thermal anomaly area, reduce the probability of coal fire being extinguished due to extreme weather, and obtain the final thermal anomaly area.

[0097] The workflow of the vegetation-rich area extraction unit comprises: calculating the normalized vegetation index of the fire area at multiple time phases based on the thermal infrared images; setting a second threshold value, dividing the thermal anomaly area based on the second threshold value to obtain the vegetation-rich area.

[0098] The normalized vegetation index NDVI of the fire area at multiple time phases is calculated by using the red band and near-infrared band of the multispectral sensor. On the basis of the final thermal anomaly area, the second threshold value is determined by adjusting the multiple relationship of the mean value and the standard deviation and combining the actual NDVI value. Finally, the area greater than the threshold value is taken as the vegetation-rich area.

[0099] In the embodiment, the 3 scenes of summer multispectral images corresponding to the years are uniformly radiometrically calibrated, atmospherically corrected, and terrain-corrected, and NDVI is calculated. The average of the calculation results of the 3 scenes of NDVI is taken, and the average value and the standard deviation of NDVI in the range of the final thermal anomaly area are extracted and counted. The average value plus one times the standard deviation is taken as the second threshold value, which retains most of the final thermal anomaly area while filtering the high-NDVI area in the subsequent processing. Finally, the area greater than the second threshold value is taken as the vegetation-rich area.

[0100] The workflow of the subsidence anomaly area extraction unit includes: obtaining a SAR image of a coal fire area, screening out a small baseline interferogram from the SAR image according to a space-time baseline threshold; performing registration and multiple views on the small baseline interferogram, removing terrain phase and orbit phase, and obtaining a differential interferogram; solving linear deformation and height error for high-coherence points in the differential interferogram, and performing atmospheric delay phase correction to obtain a surface deformation time sequence; setting a third threshold value, and dividing the surface deformation time sequence based on the third threshold value to obtain a subsidence anomaly area.

[0101] First, a small baseline differential interferogram is generated: N scenes of SAR images of the same area obtained at (t0, t1, …, t n ) are selected according to a space-time baseline threshold to form M interferograms with a baseline lower than the threshold, and then N / 2≤M≤[N(N-1)] / 2; the interferograms are registered, multiple views are performed, and flat ground and terrain phases are removed to obtain a differential interferogram by using accurate orbit POD and reference digital elevation model DEM data. Second, deformation phase estimation is performed: linear model is used to estimate the deformation phase of N images, and singular value decomposition method is used to solve the least squares solution of unknown parameters in the minimum norm sense for each small baseline set to obtain linear deformation and height error. Third, residual phase estimation is performed: on the basis of the linear model, space-time filtering is performed on the residual phase to further separate the atmospheric phase, nonlinear deformation phase, and part of the residual error and noise, and finally the deformation variable is the superposition of the linear deformation and the nonlinear deformation. Finally, external atmospheric correction data is used to correct the atmospheric delay phase, and a Delaunay grid composed of irregular triangles is used for 3D unwrapping to improve the unwrapping accuracy of low-coherence areas, and then the final average deformation rate is obtained; the confidence is set to 95%, the third threshold value is taken as the average value of the surface deformation minus 2 times the standard deviation, and the part less than the third threshold value is regarded as the subsidence anomaly area.

[0102] In this embodiment, 36 scenes of SAR images are processed using Sarscape software. The images are cropped with the test area as the range. In order to balance the spatial and temporal coherence and processing efficiency, the maximum spatial baseline threshold is set to 2% of the critical baseline, the maximum time baseline threshold is 120 days, and the minimum space-time baseline threshold is 0. A total of 114 pairs of small baseline connection diagrams are generated. The images are registered, and the range and azimuth directions are 4:1 multi-views. The 30m resolution SRTM DEM data and the precise orbit data POD are used to remove the terrain phase and the orbit phase in the interference phase, respectively. The phase is unwrapped using Delaunay Minimum Cost Flow (Delaunay MCF), and the differential interference phase is edited. The unwrapping quality of the phase is poor. The atmospheric delay phase is corrected using the Generic Atmospheric Correction Online Service for InSAR (GACOS) data. The average deformation rate and the residual phase component are estimated, and the time series of the final ground surface deformation is obtained. The third threshold is the mean value of the ground surface deformation minus 2 times the standard deviation. The part less than the third threshold is regarded as a subsidence anomaly area.

[0103] The workflow of the highlight area extraction unit includes: radiometric calibration of night light images and comparison to obtain the brightness difference of coal fire area and other ground objects; based on the brightness difference, the bright areas of different night light images are intersected to obtain the intersection area; the intersection area is divided into blocks, the brightness mean value of different blocks in different night light images is calculated, and the fire area and the mining area are segmented based on the thermal anomaly area and the subsidence anomaly area; the fourth threshold is set, and the position of the open fire point is determined based on the fourth threshold, that is, the highlight area.

[0104] The night light images are radiometrically calibrated. According to the brightness difference between the coal fire area and other ground objects, the bright areas of two night light images in the same month are intersected to filter out sporadic points caused by sensors, partial surface high radiation units and atmospheric scattering. The intersection area is divided into blocks, the brightness mean value of each block in the two images is counted, and the brightness value is equally divided into 8 parts from the minimum value to the maximum value, to obtain 8 brightness intervals. The temperature anomaly area and the subsidence anomaly area are combined to segment the fire area and the mining area. The brightness mean value and the standard deviation of the open fire point range are counted. The threshold is the brightness mean value plus 1 times the standard deviation for density segmentation to reduce the light overflow effect and further determine the position of the open fire point. The part higher than the threshold is regarded as the highlight area.

[0105] In this embodiment, first, the radiometric calibration of two scenes of LuoJia-1 night light images in the same month is performed, and the formula is expressed as

[0106] L = DN 3 / 2 ·10 -10

[0107] L is the absolute radiance-corrected radiance value (W / (m 2 DN is the image gray value.

[0108] The whole night light image pixel brightness value after calibration is counted, and all values of 0 are subtracted, which is defined as "bright area". The bright area contains fire area, mining area and a large number of scattered points. In order to classify the types of ground objects, the bright area is further processed by the following steps: the intersection of the bright areas of the two night light images of the month is processed, and the scattered points caused by the sensor, part of the surface high radiation unit and atmospheric scattering are filtered out; according to the brightness distribution, the intersection area is divided into two parts, the brightness average of each part in the two images is counted, and the brightness value is equally divided into 8 parts from the minimum value to the maximum value, 8 brightness intervals are obtained, and the temperature anomaly area and the sedimentation anomaly area are combined to separate the fire area and the mining area; the brightness average and the standard deviation σ of the range of the open fire point are counted. Due to the low resolution of the night light image, the light overflow effect causes the possible scattered open fire points to be counted into one pixel, so the density segmentation is carried out with as the threshold, and the position of the open fire point is further determined. The part higher than the threshold is regarded as the high-brightness area.

[0109] The sample demarcation module is used to demarcate the fire area characteristic sample based on the multi-element information of the fire area. The sample demarcation module comprises: a concealed fire area demarcation unit, an open fire area demarcation unit and a non-fire area demarcation unit.

[0110] The concealed fire area demarcation unit is used to construct a standard deviation ellipse with the intersection of the thermal anomaly area and the sedimentation anomaly area, to obtain a strongly constrained concealed fire area sample; the open fire area demarcation unit is used to superimpose the high-brightness area on the strongly constrained concealed fire area sample, to obtain a strongly constrained open fire area sample; and the non-fire area demarcation unit is used to establish a buffer zone outside the standard deviation ellipse, to extract the independent thermal anomaly area, the independent sedimentation anomaly area and the non-anomaly characteristic point in the image of the buffer zone, to obtain a non-fire area sample.

[0111] In the embodiment, a 1 standard deviation ellipse is constructed at the intersection of the thermal anomaly area and the subsidence anomaly area as a strong constraint hidden fire area sample range, wherein the average center of the ellipse represents the spatial distribution center of gravity of the fire area, the major axis represents the direction of the fire area distribution, and the minor axis represents the approximate range of the fire area distribution. The "highlight area" is superimposed on the basis of the strong constraint hidden fire area as a strong constraint open fire area. The non-fire area sample is determined from two aspects: ① Extract the single temperature and subsidence anomaly area, and classify it as a non-fire area to improve the ability of the learner to distinguish the anomaly area represented by the two and improve the sensitivity of the learner; ② Extract non-anomaly feature points, which are near the mean in the four data sources, and are a supplement to the non-fire area samples. Specifically, the standard deviation ellipse is established based on the temperature anomaly and the subsidence anomaly area, a 100m buffer area is established outside the ellipse, the temperature anomaly and the subsidence anomaly threshold is reduced to one standard deviation, and the independent temperature anomaly and subsidence anomaly data (areas without intersection) in the image are extracted; the data in the buffer area and the single temperature and subsidence anomaly data are masked, and the non-anomaly feature points are randomly extracted in the remaining area. The number of points is determined according to the image resolution and the image area, and the test area is required to be covered.

[0112] The recognition module is configured to construct an ensemble learning model based on the fire area feature samples, and to recognize the fire area range based on the ensemble learning model.

[0113] The workflow of the recognition module includes: oversampling the strong constraint hidden fire area samples, the strong constraint open fire area samples, and the non-fire area samples, shuffling the obtained oversampled data, and dividing the shuffled data into a training set and a test set according to a preset ratio; training an initial layer based on the training set and the test set to obtain a first output result; training a secondary layer based on the first output result to obtain a second output result; establishing a final layer recognition model based on the second output result to obtain an ensemble learning model.

[0114] In the embodiment, the extracted strong constraint hidden fire area, strong constraint open fire area, and non-fire area samples are subjected to ADASYN oversampling to balance the sample data, and the shuffled data set is divided into 70% as a training data set and 30% as a test data set. The training data set is denoted as Tr, and the test data set is denoted as Te.

[0115] The initial layer includes five base learners of The Classification and Regression Tree (CART), K-Nearest Neighbor (KNN), Back Propagation Neural Network (BPNN), Support Vector Machine (SVM) and Naive Bayes (NB). The training data set is input into the five base learners, the base learners are trained by Tr respectively, and the five trained models are used to predict Tr and Te respectively. Five-fold cross-validation is used to reduce the risk of overfitting. After 10 independent runs, Tr_tree_pred, Tr_knn_pred, Tr_bp_pred, Tr_svm_pred, Tr_nb_pred and Te_tree_pred, Te_knn_pred, Te_bp_pred, Te_svm_pred, Te_nb_pred are obtained, i.e. the first output result. The average of the 10 output results of the five models for predicting Tr is obtained, i.e. AV_Pred_tree, AV_Pred_knn, AV_Pred_bp, AV_Pred_svm and AV_Pred_nb. The accuracy, recall, precision, F1 value, Kappa coefficient and confusion matrix heat map are used to evaluate the quality of the classification results in multiple dimensions. The best comprehensive performance of the BPNN model is used as a meta-learner (StBP) to participate in the calculation of the secondary learner.

[0116] The secondary layer includes Random Forest (RF) based on Bagging ensemble, Adaptive Boosting (AdaBoost) based on Boosting ensemble and Stacking_BP (StBP) based on Stacking selected from the base learner.

[0117] Tr_Pred_Data = (Tr_tree_pred, Tr_knn_pred, Tr_bp_pred, Tr_svm_pred, Tr_nb_pred), Te_Pred_Data = (Te_tree_pred, Te_knn_pred, Te_bp_pred, Te_svm_pred, Te_nb_pred), and the three meta-learners AdaBoost, RF and StBP of the secondary layer are trained by using Tr_Pred_Data, and the trained models are used to predict Te_Pred_Data, to obtain the meta-learner prediction results, i.e. the second output result.

[0118] The three meta-learners of the secondary layer are weighted and averaged to establish the final layer AdaBoost_RF_StBP coal fire area identification model. The weight is determined by the Kappa coefficient. The greater the Kappa coefficient, the greater the weight, and vice versa. The weight formula is:

[0119]

[0120] In the formula, Kappa 1，2，3 is the Kappa value of a model, and Kappa1, Kappa2 and Kappa3 are the Kappa values of the respective models.

[0121] The weight of each model is multiplied by the prediction probability of the three classification labels naked_fire (nf), concealed_fire (cf) and unfire (uf) of each model, and a weighted average is taken to obtain the final classification label prediction probability. The calculation formula is:

[0122]

[0123]

[0124]

[0125] In the formula, P nf , P cf and P uf represent the probabilities of the naked fire area, concealed fire area and non-fire area after the weighted average of the three models, respectively. P n , P c and P u represent the probabilities of the naked fire area, concealed fire area and non-fire area predicted by each model, respectively. W 1，2，3 is the weight of a model, and W1, W2 and W3 represent the respective weights of the three models.

[0126] The general gobi and general temple are subjected to secondary masking, and are input into the AdaBoost_RF_StBP model to predict the final fire area range.

[0127] Example Three

[0128] In the embodiment, by comparing the effects of field detection and model identification, it is found that the identification model constructed in the application has a high degree of overlap with the field detection in delineating the fire area range, and the separation degree of hidden fire area, open fire area and non-fire area is good, and the misjudgment phenomenon of the three fire areas basically does not exist, which better compensates for the misjudgment phenomenon of the filtering method caused by the small vertical superposition range of temperature anomaly and subsidence anomaly, and only two misjudgment areas exist. The model trained by taking the general gobi and general temple test area as the data source and applied in the sandao dam test area still shows good identification accuracy, which can reflect that the application in the multi-factor extraction of fire area, the strong constraint of hidden fire area, open fire and non-fire area sample delineation stage can not only make the sample set represent the characteristics of coal fire combustion, but also ensure the high reliability of the sample set. The three-layer improved ensemble learning model also has greater advantages in accuracy and generalization than single machine learning model or single ensemble learning model because it integrates three kinds of ensemble learning ideas.

[0129] The above-described embodiments are only descriptions of the preferred modes of the application and do not limit the scope of the application. Without departing from the design spirit of the application, various modifications and improvements to the technical solutions of the application made by those skilled in the art shall fall within the protection scope determined by the claims of the application.

Claims

1. A coal fire area multi-source remote sensing identification method based on three-layer ensemble learning, characterized by, The method comprises the following steps: Based on multi-modal remote sensing data, the fire area multi-element information is extracted, and the multi-modal remote sensing data comprises thermal infrared images, multispectral images, SAR images and night light images; Based on the fire area multi-element information, the fire area characteristic sample is demarcated; Based on the fire area characteristic sample, an integrated learning model is constructed, and the fire area range is identified based on the integrated learning model; The fire area multi-element information comprises a thermal anomaly area, a vegetation-rich area, a settlement anomaly area and a high-light area; The extraction method of the vegetation-rich area comprises: Based on the thermal infrared images, the normalized vegetation index of the fire area at multiple time phases is calculated; A second threshold value is set, and the thermal anomaly area is divided based on the second threshold value to obtain the vegetation-rich area; The vegetation-rich area is filtered out from the thermal anomaly area; The demarcation method of the fire area characteristic sample comprises: The intersection of the thermal anomaly area after filtering out the vegetation-rich area and the settlement anomaly area is constructed into a standard deviation ellipse to obtain a strong constraint hidden fire area sample; The high-light area is superimposed on the strong constraint hidden fire area sample to obtain a strong constraint open fire area sample; A buffer area is established outside the standard deviation ellipse, and independent thermal anomaly areas, independent settlement anomaly areas and non-anomaly feature points in the image of the buffer area are extracted to obtain a non-fire area sample.

2. The coal fire area multi-source remote sensing identification method based on three-layer ensemble learning according to claim 1, characterized in that, The extraction method of the thermal anomaly area comprises: The ground surface temperature of the coal fire area at multiple time phases is inverted to obtain an inversion result; A first threshold value is set, and the inversion result is divided based on the first threshold value to obtain an initial thermal anomaly area; The adjacent images of the initial thermal anomaly area in winter of each year are intersected to obtain a first processed image; The first processed image is superimposed in units of years to obtain the thermal anomaly area.

3. The coal fire area multi-source remote sensing identification method based on three-layer ensemble learning according to claim 1, characterized in that, The extraction method of the settlement anomaly area comprises: The SAR images of the coal fire area are obtained, and small baseline interferograms are screened out from the SAR images according to a space-time baseline threshold value; The small baseline interferograms are registered, multi-visual, and the terrain phase and the orbit phase are removed to obtain difference interferograms; Linear deformation and height error of high-coherence points in the difference interferograms are solved, and atmospheric delay phase correction is performed to obtain a ground surface deformation time sequence; A third threshold value is set, and the ground surface deformation time sequence is divided based on the third threshold value to obtain the settlement anomaly area.

4. The coal fire area multi-source remote sensing identification method based on three-layer ensemble learning according to claim 1, characterized in that, The extraction method of the high-light area comprises: The night light images are radiometrically calibrated and compared to obtain the brightness difference of the coal fire area and other ground objects; Based on the brightness difference, the bright areas of different night light images are intersected to obtain an intersection area; The intersection area is divided into blocks, the brightness average of different block areas in different night light images is calculated, and the fire area and the mining area are segmented based on the thermal anomaly area and the settlement anomaly area; A fourth threshold value is set, and the position of the open fire point, i.e. the high-light area, is determined based on the fourth threshold value.

5. The coal fire area multi-source remote sensing identification method based on three-layer ensemble learning according to claim 1, characterized in that, The method for constructing the integrated learning model comprises: The strong constraint hidden fire area sample, the strong constraint open fire area sample and the non-fire area sample are oversampled, and the obtained oversampled data is shuffled to divide the shuffled data into a training set and a test set according to a preset ratio; The training set and the test set are used to train an initial layer to obtain a first output result; A secondary layer is trained based on the first output result to obtain a second output result; A final layer prediction model is established based on the second output result to obtain the ensemble learning model.

6. A coal fire area multi-source remote sensing identification system based on three-layer ensemble learning, characterized by, Comprise: An information extraction module, a sample delineation module, and an identification module; The information extraction module is used to extract fire area multi-element information based on multi-modal remote sensing data, wherein the multi-modal remote sensing data comprises thermal infrared images, multispectral images, SAR images, and night light images; The sample delineation module is used to delineate fire area characteristic samples based on the fire area multi-element information; The identification module is used to construct an ensemble learning model based on the fire area characteristic samples and identify the fire area range based on the ensemble learning model; The fire area multi-element information comprises thermal anomaly areas, vegetation-rich areas, sedimentation anomaly areas, and high-brightness areas; The extraction method of the vegetation-rich area comprises: The normalized vegetation index of the fire area at multiple time phases is calculated based on the thermal infrared images; A second threshold value is set, and the thermal anomaly area is divided based on the second threshold value to obtain the vegetation-rich area; The vegetation-rich area is filtered out from the thermal anomaly area; The delineation method of the fire area characteristic sample comprises: A standard deviation ellipse is constructed based on the intersection of the thermal anomaly area after filtering out the vegetation-rich area and the sedimentation anomaly area to obtain a strongly constrained hidden fire area sample; The high-brightness area is superimposed on the strongly constrained hidden fire area sample to obtain a strongly constrained open fire area sample; A buffer area is established outside the standard deviation ellipse, and independent thermal anomaly areas, independent sedimentation anomaly areas, and non-anomalous feature points in the image of the buffer area are extracted to obtain a non-fire area sample.

Images