Forest fire sample library generation method based on remote sensing inversion

Abstract

The invention is suitable for the technical field of disaster monitoring, and provides a forest fire sample library generation method based on remote sensing inversion, which comprises the steps of image data collection, data preprocessing, overfire area extraction, overfire area post-processing, sample library generation and the like. According to the invention, inversion extraction is carried out on the earth surface change in the remote sensing image, and a whole-process intelligent and automatic solution from fire information acquisition to sample data set production is automatically generated. According to the method, a large-scale fire sample data set can be automatically produced, the time and cost of manual labeling are greatly reduced, and efficient and accurate data support is provided for training and optimization of a fire monitoring and disaster evaluation model. According to the data set generated by the technology, the accuracy, the response speed and the adaptive capacity of a fire monitoring model are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of disaster monitoring, and particularly relates to a method for generating a forest fire sample library based on remote sensing inversion. Background Art

[0002] With the intensification of global climate change, the frequency and intensity of forest fires have been increasing year by year, posing a serious threat to the ecosystem, climate regulation, and human society. To effectively monitor and respond to forest fires, artificial intelligence (AI) models based on deep learning have been widely used in fire detection and disaster assessment. However, the development and deployment of these models are extremely dependent on large-scale and high-quality labeled sample data sets. The traditional manual annotation method consumes a large amount of manpower and time, and the cost is high, making it difficult to meet the fire monitoring requirements of large-scale, multi-time, and multi-region.

[0003] Remote sensing technology obtains rich environmental information through multi-band and multi-temporal surface images and has become one of the important tools for fire monitoring. With the continuous progress of remote sensing technology and inversion algorithms, remote sensing images show increasing application potential in fire monitoring and assessment. However, the existing technology still faces the problems of high cost and low efficiency of manual annotation, which limits the automatic generation of fire monitoring data. To address these challenges, there is an urgent need for a technical solution that can automatically retrieve and extract fire features from remote sensing images and generate the training data set required by deep learning models. Summary of the Invention

[0004] In view of the above problems, the purpose of the present invention is to provide a method for generating a forest fire sample library based on remote sensing inversion, aiming to solve the technical problems that the existing traditional manual annotation method for fire samples consumes a large amount of manpower and time and has a high cost.

[0005] The present invention adopts the following technical solutions:

[0006] The method for generating a forest fire sample library based on remote sensing inversion includes the following steps:

[0007] Step S1, Image Data Collection

[0008] Collect satellite image data within a specific time period before and after the occurrence of a forest fire, including the fire area range, according to the historical time and location of forest fire occurrence;

[0009] Step S2, Data Preprocessing

[0010] First, perform geometric correction, radiometric correction, and atmospheric correction on the satellite images, and then resample each band of the satellite images so that the satellite images of all bands are unified to the same spatial scale;

[0011] Step S3, Extract Burned Areas

[0012] Based on the characteristics of the burned area among different bands, remote sensing inversion technology is used to extract the burned area;

[0013] Step S4, Post-processing of the burned area

[0014] The burned area is processed by area filtering, noise filtering and hole filling, and the sample labels are marked;

[0015] Step S5, Generating a sample library

[0016] The burned area is cropped according to the model requirements to obtain a sample library, and the sample library is divided into a training set, a validation set and a test set.

[0017] Furthermore, in step S1, the satellite image data is downloaded from the website, and satellite images without clouds and shadows are selected, covering the geographical area of the fire. The satellite image data is a multi-spectral image or a thermal infrared image.

[0018] Furthermore, the specific process of extracting the burned area in step S3 is as follows:

[0019] Calculate the Normalized Burn Ratio (NBR) of the satellite images of the same area before and after the fire, and subtract the two NBR results to obtain the NBR difference. Calculate the segmentation threshold for automatically annotating the fire boundary and the unaffected area according to the NBR difference map;

[0020] Extract the burned area of the fire satellite image through the segmentation threshold.

[0021] Furthermore, the calculation method of the segmentation threshold is as follows:

[0022] S31. Use the fuzzy Gaussian membership function to map the pixel values of the NBR difference map to the membership range [0,1]:

[0023]

[0024] where x represents the pixel value, c represents the central value of the fuzzy Gaussian membership function, taking the global pixel value mean, σ represents the standard deviation of the Gaussian distribution, determining the degree of expansion of the membership. For each pixel, μ(x) represents its membership to the foreground or background;

[0025] S32. Determine the optimal threshold T by maximizing the between-class variance * : Calculate the total pixel mean, foreground weight, background weight, foreground mean and background mean respectively, and then combine the between-class variance of the membership. By traversing all possible thresholds T, find the threshold that maximizes the between-class variance, that is, the optimal threshold T * , and the optimal threshold is the segmentation threshold to be calculated, where:

[0026] Total pixel mean:

[0027]

[0028] Foreground weight:

[0029]

[0030] Background weight:

[0031]

[0032] Foreground mean:

[0033]

[0034] Background mean:

[0035]

[0036] Between-class variance of membership degree:

[0037]

[0038] Traverse the threshold T to find the optimal threshold T that maximizes : * :

[0039]

[0040] Pixels with values greater than or equal to T * are the burned areas, and those less than T * are non-burned areas.

[0041] Furthermore, the specific process of post-processing the burned area in step S4 is as follows:

[0042] Set an area threshold to filter out patches in the burned area with discontinuous areas smaller than the area threshold;

[0043] Use the erosion algorithm to remove small isolated pixel points in the burned area to clean up the noise;

[0044] Use the dilation algorithm to increase the area of the burned area in the image to fill small holes.

[0045] Furthermore, the specific process of generating the sample library in step S5 is as follows:

[0046] According to the size requirements of the model, crop the satellite image according to the cropping rules to obtain image patches, and each patch needs to cover the core area of the fire and the surrounding unaffected areas;

[0047] Perform data augmentation on the cropped image patches to obtain the sample library, and the data augmentation operations include random rotation, scaling, and translation operations;

[0048] The sample library is divided into a training set, a validation set, and a test set.

[0049] The beneficial effects of the present invention are as follows: Through the remote sensing inversion technology, the present invention can accurately extract key surface information before and after a fire from remote sensing satellite images, such as the fire range, etc. Specifically, for remote sensing satellite images before and after a forest fire, through preprocessing steps, the burned area is extracted based on the remote sensing inversion technology and some post-processing steps are carried out, and then sample labels are automatically marked. Finally, a high-quality fire sample data set is constructed through image cropping. Moreover, in the process of extracting the burned area, a method for calculating the segmentation threshold is designed to make the segmentation and extraction more accurate. Through the automated processing flow, the present invention effectively reduces the time and cost of manual annotation, thereby efficiently producing a large-scale data set required for the training of deep learning models. Based on these automatically generated fire sample data sets, the training efficiency and accuracy of the fire monitoring model can be significantly improved, and the data set can be continuously updated to adapt to the dynamically changing fire monitoring requirements. Brief Description of the Drawings

[0050] Figure 1 is a flowchart of a method for generating a forest fire sample library based on remote sensing inversion provided by an embodiment of the present invention;

[0051] Figure 2 is a screenshot of the query image interface;

[0052] Figure 3 is a screenshot of the setting interface for resampling preprocessing;

[0053] Figure 4 is a schematic diagram of the NBR index, difference map, and marked burned area of the image before and after the fire;

[0054] Figure 5 is a schematic diagram of image cropping. Detailed Embodiments

[0055] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] Artificial intelligence models based on deep learning have been widely used in fire detection and disaster assessment. The development and deployment of these models rely extremely on large-scale and high-quality labeled sample data sets. Remote sensing inversion technology is a technology for inferring surface or atmospheric parameters from remote sensing data, and it has important applications in many fields such as environmental monitoring, climate change research, and resource management. The present invention uses remote sensing inversion technology, which can not only accurately extract the post-fire surface change information from remote sensing images, but also automatically generate a large-scale fire sample data set, thus greatly reducing the dependence on manual annotation. Through the analysis and inversion of remote sensing images, key information such as the fire range and fire intensity can be automatically extracted, which not only improves the efficiency of data set construction, but also significantly improves the accuracy and application effect of deep learning model training. In order to illustrate the technical solution described in the present invention, specific embodiments are used for illustration below.

[0057] As Figure 1 shown, the method for generating a forest fire sample library based on remote sensing inversion provided in this embodiment includes the following steps:

[0058] Step S1, image data collection: According to the historical forest fire occurrence time and location, collect satellite image data within a specific time period before and after the forest fire, including the fire area range.

[0059] In this step, the satellite image data is downloaded from the website. For example, according to the forest fire occurrence time and location, query and obtain the satellite image data within a specific time period before and after the fire on the Copernicus Open Access Hub website. Among them, satellite images without clouds and shadows are selected and cover the specified geographical area. The downloaded images usually include multi-spectral images, thermal infrared images, etc., and these data are crucial for identifying and analyzing the changes in the fire area.

[0060] For example, for a forest fire that occurred in a certain village in a certain county in Sichuan Province, based on this event, access the Copernicus OpenAccess Hub website, and after screening, Figure 2 as shown, the following two images are downloaded: the satellite image before the fire and the satellite image after the fire, and the file names are respectively "S2B_MSIL2A_20240309T034559_N0510_R104_T47RPP_20240309T061216.SAFE" and "S2B_MSIL2A_20240329T034539_N0510_R104_T47RPP_20240329T060618.SAFE".

[0061] Step S2, Data preprocessing: First, perform geometric correction, radiometric correction, and atmospheric correction on the satellite image. Then, resample each band of the satellite image so that the satellite images of all bands are unified to the same spatial scale.

[0062] This step preprocesses the satellite image to improve the quality, accuracy, and usability of the satellite images before and after downloading, laying a good foundation for subsequent remote sensing data analysis, interpretation, and application.

[0063] First, the preprocessing includes geometric correction, radiometric correction, and atmospheric correction. Among them, since remote sensing images may be affected by geometric distortions such as terrain effects and sensor tilts during the acquisition process, geometric correction is required to ensure that the geographic coordinates of the image match the actual location on the Earth's surface, thereby providing accurate geographic reference. Second, the intensity of the electromagnetic radiation received by the remote sensing sensor may be affected by atmospheric conditions (such as scattering and absorption), resulting in distortion of the reflectance or radiation values in the image. Therefore, through radiometric correction in this step, the influence of atmospheric effects can be eliminated or reduced, and the true reflectance information of the ground objects can be restored. In addition, atmospheric correction focuses on eliminating the spectral information distortion caused by atmospheric scattering and absorption in the image, ensuring that the spectral characteristics reflected in the image are closer to the true situation of the ground objects.

[0064] After the above process is completed, resample each band of the satellite image. Since the spatial resolutions of the various bands of the satellite image may be inconsistent, direct calculation may cause pixel misalignment, affecting the accuracy of the results. Therefore, through resampling in this step, the images of all bands can be unified to the same spatial scale, ensuring the accuracy and consistency of the analysis and reducing the influence of the mixed pixel effect.

[0065] Such as Figure 3 The schematic diagram of the resampling preprocessing interface settings shown. The Sentinel-2 satellite image has multi-spectral bands, and its spatial resolutions are divided into three categories: 10 meters, 20 meters, and 60 meters. Based on the subsequent calculation formula of the Normalized Burn Ratio (NBR), in this embodiment, the near-infrared band B8 with a resolution of 10 meters and the short-wave infrared band B11 with a resolution of 20 meters are selected. To unify the spatial resolution, the B11 band is resampled to 10 meters and synthesized with the multi-band images of B2 (blue band), B3 (green band), B5 (red edge band), and B8 (near-infrared band).

[0066] Step S3, Extract the burned area: Based on the characteristics of the burned area among each band, use remote sensing inversion technology to extract the burned area.

[0067] In this step, remote sensing inversion technology is used to extract the targets in the burned area. The Normalized Burn Ratio (NBR) is used to evaluate forest fires and other burned areas, mainly for monitoring the degree of vegetation damage after a fire.

[0068] The calculation of NBR depends on the reflectance difference between the near-infrared (NIR) and short-wave infrared (SWIR) bands in the remote sensing image. By comparing the reflectances of the near-infrared and short-wave infrared bands, NBR can reflect the changes after vegetation combustion. Generally, healthy unburned vegetation has a higher reflectance in the near-infrared band and a lower reflectance in the short-wave infrared band. When the vegetation is affected by a fire, the near-infrared reflectance decreases significantly, while the short-wave infrared reflectance increases, resulting in a corresponding change in the NBR value. The calculation formula of the NBR value is as follows:

[0069]

[0070] where NIR represents the reflectance of the near-infrared band and SWIR represents the reflectance of the short-wave infrared band.

[0071] By comparing the NBR values before and after the fire, the burned area of the fire can be effectively extracted. The NBR value before the fire is higher, while the NBR value after the fire is lower. In this step, the Normalized Burn Ratio NBR of the satellite images of the same area before and after the fire is calculated, and the difference between the two Normalized Burn Ratio results is obtained to get the NBR difference. The segmentation threshold for automatically marking the fire boundary and the unaffected area is calculated according to the NBR difference map; then the burned area of the fire satellite image is extracted through the segmentation threshold. The area greater than or equal to the segmentation threshold is the burned area, and the area less than the segmentation threshold is the non-burned area.

[0072] In this step, the fire area is not directly distinguished by calculating the result of the NBR value. Since it is considered that the time, location, and environmental parameters of the image are different, there will be significant errors in directly using the NBR value to divide the fire area, especially the inability to compare the fire differences horizontally. Therefore, the difference in NBR (Delta NBR, dNBR) before and after the fire is used in this step to evaluate the impact of the fire on vegetation. By calculating an appropriate segmentation threshold, the fire boundary and the unaffected area can be automatically marked, helping to delimit the affected area.

[0073] As Figure 4 shown, by calculating the Normalized Burn Ratio (NBR) for the two images before and after the fire, the difference map of the pre-disaster and post-disaster indices (dNBR) is obtained, and the segmentation threshold is calculated based on the difference map.

[0074] Due to the characteristics of same-spectrum different objects and same-object different spectra in remote sensing images, the problem of segmenting the boundary of the burned area becomes more complex. Even after calculating the difference map and setting a fixed segmentation threshold, the boundary may be inaccurate, and there may be cases of missed segmentation or misclassification. This embodiment provides a new segmentation algorithm GFTO (Gaussian-Fuzzy Threshold Optimization), that is, a method for calculating the segmentation threshold. By using the fuzzy Gaussian membership function to weight the pixel values, fuzzy information is introduced when calculating the weights, means, and between-class variances of the foreground (burned area) and background (unburned area) of the image, making the selection of the segmentation threshold more robust and adaptable to the ambiguity and noise of complex images.

[0075] The specific calculation method of the segmentation threshold is as follows:

[0076] S31. Use the fuzzy Gaussian membership function to map the pixel values of the NBR difference map to the membership degree range [0, 1]:

[0077]

[0078] where x represents the pixel value, c represents the central value of the fuzzy Gaussian membership function, taking the global pixel value mean, σ represents the standard deviation of the Gaussian distribution, determining the degree of expansion of the membership degree, and for each pixel, μ(x) represents its membership degree of belonging to the foreground or background;

[0079] S32. Determine the optimal threshold T by maximizing the between-class variance * : Calculate the total pixel mean, foreground weight, background weight, foreground mean, and background mean respectively, and then combine the between-class variance of the membership degree. By traversing all possible thresholds T, find the threshold that maximizes the between-class variance, that is, the optimal threshold T * , and the optimal threshold is the segmentation threshold to be calculated, where:

[0080] Total pixel mean:

[0081]

[0082] Foreground weight:

[0083]

[0084] Background weight:

[0085]

[0086] Foreground mean:

[0087]

[0088] Background mean:

[0089]

[0090] Between-class variance of membership degree:

[0091]

[0092] Traverse the threshold T. For example, traverse each pixel value of the difference map to find the optimal threshold T that maximizes the best threshold T * :

[0093]

[0094] Pixels with values greater than or equal to T * are the burned areas, and those less than T * are non-burned areas. For example, if the calculated optimal threshold (i.e., the segmentation threshold) is 0.25, areas with dNBR less than 0.25 are non-burned areas, and areas with dNBR greater than or equal to 0.25 are burned areas, and they are distinguished in this way.

[0095] Step S4, Post-processing of burned areas: Process the burned areas through area filtering, noise filtering, and hole filling, and mark the sample labels.

[0096] After obtaining the burned areas through the foregoing steps, to ensure the accuracy of the results, the segmentation threshold set in the foregoing steps may result in some discontinuous small-area regions or noise (i.e., small patches). These small patches are usually caused by sensor noise or the characteristics of same-object different-spectra and same-spectra different-objects in the image itself, and do not reflect the actual fire area. Therefore, these irrelevant regions need to be removed through post-processing.

[0097] The post-processing process of this step is as follows:

[0098] S41. Set the area threshold to filter out patches in the burned areas with discontinuous areas smaller than the area threshold.

[0099] S42. Use the erosion algorithm to remove small isolated pixel points in the burned areas to clean the noise;

[0100] S43. Use the dilation algorithm to increase the area of the burned areas in the image to fill small holes.

[0101] In the specific process, first filter by the area of the patches to remove patches with areas smaller than the set area threshold. The specific area threshold can be set according to actual needs. Secondly, use morphological operation image processing technology to filter noise and fill small holes; in this embodiment, the erosion algorithm is used to remove small isolated pixel points in the image to clean the noise; at the same time, the dilation algorithm is used to increase the area of the burned areas in the image, thereby filling small holes. This series of post-processing steps helps to improve the accuracy and reliability of fire area extraction.

[0102] Step S5: Generate a sample library: The root model requirements are used to crop the overfire area to obtain a sample library, and the sample library is divided into a training set, a validation set, and a test set.

[0103] The process of this step is as follows:

[0104] S51: According to the size requirements of the model, the satellite image is cropped according to the cropping rules to obtain image patches, and each patch needs to cover the core area of the fire and the surrounding unaffected areas.

[0105] To meet the training requirements of the deep learning model, clear sizes and cropping rules are set during the image cropping in this step. During the cropping process, special attention is paid to ensuring that each image patch not only covers the core area of the fire but also includes the surrounding unaffected parts, so as to enhance the diversity and richness of the dataset.

[0106] S52: The cropped image patches are subjected to data augmentation to obtain a sample library, and the data augmentation operations include random rotation, scaling operations, and translation operations.

[0107] To further improve the generalization ability of the model and its adaptability to different environments, this embodiment further performs data augmentation operations on the cropped image patches. These operations include random rotation and scaling to simulate image effects at various perspectives and different distances, and translation to simulate different geographical locations. Through these transformations, the model can learn more diverse features during training, helping it to adapt to complex scenarios that may be encountered in reality.

[0108] S53: Divide the sample library into a training set, a validation set, and a test set.

[0109] When dividing the sample library, the proportions of fire and non-fire areas, as well as the temporal and spatial distributions of the images, need to be fully considered. To ensure that the sample sizes in each subset are sufficient and diverse, stratified sampling is performed on different types of fire data. Such a division strategy not only ensures the balance and diversity of the training set and the test set but also effectively avoids biases that may occur during the model training process, thereby improving the training effect and evaluation reliability of the model.

[0110] Taking the aforementioned satellite images as an example, the satellite images before and after the fire are overlaid with the corresponding sample label information, and the overlaid images are cropped using a 256*256 pixel window, with an overlap rate set at 20% to generate a series of training samples. To enhance the generalization ability of the model, data augmentation is applied to the cropped samples using techniques such as random rotation, scaling, and translation, thereby enhancing the diversity of the samples and the robustness of the model. As Figure 5 shown in the 12 cropped image patches and corresponding labels.

[0111] In summary, the present invention provides a method for generating a forest fire sample library based on remote sensing inversion, aiming to automatically generate a full-process intelligent and automated solution from fire information acquisition to sample dataset production by inverting and extracting surface changes in remote sensing images. This method can not only automatically produce a large-scale fire sample dataset, greatly reducing the time and cost of manual annotation, but also provide efficient and accurate data support for the training and optimization of fire monitoring and disaster assessment models. The dataset generated by this technology will significantly improve the accuracy, response speed and adaptability of the fire monitoring model, and promote the advancement of fire disaster situation analysis towards full automation.

[0112] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for generating a forest fire sample library based on remote sensing inversion, characterized in that: The method comprises the following steps: Step S1: Image data collection According to the time and location of historical forest fires, satellite image data of a specific time period before and after the forest fire, including the fire area, is collected; Step S2: Data preprocessing First, the satellite images are geometrically corrected, radiometrically corrected, and atmospherically corrected. Then, each band of the satellite images is resampled so that the satellite images of all bands are unified to the same spatial scale. Step S3: Extracting the burned area Based on the characteristics of the burned area in each band, the burned area is extracted using remote sensing inversion technology. Step S4: Post-processing of the burned area Process the burned area through area filtering, noise filtering and hole filling, and mark the sample labels; Step S5: Generate sample library The root model needs to trim the overfire area to obtain a sample library, and divide the sample library into training set, validation set and test set.

2. The method for generating a forest fire sample library based on remote sensing inversion as claimed in claim 1, characterized in that: In step S1, the satellite image data is downloaded from a website, and satellite images without clouds and shadows are selected, and cover the geographical area of ​​the fire. The satellite image data is a multispectral image or a thermal infrared image.

3. The method for generating a forest fire sample library based on remote sensing inversion as claimed in claim 2, characterized in that: The specific process of extracting the burned area in step S3 is as follows: Calculate the normalized burn index (NBR) of satellite images of the same area before and after the fire, and subtract the two normalized burn index results to obtain the NBR difference map. According to the NBR difference map, calculate the segmentation threshold for automatically marking the fire boundary and the unaffected area. Extract the burned area from fire satellite images by segmentation threshold.

4. The method for generating a forest fire sample library based on remote sensing inversion as claimed in claim 3, characterized in that: The segmentation threshold is calculated as follows: S31, using the fuzzy Gaussian membership function to map the pixel values ​​of the NBR difference map to the membership range [0,1]: Where x represents the pixel value, c represents the central value of the fuzzy Gaussian membership function, takes the global pixel value mean, σ represents the standard deviation of the Gaussian distribution, and determines the extension degree of the membership. For each pixel, μ(x) represents its membership of the foreground or background. S32. Determine the optimal threshold T by maximizing the inter-class variance * : Calculate the total pixel mean, foreground weight, background weight, foreground mean and background mean respectively, and then combine the inter-class variance of membership, and find the threshold that maximizes the inter-class variance by traversing all possible thresholds T, that is, the optimal threshold T * , the optimal threshold is the separation threshold to be calculated, where: Total pixel mean: Foreground Weight: Background Weight: Outlook mean: Background mean: The between-class variance of membership degree: Traverse the threshold T and find The optimal threshold T that maximizes * : Pixel value is greater than or equal to T * is the fire area, less than T * It is a non-fire area.

5. The method for generating a forest fire sample library based on remote sensing inversion as claimed in claim 4, characterized in that: The specific process of post-processing the burned area in step S4 is as follows: Set an area threshold to filter out discontinuous spots in the burned area whose area is smaller than the area threshold; The erosion algorithm is used to remove small isolated pixels in the burned area to clean up the noise; A dilation algorithm is used to increase the area of ​​the burned region in the image to fill small holes.

6. The method for generating a forest fire sample library based on remote sensing inversion as claimed in claim 5, characterized in that: The specific process of generating a sample library in step S5 is as follows: According to the size requirements of the model, the satellite image is cropped according to the cropping rules to obtain small image blocks. Each small block must cover the core area of ​​the fire and the surrounding unaffected areas; Perform data enhancement on the cropped image blocks to obtain a sample library. The data enhancement operations include random rotation, scaling, and translation. The sample library is divided into training set, validation set and test set.

Images