Fire identification and burned area estimation method based on multi-source heterogeneous data fusion

Abstract

The present application provides a forest fire identification and burned area estimation method based on multi-source heterogeneous data fusion, comprising: acquiring a satellite remote sensing image containing a hot spot target and pre-processing; using Faster R-CNN to detect the hot spot target of the satellite remote sensing image, obtaining an image containing a hot spot prediction box and the corresponding prediction box coordinates; performing coordinate conversion on the prediction box, using the global forest cover map for multi-source data fusion, and realizing the filtering of non-forest fire areas; respectively using NDVI threshold method and NBR threshold method to estimate the burned area; using satellite thermal infrared remote sensing data to detect forest fires, the thermal infrared remote sensing is sensitive to high temperature target identification, through the gray value difference between the forest fire area and the background area in the remote sensing image, and the fusion of the global forest cover map effectively distinguishes the fire point, does not depend on visible light, has high time resolution, wide image coverage, can realize all-weather large-scale forest fire monitoring, and the satellite revisit period is conducive to the dynamic monitoring and process analysis of forest resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of urban computing, in particular to a mountain fire identification and burned area estimation method based on multi-source heterogeneous data fusion. BACKGROUND

[0002] Under the global warming trend, the abnormally high temperature in summer makes the forest more drought, increasing the frequency and severity of forest fires. China is rich in forest resources, with a variety of plants. Once a forest fire occurs, it will spread rapidly, causing houses and trees to be destroyed, seriously damaging the forest structure and ecological environment, polluting the surrounding air, and threatening the safety of residents' lives and property. Forest fires are highly explosive and destructive, often occurring in remote areas, making rescue work difficult.

[0003] Traditional forest fire detection mainly relies on manual patrol, but this method not only consumes manpower and resources, but also has the defect of small monitoring area and monitoring dead angle. Various sensors gradually replace manual detection, such as smoke sensors, temperature sensors, etc. However, sensors are easily disturbed by the environment, resulting in low sensitivity, false positives and false negatives, and the monitoring area is also small, which cannot meet the requirements of all-around monitoring of forest fires. In recent years, visible light-based detection systems have also appeared, but their imaging must rely on external visible light sources, and are easily affected by clouds and smoke, which may not be able to detect dark fires, and cannot achieve all-weather monitoring. SUMMARY

[0004] The main purpose of the present application is to overcome the above-mentioned defects in the prior art, and to provide a mountain fire identification and burned area estimation method based on multi-source heterogeneous data fusion. The satellite thermal infrared remote sensing data is used to detect forest fires, which is sensitive to high-temperature target identification. By using the gray value difference between the forest fire area and the background area in the remote sensing image and fusing the global forest cover map, the fire point can be effectively distinguished, which does not rely on visible light, has high time resolution and wide image coverage, and can realize all-weather and large-scale forest fire monitoring. The satellite revisit period is beneficial to the dynamic monitoring and process analysis of forest resources.

[0005] The present application adopts the following technical solutions:

[0006] The mountain fire identification and burned area estimation method based on multi-source heterogeneous data fusion comprises:

[0007] Satellite remote sensing images containing hot spot targets are obtained and preprocessed to obtain preprocessed satellite remote sensing images;

[0008] The preprocessed satellite remote sensing images are made into a hot spot target data set, and Faster R-CNN is used to detect hot spot targets in the satellite remote sensing images to obtain images containing hot spot prediction boxes and corresponding prediction box coordinates;

[0009] The prediction box coordinates are mapped back to the original satellite remote sensing image to obtain geographical coordinates of the prediction box in the remote sensing image, and after coordinate system conversion, coordinates of the prediction box in the global forest cover map are obtained, and the number of pixel points with a value of 1 in the prediction box is counted, and the proportion of the number of pixel points with a value of 1 in the total pixel points is calculated, and when the proportion is less than a set threshold value, the prediction box is deleted, and non-fire area filtering is realized.

[0010] The NDVI threshold method and the NBR threshold method are respectively used to estimate the burned area.

[0011] Specifically, the Faster R-CNN is used to detect hot spot targets in the satellite remote sensing image to obtain an image containing hot spot prediction boxes and corresponding prediction box coordinates, specifically including:

[0012] First, the preprocessed remote sensing image is input into the ResNet 50 network through a bottom-up path to generate four feature maps of different scales;

[0013] The four feature maps of different scales are used as inputs of the FPN network, and after up-sampling and horizontal connection operations, corresponding processed feature maps are obtained.

[0014] The processed feature maps are input into the RPN network, the RPN network is used to extract candidate boxes and output to the ROIPooling layer, and at the same time, the processed feature maps are also input into the ROI Pooling layer to extract corresponding feature maps for each candidate box; finally, the feature maps are processed through a series of fully connected layers to obtain an image containing hot spot prediction boxes and corresponding prediction box coordinates.

[0015] Specifically, the prediction box coordinates are mapped back to the satellite remote sensing image to obtain geographical coordinates of the prediction box in the remote sensing image, and after coordinate system conversion, coordinates of the prediction box in the global forest cover map are obtained, specifically including:

[0016] The prediction box coordinates are mapped back to the satellite remote sensing image, and then the GDAL library is used to obtain geographical coordinates of the prediction box in the satellite remote sensing image, and conversion between geographical coordinates and row and column numbers is performed.

[0017] The relationship between the row and column coordinates of the image and the geographical coordinates can be represented by the following mathematical relationship.

[0018] XG=GT(0)+X*GT(1)+Y*GT(2)

[0019] YG=GT(3+X*GT(4+Y*GT(5)

[0020] Wherein, XG, YG are the geographical coordinates of the image; X, Y are the row and column coordinates of the image; GT(0), GT(3) are the longitude and latitude of the upper left corner of the image; GT(1), GT(5) are the horizontal and vertical resolution of the image; GT(2), GT(4) are the rotation coefficients of the image.

[0021] Specifically, the non-mountain fire area filtering also includes:

[0022] The global forest cover map is cut, after the image cutting, the forest cover sub-map containing the original remote sensing image is selected, the row and column coordinates of the prediction box in the forest cover sub-map are obtained, and the non-mountain fire target filtering is used.

[0023] Specifically, the non-mountain fire area filtering specifically includes:

[0024] First, the pixel points of the forest cover sub-map are traversed, when the area where the prediction box is located is traversed, the number of pixel points with a value of 1 in the prediction box is counted, and the total number of pixel points in the prediction box is divided to obtain the probability that the prediction box contains forest; if the probability is greater than a set threshold, it is considered that the prediction box contains forest, otherwise it does not contain forest;

[0025] By traversing all the prediction boxes, the non-mountain fire area is filtered out;

[0026] After the filtering operation is completed, the non-mountain fire prediction box is removed, and the mountain fire area prediction box is saved.

[0027] Specifically, the NDVI threshold method is used to estimate the burned area, specifically:

[0028] NDVI threshold method:

[0029]

[0030] Wherein, NIR is the near-infrared band, and RED is the red band; NDVI is calculated for two images before and after the occurrence of mountain fire, and the NDVI threshold is set to 0.3, and the area is divided into vegetation area and non-vegetation area; when the NDVI before the occurrence of mountain fire is greater than or equal to 0.3, and the NDVI after the occurrence of mountain fire is less than 0.3, it is considered that the pixel is a burned pixel; the total number of burned pixels is counted, and the burned area is obtained by multiplying the pixel area.

[0031] Specifically, the NBR threshold method is used to estimate the burned area, specifically:

[0032]

[0033] Wherein, NIR is the near-infrared band, and SWIR is the short-wave infrared band; NBR is calculated for two images before and after the occurrence of mountain fire, and the burned area is determined by the following formula according to the obtained NBR;

[0034] NBR before -NBR after >T NBR

[0035] wherein, NBR before and NBR after respectively represent NBR values before and after the occurrence of the mountain fire, T NDVI is a discrimination threshold value; the threshold value T NBR is set to 0.4, and a pixel greater than 0.4 is considered to be a burned pixel, and finally the number of burned pixels is counted to obtain the burned area.

[0036] Another aspect of the embodiment of the present application provides a mountain fire identification and burned area estimation system based on multi-source heterogeneous data fusion, comprising:

[0037] A preprocessing unit: acquires a satellite remote sensing image containing a hot spot target, and performs preprocessing to obtain a preprocessed satellite remote sensing image;

[0038] A target detection unit: processes the preprocessed satellite remote sensing image as a hot spot target data set, and uses FasterR-CNN to detect hot spot targets in the satellite remote sensing image to obtain an image containing a hot spot prediction box and corresponding prediction box coordinates;

[0039] A non-target filtering unit: maps the prediction box coordinates back to the original satellite remote sensing image to obtain the geographic coordinates of the prediction box in the remote sensing image, and after coordinate system conversion, obtains the coordinates of the prediction box in the global forest cover map, counts the proportion of the number of pixel points with a value of 1 in the prediction box to the total pixel points, and when the proportion is less than a set threshold value, the prediction box is deleted to realize non-mountain fire area filtering;

[0040] An area estimation unit: respectively uses an NDVI threshold method and an NBR threshold method to estimate the burned area.

[0041] Still another aspect of the embodiment of the present application is an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned mountain fire identification and burned area estimation method based on multi-source heterogeneous data fusion when executing the computer program.

[0042] Still another aspect of the embodiment of the present application is a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the above-mentioned mountain fire identification and burned area estimation method based on multi-source heterogeneous data fusion.

[0043] As can be seen from the above description of the present application, compared with the prior art, the present application has the following beneficial effects:

[0044] (1) A forest fire identification and burned area estimation method based on multi-source heterogeneous data fusion, comprising: acquiring a satellite remote sensing image containing a hot spot target, and pre-processing to obtain a pre-processed satellite remote sensing image; the pre-processed satellite remote sensing image is made into a hot spot target data set, and Faster R-CNN is used to detect the hot spot target of the satellite remote sensing image to obtain an image containing a hot spot prediction box and the corresponding prediction box coordinates; the prediction box coordinates are mapped back to the original satellite remote sensing image to obtain the geographic coordinates of the prediction box in the remote sensing image, and after coordinate system conversion, the coordinates of the prediction box in the global forest cover map are obtained, the number of pixel points with a value of 1 in the prediction box is counted, and the proportion of the total pixel points is less than the set threshold value, the prediction box is deleted, and the non-forest fire area is filtered out; NDVI threshold method and NBR threshold method are used to estimate the burned area; satellite thermal infrared remote sensing data is used to detect forest fires, which is sensitive to high-temperature target identification, and the gray value difference between the forest fire area and the background area in the remote sensing image is combined with the global forest cover map to effectively identify the fire point, which does not depend on visible light, has high time resolution and wide image coverage, can realize all-weather large-scale forest fire monitoring, and the satellite revisit period is beneficial to dynamic monitoring and process analysis of forest resources.

[0045] (2) In the target detection, Faster R-CNN is used for hot spot detection, which can better solve the problems of multi-scale and small target, and ResNet 50+FPN is used as a feature extraction network, ResNet uses a residual network structure, increases an identity mapping, and overcomes problems such as gradient disappearance and gradient explosion caused by deepening of the network; in addition, FPN realizes multi-scale information fusion, fuses low-level detail information and high-level semantic information, so that the low layer can obtain more context information when detecting small targets, and effectively solves the problem that the forest fire target occupies a small proportion and is dense. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 A forest fire identification and burned area estimation method based on multi-source heterogeneous data fusion is provided for the embodiment of the present application;

[0047] Figure 2 A Faster R-CNN network structure of Resnet50+FPN is provided for the embodiment of the present application;

[0048] Figure 3 A part of the hot spot target detection result graph is provided for the embodiment of the present application;

[0049] Figure 4The operation flow diagram for filtering non-mountain fire targets from the thermal infrared remote sensing image containing non-mountain fire regions provided by the embodiment of the present application, wherein figure (a) is a thermal target detection result image, figure (b) is an enlarged image of the target detection result, figure (c) is a natural image of the region where the prediction frame is located, and figure (d) is a result image after filtering the non-mountain fire regions;

[0050] Figure 5 The NDVI images before and after the occurrence of the mountain fire provided by the embodiment of the present application, wherein figure (a) is an NDVI image before the occurrence of the mountain fire, and figure (b) is an NDVI image after the occurrence of the mountain fire;

[0051] Figure 6 The NBR images before and after the occurrence of the mountain fire provided by the embodiment of the present application, wherein figure (a) is an NBR image before the occurrence of the mountain fire, and figure (b) is an NBR image after the occurrence of the mountain fire;

[0052] Figure 7 The system structure diagram of the mountain fire recognition and burned area estimation based on multi-source heterogeneous data fusion provided by the embodiment of the present application;

[0053] Figure 8 The schematic diagram of an electronic device provided by the embodiment of the present application;

[0054] Figure 9 The schematic diagram of an embodiment of a computer readable storage medium provided by the embodiment of the present application.

[0055] The present application will be further described in detail below in combination with the drawings and specific embodiments. DETAILED DESCRIPTION

[0056] The present application provides a mountain fire recognition and burned area estimation method based on multi-source heterogeneous data fusion, which detects the possible mountain fire points by using satellite thermal infrared remote sensing data. The thermal infrared remote sensing is sensitive to the identification of high-temperature targets, and high temperature is the most important feature of mountain fire. The fire points are effectively distinguished by the gray value difference between the mountain fire region and the background region in the remote sensing image and in combination with the global forest coverage map. The method does not rely on visible light, has high time resolution and wide image coverage, can realize all-weather and large-scale mountain fire monitoring, and the satellite revisit period is beneficial to the dynamic monitoring and process analysis of forest resources.

[0057] In the present application, the thermal target refers to, for example, the high-temperature region caused by mountain fire, the high-temperature region of desert, the high-temperature region caused by human activities, etc. The high-temperature region caused by human activities exists more in cities, such as the thermal emission of mining enterprises, fire, nuclear power plants, etc. The mountain fire target refers to a kind of fire situation that occurs in the forest and is difficult to control, which is caused by lightning or human, that is, the high-temperature thermal point region excluding the forest coverage region.

[0058] The fire identification and fire area estimation method based on multi-source heterogeneous data fusion comprises:

[0059] S1: hot spot remote sensing image data collection and data preprocessing

[0060] Satellite remote sensing images containing hot spots are obtained and preprocessed to obtain preprocessed satellite remote sensing images.

[0061] Further, step S1 specifically comprises:

[0062] S11, collect satellite thermal infrared remote sensing images containing hot spots, and perform data preprocessing on each thermal infrared remote sensing image.

[0063] S12, perform adaptive histogram equalization on the remote sensing image to uniformize the image gray scale distribution, adjust the contrast, improve the contrast between the hot spots and the background area, and highlight the hot spot brightness.

[0064] S13, the image size of the remote sensing image is too large, considering the inconvenience of labeling and the limitation of GPU memory, we cut a remote sensing image into small images of 512x512 size, select images containing hot spots, perform data labeling, and mark the hot spots in each image.

[0065] S14, perform data enhancement processing on the image, such as data rotation, data flipping, etc., to achieve the purpose of expanding the data set.

[0066] S15, make the data preprocessed image into a VOC 2012 standard format data set.

[0067] S2: training and detecting hot spot targets using Faster R-CNN model

[0068] Further, step S2 specifically comprises:

[0069] S21, build a Faster R-CNN network model with ResNet50+FPN as the backbone, and train on the training set until convergence.

[0070] S22, input the image that wants to perform hot spot target recognition into the trained model, and the hot spot target detection result can be obtained. Figure 2 Resnet50+FPN Faster R-CNN network structure.

[0071] Specifically:

[0072] Faster R-CNN is used to identify hot spot targets in satellite remote sensing images to obtain images containing hot spot prediction boxes and corresponding prediction box coordinates; specifically including:

[0073] First, the pre-processed remote sensing image is input into the ResNet 50 network through a bottom-up path to generate four feature maps of different scales;

[0074] The four feature maps of different scales are taken as inputs of the FPN network, and after upsampling and horizontal connection operations, corresponding processed feature maps are obtained;

[0075] The processed feature maps are input into the RPN network, which is used to extract candidate boxes and output to the ROIPooling layer. At the same time, the processed feature maps are also input into the ROI Pooling layer to extract the corresponding feature maps for each candidate box. Finally, the feature maps are output through a series of fully connected layers, and the result and the hotspot target box coordinates are output.

[0076] As Figure 3 Part of the hotspot target detection result map.

[0077] S3: Filtering non-wildfire areas using global forest cover map

[0078] Since the hotspot targets include high-temperature areas such as deserts, high-temperature areas caused by human activities, etc., the high-temperature caused by human activities is more present in cities, such as mining enterprises, fires, heat emissions of nuclear power plants, etc., which also have high temperatures and high gray values, and have obvious contrast with the background area, showing similar characteristics to wildfires in remote sensing images, and may be mistaken for wildfires, so it is necessary to filter out this type of target. In the embodiment of the present application, GIS platform and multi-source data fusion technology are used to filter out non-wildfire areas.

[0079] Further, step S3 specifically comprises:

[0080] S31, coordinate mapping. After hotspot target detection, an image containing hotspot prediction boxes and corresponding prediction box coordinates are obtained, and the prediction box coordinates are mapped back to the original hotspot remote sensing image.

[0081] S32, coordinate conversion. The row and column coordinates of the prediction boxes in the remote sensing image are converted into geographic coordinates through the GDAL library. At the same time, the coordinate system of the hotspot remote sensing image is converted into the coordinate system of the forest cover map.

[0082] S33, screening forest cover sub-map. The global forest cover map is cut into 10000x10000 size images, and the latitude and longitude information of the four corners of each small block is calculated. After cutting, all forest cover sub-maps are traversed in a loop, and the coordinates of their left, right, top and bottom four corners are compared with the coordinates of the four corners of the hotspot remote sensing image. The forest cover sub-map containing the remote sensing image is selected. At the same time, the coordinate conversion is performed to obtain the row and column coordinates of the hotspot prediction boxes in the forest cover sub-map.

[0083] S34, non-fire area filtering. By traversing the forest coverage sub-pixel points, the probability of containing forest in the hotspot prediction box is counted, and by traversing all the prediction boxes, non-fire areas with a forest probability lower than 0.1 are filtered out.

[0084] S35, the center point longitude and latitude of the prediction box are calculated by the longitude and latitude coordinates of the four corners of the prediction box, and the country, city, road and other information of the wildfire prediction box area are obtained by using Nominatim API.

[0085] Specifically:

[0086] The prediction box coordinates are mapped back to the original satellite remote sensing image to obtain the geographic coordinates of the prediction box in the remote sensing image. After coordinate system conversion, the coordinates of the prediction box in the global forest coverage map are obtained. The number of pixel points with a value of 1 in the prediction box accounts for the proportion of the total pixel points. When it is less than the set threshold, the prediction box is deleted, and non-fire area filtering is realized.

[0087] Through the hotspot target detection model, the image containing the hotspot prediction box and the corresponding prediction box coordinates are obtained. Since these images are small images cut from the hotspot remote sensing image, it is necessary to map the prediction box coordinates back to the original hotspot remote sensing image. Then, we combine the global forest coverage map to filter out non-fire targets. The geographic coordinates of the remote sensing image are obtained through the GDAL library, and the conversion between geographic coordinates and row and column numbers is performed.

[0088] The relationship between the row and column coordinates of the image and the geographic coordinates can be represented by the following mathematical relationship.

[0089] XG=GT(0)+X*GT(1)+Y*GT(2)(5)

[0090] YG=GT(3)+X*GT(4)+Y*GT(5)(6)

[0091] Where XG, YG are the geographic coordinates of the image; X, Y are the row and column coordinates of the image; GT(0), GT(3) are the longitude and latitude of the upper left corner of the image; GT(1), GT(5) are the horizontal and vertical resolution of the image; GT(2), GT(4) are the rotation coefficients of the image.

[0092] At the same time, since the hotspot remote sensing image and the global forest cover map belong to different coordinate systems, coordinate system conversion is also needed using the GDAL library. Due to the large size of the global forest cover map, it may cause long time consumption in filtering out non-fire targets. Therefore, the global forest cover map needs to be cut. After image cutting, the forest cover sub-map containing the wildfire remote sensing image is selected. Next, the prediction box coordinates are subjected to a series of coordinate conversion operations to obtain the row and column coordinates of the prediction box in the forest cover sub-map, which is used for filtering out non-fire targets.

[0093] Finally, non-fire target filtering is performed. First, the pixel points of the forest cover sub-map are traversed. When the area where the prediction box is located is traversed, the number of pixel points with a value of 1 (i.e., forest) in the prediction box is counted, and divided by the total number of pixel points in the prediction box to obtain the probability of containing forest in the prediction box. After many experiments, we set the threshold value to 0.1. As long as the probability is greater than 0.1, it is considered that there is forest in the prediction box, otherwise there is no forest. Thus, by traversing all the prediction boxes, non-fire areas can be filtered out. After filtering, non-fire prediction boxes are removed, and wildfire area prediction boxes are saved. For example Figure 4 For filtering out non-fire targets in the thermal infrared remote sensing image containing non-fire areas, the operation flowchart is shown in FIG. 4. FIG. 4(a) is a thermal hotspot target detection result image, FIG. 4(b) is an enlarged image of the target detection result, FIG. 4(c) is a natural image of the area where the prediction box is located, and FIG. 4(d) is a result image after filtering out non-fire areas.

[0094] S4: respectively using NDVI threshold method and NBR threshold method to estimate the burned area.

[0095] Specifically, the NDVI threshold method is used to estimate the burned area. The normalized difference vegetation index (NDVI) is a remote sensing index reflecting the vegetation condition of land cover, which quantifies vegetation by using the difference between the pigment absorption rate of chlorophyll in the red band and the high reflectivity of plant bodies in the near-infrared band. The vegetation cover before and after the wildfire will change, so the burned area can be estimated according to the difference between the NDVI before and after the wildfire. Specifically,

[0096] NDVI threshold method:

[0097]

[0098] wherein, NIR is the near-infrared band, and RED is the red band; NDVI is calculated for the two images before and after the wildfire, respectively, and the NDVI threshold is set to 0.3, dividing the area into vegetation area and non-vegetation area; when the NDVI before the wildfire is greater than or equal to 0.3, and the NDVI after the wildfire is less than 0.3, it is considered that the pixel is a burned pixel; the total number of burned pixels is counted, and multiplied by the pixel area to obtain the burned area.

[0099] As Figure 5 , the NDVI image before and after the occurrence of the mountain fire, wherein figure (a) is the NDVI image before the occurrence of the mountain fire, and figure (b) is the NDVI image after the occurrence of the mountain fire;

[0100] Specifically, the NBR threshold method is used to estimate the burned area, and the normalized burn ratio (NBR) uses the change of the reflectivity of the NIR and SWIR bands of the remote sensing image before and after the occurrence of the mountain fire to highlight the burned area, and specifically, the normalized burn ratio (NBR) is calculated according to the following formula:

[0101]

[0102] Wherein, NIR is the near-infrared band, and SWIR is the short-wave infrared band; the NBR is calculated for the two images before and after the occurrence of the mountain fire, and the burned area is determined according to the obtained NBR through the following formula:

[0103] NBR before -NBR after >T NBR

[0104] Wherein, NBR before and NBR after respectively represent the NBR values before and after the occurrence of the mountain fire, and T NDVI is a discrimination threshold; the T NBR threshold is set to 0.4, and the pixel is considered as a burned pixel if it is greater than 0.4, and finally the number of burned pixels is counted to obtain the burned area.

[0105] As Figure 6 , the NBR image before and after the occurrence of the mountain fire, wherein figure (a) is the NBR image before the occurrence of the mountain fire, and figure (b) is the NBR image after the occurrence of the mountain fire;

[0106] As Figure 7 , the mountain fire recognition and burned area estimation system based on multi-source heterogeneous data fusion is provided in another aspect of the embodiment, comprising:

[0107] The mountain fire recognition and burned area estimation system based on multi-source heterogeneous data fusion is provided in another aspect of the embodiment, comprising:

[0108] The preprocessing unit: acquires the satellite remote sensing image containing the hot spot target, and performs preprocessing to obtain the preprocessed satellite remote sensing image;

[0109] The target detection unit: the preprocessed satellite remote sensing image is made into a hot spot target data set, and the FasterR-CNN is used to detect the hot spot target of the satellite remote sensing image to obtain the image containing the hot spot prediction frame and the corresponding prediction frame coordinates;

[0110] Non-target filtering unit: the prediction box coordinates are mapped back to the original satellite remote sensing image to obtain the geographical coordinates of the prediction box in the remote sensing image, and after coordinate system conversion, the coordinates of the prediction box in the global forest coverage map are obtained, the number of pixel points with a value of 1 in the prediction box is counted, and the proportion of the number of pixel points with a value of 1 in the total pixel points is calculated, and when the proportion is less than a set threshold, the prediction box is deleted, and non-fire area filtering is realized.

[0111] Area estimation unit: NDVI threshold method and NBR threshold method are used to estimate the area of the fire.

[0112] Figure 8 As shown in the figure, the embodiment of the application provides an electronic device 800, which comprises a memory 810, a processor 820, and a computer program 811 stored in the memory 820 and capable of running on the processor 820, and the processor 820 implements the method of cross-user behavior recognition migration learning provided by the embodiment of the application when executing the computer program 811.

[0113] Since the electronic device introduced in the embodiment is the device used in the embodiment of the application, based on the method introduced in the embodiment of the application, those skilled in the art can understand the specific implementation of the electronic device of the embodiment and its various forms, so the electronic device how to implement the method in the embodiment of the application will not be described in detail, as long as the device used by those skilled in the art to implement the method in the embodiment of the application belongs to the scope of the application.

[0114] Please refer to Figure 9 , Figure 9 An embodiment of a computer readable storage medium provided by the embodiment of the application is shown in the figure.

[0115] As Figure 9 shown, the embodiment provides a computer readable storage medium 900, which stores a computer program 911, and the computer program 911 is executed by a processor to implement the method of cross-user behavior recognition migration learning provided by the embodiment of the application.

[0116] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0117] Those skilled in the art will appreciate that embodiments of the application can be provided as methods, systems, or computer program products. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0118] The mountain fire recognition and overfire area estimation method based on multi-source heterogeneous data fusion comprises the following steps: acquiring a satellite remote sensing image containing a hot spot target, and pre-processing to obtain a pre-processed satellite remote sensing image; the pre-processed satellite remote sensing image is made into a hot spot target data set, Faster R-CNN is used to detect the hot spot target of the satellite remote sensing image, and an image containing a hot spot prediction box and the corresponding prediction box coordinates are obtained; the prediction box coordinates are mapped back to the original satellite remote sensing image to obtain the geographic coordinates of the prediction box in the remote sensing image, and after coordinate system conversion, the coordinates of the prediction box in the global forest cover map are obtained, the number of pixel points with a value of 1 in the prediction box is counted, and the proportion of the total pixel points is less than the set threshold value, the prediction box is deleted, and the non-mountain fire area is filtered out; the NDVI threshold method and the NBR threshold method are used to estimate the overfire area; satellite thermal infrared remote sensing data is used to detect the mountain fire, the thermal infrared band is sensitive to high-temperature target identification, the gray value difference between the mountain fire area and the background area in the remote sensing image is used, and the global forest cover map is fused to effectively distinguish the fire point, the method does not depend on visible light, has high time resolution and wide image coverage, can realize all-weather and large-scale mountain fire monitoring, and the satellite revisit period is beneficial to dynamic monitoring and process analysis of forest resources.

[0119] In the target detection, Faster R-CNN is used for hot spot detection, which can better solve the problems of multi-scale and small target, and ResNet 50+FPN is used as a feature extraction network, the residual network structure is adopted in ResNet, the identity mapping is increased, and the problems of gradient disappearance and gradient explosion caused by deepening of the network are overcome; in addition, FPN realizes multi-scale information fusion, and the low-layer detail information and the high-layer semantic information are fused, so that the low layer can obtain more context information when detecting small targets, and the problem that the mountain fire target occupies a small proportion and is dense is effectively solved.

[0120] It is to be noted that, in the present text, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another, without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element. The above specification is merely a specific implementation of the present application, enabling one of ordinary skill in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0121] The above is only a specific embodiment of the present application, but the design concept of the present application is not limited thereto, and any non-essential modification of the present application using the concept should be regarded as an infringement of the scope of protection of the present application.

Claims

1. A method for forest fire identification and burned area estimation based on multi-source heterogeneous data fusion, characterized in that, The method comprises the following steps: Satellite remote sensing images containing hotspot targets are acquired and preprocessed to obtain preprocessed satellite remote sensing images; The preprocessed satellite remote sensing images are made into a hotspot target data set, and Faster R-CNN is used to detect the hotspot targets in the satellite remote sensing images to obtain images containing hotspot prediction boxes and corresponding prediction box coordinates; The prediction box coordinates are mapped back to the original satellite remote sensing images to obtain the geographic coordinates of the prediction boxes in the remote sensing images, and after coordinate system conversion, the coordinates of the prediction boxes in the global forest cover map are obtained, and the proportion of the number of pixel points with a value of 1 in the prediction boxes in the total pixel points is counted; when the proportion is less than a set threshold, the prediction box is deleted, and non-fire area filtering is realized; The over-burned area is estimated by using the NDVI threshold method and the NBR threshold method respectively; Faster R-CNN is used to detect the hotspot targets in the satellite remote sensing images to obtain images containing hotspot prediction boxes and corresponding prediction box coordinates, which specifically comprises: First, the preprocessed remote sensing images are input into the ResNet 50 network from bottom to top to generate four feature maps of different scales; The four feature maps of different scales are input into the FPN network, and after up-sampling and horizontal connection operations, corresponding processed feature maps are obtained; The processed feature maps are input into the RPN network, which is used to extract candidate boxes and output to the ROIPooling layer. At the same time, the processed feature maps are also input into the ROIPooling layer to extract corresponding feature maps for each candidate box. Finally, the feature maps are processed through a series of fully connected layers to obtain images containing hotspot prediction boxes and corresponding prediction box coordinates.

2. The forest fire identification and burned area estimation method based on multi-source heterogeneous data fusion according to claim 1, characterized in that, The prediction box coordinates are mapped back to the satellite remote sensing images to obtain the geographic coordinates of the prediction boxes in the remote sensing images, and after coordinate system conversion, the coordinates of the prediction boxes in the global forest cover map are obtained, which specifically comprises: The prediction box coordinates are mapped back to the satellite remote sensing images, and then the geographic coordinates of the prediction boxes in the satellite remote sensing images are obtained by using the GDAL library, and the conversion between the geographic coordinates and the row and column numbers is performed; The relationship between the row and column coordinates of the image and the geographic coordinates is represented by the following mathematical relationship: XG=GT(0)+X*GT(1)+Y*GT(2) YG=GT(3)+X*GT(4)+Y*GT(5) Where XG and YG are the geographic coordinates of the image; X and Y are the row and column coordinates of the image; GT(0) and GT(3) are the longitude and latitude of the top left corner of the image; GT(1) and GT(5) are the horizontal and vertical resolutions of the image; GT(2) and GT(4) are the rotation coefficients of the image.

3. The forest fire identification and burned area estimation method based on multi-source heterogeneous data fusion according to claim 1, characterized in that, Non-fire area filtering also includes: The global forest cover map is cut, and after the image cutting is completed, the forest cover sub-map containing the original remote sensing image is selected, and the row and column coordinates of the prediction boxes in the forest cover sub-map are obtained for non-fire target filtering.

4. The forest fire identification and burned area estimation method based on multi-source heterogeneous data fusion according to claim 3, characterized in that, Non-fire area filtering specifically comprises: First, the pixel points of the forest coverage subgraph are traversed, and when the area of the prediction box is reached, the number of pixel points with a value of 1 in the prediction box is counted, and the number is divided by the total number of pixel points in the prediction box to obtain the probability that the prediction box contains forest; if the probability is greater than a set threshold, it is considered that the prediction box contains forest, otherwise it does not contain forest; By traversing all the prediction boxes, non-fire areas are filtered out; After the filtering operation is completed, the non-fire prediction box is removed, and the fire area prediction box is saved.

5. The forest fire identification and burned area estimation method based on multi-source heterogeneous data fusion according to claim 1, characterized in that, The NDVI threshold method is used to estimate the burned area, specifically as follows: NDVI threshold method: wherein NIR is the near-infrared band, and RED is the red light band; NDVI is calculated for two images before and after the occurrence of forest fires, and the NDVI threshold is set to 0.3, and the area is divided into vegetation area and non-vegetation area; when the NDVI before the occurrence of forest fires is greater than or equal to 0.3, and the NDVI after the occurrence of forest fires is less than 0.3, it is considered that the pixel is a burned pixel; the total number of burned pixels is counted, and the burned area is obtained by multiplying the pixel area.

6. The forest fire identification and burned area estimation method based on multi-source heterogeneous data fusion according to claim 1, characterized in that, The NBR threshold method is used to estimate the burned area, specifically as follows: wherein NIR is the near-infrared band, and SWIR is the short-wave infrared band; NBR is calculated for two images before and after the occurrence of forest fires, and the burned area is determined by the following formula according to the obtained NBR; NBR before -NBR after >T NBR Where NBR before and NBR after respectively represent the NBR values before and after the occurrence of the forest fire, T NBR is the discrimination threshold value; the threshold value T NBR is set to 0.4, and the pixel is considered to be a burned pixel if it is greater than 0.4, and finally the number of burned pixels is counted to obtain the burned area.

7. A forest fire identification and burned area estimation system based on multi-source heterogeneous data fusion, characterized in that, It includes: A preprocessing unit acquires a satellite remote sensing image containing a hot spot target, and performs preprocessing to obtain a preprocessed satellite remote sensing image; A target detection unit uses the Faster R-CNN to detect the hot spot target in the satellite remote sensing image, and obtains an image containing a hot spot prediction box and corresponding prediction box coordinates; A non-target filtering unit maps the prediction box coordinates back to the original satellite remote sensing image to obtain the geographical coordinates of the prediction box in the remote sensing image, and after coordinate system conversion, obtains the coordinates of the prediction box in the global forest coverage map, counts the proportion of the number of pixel points with a value of 1 in the total pixel points in the prediction box, and deletes the prediction box when the proportion is less than a set threshold, thereby filtering out non-fire areas; An area estimation unit estimates the burned area using the NDVI threshold method and the NBR threshold method; The Faster R-CNN is used to detect the hot spot target in the satellite remote sensing image, and the image containing the hot spot prediction box and the corresponding prediction box coordinates are obtained, specifically including: First, the preprocessed remote sensing image is input into the ResNet 50 network through a bottom-up path to generate four feature maps of different scales; The four feature maps of different scales are input into the FPN network, and processed feature maps are obtained through upsampling and horizontal connection operations, respectively. The processed feature map is input into an RPN network, the RPN network is used for extracting a candidate frame and is output to an ROIPooling layer, meanwhile, the processed feature map is also input into the ROIPooling layer, a corresponding feature map is extracted for each candidate frame; finally, the feature map is subjected to a series of full connection layers, an image containing a hotspot prediction frame and corresponding prediction frame coordinates are obtained.

8. An electronic device, comprising: The application relates to a computer program product, comprising: A memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method steps of any one of claims 1-6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium and is executable on the processor to implement the method steps of any one of claims 1-6.

Images