An early forest fire detection method based on UAV aerial images

Abstract

The present invention discloses an early forest fire detection method based on unmanned aerial vehicle (UAV) aerial images. First, a bilateral filtering algorithm and a dark channel prior defogging algorithm are used to improve the quality of forest fire images. Then, to address the problem that the fire area in early forest fire images taken by UAV aerial vehicles is too small, a two-stage network Faster-RCNN with higher accuracy is selected as a basic detection network. Then, by analyzing the area size distribution of early forest fire targets, a K-means algorithm is used to optimize the anchor box of Faster-RCNN according to the characteristics that there are small forest fire targets and forest fire targets with large aspect ratio differences. Secondly, to address the problem that the pixels of early forest fire targets are too small, a feature fusion method is used to highlight the features of early forest fire images, thereby improving detection accuracy. Finally, to address the problem that the background of early forest fires in UAV aerial images is complex, an attention mechanism is introduced to reduce the interference of the complex background of the image on the early forest fire target recognition model, thereby improving the learning efficiency and detection accuracy of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of forest fire detection, and in particular relates to an early forest fire detection method based on unmanned aerial vehicle (UAV) aerial images. Background Art

[0002] With the development of society, the frequency of forest fires is increasing, causing serious impacts on the environment and human life. Therefore, early detection of forest fires and the implementation of effective measures to prevent their occurrence and spread have become important issues in today's society. Drone aerial photography allows for comprehensive, all-weather, and large-scale monitoring of forest areas. The ability to capture and transmit real-time images of forest areas allows for early warnings in the event of a fire, significantly impacting the timely detection and suppression of early-stage forest fires. With the development of drone aerial photography technology, forest fire detection technology based on drone aerial imagery has become a major research hotspot.

[0003] Currently, there are two main approaches to forest fire detection based on computer vision. One is based on machine learning processing technology. Most research in this field relies on manually extracted features such as color, shape, and texture to detect fires, but this results in low detection accuracy and lacks real-time performance. The other is based on deep learning technology. This method utilizes a deep network structure to gradually extract features from forest fire images, effectively detecting large burning areas. Because early-stage forest fire targets in drone aerial photography are small in size, have large variations in aspect ratio, and are often found against complex backgrounds, existing methods suffer from low detection accuracy and poor real-time performance. Therefore, this method proposes a method for early-stage forest fire detection based on drone aerial imagery. This method, based on the Faster-RCNN algorithm, optimizes anchor boxes, fuses image features from different layers of early-stage fires, and introduces an attention mechanism to reduce interference from complex backgrounds. This method effectively addresses the issues of low accuracy and poor real-time performance in early-stage forest fire detection in drone aerial images. Compared to similar algorithms, it offers higher accuracy and meets real-time detection requirements, making it suitable for forest fire prevention monitoring. Summary of the Invention

[0004] In view of this, the main purpose of the present invention is to provide an early forest fire detection method based on drone aerial images.

[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] A method for early forest fire detection based on drone aerial images includes the following steps:

[0007] Step 1, preprocessing: Collect and create a dataset of early forest fire images from drone aerial photography scenes. Use the bilateral filtering algorithm and the dark channel prior dehazing algorithm to preprocess the aerial forest fire image dataset to obtain the preprocessed images.

[0008] Step 2: Select a detection model and optimize the anchor box. To address the problem of early forest fire targets being too small in drone aerial photography, we selected the more accurate two-stage Faster-RCNN network as the base detection network. We also used the K-means clustering algorithm to optimize the Faster-RCNN anchor box to better fit the early forest fire targets in the drone aerial images.

[0009] Step 3: Optimize the accuracy of the aerial forest fire detection model: Although Faster-RCNN can effectively improve the detection accuracy of early forest fires in aerial images, the early forest fire targets in the aerial images studied in this paper have too few available pixels and vary in shape. Therefore, the PPM structure is introduced to improve the Faster-RCNN network. By integrating contextual features, the model receptive field is increased to further improve the detection accuracy of early forest fire images in drone aerial photography scenarios.

[0010] Step 4: Optimize the speed of the aerial forest fire detection model: To address the complex background issues in drone-photographed forest fire images, we introduced the attention mechanism SENet to improve the original Flatten layer and FC1 and FC2 fully connected layers of Faster-RCNN, thereby improving the model's learning efficiency and detection accuracy for early forest fire detection in drone-photographed scenes.

[0011] Step 5: Forest fire detection network training: To address the problem of early forest fire detection in drone aerial photography scenarios, complete the above improvements and build a network training improved Faster-RCNN model. Use the improved model to detect early forest fires in drone aerial images.

[0012] Compared with the prior art, the present invention has the following beneficial effects:

[0013] The present invention addresses the problem of low accuracy of early forest fire detection models in drone aerial photography scenarios. It can effectively eliminate the noise in the natural forest environment, solve the problem of low accuracy of existing models in detecting early forest fire images in drone aerial photography scenarios, and meet real-time detection needs, with good application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flowchart of an example of the present invention.

[0015] Figure 2 This is a diagram of the improved feature extraction network structure in an example of the present invention.

[0016] Figure 3 Schematic diagram of the overall structure of the aerial forest fire detection model based on Faster-RCNN improvement in the example of the present invention DETAILED DESCRIPTION

[0017] In order to better understand the technical solution of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings. It should be clear that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] The embodiment of the present invention provides a method for early forest fire detection based on drone aerial images, such as Figure 1 As shown, the main steps are:

[0019] Step 1: Collect and create a dataset of early forest fire images from drone aerial photography. First, remove noise from the aerial forest fire images using a bilateral filtering algorithm. Then, use a dark channel prior dehazing algorithm to dehaze the aerial forest fire images.

[0020] For example, in this step, bilateral filtering algorithm and dark channel prior dehazing algorithm are used to enhance image quality. The specific steps are as follows:

[0021] In step 1-1, the forest fire image is preprocessed using the bilateral filtering algorithm. The pixel values ​​of the output image are obtained by performing local weighted averaging on the input forest fire image, as shown in the following formula:

[0022]

[0023] in,

[0024]

[0025]

[0026] In the above formula, D is the coordinate set of the pixel points in the neighborhood of the center of the forest fire image, F(x,y) is the pixel value of the output image, f(i,j) is the pixel value of the current pixel point (i,j) in the image, ω s (i,j) and ω r (i, j) are the spatial domain weight coefficient and grayscale domain weight coefficient of the image at (i, j), f(x, y) is the pixel value of the center point (x, y), σ s and σ r are the standard deviation in the spatial domain and the standard deviation in the grayscale domain respectively;

[0027] Considering that UAVs collect forest fire images from high altitudes, the interaction between suspended particles in the air and light causes significant degradation in forest fire images captured by UAVs, which manifests as blurred and foggy images. This method uses a dark channel prior dehazing algorithm to enhance UAV aerial forest fire data.

[0028] In steps 1-2, the dark channel prior dehazing algorithm is used to extract the dark channel features of the aerial forest fire image at a fixed scale, as shown in the following formula:

[0029]

[0030] in, is the dark channel feature with Ω as the scale, I ch (x, y) are the R, G, and B channels of the blurred foggy aerial forest fire image;

[0031] In steps 1-3, the atmospheric light value and the transmission rate are calculated through the dark channel characteristics to obtain the enhanced aerial forest fire image.

[0032] Step 2: To address the issue of early-stage forest fires being too small in drone aerial photography, we selected the more accurate two-stage Faster-RCNN as the base detection network. We also used the K-means clustering algorithm to optimize Faster-RCNN's anchor boxes to better fit the early-stage forest fire targets in the drone aerial images.

[0033] Specifically, the specific steps of introducing K-means to optimize the anchor box of Faster-RCNN are as follows:

[0034] Step 2-1: Statistical analysis of early forest fire datasets captured by drones;

[0035] By analyzing a dataset of early-stage forest fires captured by drones, we found that the aspect ratios of early-stage forest fire targets in drone-photographed scenes are mostly concentrated around 1.2, with the largest aspect ratio reaching 4.77. Some fire targets also have aspect ratios between 3 and 4.7. Analysis shows that the dataset contains targets with extremely small pixels and fire targets with widely varying aspect ratios. The scale of the original Faster-RCNN anchor box does not well match the scale of the early-stage forest fire targets captured by drones. Therefore, this method uses the K-means clustering algorithm to cluster the sizes of the annotation boxes in this dataset to obtain the best-fitting anchor box.

[0036] In step 2-2, in the original K-means algorithm, the Euclidean distance formula is used to measure the distance between the target position and the cluster center point, which is not suitable for measuring the degree of fit between the anchor box and the forest fire target annotation box. Therefore, this method uses IOU to measure the distance between objects. IOU The calculation formula is shown below:

[0037] d IOU =1-IOU

[0038] in,

[0039]

[0040] Sj represents the intersection area of ​​the anchor box and the label box, and Sb represents the union area of ​​the anchor box and the label box. The specific steps of the K-means clustering algorithm are as follows:

[0041] Step 2-2-1, select the annotation box X = {X1, X2, X3, ..., Xn} of the forest fire target in n data sets;

[0042] Step 2-2-2, randomly select the labeled boxes of 9 forest fire targets as cluster centers {C1, C2, C3, …, C9};

[0043] Step 2-2-3, calculate the IOU distance between each forest fire target annotation box and the nine cluster center points respectively;

[0044] Step 2-2-4: Divide each forest fire target annotation box into the cluster center point with the smallest IOU distance;

[0045] Step 2-2-5, calculate the average value of the coordinates of each forest fire target labeling box in the 9 categories and use it as the new cluster center point;

[0046] Step 2-2-6: Determine whether the specified number of cycles has been reached or the cluster center point is no longer changing, otherwise return to step 2-2-3.

[0047] Step 3: Although Faster-RCNN can effectively improve the detection accuracy of early forest fires in aerial images, the early forest fire targets in the aerial images studied in this paper have too few available pixels and vary in shape. Therefore, the PPM structure is introduced to improve the Faster-RCNN network. By integrating contextual features, the model receptive field is increased to further improve the detection accuracy of early forest fires in UAV aerial photography scenarios;

[0048] Specifically, the detailed steps of using the feature fusion structure PPM to improve the Faster-RCNN network to improve the accuracy of early forest fire target detection in UAV aerial photography scenarios are as follows:

[0049] Take the P2 feature map as an example;

[0050] Step 3-1, perform maximum pooling downsampling on the P2 feature map using pooling kernels of size pairs (6×6), (3×3), (2×2), and (1×1), and output feature maps of sizes (26×26×256), (53×53×256), (79×79×256), and (160×160×256), respectively;

[0051] Step 3-2, perform maximum pooling downsampling on the P2 feature map using pooling kernels of size pairs (6×6), (3×3), (2×2), and (1×1), and output feature maps of sizes (26×26×256), (53×53×256), (79×79×256), and (160×160×256), respectively;

[0052] Step 3-2, perform maximum pooling downsampling on the P2 feature map using pooling kernels of size pairs (6×6), (3×3), (2×2), and (1×1), and output feature maps of sizes (26×26×256), (53×53×256), (79×79×256), and (160×160×256), respectively;

[0053] In step 3-2, the four feature maps are convolved with a 3×3 convolution kernel, and then upsampled to 160×160×256 by bilinear interpolation. Finally, they are concatenated in the channel dimension to obtain the final composite feature map.

[0054] Step 3-3, the composite feature map obtained by fusing the low-level features and high-level features of forest fire is sent to the subsequent network, and its network structure is as follows Figure 2 shown.

[0055] Step 4: To address the complex background issues in UAV aerial photography of forest fires, the attention mechanism SENet is introduced to improve the original Flatten layer and FC1 and FC2 fully connected layers of Faster-RCNN, thereby improving the model's learning efficiency and detection accuracy for early forest fire detection in UAV aerial photography scenarios.

[0056] Specifically, the detailed steps of using the attention mechanism SENet to improve the Faster-RCNN network to improve the speed of early forest fire target detection in drone aerial photography scenarios are as follows:

[0057] In step 4-1, global average pooling is used to pool the two-dimensional features of each channel of the feature matrix obtained by the ROI Pooling layer into a real number, so that the feature matrix is ​​converted from (batch_size, h, w, c) to (batch_size, 1, 1c).

[0058] In step 4-2, the pooled feature vector is passed through a fully connected layer to reduce its dimension to 1 / S (the size becomes batch_size, 1, 1, c*1 / S), and then activated using the swish activation function. Then, the feature vector is passed through another fully connected layer to become (batch_size, 1, 1, c), and finally activated by the sigmoid activation function.

[0059] In step 4-3, the obtained feature vector is multiplied by the feature matrix obtained by the ROI Pooling layer in the channel direction to obtain a weighted feature map.

[0060] Step 5: To address the problem of early forest fire detection in drone aerial photography scenarios, complete the above improvements and build a network to train the improved Faster-RCNN model. Use the improved model to detect early forest fires in drone aerial images.

[0061] The improved Faster-RCNN model in step 5 includes a feature extraction network module, an RPN module, a RoI Pooling module, an attention mechanism SENet module, and a Classification module. All of the above modules are integrated into a network that can be trained end-to-end as a whole. Its structure is as follows: Figure 3 shown.

[0062] We also compared this data with a self-built early forest fire data set from a drone aerial photography scenario. The final results are shown in Table 1:

[0063] Table 1 Comparative experimental results:

[0064] algorithm AP (%) FPS (f / s) YOLOv6 54.7 80.2 SSD 63.2 22.5 Faster-RCNN 79.4 14.1 Ours 92.6 20.2

[0065] To verify the effectiveness of the improved algorithm for detecting fire targets in aerial forest imagery, the present invention conducted comparative experiments with the YOLOv6, SSD, and Faster-RCNN models. The experiments used AP as the evaluation metric. During training, samples with an IoU greater than or equal to 0.5 were considered positive, while those less than 0.5 were considered negative. The experimental results are shown in Table 1. As can be seen from Table 1, the improved algorithm based on Faster-RCNN outperformed all other compared models in terms of AP values, achieving higher accuracy and speed than the original Faster-RCNN model. This is because the improved algorithm fully considers the characteristics of the aerial forest fire dataset and addresses the issues of high aerial image resolution, small fire area, and fire occlusion in early-stage forest fire images captured by drones. Ultimately, the improved Faster-RCNN algorithm achieved an AP value of 92.6%, outperforming the original Faster-RCNN, SSD, and YOLOv6 models. It also achieved an FPS of 20.2 f / s, demonstrating reasonable real-time detection capabilities.

[0066] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for early forest fire detection based on drone aerial images, characterized in that: The following steps are involved: Step 1: Collect and create a dataset of early forest fire images from drone aerial photography. First, remove noise from the aerial forest fire images using a bilateral filtering algorithm. Then, use a dark channel prior dehazing algorithm to dehaze the aerial forest fire images. Step 2: To address the problem of early forest fire targets being too small in drone aerial photography, we selected the two-stage Faster-RCNN network as the base detection network and used the K-means clustering algorithm to optimize the Faster-RCNN anchor boxes to fit the early forest fire targets in the drone aerial images. Step 3: Introduce the PPM structure to improve the Faster-RCNN network and enhance the model receptive field by integrating contextual features; Step 4: To address the complex background issues in drone-photographed forest fire images, we introduced the attention mechanism SENet to improve the original Flatten layer and FC1 and FC2 fully connected layers of Faster-RCNN. Step 5: To address the problem of early-stage forest fire detection in drone aerial photography scenarios, complete the aforementioned improvements and build a network to train an improved Faster-RCNN model. Use the improved model to detect early-stage forest fires in drone aerial images. In step 3, the Faster-RCNN algorithm based on ResNet50+FPN is used for early forest fire detection in drone aerial images. The ResNet50 network consists of 49 convolutional layers and 1 fully connected layer. The image scale is changed by the convolution step size. The network is divided into five stages. The image is downsampled by 2 times in each stage. The first stage consists of a 7×7 convolution. The remaining four stages introduce 3, 4, 6, and 3 residual structures composed of 1×1, 3×3, and 1×1 convolutions, respectively, and the convolution-related parameters are set in the residual block. Based on ResNet50+FPN as the feature extraction network, the PPM structure is added after the generated five feature maps P2-P6 to further improve the receptive field of the forest fire detection model. The improved feature extraction network structure is as follows; In the PPM structure, the five feature maps P2-P6 generated by the ResNet50+FPN structure are connected to the PPM structure respectively; the maximum pooling downsampling is performed using the pooling kernels of 6×6, 3×3, 2×2, and 1×1 size pairs, and the output sizes are 26×26×256, 53×53×256, 79×79×256, and 160×160×256 respectively. The four obtained feature maps are convolved with a 3×3 convolution kernel, and then upsampled to 160×160×256 through bilinear interpolation. Finally, they are spliced ​​in the channel dimension to obtain the final composite feature map; In step 4, the attention mechanism is introduced to solve the complex background problem of aerial forest fire images and prevent the occurrence of gradient vanishing due to too many network layers. The attention mechanism SENet is used to replace the original Flatten layer and FC1 and FC2 fully connected layers in the Faster-RCNN model. The specific process is as follows: In step 4.1, by using global average pooling, the two-dimensional features of each channel of the feature matrix obtained by the ROIPooling layer are pooled into a real number, so that the feature matrix is ​​converted from (batch_size, h, w, c) to (batch_size, 1, 1, c); In step 4.2, the pooled feature vector is passed through a fully connected layer to reduce its dimension to 1 / S (the size becomes batch_size, 1, 1, c*1 / S), and then activated using the swish activation function. Then, the feature vector is passed through another fully connected layer to become (batch_size, 1, 1, c), and finally activated by the sigmoid activation function. In step 4.3, the obtained feature vector is multiplied with the feature matrix obtained by the ROIPooling layer in the channel direction to obtain a weighted feature map.

2. The method for early forest fire detection based on drone aerial images according to claim 1, characterized in that: In step 1, a complete UAV aerial forest fire dataset is built. The forest fire images are first denoised using a bilateral filtering algorithm, and then dehazed using a dark channel prior dehazing algorithm to enhance the original aerial forest fire dataset. The processing steps include the following: Step 1.1: Use the bilateral filtering algorithm to preprocess the forest fire image. Perform local weighted averaging on the input forest fire image to obtain the pixel values ​​of the output image, as shown in the following formula: in, In the above formula, D is the coordinate set of the pixel points in the neighborhood of the center of the forest fire image, F(x,y) is the pixel value of the output image, f(i,j) is the pixel value of the current pixel point (i,j) in the image, ω s (i,j) and ω r (i, j) are the spatial domain weight coefficient and grayscale domain weight coefficient of the image at (i, j), f(x, y) is the pixel value of the center point (x, y), σ s and σ r are the standard deviation in the spatial domain and the standard deviation in the grayscale domain respectively; In step 1.2, the dark channel prior dehazing algorithm is used to extract the dark channel features of the aerial forest fire image at a fixed scale, as shown in the following formula: in, is the dark channel feature with Ω as the scale, I ch (x, y) are the R, G, and B channels of the blurred foggy aerial forest fire image; In step 1.3, the atmospheric light value and the transmission rate are calculated using the dark channel features to obtain the enhanced aerial forest fire image.

3. The method for early forest fire detection based on drone aerial images according to claim 1, characterized in that: In step 2, statistics on the early forest fire dataset captured by drone aerial photography revealed the following data: the aspect ratio of early forest fire targets in drone aerial photography was 1.2, with the largest aspect ratio reaching 4.

77. Some forest fire targets also had aspect ratios between 3 and 4.

7. The K-means clustering algorithm was used to cluster the annotation box sizes of this dataset to obtain the best-fitting anchor box. IOU is used to measure the distance between objects; d IOU The calculation formula is shown below: d IOU =1-IOU Among them, S j Represents the intersection area of ​​the Anchor box and the annotation box, S b Represents the combined area of ​​the Anchor box and the annotation box; The specific steps of the K-means clustering algorithm are as follows: Step 2.1, select the annotation box X = {X1, X2, X3, ..., Xn} of the forest fire target in n data sets; Step 2.2: Randomly select nine labeled boxes of forest fire targets as cluster centers {C1, C2, C3, …, C9}; Step 2.3: Calculate the IOU distance between each forest fire target annotation box and the nine cluster center points respectively; Step 2.4: Classify each forest fire target annotation box into the cluster center point with the smallest IOU distance; Step 2.5: Calculate the average coordinates of each forest fire target label box in the nine categories and use it as the new cluster center point; Step 2.6: Determine whether the specified number of cycles has been reached or the cluster center point is no longer changing, otherwise return to step 2.

3.

4. The method for early forest fire detection based on drone aerial images according to claim 1, characterized in that: In step 5, use Pytorch to build the improved model; based on the above improved Faster-RCNN model, including the feature extraction network module, RPN module, RoIPooling module, attention mechanism SENet module and Classification module, all the above modules are integrated into a network and trained end-to-end as a whole.

Images