An optical remote sensing forest fire detection method based on a CNN-Transformer hybrid backbone network

Abstract

The application discloses a kind of forest fire detection methods of optical remote sensing based on CNN-Transformer hybrid backbone network, belong to remote sensing image processing and deep learning target detection technical field.The method first obtains forest area optical remote sensing image, image is cropped, normalized, labeled and data enhancement processing, constructs forest fire detection dataset;Then construct FCT-FireNet forest fire detection network, the FCT-FireNet includes fire context enhancement type CNN-Transformer hybrid backbone network, boundary context feature fusion network, small fire point enhancement detection head and fire confidence calibration module.The hybrid backbone network extracts fire point luminance mutation, flame edge and texture details by local convolution branch, models long-distance dependence between smoke diffusion, vegetation shelter and terrain background by window self-attention branch, and realizes the adaptive fusion of local detail features and global context features using the gated cross-branch fusion module.

Description

Technical Field

[0001] This invention relates to the field of optical remote sensing image target detection technology, specifically to an optical remote sensing forest fire detection method based on a CNN-Transformer hybrid backbone network. Background Technology

[0002] Forests are an important component of Earth's terrestrial ecosystems, playing a vital role in regulating climate, maintaining ecological balance, and protecting biodiversity. However, forest fires are characterized by their suddenness, destructiveness, and difficulty in prevention and control, posing a significant threat to the ecological environment and human life and property. Because forest areas are typically sparsely populated, traditional ground-based sensors are insufficient for monitoring them. Optical remote sensing images, with their wide coverage, high resolution, and real-time acquisition capabilities, have become a crucial source of information for forest fire monitoring.

[0003] Currently, remote sensing methods for forest fire detection mainly include traditional image processing techniques and deep learning-based intelligent detection methods. Traditional methods primarily rely on brightness temperature thresholding, spectral analysis, edge and texture anomaly detection, and statistical model analysis. Although these methods are relatively inexpensive to implement, they suffer from problems such as strong threshold dependence, poor feature universality, high computational complexity, and insufficient real-time performance, making them unsuitable for remote sensing image scenarios with high resolution, small fire targets, and complex backgrounds.

[0004] With the development of deep learning, object detection methods based on convolutional neural networks (CNNs), such as the YOLO series algorithms, have been widely used in forest fire detection. However, existing deep learning methods still have the following shortcomings in optical remote sensing images: 1. In the early stages of a fire, small fire spots have a small target area and weak texture, making them easy to miss. 2. The fire boundary is blurred, and traditional upsampling and feature fusion methods result in low positioning accuracy; 3. In forest scenes, flames are easily obscured by smoke and trees, and existing detection heads cannot fully model the long-distance dependence of the obscured areas; 4. The multi-scale fire target detection effect is limited, and it is difficult to simultaneously detect small targets, local fires and large-area fires; 5. Remote sensing equipment has limited computing resources, and the model needs to balance lightweight design and accuracy, but existing methods are unable to achieve this balance.

[0005] Therefore, there is an urgent need for a forest fire detection method applicable to optical remote sensing images. This method should be able to accurately extract small-scale fire point features, enhance fire boundary localization, model the contextual relationship of occlusion areas, and achieve high-precision fire detection in complex forest scenes while ensuring lightweight design. Summary of the Invention

[0006] This invention provides an optical remote sensing forest fire detection method based on a CNN-Transformer hybrid backbone network, which solves the problems of existing forest fire remote sensing detection methods, such as difficulty in extracting small-scale fire point features, low accuracy in fire boundary positioning, insufficient recognition ability under smoke and tree obstruction conditions, limited multi-scale fire detection performance, and difficulty in balancing detection accuracy and lightweight design.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: Acquire optical remote sensing images of forest fires, preprocess and annotate the images, construct a forest fire detection dataset, and divide it into training set, validation set and test set according to a set ratio; An FCT-FireNet forest fire detection model based on a CNN-Transformer hybrid backbone network is constructed. The model includes a CNN-Transformer hybrid backbone network (FCT-Backbone), a gated cross-branch fusion module (GCFM), a boundary context feature fusion network (BCFN), a small fire point enhancement detection head (SFED Head), and a fire confidence calibration module (FCCM). The training set is input into the FCT-FireNet forest fire detection model, and the optimal model parameters are obtained through iterative training and parameter optimization. The optical remote sensing image to be detected is input into the trained FCT-FireNet model to achieve automatic detection of forest fire targets and output fire location, category information and detection confidence.

[0008] Furthermore, the methods for preprocessing and labeling optical remote sensing images are as follows: Collect publicly available forest fire remote sensing datasets and forest fire optical images acquired by satellite remote sensing and UAV remote sensing to construct a forest fire dataset covering different seasons, terrains, vegetation densities, and lighting conditions. The original images are subjected to size unification, data augmentation, and annotation processing, and then divided into training, validation, and test sets according to a preset ratio.

[0009] Furthermore, the improved modules of the FCT-FireNet forest fire detection model are designed as follows: To address the challenges of small-scale fire points, weak texture features, and severe interference from complex backgrounds in the early stages of forest fires, a hybrid CNN-Transformer backbone network (FCT-Backbone) is constructed within the core network. Convolutional branches extract local detail features such as flame texture, fire point edges, and smoke contours; Transformer branches model long-range contextual dependencies between the fire area and its surrounding environment; and a gated cross-branch fusion module (GCFM) adaptively fuses local texture information with global contextual information, thereby enhancing the representation of fire target features and improving the accuracy of small-scale fire point detection.

[0010] To address the issues of blurred forest fire edges and inaccurate boundary localization caused by smoke diffusion, a Boundary Context Feature Fusion Network (BCFN) is designed. This network extracts shallow spatial edge information through a boundary feature extraction module, extracts deep semantic information through a semantic feature enhancement module, and utilizes a boundary-guided fusion mechanism to achieve the collaborative fusion of shallow boundary features and deep semantic features, thereby improving the accuracy of fire boundary localization and the integrity of the target outline.

[0011] To address the challenges of easily missing small initial fire points and the difficulty in detecting multi-scale fire targets, a Small Fire Point Enhancement Detection Head (SFED Head) is designed. This head utilizes a high-resolution feature layer to construct a small target detection branch, while simultaneously combining a multi-scale feature pyramid structure to construct medium-scale and large-scale fire detection branches. This enables collaborative detection of fire targets at different scales, improving the model's ability to detect initial fire points and complex fire scenarios.

[0012] To address the high false detection rate caused by clouds, bright bare ground, building reflections, and smoke interference in forest scenes, a Fire Confidence Calibration Module (FCCM) was designed. This module comprehensively utilizes flame area features, smoke context features, multi-scale prediction consistency features, and background interference features to recalibrate the confidence of detection results, thereby improving the model's ability to distinguish real fire targets and reducing the false detection rate in complex background environments.

[0013] Furthermore, the FCT-FireNet model is trained using a joint loss function, including bounding box regression loss, classification loss, target confidence loss, boundary awareness loss, and confidence calibration loss; multi-task collaborative optimization improves the model's overall performance in fire target localization, classification, and boundary recognition.

[0014] Compared with the prior art, the present invention has the following beneficial effects: (1) Construct a hybrid backbone network that works in collaboration between CNN and Transformer to achieve effective fusion of local texture information and global context information, thereby improving the ability to extract fire target features in complex forest scenarios; (2) A boundary context feature fusion network is proposed to enhance the ability to express fire boundaries and semantic information and improve the accuracy of fire target localization; (3) Design a small fire point enhanced detection head to effectively solve the problem of missed detection of small target fire points in the early stage of forest fire and improve the detection performance of multi-scale fire targets; (4) Introduce a fire confidence calibration mechanism to reduce false detections caused by cloud, bare ground reflection and smoke interference, and improve the reliability of detection results; (5) The modules work together to maintain high detection accuracy while taking into account the need for lightweight models, making it suitable for various forest fire monitoring scenarios such as satellite remote sensing, UAV remote sensing and ground monitoring platforms. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating the overall framework of the FCT-FireNet forest fire detection network provided by the present invention. Figure 2 This is a diagram of the CNN-Transformer hybrid backbone network structure provided by the present invention; Figure 3 This is a structural diagram of the gated cross-branch fusion module provided by the present invention; Figure 4 A diagram of the boundary context feature fusion network structure provided by this invention; Figure 5 This is a structural diagram of the small fire point enhancement detection head provided by the present invention; Figure 6 This is a structural diagram of the fire confidence calibration module provided by the present invention; Figure 7 The structural diagram provided for this invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used herein in the specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application.

[0018] To address the shortcomings of existing optical remote sensing methods for forest fire detection in complex forest scenarios, such as weak small fire point feature extraction, inaccurate fire boundary localization, missed detections due to smoke and tree obstruction, and false detections due to complex background interference, this invention proposes an optical remote sensing forest fire detection method based on a CNN-Transformer hybrid backbone network. This method constructs an FCT-FireNet forest fire detection network, as follows: Figure 1 As shown, the network mainly includes a CNN-Transformer hybrid backbone network, a gated cross-branch fusion module, a boundary context feature fusion network, a small fire point enhancement detection head, and a fire confidence calibration module.

[0019] First, optical remote sensing images of forest areas are acquired, preprocessed, and labeled to construct a forest fire detection dataset. These optical remote sensing images can be sourced from satellite remote sensing platforms, UAV remote sensing platforms, airborne remote sensing platforms, and publicly available forest fire image datasets. The acquired images cover forest fire scenes under different lighting conditions, vegetation densities, terrain environments, smoke obscuration levels, and different stages of fire development.

[0020] The acquired optical remote sensing images are cropped, scaled, normalized, and labeled, with fire areas in the images labeled as fire target categories. Preferably, the remote sensing images are cropped or scaled to a fixed size, such as 640×640, and severely blurred, heavily occluded, or poorly labeled images are removed. To improve the model's generalization ability, the training set images are subjected to random flipping, random rotation, scaling, brightness perturbation, color perturbation, and Mosaic stitching enhancement. The processed dataset is then divided into training, validation, and test sets according to a set ratio.

[0021] To address the challenges of small fire spots being small in scale, exhibiting weak brightness variations, and having texture features easily lost in deep networks during the initial stages of forest fires, this invention constructs a CNN-Transformer hybrid backbone network, such as... Figure 2 As shown, the CNN-Transformer hybrid backbone network includes a local fire point texture extraction branch, a window context self-attention branch, and a gated cross-branch fusion module. The input image is first processed through convolutional layers for preliminary feature extraction to obtain a basic feature map, which is then input into the local fire point texture extraction branch and the window context self-attention branch, respectively.

[0022] The local fire point texture extraction branch is mainly used to extract texture features of flame edges, small fire point brightness abrupt changes, smoke outlines, and weak fire point textures in gaps in the tree canopy. This branch can be composed of depthwise separable convolutions, pointwise convolutions, batch normalization layers, and activation functions. For the input feature map X, the local fire point texture extraction process can be represented as: Here, DWConv represents depthwise convolution, PWConv represents pointwise convolution, and F_cnn represents the local fire point texture features output by the convolutional branch. This structure enhances the model's ability to extract local fire point edges and fine-grained textures with lower computational cost.

[0023] To address the complex contextual relationships within forest fire remote sensing images caused by factors such as smoke diffusion, tree occlusion, mountain shadows, and cloud / fog backgrounds, this invention introduces a window contextual self-attention branch into the backbone network. This branch divides the input feature map into multiple local windows, calculates the query matrix Q, key matrix K, and value matrix V within each window, and obtains contextually enhanced features within the window through a self-attention mechanism. The calculation process is as follows: Where Q represents the query matrix, K represents the key matrix, V represents the value matrix, and d represents the feature channel dimension. Through the window context self-attention mechanism, spatial correlation information between the fire point area, smoke diffusion area, vegetation-occupied area, and surrounding background can be captured, thereby improving the model's ability to identify occluded fire points and fire targets in complex backgrounds.

[0024] To address the issue that convolutional branches focus only on local textures while Transformer branches are more biased towards global context, directly adding these two types of features can easily lead to redundancy or conflicts. This invention proposes a gated cross-branch fusion module, such as... Figure 3 As shown. This module converts the local texture features F output by the convolution branch. cnn Contextual features F of the Transformer branch output trans The layers are concatenated, and a fusion weight G is generated using a convolutional layer and a sigmoid activation function, specifically as follows: Where σ represents the Sigmoid activation function, Conv represents the convolution operation, [F cnn ,F trans The symbol ] indicates a feature concatenation operation. Subsequently, the two types of features are weighted and fused according to the fusion weight G to obtain the enhanced fire feature map F. out : Here, ⊙ represents element-wise multiplication. Through this module, when the detection area is a small fire spot or the edge of a flame, the network can increase the feature weights of the convolutional branch; when the detection area is obscured by smoke, vegetation, or complex background interference, the network can increase the feature weights of the Transformer branch, thereby achieving adaptive fusion of local texture features and global context features.

[0025] To address the problem of inaccurate detection box localization caused by blurred edges and unclear fire target outlines due to smoke diffusion in forest fires, this invention proposes a boundary context feature fusion network, such as... Figure 4 As shown, the network includes a boundary feature extraction module, a semantic feature enhancement module, and a boundary-guided fusion module. The boundary feature extraction module is used to extract flame edges, smoke outlines, and fire spread boundary information from shallow high-resolution features; the semantic feature enhancement module is used to extract high-level semantic information of the fire area from deep features; and the boundary-guided fusion module is used to guide the fusion of shallow spatial features and deep semantic features using boundary saliency information.

[0026] Boundary saliency maps can be generated from learnable edge convolutional units, and the calculation process is as follows: Among them, F low Represents a shallow, high-resolution feature map, Conv edge Let B represent the learnable edge convolutional unit, and let B represent the boundary saliency map. Subsequently, the boundary saliency map B is applied to the fusion process of shallow spatial features and deep semantic features: Among them, F high This represents the deep semantic feature map, Up represents the upsampling operation, and F represents the deep semantic feature map. fuse This represents the fused feature map after boundary enhancement. This structure preserves more spatial details in the fire boundary region and stronger semantic information in the non-boundary region, thereby improving the accuracy of fire target boundary localization.

[0027] To address the issue that initial forest fire ignition points are small in scale and have a low pixel count in optical remote sensing images, making them easily weakened or even lost during multiple downsampling processes, this invention proposes a small fire point enhancement detection head, such as... Figure 5 As shown. The small fire point enhancement detection head includes a high-resolution small target detection branch and a multi-scale fire detection branch. The high-resolution small target detection branch receives high-resolution feature maps from shallow or mid-shallow layers, such as P2 or P3 feature maps, to detect early small-scale fire points, smoldering fire points, and weak fire points in canopy gaps; the multi-scale fire detection branch receives feature maps of different scales such as P3, P4, and P5 to detect medium-scale fire points, large-area open flames, and smoke-covered fire areas.

[0028] Each detection branch includes a classification sub-branch, a bounding box regression sub-branch, and a target confidence sub-branch. The classification sub-branch outputs the fire category probability, the bounding box regression sub-branch outputs the fire target location coordinates, and the target confidence sub-branch outputs the probability that the detection box contains a fire target. By having the high-resolution small target detection branch work in conjunction with the multi-scale fire detection branch, the model's ability to detect fire targets at different scales can be improved.

[0029] To address the issue of false detections caused by the similarity in color or brightness between bright areas of clouds and fog, reflective areas of bare ground, red rooftops of buildings, areas reflecting sunlight, and reflective areas of water in optical remote sensing forest scenes, leading to false alarms, this invention proposes a fire confidence calibration module, such as... Figure 6 As shown, this module comprehensively utilizes the flame response features inside the detection frame, the smoke context features around the detection frame, the multi-scale prediction consistency features, and the background interference features to recalibrate the initial confidence level of the detection head output.

[0030] For the initial confidence level C output by the detection head det The fire confidence calibration module first extracts the flame response features R inside the detection frame. fire Contextual response features of surrounding smoke R context Multi-scale prediction consistency feature R consistency and background interference features R background Then, the above features are input into a multilayer perceptron to generate a calibration factor R_cal: The final calibrated confidence level C_final is obtained: Here, MLP stands for Multilayer Perceptron, and σ represents the Sigmoid activation function. This module reduces the probability of non-fire-prone areas being misidentified as fire points, improving the reliability of detection results in complex forest remote sensing scenarios.

[0031] During model training, the training set is input into the FCT-FireNet forest fire detection network. The network's forward propagation yields predicted bounding boxes, class probabilities, target confidence scores, boundary saliency maps, and calibration confidence scores. This invention employs a joint loss function to train the network. This joint loss function includes bounding box regression loss, classification loss, target confidence loss, boundary awareness loss, and confidence calibration loss. The total loss function can be expressed as: Among them, L box L represents the bounding box regression loss. cls L represents the classification loss. objL represents the target confidence loss. edge L represents the boundary-aware loss. cal The confidence calibration loss represents the loss, and λ1, λ2, λ3, λ4, and λ5 represent the weight coefficients of each loss term. The bounding box regression loss optimizes the positional relationship between the predicted and ground truth bounding boxes; the classification loss optimizes the fire category recognition capability; the target confidence loss determines whether the detected bounding box contains a fire target; the boundary awareness loss constrains the consistency between the boundary saliency map and the actual fire boundary; and the confidence calibration loss constrains the consistency between the calibration confidence and the actual detection quality.

[0032] In the actual detection process, the optical remote sensing image of the forest area to be detected is input into the trained FCT-FireNet forest fire detection model. The model first extracts local fire texture features and global contextual semantic features through a CNN-Transformer hybrid backbone network; then, it achieves adaptive fusion of the two types of features through a gated cross-branch fusion module; next, it enhances the response of the fire boundary area through a boundary context feature fusion network; then, it outputs the initial fire detection results through a small fire point enhancement detection head; finally, it calibrates the detection results through a fire confidence calibration module, and outputs the final forest fire target bounding box, category information, and calibrated confidence score.

[0033] In summary, this invention enhances the local texture and global context representation capabilities of optical remote sensing forest fire targets by employing a CNN-Transformer hybrid backbone network, achieves adaptive fusion of different features through a gated cross-branch fusion module, improves the accuracy of fire boundary localization through a boundary context feature fusion network, enhances the detection capability of small-scale fire points in the early stages through a small fire point enhancement detection head, and reduces the false detection rate in complex backgrounds through a fire confidence calibration module, thereby achieving high-precision detection of forest fire targets in complex forest remote sensing scenarios.

[0034] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of protection of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of this invention patent should be determined by the appended claims.

Claims

1. An optical remote sensing method for forest fire detection based on a CNN-Transformer hybrid backbone network, characterized by: Step 1: Obtain and organize the forest fire optical remote sensing image dataset, preprocess and label the images, and divide them into training set, validation set and test set according to the set ratio; Step 2: Construct the FCT-FireNet forest fire remote sensing image target detection network, which includes a CNN-Transformer hybrid backbone network, a boundary context feature fusion network, a small fire point enhancement detection head, and a fire confidence calibration module; Step 3: Use the training set to train and optimize the parameters of the FCT-FireNet forest fire remote sensing image target detection network built in Step 2; Step 4: Input the optical remote sensing image of the forest area to be detected into the trained FCT-FireNet network, and output the location, category information and confidence results of the forest fire target.

2. The optical remote sensing forest fire detection method based on a CNN-Transformer hybrid backbone network as described in claim 1, characterized in that: The CNN-Transformer hybrid backbone network described in step 2 includes a local fire texture extraction branch, a window context self-attention branch, and a gated cross-branch fusion module. The local fire texture extraction branch uses a convolutional structure to extract fire edge, brightness variation, and smoke texture features. The window context self-attention branch uses a window self-attention mechanism to model the long-distance dependency between the fire region, the smoke diffusion region, and the forest background. The gated cross-branch fusion module adaptively fuses the features from the convolutional branch and the Transformer branch to obtain an enhanced fire feature representation.

3. The optical remote sensing forest fire detection method based on a CNN-Transformer hybrid backbone network as described in claim 1, characterized in that: The boundary context feature fusion network described in step 2 includes a boundary feature extraction module, a semantic feature enhancement module, and a boundary-guided fusion module. The boundary feature extraction module is used to extract flame edges, smoke outlines, and fire spread boundary information. The semantic feature enhancement module is used to extract deep semantic features of the fire area. The boundary-guided fusion module fuses shallow spatial features and deep semantic features based on boundary saliency information to improve the accuracy of fire target boundary positioning.

4. The optical remote sensing forest fire detection method based on a CNN-Transformer hybrid backbone network as described in claim 1, characterized in that: The small fire point enhancement detection head in step 2 includes a high-resolution small target detection branch and a multi-scale fire detection branch. The high-resolution small target detection branch uses high-resolution feature maps to detect initial small-scale fire points. The multi-scale fire detection branch uses feature maps of different scales to detect medium-scale fire points, large-area open flames, and smoke-covered fire areas, thereby improving the model's ability to detect fire targets of different scales.

5. The optical remote sensing forest fire detection method based on a CNN-Transformer hybrid backbone network as described in claim 1, characterized in that: The fire confidence calibration module in step 2 comprehensively utilizes the flame response features inside the detection frame, the surrounding smoke context features, the multi-scale prediction consistency features, and the background interference features to recalibrate the initial detection confidence. The background interference features include the color and texture features of cloud and fog bright areas, bare ground reflective areas, building roof areas, and solar reflective areas. The fire confidence calibration module reduces the false detection rate in complex background environments and improves the reliability of detection results.

6. The optical remote sensing forest fire detection method based on a CNN-Transformer hybrid backbone network as described in claim 2, characterized in that: The gated cross-branch fusion module generates fusion weights based on the output features of the convolutional branch and the output features of the Transformer branch, and performs weighted fusion of local texture features and global context features according to the fusion weights, thereby achieving adaptive expression of fire features under different scenarios.

7. The optical remote sensing forest fire detection method based on a CNN-Transformer hybrid backbone network as described in claim 1, characterized in that: The loss functions used in step 3 for training include bounding box regression loss, classification loss, target confidence loss, boundary-aware loss, and confidence calibration loss; the boundary-aware loss is used to constrain the consistency between the boundary saliency map and the actual fire boundary; the confidence calibration loss is used to constrain the consistency between the calibration confidence and the actual detection quality.

Images