A drone-based early forest fire smoke detection system and method

Abstract

This invention discloses an early forest fire smoke detection system and method using unmanned aerial vehicles (UAVs), belonging to the field of intelligent forest fire monitoring and computer vision technology. The system includes an image acquisition module, a preprocessing module, a smoke detection network module, a high-resolution feature preservation module, a frequency domain residual enhancement module, a training supervision enhancement module, and a detection output module. The method includes: acquiring forest scene images collected by a UAV and preprocessing the images; inputting the preprocessed images into the smoke detection network, preserving shallow, small-scale smoke details through the high-resolution feature preservation module; enhancing the frequency domain response of the feature map through the frequency domain residual enhancement module to improve the distinguishability between smoke targets and complex natural backgrounds; introducing auxiliary segmentation supervision and normalized Wasserstein distance constraints during the training phase to enhance the consistency of smoke region response and the stability of small-scale smoke target localization; and finally outputting the location and confidence level of the smoke target. This system and method can improve the detection accuracy of early weak smoke targets in UAV inspection scenarios, reduce the false detection rate and false negative rate in complex backgrounds, and are suitable for early forest fire warning.

Description

[0001] This invention belongs to the fields of forest fire monitoring and early warning, UAV remote sensing image processing and artificial intelligence target detection technology, specifically relating to an early forest fire smoke UAV detection system and method. Background Technology

[0002] Forest fires are characterized by their suddenness, rapid spread, and wide-ranging destruction, severely impacting forest resources, the ecological environment, and the safety of life and property. Smoke typically appears in the smoldering stage before open flames form; therefore, timely detection of early smoke is a crucial technical means for forest fire early warning. With the development of image processing, deep learning, and edge computing technologies, visual image-based forest fire smoke detection methods have been gradually applied in the field of fire monitoring. For example, CN119785216A discloses an early forest fire smoke detection system and method. This system constructs a YOLOv7-Smockey smoke detection network and combines it with PyTorch, TensorRT, and NVIDIA Jetson Nano embedded platforms to achieve real-time smoke detection and video stream delivery in network camera scenarios. This type of solution primarily focuses on the edge deployment, inference acceleration, and real-time operation capabilities of the smoke detection model. CN117409191B discloses a fire inspection and early warning method based on UAVs and an improved YOLOv8 target detection algorithm. It uses a UAV to collect real-time video streams and deploys the improved target detection model to a development board to achieve fire signal detection and alarm. This type of solution improves the automation level of UAV fire inspection. CN106600888A discloses an automatic forest fire detection method and system. It uses video images for smoke pre-judgment and determines whether the smoke is real smoke by obtaining high-frequency components through wavelet decomposition, thereby improving the timeliness and reliability of forest fire monitoring.

[0003] However, the aforementioned existing technologies still have certain shortcomings. First, some early smoke detection solutions are mainly geared towards deployment scenarios with fixed cameras or embedded platforms, focusing on real-time model operation and inference acceleration. They lack targeted high-resolution feature preservation mechanisms to address issues such as small target scale, low contrast, blurred boundaries, and weak texture in early smoke under UAV long-distance imaging conditions. Second, while some UAV fire inspection solutions combine the UAV platform with target detection algorithms, their focus is primarily on fire target detection, video stream processing, and alarm procedures, with insufficient consideration given to the issue of easily lost detailed information during continuous downsampling of early smoke in the detection network. Third, traditional wavelet or high-frequency component analysis methods are mostly applied to the original video image or suspected smoke areas, failing to fully address the problem of insufficient utilization of smoke frequency domain discrimination information by the intermediate feature layers of deep detection networks. Furthermore, for small-scale, diffuse smoke targets, traditional bounding box regression methods based on intersection-union ratio (IoU) are easily affected by pixel-level offsets, leading to unstable localization.

[0004] Therefore, there is an urgent need for an early smoke detection system and method for UAV forest patrol scenarios, which can improve the feature preservation ability of small-scale weak smoke targets, the discrimination ability in complex backgrounds, and the stability of target localization while ensuring detection efficiency. To solve the above problems, this invention addresses the characteristics of early smoke in UAV-acquired images, namely "small scale, low contrast, blurred boundaries, weak texture, and strong background interference," by introducing a high-resolution feature preservation structure to reduce the loss of shallow smoke details during downsampling; through a frequency domain residual enhancement module, the frequency domain response of smoke is modeled and enhanced in the intermediate feature layer of the detection network, improving the ability to distinguish smoke from complex backgrounds such as clouds, mountain shadows, and vegetation textures; and auxiliary supervision constraints are introduced during the training phase to enhance the consistency of smoke region response and the stability of small-scale target localization, thereby improving the accuracy and reliability of UAV early smoke detection in forest fires. Summary of the Invention

[0005] (a) Technical problems to be solved To address the problems of existing UAV-based early forest fire smoke detection methods, such as easy omission of small-scale smoke targets, insufficient expression of weak-texture smoke features, high false detection rate under complex natural background interference, unstable box-level supervision signals, and high computational cost of some models, this invention provides an UAV-based early forest fire smoke detection system and method.

[0006] This invention aims to achieve the following objectives: 1. Improve the detection accuracy of small-scale, low-contrast, and weakly textured smoke targets in UAV images; 2. Reduce false positives caused by complex backgrounds such as clouds, mountains, vegetation textures, and changes in lighting; 3. Improve the stability of the small target bounding box regression process; 4. Improve model training performance without adding an extra supervision branch during the inference phase; 5. Achieving a better balance between detection accuracy and computational complexity to adapt to applications such as drone inspections and early warning of forest fires. (II) Technical Solution This invention provides an early forest fire smoke detection system and method using drones. The system includes a drone image acquisition module, a smoke image data processing module, a weak smoke detection network module, a training period auxiliary supervision module, a model training module, a model inference module, and an early warning result output module.

[0007] The UAV image acquisition module is used to acquire image or video data collected during UAV inspections in forests, mountains, forest edges, or other outdoor environments.

[0008] The smoke image data processing module is used to perform size normalization, annotation conversion, dataset partitioning, and data augmentation on the acquired image data.

[0009] The weak smoke detection network module is used to detect early forest fire smoke targets in input images and output the location and confidence information of the smoke targets. The weak smoke detection network module is based on a single-stage target detection network and includes a backbone feature extraction network, a multi-scale feature fusion network, and a detection head.

[0010] The weak smoke detection network module includes a high-resolution feature preservation module and a frequency domain residual enhancement module.

[0011] The high-resolution feature preservation module is used to retain shallow, weak smoke detail information during feature extraction and feature fusion. The high-resolution feature preservation module includes a P2 high-resolution detection branch and a Space-to-Depth downsampling structure. The P2 high-resolution detection branch is used to improve the detection capability of small-scale smoke targets, and the Space-to-Depth downsampling structure is used to reduce the loss of weak smoke detail information during traditional downsampling.

[0012] Furthermore, the weak smoke detection network module includes four detection branches at four scales: P2, P3, P4, and P5. The P2 detection branch corresponds to a 1 / 4 resolution feature layer of the input image and is used to enhance the representation ability of small-scale, low-contrast, and weak-texture smoke targets.

[0013] The Space-to-Depth downsampling structure rearranges the local spatial neighborhood information in the input feature map to the channel dimension, and uses at least one convolutional layer to perform local fusion and channel adjustment on the rearranged features, thereby reducing spatial resolution while retaining shallow local response information.

[0014] The frequency domain residual enhancement module is used to transform the feature map to the frequency domain and modulate the weak smoke response based on the amplitude spectrum statistical characteristics. In the frequency domain residual enhancement module, the phase spectrum is kept unchanged, and only the amplitude spectrum is subjected to bandgap modulation and radial spectrum prior modulation to improve the ability to distinguish between smoke targets and complex natural backgrounds.

[0015] The training-phase auxiliary supervision module is used to introduce auxiliary supervision constraints during the model training phase. These constraints include weakly supervised auxiliary segmentation constraints and normalized Gaussian Wasserstein distance constraints. The weakly supervised auxiliary segmentation constraints enhance the consistency of response in smoke regions, while the normalized Gaussian Wasserstein distance constraints improve the localization stability of small-scale smoke targets.

[0016] The model training module is used to train the weak smoke detection network module based on a deep learning framework.

[0017] The model inference module is used to deploy the trained smoke detection model on a server, edge computing device, or drone-supporting computing platform, and to perform smoke detection on real-time acquired images.

[0018] The early warning result output module is used to output the detection results to the monitoring terminal or fire early warning platform in the form of image selection, video stream, alarm signal or structured detection results.

[0019] Furthermore, the training-period auxiliary supervision module only participates in computation during the model training phase and is removed or not involved in computation during the model inference phase, thereby avoiding increasing the computational burden during the inference phase.

[0020] This invention also provides a method for early forest fire smoke detection using unmanned aerial vehicles (UAVs) based on weak visual signal enhancement, comprising the following steps: S1: Collect image data of forest scenes by drones. Collect images or videos of forests, mountains, agricultural and forestry border areas or other natural environments in the wild through drone platforms, and obtain data samples containing early smoke targets and smoke-free background images. S2: Construct an early forest fire smoke detection dataset, filter, classify and label the collected images, generate a dataset containing smoke target bounding box information, and divide it into training set, validation set and test set according to a preset ratio; S3: Enhance the smoke image data by performing color perturbation, geometric transformation, random flipping, random erasure, copy-paste, and mosaic enhancement on the training images; S4: Construct a weak smoke detection network. Based on the single-stage target detection network, introduce the P2 high-resolution detection branch to form a multi-scale detection structure, and set a Space-to-Depth downsampling structure at the key downsampling positions of the backbone network and feature fusion network. S5: Construct a frequency domain residual enhancement module. After channel compression, the input feature map is subjected to a two-dimensional real-valued fast Fourier transform to obtain the amplitude spectrum and phase spectrum. Keeping the phase spectrum unchanged, the amplitude spectrum is subjected to gated modulation based on radial frequency band statistics and radial spectrum prior modulation. Subsequently, it is restored to the spatial domain by inverse Fourier transform and fused with the spatial branch and residual path. S6: Construct a training-period auxiliary supervision mechanism, generate weakly supervised auxiliary segmentation constraints based on bounding box annotations, and introduce normalized Gaussian Wasserstein distance as an auxiliary term for small target bounding box regression; S7: Train the weak smoke detection network. Use training set images and corresponding annotations to train the model, and evaluate the model performance on the validation set to obtain the trained smoke detection model. S8: Perform early smoke detection for drones. Input the image of the drone to be detected into the trained smoke detection model, and output the detection box, category and confidence score of the smoke target. S9: Output early warning results for forest fires. When a smoke target is detected and the confidence level meets the preset threshold, output the early smoke detection results and send the results to the monitoring platform, server or early warning terminal.

[0021] In one implementation, the weak smoke detection network adds a P2 detection branch to the existing P3, P4, and P5 detection branches, forming a four-scale detection structure of P2, P3, P4, and P5. The P2 detection branch corresponds to a 1 / 4 resolution feature layer of the input image. Compared to conventional 1 / 8, 1 / 16, and 1 / 32 resolution detection branches, it can retain more shallow spatial details, thereby improving the detection capability of small-scale, weak-texture, and low-contrast smoke targets.

[0022] In one implementation, the Space-to-Depth downsampling structure is used to replace part of the traditional convolutional downsampling operation with a stride of 2. Let the input feature map be:

[0023] in, Indicates the spatial dimensions of the feature map. This represents the number of input channels. The Space-to-Depth operation rearranges information from the local spatial neighborhood to the channel dimension, resulting in:

[0024] Subsequently, at least one convolutional layer with a stride of 1 is used to locally fuse and adjust the channels of the rearranged features to obtain the output features:

[0025] in:

[0026] In one specific implementation, the convolutional layer uses a convolutional kernel size of... The convolutional layer, namely:

[0027] Through the above structure, the feature responses originally located in the local spatial neighborhood are transferred to the channel dimension and then fused through convolution, thereby reducing the direct loss of shallow weak smoke responses while reducing spatial resolution.

[0028] In one implementation, the frequency domain residual enhancement module includes a spatial branch, a frequency domain branch, and a residual fusion path.

[0029] For the feature Xc of the c-th channel in the input feature map, the frequency domain branch first performs a two-dimensional real-valued fast Fourier transform on it:

[0030] in, Indicates amplitude spectrum, Let u represent the phase spectrum, and v represent the frequency domain coordinates.

[0031] Since the phase spectrum mainly reflects the spatial structure information of the features, and the amplitude spectrum mainly reflects the energy distribution of different frequency components, the frequency domain branch keeps the phase spectrum unchanged and modulates the amplitude spectrum.

[0032] Specifically, the frequency domain half-spectrum is divided into low-frequency, mid-frequency, and high-frequency regions according to the normalized radial frequency, and a channel spectrum description vector is constructed:

[0033] in, Indicates the average amplitude in the low-frequency region. Indicates the average amplitude in the mid-frequency region. Indicates spectral attenuation characteristics, This indicates low-frequency concentration characteristics.

[0034] The spectral attenuation characteristic is defined as:

[0035] The low-frequency concentration characteristic is defined as:

[0036] in, Indicates the average amplitude in the high-frequency region. This represents the average amplitude across the entire half-spectrum region, and ε is the numerical stability term.

[0037] Inputting the channel spectrum description vector into a lightweight multilayer perceptron yields the channel spectrum gating coefficients:

[0038] Simultaneously, a radial spectrum prior is introduced:

[0039] in, τ represents the normalized radial frequency coordinate, and τ represents the spectral attenuation bandwidth control parameter.

[0040] The modulated amplitude spectrum is represented as follows:

[0041] Subsequently, the modulated amplitude spectrum and the original phase spectrum are recombined, and the frequency domain enhancement features are obtained by inverse two-dimensional real-valued fast Fourier transform. .

[0042] The frequency domain residual enhancement module ultimately outputs the result through residual fusion:

[0043] in, For input features, Output features for spatial branching. For the frequency domain branch output features, α and β are the learnable channel coefficients, and ⊙ represents the channel scaling operation.

[0044] In one implementation, the training-phase auxiliary supervision module includes an auxiliary segmentation branch. This auxiliary segmentation branch operates on the highest resolution feature layer P2 and outputs a single-channel response map.

[0045] in, This indicates a lightweight segmentation prediction head. This represents the Sigmoid activation function.

[0046] Since the training data only provides bounding box annotations and not pixel-level smoke masks, the auxiliary segmentation branch generates weakly supervised region constraints based on the bounding boxes, enabling the detection network to learn the response differences between regions inside and outside the smoke boxes. This constraint is used to enhance the consistency of responses within the smoke region, rather than to obtain precise pixel-level segmentation results.

[0047] The auxiliary segmentation loss is added as a regularization term to the overall training objective:

[0048] in, Indicates detection loss, Let λseg represent the auxiliary segmentation loss, and let λseg represent the weight of the auxiliary segmentation loss.

[0049] Furthermore, the auxiliary segmentation constraint employs a preheating mechanism to gradually increase the weight, thereby reducing the impact of unstable region responses on the main detection task during the early stages of training.

[0050] In one implementation, the training-phase auxiliary supervision module further includes a normalized Gaussian. Distance constraint. Predict the bounding box. and real frame They are modeled as two-dimensional Gaussian distributions. and The second-order Wasserstein distance between the two is defined as:

[0051] Where mp and mg represent the mean of the Gaussian distributions corresponding to the predicted and ground truth boxes, respectively, and Σp and Σg represent the covariance matrices of the Gaussian distributions corresponding to the predicted and ground truth boxes, respectively.

[0052] Normalized Gaussian Wasserstein similarity is defined as:

[0053] Where C is the normalization constant. The corresponding loss is defined as:

[0054] The final bounding box regression loss is:

[0055] Where LIoU represents the bounding box regression loss based on the intersection-union ratio. This represents the normalized Gaussian Wasserstein distance loss weights.

[0056] Furthermore, to avoid unnecessary impact on the regression of medium- and large-scale targets, the NWD constraint is only applied when the area of ​​the true bounding box is less than a preset area threshold. Enabled at: when hour:

[0057] when hour:

[0058] in, Represents a real box The area. In one specific implementation, Set to 1024 pixels squared.

[0059] (III) Beneficial Effects This invention significantly enhances the preservation of shallow, fine-grained, weak visual cues by integrating a P2 high-resolution detection branch with a Space-to-Depth lossless downsampling structure, solving the problem of information loss in small-scale smoke during multiple downsampling processes. Combined with a frequency domain residual enhancement module for targeted modulation of the amplitude spectrum, it greatly improves the discriminative power of smoke signals against complex natural backgrounds such as clouds and vegetation while maintaining the phase structure. Simultaneously, this invention introduces weakly supervised assisted segmentation constraints and a Normalized Gaussian Wasserstein distance (NWD) optimization mechanism, which not only enhances the consistency of smoke region responses but also fundamentally alleviates the technical challenge of displacement sensitivity in small target bounding box regression, improving positioning accuracy and training stability. Experimental results show that, while maintaining an extremely lightweight architecture (approximately 3.473M parameters), this invention achieves a deep synergy between detection performance (AP50 reaching 0.825) and computational efficiency, significantly reducing the risk of missed detections and false alarms in UAV inspections, and providing efficient and reliable technical support for early warning of forest fires. Attached Figure Description

[0060] Figure 1 This is a flowchart of the method for early forest fire smoke detection using unmanned aerial vehicles based on weak visual signal enhancement according to the present invention. Figure 2 This is a schematic diagram of the overall structure of the weak smoke detection network of the present invention; Figure 3 This is a schematic diagram of the P2 high-resolution detection branch and multi-scale feature fusion structure of the present invention; Figure 4 This is a schematic diagram of the Space-to-Depth downsampling structure of the present invention; Figure 5 This is a schematic diagram of the frequency domain residual enhancement module structure of the present invention; Figure 6 This is a schematic diagram of the frequency domain amplitude spectrum modulation process of the present invention; Figure 7 This is a schematic diagram of the auxiliary supervision structure during the training period of the present invention; Figure 8 This is a schematic diagram of the smoke detection results output by the UAV of the present invention. Detailed Implementation

[0061] To better illustrate the technical solution of the present invention, the present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are only for explaining the present invention and are not intended to limit the scope of protection of the present invention. Without departing from the concept of the present invention, those skilled in the art can make conventional adjustments to the specific number of network layers, parameter scale, number of training rounds, deployment devices, and thresholds.

[0062] First, the technical terms involved in this invention will be explained.

[0063] Early forest fire smoke images taken by drones: These are images of forests, mountains, or other natural landscapes captured by visible light cameras mounted on drones, which may contain faint smoke from the early or smoldering stages of a fire.

[0064] Weak visual signals refer to visual responses in an image characterized by low contrast, weak texture, blurred boundaries, low visual saliency, and easy obscuration by background interference in the target area. In this invention, early smoke is considered a weak visual signal in a complex natural background.

[0065] P2 detection branch: refers to the detection branch in the object detection network that corresponds to higher spatial resolution. Its feature map stride is usually 1 / 4 of the input image, which is used to enhance the detection capability of small-scale objects.

[0066] Space-to-Depth downsampling refers to rearranging local neighborhood information in the spatial dimension to the channel dimension to reduce information loss caused by traditional downsampling.

[0067] Frequency domain residual enhancement module: This module modulates the feature response based on amplitude spectrum statistical features after frequency domain transformation of the feature map, and then fuses it back to the spatial domain features through residual method.

[0068] AuxSeg: refers to the weakly supervised auxiliary segmentation branch during the training period. It forms region-level supervision constraints based on the bounding box annotations, which is used to enhance the consistency of response in smoke regions.

[0069] NWD: Normalized Gaussian Wasserstein distance, used to measure the difference between the Gaussian distributions corresponding to the predicted bounding box and the ground truth bounding box. In small object detection, it can alleviate the problem that IoU is too sensitive to small positional shifts.

[0070] Example 1 This embodiment provides an early forest fire smoke detection system and method using a drone. The system takes forest scene images captured by a drone as input, outputs smoke target detection results through a weak smoke detection network, and provides early fire warnings based on the detection results. The method flow of this embodiment is as follows: Figure 1 As shown.

[0071] Step 1: Acquisition of drone smoke image data and construction of dataset like Figure 1 As shown, drone image data containing early forest fire smoke was first collected. The data sources included publicly available drone smoke images and self-collected images of real outdoor scenes. The images included smoke-containing images and smoke-free images. In the smoke-containing images, smoke targets were characterized by small scale, low contrast, blurred boundaries, and diffuse morphology.

[0072] In this embodiment, the constructed drone smoke dataset includes 15,881 images, of which 12,877 contain smoke and 3,004 are smoke-free. The dataset is divided into training, validation, and test sets in an 8:1:1 ratio, with 12,652 images in the training set, 1,617 in the validation set, and 1,612 in the test set. Smoke instances with a bounding box area less than 1024 pixels squared are considered small target smoke instances.

[0073] Boundary boxes are labeled for smoke targets in the image, including the smoke target category and bounding box coordinates. The labeled data is then converted into a data format that can be read by the object detection network.

[0074] Step 2: Data Augmentation Processing like Figure 1 As shown, after the dataset is constructed, data augmentation is performed on the input images during the training phase to improve the model's adaptability to complex natural backgrounds, scale changes, and lighting changes.

[0075] In this embodiment, data enhancement includes HSV color perturbation, image rotation, translation transformation, scaling, shear transformation, horizontal flipping, Mosaic image enhancement, Copy-Paste enhancement, and random erasure enhancement.

[0076] In this embodiment, the HSV parameter is set to , , The rotation angle is set to The translation ratio was set to 0.08, the scaling ratio to 0.3, the shearing ratio to 1.0, the horizontal flip probability to 0.5, the copy-paste probability to 0.2, and the random erase probability to 0.3. Mosaic enhancements were enabled early in training and disabled in the last few training rounds to allow the model to gradually revert to the real image distribution.

[0077] Step 3: Construct a weak smoke detection network like Figure 2 As shown, the weak smoke detection network constructed in this embodiment includes a backbone network, a feature fusion network, a frequency domain residual enhancement module, and a multi-scale detection head. The weak smoke detection network is based on a single-stage target detection network, and in this embodiment, the YOLOv8 target detection network is used as a specific implementation.

[0078] The basic network consists of a backbone feature extraction network, a feature pyramid fusion network, a path aggregation network, and a detection head. A typical YOLOv8 detection head is usually set on feature layers with strides of 8, 16, and 32, corresponding to detection scales P3, P4, and P5. For early-stage smoke detection tasks on UAVs, due to the small size of the targets and weak visual response, relying solely on these scales can easily lead to missed detections of small smoke targets.

[0079] Therefore, this embodiment adds a P2 detection branch to the basic network, enabling the network to perform four-scale detection capabilities: P2, P3, P4, and P5. The P2 branch corresponds to 1 / 4 resolution of the input image and is used to enhance the detection capability of small-scale smoke targets. The P2 high-resolution detection branch and the multi-scale feature fusion structure are as follows... Figure 3 As shown.

[0080] Step 4: Configure the Space-to-Depth downsampling structure like Figure 4 As shown, at key downsampling locations in the base network, some traditional convolutional downsampling with a stride of 2 is replaced with a Space-to-Depth downsampling structure.

[0081] Let the input feature map be:

[0082] in, The spatial dimensions of the input feature map, The number of input channels. The Space-to-Depth operation will convert each... Pixels or features in the spatial neighborhood are rearranged to the channel dimension, resulting in:

[0083] Its dimensions are:

[0084] This process does not directly discard the response at spatial location, but rather transfers local spatial information to the channel dimension. Subsequently, the rearranged features are subjected to channel compression and local fusion through at least one convolutional layer with a stride of 1, yielding the output features:

[0085] in:

[0086] In this embodiment, the convolution is preferably... Convolution, i.e.:

[0087] Therefore, while reducing the spatial size of the feature map, more shallow local details are preserved, making it less likely that the weak texture and fine-grained boundary cues of the early smoke will be directly weakened during the downsampling process.

[0088] Step 5: Construct the frequency domain residual enhancement module like Figure 5 As shown, a frequency domain residual enhancement module is set up in the weak smoke detection network. For ease of description, in this embodiment, the frequency domain residual enhancement module is referred to as the RFBG-Lite module. The frequency domain residual enhancement module includes a spatial branch, a frequency domain branch, and a residual fusion path.

[0089] First, the input features conduct Convolutional compression reduces the channel dimension and decreases the computational overhead in the subsequent frequency domain. Then, a two-dimensional real-valued Fast Fourier Transform is performed on the compressed features.

[0090] For the One channel:

[0091] in, For amplitude spectrum, This is the phase spectrum.

[0092] Since the phase spectrum mainly determines the spatial structure and location of an image or feature, and the amplitude spectrum mainly determines the energy distribution of different frequency components, this embodiment keeps the phase spectrum unchanged and modulates only the amplitude spectrum.

[0093] like Figure 6 As shown, the frequency domain half-spectrum is divided into low-frequency, mid-frequency, and high-frequency regions according to the normalized radial distance. Let the average amplitude of the low-frequency region be... The average amplitude in the mid-frequency region is The average amplitude in the high-frequency region is The overall half-spectrum average amplitude is Construct the channel spectral description vector:

[0094] in:

[0095] It is a numerically stable term.

[0096] Will Inputting a lightweight multilayer perceptron yields the spectral gating coefficients:

[0097] Introducing radial spectrum priors:

[0098] in, Represents the normalized radial frequency. This represents the a priori attenuation bandwidth of the spectrum.

[0099] The modulated amplitude spectrum is as follows:

[0100] Subsequently, the modulated amplitude spectrum With the original phase spectrum Recombined, the frequency domain enhancement features are obtained through inverse two-dimensional real-valued fast Fourier transform:

[0101] Meanwhile, the spatial branch preserves local texture and edge responses through lightweight convolution, outputting spatially enhanced features. .

[0102] Finally, the output is obtained through residual fusion:

[0103] in, and These are learnable channel weights. This structure enables the network to simultaneously utilize local information in the spatial domain and global energy distribution information in the frequency domain, improving its ability to distinguish between smoke and complex backgrounds.

[0104] Step 6: Set up weak supervision to assist in branch splitting like Figure 7 As shown, in order to improve the problem of insufficient constraints of bounding box annotation on diffuse smoke regions, this embodiment sets a weakly supervised auxiliary segmentation branch AuxSeg during the training phase.

[0105] This auxiliary segmentation branch is connected to the P2 high-resolution feature layer and outputs a single-channel smoke response map:

[0106] in, It is a lightweight segmentation prediction head.

[0107] Since the dataset does not provide pixel-level smoke masks, this embodiment constructs weakly supervised region constraints based on the real bounding boxes, resulting in higher responses inside the smoke boxes and lower responses outside the boxes. This constraint does not require obtaining precise segmentation boundaries, but rather enhances the model's learning of the consistency within the smoke regions.

[0108] The total loss function is:

[0109] in, Weights are used to assist in segmentation loss.

[0110] In this embodiment, the AuxSeg weights adopt a warm-up strategy, that is, a small weight is set in the early stage of training and gradually increased as training progresses, so as to avoid the auxiliary segmentation branch interfering with the main detection task before the model has stabilized and converged.

[0111] Step 7: Set NWD Small Goal Regression Auxiliary Constraints like Figure 7 As shown, the training-phase auxiliary supervision structure also includes NWD (Non-Wide Object Decision Making) small-object regression auxiliary constraints. For small-scale smoke targets, even a small pixel shift in the predicted bounding box can cause a significant change in IoU, resulting in fluctuations in the bounding box regression loss. Therefore, this embodiment introduces NWD as an auxiliary regression constraint.

[0112] Predicted box and real frame They are modeled as two-dimensional Gaussian distributions. and The Wasserstein distance between the two is:

[0113] Normalized similarity is:

[0114] The corresponding loss is:

[0115] The bounding box regression loss is:

[0116] In this embodiment, to avoid affecting the normal regression of medium and large targets, regression is only performed when the actual bounding box area is less than the area threshold. When to enable NWD: when hour:

[0117] when hour:

[0118] In this embodiment, Set to 1024 pixels squared. Set to 0.5. Set to 6.0.

[0119] The above parameters are the specific settings in this embodiment. Those skilled in the art can make adaptive adjustments based on the input image resolution, smoke target scale distribution, and model training status.

[0120] Step 8: Model Training like Figure 1 As shown, after completing the construction of the network structure and the training-period auxiliary supervision mechanism, the smoke detection network is trained.

[0121] The model training in this embodiment is implemented using the PyTorch deep learning framework. The training hardware environment uses an Intel Xeon Silver 4210 processor and an NVIDIA Quadro RTX 5000 GPU.

[0122] In this embodiment, the training parameters are set as follows: Input image size: 640 × 640; Optimizer: AdamW; Initial learning rate: 0.001; Final learning rate factor: 0.005; Weight decay: 0.0008; Training epochs: 300; Warmup: 5 epochs; Batch size: 16; Data loading threads: 8; Learning rate scheduling: Cosine annealing; Early stopping patience: 30 epochs; Training strategies: rectangular training, multi-scale training, SyncBN.

[0123] During training, COCO format evaluation metrics were used, including AP, AP50, AP75, APs, and ARs. Among them, APs and ARs were used to specifically evaluate the detection capability of small-scale smoke targets.

[0124] Step 9: Model Inference and Result Output like Figure 8 As shown, after training is complete, the training-phase auxiliary segmentation branch and NWD auxiliary supervision computation are removed or turned off, leaving only the main detection network for inference.

[0125] The system inputs real-time images captured by the drone into a trained weak smoke detection network. The model outputs smoke detection bounding boxes, category confidence scores, and location information. When the detection confidence score exceeds a preset threshold, the system outputs the detection results to the early warning platform.

[0126] The output includes images with labeled smoke detection boxes, smoke target categories and confidence levels, coordinates of smoke target bounding boxes, video stream detection results, and early fire warning signals.

[0127] In one implementation, the trained model is deployed to an edge computing platform, an unmanned aerial vehicle (UAV) computing device, or a ground server for real-time or near-real-time forest fire smoke detection.

[0128] Step 10: Implementation Results On the drone smoke test set, the method described in this embodiment achieved detection results of AP 0.434, AP50 0.825, AP75 0.393, APs 0.291, and ARs 0.433. Compared with models such as YOLOv8n, YOLOv9t, YOLOv10n, YOLOv11n, YOLOv8s, and RT-DETR-L, this embodiment achieves superior results in AP, AP50, AP75, and APs.

[0129] Regarding false positives and false negatives, compared to the YOLOv8n baseline model, this embodiment reduces the bounding box level false positive rate from 0.2230 to 0.1792 and the bounding box level false negative rate from 0.3129 to 0.2307; the image level false positive rate is reduced from 0.0406 to 0.0221 and the image level false negative rate is reduced from 0.2826 to 0.1969.

[0130] In terms of computational complexity, the model in this embodiment has 3.473M parameters and 19.232 GFLOPs of computation. Compared with RT-DETR-L's 30.964M parameters and 107.992 GFLOPs, this embodiment is more suitable for the actual deployment needs of UAV forest fire early monitoring.

[0131] In summary, this invention can improve the accuracy and stability of early forest fire smoke detection in complex natural environments, reduce the risk of missed detection of small-scale weak smoke targets, and improve the practicality of UAV forest fire early warning systems without significantly increasing the computational overhead of the inference stage.

Claims

1. A drone-based early forest fire smoke detection system, characterized in that, The system includes: The image acquisition module is used to acquire forest scene images collected by the drone; The preprocessing module is used to perform resizing, normalization, and data augmentation on the forest scene image; The smoke detection network module is used to extract features and detect smoke targets in the preprocessed forest scene image, and output the location and confidence score of the smoke targets. The high-resolution feature preservation module, including the P2 high-resolution detection branch and the Space-to-Depth downsampling structure, is used to preserve shallow weak smoke details during feature extraction and feature fusion in the smoke detection network module. The frequency domain residual enhancement module is used to perform a two-dimensional real-valued fast Fourier transform on at least one layer of feature map, keeping the phase spectrum unchanged, and modulating the amplitude spectrum based on the radial frequency band statistics of the amplitude spectrum and the spectral prior to obtain the frequency domain enhancement features for smoke detection. The training supervision enhancement module is used to introduce weakly supervised auxiliary segmentation constraints and normalized Gaussian Wasserstein distance constraints during the model training phase to improve the consistency of response in smoke regions and the stability of localization of small-scale smoke targets. The detection output module is used to output the location and confidence level of the smoke target based on the multi-scale features after high-resolution preservation and frequency domain enhancement.

2. The system according to claim 1, characterized in that, The P2 high-resolution detection branch corresponds to the 1 / 4 scale feature map of the input image, which is used to enhance the detection capability of small-scale, low-contrast and weak-texture smoke targets.

3. The system according to claim 1, characterized in that, The Space-to-Depth downsampling structure is used to rearrange the local spatial neighborhood information of the input feature map to the channel dimension, resulting in rearranged features with halved spatial size and increased number of channels. The rearranged features are then locally fused and channel adjusted through at least one convolutional layer to obtain the output features.

4. A method for detecting early forest fire smoke using drones, characterized in that, Includes the following steps: S1, acquire forest scene images collected by the drone, and preprocess the forest scene images; S2, input the preprocessed forest scene image into the smoke detection network, and extract multi-level image features through the backbone network; S3, through the high-resolution feature preservation module, the shallow smoke features are preserved and enhanced to obtain multi-scale features containing high-resolution weak smoke information; S4, At least one layer of feature map is enhanced in the frequency domain by the frequency domain residual enhancement module to obtain frequency domain enhanced features; S5 uses a detection head to predict the multi-scale features after high-resolution preservation and frequency domain enhancement, and outputs the location and confidence level of the smoke target.

5. The method according to claim 4, characterized in that, The high-resolution feature preservation module includes a P2 high-resolution detection branch and a Space-to-Depth downsampling structure; The P2 high-resolution detection branch corresponds to the 1 / 4 scale feature map of the input image, which is used to enhance the detection capability of small-scale early smoke targets; The Space-to-Depth downsampling structure is used to rearrange the local spatial information of the input feature map to the channel dimension, and to perform local fusion and channel adjustment on the rearranged features.

6. The method according to claim 4, characterized in that, The frequency domain residual enhancement module includes a spatial branch, a frequency domain branch, and a residual fusion branch; The spatial branches are used to preserve local spatial details; The frequency domain branch is used to perform frequency domain transformation on the feature map and modulate the amplitude spectrum of the frequency domain features; The residual fusion branch is used to fuse the original input features, spatial enhancement features, and frequency domain enhancement features.

7. The method according to claim 6, characterized in that, The frequency domain branching includes the following steps: Adjust the channel dimension of the input feature map; The adjusted feature map is subjected to a two-dimensional real-valued fast Fourier transform to obtain complex frequency domain features; The complex frequency domain features are decomposed into amplitude spectrum and phase spectrum; Channel gating weights are generated based on radial frequency band statistics of the amplitude spectrum; Spectral priors are generated based on normalized radial frequency coordinates; The amplitude spectrum is modulated using the channel gating weights and the spectral priors; The modulated amplitude spectrum and the original phase spectrum are reconstructed in the frequency domain, and the frequency domain enhancement features are obtained by inverse fast Fourier transform. The radial frequency band statistics include the average amplitude in the low-frequency region, the average amplitude in the mid-frequency region, the spectral attenuation index, and the low-frequency concentration index. The spectral attenuation index is determined based on the average amplitude in the low-frequency region and the average amplitude in the high-frequency region, and the low-frequency concentration index is determined based on the average amplitude in the low-frequency region and the overall average amplitude of the spectrum.

8. The method according to claim 4, characterized in that, A training supervision enhancement mechanism is introduced during the training phase, which includes weakly supervised auxiliary splitting branch and normalized Gaussian Wasserstein distance constraint. The weakly supervised auxiliary segmentation branch is connected to the P2 high-resolution feature layer and generates a region consistency constraint based on the smoke target box annotation, which is used to make the internal region of the smoke target box have a higher response than the external region of the smoke target box. The weakly supervised auxiliary segmentation branch is used only during the training phase and is removed or not involved in the computation during the inference phase. The normalized Gaussian Wasserstein distance constraint models the predicted box and the ground truth box as two-dimensional Gaussian distributions respectively, and constructs the localization loss based on the Wasserstein distance between the two two-dimensional Gaussian distributions; The normalized Gaussian Wasserstein distance constraint is only enabled for smoke target boxes with an area smaller than a preset pixel area threshold.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method according to any one of claims 4 to 8.

Images