Unmanned aerial vehicle forest fire detection method and system fusing environmental physical prior

Abstract

The application discloses a UAV forest fire detection method and system fusing environmental physical priori, and the detection method comprises the following steps: acquiring a UAV image frame and synchronous data parameters, converting a wind direction in a natural coordinate system into a projected diffusion direction in an image coordinate system according to a camera internal parameter and a posture parameter; calculating a physical confidence, and adjusting the fusion strength of a physical attention mask and a coordinate attention weight by using the physical confidence or a smoothing gating coefficient generated from the physical confidence; acquiring a target imaging scale to adaptively adjust a multi-hole rate feature branch weight; in a training stage, a wind direction coupling form adaptive loss function is constructed.The application further discloses a UAV forest fire detection system fusing environmental physical priori.The application can enhance smoke diffusion direction perception when the environmental priori is reliable, and can degrade into a visual detection mode when the environmental priori is unreliable, so that the detection stability in a strong wind, high flight height and irregular smoke scene is improved, and fire warning linkage is supported.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, drone inspection and disaster early warning technology, and in particular to a drone forest fire detection method and system that integrates prior knowledge of environmental physics. Background Technology

[0002] In recent years, with the popularization of drone inspections in forest fire prevention, urban fire fighting and other fields, how to use airborne visual data to achieve early detection of fires has become a key research direction. However, most existing methods are directly applied to general target detection models, which mainly face the following challenges: (1) The image formats of public datasets are different, there are damaged files and mismatched annotations, which affects the stability of training; (2) Relying only on image data, it is not possible to use the prior knowledge of the environment such as wind speed and wind direction measured by drones, which leads to serious misjudgment of the direction of smoke diffusion under wind interference; (3) The impact of the change of drone flight altitude on the target scale is not considered, and the feature extraction of the fixed receptive field cannot adapt to the drastic scale change from 50m to 500m; (4) General feature extraction and loss functions are difficult to cope with the diffuseness and irregular shape of smoke targets. High transparency leads to missed detections and inaccurate positioning; and the loss function has a fixed penalty for the shape of irregular smoke targets, resulting in low positioning accuracy; (5) Existing pure vision large models usually rely on expanding the network size and increasing the feature dimension to improve accuracy, which has a high computational cost and power consumption, making it difficult to perform low-latency real-time inference on UAV airborne edge devices with limited computing power; (6) There is a lack of a complete closed-loop evaluation process from training to verification to visualization testing, making it difficult to objectively measure and deliver model performance; (7) Most current systems are limited to the detection stage and cannot effectively integrate UAV inspection and fire extinguishing devices to achieve early warning linkage.

[0003] To address the aforementioned problems, this invention urgently needs to design a drone-based forest fire detection technology that integrates prior environmental physical knowledge, including a systematic solution encompassing data construction, model optimization, robustness verification, and early warning linkage, in order to solve at least one of the aforementioned technical issues. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned shortcomings of the prior art and provide a method and system for detecting forest fires by drones that integrates environmental physical priors, so as to solve the problem of insufficient detection stability of the prior art in strong wind, high flight altitude and irregular smoke scenarios, thereby improving the robustness and edge deployment adaptability of drone forest fire detection.

[0005] The technical solution of this invention is:

[0006] One of the present inventions is a drone-based forest fire detection method that integrates prior environmental physical knowledge, comprising the following steps:

[0007] S1: Acquire image information collected by the UAV and data parameters corresponding to the image information, including wind speed, wind direction, UAV relative altitude and attitude parameters; synchronize the image information and the data parameters according to the timestamp, and construct training samples based on the valid images and the synchronized data parameters;

[0008] S2: Using the training samples, construct and train a fire target detection network. The construction method includes:

[0009] A. Perform preliminary processing on the acquired valid image information to obtain candidate smoke regions; the first frame uses the visual candidate region of the current image information, and subsequent frames use the union of the visual candidate region of the current image information and the outer expansion region of the smoke detection box of the previous frame;

[0010] B. Obtain the current layer feature map extracted by the backbone network from the effective image information; convert the measured wind direction angle in the natural coordinate system into the projected diffusion direction in the image coordinate system according to the camera intrinsic parameters and the pose parameters, and generate a physical attention mask according to the projected diffusion direction and the wind speed; extract the optical flow direction in the candidate smoke region, calculate the physical confidence based on the time synchronization error between the image information and sensor data, the variance of environmental parameters, and the consistency between the optical flow direction and the projected diffusion direction, adjust the fusion strength of the physical attention mask and the coordinate attention weights using the physical confidence or the smoothing gating coefficient generated therefrom, recalibrate the current layer feature map, and output the recalibrated feature map;

[0011] C. Obtain the axial distance from the camera to the target reference plane, calculate the target imaging scale based on the camera focal length and category reference size; generate the basic weights of multiple dilated convolution branches based on the target imaging scale, and use a residual weight network with the relative flight altitude of the UAV as input to correct the basic weights, and output multi-scale fusion features;

[0012] D. Using the features output by the physical information fusion coordinate attention module and the imaging scale-driven multi-scale feature enhancement network, the prediction box parameters are obtained through the detection network prediction; the center deviation of the prediction box is projected onto the projection diffusion direction and its normal direction respectively, and the parallel direction error and normal direction error are differentially weighted using the physical confidence, and the shape penalty term is adjusted in combination with the deviation of the real box shape to construct a wind direction coupled shape adaptive loss function; the fire target detection network is optimized and trained using the loss function as the training objective;

[0013] S3: Using the trained fire target detection network, the performance is verified using an independent validation set and a hierarchical test subset divided according to wind speed, UAV relative flight altitude and smoke morphology deviation. The verified target detection model and its corresponding model parameters are output.

[0014] Further, S1 specifically includes: validating the acquired raw image information and isolating image information that cannot be opened or decoded normally; for image information that passes the verification, aligning the image frame timestamp with the sensor timestamp of the data parameters, calculating the mean and variance of environmental parameters within the interpolation window formed by adjacent sensor samples, and associating the mean and variance of the environmental parameters with the corresponding image information; enhancing the associated image information, and dividing the enhanced data into a training set and a validation set to form the training samples.

[0015] Furthermore, S2-B specifically includes the following steps:

[0016] B1: Generate candidate smoke regions using a lightweight visual candidate region generation unit that does not receive environmental parameters;

[0017] B2: Perform global average pooling on the input feature map along the height and width directions respectively to obtain the height direction feature vector and the width direction feature vector; after concatenation, convolution dimensionality reduction, activation and segmentation, the height direction feature vector and the width direction feature vector are used to generate the initial height direction attention weight and the initial width direction attention weight.

[0018] B3: Based on the attitude parameters, the measured wind direction angle in the natural coordinate system is converted into the projected diffusion direction in the image coordinate system, and a physical attention mask is generated based on the projected diffusion direction and wind speed; and the optical flow direction is extracted in the candidate smoke region to obtain the physical confidence level, and a smoothing gating coefficient is generated from the physical confidence level.

[0019] B4: Pool the physical attention mask along the height and width directions to obtain the height physical weights and width physical weights; use the smoothing gating coefficients generated by the physical confidence to adjust the fusion strength of the height physical weights and the initial height attention weights, and the fusion strength of the width physical weights and the initial width attention weights, respectively.

[0020] B5: Multiply the input feature map element-wise with the fused height and width attention weights to output the recalibrated feature map.

[0021] Furthermore, the physical confidence level is obtained using the following formula:

[0022] ;

[0023] In the formula, Indicates physical confidence level. This is the time difference between the image frame timestamp and the sensor timestamp. The variance of environmental parameters within the interpolation window; The variance decay scale; The time decay scale; The direction of optical flow in the candidate smoke region; The projection diffusion direction in the image coordinate system;

[0024] Set the physical branch closure threshold and physical branch fully enabled threshold According to the physical confidence level and , The relative relationship between them generates smooth gating coefficients with values ​​between 0 and 1. :

[0025] ;

[0026] when Not greater than Close the physical branch when lie in and The physical branch contribution is linearly adjusted between the two. Not less than When fully enabling physical branches.

[0027] Furthermore, S2-C specifically includes the following steps:

[0028] C1: Define the gimbal pitch angle as the angle between the camera optical axis and the vertical downward direction. Calculate the axial distance from the camera to the target reference plane based on the UAV's relative flight altitude, and calculate the target imaging scale based on the camera focal length and category reference size.

[0029] C2: Generate basic weights based on the matching degree between the equivalent receptive fields of multiple dilated convolution branches and the target desired receptive field, wherein the target desired receptive field is calculated by the target imaging scale; add the logarithm of the basic weights to the output of the residual weight network with the relative flight altitude of the UAV as input and then normalize the result to obtain the fusion weights of each dilated convolution branch.

[0030] C3: The feature maps output by each dilated convolution branch are weighted and summed according to the fusion weights to obtain multi-scale fusion features.

[0031] Further, S2-D specifically includes the following steps: constructing a parallel unit vector and a normal unit vector orthogonal to it based on the projection diffusion direction; projecting the deviation between the center of the predicted box and the center of the real box onto the parallel unit vector and the normal unit vector respectively to obtain the parallel direction error and the normal direction error; calculating a basic visual distance term independent of environmental parameters based on the center distance between the predicted box and the real box; differentially weighting the parallel direction error and the normal direction error using the smoothing gating coefficient, and generating a shape deviation adjustment factor based on the absolute logarithm of the aspect ratio of the real box to dynamically adjust the shape penalty term of the loss function; when environmental data is missing or the physical confidence is lower than a preset threshold, the smoothing gating coefficient decreases, the orientation localization additional term decays, and the basic visual distance term still participates in regression optimization.

[0032] Furthermore, it also includes a real-time drone inspection and firefighting linkage step based on the target detection model:

[0033] (1) Load the verified target detection model output in step S3;

[0034] (2) Image information and data parameters are acquired in real time through the UAV’s onboard acquisition equipment, and the image information is preprocessed by scaling and normalization;

[0035] (3) Input the preprocessed image information into the target detection model, and after processing by the target detection model, output the detection results of smoke and / or flame targets. The detection results include prediction box parameters, category and confidence information.

[0036] (4) When smoke and / or flame targets are detected, a comprehensive early warning judgment is made based on the relationship between the target confidence level and the preset threshold, combined with the current environmental parameters;

[0037] (5) When the warning is triggered, the following linkage operations will be executed:

[0038] a. Generate early warning signals and push them synchronously to the monitoring system via communication links;

[0039] b. Issue a linkage control command to the matching fire extinguishing device. The control command includes target location information and action command. After receiving the command, the fire extinguishing device performs the fire extinguishing operation.

[0040] (6) Store the detection results, early warning records and linkage operation logs.

[0041] One of the inventions is an unmanned aerial vehicle (UAV) forest fire detection system that integrates prior knowledge of environmental physics, comprising:

[0042] The data engineering and management subsystem is used to acquire image information collected by the UAV and the data parameters corresponding to the image information, synchronize the image information and the data parameters according to the timestamp, and construct training samples based on the valid images and the synchronized data parameters.

[0043] The model training and optimization subsystem includes a physical information fusion coordinate attention module, an imaging scale-driven multi-scale feature enhancement network, and a wind direction coupled morphological adaptive loss construction module, which are used to execute step S2 in the UAV forest fire detection method according to claim 1, so as to construct and train the fire target detection network.

[0044] The model validation and delivery subsystem is used to validate the performance of the trained fire target detection network using an independent validation set and a hierarchical test subset divided according to wind speed, relative flight altitude and smoke pattern deviation, and output the validated target detection model and the corresponding model parameters.

[0045] The UAV real-time inspection and early warning linkage subsystem is used to load a validated target detection model, perform frame-by-frame detection on the image information collected by the UAV in real time, and trigger an early warning when smoke and / or flames are detected and send control commands to the linkage fire extinguishing device.

[0046] Furthermore, the physical information fusion coordinate attention module includes:

[0047] A lightweight visual candidate region generation unit is used to perform preliminary processing on the current image information and output candidate smoke regions; the lightweight visual candidate region generation unit does not receive environmental parameters;

[0048] The wind direction projection conversion unit is used to convert the measured wind direction angle in the natural coordinate system into the projected diffusion direction in the image coordinate system based on the camera intrinsic parameters and attitude parameters.

[0049] The physical confidence calculation unit is used to extract the optical flow direction within the candidate smoke area and calculate the physical confidence based on the time synchronization error between the image frame and the sensor, the variance of environmental parameters, and the consistency between the optical flow direction and the projection diffusion direction.

[0050] The gated fusion unit is used to adjust the fusion strength of the physical attention mask and the coordinate attention weight using the physical confidence or the smooth gating coefficient generated therefrom, to recalibrate the current feature map output by the backbone network, and output the recalibrated feature map.

[0051] Furthermore, the imaging scale-driven multi-scale feature enhancement network includes:

[0052] The imaging scale calculation unit is used to calculate the target imaging scale based on the UAV's relative flight altitude and the gimbal's pitch angle.

[0053] The multi-scale feature fusion unit is used to generate the basic weights of multiple dilated convolution branches according to the target imaging scale, and to correct the basic weights using a residual weight network with the relative flight altitude as input. The unit performs multi-scale feature fusion on the multi-scale feature maps of different layers output by the backbone network and outputs multi-scale fused features.

[0054] The beneficial effects of this invention are:

[0055] 1. It has achieved full automation of the entire process from data to model to early warning linkage. More importantly, the trained model can automatically identify smoke and flames from drone video streams and output detection results, while simultaneously linking with early warning systems and fire extinguishing devices to achieve seamless integration of intelligent early fire monitoring and emergency response processes.

[0056] 2. Through automated data cleaning and organization, combined with Mosaic enhancement, the quality and diversity of training data are guaranteed from the source, ensuring data quality and process reproducibility; and the relevant images basically cover various forms of smoke in different scenarios, ensuring that the model fully learns the diverse features of smoke.

[0057] 3. By introducing environmental physical priors such as wind speed and direction through the PI-CA module, the wind direction in the natural coordinate system is projected onto the image coordinate system. Physical confidence is generated by combining the consistency between the optical flow direction of the candidate smoke region and the projected diffusion direction. This physical confidence is then used to generate smoothing gating coefficients, adaptively adjusting the fusion strength of the physical attention mask and coordinate attention weights. When the environmental priors are reliable, this module enhances the feature response in the smoke diffusion direction, improving the model's ability to perceive targets drifting with the wind. When the environmental priors are unreliable, the physical branch automatically decays, and the model degenerates into a pure visual detection mode, avoiding attention shifts caused by erroneous physical priors. Therefore, the PI-CA module maintains stable feature extraction capabilities under different environmental conditions, significantly improving the accuracy of smoke detection under complex meteorological conditions such as strong winds.

[0058] 4. The MAFE-Net module estimates the target's imaging scale in the image in real time based on the UAV's relative flight altitude and gimbal pitch angle, and adaptively adjusts the fusion weights of multiple dilated convolution branches accordingly. This allows the network to automatically select a receptive field combination that matches the current target scale at different flight altitudes, overcoming the limitation of traditional fixed receptive field structures that are difficult to adapt to drastic changes in target scale. It can maintain stable multi-scale feature representation capabilities within a wide flight altitude range of 50m to 500m, and does not rely on large-scale network structures, effectively reducing computational redundancy.

[0059] 5. The predicted bounding box center deviation is decomposed into parallel error and normal error along the projection diffusion direction using the WD-SCA-ShapeIoU loss function. Physical confidence is used to differentially weight the errors in these two directions, allowing the network to prioritize constraining the positioning deviation along the wind direction when the environmental prior is reliable. A shape deviation adjustment factor is generated by combining the logarithmic absolute value of the aspect ratio of the true bounding box, dynamically adjusting the strength of the shape penalty term. This allows the loss function to adaptively adjust the shape constraint according to the flat or elongated shape of the target. This loss function, independent of environmental parameters, maintains a stable constraint on the predicted bounding box center position through its basic visual distance term. When environmental parameters are reliable, it forms physical direction enhancement; when environmental parameters are unreliable, the network can still complete regression optimization through the basic visual distance term, thus avoiding sudden changes in detection performance caused by sensor anomalies. This enables accurate bounding box localization under various environmental conditions.

[0060] 6. The PI-CA module injects wind field physical priors during the feature extraction stage to enhance direction awareness. MAFE-Net adaptively adjusts the receptive field based on flight altitude during the multi-scale fusion stage to cover scale variations. The WD-SCA-ShapeIoU loss function introduces wind direction-coupled direction constraints and shape-adaptive penalties during the training and optimization stage. These three modules work synergistically and complement each other: the physical confidence provided by PI-CA provides a reliable basis for the direction-differential weighting of WD-SCA-ShapeIoU; the multi-scale fusion features output by MAFE-Net and the recalibrated features output by PI-CA are combined for WD-SCA-ShapeIoU to calculate the loss. This ensures the entire network maintains high-precision fire target detection performance even in complex scenarios such as strong wind interference, flight altitude changes, and irregular smoke morphology. Furthermore, all three modules are based on a lightweight design with controllable computational overhead, meeting the low-latency requirements of real-time inspections and supporting an integrated fire prevention and control process from image acquisition and intelligent analysis to early warning linkage.

[0061] 7. This invention does not rely on simply expanding the size of the visual model to improve detection accuracy. Instead, it uses low-cost sensor priors such as wind speed, wind direction, flight altitude and attitude to guide attention and multi-scale feature selection. It improves detection robustness with less additional computation and reduces the model's dependence on computing power stacking. It is suitable for real-time inference on UAV onboard edge devices with limited computing power and power consumption, and has strong edge computing adaptability.

[0062] 8. This invention provides a complete system engineering solution from data preparation, model training, evaluation and verification to final deployment. The model can be encapsulated as an API interface to process video streams transmitted by drones in real time and supports linkage with fire extinguishing devices, which lowers the threshold for technology implementation. It has a high degree of systematization and engineering and is practical. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the workflow of the physical information fusion coordinate attention module according to an embodiment of the present invention;

[0064] Figure 2 This is a schematic diagram of an imaging scale-driven multi-scale feature enhancement network according to an embodiment of the present invention;

[0065] Figure 3 This is a schematic diagram of the wind direction coupling morphology adaptive loss function calculation process according to an embodiment of the present invention;

[0066] Figure 4 This is a schematic diagram of the ablation experiment results of an embodiment of the present invention;

[0067] Figure 5 This is a schematic diagram of the PR curve and confusion matrix according to an embodiment of the present invention;

[0068] Figure 6 This is a schematic diagram of typical scene detection results according to an embodiment of the present invention. Detailed Implementation

[0069] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0070] Example 1

[0071] This embodiment provides a drone-based forest fire detection method that integrates prior environmental physical knowledge. It mainly includes three core stages: data engineering, model building and training, and evaluation and verification. The trained model possesses automatic recognition capabilities: after inputting a drone video stream or static image, the model analyzes frame by frame, detecting smoke and flame targets and marking their locations and confidence levels. Subsequently, it triggers early warning signals and fire suppression linkage commands based on pre-set thresholds.

[0072] Phase 1 – Modular Data Engineering and Preprocessing: Acquire image information collected by the UAV and corresponding data parameters, including wind speed, wind direction, UAV relative altitude, and attitude parameters; synchronize the image information and data parameters according to timestamps, and construct training samples based on valid images and synchronized data parameters, as detailed below:

[0073] 1.1 Data Cleaning and Quality Filtering: This embodiment uses an automated script to batch read raw image files and check their integrity. Images that cannot be opened or decoded normally are moved to a backup folder along with their annotation files to isolate corrupted files. A cleaning report is generated, and a cleaning log is recorded. This prevents training interruptions due to individual file corruption, ensuring that all data entering the training set is usable. For example, this embodiment processed 13,570 raw images, including 12,631 pure smoke images and 1,102 mixed smoke and flame images, removing a small number of invalid samples.

[0074] 1.2 Dataset Reconstruction and Enhancement: For the cleaned and valid data, training and validation sets are established according to the object detection dataset format, organized into a standardized directory structure, and corresponding configuration files are generated for easy model reading later. Considering the significant variations in shooting distance during drone inspections and the obvious scale differences of smoke in the image, this embodiment introduces a Mosaic enhancement strategy. This strategy randomly combines multiple images from different scenes into a new sample, which is equivalent to mixing targets of different scales in a training batch, helping to improve the model's adaptability to scale changes.

[0075] 1.3 Data Parameter Synchronization and Association: Align the image frame timestamp with the sensor timestamp of the data parameters, calculate the mean and variance of environmental parameters such as wind speed and wind direction within the interpolation window formed by adjacent sensor samples, and synchronously associate the UAV's relative flight altitude and attitude parameters, and associate the mean, variance, relative flight altitude and attitude parameters of the environmental parameters with the corresponding image frames.

[0076] The correlated data parameters and the preprocessed image frames together form a training sample containing image and synchronization data parameter information.

[0077] In this embodiment, the attitude parameters include the UAV heading angle, gimbal pitch angle, and camera roll angle.

[0078] Phase 2 – Dedicated Model Construction and Optimization Training: Using the training samples constructed in Phase 1, a fire target detection network is constructed and trained. The fire target detection network includes a physical information fusion coordinate attention module (PI-CA), an imaging scale-driven multi-scale feature enhancement network (MAFE-Net), and a wind direction coupled shape adaptive loss function (WD-SCA-ShapeIoU).

[0079] In this embodiment, the fire target detection network can adopt a single-stage target detection network structure, specifically an improved lightweight single-stage target detection network structure. This stage specifically includes the following steps:

[0080] S201: Obtain the current image frame and its corresponding data parameters from the training samples. Use a lightweight visual candidate region generation unit that does not receive environmental parameters to perform preliminary processing on the current image frame to obtain candidate smoke regions. The first frame uses the visual candidate region of the current image frame, and subsequent frames use the union of the visual candidate region of the current image frame and the outer expansion region of the smoke detection box of the previous frame. Based on the camera intrinsic parameters and attitude parameters including heading angle, gimbal pitch angle and camera roll angle, convert the measured wind direction angle in the natural coordinate system into the projected diffusion direction in the image coordinate system. Extract the optical flow direction in the candidate smoke region, and calculate the physical confidence based on the time synchronization error between the image frame and the sensor, the variance of environmental parameters, and the consistency between the optical flow direction and the projected diffusion direction. Use the physical confidence or the smoothing gating coefficient generated therefrom to adjust the fusion strength of the physical attention mask and the coordinate attention weight, and output the recalibrated feature map.

[0081] This step S201 is specifically implemented through the Physical Information Fusion Coordinate Attention Module (PI-CA):

[0082] The Physical Information Fusion Coordinate Attention Module (PI-CA) incorporates projection diffusion direction and wind speed information in the image coordinate system on top of standard coordinate attention. This module first extracts directional attention weights along the height and width directions, then constructs a physical attention mask based on the projection diffusion direction, and finally applies it using physical confidence. Generated smooth gating coefficients The fusion strength of the physical attention mask and the original coordinate attention weights is controlled. Thus, when the environmental prior is reliable, the network enhances the feature responses of areas related to the smoke diffusion direction, while avoiding attention shifts caused by incorrect wind directions when the environmental prior is unreliable.

[0083] like Figure 1 As shown, the workflow of the Physical Information Fusion Coordinate Attention Module (PI-CA) includes the following steps:

[0084] A. Perform global average pooling on the input feature map along the height and width directions respectively to obtain the height direction feature vector and the width direction feature vector; after concatenation, convolution dimensionality reduction, activation and segmentation, the height direction feature vector and the width direction feature vector are used to generate the initial height direction attention weight and the initial width direction attention weight.

[0085] First, global average pooling is performed along the height and width directions of the input feature map to obtain the following directional features:

[0086] ;

[0087] ;

[0088] In the formula, W is the width of the feature map. For the c-th channel, the coordinate position The eigenvalue at the location; H is the height of the feature map.

[0089] The height direction feature vector obtained above With width direction feature vector The concatenation is performed along the channel dimension, then dimensionality is reduced by a 1×1 convolutional layer and transformed using a non-linear activation function to obtain the fused feature f:

[0090] ;

[0091] In the formula, Represents a 1×1 convolution; This is the activation function.

[0092] The fused feature f is split along the channel dimension into and , which correspond to the height and width directions respectively.

[0093] Then, the initial height-oriented attention weights are generated. Attention weights in the initial width direction :

[0094] ;

[0095] ;

[0096] In the formula, and It consists of two feature vectors containing information in the height and width directions. and It consists of two independent 1×1 convolutional layers. This is the Sigmoid activation function.

[0097] B. A lightweight visual candidate region generation unit is used to generate candidate smoke regions. Based on the camera intrinsic parameter matrix K, the UAV heading angle ψ, the gimbal pitch angle η, and the camera roll angle φ, the measured wind direction angle θ in the natural coordinate system is converted into the projected diffusion direction in the image coordinate system. And according to the projection diffusion direction A physical attention mask is generated based on the wind speed v; and the optical flow direction is extracted from the candidate smoke region to obtain the physical confidence level. And generate smooth gating coefficients from the physical confidence level. .

[0098] First, to reduce the impact of asynchronous image frames and environmental sensor data, local wind field fluctuations, or sensor anomalies on physical priors, this embodiment sets up a lightweight visual candidate region generation unit that does not include a physical attention branch. This unit includes a sequentially connected image scaling layer, three depthwise separable convolutional layers, a 1×1 convolutional layer, and a sigmoid activation layer, outputting a smoke saliency probability map. Thresholding segmentation, connected component analysis, and minimum area filtering are performed on the probability map to generate visual candidate regions for the current frame. For the first frame of the video, the current frame's visual candidate region is directly used; for subsequent frames, the union of the current frame's visual candidate region and the region of the smoke detection box from the previous frame expanded by a preset ratio is taken as the current frame's candidate smoke region. This unit does not accept wind speed, wind direction, or flight altitude parameters to avoid forming a circular dependency with the physical attention branch.

[0099] Secondly, based on the camera intrinsic parameter matrix K, the UAV heading angle ψ, the gimbal pitch angle η, and the camera roll angle φ, the measured wind direction angle θ in the natural coordinate system is converted into the projected diffusion direction in the image coordinate system. Let the natural coordinate system adopt the East-North-Sky direction, the camera intrinsic parameter matrix be K, the UAV heading angle be ψ, the gimbal pitch angle be η, and the camera roll angle be φ. The unit wind direction vector in the natural coordinate system is expressed as:

[0100] ;

[0101] in, θ is the unit wind direction vector in the natural coordinate system; θ is the natural wind direction angle measured by the sensor.

[0102] Based on the UAV's heading angle, gimbal pitch angle, and camera roll angle, establish the rotation relationship from the natural coordinate system to the camera coordinate system:

[0103] ;

[0104] ;

[0105] ;

[0106] in, This is the rotation matrix from the natural coordinate system to the camera coordinate system; This is the heading rotation matrix; η is the gimbal pitch and rotation matrix; ψ is the heading angle; η is the gimbal pitch angle.

[0107] Compensated wind direction vector :

[0108] ;

[0109] In the formula, This represents the roll compensation performed around the camera's optical axis in the camera coordinate system, where φ is the camera roll angle. A rotation matrix for the fixed mounting between the camera and the drone body. When and When φ is approximately zero, it is set to zero; otherwise, it is set to the compensated wind direction vector. This formula performs complete roll compensation, where The scroll threshold, This represents the allowable projection direction error.

[0110] Select a reference point in the natural coordinate system and order Projecting two spatial points onto the image plane:

[0111] ;

[0112] Where T is the camera's position vector in the natural coordinate system; s is the positive scale selected along the wind direction; π represents the camera coordinates; π represents the homogeneous coordinate normalized projection. These are pixel coordinates; The direction of smoke diffusion is the projection of the smoke in the image coordinate system. Let be the image plane pixel point projected from the i-th spatial point.

[0113] Then, a polar coordinate system is established with the image center as the origin. The polar radius r and polar angle ϕ(i,j) of each pixel are calculated, and the projection diffusion direction is used as the coordinate system. Replace natural wind direction that has not undergone attitude change The wind direction influence function is generated, and then combined with the wind speed v to generate the wind speed influence function, and finally a physical attention mask is constructed.

[0114] The wind direction influence function It can be obtained through the following formula:

[0115] ;

[0116] The wind speed influence function It can be obtained through the following formula:

[0117] ;

[0118] The physical attention mask It can be obtained through the following formula:

[0119] ;

[0120] Simultaneously, optical flow vectors of adjacent frames are calculated within the candidate smoke region, and the circumferential average direction is obtained using the optical flow amplitude as a weight; then, a direction consistency term is constructed based on the angle between the optical flow vector and the projected diffusion direction to obtain the physical confidence level of visual-wind field consistency.

[0121] ;

[0122] in, Indicates physical confidence level. Image frame timestamp With sensor timestamps Time difference; The variance of environmental parameters within the interpolation window; The variance decay scale; The time decay scale; The direction of optical flow in the candidate smoke region; This represents the projection diffusion direction in the image coordinate system.

[0123] Set the physical branch closure threshold and physical branch fully enabled threshold According to the physical confidence level and , The relative relationship between them generates smooth gating coefficients with values ​​between 0 and 1. :

[0124] ;

[0125] when Not greater than Close the physical branch when lie in and The physical branch contribution is linearly adjusted between the two. Not less than The physical branch is fully activated at that time. Understandably, this is affected by factors such as time lag, environmental fluctuations, or increased visual-wind field direction deviation. Automatically decreases.

[0126] To avoid incorrect physical prior guidance caused by environmental sensor malfunctions or local wind field disturbances, this embodiment does not directly use the measured wind direction as the fixed attention direction. Instead, it generates physical confidence based on the time synchronization error between the image frame and sensor data, the variance of environmental parameters within the interpolation window, and the consistency of optical flow direction in the candidate smoke region. The physical confidence level Used to characterize the consistency between the current environmental physical priors and the motion state of smoke in the image, and used to generate smooth gating coefficients. When physical confidence level At higher levels, the smoothing gate coefficient Increase, enhance the moderating effect of the physical attention branch on visual features; when physical confidence... At a low level, the smoothing gate coefficient Reduce or zero the physical attention branch, or reduce or turn it off, causing the detection network to degenerate into a detection mode that primarily relies on visual features.

[0127] C. Pool the physical attention mask along the height and width directions respectively to obtain the physical weights in the height direction. Physical weights in the width direction ;Utilizing physical confidence Generated smooth gating coefficients Adjust the physical weights in the height direction respectively. The fusion strength with the initial height-direction attention weights and the width-direction physical weights The fusion strength with the initial width direction attention weight.

[0128] Specifically, physical attention mask Pooling along the height and width directions yields... and :

[0129] ;

[0130] Physical weight in the height direction Physical weights in the width direction Fusion with initial coordinate attention weights:

[0131] ;

[0132] ;

[0133] In the formula, Indicates the smoothing gate coefficient. and is the learnable fusion coefficient. When As the contribution of the physical branch decreases, the contribution of the physical branch also decreases synchronously.

[0134] D. Multiply the input feature map element-wise with the fused height and width attention weights to output the recalibrated feature map.

[0135] Specifically, the final output recalibration features are:

[0136] ;

[0137] in, For the c-th channel, the coordinate position The eigenvalue at that location.

[0138] This module enables the network to sense the physical direction of smoke diffusion, reducing false detections caused by wind interference.

[0139] S202: Define the gimbal pitch angle as the angle between the camera optical axis and the vertical downward direction. Calculate the axial distance from the camera to the target reference plane based on the UAV's relative flight altitude, and calculate the target imaging scale based on the camera focal length and category reference size. Generate the base weights of multiple dilated convolution branches based on the target imaging scale, and use a residual weight network with the UAV's relative flight altitude as input to correct the base weights, thereby obtaining multi-scale fusion features.

[0140] This step S202 is specifically implemented through an imaging scale-driven multi-scale feature enhancement network (MAFE-Net):

[0141] The Imaging Scale Driven Multi-Scale Feature Enhancement Network (MAFE-Net) estimates the target's imaging scale in the image using the UAV's relative flight altitude and gimbal pitch angle, and adjusts the fusion weights of multiple dilated convolutional branches based on this imaging scale. The category reference size L is statistically obtained from the target bounding boxes, simultaneous shooting distance, and camera focal length in the training set, and is not the actual target size known beforehand during the detection phase. Compared to fixed-weight multi-scale feature fusion methods, this module increases the contribution of the large receptive field branch in high flight altitude scenes and increases the contribution of the small receptive field and detail branches in low flight altitude scenes.

[0142] To address the issue of drastic changes in target scale caused by variations in UAV altitude, this embodiment designs an imaging scale-driven multi-scale feature enhancement network (MAFE-Net). The gimbal pitch angle η is defined as the angle between the camera optical axis and the vertically downward direction. When the target reference plane is approximately horizontal, the axial distance from the camera to the target reference plane is... Based on the pinhole imaging relationship, the projection scale of the category reference size L in the image is: Where f is the camera focal length, The relative flight altitude of the drone is L, and the category reference size is obtained from the training set. In other words, L is the category reference size obtained from the target bounding boxes, synchronous shooting distance, and camera focal length in the training set, not the actual target size known beforehand during detection. For example... Figure 2 As shown: For the four parallel convolution branches with dilation rates of 1, 2, 3, and 5, the four parallel convolution branches with dilation rates of 1, 2, 3, and 5 respectively perform dilated convolution on the input feature map and output feature maps F1, F2, F3, and F4 respectively.

[0143] Based on the equivalent receptive field and the target desired receptive field of each dilated convolution branch The degree of matching generates the base weights, wherein the target desired receptive field It is determined by the target imaging scale Calculation yields ( κ is a calibration coefficient used to unify the pixel scale and the receptive field scale. Then... Relative flight altitude of drones The input is a two-layer fully connected residual weighted network. After summing and normalizing with Softmax, the final fusion weights of each dilated convolutional branch are obtained. .

[0144] The design uses an imaging model to provide an interpretable base assignment, and a learning network is used to correct for camera calibration errors, terrain undulations, and differences in target reference size. For the i-th target in the training samples, the assignment is based on the pixel width of its bounding box. Pixel height Synchronous shooting distance Equivalent reference size for calculating camera focal length f And take category c The median is used to obtain the category reference size. These are obtained using the following formulas;

[0145] ;

[0146] ;

[0147] During detection, the corresponding candidate regions are selected or weighted based on their category probabilities. Therefore, there is no need to know the true nature of the target in advance.

[0148] Actual dimensions.

[0149] The outputs of each branch are fused according to the following formula to obtain the multi-scale fused feature F:

[0150] ;

[0151] The channels are then adjusted using 1×1 convolutions. This module can be inserted into each layer of the Bidirectional Feature Pyramid Network (BiFPN), increasing the weight of the hole rate branch (expanding the receptive field) at high altitudes and emphasizing the detail branch at low altitudes, thus enabling adaptive matching of target scale at different flight altitudes.

[0152] Compared to PANet's unidirectional fusion, this embodiment adds cross-scale skip connections and repeated fusion blocks, which improves the model's ability to capture features of multi-scale targets such as smoke and flames without significantly increasing the computational load, laying the foundation for subsequent lightweight deployment.

[0153] S203: Based on the recalibrated feature map in step S201 and the multi-scale fusion feature prediction in step S202, the prediction box parameters are obtained; the center deviation of the prediction box is projected onto the projection diffusion direction and its normal direction respectively; the parallel direction error and normal direction error are differentially weighted using the physical confidence level; and the shape penalty term is adjusted in combination with the deviation of the real box shape to construct the wind direction coupled shape adaptive loss function (WD-SCA-ShapeIoU).

[0154] This step S203 is specifically implemented through the wind direction coupled shape adaptive loss function (WD-SCA-ShapeIoU):

[0155] The Wind Direction Coupled Shape Adaptive Loss Function (WD-SCA-ShapeIoU) decomposes the deviation between the predicted box center and the true box center into a parallel error along the projected diffusion direction and a normal error perpendicular to the projected diffusion direction, and then bases it on the physical confidence level. Generated smooth gating coefficients Additional option for adjusting orientation and positioning.

[0156] Furthermore, the loss function calculates the shape deviation based on the aspect ratio of the true bounding box to adaptively adjust the shape penalty term. This is useful when environmental parameters are missing or physical confidence is low. At a low level, the smoothing gate coefficient As the orientation positioning addition decreases, it automatically decays, while the basic visual distance term, which does not depend on environmental parameters, still participates in regression optimization.

[0157] To more accurately locate smoke targets that spread with the wind and have irregular boundaries, this invention constructs a wind-direction coupled morphological adaptive loss based on Shape-IoU, specifically including the following steps:

[0158] A. Based on the projection diffusion direction Construct a parallel unit vector and a normal unit vector orthogonal to it; project the deviation between the predicted box center and the ground truth box center onto the parallel unit vector and the normal unit vector respectively to obtain the parallel direction error. Error with normal direction .

[0159] Specifically, let Δc be the deviation between the center of the predicted bounding box and the center of the true bounding box, and let the unit vector of the projection diffusion direction be... Its normal unit vector is Indicates the center deviation along the diffusion direction. This indicates the center deviation in the vertical diffusion direction.

[0160] B. Calculate the basic visual distance term, independent of environmental parameters, based on the center distance between the predicted bounding box and the ground truth bounding box; utilize a smoothing gating coefficient. Regarding the parallel direction error and normal direction error Differential weighting is applied, and a shape deviation adjustment factor is generated based on the absolute logarithm of the aspect ratio of the actual frame to dynamically adjust the Shape-IoU shape penalty term; when environmental data is missing or physical confidence is low... When the threshold is below, the smoothing gating coefficient The directional positioning additional term is reduced, but the basic visual distance term still participates in the regression optimization.

[0161] Specifically, utilizing The positioning penalty is applied in two directions, and the Shape-IoU penalty term is adjusted using the deviation of the actual bounding box shape, such as... Figure 3 As shown:

[0162] First, calculate the relationship between the predicted bounding box B and the ground truth bounding box. Intersection over Union (IoU):

[0163] ;

[0164] Based on the width of the actual box and height Calculate the orientation weighting coefficients, where scale is a scaling factor related to the target size:

[0165] ;

[0166] ;

[0167] Introducing shape distance To measure the offset of the center point, where c is the diagonal length of the minimum bounding box:

[0168]

[0169] In the formula, The x-coordinate of the center point of the prediction box; The ordinate of the center point of the prediction box; The x-coordinate of the center point of the true bounding box; The ordinate is the center point of the true bounding box.

[0170] Calculate the shape difference weights between the predicted bounding box and the ground truth bounding box in the width and height directions respectively:

[0171] ;

[0172] This leads to the shape penalty term. Where 𝜃=4 is the adjustment factor:

[0173] ;

[0174] Then, based on the width w of the actual frame gt and height h gt Calculate the morphological deviation metric:

[0175] ;

[0176] Generate adaptive adjustment factor :

[0177] ;

[0178] The final loss function is defined as:

[0179] ;

[0180] in, The loss value is represented by D, which represents the basic visual distance term independent of environmental parameters. , Indicates the smoothing gate coefficient. Indicates the error in the parallel direction. Indicates the error in the normal direction. These are the error weights for the parallel direction and the normal direction, respectively. This represents the shape penalty term, and ≥ ; At higher levels, the model strengthens directional constraints consistent with the real wind field, in addition to the basic visual distance term. At lower values, only the physical direction addition is attenuated, while the basic visual distance term D is always retained; μ is adaptively adjusted based on the deviation of the true bounding box shape. Therefore, the projection diffusion direction and physical confidence used in PI-CA simultaneously participate in feature enhancement and bounding box regression. The wind direction coupled with morphological adaptive loss function in this embodiment is optimized and trained on the bounding box regression task of the fire target detection network to ensure that the position and shape of the predicted boxes output by the network closely approximate real smoke and / or flame targets.

[0181] This embodiment constructs a complete fire target detection network through the aforementioned steps S201-S203. Based on this, this embodiment trains the fire target detection network using the PyTorch framework, with the main hyperparameter settings as follows: images in the training samples constructed in the first stage are uniformly scaled to 1280×1280 resolution, trained for 100 epochs, using a stochastic gradient descent optimizer with momentum, an initial learning rate of 0.01, a momentum coefficient of 0.93, and the learning rate decays according to a cosine function as the training progresses. The trained model can automatically identify smoke and flames from input images.

[0182] The third stage – systematic evaluation, validation and deployment: using an independent validation set and a hierarchical test subset divided according to wind speed, relative flight altitude and smoke morphology deviation, the performance of the trained fire target detection network is validated, and the validated target detection model and corresponding model parameters are output.

[0183] 3.1 Multi-level performance evaluation:

[0184] (1) Lightweight backbone network comparison experiment: In order to verify the impact of different lightweight backbone networks on the computational cost and detection accuracy of the model, this embodiment embeds three lightweight backbone networks, GhostNet, ShuffleNetV2 and MobileNetV3, into the fire target detection network constructed in this invention for comparison experiment.

[0185] In one specific embodiment, an improved lightweight single-stage object detection network structure is adopted, with MobileNetV3 as the backbone network. MobileNetV3 reduces computational cost through depthwise separable convolutions and enhances feature representation by combining SE attention mechanism. Experimental results show that after embedding this backbone network, the model's FLOPs are reduced to 2.3G while maintaining high detection accuracy, making it suitable for embedded deployment requirements of UAVs.

[0186] In one specific embodiment, an improved lightweight single-stage object detection network structure is adopted, with GhostNet as the backbone network. GhostNet generates basic features through ordinary convolutions, and then uses inexpensive operations to generate redundant features to reduce computational cost. Experimental results show that after embedding this backbone network, the model's FLOPs are reduced to 8.0G, and mAP@0.5 reaches 81.1%, achieving a good balance between accuracy and lightweight design.

[0187] In one specific embodiment, ShuffleNetV2 is used as the backbone network. The YOLOv12 baseline model has a FLOPs of 15.8 G and an mAP@0.5 of 80.90%. After embedding ShuffleNetV2, the model's FLOPs decrease to 1.8 G, which is 11.4% of the baseline, while the mAP@0.5 remains at 80.70%. Based on this, and with the addition of data augmentation strategies, the model's FLOPs reach 1.9 G, and the mAP@0.5 increases to 82.50%.

[0188] The comparative experimental results above show that all three lightweight backbone networks can reduce the computational load of the model to varying degrees. Among them, ShuffleNetV2 has the most significant effect in terms of lightweighting. When combined with data augmentation, it can improve accuracy while reducing the computational load, which is beneficial for adapting to the embedded deployment requirements of UAVs.

[0189] (2) Ablation experiment: To verify the effectiveness of the core improved module of the present invention, an ablation experiment was designed in this embodiment. The experimental results are as follows: Figure 4 As shown.

[0190] After embedding the Physical Information Fusion Coordinate Attention (PI-CA) module separately, the overall mAP@0.5 was improved by 0.0392. This result shows that the Physical Information Fusion Coordinate Attention mechanism can enhance the model's ability to capture spatial features of irregular, blurred targets such as smoke.

[0191] Using the wind-coupled shape adaptive loss function (WD-SCA-ShapeIoU) alone improved the overall mAP@0.5 by 0.0266. This result demonstrates that shape-aware loss can improve the bounding box regression accuracy for irregular targets.

[0192] When the physical information fusion coordinate attention module is used in combination with the wind direction coupling morphological adaptive loss function, the overall mAP@0.5 reaches 0.7581, which meets the delivery acceptance threshold of ≥0.7 set in this embodiment.

[0193] The above results demonstrate the effectiveness of the two improvements proposed in this embodiment. They significantly improve the model detection accuracy while maintaining high real-time performance, even with only an increase of 0.8ms inference time, thus meeting the needs of real-time UAV inspection.

[0194] In addition to commonly used metrics such as mAP@0.5, this embodiment also statistically analyzed the precision and recall rates for the smoke and flame categories, and plotted PR curves and confusion matrices (e.g., ...). Figure 5 As shown in the figure, this example visually demonstrates the model's performance and class misclassification at different confidence thresholds. This embodiment uses commonly used metrics in object detection, such as precision, recall, and mAP@0.5, for quantitative evaluation. Based on the test statistics, Figure 5 Among the operating points indicated by the PR curves, the precision for fire targets was 0.9432 and the recall was 0.8985, indicating that the model has high accuracy and detection stability for fire targets. The precision for smoke targets was 0.6667 and the recall was 0.4497, indicating that there are still some missed detections of smoke targets due to their diffuse boundaries, high transparency, and strong background similarity, but this is an improvement compared to the baseline model. Figure 5 The PR curve is used to reflect the relationship between precision and recall under different confidence thresholds; the normalized confusion matrix is ​​used to reflect the classification confusion between smoke, flame and background under the selected fixed threshold.

[0195] (3) Simulated stress test: This embodiment also developed an offline log analysis module to statistically analyze the target detection rate, size sensitivity, and inference time, and generate a statistical analysis report to verify the stability and robustness of the model in the actual deployment environment. Specifically, the simulated stress test uses offline playback of UAV inspection logs and test set samples, and divides the test scenarios according to different wind speed levels, relative flight altitude ranges, and smoke morphology deviations. The target detection rate, false detection rate, average inference time, and mAP of the high flight altitude scenario are statistically analyzed for each scenario to evaluate the stability of the model under strong wind, high flight altitude, and irregular smoke conditions.

[0196] 3.2 Visualization Result Validation and Model Delivery:

[0197] Load the pre-trained, optimal model, perform inference on the test set images, and plot the detection results directly onto the original image, providing intuitive performance proof. Figure 6 The model's detection results in typical scenarios are shown to demonstrate that the model can correctly select smoke and / or flame targets in complex backgrounds and provide corresponding confidence scores, thereby verifying the model's visualization detection effect on actual UAV inspection images.

[0198] Based on core metrics such as mAP@0.5, an acceptance threshold is set, and a text verification report containing all key metrics and conclusions is automatically generated to complete the standardized delivery of the model.

[0199] 3.3 System Integration and Deployment:

[0200] After the model passes acceptance testing, this embodiment exports it into a general inference format (ONNX or TorchScript) and encapsulates it as an API service. This service receives video streams / image sequences transmitted from the drone in real time, analyzes them frame by frame, and automatically pushes early warning information when smoke or flames are detected, and sends linkage commands to the corresponding fire extinguishing devices.

[0201] The following is a preferred embodiment of the UAV forest fire detection method of the present invention:

[0202] 1. Data Preparation Stage: This embodiment processes 13,570 original images, including 12,631 pure smoke images and 1,102 mixed smoke and flame images. A data cleaning script is run to remove damaged images, generate a cleaning report, and record the cleaning log. Then, Mosaic data augmentation is used to expand the sample, ultimately forming a standardized dataset and generating a data.yaml configuration file.

[0203] 2. Model Training Phase: In the configured PyTorch environment, an improved deep convolutional neural network is built. The improvements in this embodiment include the following three parts:

[0204] (1) Embedded Physical Information Fusion Coordinate Attention Module (PI-CA). A PI-CA module is inserted after a specific layer of the backbone network. This module receives not only the feature map but also the wind speed v and wind direction angle θ (in radians) measured by the UAV. Taking an input feature map of size 2×64×128×128 as an example, the output shape of the PI-CA module remains unchanged, and the computational overhead increases by less than 5%. In the specific implementation, σ=π / 4, α=2.0, and the learnable coefficient α when fusing the physical weights with the initial attention weights is set. h a w Initialized to 0.5. Image frames and environmental sensor data are aligned according to timestamps; in this embodiment, interpolation windows are constructed using adjacent sensor data, based on the time difference Δt and window variance. and the direction of optical flow With the direction of projection diffusion Consistency calculation of physical confidence level Then by Generate smooth gating coefficients and will As a fusion coefficient of the PI-CA physical branch. When sensor data is missing or When the value is below a preset threshold, the weights of the physical branches decay, and the model primarily relies on visual features for detection. In one optional embodiment, the number of output channels of the three depthwise separable convolutional layers in the candidate region generation unit are 16, 32, and 64, respectively. The input image is scaled to 320×320, the saliency probability threshold is 0.35, the minimum connected component area is 0.1% of the image area, and the expansion ratio of the detection box in the previous frame is 1.2; the roll threshold... Take 3°, allowing for projection direction error. Set the value to 2°; Physical branch closure threshold. Set the threshold to 0.25, the threshold for fully enabling physical branches. Take 0.60. The above values ​​are determined by the camera calibration results and the independent validation set, and can be adjusted according to the sensor sampling frequency, camera installation error, and flight platform vibration level.

[0205] (2) Introduce an imaging scale-driven multi-scale feature enhancement network (MAFE-Net). An imaging scale-driven multi-scale feature enhancement network (MAFE-Net) is connected after each layer of the bidirectional feature pyramid network (BiFPN), with four dilated convolution branches and dilation rates of 1, 2, 3, and 5. The relative flight altitude of the UAV is then... A two-layer fully connected network (16-dimensional hidden layers) is input, and branch weights are generated. During testing, a 2×64×64×64 feature map and random altitude are input; the output shape remains unchanged, while the weight shape is 2×4, demonstrating that the weights dynamically change with altitude. This embodiment calculates equivalent reference dimensions for smoke and flame categories based on training set bounding boxes, synchronized flight altitude, gimbal pitch angle, and camera focal length. and adopt each category The median is used as the category reference size. During the inference phase, the category probabilities of candidate regions are used to weight the reference sizes of smoke and flame.

[0206] (3) The wind-coupled shape adaptive loss function (WD-SCA-ShapeIoU) is adopted. During the training phase, the wind-coupled shape adaptive loss function is used as the bounding box regression loss to replace the CIoU loss in the original target detection network, and the predicted box localization error is adaptively constrained according to the projection diffusion direction, smoothing gating coefficient, and deviation of the ground truth box shape. This function calculates for each ground truth box. This leads to the adjustment factor μ, and the final loss value is... In the example, the loss value for both sets of predicted boxes and ground truth boxes is 0.3445, which is a stable value.

[0207] Training parameters: input size 1280×1280, batch size 2, total number of rounds 100, initial learning rate 0.01, momentum 0.93, learning rate decays according to a cosine function. After embedding the physical information fusion coordinate attention module and the wind direction coupled morphological adaptive loss function into the fire target detection network, the experiment yielded mAP@0.5=0.7581. The newly added vision-wind field consistency gating, imaging scale driven weights, and wind direction coupled loss used the same training configuration, and their effects in strong wind, high flight altitude, and irregular smoke scenarios were evaluated using a hierarchical ablation scheme.

[0208] 3. Model Validation and Delivery Phase:

[0209] Quantitative evaluation: In addition to calculating the conventional mAP, precision, and recall, a stratified test subset was constructed based on smoke morphology deviation, UAV flight altitude, and wind speed level. Ablation experiments included the YOLOv12 baseline model, standard coordinate attention, PI-CA, PI-CA with visual-wind field consistency gating, imaging scale-driven multi-scale feature enhancement network (MAFE-Net), wind direction coupled morphology adaptive loss function (WD-SCA-ShapeIoU), and the complete combined model; the false positive rate in strong wind scenes, mAP in high flight altitude scenes, IoU of irregular smoke localization, and single-frame inference time were statistically analyzed to verify the technical effects of each distinguishing technical feature and its combination.

[0210] Visualization test: Use the best model to infer the test set and generate a detection result map. The detection box is labeled with confidence and data parameters (wind speed, wind direction, relative flight altitude).

[0211] Acceptance Report: The system automatically generates an acceptance report, confirming that the overall mAP@0.5=0.7581 meets the delivery requirement of ≥0.7.

[0212] 4. System Deployment Phase: Convert the accepted model to ONNX format, encapsulate it into a RESTful interface using FastAPI, and connect it to the drone's video stream / image sequence for real-time detection. Whenever the model detects smoke or flames, the system will trigger an alert and send a linkage command to the fire suppression system. The deployed interface will display the detection footage and alarm records in real time.

[0213] Example 2

[0214] A drone-based forest fire detection system integrating prior environmental physical knowledge includes:

[0215] The data engineering and management subsystem is used to acquire image frames collected by the UAV and the corresponding wind speed, wind direction, relative flight altitude, and attitude parameters of the image frames, synchronize them according to timestamps, and construct training samples based on valid images and synchronized data parameters. The data parameters include environmental parameters (wind speed, wind direction) and flight state parameters (UAV relative flight altitude, attitude parameters).

[0216] The model training and optimization subsystem is used to construct and train a fire target detection network, which includes a physical information fusion coordinate attention module (PI-CA), an imaging scale-driven multi-scale feature enhancement network (MAFE-Net), and a wind direction coupled morphological adaptive loss construction module.

[0217] The model validation and delivery subsystem is used to validate the performance of the trained fire target detection network using an independent validation set and a hierarchical test subset divided according to wind speed, relative flight altitude and smoke pattern deviation, and output the validated target detection model and the corresponding model parameters.

[0218] The UAV real-time inspection and early warning linkage subsystem is used to load a validated target detection model, perform frame-by-frame detection on the video stream / image sequence collected by the UAV in real time, and trigger an early warning when smoke and / or flames are detected and send control commands to the linkage fire extinguishing device.

[0219] Furthermore, the Physical Information Fusion Coordinate Attention Module (PI-CA) includes:

[0220] A lightweight visual candidate region generation unit is used to perform preliminary processing on the current image frame and output candidate smoke regions; the lightweight visual candidate region generation unit does not receive environmental parameters;

[0221] The wind direction projection conversion unit is used to convert the measured wind direction angle in the natural coordinate system into the projected diffusion direction in the image coordinate system based on the camera intrinsic parameters and attitude parameters including heading angle, gimbal pitch angle and camera roll angle.

[0222] The physical confidence calculation unit is used to extract the optical flow direction within the candidate smoke area and calculate the physical confidence based on the time synchronization error between the image frame and the sensor, the variance of environmental parameters, and the consistency between the optical flow direction and the projection diffusion direction.

[0223] The gated fusion unit is used to adjust the fusion strength of the physical attention mask and the coordinate attention weight using the physical confidence or the smooth gating coefficient generated therefrom, to recalibrate the current feature map output by the backbone network, and output the recalibrated feature map.

[0224] Furthermore, the imaging scale-driven multi-scale feature enhancement network (MAFE-Net) includes:

[0225] The imaging scale calculation unit is used to calculate the target imaging scale based on the UAV's relative flight altitude and the gimbal's pitch angle.

[0226] The multi-scale feature fusion unit is used to generate the basic weights of multiple dilated convolution branches according to the target imaging scale, and to correct the basic weights using a residual weight network with the relative flight altitude as input. The unit performs multi-scale feature fusion on the multi-scale feature maps of different layers output by the backbone network and outputs multi-scale fused features.

[0227] Furthermore, the wind direction coupling morphology adaptive loss construction module is used to predict the bounding box parameters based on the recalibrated feature map and the multi-scale fusion feature, project the center deviation of the bounding box onto the projection diffusion direction and its normal direction respectively, use the physical confidence to differentially weight the parallel direction error and the normal direction error, and combine the shape penalty term with the deviation of the true bounding box shape to construct the wind direction coupling morphology adaptive loss function.

[0228] In this embodiment, after the UAV real-time inspection and early warning linkage subsystem loads the final model, it can automatically analyze the UAV video stream. Once smoke or flames are detected, an early warning is triggered, and the fire extinguishing device is linked to perform early fire extinguishing operations. The system acquires inspection video streams (1280×1280 resolution) / image sequences in real time through UAV onboard acquisition equipment (such as visible light / infrared cameras, wind speed / direction sensors, and altimeters). The video streams / image sequences are input into the deployed target detection model for real-time inference, identifying smoke and / or flame targets and outputting detection results. For identified fire targets, the system automatically triggers an early warning and sends control commands to the linked fire extinguishing device, achieving a fully automated closed loop from data acquisition and intelligent identification to emergency response. The specific implementation method includes the following steps:

[0229] (1) Image information and data parameters are acquired in real time through the UAV's onboard acquisition equipment. The data parameters include environmental parameters and flight status parameters. The environmental parameters include wind speed v and wind direction θ. The flight status parameters include relative flight altitude. In addition to the UAV heading angle ψ, gimbal pitch angle η, and camera roll angle φ; the acquired image information is scaled, normalized, preprocessed, and enhanced with Mosaic;

[0230] (2) Load the target detection model that has passed the third stage of verification, input the preprocessed image information into the target detection model, and output the candidate smoke region of the current frame through the lightweight visual candidate region generation unit of the target detection model; wherein, for the first frame, the visual candidate region of the current frame is directly used, and for subsequent frames, the union of the visual candidate region of the current frame and the outer expansion region of the smoke detection box of the previous frame is used.

[0231] (3) Embed the current layer feature map output by the backbone network into the PI-CA module, and input the wind speed v and wind direction θ into the physical branch of the PI-CA module; convert the measured wind direction θ in the natural coordinate system into the projected diffusion direction in the image coordinate system according to the camera intrinsic parameters, as well as the heading angle, gimbal pitch angle and camera roll angle; extract the optical flow direction in the candidate smoke area, calculate the physical confidence based on the time synchronization error, environmental parameter variance and the consistency between the optical flow direction and the projected diffusion direction, and generate the smoothing gate coefficient from the physical confidence; use the smoothing gate coefficient to adjust the fusion strength of the physical attention mask and the coordinate attention weight, recalibrate the current layer feature map, and output the recalibrated feature map;

[0232] (4) Input the multi-scale feature maps output from different layers of the backbone network into the MAFE-Net module, and simultaneously input the relative flight altitude. Input MAFE-Net; based on relative flight altitude The axial distance from the camera to the target reference plane is calculated using the gimbal pitch angle, and the target imaging scale is calculated by combining the camera focal length and category reference size; fusion weights of multiple dilated convolution branches are generated based on the target imaging scale, and the multi-scale feature maps are weighted and fused to output multi-scale fused features;

[0233] (5) The recalibrated feature map is combined with the multi-scale fusion feature and then input into the detection head. The detection head predicts and outputs the bounding box parameters, category and confidence of smoke and / or flame targets.

[0234] (6) When smoke and / or flame targets are detected, a comprehensive early warning judgment is made based on the relationship between the target confidence level and the preset threshold, combined with the current environmental parameters. For example, when the target confidence level is greater than or equal to the preset threshold and the current physical confidence level is not lower than the preset threshold, it is judged as a valid fire warning; when the target confidence level is greater than or equal to the preset threshold but the current physical confidence level is lower than the preset threshold, it is marked as an early warning to be confirmed and manually reviewed by the ground station.

[0235] (7) When the warning is triggered, the following linkage operations will be performed:

[0236] a. Generate early warning signals, including the target category, location information, confidence level, target image and timestamp, etc., and push them synchronously to the ground station monitoring platform, mobile APP and on-site sound and light alarm system through the communication link;

[0237] b. Issue a linkage control command to the supporting fire extinguishing device. The control command includes target location information and action instructions. After receiving the command, the fire extinguishing device performs the fire extinguishing operation. The fire extinguishing device may be a drone carrying fire extinguishing bombs or a ground fire extinguishing unit.

[0238] (8) The ground station monitoring center displays the monitoring screen, early warning information and fire extinguishing device status in real time. At the same time, it stores the detection results, early warning records, linkage operation logs and video streams to form a complete inspection and disposal data chain for post-event traceability and analysis.

[0239] In summary, this invention enhances the feature response along the smoke diffusion direction through the PI-CA module when the environmental prior is reliable, and automatically degrades to a pure visual detection mode when the environmental prior is unreliable, thereby improving the detection stability in complex scenarios such as strong winds, high flight altitudes, and irregular smoke. By projecting the wind direction from the natural coordinate system to the image coordinate system and generating physical confidence based on the consistency between the optical flow direction of the candidate smoke region and the projected diffusion direction, the environmental physical prior adaptively participates in feature recalibration and localization regression according to its reliability. The MAFE-Net module estimates the target imaging scale based on the UAV's relative flight altitude and gimbal pitch angle, and adaptively adjusts the fusion weights of multi-scale feature branches to improve the model's adaptability to drastic scale changes. The WD-SCA-ShapeIoU loss function decomposes the predicted box center deviation into parallel error and normal error along the projected diffusion direction, uses physical confidence for differentiated weighting, and adaptively adjusts the shape penalty term in conjunction with the deviation of the real box shape, thereby improving the localization accuracy of irregular smoke targets. The above modules work together to effectively support the real-time inference deployment of UAVs and fire early warning linkage.

Claims

1. A method for detecting forest fires using unmanned aerial vehicles (UAVs) that integrates prior knowledge of environmental physics, characterized in that, Includes the following steps: S1: Acquire image information collected by the UAV and data parameters corresponding to the image information, including wind speed, wind direction, UAV relative altitude and attitude parameters; The image information and data parameters are synchronized according to the timestamp, and training samples are constructed based on the valid images and the synchronized data parameters. S2: Using the training samples, construct and train a fire target detection network, including: A. Perform preliminary processing on the acquired valid image information to obtain candidate smoke regions; the first frame uses the visual candidate region of the current image information, and subsequent frames use the union of the visual candidate region of the current image information and the outer expansion region of the smoke detection box of the previous frame; B. Obtain the current layer feature map extracted by the backbone network from the effective image information; convert the measured wind direction angle in the natural coordinate system into the projected diffusion direction in the image coordinate system according to the camera intrinsic parameters and the pose parameters, and generate a physical attention mask according to the projected diffusion direction and the wind speed; extract the optical flow direction in the candidate smoke region, calculate the physical confidence based on the time synchronization error between the image information and sensor data, the variance of environmental parameters, and the consistency between the optical flow direction and the projected diffusion direction, adjust the fusion strength of the physical attention mask and the coordinate attention weights using the physical confidence or the smoothing gating coefficient generated therefrom, recalibrate the current layer feature map, and output the recalibrated feature map; C. Obtain the axial distance from the camera to the target reference plane, calculate the target imaging scale based on the camera focal length and category reference size; generate the basic weights of multiple dilated convolution branches based on the target imaging scale, and use a residual weight network with the relative flight altitude of the UAV as input to correct the basic weights, and output multi-scale fusion features; D. Using the features output by the physical information fusion coordinate attention module and the imaging scale-driven multi-scale feature enhancement network, the prediction box parameters are obtained through the detection network prediction; the center deviation of the prediction box is projected onto the projection diffusion direction and its normal direction respectively, and the parallel direction error and normal direction error are differentially weighted using the physical confidence, and the shape penalty term is adjusted in combination with the deviation of the real box shape to construct a wind direction coupled shape adaptive loss function; the fire target detection network is optimized and trained using the loss function as the training objective; S3: Using the trained fire target detection network, the performance is verified using an independent validation set and a hierarchical test subset divided according to wind speed, UAV relative flight altitude and smoke morphology deviation. The verified target detection model and its corresponding model parameters are output.

2. The UAV forest fire detection method based on environmental physical priors as described in claim 1, characterized in that, S1 specifically includes: validating the acquired raw image information and isolating image information that cannot be opened or decoded normally; for image information that passes the verification, aligning the image frame timestamp with the sensor timestamp of the data parameters, calculating the mean and variance of environmental parameters within the interpolation window formed by adjacent sensor samples, and associating the mean and variance of the environmental parameters with the corresponding image information; enhancing the associated image information, and dividing the enhanced data into a training set and a validation set to form the training samples.

3. The UAV forest fire detection method based on environmental physical priors as described in claim 1, characterized in that, S2-B specifically includes the following steps: B1: Generate candidate smoke regions using a lightweight visual candidate region generation unit that does not receive environmental parameters; B2: Perform global average pooling on the input feature map along the height and width directions respectively to obtain the height direction feature vector and the width direction feature vector; after concatenation, convolution dimensionality reduction, activation and segmentation, the height direction feature vector and the width direction feature vector are used to generate the initial height direction attention weight and the initial width direction attention weight. B3: Based on the attitude parameters, the measured wind direction angle in the natural coordinate system is converted into the projected diffusion direction in the image coordinate system, and a physical attention mask is generated based on the projected diffusion direction and wind speed; and the optical flow direction is extracted in the candidate smoke region to obtain the physical confidence level, and a smoothing gating coefficient is generated from the physical confidence level. B4: Pool the physical attention mask along the height and width directions to obtain the height physical weights and width physical weights; use the smoothing gating coefficients generated by the physical confidence to adjust the fusion strength of the height physical weights and the initial height attention weights, and the fusion strength of the width physical weights and the initial width attention weights, respectively. B5: Multiply the input feature map element-wise with the fused height and width attention weights to output the recalibrated feature map.

4. The UAV forest fire detection method based on environmental physical priors according to claim 3, characterized in that, The physical confidence level is obtained using the following formula: ； In the formula, Indicates physical confidence level. This is the time difference between the image frame timestamp and the sensor timestamp. The variance of environmental parameters within the interpolation window; The variance decay scale; The time decay scale; The direction of optical flow in the candidate smoke region; The projection diffusion direction in the image coordinate system; Set the physical branch closure threshold and physical branch fully enabled threshold According to the physical confidence level and , The relative relationship between them generates smooth gating coefficients with values ​​between 0 and 1. : ； when Not greater than Close the physical branch when lie in and The physical branch contribution is linearly adjusted between the two. Not less than When fully enabling physical branches.

5. The UAV forest fire detection method based on environmental physical priors as described in claim 1, characterized in that, S2-C specifically includes the following steps: C1: Define the gimbal pitch angle as the angle between the camera optical axis and the vertical downward direction. Calculate the axial distance from the camera to the target reference plane based on the UAV's relative flight altitude, and calculate the target imaging scale based on the camera focal length and category reference size. C2: Generate basic weights based on the matching degree between the equivalent receptive fields of multiple dilated convolution branches and the target desired receptive field, wherein the target desired receptive field is calculated by the target imaging scale; add the logarithm of the basic weights to the output of the residual weight network with the relative flight altitude of the UAV as input and then normalize the result to obtain the fusion weights of each dilated convolution branch. C3: The feature maps output by each dilated convolution branch are weighted and summed according to the fusion weights to obtain multi-scale fusion features.

6. The UAV forest fire detection method based on fusion of environmental physical priors as described in claim 1, characterized in that, The S2-D algorithm specifically includes the following steps: constructing a parallel unit vector and a normal unit vector orthogonal to it based on the projection diffusion direction; projecting the deviation between the center of the predicted box and the center of the real box onto the parallel unit vector and the normal unit vector respectively to obtain the parallel direction error and the normal direction error; calculating a basic visual distance term independent of environmental parameters based on the center distance between the predicted box and the real box; differentially weighting the parallel direction error and the normal direction error using the smoothing gating coefficient, and generating a shape deviation adjustment factor based on the absolute logarithm of the aspect ratio of the real box to dynamically adjust the shape penalty term of the loss function; when environmental data is missing or the physical confidence is lower than a preset threshold, the smoothing gating coefficient decreases, the orientation localization additional term decays, and the basic visual distance term still participates in regression optimization.

7. The UAV forest fire detection method based on environmental physical priors as described in claim 1, characterized in that, It also includes a real-time drone inspection and firefighting linkage step based on the target detection model: (1) Load the verified target detection model output in step S3; (2) Image information and data parameters are acquired in real time through the UAV’s onboard acquisition equipment, and the image information is preprocessed by scaling and normalization; (3) Input the preprocessed image information into the target detection model, and after processing by the target detection model, output the detection results of smoke and / or flame targets. The detection results include prediction box parameters, category and confidence information. (4) When smoke and / or flame targets are detected, a comprehensive early warning judgment is made based on the relationship between the target confidence level and the preset threshold, combined with the current environmental parameters; (5) When the warning is triggered, the following linkage operations will be executed: a. Generate early warning signals and push them synchronously to the monitoring system via communication links; b. Issue a linkage control command to the matching fire extinguishing device. The control command includes target location information and action command. After receiving the command, the fire extinguishing device performs the fire extinguishing operation. (6) Store the detection results, early warning records and linkage operation logs.

8. A UAV forest fire detection system integrating prior environmental physics, characterized in that, include: The data engineering and management subsystem is used to acquire image information collected by the UAV and the data parameters corresponding to the image information, synchronize the image information and the data parameters according to the timestamp, and construct training samples based on the valid images and the synchronized data parameters. The model training and optimization subsystem includes a physical information fusion coordinate attention module, an imaging scale-driven multi-scale feature enhancement network, and a wind direction coupled morphological adaptive loss construction module, which are used to execute step S2 in the UAV forest fire detection method according to claim 1, so as to construct and train the fire target detection network. The model validation and delivery subsystem is used to validate the performance of the trained fire target detection network using an independent validation set and a hierarchical test subset divided according to wind speed, relative flight altitude and smoke pattern deviation, and output the validated target detection model and the corresponding model parameters. The UAV real-time inspection and early warning linkage subsystem is used to load a validated target detection model, perform frame-by-frame detection on the image information collected by the UAV in real time, and trigger an early warning when smoke and / or flames are detected and send control commands to the linkage fire extinguishing device.

9. The UAV forest fire detection system integrating environmental physical priors as described in claim 8, characterized in that, The physical information fusion coordinate attention module includes: A lightweight visual candidate region generation unit is used to perform preliminary processing on the current image information and output candidate smoke regions; the lightweight visual candidate region generation unit does not receive environmental parameters; The wind direction projection conversion unit is used to convert the measured wind direction angle in the natural coordinate system into the projected diffusion direction in the image coordinate system based on the camera intrinsic parameters and attitude parameters. The physical confidence calculation unit is used to extract the optical flow direction within the candidate smoke area and calculate the physical confidence based on the time synchronization error between the image frame and the sensor, the variance of environmental parameters, and the consistency between the optical flow direction and the projection diffusion direction. The gated fusion unit is used to adjust the fusion strength of the physical attention mask and the coordinate attention weight using the physical confidence or the smooth gating coefficient generated therefrom, to recalibrate the current feature map output by the backbone network, and output the recalibrated feature map.

10. The UAV forest fire detection system based on integrated environmental physical priors as described in claim 8 or 9, characterized in that, The imaging scale-driven multi-scale feature enhancement network includes: The imaging scale calculation unit is used to calculate the target imaging scale based on the UAV's relative flight altitude and the gimbal's pitch angle. The multi-scale feature fusion unit is used to generate the basic weights of multiple dilated convolution branches according to the target imaging scale, and to correct the basic weights using a residual weight network with the relative flight altitude as input. The unit performs multi-scale feature fusion on the multi-scale feature maps of different layers output by the backbone network and outputs multi-scale fused features.

Images