A multi-modal lightweight forest fire detection method suitable for a vertical take-off and landing fixed-wing unmanned aerial vehicle platform

Abstract

The application discloses a kind of multi-modal lightweight forest fire detection methods suitable for vertical take-off and landing fixed wing unmanned aerial vehicle platform, comprising: using multi-modal early fusion method to the visible light image and infrared image of unmanned aerial vehicle synchronous collection are carried out feature alignment and weighted fusion, obtain fusion feature;Fusion feature is carried out lightweight downsampling and multi-scale feature extraction, obtain multi-scale feature set;Using bidirectional feature pyramid network to multi-scale feature set is carried out cross-scale feature fusion, and extract dynamic change information, obtain time series feature;Using the spatial pyramid method based on hollow convolution to time series feature is carried out multi-scale context enhancement and utilizes lightweight Transform module to carry out global context and local texture modeling, obtain high-level semantic feature;Based on high-level semantic feature, through multi-task output head parallel generation target detection frame, fire grade classification, predict uncertainty estimation and pixel-level fire probability graph.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and fire safety technology, and in particular relates to a multimodal lightweight forest fire detection method applicable to vertical take-off and landing fixed-wing unmanned aerial vehicle platforms. Background Technology

[0002] Currently, early detection of forest fires mainly relies on visible light RGB cameras and IR infrared cameras mounted on quadcopter drones, using traditional target detection algorithms based on architectures such as YOLO and SSD. This approach has the following shortcomings: 1. Traditional quadcopter platforms have short endurance and small operating radius, making them unsuitable for continuous fire monitoring tasks in large-scale forest areas; 2. Existing detection algorithms are mostly designed for single-frame RGB images. When drones are patrolling at night, weak fires or light smoke with low contrast may be missed. Alternatively, they may only use IR high-temperature points for detection, but thermal cameras have low resolution and large interference from ground reflections, making it easy to miss small fires when detecting them at a distance. It is also difficult to distinguish between high-temperature non-burning objects and initial flames. 3. Existing RGB and IR image fusion methods mostly adopt a multi-branch fusion structure, which leads to the lack of sharing of early features and the loss of light smoke or edge information; at the same time, the parameters of the dual-branch structure double, making it difficult to compress the overall model to the range that can be deployed on edge SoCs. 4. Currently, many mainstream models lack the ability to model visual targets like smoke, which are low-contrast, blurry in shape, and slow in diffusion. Because smoke appears close to the background in grayscale in RGB images and has a blurred temperature transition in IR images, it cannot be extracted by simple brightness or color thresholds. Traditional detection methods often rely on the bright or high-temperature areas of flames, and are poor at recognizing features with blurred diffusion boundaries. They ignore the temporal and shape changes of smoke, which is especially evident in long-distance aerial photography, leading to missed detections and false alarms.

[0003] 5. Furthermore, fire detection is not merely about detection; it also requires providing high-level semantic information such as semantic judgment of fire severity and assessment of hazard level. However, existing neural networks generally only provide bounding boxes and confidence scores, lacking the ability to classify fires, output fire source area masks, and measure uncertainty, making it difficult to meet the needs of real-time response, risk management, and task scheduling.

[0004] 6. Current models are mostly large-scale convolutional networks, which are complex in structure and have high inference resources, making them difficult to deploy on SoCs mounted on VTOL flight platforms. The inference process generally involves pushing video streams to the server via the communication module before inference. Some detection models, such as YOLOv5m or dual-branch Transformer-Net, have many parameters, and even after using INT8 quantization, they still have high power consumption and high memory usage. In practice, when deployed on edge SoC platforms, the recognition frame rate is low, which is difficult to meet the needs of real-time application scenarios.

[0005] Therefore, this invention provides a multimodal lightweight forest fire detection method suitable for VTOL UAVs. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes a multimodal lightweight forest fire detection method applicable to vertical take-off and landing fixed-wing unmanned aerial vehicle platforms, thereby resolving the issues present in the prior art.

[0007] To achieve the above objectives, the present invention provides a multimodal lightweight forest fire detection method suitable for vertical take-off and landing fixed-wing unmanned aerial vehicle platforms, comprising: Visible light and infrared images are acquired synchronously by a drone. A multimodal early fusion method is used to perform feature alignment and weighted fusion on the visible light and infrared images to obtain fused features. The fused features are subjected to lightweight downsampling and multi-scale feature extraction to obtain a multi-scale feature set; A bidirectional feature pyramid network is used to perform cross-scale feature fusion on the multi-scale feature set and extract dynamic change information to obtain temporal features; The temporal features are enhanced with multi-scale context using a spatial pyramid method based on dilated convolution to obtain context-enhanced features. The lightweight Transformer module is used to perform global context and local texture modeling on the context enhancement features to obtain high-level semantic features; Based on the aforementioned high-level semantic features, target detection boxes, fire level classifications, prediction uncertainty estimates, and pixel-level fire probability maps are generated in parallel through multi-task output heads.

[0008] Compared with the prior art, the present invention has the following advantages and technical effects: This invention first significantly reduces model complexity and computational overhead through multimodal early fusion and lightweight network design, enabling efficient deployment on edge computing devices mounted on VTOL UAVs to meet the real-time requirements of forest fire monitoring. Secondly, by combining bidirectional feature pyramids and temporal modeling, it effectively enhances the feature extraction capability for low-contrast targets such as weak fires and light smoke, and leverages their dynamic characteristics to significantly improve detection accuracy and robustness in nighttime and complex scenarios. Furthermore, through the synergistic effect of multi-scale context enhancement and a lightweight Transformer, it strengthens the perception of subtle features such as smoke edges and obscured flames while maintaining model lightweightness. Finally, the multi-task parallel output mechanism not only provides fire source location information but also simultaneously outputs fire severity level, uncertainty metric, and pixel-level heatmaps, providing more comprehensive and refined data support for fire assessment and emergency decision-making. Attached Figure Description

[0009] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a structural diagram of the backbone according to an embodiment of the present invention; Figure 2 This is a structural diagram of the Neck according to an embodiment of the present invention; Figure 3 This is a diagram of the Heads structure according to an embodiment of the present invention; Figure 4 This is a technical flowchart of an embodiment of the present invention. Detailed Implementation

[0010] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0011] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0012] like Figure 1 As shown, the Backbone performs downsampling and pyramid construction (Rep-DownBlock, SimAM, GeM, DW / PW), outputting P3-48 / P4-96 / P5-96. Simultaneously, P3 and P4 are weighted and horizontally fused to WAddP3 / P4. For example... Figure 2As shown, the Neck is upsampled from top to bottom using FPN and then fused laterally with weighted P4 and P3. After enhancement by CBAM, TSM-Lite, Residual-ASPP, and 1×1 Fuse, the resulting features are sent to TinyViT. Figure 3 As shown, these are the Heads, which generate three task heads—detection, grading, and uncertainty—above TinyViT B1 / B2, and output a Mask32 mask from B2 via a lightweight branch that performs adaptive average pooling, channel compression, and scale smoothing.

[0013] like Figure 4 As shown, this embodiment provides a multimodal lightweight forest fire detection method suitable for vertical take-off and landing fixed-wing UAV platforms. This method effectively improves the detection capability of fire sources and smoke in complex forest environments through network structure optimization, multimodal adaptation, temporal modeling, and multi-task output, and adapts to the real-time deployment requirements of low-power edge devices. Specifically, it includes the following steps: Step 1: Multimodal Early Fusion Architecture (Early Fusion + WeightGate): Acquire visible light and infrared images simultaneously captured by the UAV. Employ a multimodal early fusion method to align and weight the features of the visible light and infrared images, resulting in fused features. This process includes: extracting shallow features from the visible light and infrared images using convolutional layers with shared weights, obtaining shallow visible light and shallow infrared features; generating weighted gating coefficients using fully connected layers and activation functions based on the global statistics of the shallow visible light and shallow infrared features; and finally, weighting and summing the shallow visible light and shallow infrared features using these weighted gating coefficients to obtain the fused features.

[0014] First, the RGB and infrared (IR) six-channel images acquired by the airborne dual sensors are input and convolved using a 1×1 method, and then learnable gating weights are introduced. During the daytime, Automatically reduces heat to avoid false alarms due to high temperatures on vegetation. This is especially important at night or when using low-heat lighting. This enhances and strengthens low-contrast fire and smoke signals. Early features are shared, preventing light smoke or thin fire from being weakened in deeper networks, while requiring fewer parameters than traditional dual-branch post-fusion methods.

[0015] Further, (1) Channel alignment and preliminary refinement: The six-channel features of RGB (3ch) and IR (3ch) are concatenated at the pixel level, compressed to 32 channels by 1×1 convolution, and two gate coefficients are generated. The spectral proportion is dynamically adjusted according to the scene brightness, and the weight of IR is automatically increased at night.

[0016] Let the visible light and infrared images simultaneously acquired by the UAV be:

[0017] Two images are convolved with the same set of 3×3 layers to obtain shallow features of equal width:

[0018]

[0019] Where C represents the number of shallow unified channels. , To share weights, For shared bias.

[0020] (2) Generation of weight-gating coefficients (Weight-Gate).

[0021] Global average pooling (squeeze):

[0022]

[0023] Cascade the two:

[0024] SE-style two-layer full connection:

[0025] in, For the dimensionality reduction weight matrix, For the weight matrix of increased dimension, , for , The corresponding bias. Activation function. Compression ratio r=8.

[0026] Softmax normalization yields the gating coefficients:

[0027] in:

[0028]

[0029]

[0030] (3) Weighted early fusion.

[0031] Spatial location of all channels :

[0032] Among them, scalar The effect is broadcast to the entire feature map.

[0033] Step 2, Lightweight Reparameterization (Rep-DownBlock): Based on the fused features, a reparameterizable downsampling module is used for processing to obtain the downsampled feature map. To suppress aliasing and preserve edge details during downsampling without increasing deployment complexity, the Rep-DownBlock module is proposed. During training, three branches are connected in parallel, and during inference, the three branches are reparameterized into a single 3×3, s=2 convolution output. This module can seamlessly replace ordinary downsampling convolutions with almost no change in parameters and computing power, but significantly reduces jagged edges and blur artifacts. It is more effective at handling small flames and fine smoke edges, making it suitable for real-time deployment on edge devices.

[0034] Furthermore, this invention employs a re-parameterizable downsample block (stride=2) in the shallow layers. During the training phase, a multi-branch structure is used to enhance the shallow layer representation; during the inference phase, all branches are equivalently folded into a single 3×3 convolution with a stride of 2, thereby achieving shallow nonlinear enhancement while completing one downsampling operation, ensuring accuracy without increasing inference overhead.

[0035] (1) Training phase (multi-branch): Training input features Rep-DownBlock consists of three parallel branches: 3×3 Conv-BN, with a step size of... Spatial modeling branch, 1×1 Conv-BN, step size The channel hybrid branch also includes a fixed low-pass branch.

[0036]

[0037]

[0038]

[0039] Among them, the low-pass downsampling operator BlurPool fuzzy pooling is used, and its output is multiplied by the channel after BN to learn the gating coefficients. The three branches are added element-wise along the channel dimension, and the activation function... For ReLU, It is the Sigmoid activation function. Learnable scalars are set per channel to achieve better fine-grained gating. This is the final training output.

[0040] BatchNorm(BN) is defined as: for a single output channel Input channel index BN acts on the feature tensor Each spatial location :

[0041]

[0042] Will Substitute into the formula:

[0043]

[0044] For any convolution BN parameters ,in The learned scaling factor. For the learned translation offset, For the running average, Operating variance The constant used for numerical stability is denoted as:

[0045] The convolution after folding can be written as:

[0046] (2) One-time reparameterization of the inference state.

[0047] Channel blending branch: Incorporate the BN of the channel blending branch into the 1×1 convolution weights and biases:

[0048]

[0049] To avoid inconsistent core sizes, the folded 1×1 core Center fill is 3×3:

[0050] Location This indicates that the 3×3 core is located at its center, with the remaining positions filled with 0.

[0051] Fixed low-pass branch: fuzzy pooling branch Treat it as a fixed low-pass kernel for channel-by-channel convolution, then adjust the stride. Sampling, using a fixed 3×3 kernel binomial, yields a Gaussian approximation:

[0052]

[0053] use Given fixed weights for depthwise convolutions, with the kernel size matching that of the inference kernel:

[0054] Incorporate the BN of this branch to obtain the equivalent weights and biases:

[0055]

[0056] Gating can be learned through the passage:

[0057] Kernel and bias accumulation:

[0058]

[0059] Therefore, inference only requires a single 3×3 convolution:

[0060] During the inference phase, the synthesized single 3×3 convolution is equivalent to the output value of the three branches in the training state, while its parameter size, memory access and MAC count are the same as those of a regular 3×3 convolution, thus eliminating the additional inference overhead caused by multiple branches.

[0061] Step 3, Lightweight Attention Module (SimAM+GEM): The downsampled feature map is processed by a zero-parameter attention module and global expectation modulation to obtain a multi-scale feature set.

[0062] The SimAM spatial attention and GEM channel attention mechanisms, which are zero-parameter attention mechanisms, are introduced to enhance the model's ability to recognize edge textures and thermal responses. Both of these modules have no trainable convolutional kernel parameters and zero computational overhead during the inference stage. While maintaining a lightweight structure, the model enhances its ability to focus on key areas such as smoke and hot spots.

[0063] Furthermore, let the input feature tensor First, it is obtained through the zero-parameter attention module SimAM. Then, the output is modulated by the Global Desired Modulation (GEM). The data is then fed into subsequent networks. Neither introduces learnable parameters, and the computational cost is negligible.

[0064] (1) Zero-parameter attention module (SimAM): Mean and variance of the entire graph (calculated independently for each channel):

[0065]

[0066] Energy (standardized measure of deviation from mean state):

[0067] Generate spatial weights and multiply them back into the original features:

[0068]

[0069] in :

[0070] The further away from the mean, such as extremely bright or dark pixels, the more energy... Increase A decrease in the value suppresses these outliers; conversely, areas close to the mean, such as smooth gray fog textures, The value is higher, so it is retained.

[0071] (2) Global Desire Modulation (GEM): Channel statistics:

[0072]

[0073] Zero-parameter normalized weights:

[0074] Channel recalibration:

[0075] When the overall brightness of this channel is high and the variance is small, such as when there is thermal reflection noise, It becomes smaller; when the brightness is moderate and the texture is highly undulating (smoke lines), The number of channels is increased to automatically balance hotspots and background. The features output by the GEM module are expanded from 32 to 48 through 1×1 convolutions to match the input dimension of the subsequent P3 fusion node.

[0076] Step 4: Lightweight Bidirectional Pyramid Network (Slim-BiFPN): Based on a multi-scale feature set, a lightweight bidirectional pyramid network is used to fuse cross-scale features to obtain enhanced multi-scale features. The lightweight bidirectional pyramid network includes a bottom-up path and a top-down path. The process of obtaining enhanced multi-scale features includes: at the fusion nodes of the lightweight bidirectional pyramid network, depthwise separable convolutions are first used to transform each input feature, and then learnable weight coefficients are used to adaptively weight and fuse the transformed features; after completing the adaptive weighted fusion, a dual attention mechanism including channel attention and 5×5 spatial attention is further used at specific intermediate layers to enhance the fused features, resulting in enhanced multi-scale features.

[0077] In the mid-to-high-level feature extraction stage, this invention employs a closed-loop fusion of two paths: bottom-up semantic upconvolution and top-down detail backpropagation. All fusion nodes are normalized and weighted using learnable coefficients wAdd, allowing the network to automatically adjust the importance of different scales. All convolutional layers are replaced with depthwise separable convolutions (DWConv). A 3×3 DWConv layer is added after each upsampling step to suppress aliasing textures, resulting in less overall computation than the traditional PANet.

[0078] To further enhance mesoscale saliency, a CBAM-k5 module is introduced into the FP4 fusion layer. First, SE-style channel weights are used to suppress redundant feature channels, and then 5×5 spatial attention is used to recalibrate the feature map. Compared with conventional 7×7 convolution, the number of parameters and FLOPs of CBAM-k5 are reduced by about half, but its receptive field can still completely cover the edge region of the smoke.

[0079] This design significantly enhances the response at the edge of light smoke while maintaining the global receptive field of dense smoke. Without increasing the width of the main body, both small-scale flames and large-scale smoke plumes can achieve stable and balanced feature expression at the most suitable scale.

[0080] Furthermore, the lightweight bidirectional pyramid, based on the bottom-up P3, P4, and P5, employs depthwise separable convolution (…). The algorithm performs smoothing and sampling, and then normalizes the data using learnable non-negative weighted coefficients for cross-scale fusion. CBAM-k5 (channel attention + 5×5 spatial attention) is injected after the FP4 layer to enhance mesoscale saliency and edge response. The entire module uses only depthwise separable convolutions and a shared MLP, significantly improving the multi-scale separability of smoke and fire targets while maintaining low parameter and MAC costs.

[0081] (1) Depthwise separable convolution: Depthwise separable convolution DWConv: Let the input feature tensor The kernel set of DWConv:

[0082] Set step size ,filling Then the output will be:

[0083]

[0084]

[0085] Each channel is convolved independently, each... Only applies to the first input tensor Each channel is convolved separately from other channels. There is no cross-channel summation; computation is completely split. When the input channels... Equal to output channel equal At that time, DWConv's MAC is that of ordinary convolution. .

[0086] Pointwise convolution PWConv: Let the input features be... Weight bias The output of PWConv is :

[0087] Pointwise convolution PWConv only performs linear combination across channels and does not change the spatial resolution. Its essence is 1×1 convolution. It achieves channel number transformation and linear combination through cross-channel weighting. It is often used for dimensionality increase and decrease or combined with DWConv to form depthwise separable convolution DSConv.

[0088] The computational workload is as follows:

[0089] The computational cost ratio of depthwise separable convolution to ordinary convolution:

[0090] Therefore, DSConv can significantly reduce computational and storage overhead without sacrificing representational power.

[0091] (2) arrive Bottom-up downsampling and channel alignment:

[0092]

[0093]

[0094] Uplink fusion:

[0095]

[0096] This is bilinear upsampling.

[0097] Learnable weighted fusion wAddP4:

[0098]

[0099]

[0100] in A constant used for numerical stability. For unnormalized learnable scalars, For learnable weights, Activation function:

[0101] After wAddP4, FP4 is output:

[0102] (3) CBAM-k5 significantly enhanced: The serial execution channel on FP4 notices spatial attention, highlighting smoke edges and flame highlights without altering the tensor size:

[0103] 1) CBAM-k5 Channel Attention: Global Average Pooling (GAP):

[0104] Global Max Pooling (GMP):

[0105] Shared MLP (two-layer FC, compression ratio r=8):

[0106]

[0107] in Activation function Sigmoid activation after fusion:

[0108] This represents a channel mask of length C, where each dimension is... Inside, indicating the first The strength of channel retention or suppression Activate Sigmoid:

[0109] Weighted original features :

[0110] 2) CBAM-k5 Spatial Attention (kernel k=5): Features converge again along the channel:

[0111]

[0112]

[0113] Sigmoid activation after 5×5 depthwise separable convolution:

[0114] For spatial attention masks, for each spatial location Given a weight, the magnitude is... The inner part indicates the strength of preservation or suppression at that position. This is the Sigmoid activation function.

[0115] Spatial weighting:

[0116] Where ⊙ represents pixel-by-pixel multiplication, the size remains unchanged.

[0117] Output:

[0118] This module performs channel attention and spatial attention sequentially, only recalibrating saliency without introducing size transformations.

[0119] (4) Multi-scale crossover from FP4 to P3:

[0120]

[0121] Upsampling and smoothing:

[0122]

[0123] Learnable weighted fusion wAddP3:

[0124]

[0125]

[0126] Used for numerical stability , During training, the optimal weights for low-level details and uplink semantics are automatically learned. for Activation function.

[0127] The final FP3-48 is then generated via DWConv:

[0128] Step 5, Temporal Modeling Module (TSM-Lite): Based on the specified level feature sequence in the enhanced multi-scale features, the temporal shift method is used to process it to obtain temporal features containing temporal dynamic information.

[0129] The process of obtaining temporal features includes: dividing the feature channel into a forward part, a static part, and a backward part; performing a shift operation on the features of the forward part to the previous frame and a shift operation on the features of the backward part to the next frame, while keeping the features of the static part unchanged, to obtain the shifted feature blocks; and concatenating the shifted feature blocks to obtain temporal features containing temporal dynamic information.

[0130] A channel-level temporal migration strategy is used to perform temporal migration on multi-frame sequences in a zero-parameter manner. The introduction of a lightweight temporal sensing path can enhance the representation capability of continuous dynamic features such as smoke drift and flame flicker.

[0131] Furthermore, let the input time-series feature sequence be... ,in The network here is from FP3-48, channel C=48. TSM-Lite introduces timing awareness by time-shifting a small number of channels, involving only memory movement, zero parameters, and almost zero arithmetic overhead, while also preserving the spatial semantics of most channels.

[0132] (1) Channel segmentation:

[0133]

[0134] Proportioning the feature channels Divided into forward blocks Static blocks Backward block Three segments. And arrange the features in order:

[0135]

[0136] By using channels to perceive temporal information, a clear temporal perception can be obtained without diluting spatial semantics. Specifically, the original 48 channels are split into 6 channels (shifted to the next frame), 36 channels (keeping the current frame), and 6 channels (shifted to the previous frame). These channels are then re-merged in their original order to obtain the complete feature tensor at time t. .

[0137] (2) Timing shift Perform a zero-parameter channel shift on each frame:

[0138] because The next frame is needed; online inference introduces a 1-frame latency, making... Perform boundary frame processing.

[0139] splicing output:

[0140] Then Input the Residual-ASPP branch and use multi-dilation rate dilated convolution to perform multi-scale enhancement on the sparse smoke cues.

[0141] Step 6: Residual-ASPP pooling: The spatial pyramid method based on dilated convolution is used to perform multi-scale context enhancement on the temporal features to obtain context-enhanced features.

[0142] The spatial pyramid method based on dilated convolution is used to perform multi-scale context enhancement on temporal features. The process of obtaining context-enhanced features includes: extracting multi-scale context features in parallel using depthwise separable convolutions with different dilation rates based on temporal features; concatenating the multi-scale context features along the channel dimension and performing channel compression and feature fusion using 1×1 convolution to obtain fused multi-scale features; and weighting and summing the fused multi-scale features with the original temporal features using learnable residual weights to obtain context-enhanced features.

[0143] An ASPP branch is added after the TSM-Lite module. The ASPP module consists of three parallel branches: a 1×1 convolution branch, a 3×3 convolution branch with a dilation rate of 3, and a 3×3 convolution branch with a dilation rate of 6. The three outputs are concatenated along the channel dimension, then subjected to 1×1 convolution for dimensionality reduction and feature fusion, and finally added back to the backbone features as residuals. The dilated convolution used achieves an effective receptive field of 13×13 without increasing the number of parameters, significantly improving the contrast between sparse high-altitude smoke and the background in multi-scale contexts. For residual protection, a learnable residual weight coefficient γ is introduced. When the ASPP has little effect on the flame detection branch, γ automatically decreases, thus avoiding negative impacts on fire source detection accuracy.

[0144] To explicitly transform the channel temporal shifts and differential information generated by TSM-Lite into discriminable spatial responses and multi-scale contexts, the TSM-Lite output... The input is then followed by a dilated convolution branch and fused using residuals. Deeply separable convolutions with varying dilation rates are applied to the input to capture multi-scale cues of background motion features, ranging from thin smoke wisps in small receptive fields to sparse smoke plumes in large receptive fields. The output of the ASPP parallel branch is fused and aligned using 1×1 channel projection / compression, preserving temporally relevant spatial evidence while responding to and amplifying the spatiotemporal features of slowly moving smoke with weak amplitudes across frames, controlling computational load and suppressing noise. A further 1×1 channel compression is applied, followed by back-addition using learnable gated residuals, amplifying temporally relevant spatial evidence without destroying highly significant core features such as the original flame.

[0145] (1) Hollow receptive field and recognition ability: For a Convolution kernel, void ratio Its equivalent receptive field size is:

[0146] One design features a 3×3 kernel with a void ratio d=3 and an equivalent receptive field of 7×7; the other design features a 3×3 kernel with a void ratio d=6 and an equivalent receptive field of 13×13. This design can achieve a receptive field of 128... It covers the diffusion area of ​​thin smoke at a feature resolution of 128 while preserving the local texture structure of the flame, significantly enhancing the recognition ability of small-scale, low-contrast targets.

[0147] (2) Parallel extraction of multi-scale context using three-branch dilated convolution: For input features Three different convolutional methods with varying dilation rates were used for processing:

[0148]

[0149]

[0150] in, Indicates the void ratio, This is a convolution operation.

[0151] (3) Feature splicing + compression: The three outputs are concatenated and then compressed through a 1×1 convolution to obtain the fused features:

[0152] (4) Weighted fusion of residuals: Introduce a learnable residual fusion coefficient Used to control the impact of ASPP branches on the final feature:

[0153] in , For learnable scalars, Using the sigmoid activation function, the network can automatically adjust the ASPP contribution. However, if the hole feature is not conducive to flame localization... It will converge to a smaller value. This avoids weakening the flame positioning effect of the main characteristics.

[0154] The output carries two core types of information: temporal difference features from TSM-Lite, capturing smoke drift and flame flickering with zero parameters; and multi-scale contextual semantics from ASPP, improving the perception of sparse regions and edge information. It will be fed into the TinyViTB1 / B2 module to further construct high-level semantic representations.

[0155] Step 7: Lightweight Transformer, TinyViT Dual-Block Design. A lightweight Transformer module is used to model the global context and local texture of the context-enhanced features, resulting in high-level semantic features.

[0156] The process of using a lightweight Transformer module to model global context and local texture of context-enhanced features to obtain high-level semantic features includes: First, based on the context-enhanced features, a large window partitioning strategy is used to perform window self-attention calculation to capture global context information. Then, a feedforward neural network is used to enhance the nonlinear expressive power, completing the first-level Transformer processing to obtain the first-level output features. After spatial downsampling and channel adjustment of the first-level output features, a small window partitioning strategy is used to perform window self-attention calculation to focus on local texture details. Then, a feedforward neural network is used to complete the second-level Transformer processing to obtain the second-level output features. The second-level output features are used as high-level semantic features.

[0157] Building upon the local details and multi-scale semantics already provided by Residual-ASPP and Slim-BiFPN, this invention further embeds a two-stage TinyViT lightweight Transformer after the temporal difference module. TinyViT-B1 is responsible for constructing a wide-range context, performing window self-attention on the feature map with a large window to capture the sparse distribution of thin smoke over large areas. TinyViT-B2 performs multi-head self-attention fusion on the shallow and deep features, supplementing mid-scale texture details, especially providing higher perception capability for fragmented flames obscured by tree branches. The dual-block cascade retains the local accuracy of convolutional networks while introducing global dependencies into the Transformer, forming a complementary relationship. Because TinyViT-B1 / B2 employs depthwise separable convolutions, layer normalization, and sparse MHSA, power consumption and latency remain almost unchanged.

[0158] TinyViT-B2 has aggregated both shallow and deep MHSA, providing ample context. After output, the Mask32 branch diverges here. On one hand, it inherits the global semantics of shallow and deep fusion, maintaining a complete perception of the overall outline of the smoke; on the other hand, it retains the high-frequency texture information of the flame, providing fine-grained boundaries for the pixel-level probability map and significantly improving the detection accuracy of occluded flames, achieving an efficient fusion of convolutional locality and Transformer globality.

[0159] Furthermore, the TinyViT module consists of two lightweight Transformer blocks (Block-1 and Block-2), which perform attention modeling for different semantic levels. Block-1 (B1) is responsible for building large window context dependencies and enhancing semantic aggregation of light smoke and blurred areas. Block-2 (B2) strengthens mesoscale local texture modeling to improve the discernibility of occluded flames and boundary details.

[0160] (1) TinyViT-B1 large window context modeling: Input from the output feature tensor of the Residual-ASPP module:

[0161] in =128, = =128.

[0162] 1) Window partitioning: Divide the feature map into... × Large windows, where features within each window are flattened into a two-dimensional matrix:

[0163] in Divided into Each window performs multi-head self-attention computation independently.

[0164] 2) Head Self-Attention (Window MSA): QKV mapping: Feature matrix within each window Generate query matrix through linear mapping Key matrix Value matrix :

[0165]

[0166]

[0167] in Mapping weights The projection weights are to be learned.

[0168] Multi-head attention calculation (including relative position bias): For each window's features, use the multi-head self-attention formula:

[0169] in The number of attention heads per head dimension =4, This is used to offset the increase in variance of the inner product as the dimension increases, thus maintaining the numerical stability of the softmax value. (Relative position offset) These are learnable parameters that reflect the spatial relationships between different positions within the encoding window.

[0170] Multi-head self-attention output projection: Attention calculation results By outputting the weight matrix Map back to the original dimension :

[0171]

[0172] Output , which is the feature matrix after window attention enhancement.

[0173] 3) Residual Connections and Feedforward Networks (Residual+MLP): Input feature tensor to the window:

[0174] in For the number of windows, C is the number of pixels within the window, and C is the number of channels.

[0175] Residual connection attention output: Attention Output After Dropout, compared with the original input Addition forms residual connection:

[0176] in This indicates element-wise multiplication. This is the Dropout mask. It follows a Bernoulli distribution. This represents the Dropout probability.

[0177] Residual Connectivity Feedforward Network (MLP): After passing through LayerNorm and two fully connected MLP networks, residual connections are formed again:

[0178]

[0179] in, , These are the weights of two fully connected layers in an MLP. , , It is the corresponding bias vector. To normalize the LayerNorm layer, normalize the feature dimensions of each sample.

[0180] For GELU activation function:

[0181] 4) Window Merge: By stitching together all the windows, we can restore the complete feature map:

[0182] (2) TinyViT-B2 mesoscale semantic modeling module: 1) Spatial convolution dimensionality reduction and normalization: Features output by Block-1 The number of channels was reduced to 96 using a depthwise separable convolution DSConv with a stride of 2, and the feature map scale was adjusted. Then, LayerNorm normalization was performed, resulting in:

[0183] 2) Window division and flattening: dividing the feature map Divided into sizes , totaling Each of the following non-overlapping windows is flattened into a feature matrix:

[0184] 3) Small Window Multi-Head Self-Attention (Window MSA): For each window Multi-head self-attention (MHSA) computation is performed independently to model the semantic relationships between pixels within a window, for each window. Independently perform multi-head self-attention computation.

[0185] QKV linear mapping:

[0186]

[0187]

[0188] Multi-head self-attention calculation: Calculate the multi-head attention matrix:

[0189] Attention output characteristics:

[0190] Among them, single-head dimension Head count The relative positional bias of learning The output projection matrix .

[0191] 4) Residual Connection and Feedforward Network (MLP): Output for each window Further enhance the nonlinear representation through residual connections and MLP networks:

[0192]

[0193] 5) Window Merge: All enhanced window feature matrices Reassemble into a complete feature map:

[0194] TinyViT-B2 layer integrates both deep and shallow multi-head self-attention (MHSA) and has the richest semantics; the mask branch bifurcates here, which can capture high-frequency information such as the fine texture of the flame, as well as effectively model low-frequency information such as the overall outline of the smoke.

[0195] Step 8, Unified Output of Multiple Tasks (Detect / Level / logσ² / Mask32): Based on high-level semantic features, target detection boxes, fire level classification, prediction uncertainty estimation, and pixel-level fire probability maps are generated in parallel through the output heads of multiple tasks.

[0196] The neck output first undergoes a shared 1×1 convolution to preserve global semantics, then is further processed by group depthwise separable convolutions to derive four lightweight branches, corresponding to object detection, fire severity level, and uncertainty estimation, respectively. Each head uses a 3×3 group depthwise separable convolution with 64 channels and 2 groups, achieving spatial awareness and task decoupling with minimal computational power. Furthermore, the 32 channels within each group can share texture and color information, unlike DWConv where information fragmentation from flame and smoke features degrades discriminative ability.

[0197] To provide low-overhead, fine-grained pixel-level fire / smoke cue maps, the Mask32 branch branches off from the TinyViT-B2 output. During training, gradient isolation is applied to the main features, and the branch only updates its own parameters, thus not interfering with the bounding box localization and semantic representation of the main detection and classification paths. This design provides stable fire and smoke heatmap visualization and auxiliary information with minimal computational overhead, and is more sensitive to occlusion and sparse smoke scenes, which is beneficial for backend decision-making and interpretable display.

[0198] Furthermore, this step designs a lightweight output module that unifies multiple tasks, enabling parallel inference of multiple tasks such as target detection, fire level classification, smoke recognition, uncertainty estimation, and pixel-level mask output.

[0199] The input features are derived from the output of the TinyViT-B2 layer:

[0200] in .

[0201] (1) Shared 1×1 convolution First, the feature dimension is unified through a shared 1×1 convolution, reducing the number of channels to 64, so that features output from the semantic layer can be shared across multiple tasks:

[0202]

[0203] This operation achieves channel dimension alignment with minimal computational cost, facilitating efficient feature sharing among subsequent task branches.

[0204] (2) Fire severity classification 1) GDW (Group Depthwise Conv): To ensure independence between different tasks, GDWConv introduces limited channel coupling bandwidth, maintaining multi-channel collaborative representation of color and texture while keeping costs under control. This mitigates the information fragmentation and decreased discriminative power issues caused by DWConv. Each task employs an independent group depthwise separable convolutional GDW, with a group number of... =2, denoted as:

[0205] Each branch structure is as follows:

[0206] The mathematical formula for GDW is expressed as follows: Let the input features be:

[0207] G is the number of groups:

[0208] Divide the channels into groups:

[0209]

[0210] Perform a k×k depthwise convolution independently for each channel in each group. :

[0211] The final output will be a concatenation of the results from each group along the channel dimension:

[0212] 2) Fire / Smoke Detection Box and Category Branch (Det)

[0213]

[0214] Each grid cell of the detection head The corresponding output is a set of 10-dimensional vectors:

[0215] Where the bounding box regression value is Category score: , For background or no fire, For tiny flames, It is an open flame (high brightness). It is a thin smoke (low contrast, gray haze). It is dense smoke (high density, dark ash). The six-dimensional classification, consisting of fire and smoke, is normalized using softmax to obtain the class probabilities. .

[0216]

[0217] A dual threshold is used to filter positive and negative samples: IoU > 0.5 is considered a positive sample, and IoU < 0.4 is considered a negative sample. Other samples are not included in gradient backpropagation. The following joint loss is used:

[0218] in , These are adjustable weighting coefficients.

[0219] For bounding box regression loss, CIoU+Smooth L1 combined loss is selected:

[0220] in For predicted bounding boxes and ground truth bounding boxes, To predict the coded coordinates and the actual coded coordinates, , This is an adjustable coefficient.

[0221] For classification loss, choose Focal Loss:

[0222] in As a unique hot tag, Here, γ represents the class probability obtained through softmax normalization, and γ is the hard sample adjustment factor. This represents the category weight.

[0223] 3) Fire severity classification branches (Cls, Class 3):

[0224] Each grid cell outputs a 3D vector:

[0225] in It is at the initial fire level. It is a medium fire level. It is classified as a major fire.

[0226] Each grid cell is normalized using softmax to obtain the probability of three types of fires:

[0227] The loss function uses weighted cross-entropy:

[0228]

[0229] The weights are calculated using the reciprocal of the frequency of each dataset category. To alleviate sample imbalance, This is a unique hot tag.

[0230] 4) Uncertainty branching ( (1 channel) To enhance the model's robustness to anomalous samples or ambiguous scenarios, an uncertainty estimation branch is introduced to explicitly regress the prediction uncertainty for each grid cell:

[0231]

[0232] The output of the grid cell is represented as:

[0233] If the data annotation provides confidence intervals or subjective assessments True value of variance Supervised learning is performed using mean squared error to regress on uncertainties:

[0234] If the training set does not provide explicit uncertainty labeling, this module uses heteroscedasticity modeling to model the prediction uncertainty. Used as the base loss for the corresponding main task With dynamic weights, construct the following self-supervised objective:

[0235] The first term represents the reweighted loss for the corresponding main task, and the second term is a regularization term that suppresses uncertainty expansion. This structure enables the network to automatically identify difficult samples and reduce their gradient contribution, thereby achieving hard example deweighting and improving training stability and generalization ability.

[0236] (3) Mask32 masking branch (pixel-level probability map, gradient isolation) To obtain a stable pixel-level probability map, this branch starts from the output features of TinyViT-B2, performs channel compression, upsampling and spatial smoothing before making pixel-by-pixel predictions; at the same time, gradient isolation is used to ensure that the main trunk and the detection and classification main paths are not disturbed by the loss of this branch.

[0237] For input features Apply the gradient isolation (Stop-Grad) operator:

[0238]

[0239] in This means that the gradient identity mapping is blocked during backpropagation, so only the mask branch parameters are updated during the training phase. The backbone weights are unaffected by gradient isolation.

[0240] Independent downsampling:

[0241] in For adaptive average pooling, its output is:

[0242] in:

[0243]

[0244]

[0245] To output grid rows, adaptive average pooling can parameterlessly and stably align features of arbitrary size to a specified grid, providing non-overlapping coverage, noise-resistant smoothing, and extremely low computational overhead.

[0246] Channel compression:

[0247] Spatial smoothing:

[0248] DWConv performs 3×3 convolutions on each channel to suppress checkerboard or jagged artifacts introduced by upsampling and to refine edges. Finally, it outputs a Mask32 probability map after a 1×1 convolution.

[0249] Pixel-level classification probability:

[0250] in It is the Sigmoid activation function. This is the bias. The output is the mask probability per pixel.

[0251] Downsampling the label mask to 32×32, we get:

[0252] Let the number of positive and negative pixels be... , Define pixel weights:

[0253] The loss function uses composite mask loss:

[0254]

[0255]

[0256] in For weighted binary cross-entropy, For Dice's loss, This is the composite mask loss.

[0257] Loss gradient isolation:

[0258] This step utilizes a Shared1×1 design with grouped depthwise separable convolutions. This invention achieves simultaneous output of four channels: bounding box, fire level, uncertainty, and pixel mask, with extremely low parameters and computational overhead. Mask32 employs a gradient isolation mechanism to ensure that the backbone detection accuracy is not affected, while providing high-resolution semantic information for area evaluation and false detection suppression.

[0259] This invention performs feature-level early fusion of RGB / IR at the network input: first, shallow alignment is completed with 1×1 shared convolution, and then learnable channel weights (Weight-Gate) are introduced to suppress redundancy, highlight cues with strong cross-modal differences, avoid the dilution of weak features (distant smoke, weak flames) by late fusion, and coordinate with subsequent pyramid scale.

[0260] This invention adopts a three-branch parallel structure. During the training phase, learnable gated weighted fusion is used. During the inference phase, convolution and BN folding are reparameterized into a single equivalent convolution. This achieves high-quality downsampling with anti-aliasing and edge preservation without increasing MAC and memory, retaining high-frequency information such as smoke filaments and contours, while providing more stable multi-scale input for subsequent networks.

[0261] This invention forms a zero-parameter attention chain by connecting shallow backbone spatial attention SimAM and channel attention GEM, which improves the response to low-contrast smoke and sparse textures, without introducing additional trainable parameters or significant computational overhead.

[0262] This invention constructs a bidirectional fusion pyramid with both top-down and bottom-up approaches. All fusion points employ learnable weighting (wAdd) and replace conventional convolutions with depthwise separable convolutions (DSConv), achieving a balance between structural compression and feature integrity. This enhances the robustness of multi-scale fire source and smoke target detection while controlling the number of parameters and computational cost.

[0263] This invention introduces channel-level, parameter-free temporal offsets into the mid-level feature flow, performing forward / backward one-frame displacements on only some channels while keeping the remaining channels unchanged, thus forming a lightweight temporal model. It can enhance the perception of dynamic spatiotemporal features such as flame jumps and smoke drifts without adding new parameters or significant computing power.

[0264] This invention addresses the shortcomings of TSM-Lite, which only performs temporal rearrangement and lacks explicit spatial modeling. It follows up with the Residual-ASPP (Residual Spatial Pyramid) and uses multi-rate dilatation dilated convolution to perform multi-scale amplification and alignment of the differential cues injected by TSM in the spatial dimension. It also uses residual gating to suppress over-enhancement, thereby taking into account the multi-scale enhancement of thin smoke and small-scale background motion, while preserving the characteristics of the main flame.

[0265] This invention employs a two-layer lightweight Transformer, TinyViT-B1 / B2: B1 constructs a long-range context with a large window, while B2 uses a small window to finely model medium-scale textures and occluded flames. By combining the global perception of Transformer with the local precision of convolution, it improves the global understanding of fire level judgment.

[0266] This invention unifies high-level features into a specified channel using a shared 1×1 pair, and then constructs a four-branch Header: Detect outputs detection and localization, Cls determines the fire severity level, logσ² outputs heterogeneous uncertainty, and Mask32 outputs a 32×32 pixel-level mask. Each branch shares the main trunk but is gradient-decoupled, enhancing expressive power without interfering with the main detection path.

[0267] This invention addresses the multimodal early forest fire identification task on VTOL (Dynamic Thrustering Online) platforms. It employs a systematic and lightweight design focusing on low-power real-time operation, visibility of weak targets, temporal consistency, and deployability. Compared to conventional solutions, it offers the following advantages: 1. Platform compatibility and deployability: The system is designed from the bottom layer for VTOL navigation and edge SoC constraints. Through Rep-DownBlock heavy parameter downsampling and full-link DSConv compression, the inference graph is simple and easy to quantize. It can run stably on devices such as RK3588 and Jetson Nano, meeting the requirements of low power consumption and real-time performance.

[0268] 2. Early fusion improves modal collaboration and parameter efficiency: Early-Fusion+Weight-Gate is used to align RGB and IR shallow features with 1×1 shared convolution in the input stage, and learnable channel gating is used to suppress redundancy and highlight cross-modal difference cues. Compared with late-fusion, it can better preserve low-contrast targets such as distant light smoke / weak flames, while reducing invalid parameters and redundant calculations.

[0269] 3. Structural Compression + High-Quality Downsampling + Stable Multi-Scale Fusion: Rep-DownBlock uses three-branch gating during training to learn anti-aliasing and edge preservation, and folds into a single equivalent convolution during inference, achieving high-quality downsampling without increasing MAC and GPU memory. The pyramid structure adopts Slim-BiFPN, whose wAdd is a learnable weighted and fully replaced DSConv to achieve bidirectional multi-scale information interaction and effectively control parameters.

[0270] 4. More robust recognition of weak targets / nighttime scenes: The shallow zero-parameter attention chain SimAM and GEM improve the response to low-contrast textures and light smoke boundaries. CBAM-k5 is introduced at FP4 to enhance mesoscale saliency and avoid overfitting caused by attention stacking at the same resolution. TinyViT-B1 and B2 large and small windows are embedded upstream to perform hierarchical context and mesoscale texture modeling, making up for the convolution's dependence on long distances and occlusion details. This results in higher recall and more stable boundaries in nighttime and distant light smoke scenes.

[0271] 5. Lightweight Temporal Modeling and Multi-Scale Spatial Magnification Collaboration: The middle layer introduces TSM-Lite to perform ±1 frame parameter-free shifting on a small proportion of channels, while maintaining static baselines for the remaining channels. This injects temporal neighborhood information with near-zero overhead. Subsequently, Residual-ASPP employs dilated convolutions with medium to small dilation rates to perform multi-scale magnification and alignment of the differential cues between adjacent frames in the spatial dimension, with residual scaling γ controlling the enhancement intensity. This BiFPN-TSM-Lite-Residual-ASPP link, centered on learnable weighted fusion and residual back-addition, does not significantly dilute effective information but instead improves temporal consistency and contextual modeling, particularly beneficial for light smoke boundaries and flickering flames.

[0272] 6. Multi-task decoupling output and uncertainty-supported decision-making: After unifying high-level features with shared 1×1 convolutions, a four-branch head is constructed. The detection box, fire level, heterogeneous uncertainty, and pixel-level mask are output in parallel through GDW. Each branch shares the main trunk but the gradient is decoupled, thereby improving the overall semantic expression capability without affecting the main detection path. At the same time, the uncertainty output by logσ² can be used to reduce the weight of difficult and noisy samples during the training phase, and the alarm threshold or risk handling strategy can be adaptively adjusted according to the uncertainty magnitude during the deployment phase.

[0273] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multimodal lightweight forest fire detection method applicable to vertical take-off and landing fixed-wing unmanned aerial vehicle platforms, characterized in that, Includes the following steps: Visible light and infrared images are acquired synchronously by a UAV. A multimodal early fusion method is used to align and weight the features of the visible light and infrared images to obtain fused features. The process of feature alignment and weighted fusion using the multimodal early fusion method includes: extracting shallow features from the visible light and infrared images using convolutional layers with shared weights to obtain visible light shallow features and infrared shallow features; generating weighted gating coefficients using fully connected layers and activation functions based on the global statistics of the visible light and infrared shallow features; and weighting and summing the visible light and infrared shallow features using the weighted gating coefficients to obtain the fused features. The fused features are subjected to lightweight downsampling and multi-scale feature extraction to obtain a multi-scale feature set; A bidirectional feature pyramid network is used to perform cross-scale feature fusion on the multi-scale feature set and extract dynamic change information to obtain temporal features; The temporal features are enhanced with multi-scale context using a spatial pyramid method based on dilated convolution to obtain context-enhanced features. The lightweight Transformer module is used to perform global context and local texture modeling on the context enhancement features to obtain high-level semantic features; Based on the aforementioned high-level semantic features, target detection boxes, fire level classifications, prediction uncertainty estimates, and pixel-level fire probability maps are generated in parallel through multi-task output heads.

2. The multimodal lightweight forest fire detection method applicable to vertical takeoff and landing fixed-wing UAV platforms according to claim 1, characterized in that, The process of performing lightweight downsampling and multi-scale feature extraction on the fused features to obtain a multi-scale feature set includes: A reparameterizable downsampling module is constructed. During the training phase, the reparameterizable downsampling module adopts a parallel structure containing a spatial convolution branch, a channel mixing branch, and a low-pass filtering branch. During the inference phase, the parallel structure is reparameterized into a single convolution kernel. Based on the fused features, a reparameterizable downsampling module is used for processing to obtain the downsampled feature map. The downsampled feature map is then processed by a zero-parameter attention module and global expectation modulation to obtain a multi-scale feature set.

3. The multimodal lightweight forest fire detection method applicable to vertical takeoff and landing fixed-wing UAV platforms according to claim 2, characterized in that, The process of using a bidirectional feature pyramid network to perform cross-scale feature fusion on the multi-scale feature set and extracting dynamic change information to obtain temporal features includes: Based on the aforementioned multi-scale feature set, a lightweight bidirectional pyramid network is used to perform cross-scale feature fusion to obtain enhanced multi-scale features. Based on the specified level feature sequence in the enhanced multi-scale features, a time-shifting method is used to process the data to obtain time-series features containing time-series dynamic information.

4. The multimodal lightweight forest fire detection method applicable to vertical takeoff and landing fixed-wing UAV platforms according to claim 3, characterized in that, The lightweight bidirectional pyramid network includes: a bottom-up path and a top-down path; The process of obtaining enhanced multi-scale features includes: at the fusion node of the lightweight bidirectional pyramid network, firstly, depthwise separable convolution is used to transform each input feature, and then learnable weight coefficients are used to adaptively weight and fuse the transformed features; after completing the adaptive weighted fusion, a dual attention mechanism including channel attention and 5×5 spatial attention is further used at a specific intermediate level to enhance the fused features, thereby obtaining the enhanced multi-scale features.

5. The multimodal lightweight forest fire detection method applicable to vertical takeoff and landing fixed-wing UAV platforms according to claim 3, characterized in that, The process of obtaining time series features includes: The feature channel is divided into a forward part, a static part, and a backward part; The features of the forward part are shifted to the previous frame, and the features of the backward part are shifted to the next frame, while keeping the features of the static part unchanged, to obtain the shifted feature block. The shifted feature blocks are spliced ​​together to obtain temporal features containing temporal dynamic information.

6. The multimodal lightweight forest fire detection method applicable to vertical takeoff and landing fixed-wing unmanned aerial vehicle platforms according to claim 5, characterized in that, The process of using a spatial pyramid method based on dilated convolution to perform multi-scale context enhancement on the temporal features to obtain context-enhanced features includes: Based on the aforementioned temporal features, multi-scale contextual features are extracted in parallel using depthwise separable convolutions with different dilation rates. The multi-scale context features are concatenated along the channel dimension, and 1×1 convolution is used for channel compression and feature fusion to obtain the fused multi-scale features. The fused multi-scale features and the original temporal features are weighted and summed using learnable residual weights to obtain the context-enhanced features.

7. The multimodal lightweight forest fire detection method applicable to vertical takeoff and landing fixed-wing UAV platforms according to claim 6, characterized in that, The process of using the lightweight Transformer module to perform global context and local texture modeling on the context enhancement features to obtain high-level semantic features includes: Based on the aforementioned context enhancement features, a large window partitioning strategy is first used to perform window self-attention calculation to capture global context information. Then, a feedforward neural network is used to enhance the nonlinear expression capability, and the first-level Transformer processing is completed to obtain the first-level output features. After spatial downsampling and channel adjustment of the first-level output features, a small window partitioning strategy is used to perform window self-attention calculation to focus on local texture details. Then, through feedforward neural network processing, the second-level Transformer processing is completed to obtain the second-level output features. The features output from the second level are used as the high-level semantic features.

Images