A system for small fire point segmentation in satellite remote sensing images

Abstract

This invention discloses a system for segmenting minimal fire points in satellite remote sensing imagery, relating to the fields of image processing and remote sensing technology. The system includes an encoder, a fire-oriented directional context modulation module, a decoder, and a multi-scale Mamba attention module. The encoder suppresses background redundancy responses and enhances minimal fire point features through a cross-layer gating mechanism; the fire-oriented directional context modulation module enhances the response of directional expansion features of fire points through multi-directional context modeling; the decoder fuses encoder features through skip connections; and the multi-scale Mamba attention module utilizes a state-space model to model long-range spatial dependencies, enhancing the globally consistent representation of multiple minimal fire points. This invention solves the problems of extremely small fire point target scale, complex background, difficulty in capturing directional diffusion, and poor global consistency in satellite imagery, achieving high-precision and robust segmentation of minimal fire points and improving the accuracy and efficiency of fire monitoring.

Description

Technical Field

[0001] This invention relates to the fields of image processing and remote sensing technology, and in particular to a system for segmenting extremely small fire points in satellite remote sensing images. Background Technology

[0002] Satellite remote sensing, with its ability to provide large-scale and continuous observation, has become an important tool for fire monitoring. By analyzing multispectral images acquired by satellites, the location of fire points can be detected in a timely manner, and the spread of fire can be assessed, providing crucial information for fire early warning and emergency response. However, fire points in satellite remote sensing images are typically characterized by extremely small scale, sparse spatial distribution, and complex and variable backgrounds, posing a significant challenge to accurate segmentation.

[0003] In existing technologies, fire detection methods are mainly divided into two categories: traditional thresholding methods and deep learning methods. Traditional methods are based on the radiation characteristics of fire in the infrared band, using fixed or adaptive thresholds to identify fire pixels. These methods rely on manually designed features and statistics, which may be effective in simple contexts, but are prone to producing a large number of false positives or false negatives in complex scenes, and the threshold settings are difficult to generalize to different regions and seasons. With the development of deep learning, segmentation models based on convolutional neural networks (CNNs) have been introduced into fire detection tasks. These models can automatically learn multi-level features, improving detection accuracy to some extent. However, CNNs have a limited receptive field, making it difficult to capture long-range spatial dependencies. For multiple sparsely distributed, extremely small fire points, inconsistent local responses and the omission of some fire points can easily occur. At the same time, CNNs are insufficient in modeling the directional extension features of targets (such as fire lines spreading along the wind direction), resulting in incomplete segmentation of thin, elongated fire points.

[0004] In recent years, Transformer-based models have achieved global context modeling through self-attention mechanisms, but their high computational complexity and slow inference speed make them difficult to meet real-time monitoring requirements. The Mamba state-space model, as an emerging architecture, possesses powerful global modeling capabilities while maintaining linear complexity. However, existing Mamba segmentation networks primarily target general targets and are not optimized for the small scale, directionality, and sparsity of fire points. In the task of segmenting extremely small fire points, problems such as insufficient feature extraction and inadequate background suppression still exist. Existing technologies, whether traditional thresholding methods or deep learning methods, have significant limitations in processing extremely small fire points in satellite remote sensing imagery. Traditional methods rely on fixed spectral thresholds or simple contextual statistics. For fire point targets occupying only a few pixels (e.g., 5 or 10 pixels), their radiation signals are easily submerged by surrounding background noise, resulting in a very high false negative rate. While CNN-based segmentation models improve detection capabilities through multi-level feature extraction, their receptive field is limited, and they easily lose detailed information about extremely small fire points during downsampling, resulting in weak or even absent responses in high-level feature maps in scenarios such as fire point recognition with ≤10 pixels. While Transformer-type models can model global dependencies, their self-attention mechanism tends to dilute local salient features in a large background, making it difficult to reliably identify extremely small fire points, such as those at the 5-pixel level. Therefore, there is an urgent need for a segmentation scheme that can simultaneously preserve the details of extremely small targets, enhance directional features, and maintain global consistency, in order to improve the detection accuracy and robustness of extremely small fire points in satellite remote sensing imagery. Summary of the Invention

[0005] This invention overcomes the problems of existing technologies in fire identification using satellite imagery, such as extremely small fire target scale, complex background, difficulty in capturing directional diffusion, and poor global consistency. It achieves high-precision and robust segmentation of extremely small fire points, thereby improving the accuracy and efficiency of fire monitoring.

[0006] To achieve the above objectives, the present invention adopts the following solution: A system for segmenting extremely small fire points in satellite remote sensing imagery, comprising: The encoder consists of multiple cascaded cross-layer gated residual U-shaped modules. Each cross-layer gated residual U-shaped module is used to downsample the input feature map and extract multi-scale context features, and suppress background redundant response through cross-layer gated mechanism. A fire-oriented directional context modulation module, connected after at least one intermediate level of the encoder, is used to perform multi-directional context modeling on the features output by the encoder to enhance the response of the directional extension features of the fire point along the wind direction and terrain orientation. The decoder consists of multiple decoding units corresponding to the encoder. Each decoding unit is used to upsample the input features and fuse them with the corresponding level features from the encoder. The multi-scale Mamba attention module, which is embedded in each hop connection path between the encoder and decoder, is used to fuse the features of the current scale encoder as well as the features of the adjacent scale encoder, and to model long-range spatial dependencies using a state space model to enhance the global consistency representation of multiple minimal fires. The encoder receives satellite remote sensing images as input, and after feature extraction at each level, it is modulated by a fire-oriented directional context modulation module. The modulated features are then input to subsequent coding units and to a multi-scale Mamba attention module via skip connections. The decoder receives features from each level of the encoder via skip connections, processes them through the multi-scale Mamba attention module, upsamples them at each level, and fuses them to finally output the fire point segmentation result.

[0007] As a preferred option, the cross-layer gated residual U-shaped module is equipped with an encoding sub-module and a decoding sub-module; The encoding submodule consists of multiple sequentially connected downsampling units, used to downsample the input feature map step by step and extract multi-scale contextual features; the decoding submodule consists of multiple sequentially connected upsampling units, each upsampling unit is used to upsample the features from adjacent higher-level upsampling units, and concatenate and fuse them with the features output by the corresponding level downsampling unit to obtain the fused features of the current layer; each upsampling unit is also used to sequentially process the fused features of the current layer with global average pooling, 1×1 convolution and sigmoid function to generate the initial gating weights of the current layer. In the decoding submodule, the upsampling unit of the adjacent higher layer passes its generated initial gating weight to the upsampling unit of the current layer, and the weight is superimposed with the initial gating weight of the current layer to become the final gating weight of the current layer. The upsampling unit of the current layer is also used to perform channel-level multiplication operation on the final gating weight and the fused feature of the current layer to obtain the modulated feature, and output the modulated feature to the upsampling unit of the adjacent lower layer or as the output of the cross-layer gating residual U-shaped module.

[0008] Preferably, the fire-oriented direction context modulation module includes a global context branch, a direction perception branch, a local context branch, and a fusion unit; The global context branch sequentially performs global average pooling, 1×1 convolution, and upsampling on the input features to output global context features. The orientation-aware branch performs strip pooling and depthwise separable one-dimensional convolution on the input features along the horizontal, vertical and diagonal directions to generate attention maps for each direction. The attention maps are then normalized by the Sigmoid function and superimposed. During superposition, weighted fusion is performed using learnable gating parameters to output orientation-aware features. The local context branch performs local average pooling and 1×1 convolution on the input features and outputs local context features. The fusion unit is used to concatenate global context features, orientation-aware features, and local context features along the channel dimension, and then add them element-wise with the input features after a 1×1 convolution to output the modulated features.

[0009] As a preferred embodiment, the multi-scale Mamba attention module includes a spatial attention weighted branch, a Mamba global modeling branch, and a fusion unit; The spatial attention weighted branch downsamples the features of adjacent high-level structures and generates a first spatial attention map through the spatial attention mechanism. It upsamples the features of adjacent low-level structures and generates a second spatial attention map through the spatial attention mechanism. The first spatial attention map is then multiplied element-wise with the current scale features to obtain the first weighted feature. The second spatial attention map is then multiplied element-wise with the current scale features to obtain the second weighted feature. The Mamba global modeling branch reconstructs the current-scale features into a sequence, which is then processed by a linear mapping layer, a state-space model operator, and an activation function before being reconstructed back into a two-dimensional feature space to obtain the global modeling features. The fusion unit overlays the current scale features, the first weighted features, the second weighted features, and the global modeling features, and inputs the overlaid features into the residual bottleneck structure for feature fusion and channel adjustment, outputting the fused multi-scale features.

[0010] Preferably, when the multi-scale Mamba attention module is located at the first level of the encoder, the spatial attention weighting branch only upsamples the adjacent low-level features to generate the second spatial attention map, and the features superimposed by the fusion unit do not include the first weighted feature; when the multi-scale Mamba attention module is located at the fifth level of the encoder, the spatial attention weighting branch only downsamples the adjacent high-level features to generate the first spatial attention map, and the features superimposed by the fusion unit do not include the second weighted feature.

[0011] Preferably, the decoder has multiple bypass outputs, each of which corresponds to a feature map of a decoding level. The system also includes a deep supervision module, which calculates the mixing loss between the prediction result of each bypass output and the real fire point mask, and sums the mixing losses of each bypass output according to a preset decreasing weight to obtain the total loss. The mixing loss consists of binary cross-entropy loss and IoU loss, and the decreasing weight decreases sequentially as the decoding level corresponding to the bypass output increases.

[0012] As a preferred option, the satellite remote sensing imagery used is multispectral imagery from the Landsat-8 satellite, with the input bands being a combination of bands 7, 6, and 5.

[0013] Preferably, in the direction-aware branch, multiple strip lengths of different scales are used when performing strip pooling along each direction. Each scale corresponds to a set of strip pooling operations, generating a direction attention map at that scale. The direction attention maps of multiple scales under the same direction are concatenated along the channel dimension and weighted and summed through a learnable fusion weight matrix to obtain the fused direction attention map for that direction. The learnable fusion weight matrix is ​​updated through backpropagation during training to adaptively adjust the contribution of features of different scales to the final direction-aware features. The fused direction attention maps of each direction are superimposed to output multi-scale direction-aware features.

[0014] The present invention includes at least the following beneficial effects: (1) By using the cross-layer gated residual U-shaped module in the encoder, the cross-layer gated mechanism is used to transmit high-level semantic information step by step in the decoding stage, adaptively suppressing the redundant response of complex backgrounds, significantly enhancing the discrimination features of extremely small and weak response fire points, and greatly improving the recognition ability of extremely small fire points compared with existing methods; (2) The fire-oriented directional context modulation module explicitly models the anisotropic diffusion features of fire points along wind direction and terrain direction through multi-directional strip pooling and learnable gated parameters, effectively improving the segmentation integrity of slender or continuous fire points, especially in scenarios where the shape of fire points is significantly affected by environmental factors, the recognition accuracy is significantly improved; (3) The multi-scale Mamba attention module integrates spatial attention weighting and Mamba global modeling, in skip connections Introducing long-range dependencies enables multiple sparsely distributed tiny fire points to maintain consistent response intensity across a wide range of scenes, avoiding local missed detections and significantly enhancing global consistency; (4) Based on the cross-layer gated residual U-shaped module and the direction modulation module, the internal implementation is further refined. Through the step-by-step transfer of gated weights and multi-branch direction perception, the feature modulation is more accurate and the direction context modeling is more comprehensive, thus achieving more robust fire point recognition in complex backgrounds; (5) Through the detailed design of the multi-scale Mamba attention module, hierarchical adaptive simplification, and deep supervision hybrid loss strategy, the flexibility and training stability of the model are further improved. While maintaining the advantage of lightweight design, the convergence speed is accelerated, so that the final segmentation results achieve excellent levels in both pixel-level accuracy and regional integrity. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the overall network structure of the system according to the present invention; Figure 2 This is a schematic diagram of the structure of a cross-layer gated residual U-shaped module according to the present invention; Figure 3 This is a schematic diagram of the structure of a fire-oriented directional context modulation module according to the present invention; Figure 4 This is a schematic diagram of the structure of a multi-scale Mamba attention module according to the present invention; Figure 5 A pixel statistics chart of fire points in the Oceania dataset; Figure 6 A pixel statistics chart of fire points in the Asia4 dataset; Figure 7 Figure showing the comparison experiment results for the Oceania dataset; Figure 8 This is a figure showing the experimental results of comparing the Asia4 dataset of this invention; Figure 9This is a visualization of the multi-stage feature representation under different module ablation configurations of the present invention. Detailed Implementation

[0016] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0017] This invention provides a system for segmenting minimal fire points in satellite remote sensing imagery, comprising: The encoder consists of multiple cascaded cross-layer gated residual U-shaped modules. Each cross-layer gated residual U-shaped module is used to downsample the input feature map and extract multi-scale context features, and suppress background redundant response through cross-layer gated mechanism. A fire-oriented directional context modulation module, connected after at least one intermediate level of the encoder, is used to perform multi-directional context modeling on the features output by the encoder to enhance the response of the directional extension features of the fire point along the wind direction and terrain orientation. The decoder consists of multiple decoding units corresponding to the encoder. Each decoding unit is used to upsample the input features and fuse them with the corresponding level features from the encoder. The multi-scale Mamba attention module, which is embedded in each hop connection path between the encoder and decoder, is used to fuse the features of the current scale encoder as well as the features of the adjacent scale encoder, and to model long-range spatial dependencies using a state space model to enhance the global consistency representation of multiple minimal fires. The encoder receives satellite remote sensing images as input, and after feature extraction at each level, it is modulated by a fire-oriented directional context modulation module. The modulated features are then input to subsequent coding units and to a multi-scale Mamba attention module via skip connections. The decoder receives features from each level of the encoder via skip connections, processes them through the multi-scale Mamba attention module, upsamples them at each level, and fuses them to finally output the fire point segmentation result.

[0018] The encoder consists of multiple cascaded cross-layer gated residual U-shaped modules. Each module is a small U-shaped network containing encoding and decoding sub-modules. These modules progressively downsample the input feature map to extract multi-scale contextual features. Simultaneously, a cross-layer gating mechanism adaptively suppresses redundant responses in complex backgrounds, thereby enhancing fire detection features with minimal and weak responses. Specifically, the cross-layer gating mechanism is implemented in the decoding stage: each decoding layer generates a fused feature after fusing features from the corresponding encoding layer and the previous decoding layer. This fused feature is then generated using global average pooling and 1×1 convolution to produce initial gating weights. These weights are superimposed with gating weights passed from a higher decoding layer to form the final gating weights. These final gating weights are then multiplied by the current layer's fused feature at the channel level to obtain the modulated feature. This process allows high-level semantic information to progressively guide the selection of low-level features, highlighting fire-related responses.

[0019] A fire-oriented directional context modulation module is connected after at least one intermediate level of the encoder, for example, it can be inserted after the fourth stage of the encoder in a practical deployment. This modulation module consists of three parallel branches: a global context branch extracts the global response intensity of the entire image through global average pooling and 1×1 convolution; a direction-aware branch performs strip pooling and depthwise separable one-dimensional convolution along the horizontal, vertical, and diagonal directions to generate attention maps for each direction, and then performs weighted fusion using learnable gating parameters to explicitly model the anisotropic diffusion characteristics of fire points influenced by wind direction and terrain; and a local context branch supplements neighborhood detail information through local average pooling. The outputs of the three branches are concatenated along the channel dimension, compressed by 1×1 convolution, and then added to the input feature residuals, thereby effectively enhancing the expressive power of the directional expansion features of fire points without destroying the original discriminative information.

[0020] The decoder consists of multiple decoding units corresponding to encoder levels. Each decoding unit upsamples the input features and fuses them with features from the corresponding encoder level via skip connections. A multi-scale Mamba attention module is embedded in each skip connection path. This module fuses encoder features at the current scale with those at adjacent scales and models long-range spatial dependencies using a state-space model. Specifically, the multi-scale Mamba attention module includes a spatial attention weighting branch and a Mamba global modeling branch: the spatial attention weighting branch generates a spatial attention map by downsampling adjacent high-level features and upsampling adjacent low-level features, and multiplies it element-wise with the current scale features to obtain cross-scale attention weighted features; the Mamba global modeling branch reconstructs the current scale features into a sequence, processes it sequentially through linear mapping, state-space model operators (i.e., Mamba core operators), and activation functions, and then reconstructs it back into a two-dimensional feature space, thereby capturing the spatial correlation between multiple tiny fire points in a large-scale scene. Finally, the current scale features, cross-scale attention weighted features, and Mamba global modeling features are superimposed and input into a residual bottleneck structure for fusion and channel adjustment, outputting multi-scale features with global consistency.

[0021] During system operation, satellite remote sensing imagery is first input into the encoder. It is then downsampled step-by-step through multiple cascaded, cross-layer gated residual U-shaped modules, with each level outputting feature maps at different scales. After a preset intermediate level in the encoder (such as the fourth stage), the feature maps enter a fire-oriented directional context modulation module for directional context enhancement. The modulated features are then input into subsequent encoding modules for high-level semantic extraction, and simultaneously passed to the corresponding multi-scale Mamba attention module in the decoder via skip connections. The decoder begins step-by-step upsampling. Each decoding unit receives upsampled features from the previous decoding layer and simultaneously receives corresponding encoding layer features processed by the multi-scale Mamba attention module via skip connections. These features are then fused and upsampled again. This process is repeated until the input image resolution is restored, ultimately outputting a fire point segmentation probability map. Throughout this process, the cross-layer gating mechanism and the directional modulation module work together to enable the network to accurately locate extremely small fire points while maintaining their morphological continuity, while the multi-scale Mamba attention module ensures that multiple sparse fire points have consistent response strength across a large scene, avoiding missed detections.

[0022] As a concrete example, the overall architecture of the FireMambaNet network designed in this solution is as follows: Figure 1As shown, FireMambaNet employs a nested U-shaped encoder-decoder framework, primarily composed of three core components: a cross-layer gated residual U-block (CG-RSU), a fire-aware directional context modulation (FDCM) module, and a multi-scale Mamba attention module (M2AM). The network uses raw satellite imagery as input. The encoder consists of six cascaded CG-RSU modules, and the encoder progressively extracts multi-scale features rich in contextual information through downsampling operations. Specifically, Stages 1-4 employ CG-RSU7, CG-RSU6, CG-RSU5, and CG-RSU4 structures respectively to fully model low- and mid-level features, while Stages 5 and 6 use a lightweight CG-RSU4F structure for high-level semantic feature extraction and bottleneck representation. To enhance the network's ability to perceive the directional extension features of fire points, FDCM is used in the fourth stage of the encoder to perform multi-directional context modeling of mid-to-high-level features, thereby strengthening the response of fire points along directional structures such as wind direction and terrain orientation. In the decoding stage, the decoder adopts a structure symmetrical to the encoder and fuses contextual features through skip connections. Notably, the proposed multi-scale Mamba attention module (M2AM) is embedded in the skip connection process to enhance the globally consistent representation of extremely small and sparsely distributed fire points in a large-scale scene. Furthermore, a deep supervision strategy with six side-channel output branches (s1-s6) is employed to guide multi-scale learning during network training. Finally, the network outputs fire point segmentation results, achieving robust detection of extremely small satellite fire points in complex backgrounds.

[0023] This system significantly improves the segmentation accuracy and robustness of extremely small fire points in complex backgrounds through the organic cooperation of multiple modules. Compared with traditional convolutional neural network methods, this system can more effectively capture the weak response features of extremely small targets and performs better in terms of directional expansion and global consistency. Compared with Transformer-based methods, this system significantly reduces computational complexity and inference latency while maintaining strong global modeling capabilities. Compared with other Mamba-based segmentation networks, this system further optimizes the ability to characterize the morphology and spatial distribution of fire points by introducing cross-layer gating and directional context modulation.

[0024] In another technical solution, the cross-layer gated residual U-shaped module is equipped with an encoding sub-module and a decoding sub-module. The encoding submodule consists of multiple sequentially connected downsampling units, used to downsample the input feature map step by step and extract multi-scale contextual features; the decoding submodule consists of multiple sequentially connected upsampling units, each upsampling unit is used to upsample the features from adjacent higher-level upsampling units, and concatenate and fuse them with the features output by the corresponding level downsampling unit to obtain the fused features of the current layer; each upsampling unit is also used to sequentially process the fused features of the current layer with global average pooling, 1×1 convolution and sigmoid function to generate the initial gating weights of the current layer. In the decoding submodule, the upsampling unit of the adjacent higher layer passes its generated initial gating weight to the upsampling unit of the current layer, and the weight is superimposed with the initial gating weight of the current layer to become the final gating weight of the current layer. The upsampling unit of the current layer is also used to perform channel-level multiplication operation on the final gating weight and the fused feature of the current layer to obtain the modulated feature, and output the modulated feature to the upsampling unit of the adjacent lower layer or as the output of the cross-layer gating residual U-shaped module.

[0025] The cross-layer gated residual U-shaped module serves as the core component of the system, internally comprising an encoding submodule and a decoding submodule. These two submodules work together to achieve multi-scale feature extraction and gated modulation. The encoding submodule consists of multiple sequentially connected downsampling units. Each downsampling unit typically includes a convolutional layer or max-pooling layer with a stride of 2, followed by batch normalization and activation functions. This downsampling progressively expands the receptive field and extracts multi-scale contextual information from local to global perspectives. The decoding submodule consists of multiple sequentially connected upsampling units, their number corresponding to the number of downsampling units in the encoding submodule, forming a symmetrical structure. Each upsampling unit first receives the feature map output from an adjacent higher-layer upsampling unit and upsamples it using bilinear interpolation or transposed convolution to align its spatial dimensions with the corresponding encoding layer features. Figure 1 Then, the upsampled features are concatenated and fused with the features output by the corresponding downsampled unit of the coding layer in the channel dimension to obtain the fused features of the current layer.

[0026] To address the challenge of representing extremely small fire targets in satellite remote sensing imagery effectively using traditional CNNs, a cross-layer gated residual U-block (CG-RSU) was designed. By introducing a cross-layer gated modulation mechanism, CG-RSU can adaptively filter feature responses at different scales, effectively suppressing redundancy and noise interference in complex backgrounds, while significantly enhancing the discriminative features of extremely small and weak-response fire points. CG-RSU employs a nested encoder-decoder structure and is the core building block of FireMambaNet, as shown in the diagram. Figure 2 As shown.

[0027] During the encoding phase, CG-RSU extracts contextual information from local to global levels through a multi-layer downsampling structure, effectively capturing the spatial and semantic differences between the fire point and its surrounding background. This multi-feature extraction process can be represented as: Where F i It is the extracted first i Layer features, Represents max pooling operation. For convolution mapping, L The number of layers that the network dynamically adjusts.

[0028] During the decoding phase, CG-RSU achieves a full fusion of high-level semantic information and low-level fine-grained spatial information by upsampling layer by layer and concatenating the samples with encoded features at the corresponding scale. This process can be represented as: in Indicates the first i Features after layer fusion This indicates an upsampling operation.

[0029] Unlike traditional RSU or single-layer gating structures, this scheme innovatively designs a cross-layer gating mechanism. Specifically, CG-RSU establishes a hierarchical transmission path for gating information between adjacent decoding layers, enabling higher-level semantic features to explicitly guide the selection process of lower-level features. This continuously strengthens the discriminative information related to the fire point during multi-scale fusion and suppresses background redundancy responses. The process of this cross-layer gating can be represented as follows: in This indicates the characteristics after gated modulation. G represents channel multiplication. i The gating weight is represented by the following formula: in Indicates global average pooling. For 1×1 convolution weights, This is the Sigmoid function.

[0030] In another technical solution, the fire-oriented direction context modulation module includes a global context branch, a direction-aware branch, a local context branch, and a fusion unit; The global context branch sequentially performs global average pooling, 1×1 convolution, and upsampling on the input features to output global context features. The orientation-aware branch performs strip pooling and depthwise separable one-dimensional convolution on the input features along the horizontal, vertical and diagonal directions to generate attention maps for each direction. The attention maps are then normalized by the Sigmoid function and superimposed. During superposition, weighted fusion is performed using learnable gating parameters to output orientation-aware features. The local context branch performs local average pooling and 1×1 convolution on the input features and outputs local context features. The fusion unit is used to concatenate global context features, orientation-aware features, and local context features along the channel dimension, and then add them element-wise with the input features after a 1×1 convolution to output the modulated features.

[0031] The fire-oriented directional context modulation module, a component specifically designed to enhance the directional awareness of fire points, comprises three parallel branches: a global context branch, a direction-aware branch, and a local context branch, along with a fusion unit. Each branch processes the input features from different perspectives, ultimately integrating complementary information for output. The direction-aware branch performs strip pooling on the input feature map along the horizontal, vertical, and diagonal directions (e.g., 45 degrees). The feature maps output from the three branches are concatenated along the channel dimension to form a feature map with increased channel count. Then, a 1×1 convolutional layer is used for channel compression and feature fusion, allowing for the interactive integration of information from different branches. Finally, the fused feature map is element-wise added to the module's input feature map to achieve residual connections, thus injecting direction awareness and contextual information while preserving the original features, resulting in modulated features. This design enables the module to effectively enhance the response to directional extension features of fire points, such as wind direction and terrain orientation, without compromising the original discriminative information.

[0032] To enhance the network's ability to identify fire points exhibiting clear directional diffusion characteristics in spatial morphology, a fire-aware directional context modulation (FDCM) module was specifically designed. This module, through direction-sensitive context modeling, significantly improves the network's ability to perceive weak fire points and their surrounding spatial relationships. Figure 3 As shown, FDCM consists of three complementary context modeling branches. The first branch uses global average pooling combined with 1×1 convolutions to perform global context compression and channel recalibration on the feature map, capturing the global response intensity and semantic prior constraints of fire points across the entire image. This process can be represented as: in This represents the features after modeling the first branch. This indicates the characteristics of the output of the CG-RSU4 module. Indicates global average pooling. Represents a 1×1 convolution mapping. This indicates bilinear interpolation to the original spatial dimensions.

[0033] The second branch is a direction-aware branch based on strip pooling. It performs directional aggregation of features along the horizontal (0°), vertical (90°), and diagonal (45°) directions, respectively. By modeling elongated spatial dependencies through depthwise separable one-dimensional convolutions, it effectively enhances the network's ability to perceive features extending directionally along wind direction and surface structure constraints, while suppressing interference from unstructured noise in the background region. Furthermore, in the direction-aware branch, strip responses from different directions are normalized using a sigmoid function and then fused, with learnable gating parameters introduced. This achieves adaptive modulation of directional attention. The design maintains network stability during early training and progressively enhances the guiding role of directional context on fire point features during convergence. The perception process of this branch can be represented as: in This represents the features after the second branch is modeled. For learnable gating parameters, This represents the set of directions for modeling, and its value is... ; This represents the Sigmoid activation function; express One-dimensional convolution per channel in the direction; Along the strip direction Adaptive average pooling operation; Let be the rotation transformation matrix.

[0034] The third branch employs local average pooling combined with convolution operations to supplement the modeling of local contextual information within the fire point's neighborhood, thus compensating for the shortcomings of global and directional modeling in depicting local details. This process can be represented as: in This represents the features after modeling the third branch. This indicates local average pooling.

[0035] Furthermore, the output features of the three branches The features are concatenated along the channel dimension and compressed and fused using 1×1 convolutions to form a comprehensive representation that is both orientation-aware and multi-scale context-complementary. Finally, the residual connections are used to add the original features to output the final feature. This effectively enhances feature representation capabilities without compromising the original discriminative information. This process can be represented as: .

[0036] In another technical solution, the multi-scale Mamba attention module includes a spatial attention weighted branch, a Mamba global modeling branch, and a fusion unit; The spatial attention weighted branch downsamples the features of adjacent high-level structures and generates a first spatial attention map through the spatial attention mechanism. It upsamples the features of adjacent low-level structures and generates a second spatial attention map through the spatial attention mechanism. The first spatial attention map is then multiplied element-wise with the current scale features to obtain the first weighted feature. The second spatial attention map is then multiplied element-wise with the current scale features to obtain the second weighted feature. The Mamba global modeling branch reconstructs the current-scale features into a sequence, which is then processed by a linear mapping layer, a state-space model operator, and an activation function before being reconstructed back into a two-dimensional feature space to obtain the global modeling features. The fusion unit overlays the current scale features, the first weighted features, the second weighted features, and the global modeling features, and inputs the overlaid features into the residual bottleneck structure for feature fusion and channel adjustment, outputting the fused multi-scale features.

[0037] The multi-scale Mamba attention module, a key component embedded in the skip connection path between the encoder and decoder, internally comprises a spatial attention weighting branch, a Mamba global modeling branch, and a fusion unit. These three parts work together to achieve long-range dependency modeling of cross-scale features and enhance global consistency. This module receives three inputs: feature maps output by the current-scale encoder, feature maps output by neighboring higher-level encoders, and feature maps output by neighboring lower-level encoders. These feature maps have different spatial resolutions and semantic levels, providing rich multi-scale information. The spatial attention weighting branch first downsamples the neighboring higher-level feature maps to make their spatial dimensions similar to those of the current-scale features. Figure 1 The downsampled feature map is used to generate a first spatial attention map through a spatial attention mechanism. Simultaneously, upsampling is performed on adjacent low-level feature maps. The upsampled feature map is then used to generate a second spatial attention map through the same spatial attention mechanism. The first spatial attention map is then multiplied element-wise with the current-scale feature map to obtain a first weighted feature, which highlights fire candidate regions guided by higher-level semantics. The second spatial attention map is then multiplied element-wise with the current-scale feature map to obtain a second weighted feature, which enhances the fine structure of fire points guided by low-level details.

[0038] The Mamba global modeling branch is responsible for capturing long-range spatial dependencies in a large-scale scene. This branch first reconstructs the feature map at the current scale, converting it from a two-dimensional form to a sequence. The sequence is then processed sequentially: first, a linear mapping layer projects the feature vectors to an appropriate dimensional space; then, a state-space model operator is used for sequence modeling. This operator captures the dependencies between any two positions in the sequence within linear complexity while maintaining sensitivity to long-range information. Next, non-linearity is introduced through an activation function such as SiLU; finally, another linear mapping layer is applied. The processed sequence is then reconstructed back to its original two-dimensional form, yielding the global modeling features. This process enables the module to perceive the spatial relationships between multiple tiny fire points that are far apart in the image, ensuring they have consistent response strengths in the segmentation results.

[0039] The fusion unit is responsible for integrating the outputs of the two branches with the original features. Specifically, it adds the current-scale feature map, the first weighted feature, the second weighted feature, and the global modeling feature element-wise to obtain the fused multi-scale feature map. The superimposed feature map is then input into a residual bottleneck structure for further processing. The residual bottleneck structure typically consists of two 1×1 convolutional layers and one 3×3 convolutional layer. The number of channels in the intermediate layers is first compressed and then expanded to reduce computational cost, while residual connections are introduced to ensure smooth gradients. After processing by the residual bottleneck structure, the final fused multi-scale feature map is output. This feature map retains the local details of the original scale while incorporating contextual information from adjacent scales and global long-range dependencies.

[0040] In complex contexts, fire points exhibit a sparse distribution. To further enhance the network's ability to model the global consistency of multiple minimal fire points, a novel multi-scale Mamba attention module (M2AM) is designed. Figure 1 The M2AM shown is embedded between the encoder and decoder. By fusing features from adjacent high and low layers, a state-space modeling-based Mamba module is introduced to capture long-range spatial dependencies, thereby enhancing the ability to perceive the correlation of minimal fire points in a wide range of scenes.

[0041] like Figure 4 As shown in section (a), M2AM mainly consists of three parts: a spatial attention weighting branch, a Mamba global modeling branch, and a feature fusion layer. The spatial attention weighting branch is based on features at the current scale. With this as the core, the characteristics of adjacent high-rise buildings are introduced. With low-level features Spatial attention guidance enables effective interaction of cross-scale contextual information. Specifically, spatial alignment is achieved through upsampling or downsampling operations, and spatial attention mechanisms are used to adaptively emphasize potential fire points and suppress interference from complex backgrounds. This process can be represented as follows: in and These represent upsampling and downsampling, respectively. This represents the spatial attention mechanism. This is for element-wise multiplication.

[0042] The Mamba global modeling branch aims to model the spatial dependencies of tiny fire points in a large-scale scene using a state-space model. First, to model the spatial dependencies of tiny fire points in a large-scale scene, two-dimensional features are used... The mapping is reconstructed into a sequence form: Then, a Mamba module based on state-space modeling is introduced to capture long-range spatial dependencies, represented as: in Represents a linear mapping layer. Operators representing state-space models This represents the SiLU activation function. This represents the Hadamard product.

[0043] Finally, the output features are reconstructed back into the two-dimensional feature space. This process can be represented as follows: In the feature fusion layer, M2AM will combine the current features Cross-scale attention features and global dependency modeling results The features are superimposed and fused using a lightweight residual bottleneck structure (Res-Bottleneck) for channel adjustment, resulting in more discriminative multi-scale features. , is represented as: in This indicates the residual bottleneck fusion module.

[0044] Furthermore, considering the special characteristics of the first and fifth layer network structures, such as Figure 4As shown in sections (b) and (c), simplified versions of M2AM-1 and M2AM-5 were further designed for feature fusion scenarios that only include the input of the next stage or the previous stage, respectively, so as to ensure model performance while taking into account structural flexibility and computational efficiency.

[0045] When the multi-scale Mamba attention module is located at the first level of the encoder, the spatial attention weighting branch only upsamples the adjacent low-level features to generate the second spatial attention map, and the features superimposed by the fusion unit do not include the first weighted feature; when the multi-scale Mamba attention module is located at the fifth level of the encoder, the spatial attention weighting branch only downsamples the adjacent high-level features to generate the first spatial attention map, and the features superimposed by the fusion unit do not include the second weighted feature.

[0046] In the first layer of the encoder, since there are no higher-level feature maps as input, the spatial attention weighting branch cannot generate the first spatial attention map and the corresponding first weighted feature. To address this, the module is configured to generate the second spatial attention map by upsampling only adjacent lower-level feature maps, and then use only the second spatial attention map to element-wise multiply with the current-scale feature map to obtain the second weighted feature. Accordingly, when the fusion unit performs feature stacking, the features involved in the stacking only include the current-scale feature map, the second weighted feature, and the global modeling feature output by the Mamba global modeling branch, excluding the first weighted feature. This simplified design avoids computational errors caused by missing input while still retaining the ability to obtain supplementary information from lower-level details, ensuring the richness of the network's first-layer features. In the fifth layer of the encoder, since there are no adjacent lower-level feature maps as input, the spatial attention weighting branch cannot generate the second spatial attention map and the corresponding second weighted feature. At this time, the module is configured to generate the first spatial attention map by downsampling only adjacent higher-level feature maps, and then use only the first spatial attention map to element-wise multiply with the current-scale feature map to obtain the first weighted feature. When the fusion unit performs feature overlay, the features involved in the overlay only include the current scale feature map, the first weighted feature, and the global modeling feature output by the Mamba global modeling branch, excluding the second weighted feature. This configuration allows the fifth-layer features to still be guided by high-level semantics, enhancing the understanding of the global context.

[0047] In another technical solution, the decoder has multiple bypass outputs, each of which corresponds to a feature map of a decoding level. The system also includes a deep supervision module, which calculates the mixing loss between the prediction result of each bypass output and the real fire point mask, and weights the mixing loss of each bypass output according to a preset decreasing weight to obtain the total loss. The mixing loss consists of binary cross-entropy loss and IoU loss, and the decreasing weight decreases sequentially as the decoding level corresponding to the bypass output increases.

[0048] The decoder generates multiple bypass outputs during the progressive upsampling process. Each bypass output corresponds to a feature map of a decoding layer. These feature maps are converted into fire probability maps with the same resolution as the input image by a segmentation head. The decoder can be configured with six bypass outputs. The deep supervision module is responsible for calculating the mixing loss between the prediction result of each bypass output and the real fire mask. The mixing loss consists of two parts: binary cross-entropy loss and IoU loss. The binary cross-entropy loss calculates the difference between the predicted probability and the real label on a pixel-by-pixel basis, focusing on pixel-level classification accuracy and effectively supervising the correct classification of each pixel. The IoU loss calculates the intersection-union ratio between the predicted fire region and the real fire region, measuring the segmentation quality from the perspective of the entire region. It is particularly effective for fire targets with very small areas because it directly optimizes the degree of region overlap rather than individual pixels.

[0049] The deep supervision module further weights and sums the mixed losses of each bypass output according to preset decreasing weights to obtain the total loss for backpropagation training. The decreasing weights follow the principle of decreasing with increasing decoding level. Shallow outputs have higher spatial resolution and contain richer detailed information, which is crucial for accurate fire point localization, so they are given larger weights to enhance fine-grained localization capabilities. Deep outputs have lower resolution but contain more semantic information, which assists in the overall fire point discrimination, so they are given smaller weights to avoid dominating the training direction while maintaining training stability.

[0050] To alleviate the severe foreground-background imbalance caused by sparsely pixelated fire targets, a multi-scale weighted hybrid loss function is adopted as the optimization objective of the network. This loss function consists of binary cross-entropy (BCE) loss and IoU loss, which constrain the network training from two levels: pixel-level discrimination and region-level consistency, respectively.

[0051] Among them, BCE loss focuses on pixel-level supervision, encouraging the network to accurately distinguish between fire point pixels and background pixels, which helps learn a reliable fire point probability response. However, in satellite fire point scenarios, background pixels dominate, and using BCE alone can easily lead to the model being biased towards the background region, thus weakening its sensitivity to extremely small fire points. To alleviate this problem, IoU loss is introduced to directly optimize the spatial overlap between the predicted fire point region and the ground truth annotation. Compared to pixel-level loss, IoU loss focuses more on the overall structure of the target, and is particularly effective for fire point targets with extremely small areas and sparse spatial distribution. Considering both pixel-level accuracy and region-level consistency, the supervision loss at a single scale can be expressed as: in, Represents BCE loss, Represents IoU loss, This represents the output of the i-th layer of the network. G This represents the corresponding real fire point mask. This is the Sigmoid activation function.

[0052] Furthermore, considering the high sensitivity of minimal fire points to fine-grained spatial localization, a deep supervision mechanism is employed to jointly optimize the network's multi-scale outputs, assigning decreasing weights to outputs at different levels. Specifically, shallow features have higher spatial resolution and are therefore assigned greater weights to enhance fine-grained fire point localization; while deep features focus on high-level semantic representation, and their weights decay layer by layer to maintain training stability. Therefore, the final overall loss function is defined as: .

[0053] In another technical solution, the satellite remote sensing imagery uses multispectral images from the Landsat-8 satellite, with the input bands being a combination of bands 7, 6, and 5.

[0054] In another technical solution, during strip pooling along each direction in the direction-aware branch, multiple strip lengths at different scales are used, with each scale corresponding to a set of strip pooling operations, generating a direction attention map at that scale. The direction attention maps at multiple scales in the same direction are concatenated along the channel dimension and weighted and summed using a learnable fusion weight matrix to obtain the fused direction attention map for that direction. The learnable fusion weight matrix is ​​updated through backpropagation during training to adaptively adjust the contribution of features at different scales to the final direction-aware features. The fused direction attention maps from each direction are then superimposed to output multi-scale direction-aware features.

[0055] The orientation-aware branch incorporates multiple parallel strip pooling units, each corresponding to a preset strip length. These strip lengths are set based on the height of the input feature map, specifically taking values ​​of 1 / 4, 1 / 2, or equal to the input feature map height, corresponding to short, medium, and long strip scales, respectively. Short strip pooling units focus on directional textures within a local area, capturing the directional features of fire edges or small branches. Medium strip pooling units cover a medium-sized area, suitable for modeling fire lines or smoke spread bands of general width. Long strip pooling units span the entire feature map height, capturing continuous directional structures across the entire image, such as long strip fire lines formed by strong winds. Each strip pooling unit performs strip pooling operations along the horizontal, vertical, and diagonal directions. Specifically, the feature map is first rotated to the corresponding direction, then pooled using a strip pooling kernel of the specified length, followed by enhanced directional context modeling through depthwise separable one-dimensional convolution, and finally, an attention map at that scale is generated using the sigmoid function. Thus, for each direction, three attention maps at different scales are generated, reflecting the probability of fire point distribution in different spatial ranges along that direction. The directional attention maps output by multiple strip pooling units are concatenated along the channel dimension to form a joint feature representation containing information from all directions and scales. The concatenated feature map is then fed into a 1×1 convolutional layer for convolutional fusion. This convolutional layer adaptively weights and combines the attention maps of different directions and scales using learnable weight parameters, thereby generating multi-scale direction-aware features. This fusion process allows the network to automatically select the most appropriate scale combination based on the specific scene; for example, it may rely more on long strip features for thin fire lines, while it may rely more on short strip features for discrete small fire points. The fused multi-scale direction-aware features are then fed into the fusion unit along with global and local context features for further processing. In actual operation, when the input feature map passes through the orientation-aware branch, it first passes through multiple strip pooling units in parallel. Each unit independently calculates the attention map at its own scale, and then the maps are concatenated and fused by convolution. Finally, an orientation-aware feature containing multi-scale orientation information is output, which is sensitive to the directional expansion of the fire point at different widths.

[0056] The main solution of this application is to achieve extremely accurate identification of very small fire points, reaching an identification effect that is difficult to achieve with conventional techniques. This is also a major application scenario of this application. In another technical scenario, after accurately identifying very small fire points, we can also perform connected component analysis to further segment the fire points into regions, so as to facilitate the subsequent prediction of fire combustion patterns. Connected component segmentation is performed using a post-processing module. The post-processing module performs connected component analysis on the fire point segmentation results output by the decoder, marks all connected regions and calculates the number of pixels in each connected region. Connected regions with a pixel count less than a preset threshold are removed from the segmentation results as noise regions. Morphological closing operations are performed on the fire point regions retained after noise removal. The morphological closing operation uses circular or square structuring elements to fill the holes inside the fire point regions and connect adjacent broken fire point regions. The post-processing module outputs the optimized fire point segmentation results after noise removal and morphological closing processing.

[0057] The post-processing module consists of two main components: a connected component analysis unit and a morphological processing unit. The connected component analysis unit first receives the fire point segmentation probability map output from the decoder. In this probability map, each pixel's value is between 0 and 1, representing the probability that the pixel belongs to a fire point. For connected component analysis, the probability map needs to be binarized. Typically, a preset threshold is used to convert the probability map into a binary image; for example, the threshold can be set to 0.5, meaning pixels with a probability greater than 0.5 are marked as fire points, otherwise they are marked as background. After binarization, the connected component analysis unit performs connected component labeling on the binary image. A common method is to use the eight-neighbor connectivity criterion, traversing all pixels in the image, grouping interconnected fire point pixels into the same connected region, and assigning a unique identifier to each region. Then, the number of pixels contained in each connected region is counted, i.e., the area of ​​the region. For connected regions with fewer than the preset threshold of pixels, they are identified as noise regions, and all pixel values ​​in these regions are set to zero, thus removing them from the segmentation result. The specific value of the preset threshold can be set according to the spatial resolution of the satellite imagery and the actual scale of the fire point. For example, for Landsat imagery with a resolution of 30 meters, the fire point usually only occupies a few pixels, so the threshold can be set to 3 to 5 pixels. Isolated noise points smaller than this value are very likely to be false detections. For higher resolution imagery, the threshold can be increased appropriately. This step effectively filters out sporadic false detections caused by complex backgrounds.

[0058] The morphological processing unit performs morphological closing operations on the image processed by the connected component analysis unit. The closing operation consists of two steps: dilation and erosion, using structuring elements of the same size, typically circular or square. Circular structuring elements better preserve isotropic morphological features and are suitable for targets like fire points without a strong directional preference; square structuring elements are computationally simpler. The size of the structuring element needs to be pre-set based on the spatial resolution of the satellite image. For example, for a 30-meter resolution Landsat image, a circular structuring element with a radius of 3 pixels can be selected, which corresponds to approximately a 90-meter spatial range, sufficient to connect fire point regions broken due to weak responses without excessively merging separated fire points. The dilation operation assigns the maximum pixel value within the structuring element's coverage area to the central pixel, thus expanding the fire point region outward, filling small internal holes and connecting adjacent broken areas. The erosion operation removes the excess edges resulting from the expansion, restoring the region to a near-original size with filled holes and connected breaks. Through closing operations, the tiny pores inside the fire point region are filled, and adjacent but slightly broken fire point regions are connected into a continuous whole, thus significantly improving the morphological integrity of the segmentation result. The post-processing module finally outputs the optimized fire point segmentation result after noise removal and morphological closing processing, which can be directly used for subsequent fire combustion trend prediction.

[0059] It is worth noting that although connected component identification and fire point segmentation typically add some non-fire point pixels to the fire point region and segment some fire point pixels into the non-fire point region, this does not affect the accurate identification of extremely small fire points achieved by the technical solution provided above in this application. Based on the identified extremely small fire point information, the system can complete independent fire point judgment. Therefore, the processing effect of the subsequent processing module benefits from the previously completed extremely small fire point judgment, achieving optimal connected component partitioning without affecting the accuracy of extremely small fire point identification. The post-processing module removes small-area noise points through connected component analysis and fills holes and broken connection regions through morphological closing operations. Compared with the segmentation schemes of existing technologies, this scheme can significantly improve the purity and morphological integrity of the segmentation results, effectively avoiding sporadic false detections and fire point breaks caused by imperfect model predictions. This makes the final output more consistent with the spatial distribution characteristics of actual fire points, improving the reliability and usability of the system in practical applications.

[0060] To further illustrate the effectiveness of the system design proposed in this application, experiments were conducted using two subsets of the Active Fire dataset with significant pixel-level sparsity: Oceania and Asia4. These datasets are imaged from the Landsat-8 satellite OLI sensor, covering 11 multispectral bands including visible, near-infrared, and short-wave infrared, with a spatial resolution of 30m. Due to resolution limitations, fire points typically occupy only a very small number of pixels in the images, exhibiting a clear spatial sparsity characteristic. The Oceania and Asia4 datasets contain 2200 and 4900 images, respectively. To better illustrate the distribution of fire points, fire point pixels were statistically analyzed across five intervals (0-5, 6-10, 11-20, 21-50, >50). The statistical results... Figure 5 and Figure 6 As shown.

[0061] like Figure 5 As shown, in the Oceania dataset, 1800 images (81.8%) have fewer than 5 fire point pixels, while only 205 images (approximately 9.3%) have more than 20 fire point pixels, indicating a significant imbalance in the distribution of fire points across the dataset. Similarly, as... Figure 6 As shown, in the Asia4 dataset, approximately 82.3% of the images (4033 images) contain fewer than 5 fire pixels, while only 18 images (0.4%) contain more than 50 fire pixels, exhibiting a more pronounced imbalance. Furthermore, although both datasets predominate in small fire targets, there are still differences in the distribution of fire size. The Asia4 dataset contains more medium-sized fires (6–20 pixels), while the Oceania dataset contains relatively more large fires (>20 pixels). This cross-regional difference further increases the difficulty of model generalization. Overall, the fire distribution in both subsets reflects the small and sparse characteristics of fires in satellite imagery, effectively validating the model's performance.

[0062] Experiments were conducted on the Oceania and Asia4 datasets. Each dataset was split into training, validation, and test sets in a 7:1:2 ratio. All network models were implemented on a single NVIDIA GTX 4070s GPU using the PyTorch framework. The Adam optimizer was used during network training, with an initial learning rate of 0.005 and a weight decay factor of 1×10⁻⁶. -4 The batch size was set to 4. Training lasted for 100 epochs, and a multinomial learning rate decay strategy was used to dynamically adjust the learning rate during training.

[0063] The model's performance is evaluated using two metrics: Intersection over Union (IoU) and F1 score. IoU assesses the spatial consistency between the predicted fire zone and the measured ground mask, while F1 score comprehensively measures the model's accuracy and detection capability in segmenting fire points. The specific calculation formulas are as follows: Where TP represents the pixel correctly predicted as a fire point, FP represents the pixel incorrectly predicted as a fire point, and FN represents the missing fire point pixel.

[0064] To further verify the effectiveness of the proposed system, we selected a set of relatively advanced existing segmentation methods for comprehensive comparison, including: 1) CNN-based segmentation networks: UNet, U2Netp, PSPNet, CorrNet, SeaNet, FPSU2Net; 2) Transformer-based segmentation networks: SegFormer, PGNet; 3) Mamba-based segmentation networks: AfaMamba, P-Mamba.

[0065] Table 1 shows the quantitative experimental results of the system design in this application compared with other existing networks on the Oceania dataset. Compared with the best CNN-based segmentation network (FPSU2Net), the system design in this application improves IoU by 1.81% and F1 score by 0.80%, indicating that the constructed network has stronger advantages in terms of overall consistency of fire point regions and pixel-level classification accuracy.

[0066] Compared to Transformer-based segmentation networks (such as SegFormer and PGNet), its IoU and F1 scores are significantly lower than the system design adopted in this application. The Transformer architecture focuses on global modeling capabilities and has advantages in modeling large-scale targets and contextual relationships. However, when faced with a large number of extremely small-scale fire targets with very low pixel proportions in the Oceania dataset, global features can easily overwhelm local salient information, resulting in limited ability to identify tiny fire points.

[0067] Furthermore, compared to Mamba-based segmentation networks (such as AfaMamba and P-Mamba), although Mamba outperforms Transformer in terms of long sequence modeling and feature propagation efficiency, its overall performance is still lower than the design in this application, but significantly better than the pure Transformer architecture. This indicates that the Mamba structure alleviates the problem of global modeling being unfriendly to small targets to some extent, but it still has shortcomings in characterizing fire point boundaries and fine-grained structures in complex backgrounds.

[0068] From the perspective of network backbone structure, the CG-RSU backbone adopted in this application achieves more comprehensive multi-scale feature fusion while maintaining a smaller number of network layers and parameter scale, effectively enhancing the perception ability of local fire point morphology and weak response regions. This enables the network to have better stability and generalization performance while ensuring accuracy.

[0069] Table 1. Comparison Experiment Results of Oceania Dataset To better compare the network's performance in segmenting fire points at different scales, such as Figure 7 As shown, scenes with fire points of different scales and shapes from the Oceania dataset were selected for demonstration. The fire point pixel size gradually decreased from pixel = 120 to pixel = 4. Figure 7 As shown in (a) and (b), for fire point samples with larger pixel sizes (pixels = 120 and 93), most CNN-based methods can detect the main fire point region, but they are insufficient in maintaining the continuity of the elongated structure, and the prediction results generally show breaks or local missed detections. Furthermore, the prediction results of segmentation networks based on Transformer and Mamba (SegFormer, AfaMamba, and P-Mamba) contain many false predictions. In contrast, the design in this application can better maintain the overall connectivity of the fire points, and the prediction results are highly consistent with the ground truth annotations in terms of morphology and spatial distribution, with a significant reduction in false detection pixels.

[0070] like Figure 7 As shown in (c) and (d), the differences between the different methods become more pronounced when the fire point scale is reduced to a medium scale (pixels = 57 and 16). Some methods (such as SeaNet and FPSU2Net) produce a large number of missed pixels under complex background interference, resulting in incomplete fire point structures. Mamba-based methods (AfaMamba and P-Mamba) exhibit large and obvious false detections in local areas, severely affecting detection accuracy. Our proposed scheme can still accurately characterize the fire point contour at this scale and maintain a good balance between false detections and missed detections.

[0071] like Figure 7 As shown in (e) and (f), in the most challenging extremely small-scale fire scene (pixels = 12 and 4), most comparison methods exhibit varying degrees of missed detections, with some methods (such as UNet and SeaNet) even showing complete missed detections. In contrast, the Mamba-based method and the system design of this application can stably identify fire locations, and the system design of this application can predict almost perfectly.

[0072] Overall, the visualization results on the Oceania dataset show that the proposed scheme has significant advantages in maintaining the continuity of fire points in slender structures, detecting small-scale targets, and suppressing false detections in complex backgrounds, further verifying its robustness and effectiveness under different fire point morphologies and scales.

[0073] In summary, the experimental results fully verify the effectiveness and superiority of the technical solution proposed in this application for fire point segmentation tasks at different scales, and it is especially suitable for application scenarios in the Oceania dataset where complex backgrounds and extremely small fire points coexist.

[0074] To better evaluate the model's performance, as shown in Table 2, we also quantitatively compared the proposed method with several representative segmentation networks on the Asia4 dataset. Our application also achieved state-of-the-art performance in both IoU and F1, reaching 85.65% and 92.26% respectively, further validating the effectiveness of this system design under different regions and data distributions. Compared to the high-performing CNN-based segmentation network FPSU2Net, our approach improves IoU by 2.07% and F1 score by 1.21%. This performance improvement is more significant compared to the Oceania dataset, indicating stronger robustness in preserving the integrity of fire points and suppressing false detections. Compared to Transformer-based segmentation networks (SegFormer, PGNet), their performance further declines on the Asia4 dataset, with significantly lower IoU and F1 scores. This indicates that in the Asia4 dataset, where there are numerous fire targets with high background interference, low contrast, and drastic scale changes, the Transformer architecture, which relies on global modeling, struggles to effectively highlight the local salient features of the fire targets, thus limiting its segmentation performance. For Mamba-based segmentation networks, such as AfaMamba and P-Mamba, their overall performance falls between CNN and Transformer methods, exhibiting good stability, but they still significantly lag behind this application in IoU and F1 scores.

[0075] Table 2 Comparison Experiment Results of Asia4 Dataset To better evaluate the model's performance, we selected different scenes at different scales with multiple fire points for demonstration. The results are as follows: Figure 8 As shown. Figure 8As shown in the diagram, in different scenarios with multiple fire points at different scales, CNN-based methods (such as UNet and SeaNet) have a large number of blue pixels, indicating that these methods have a serious problem of missing fire points. Furthermore, the missed fire point rate increases as the number of fire point pixels decreases. Transformer-based methods also have a certain missed fire rate, although it is lower than that of CNN-based methods. Mamba-based methods have fewer blue pixels but more red pixels, indicating that while their missed fire point rate is low, their false positive rate is high, making it difficult to identify extremely small fire point targets. In contrast, our system design uses mostly yellow pixels and exhibits stable performance across different scales and scenarios with multiple fire points. For a single fire point, such as... Figure 8 As shown in the datasheet, CNN-based and Transformer-based methods exhibit significant false negatives, even failing to effectively distinguish between fire points and complex backgrounds. While U2Net and FPSU2Net have lower false negative rates, they have higher false positive rates compared to our system design. Mamba-based methods suffer from both false positives and false negatives in this scenario, making it difficult to reliably identify fire targets. In contrast, our system design demonstrates more stable performance, such as... Figure 8 As shown in d, the system design of this application can completely separate fire points.

[0076] In summary, in complex backgrounds and scenarios with multiple fire points at different scales, the system design of this application exhibits better robustness and more refined segmentation capabilities. Furthermore, this application not only demonstrates superior performance in overall statistical metrics but also shows significant advantages in pixel-level detail and the recognition of multiple extremely small targets.

[0077] To verify the contribution of each key module to the satellite fire detection performance, ablation experiments were conducted on the CG-RSU, FDCM, and M2AM modules, as well as ablation experiments on different bands of the Landsat 8 satellite. The specific design of the ablation experiments is as follows: To verify the effectiveness of each module in the system design of this application, eight ablation experiments (No. 1-No. 8) were conducted on the Oceania and Asia4 datasets, and the results are shown in Table 3. Furthermore, to better demonstrate the effectiveness of each module, we selected two representative images to display the feature maps from the ablation experiments, and the results are shown below. Figure 9 As shown. Visualization results of multi-stage feature representation under different module ablation configurations, where... Figure 9 (a) to (h) correspond to the feature heatmaps of the models under the module combinations of groups 1 to 8 in Table 3, respectively.

[0078] As shown in Table No. 1, the baseline model (No. 1) achieved IoU of 83.91% and 81.16% on the Oceania and Asia4 datasets, respectively. Building upon this, as shown in Tables 3 Nos. 2-4, by individually adding the CG-RSU, M2AM, and FDCM modules, the model achieved varying degrees of performance improvement on both the Oceania and Asia4 datasets, with IoU improvements ranging from 1.9% to 2.6%. Figure 9 As shown in the ad, compared to the baseline model, the feature map can clearly identify fire points after adding the modules individually, verifying the effectiveness of each module in multi-scale stable modeling, long-range dependency perception, and orientation-aware context compensation.

[0079] Furthermore, Table 3, numbers 5-7, presents the experimental results for different combinations of dual modules. Combined with... Figure 8 As can be observed in the example, for scenarios with multiple fire points, any combination of two modules outperforms a single-module configuration, identifying more fire points. This indicates a good complementarity between multi-scale stable modeling, long-range dependency perception, and structure-aware modulation. Specifically, as shown in Table 3, No. 6, the combination of CG-RSU and FDCM (No. 6) improved IoU by 3.94% on the Oceania dataset. The combination of M2AM and FDCM (as shown in Table 3, No. 7) performed even better on the Asia4 dataset, improving IoU by 3.70%, demonstrating that orientation-aware contextual information has stronger generalization ability in complex regional scenes.

[0080] When all three modules are introduced simultaneously (No. 8), the overall model performance reaches its optimum, achieving 88.51% IoU and 93.76% F1 on the Oceania dataset, and 85.65% and 92.26% respectively on the Asia4 dataset. Compared to the baseline model, the proposed system design improves IoU by 4.60% and 4.49% respectively. In summary, the modules have significant synergistic gain effects in the fire detection task, effectively improving the model's robustness and generalization ability in complex backgrounds and cross-regional scenarios.

[0081] Table 3 Module Ablation Experiment Results Since FDCM plays a crucial role in feature modulation within the network, its insertion position can significantly impact overall performance. Therefore, this application further conducted ablation experiments on different insertion positions of FDCM in the network (stages 1-5), and the results are shown in Table 4. As shown in Table 4, when FDCM is placed in a shallow layer (stage 1), model performance significantly decreases, with an IoU of only 42.95% on the Oceania dataset, indicating that orientation-aware context is difficult to effectively model high-level semantic information in low-level feature stages. As FDCM gradually moves to the middle layers, model performance significantly improves, with stage 4 achieving the best results on both datasets, reaching IoUs of 88.51% and 85.65% on the Oceania and Asia4 datasets, respectively. In contrast, when FDCM moves further to a higher layer (stage 5), performance declines to some extent, indicating that features at deeper layers tend to converge semantically, limiting the gain of orientation-aware modulation. Therefore, this application deploys FDCM in stage 4 to fully leverage its crucial feature modulation role.

[0082] Table 4. Results of FDCM site ablation experiments To analyze the impact of different spectral bands on fire detection performance, this application conducted ablation experiments with multi-band combinations on the Oceania and Asia4 datasets. The experimental results are shown in Table 5. As can be seen from the table, the contributions of different bands to fire detection performance vary significantly. Specifically, as shown in No. 1 of Table 5, the model performance is extremely low when only band 432 is used, with IoU of only 15.79% and 11.25% on the Oceania and Asia4 datasets, respectively, indicating that relying solely on visible light information is insufficient to effectively distinguish fire points from complex backgrounds. In contrast, as shown in Nos. 3-7 of Table 5, the model performance is significantly improved after introducing near-infrared (B5) and short-wave infrared (B6, B7) bands. In particular, the combination of bands 7, 6, and 5 demonstrated stable and excellent detection capabilities on both datasets. Specifically, the 7, 6, and 5 band combination achieved 88.51% IoU and 93.76% F1 on the Oceania dataset, and 85.65% IoU and 92.26% F1 on the Asia4 dataset, exhibiting the best overall performance. Furthermore, as shown in Table 5, No. 8, using all bands (ALL) for joint modeling may introduce some redundant or weak discriminative information, resulting in no significant improvement in the overall performance of the network model. Therefore, this application selects bands 7, 6, and 5 as the optimal band combination for fire detection.

[0083] Table 5 Ablation experimental results of different band combinations To further evaluate the model's performance in terms of computational complexity and inference efficiency, this application conducted comparative experiments on several representative semantic segmentation models under a unified experimental environment. All models were tested for inference on an NVIDIA RTX 4070S GPU with an input image size of 256×256. The experimental results are shown in Table 6.

[0084] In terms of model size and computational complexity, traditional CNN methods such as UNet and PSPNet have a large parameter size, reaching 39.42 M and 52.49 M parameters respectively. Although they can achieve high inference frame rates on GPUs, their model redundancy is quite significant. In contrast, lightweight networks such as SeaNet, U2Netp, and FPSU2Net significantly reduce the parameter size, with U2Netp and FPSU2Net both having around 1.5 M parameters. However, they still have some limitations in terms of complex feature modeling capabilities. Transformer-based or hybrid architecture methods (such as SegFormer, PGNet, and P-Mamba) have advantages in modeling global context, but their parameter size or inference efficiency varies considerably. For example, PGNet has a parameter size as high as 72.71 M, while P-Mamba, although having lower computational cost, still suffers from some impact on inference speed.

[0085] In comparison, the system design proposed in this application achieves a more balanced performance between model size, computational complexity, and inference efficiency. Specifically, the proposed model is based on the CG-RSU architecture, with only 1.56 M parameters and a computational cost of 16.10 GF1OPs, achieving an inference speed of 42.80 FPS on an RTX4070S. While maintaining low model complexity, the model still possesses stable inference efficiency, indicating that the system design in this application achieves a good trade-off between lightweight design and feature representation capabilities, and has high potential for practical deployment.

[0086] Table 6 Comparison of Model Complexity and Size As described in the preceding descriptions of the Oceania and Asia4 datasets, satellite fire point segmentation tasks in practical applications generally face challenges such as extremely small target scale, imbalanced pixel distribution, and significant regional differences. Against this backdrop, as shown in Tables 1 and 2, this application achieves stable and leading performance on both datasets, demonstrating that the proposed network exhibits good robustness and generalization ability in complex backgrounds and cross-regional scenarios. Compared to traditional CNN methods, the CG-RSU backbone introduced in this application effectively alleviates the problem of detail loss caused by multiple downsampling through multi-scale stable feature fusion, enabling extremely small fire points and slender structures to be more fully expressed in high-level semantic features. In contrast, Transformer-based methods, due to their reliance on global modeling mechanisms, are easily dominated by large-area background information in scenes with pixel-level high imbalance, thus limiting their ability to identify fire points with weak local responses; while Mamba-based methods show certain advantages in long-range dependency modeling, but still have shortcomings in characterizing fire point boundaries and fine-grained structures under complex background conditions. This application introduces M2AM and FDCM modules to achieve effective synergy between long-range dependency perception and direction-aware context modeling, thereby significantly improving the model's stability and refined representation capabilities in fire point segmentation tasks at different scales. Furthermore, the band ablation experimental results shown in Table 5 further verify the key role of near-infrared and short-wave infrared bands in enhancing the separability of fire point and background spectra. Reasonable band combinations (such as the 765 band) are more beneficial to improving the overall segmentation performance of the model than simply increasing the input dimension.

[0087] In comparative experiments on the Oceania and Asia4 datasets, this application further conducted a detailed analysis of a subset of extremely small fire points (≤10 pixels). Figure 5 and Figure 6 As shown, over 80% of the images in both datasets contain fewer than 5 fire point pixels, while images containing 6 to 10 pixels also constitute a significant proportion, fully demonstrating the challenge of identifying extremely small fire points. Figure 7-9In the visualization results, for extremely small fire point samples with pixel sizes of only 4 and 12, most comparison methods (such as UNet, SeaNet, SegFormer, etc.) showed complete false negatives or large-area false positives, while the system of this application can accurately locate the fire point position and maintain morphological integrity. Especially in the extremely small target scene with 4 pixels, the system can still output segmentation results that are highly consistent with the real annotation, proving its ability to preserve extremely weak fire point signals. The ablation experiments in Table 3 further show that after introducing CG-RSU, FDCM and M2AM modules, the model's recognition performance for fire point targets as small as 5 pixels is significantly improved. By preserving shallow details through cross-layer gating mechanism, enhancing anisotropic features through orientation modulation, and maintaining sparse fire point consistency through Mamba global modeling, the system achieves the best IoU and F1 scores in the extremely small fire point segmentation task with ≤10 pixels, which is significantly better than existing CNN, Transformer and Mamba-like methods.

[0088] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A system for small fire point segmentation in satellite remote sensing imagery, characterized in that, include: The encoder consists of multiple cascaded cross-layer gated residual U-shaped modules. Each cross-layer gated residual U-shaped module is used to downsample the input feature map and extract multi-scale context features, and suppress background redundant response through cross-layer gated mechanism. A fire-oriented directional context modulation module, connected after at least one intermediate level of the encoder, is used to perform multi-directional context modeling on the features output by the encoder. The decoder consists of multiple decoding units corresponding to the encoder. Each decoding unit is used to upsample the input features and fuse them with the corresponding level features from the encoder. The multi-scale Mamba attention module, which is embedded in each hop connection path between the encoder and decoder, is used to fuse the current-scale encoder features and the features of the neighboring-scale encoders, and to model long-range spatial dependencies using a state-space model. The encoder receives satellite remote sensing images as input, and after feature extraction at each level, it is modulated by a fire-oriented directional context modulation module. The modulated features are then input to subsequent coding units and to a multi-scale Mamba attention module via skip connections. The decoder receives features from each level of the encoder via skip connections, processes them through the multi-scale Mamba attention module, upsamples them at each level, and fuses them to finally output the fire point segmentation result.

2. The system for small fire point segmentation in satellite remote sensing images according to claim 1, characterized in that, The cross-layer gated residual U-shaped module is internally equipped with an encoding sub-module and a decoding sub-module; The encoding submodule consists of multiple sequentially connected downsampling units, used to downsample the input feature map step by step and extract multi-scale contextual features; the decoding submodule consists of multiple sequentially connected upsampling units, each upsampling unit is used to upsample the features from adjacent higher-level upsampling units, and concatenate and fuse them with the features output by the corresponding level downsampling unit to obtain the fused features of the current layer; each upsampling unit is also used to sequentially process the fused features of the current layer with global average pooling, 1×1 convolution and sigmoid function to generate the initial gating weights of the current layer. In the decoding submodule, the upsampling unit of the adjacent higher layer passes its generated initial gating weight to the upsampling unit of the current layer, and the weight is superimposed with the initial gating weight of the current layer to become the final gating weight of the current layer. The upsampling unit of the current layer is also used to perform channel-level multiplication operation on the final gating weight and the fused feature of the current layer to obtain the modulated feature, and output the modulated feature to the upsampling unit of the adjacent lower layer or as the output of the cross-layer gating residual U-shaped module.

3. The system for small fire point segmentation in satellite remote sensing images according to claim 1, characterized in that, The fire-oriented direction context modulation module includes a global context branch, a direction-aware branch, a local context branch, and a fusion unit; The global context branch sequentially performs global average pooling, 1×1 convolution, and upsampling on the input features to output global context features. The orientation-aware branch performs strip pooling and depthwise separable one-dimensional convolution on the input features along the horizontal, vertical and diagonal directions to generate attention maps for each direction. The attention maps are then normalized by the Sigmoid function and superimposed. During superposition, weighted fusion is performed using learnable gating parameters to output orientation-aware features. The local context branch performs local average pooling and 1×1 convolution on the input features and outputs local context features. The fusion unit is used to concatenate global context features, orientation-aware features, and local context features along the channel dimension, and then add them element-wise with the input features after a 1×1 convolution to output the modulated features.

4. A system for segmenting extremely small fire points in satellite remote sensing images according to claim 1, characterized in that, The multi-scale Mamba attention module includes a spatial attention weighted branch, a Mamba global modeling branch, and a fusion unit; The spatial attention weighted branch downsamples the features of adjacent high-level structures and generates a first spatial attention map through the spatial attention mechanism. It upsamples the features of adjacent low-level structures and generates a second spatial attention map through the spatial attention mechanism. The first spatial attention map is then multiplied element-wise with the current scale features to obtain the first weighted feature. The second spatial attention map is then multiplied element-wise with the current scale features to obtain the second weighted feature. The Mamba global modeling branch reconstructs the current-scale features into a sequence, which is then processed by a linear mapping layer, a state-space model operator, and an activation function before being reconstructed back into a two-dimensional feature space to obtain the global modeling features. The fusion unit overlays the current scale features, the first weighted features, the second weighted features, and the global modeling features, and inputs the overlaid features into the residual bottleneck structure for feature fusion and channel adjustment, outputting the fused multi-scale features.

5. A system for segmenting minimal fire points in satellite remote sensing images according to claim 4, characterized in that, When the multi-scale Mamba attention module is located in the first level of the encoder, the spatial attention weighting branch only upsamples the adjacent low-level features to generate the second spatial attention map, and the features superimposed by the fusion unit do not include the first weighted features. When the multi-scale Mamba attention module is located at the fifth level of the encoder, the spatial attention weighted branch only downsamples the adjacent high-level features to generate the first spatial attention map, and the features superimposed by the fusion unit do not include the second weighted features.

6. A system for segmenting minimal fire points in satellite remote sensing images according to claim 1, characterized in that, The decoder has multiple bypass outputs, each of which corresponds to a feature map of a decoding level. The system also includes a deep supervision module, which calculates the mixing loss between the prediction result of each bypass output and the real fire point mask, and then sums the mixing losses of each bypass output according to a preset decreasing weight to obtain the total loss. The mixing loss consists of binary cross-entropy loss and IoU loss, and the decreasing weight decreases sequentially as the decoding level corresponding to the bypass output increases.

7. A system for segmenting minimal fire points in satellite remote sensing images according to claim 1, characterized in that, The satellite remote sensing imagery uses multispectral images from the Landsat-8 satellite, with input bands consisting of a combination of bands 7, 6, and 5.

8. A system for segmenting minimal fire points in satellite remote sensing imagery according to claim 3, characterized in that, In the direction-aware branch, strip pooling along each direction uses multiple strip lengths at different scales, with each scale corresponding to a set of strip pooling operations, generating a direction attention map at that scale. The direction attention maps at multiple scales in the same direction are concatenated along the channel dimension and weighted and summed using a learnable fusion weight matrix to obtain the fused direction attention map for that direction. The learnable fusion weight matrix is ​​updated through backpropagation during training to adaptively adjust the contribution of features at different scales to the final direction-aware features. The fused direction attention maps from each direction are then superimposed to output multi-scale direction-aware features.

Images