A SegFormer water body fine segmentation model fusing dynamic weights

Abstract

The application discloses a SegFormer water body fine segmentation model fusing dynamic weights, comprising: a mixed vision Transformer encoder module, which is used for layered feature extraction on input remote sensing images, and constructs a multi-scale feature pyramid from details to semantics; a dynamic weight fusion module connected after the mixed vision Transformer encoder module, which is used for receiving features of each layer of the multi-scale feature pyramid, and adaptively generating dynamic fusion weights according to the contribution of each scale feature to the current input image, weighting and fusing features of each layer after uniform spatial resolution, and outputting fusion features; a dual attention enhancement module connected after the dynamic weight fusion module, which is used for respectively recalibrating the channel dimension and the spatial dimension of the fusion features, multiplying the enhancement results of the two dimensions, and outputting final segmentation features. The application effectively improves the segmentation precision and detail retention capability of the model for multi-scale water bodies in complex scenes.

Description

Technical Field

[0001] This invention relates to the fields of remote sensing image processing and water segmentation technology, and in particular to a SegFormer water segmentation model that incorporates dynamic weights. Background Technology

[0002] Currently, with the deepening development of refined water resource management, high-precision and high-efficiency water body segmentation has become a core requirement for high-resolution remote sensing image water body segmentation tasks. Traditional water body segmentation methods, such as the Normalized Difference Water Index (NDWI), and machine learning models such as Support Vector Machines (SVM) and Random Forests, mainly rely on statistical analysis of spectral features. They are difficult to effectively handle complex issues of homonymous and heteronymous objects and mixed pixels among ground features, especially when faced with detail blurring, missing small water bodies, and complex background interference in high-resolution images, resulting in a significant decrease in segmentation accuracy. Although deep learning models such as U-Net and SegFormer have improved segmentation performance in general scenarios in recent years by mining deep semantic features, their inherent static multi-scale fusion strategies still have shortcomings such as insufficient adaptive perception capabilities and inadequate integration of detailed features and contextual information. They cannot meet the requirements of large-scale monitoring for refined and robust water body information extraction, as detailed below:

[0003] The Transformer architecture suffers from insufficient local detail preservation. Although models such as U-Net and SegFormer have become commonly used frameworks for water segmentation due to their encoder-decoder structures or multi-level attention mechanisms, their limitations are becoming increasingly apparent as the requirements for segmentation accuracy and generalization ability increase. While Transformer-type models can effectively capture long-range dependencies and global semantics, their ability to model and preserve local details and edge information in high-resolution remote sensing imagery remains insufficient, easily leading to blurred water body boundaries and irregular contours, especially at the junctions of small water bodies and complex terrain.

[0004] Multi-scale feature fusion strategies lack dynamic adaptability. Existing methods typically employ preset or static weights for multi-scale feature fusion, making it difficult to dynamically adjust based on the actual scale distribution and morphological features of water bodies in the input image. This rigid fusion mechanism results in a lack of targeted perception and modeling capabilities when facing targets of different scales, such as wide water bodies and small ditches. It fails to achieve an adaptive balance between deep semantic features and shallow detail features, leading to blurred boundaries or the omission of small water bodies, thus limiting the model's accuracy and robustness in complex geographical environments.

[0005] Attention mechanisms fail to coordinate the modeling of scale and detail features. Currently used attention mechanisms often operate independently on spatial or channel dimensions, lacking the ability to collaboratively enhance multi-scale features and detail information. Single-dimensional attention struggles to simultaneously address the scale differences and local detail features of water targets, resulting in a rigid multi-scale feature fusion process. This hinders accurate feature selection and fusion in complex scenarios, impacting the model's performance in discriminating the integrity of water structures and the clarity of boundaries.

[0006] In summary, while existing improved models based on multi-scale feature fusion have made progress in complex scene segmentation, they still have limitations in water body extraction applications: First, they struggle to effectively mitigate detail loss and boundary blurring caused by low-resolution source data or complex interference; second, segmentation results often result in the omission or misclassification of small water bodies; and third, they largely rely on preset or static fusion strategies, making it difficult to dynamically and adaptively integrate features based on the actual scale distribution and morphological characteristics of water bodies in the input image. These problems severely restrict the reliability of water body segmentation results in key scenarios such as water resource management and climate change research. Therefore, developing a novel segmentation model capable of adaptively fusing multi-scale features and enhancing detail preservation is a pressing technical challenge in the field of remote sensing image processing and water body segmentation. Summary of the Invention

[0007] The purpose of this invention is to provide a SegFormer water body fine segmentation model that integrates dynamic weights, which can effectively solve the problems of fuzzy details, omission of small water bodies, and insufficient multi-scale adaptive sensing capabilities in traditional water body segmentation, and significantly improve the automation, refinement and reliability of dynamic water resource monitoring and assessment.

[0008] The technical solution adopted in this invention is as follows:

[0009] A SegFormer water body fine segmentation model incorporating dynamic weights includes:

[0010] The hybrid vision Transformer encoder module is used to extract hierarchical features from the input remote sensing image and construct a multi-scale feature pyramid from details to semantics. The hybrid vision Transformer encoder module includes four progressive downsampling stages, each stage containing an overlapping block embedding unit and a Transformer block. The Transformer block integrates an efficient self-attention unit and an enhanced hybrid feedforward network unit.

[0011] The dynamic weight fusion module, connected after the hybrid vision Transformer encoder module, is used to receive the features of each layer of the multi-scale feature pyramid, and adaptively generate dynamic fusion weights based on the contribution of each scale feature to the current input image. It then performs weighted fusion on the features of each layer after unifying the spatial resolution and outputs the fused features.

[0012] The dual attention enhancement module, connected after the dynamic weight fusion module, is used to recalibrate the fused features in both the channel dimension and the spatial dimension, and multiply the enhancement results of the two dimensions to output the final segmentation features.

[0013] The overlapping block embedding unit uses a convolution operation with an overlap rate of 50% to map the input image to the embedding space and uses layer normalization for stabilization.

[0014] The efficient self-attention unit uses sequence reduction ratios of 8, 4, 2, and 1 in the four stages of the encoder to progressively compress key-value pairs.

[0015] The enhanced hybrid feedforward network unit includes depthwise separable convolutional layers and TeLU activation function layers.

[0016] The dynamic weight fusion module includes:

[0017] The spatial resolution unification unit is used to unify the input multi-scale features to the spatial resolution of the finest-grained features through bilinear interpolation.

[0018] Multiple scale estimators, each corresponding to a feature at a specific scale, are used to extract global statistical information of the feature at that scale and calculate its importance score; the scale estimators sequentially include a convolutional layer, a TeLU activation layer, a global average pooling layer, and another convolutional layer.

[0019] The weight normalization unit is used to concatenate the importance scores of each scale along the channel dimension and then normalize them using the Softmax function to obtain the dynamic fusion weights corresponding to each scale.

[0020] The weighted fusion unit is used to multiply the features of each layer after unifying the resolution with the corresponding dynamic fusion weights element by element and then sum them to obtain the fused features.

[0021] The dual attention enhancement module includes:

[0022] The channel attention submodule is used to perform global average pooling on the input features, which are then processed by a multilayer perceptron and activated by a Sigmoid function to generate channel weight vectors, which are then multiplied with the input features channel by channel.

[0023] The spatial attention submodule is used to perform global max pooling and global average pooling on the input features respectively. The pooling results are concatenated in the channel dimension and then activated by a convolutional layer and a Sigmoid function to generate a spatial weight matrix, which is then multiplied element-wise with the input features.

[0024] The fusion unit is used to multiply the outputs of the channel attention submodule and the spatial attention submodule element by element to obtain the final enhanced features.

[0025] The spatial resolution unification unit uses bilinear interpolation to unify the four scale features output by the encoder to the same spatial resolution as the finest-grained feature.

[0026] The specific processing steps of the scale estimator include the following:

[0027] First pass Convolutional layers capture local feature interaction information;

[0028] The nonlinear expressive power is further enhanced by a custom TeLU activation function;

[0029] The feature map is then compressed into a global feature vector using global average pooling (GAP).

[0030] Finally passed Convolutional layers map global features to a single-dimensional importance score:

[0031] (2)

[0032] in The scale estimator for the i-th feature is expressed mathematically as follows:

[0033] (3)

[0034] After obtaining the original importance scores s1, s2, s3, and s4 for each scale, they are concatenated along the channel dimension to form a unified weight vector S. .

[0035] The multi-scale features output by the hybrid vision Transformer encoder module include feature maps with four different downsampling factors.

[0036] The convolutional layers in the spatial attention submodule use 7×7 convolutional kernels.

[0037] This invention uses a hybrid visual Transformer (MVT) as the encoder and constructs a multi-level feature pyramid through overlapping block embedding, sequence reduction self-attention, and an enhanced hybrid feedforward network to improve the modeling ability of local details and global semantics. In the decoder, a dynamic weight fusion (DWF) module is introduced, which adaptively learns and assigns fusion weights to features at different scales based on the semantic content of the input image, achieving accurate perception of water bodies at multiple scales. Furthermore, a dual attention enhancement (DAF) module is integrated, which extracts key features and suppresses background interference through the synergistic effect of channel and spatial attention, improving the segmentation accuracy of complex boundaries and small water bodies. This method effectively solves the problems of blurred details, missed segmentation of small water bodies, and insufficient multi-scale adaptive perception capabilities in traditional water body segmentation, significantly improving the automation, precision, and reliability of dynamic water resource monitoring and assessment. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the principle of the present invention;

[0040] Figure 2 This is a structural block diagram of the MVT module described in this invention;

[0041] Figure 3 This is a block diagram of the dynamic weight fusion multi-scale feature module described in this invention;

[0042] Figure 4 This is a comparison chart of the average inference time per pixel between the present invention and existing multi-model approaches;

[0043] Figure 5 This is a performance comparison chart of the present invention and existing multi-models on SID based on ROC and PR curves;

[0044] Figure 6 This is a probability distribution diagram of water body segmentation on SID for the present invention and existing models. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] like Figure 1 , 2 As shown in Figure 3, the present invention includes a hybrid visual Transformer encoder module for performing hierarchical feature extraction on input remote sensing images and constructing a multi-scale feature pyramid from details to semantics. The hybrid visual Transformer encoder module includes four progressive downsampling stages, each stage containing an overlapping block embedding unit and a Transformer block. The Transformer block integrates an efficient self-attention unit and an enhanced hybrid feedforward network unit. The multi-scale features output by the hybrid visual Transformer encoder module include feature maps with four different downsampling factors.

[0047] This invention presents a SegFormer water body fine segmentation model with integrated dynamic weights, based on a MixVision Transformer (MVT) module. Employing a hierarchical Transformer architecture, it is specifically optimized for high-resolution remote sensing image water body segmentation tasks. The MVT module consists of four progressive downsampling stages, each including Overlap Patch Embed (OPE) and a Transformer block. OPE maps the input image to the embedding space using overlapping convolution operations and applies LayerNorm for stabilization. Each Transformer block integrates an efficient self-attention module, a feedforward network (FFN), and innovatively introduces deep convolution (DWConv) in the channel dimension to enhance local spatial awareness.

[0048] The improved MVT module specifically includes the following three key components:

[0049] Overlapping Block Embedding (OPE) Mechanism. To address the information fragmentation and loss of boundary details caused by non-overlapping image segmentation in traditional Vision Transformers, this invention employs an overlapping block embedding strategy. This mechanism achieves a feature extraction rate of up to 50% between image blocks through convolution operations, ensuring information continuity between adjacent image blocks and effectively mitigating the loss of water body edge and fine structural information caused by rigid segmentation. This lays a richer foundation of underlying features for subsequent fine segmentation.

[0050] Efficient Self-Attention Mechanism. To balance the model's global modeling capability with computational efficiency, this invention designs an efficient self-attention mechanism employing a sequence reduction strategy. This mechanism progressively compresses key-value pairs in four stages of the encoder using sequence reduction ratios of 8, 4, 2, and 1, respectively. This strategy significantly reduces the computational complexity and memory overhead of self-attention operations while maintaining the model's effective capture of global semantic information of the image, enabling it to efficiently process large-scale remote sensing images.

[0051] Enhanced Mix-FFN. To improve the model's ability to represent local features and express nonlinearities while ensuring training stability, this invention enhances the traditional feedforward network. This hybrid feedforward network introduces depthwise separable convolution to explicitly model local spatial context information, building upon the standard feedforward structure. Furthermore, a custom TeLU activation function is used instead of the conventional activation function, effectively suppressing gradient explosion or numerical instability risks that may occur in deep networks while enhancing nonlinear expression, thereby improving the model's overall robustness and feature learning ability.

[0052] A dynamic weight fusion module, connected after the hybrid vision Transformer encoder module, receives features from each layer of the multi-scale feature pyramid and adaptively generates dynamic fusion weights based on the contribution of each scale feature to the current input image. It then performs weighted fusion of the features from each layer after unifying the spatial resolution and outputs the fused features. The dynamic weight fusion module includes:

[0053] A spatial resolution unification unit is used to unify the input multi-scale features to the spatial resolution of the finest-grained features through bilinear interpolation; the spatial resolution unification unit uses bilinear interpolation to unify the four scale features output by the encoder to the same spatial resolution as the finest-grained features.

[0054] Multiple scale estimators, each corresponding to a feature at a specific scale, are used to extract global statistical information of that feature and calculate its importance score. Each scale estimator sequentially includes a convolutional layer, a TeLU activation layer, a global average pooling layer, and another convolutional layer. The dynamic weight fusion multi-scale feature module is a core component of the SegFormer water body fine-grained segmentation model decoder. This module consists of multiple scale estimators and a weight normalization layer. An attention-based weight generation strategy is employed, dynamically assigning weights based on the actual contribution of features at different scales to the current input image.

[0055] The specific steps are as follows:

[0056] The module uses bilinear interpolation to transform the multi-scale features output by the encoder. Unify to the same spatial resolution (i.e., the finest granular feature) Maintain consistency (Size), to ensure spatial consistency between subsequent weight calculations and feature fusion:

[0057] (1)

[0058] in , This represents the first (after interpolation, i.e., after resampling and normalization) digit. Each feature map This represents the bilinear interpolation operation. Indicates the input number of the first... One original feature map, Indicates the interpolation target size parameter ( Target height, (Target width).

[0059] 1) Each scale estimator evaluates the normalized features The process involves extracting global statistics and calculating corresponding importance scores. The scale evaluator is implemented using convolution, activation, pooling, and compression, first through... Convolutional layers capture local feature interaction information, which is then enhanced with a custom TeLU activation function to improve non-linear expressiveness. Global average pooling (GAP) then compresses the feature map into a global feature vector, and finally... Convolutional layers map global features to a single-dimensional importance score:

[0060] (2)

[0061] in The scale estimator for the i-th feature is expressed mathematically as follows:

[0062] (3)

[0063] in, Indicates the first The output features of each feature extraction module This represents the convolution operation. Indicates global average pooling. Let S represent the Tanh-ELU activation function. After obtaining the original importance scores s1, s2, s3, s4 for each scale, they are first concatenated along the channel dimension into a unified weight vector S: (Where B is the batch size)

[0064] 2) Using the Softmax function to... After normalization, the sum of the weights at each scale is equal to 1, resulting in the final dynamic fusion weight W:

[0065] (4)

[0066] in w i Let be the ith weight channel in W, corresponding to the fusion weight of the ith scale feature. This is the partition function.

[0067] 3) Normalize features at each scale With the corresponding dynamic weight w i Element-wise multiplication is performed, and then all weighted features are summed to obtain the final fused feature F:

[0068] (5)

[0069] This fusion process can adaptively integrate the advantages of multi-scale features, retaining the detailed information of low-scale features while fusing the global semantic information of high-scale features, thereby improving the discriminative ability of feature representation.

[0070] The weight normalization unit is used to concatenate the importance scores of each scale along the channel dimension and then normalize them using the Softmax function to obtain the dynamic fusion weights corresponding to each scale.

[0071] The weighted fusion unit is used to multiply the features of each layer after unifying the resolution with the corresponding dynamic fusion weights element by element and then sum them to obtain the fused features.

[0072] A dual-attention enhancement module, connected after the dynamic weight fusion module, is used to recalibrate the fused features in both the channel dimension and spatial dimension, and multiply the enhancement results of the two dimensions to output the final segmentation features. The dual-attention enhancement module includes:

[0073] The channel attention submodule is used to perform global average pooling on the input features, which are then processed by a multilayer perceptron and activated by a Sigmoid function to generate channel weight vectors, which are then multiplied with the input features channel by channel.

[0074] The spatial attention submodule is used to perform global max pooling and global average pooling on the input features respectively. The pooling results are concatenated in the channel dimension and then activated by a convolutional layer and a Sigmoid function to generate a spatial weight matrix, which is then multiplied element-wise with the input features.

[0075] The fusion unit is used to multiply the outputs of the channel attention submodule and the spatial attention submodule element by element to obtain the final enhanced features.

[0076] The dual attention enhancement module is a crucial component of the decoder in the SegFormer water body fine segmentation model, which incorporates dynamic weights. This module employs a parallel dual-branch structure, recalibrating features from both the channel and spatial dimensions. The outputs of the two sub-modules are fused through element-wise multiplication, achieving dual feature enhancement.

[0077] Within the SegFormer model framework with fused dynamic weights, the channel attention submodule first aggregates the spatial information of each channel from the multi-scale features received by the decoder through global average pooling, generating a channel-level global statistical description. This step aims to capture the dependencies between different feature channels, and then dynamically calibrate the response intensity of each channel through a subsequent reweighting mechanism. This allows the model to adaptively strengthen feature channels related to water semantics, suppress redundant or interfering information, and optimize feature representation. The core of this module is to dynamically calibrate the importance of each channel of the input features using the learned channel weight vector. For input features... Its output The calculation can be expressed as:

[0078] (6)

[0079] in, Indicates global average pooling. It is a fully connected layer with a bottleneck structure. It is the Sigmoid activation function. This indicates channel-wise multiplication.

[0080] The spatial attention submodule operates in parallel with the channel attention branch, focusing on the spatial dimension of the feature map. It efficiently extracts cross-channel spatial saliency information by performing global max pooling and global average pooling on the input features, and then concatenating and fusing their outputs along the channel dimension. This mechanism generates a spatial weight matrix that dynamically highlights key locations in the feature map corresponding to water bodies, thereby performing fine-grained spatial recalibration of the features and effectively mitigating the problem of coarse boundary segmentation. The core of this module is generating a spatial weight map to dynamically enhance key spatial locations related to water bodies. For the input features... Its output The calculation can be expressed as:

[0081] (7)

[0082] in, and These represent global max pooling and average pooling along the channel dimension, respectively. Indicates channel splicing. Represent a Convolutional layer For the Sigmoid function, This represents element-wise multiplication.

[0083] The overlapping block embedding unit maps the input image to the embedding space using a convolution operation with a 50% overlap rate and applies layer normalization for stabilization. The efficient self-attention unit progressively compresses key-value pairs using sequence reduction ratios of 8, 4, 2, and 1 in the four stages of the encoder. The enhanced hybrid feedforward network unit includes depthwise separable convolutional layers and TeLU activation function layers. The convolutional layers in the spatial attention submodule use 7×7 convolutional kernels.

[0084] This invention introduces an improved hybrid visual Transformer (MVT) encoder, employing a hierarchical MVT structure in the encoder portion. It mitigates information loss in image blocks through an overlapping block embedding mechanism, and utilizes an efficient self-attention strategy with sequence reduction to maintain global modeling capabilities while reducing computational complexity. Furthermore, it introduces depthwise separable convolutions and a custom activation function into the feedforward network to enhance local feature modeling and nonlinear representation capabilities. This constructs a robust feature pyramid from detail to semantics for multi-scale water segmentation tasks. The proposed hierarchical MVT module alleviates information loss in image blocks through an overlapping block embedding mechanism, and the hybrid feedforward network with efficient self-attention using a sequence reduction strategy and the introduction of depthwise convolutions reduces computational complexity while enhancing the model's multi-scale feature modeling capabilities from local details to global semantics, laying a solid feature foundation for refined segmentation.

[0085] Furthermore, by adding a Dynamic Weight Fusion (DWF) module, an attention-based dynamic weight generator is introduced into the decoder. This module can adaptively learn and assign optimal fusion weights to feature maps at different levels based on the content of the input image. When dealing with wide bodies of water, it emphasizes high-level semantic features, while automatically enhancing the contribution of low-level high-resolution features when segmenting small rivers and ditches. This effectively solves the limitations of traditional fixed-weight fusion in multi-scale water perception, namely, the insufficient adaptive perception capability in complex multi-scale scenes.

[0086] Finally, a dual-attention feature enhancement (DAF) module is integrated, which introduces parallel channel attention (CA) and spatial attention (SA) sub-modules after feature fusion. The CA module models the channel dimension to filter key semantic features, while the SA module reweights the spatial dimension to focus on important regions and fragile boundaries. Their synergistic effect significantly suppresses background interference in the feature maps, enhances the ability to distinguish small bodies of water and complex contours, refines and enhances feature representation, and effectively compensates for the attenuation of detailed information in deep networks.

[0087] This invention significantly improves the accuracy, detail preservation, and adaptability of water body segmentation in remote sensing images across multiple scales by constructing an encoder-decoder architecture based on an improved hybrid vision Transformer (MVT) and integrating a dynamic weight fusion module and a dual attention mechanism. This architecture not only achieves more accurate and complete water body extraction in complex scenes but also maintains high inference efficiency, providing reliable and efficient remote sensing analysis technology support for applications such as dynamic water resource monitoring and flood risk assessment.

[0088] The following provides further explanation and description of the actual technical effects of the technical solution of the present invention:

[0089] (1) Evaluation of the SegFormer water body fine segmentation model with dynamic weights

[0090] To evaluate the performance of the SegFormer water body fine segmentation model with fused dynamic weights, overall accuracy, recall for binary water body categories, F1 score, intersection-over-union ratio (IoU), mean intersection-over-union ratio (mIoU), Matthews correlation coefficient (MCC), and inference time were used as evaluation metrics. Performance was evaluated on a self-built SID dataset and a publicly available GID dataset. In the binary segmentation task, water body pixels were defined as the positive class (1), and non-water body pixels as the negative class (0). The calculation of the above metrics is based on the comparison between the model's pixel-by-pixel prediction results and the ground truth annotations on the test set. The calculation formulas are as follows:

[0091] Overall accuracy measures the proportion of pixels correctly predicted by the model out of the total number of pixels:

[0092] (15)

[0093] In this context, TP, TN, FP, and FN represent the number of true positive, true negative, false positive, and false negative pixels, respectively.

[0094] Recall rate reflects the model's ability to identify water pixels:

[0095] (16)

[0096] A higher recall rate indicates a lower probability that the model will miss water pixels.

[0097] The F1 score, which combines precision and recall, is used to measure the overall performance of binary segmentation.

[0098] (17)

[0099] Binary intersection-union ratio (IoU) is used to measure the degree of overlap between water body predictions and actual annotations:

[0100] (18)

[0101] The mean intersection-union ratio (mIoU) is the arithmetic mean of the IoUs of the positive and negative classes, and is used to evaluate the overall water body segmentation performance.

[0102] The Matthews correlation coefficient (MCC) comprehensively measures the correlation between model predictions and true labels, and is particularly suitable for class imbalance problems. Its calculation formula is as follows:

[0103] (19)

[0104] Inference time is a key metric for measuring the value of a model deployment. To measure the efficiency of a model in real-world applications, we introduce pixel-level average inference time:

[0105] (20)

[0106] in, This represents the total inference time of the model across the entire test set. This represents the total number of pixels in the test set. This invention uses milliseconds (ms) as the unit of measurement to ensure fair comparison between different models.

[0107] After training and prediction, based on the model evaluation metrics mentioned above, the experimental results of this model and the current mainstream models on the self-built super-resolution dataset are shown in Table 1.

[0108] Table 1. Comparison of metrics (%) of each model on the super-resolution dataset

[0109] The SegFormer water body fine segmentation model with dynamic weights proposed in this invention achieves optimal performance on most metrics. In the core metric mIoU, DWFSeg reaches 92.40%, an improvement of 1.42 percentage points compared to the UNet++ model. The overall accuracy (OA) reaches 98.08%, demonstrating the model's good performance in pixel-level classification. Notably, the SegFormer water body fine segmentation model with dynamic weights achieves a precision of 94.44%, significantly higher than other comparative models, indicating its advantage in reducing false positives. The F1-Score is 93.05%, IoU is 87.00%, and the Matthews correlation coefficient (MCC) is 91.95%, all of which verify that the SegFormer water body fine segmentation model with dynamic weights achieves an excellent balance between high recall and precision.

[0110] The experimental results of this model compared with current mainstream models on the public GID dataset are shown in Table 2.

[0111] Table 2 Comparison of metrics for each model on the public GID dataset (%)

[0112]

[0113] The SegFormer water body fine segmentation model with dynamic weights significantly outperformed the comparison models across all evaluation metrics. In the core metric mIoU, the model achieved 87.95%, a 4.81 percentage point improvement over the second-best UNet++ model. The recall rate reached 92.99%, a 12.83 percentage point improvement over UNet++, indicating a significant reduction in the false negative rate for water body pixels. The F1-Score reached 96.37%, demonstrating a good balance between precision and recall. The overall accuracy (OA) was 96.14%, and the Matthews correlation coefficient (MCC) was 86.96%, further validating the model's stable performance.

[0114] This invention compares the inference performance of each model on the super-resolution dataset based on the average inference time, as shown in Figure 4.

[0115] All models were tested under the same hardware environment (NVIDIA RTX 3090 GPU). Figure 4(a) shows that the proposed model exhibits superior inference performance while maintaining high segmentation accuracy, with an average inference time of 3.51 × 10⁻ per pixel. 5 ms, standard deviation is 1.14×10⁻ 6The speed is improved by approximately 5.9% compared to the UNet++ model, and the inference stability is better. Figure 4(b) shows that the average inference time per pixel of the model in this invention is 1.337 × 10⁻ on GID. 4 ms, standard deviation is 1.8×10⁻ 6 Compared to the UNet++ model's 1.737×10⁻ 4 In milliseconds, DWFSeg improved inference speed by approximately 23.0%. This is compared to the slowest inference model, ISANet (2.277 × 10⁻⁻⁴). 4 Compared to the previous model (ms), the speed improvement reached 41.3%. At the same time, the standard deviation of DWFSeg is better than most of the comparison models, indicating that its inference process has better stability.

[0116] Two key validation modules are introduced: ROC Curves Comparison with Best Thresholds and Precision–Recall Curves Comparison. These modules are validated on the SID dataset to reveal the essential performance differences of the DWFSeg model.

[0117] Figure 5 (a) The ROC curve comparison results show that the Area Under Curve (AUC) of the proposed model is superior to the comparative model. The curve maintains a high True Positive Rate (TPR) in the low False Positive Rate (FPR) range, indicating that the model has stronger separability for the target region. At the optimal threshold point, the TPR of the proposed model is higher than that of other models, indicating that it has higher sensitivity to small targets. The proposed model improves recall while maintaining a low false positive rate, reflecting its advantages in feature representation and spatial structure modeling. Figure 5 (b) In the PR curve, the model of this invention maintains higher accuracy in the high recall interval compared to other networks, and its PR-AUC also reaches a superior level. Especially when the recall approaches 1.0, the accuracy of some models drops significantly, while the curve of the model of this invention drops more gently, indicating that it can effectively suppress noise prediction while detecting complete targets, and has better stability and generalization ability. These two sets of indicators show that the model of this invention has advantages in both overall recognition ability and detection stability.

[0118] This study also compares the models based on their class-wise probability distribution, illustrating the essential differences in output quality and class separability. The distribution comparison results for ISANet, SegFormer, SegNet, UNet++, and DWFSeg are presented.

[0119] Figure 6 In comparison, DWFSeg exhibits a more polarized distribution of predicted probabilities. The predicted probabilities for background samples are concentrated around 0, showing a narrow and stable distribution, while the predicted probabilities for water samples are concentrated around 1, with a clear peak. The fewest samples are near the threshold of 0.5, indicating that the model is more decisive in identifying uncertain regions. In contrast, models such as SegNet and UNet++ have a large number of mixed samples near the 0.5 region, reflecting that their probability outputs are not reliable enough and are prone to low-confidence predictions in ambiguous areas.

[0120] (2) Ablation Experiment Analysis of SegFormer Water Body Fine Segmentation Model with Dynamic Weights

[0121] To verify the contributions of the proposed Dynamic Weight Fusion (DWF) module and Dual Attention Feature Enhancement (DAF) module to model performance, we conducted ablation experiments on the SID and GID datasets. The experiments started with a baseline model without any modules, and then sequentially added the DWG module, the DAF module, and a combination of both, comparing the segmentation performance metrics for each configuration.

[0122] Table 3 Performance comparison of different module combinations in the ablation experiment on GID and super-resolution datasets (%)

[0123]

[0124] On SID, the baseline model achieved an mIoU of 86.20% and an OA of 96.22%. Adding the DWG module alone significantly improved mIoU to 91.06%, OA to 97.69%, IoU from 76.72% to 84.77%, and accuracy from 84.59% to 91.51%, demonstrating the crucial role of the dynamic weight generator in preserving segmentation in high-resolution images [A15.1]. Adding the DAF module alone achieved an mIoU of 89.62%, OA of 97.34%, and accuracy of 92.42%, showcasing the high-resolution image optimization capabilities of the attention mechanism [A16.1]. The complete model demonstrated the most outstanding performance, achieving an mIoU of 92.40%, an OA of 98.08%, an F1-Score of 93.05%, an IoU of 87.00%, and an accuracy of 94.44%, outperforming all other configurations in all metrics.

[0125] On the GID dataset, the baseline model achieved an mIoU of 83.62% and an overall accuracy (OA) of 94.49%. Introducing the DWG module alone improved the mIoU to 84.87% and the OA to 95.03%, with IoU increasing from 73.75% to 75.63% and accuracy from 79.96% to 82.46%, demonstrating that the dynamic weighting mechanism effectively enhanced the multi-scale feature fusion capability. Introducing the DAF module alone further improved the mIoU to 84.75%, the OA to 95.11%, and the accuracy to 84.98%, showcasing the optimization effect of the attention mechanism on feature selection. When the DWG and DAF modules are used simultaneously, the model performance reaches its optimal level, with mIoU significantly improved to 87.95%, OA reaching 96.14%, recall increasing from 90.47% to 92.99%, F1-Score increasing from 84.90% to 89.19%, and IoU reaching 80.49%. This demonstrates that the two modules have a synergistic enhancement effect.

[0126] Comprehensive analysis shows that the DWG module primarily enhances the model's ability to identify water bodies of different sizes by adaptively weighting and fusing multi-scale features. The DAF module, on the other hand, enhances the selectivity and discriminativeness of feature representation through channel and spatial attention mechanisms. The synergistic effect of the two modules improved the mIoU relative to the baseline by 6.20 percentage points on the super-resolution dataset and by 4.33 percentage points on the GID dataset, validating their effectiveness in water segmentation tasks.

[0127] This invention enhances the encoder's ability to collaboratively model local details and global semantics by introducing an improved Hybrid Vision Transformer (MVT) module; it designs a Dynamic Weighted Fusion (DWF) module to achieve adaptive weighted fusion of multi-scale features; and it further integrates a Dual Attention Enhancement (DAF) module to improve the discriminativeness and robustness of feature representation through the synergistic effect of channel and spatial attention, thereby achieving accurate and complete segmentation of multi-scale water bodies in complex scenes. Experimental results on the self-built SID dataset and the publicly available GID dataset show that this invention significantly outperforms mainstream comparative models in core metrics such as average intersection-over-union ratio, overall accuracy, and F1-Score, while maintaining high inference efficiency and stability, providing reliable technical support for the fine extraction and dynamic monitoring of remotely sensed water bodies.

[0128] In the description of this invention, it should be noted that directional terms such as "center", "lateral", "longitudinal", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", and "counterclockwise" indicate the orientation and positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. They should not be construed as limiting the specific protection scope of this invention.

[0129] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification and claims of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0130] Note that the above description is merely a preferred embodiment and application of the technical principles of the present invention. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the specific embodiments described herein, and may include many other effective embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A SegFormer water body fine segmentation model incorporating dynamic weights, characterized in that: include: The hybrid vision Transformer encoder module is used to extract hierarchical features from the input remote sensing image and construct a multi-scale feature pyramid from details to semantics. The hybrid vision Transformer encoder module includes four progressive downsampling stages, each stage containing an overlapping block embedding unit and a Transformer block. The Transformer block integrates an efficient self-attention unit and an enhanced hybrid feedforward network unit. The dynamic weight fusion module, connected after the hybrid vision Transformer encoder module, is used to receive the features of each layer of the multi-scale feature pyramid, and adaptively generate dynamic fusion weights based on the contribution of each scale feature to the current input image. It then performs weighted fusion on the features of each layer after unifying the spatial resolution and outputs the fused features. The dual attention enhancement module, connected after the dynamic weight fusion module, is used to recalibrate the fused features in both the channel dimension and the spatial dimension, and multiply the enhancement results of the two dimensions to output the final segmentation features.

2. The SegFormer water body fine segmentation model with fused dynamic weights according to claim 1, characterized in that, The overlapping block embedding unit uses a convolution operation with an overlap rate of 50% to map the input image to the embedding space and uses layer normalization for stabilization.

3. The SegFormer water body fine segmentation model with fused dynamic weights according to claim 1, characterized in that, The efficient self-attention unit uses sequence reduction ratios of 8, 4, 2, and 1 in the four stages of the encoder to progressively compress key-value pairs.

4. The SegFormer water body fine segmentation model with fused dynamic weights according to claim 1, characterized in that, The enhanced hybrid feedforward network unit includes depthwise separable convolutional layers and TeLU activation function layers.

5. The SegFormer water body fine segmentation model with fused dynamic weights according to claim 1, characterized in that, The dynamic weight fusion module includes: The spatial resolution unification unit is used to unify the input multi-scale features to the spatial resolution of the finest-grained features through bilinear interpolation. Multiple scale estimators, each corresponding to a feature at a specific scale, are used to extract global statistical information of the feature at that scale and calculate its importance score; the scale estimators sequentially include a convolutional layer, a TeLU activation layer, a global average pooling layer, and another convolutional layer. The weight normalization unit is used to concatenate the importance scores of each scale along the channel dimension and then normalize them using the Softmax function to obtain the dynamic fusion weights corresponding to each scale. The weighted fusion unit is used to multiply the features of each layer after unifying the resolution with the corresponding dynamic fusion weights element by element and then sum them to obtain the fused features.

6. The SegFormer water body fine segmentation model with fused dynamic weights according to claim 1, characterized in that, The dual attention enhancement module includes: The channel attention submodule is used to perform global average pooling on the input features, which are then processed by a multilayer perceptron and activated by a Sigmoid function to generate channel weight vectors, which are then multiplied with the input features channel by channel. The spatial attention submodule is used to perform global max pooling and global average pooling on the input features respectively. The pooling results are concatenated in the channel dimension and then activated by a convolutional layer and a Sigmoid function to generate a spatial weight matrix, which is then multiplied element-wise with the input features. The fusion unit is used to multiply the outputs of the channel attention submodule and the spatial attention submodule element by element to obtain the final enhanced features.

7. The SegFormer water body fine segmentation model with fused dynamic weights according to claim 5, characterized in that, The spatial resolution unification unit uses bilinear interpolation to unify the four scale features output by the encoder to the same spatial resolution as the finest-grained feature.

8. The SegFormer water body fine segmentation model with fused dynamic weights according to claim 5, characterized in that, The specific processing steps of the scale estimator include the following: First pass Convolutional layers capture local feature interaction information; The nonlinear expressive power is further enhanced by a custom TeLU activation function; The feature map is then compressed into a global feature vector using global average pooling (GAP). Finally passed Convolutional layers map global features to a single-dimensional importance score: （2） in The scale estimator for the i-th feature is expressed mathematically as follows: （3） After obtaining the original importance scores s1, s2, s3, and s4 for each scale, they are concatenated along the channel dimension to form a unified weight vector S. .

9. The SegFormer water body fine segmentation model with fused dynamic weights according to claim 1, characterized in that, The multi-scale features output by the hybrid vision Transformer encoder module include feature maps with four different downsampling factors.

10. The SegFormer water body fine segmentation model with fused dynamic weights according to claim 6, characterized in that, The convolutional layers in the spatial attention submodule use 7×7 convolutional kernels.

Images