Method for training a medical image segmentation model based on multi-modal scan mamba

Abstract

The present application relates to the technical field of image processing, in particular to a medical image segmentation model training method based on multi-mode scanning Mamba. In the present application, the multi-mode scanning Mamba module adopts five different scanning strategies of forward, backward, spiral, sawtooth and block, and dynamically integrates diversified spatial dependence through a learnable fusion mechanism; the frequency and boundary perception hybrid attention module organically combines multi-scale frequency domain attention with boundary perception attention based on the Sobel operator, simultaneously enhances feature expression in the frequency domain and the spatial domain in two dimensions, and realizes fine boundary positioning.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a training method for a medical image segmentation model based on multimodal scanning Mamba. Background Technology

[0002] Medical image segmentation plays a crucial role in computer vision and clinical decision-making, aiding in accurate diagnosis, treatment planning, and disease monitoring. Convolutional neural network-based architectures (such as U-Net and TBConvL-Net) offer powerful local representation learning capabilities, but their inherent local receptive fields still present fundamental limitations in modeling long-range dependencies.

[0003] Transformer-based methods enhance global context modeling through self-attention; however, their quadratic computational complexity and weak inductive bias impact efficiency and generalization ability, especially when training data is insufficient. State-space models (SSMs) and the Mamba architecture have emerged as efficient alternatives, capable of linear-time modeling of long-distance interactions. Representative models (such as U-Mamba and VM-UNet) demonstrate competitive performance by embedding SSM modules into the U-Net framework.

[0004] However, existing SSM-based image segmentation methods still have significant limitations in modeling complex spatial dependencies, multi-scale feature learning, and fine boundary localization. Existing methods typically rely on fixed and simple spatial scanning schemes (such as row-by-row or column-by-column scanning), limiting their ability to model diverse spatial dependencies and complex anatomical structures. Furthermore, these methods underutilize low-frequency structural information and high-frequency details, and their static feature fusion strategies are ill-suited to the heterogeneous needs of clinical imaging. Summary of the Invention

[0005] Based on this, the present invention provides a method for training a medical image segmentation model based on multimodal scanning Mamba, which solves at least one problem in the prior art.

[0006] In a first aspect, the present invention provides a method for training a medical image segmentation model based on multi-modal scanning Mamba, which includes the following steps: A medical image segmentation model is constructed, comprising an encoder, a skip connection layer, and a decoder, wherein the decoder includes a multi-modal scanning Mamba module; The medical image segmentation model is trained using a medical image dataset. In the multi-modal scanning Mamba module, forward scanning, backward scanning, spiral scanning, zigzag scanning, and block scanning are used to sequence the feature maps. The feature sequences are processed by selecting a state-space model and feature fusion is performed based on learnable fusion weights. The parameters of the medical image segmentation model are updated based on the target loss.

[0007] In some optional embodiments, in the multi-mode scanning Mamba module, the input feature map is layer-normalized, then processed by the MPSMamba module, and the output feature map of the MPSMamba module is residually connected with the input feature map; the intermediate feature map after residual connection is layer-normalized, then processed by a multilayer perceptron, and the output feature map of the multilayer perceptron is residually connected with the intermediate feature map to obtain the output feature map of the multi-mode scanning Mamba module.

[0008] In some optional embodiments, in the MPSMamba module, the input feature map is layer-normalized to obtain a layer-normalized feature map; the layer-normalized feature map is then processed sequentially through a linear layer, a depthwise separable convolutional layer, the MPSSM module, and the layer-normalized layer, and then element-wise multiplied with the feature map activated by the linear layer and the SiLU function to obtain an enhanced feature map; the enhanced feature map is then processed by the linear layer and residually concatenated with the input feature map to obtain the output feature map of the MPSMamba module.

[0009] In some optional embodiments, in the MPSSM module, the input feature map of the MPSSM module is serialized by forward scanning, backward scanning, spiral scanning, sawtooth scanning and block scanning respectively to obtain a feature sequence; after layer normalization, the feature sequence is processed by selecting a state space model and fused according to learnable fusion weights; the fused feature map after feature fusion is linearly projected to obtain the output feature map of the MPSSM module.

[0010] In some optional embodiments, the skip connection layer includes an FBHA module, which applies multi-scale frequency domain attention and Sobel-based boundary-aware attention to the input feature map, respectively, then concatenates it with the input feature map, then performs gated residual connection with the input feature map, and finally performs convolution to obtain the output feature map of the FBHA module.

[0011] In some optional embodiments, applying multi-scale frequency domain attention includes: constructing a Gaussian pyramid based on the input feature map, calculating a frequency pyramid, feeding the features of the frequency pyramid into the frequency enhancement module for frequency domain attention learning, and then feeding them into the gate control multi-scale feature fusion module for multi-scale feature fusion.

[0012] In some optional embodiments, in the frequency enhancement module, the input feature map is sequentially subjected to convolution, batch normalization, ReLU activation, convolution, and Sigmoid activation, and then multiplied element-wise with the input feature map to obtain the output feature map of the frequency enhancement module.

[0013] In some optional embodiments, in the gated multi-scale feature fusion module, the output feature map of the enhanced frequency module is upsampled and concatenated along the channel dimension, and then sequentially passed through convolution, batch normalization, ReLU activation and convolution to obtain a multi-scale fused feature map; then the multi-scale fused feature map is connected to the input feature map of the FBHA module through a gated residual connection.

[0014] In some optional embodiments, applying boundary-aware attention based on the Sobel operator includes: converting the input feature map into a grayscale image, calculating the gradients in the horizontal and vertical directions using the Sobel operator, concatenating the gradient feature map with the input feature map, and calculating the boundary attention weights; furthermore, sequentially performing convolution, ReLU activation, convolution, and Sigmoid activation on the input feature map to obtain a predicted boundary map; and obtaining a boundary enhancement feature map based on the boundary attention weights and the predicted boundary map.

[0015] Secondly, the present invention provides a medical image segmentation method, which includes the following steps: The medical image is input into the medical image segmentation model, and the medical image segmentation model outputs the segmentation result; The medical image segmentation model is obtained by the medical image segmentation model training method based on multimodal scanning Mamba.

[0016] Thirdly, the present invention provides a medical image segmentation device, comprising: At least one processor; and memory that is communicatively connected to at least one processor; The memory stores instructions that, when executed by at least one processor, implement the medical image segmentation method.

[0017] Fourthly, the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the medical image segmentation method.

[0018] Due to the adoption of the above technical solutions, the embodiments of the present invention have at least the following beneficial effects: (1) A state-space driven multi-mode scanning Mamba module was designed, which integrates five complementary spatial scanning modes—forward, backward, spiral, zigzag and block; and adaptively balances their contributions through a learnable fusion scheme, thereby enabling more expressive and robust modeling of complex spatial dependencies. (2) A frequency and boundary-aware hybrid attention mechanism was developed, which jointly utilizes multi-scale frequency domain cues and boundary information based on the Sobel operator to enhance structural representation by simultaneously enhancing global spectral features and fine-grained boundary details; (3) The medical image segmentation model has high computational efficiency and excellent segmentation accuracy. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the network structure of a medical image segmentation model in one embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the network structure of a frequency and boundary awareness hybrid attention module in one embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the network structure of a multi-scale frequency domain attention module in one embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of the network structure of the enhanced frequency module in one embodiment of the present invention.

[0023] Figure 5 This is a schematic diagram of the network structure of a boundary-aware attention module based on Sobel in one embodiment of the present invention.

[0024] Figure 6 This is a schematic diagram of the network structure of the multi-mode scanning Mamba module in one embodiment of the present invention.

[0025] Figure 7 This is a schematic diagram of the network structure of the MPSSM module in one embodiment of the present invention.

[0026] Figure 8 This is a schematic diagram of the MPSSM module performing multi-mode scanning in one embodiment of the present invention.

[0027] Figure 9 This is a schematic diagram of five scanning strategies in one embodiment of the present invention.

[0028] Figure 10 This section presents partial visualizations of the segmentation results for each model on the ACDC dataset.

[0029] Figure 11 This is a partial visualization of the segmentation results of each model on the ISIC2018 dataset.

[0030] Figure 12 This section presents partial visualizations of the segmentation results for each model on the GlaS dataset.

[0031] Figure 13 This is a visualization of the segmentation results of a quantitative study on the ablation of different modules in one embodiment of the present invention.

[0032] Figure 14This is a visualization of the quantitative experimental segmentation results of the progressive scanning strategy of the MPSMamba module in one embodiment of the present invention. Detailed Implementation

[0033] The following will provide a clear and complete description of the concept and technical effects of the present invention, so as to fully explain the purpose, solution and effects of the present invention.

[0034] To at least partially overcome the limitations of existing SSM-based models in complex spatial dependency modeling, multi-scale feature learning, and fine boundary localization, this invention provides a training method for a medical image segmentation model based on multi-modal scanning Mamba. Specifically, a medical image segmentation model is trained, and medical image segmentation is achieved through this model. For ease of representation, this medical image segmentation model is referred to as FBAMPSM.

[0035] like Figure 1 As shown, FBAMPSM employs a U-shaped architecture comprising an encoder, a decoder, and skip connection layers. The encoder extracts multi-scale features from the input image in four stages using a pre-trained PVTv2 module. The skip connection layers refine the features in four stages using a frequency and boundary-aware hybrid attention module (FBHA module). The decoder upsamples the features in four stages using a multi-modal scanning Mamba module (MPSM module). In other embodiments, the number of stages in the pre-trained PVTv2, FBHA, and MPSM modules can vary, for example, all configured with three, five, or six stages.

[0036] The input image can be 224×224 pixels. In the encoder, the input image is processed by a pre-trained PVTv2 algorithm in four stages, generating multi-scale feature maps with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32, respectively, with channel dimensions of 64, 128, 320, and 512. The output feature maps from each stage of the pre-trained PVTv2 algorithm are then processed through the FBHA module for each of the four stages. The output feature maps from each stage of the FBHA module are then residually concatenated with the output feature maps from each stage of the pre-trained PVTv2 algorithm, and then processed through the MPSM module for each of the four stages.

[0037] The Frequency and Boundary Aware Hybrid Attention (FBHA) module applies multi-scale frequency domain attention and boundary-aware attention to the input feature map (the output feature map of each stage of the encoder), then concatenates it with the input feature map, performs a gated residual connection with the input feature map, and finally outputs it after convolution. Specifically, as shown... Figure 2 As shown, the input feature map It is processed by two attention branches to capture complementary global and boundary-sensitive information, as shown below: ; ; in, Indicates the input feature map, and These represent the multi-scale frequency domain attention mechanism and the Sobel-based boundary-aware attention mechanism, respectively. This represents the output feature map after applying multi-scale frequency domain attention. This represents the output feature map after applying boundary-aware attention.

[0038] To effectively integrate the outputs of the two branches, the input feature map The output feature maps from the two attention branches are concatenated as follows: ; in, Indicates the input feature map, This indicates a splicing operation. This represents the spliced ​​feature map. Represents the space of real numbers. Indicates batch size, Indicates the number of channels in the feature map. Indicates the feature map height. Indicates the width of the feature map.

[0039] To enable the model to adaptively determine the contribution of each attention branch based on the characteristics of the input features, adaptive weights are computed through a learnable fusion network. The enhanced feature map (output feature map of the FBHA module) is obtained through gated residual connections, as shown below: ; ; in, This indicates that the Sigmoid function is activated. Indicates that the ReLU function is activated; This represents a k×k convolution operation, for example This represents a 1×1 convolution operation. This represents a 3×3 convolution operation; Indicates the input feature map, This represents the output feature map after passing through the FBHA module.

[0040] In the FBHA module, a multi-scale frequency domain attention mechanism adaptively enhances features across multiple frequency bands, effectively integrating multi-scale frequency enhancement while preserving necessary original information. For example... Figure 3 As shown, given the input feature map Construct a Gaussian pyramid ,in , Indicates the number of levels in the Gaussian pyramid ( (positive integers greater than 1), subsequent layers ( arrive Obtained through the following methods: ; in, This indicates a Gaussian blur operation. Standard deviation A 5×5 Gaussian kernel, This indicates average pooling downsampling with a step size of 2.

[0041] Frequency Pyramid The calculation is as follows: ; in, This indicates a bilinear interpolation upsampling operation. Here, Encodes high-frequency details, while deeper layers gradually represent lower-frequency structures; It contains low-frequency global information.

[0042] The features of the frequency pyramid are fed into the Enhanced Frequency Module (EFB) for frequency domain attention learning. For example... Figure 4 As shown, in the frequency enhancement module, features are sequentially processed through convolution, batch normalization, ReLU activation, convolution again, and Sigmoid activation. In other words, each frequency component in the frequency pyramid... Enhancement is achieved through a separate attention network, as shown below: ; ; in, This indicates that the Sigmoid function is activated. and These represent 1×1 and 3×3 convolution operations, respectively. Indicates batch normalization. This indicates that the ReLU function is activated. Indicates the first i Attention weights of layers This represents the enhanced frequency component.

[0043] The EFB-enhanced features are input into the gated multi-scale feature fusion module (GMFF) for multi-scale feature fusion. First, the enhanced features are upsampled to the original resolution and concatenated along the channel dimension, as shown below: ; ; in, This represents the feature map after upsampling. Indicates upsampling, Indicates the size of the feature map. This represents the height of the target feature map after upsampling. This represents the width of the target feature map after upsampling. This represents the spliced ​​feature map. This indicates a splicing operation.

[0044] Secondly, the concatenated feature maps are sequentially processed through convolution, batch normalization, ReLU activation, and convolution again, as shown below: ; in, This represents a multi-scale fused feature map. This represents a 1×1 convolution operation. Indicates batch normalization. This indicates that the ReLU function is activated.

[0045] Then, the gated residual connections selectively fuse the original features (input feature map) and the enhanced features (multi-scale fused feature map), as shown below: ; ; in, This represents a 1×1 convolution operation. Indicates the gating weight, This represents the output feature map after applying multi-scale frequency domain attention. This indicates element-wise multiplication.

[0046] In the FBHA module, a Sobel-based boundary-aware attention mechanism explicitly captures gradient information to guide attention learning. Its dual-gating mechanism only enhances features when both the attention and boundary detection signals indicate a boundary region. For example... Figure 5 As shown, given the input feature map First, it is converted into a grayscale image to reduce complexity and focus on structural information, as shown below: ; in, Indicates grayscale conversion. This represents a grayscale image.

[0047] Then, the Sobel operator is used to calculate the gradients in the horizontal and vertical directions, as shown below: ; in, and These represent the horizontal gradient and the vertical gradient, respectively. This represents the computation process of the Sobel operator. and These represent the horizontal and vertical convolution kernels computed by the Sobel operator, respectively. In this embodiment of the invention, the Sobel operator's convolution kernel is shown below: ; .

[0048] The gradient feature map is concatenated with the input feature map, and the boundary attention weights are calculated as follows: ; ; in, Indicates the input feature map, This represents the spliced ​​feature map. Indicates the boundary attention weights; Indicates a splicing operation; Indicates that the Sigmoid function is activated; This represents a k×k convolution operation, for example This represents a 1×1 convolution operation. This represents a 3×3 convolution operation; Indicates batch normalization; This indicates that the ReLU function is activated.

[0049] To adaptively detect boundaries, a learnable boundary branch is introduced, as follows: ; in, This represents the predicted boundary map.

[0050] The final boundary enhancement feature map is as follows: ; in, This represents the boundary enhancement feature map, i.e., the output feature map based on the Sobel-based boundary-aware attention mechanism; This indicates element-wise multiplication.

[0051] The Multimodal Scanning Mamba module (MPSM module) employs a dual residual design, comprising a LayerNorm, an MPSMamba module, and a Multilayer Perceptron (MLP). The core MPSMamba module serializes feature maps along multiple scan paths and models multidirectional dependencies through a selective state space (S6) mechanism. Specifically, as... Figure 6As shown, for the input feature map of the i-th stage (i=1, 2, 3 or 4) ,in, This represents the input feature map of the FBHA module. The image shows the feature map after processing by the FBHA module. The processing flow of the MPSM module is as follows: For the fourth stage (i=4): ; ; ; For the first three stages (i=1, 2, or 3): ; ; ; in, This represents the input feature map of the MPSM module; Indicates the skip connection layer. Output feature map of stage (i=1, 2, 3 or 4), Indicates the MPSM module number The output feature map of the stage (i=1, 2, 3 or 4), for example, This represents the output feature map of stage 4 of the skip connection layer. This represents the output feature map after processing in stage 4 of the MPSM module. This represents the intermediate feature map output by the MPSMamba module; Representation layer normalization, This indicates that the MPSMamba module handles this. This indicates multilayer perceptron processing.

[0052] In the MPSMamba module, layer normalization is first applied to stabilize activations, followed by linear projection for channel alignment. Depthwise separable convolutions then extract local spatial correlations at low computational cost. The resulting features are processed by the Multi-Mode Scanning State Space Module (MPSSM module), which models long-range dependencies through sequential scanning along five complementary directions (forward, backward, spiral, zigzag, and block traversals). A second normalization layer and a gated branch (linear layer + SiLU function) provide adaptive channel modulation, selectively enhancing informative responses. The final linear projection and residual connections ensure context preservation. Formally, for the input feature map... The MPSMamba module's processing procedure is as follows: ; ; ; ; in, This represents the feature map after passing through the normalization layer. Representation layer normalization, Indicates linear projection. This represents depthwise separable convolution. This indicates that the MPSSM module is processing the data. This represents the feature map of the feature branches after multi-mode scanning state space modeling and local convolution processing. Indicates SiLU function activation. Represents the gated branch feature map. This represents the output feature map after layer normalization and residual connection with the MPSMamba module.

[0053] like Figure 7 and Figure 8 As shown, the MPSSM module performs an omnidirectional scan on the input features to generate multi-mode sequences, processes each sequence through the S6 mechanism, and adaptively fuses the multi-directional global context during the destacking stage to obtain a unified representation. Specifically, to capture the spatial dependencies of heterogeneity, the MPSSM module employs the following five scanning strategies with different geometric features: (1) Forward scanning preserves the natural row priority order; (2) Backward scan to capture complementary inversion dependencies; (3) Helical scanning effectively encodes radial and centrally symmetric structures; (4) Sawtooth scanning, maintaining horizontal-vertical continuity while converting diagonal lines; (5) Block scanning preserves local spatial coherence and fine-grained texture.

[0054] These five scanning strategies are as follows: Figure 9 As shown, these scanning strategies are used for input feature maps. Generate five sequence representations, as shown below: ; ; in, , and These represent helical scan, zigzag scan, and block scan operations, respectively. Indicates a backward scan operation. This represents a set of multi-mode sequences generated by various scanning strategies. Representation layer normalization, This indicates the selection of a state-space model for processing. These represent the feature maps output after normalization and selection of the state-space model for the forward scan sequence, backward scan sequence, spiral scan sequence, zigzag scan sequence, and block scan sequence, respectively.

[0055] Each scanning mode has its unique advantages: forward / backward scanning models horizontal dependence, spiral scanning encodes rotational structures, zigzag scanning captures diagonal transformations, and block scanning preserves fine local textures. To adaptively integrate orientation information, learnable fusion weights are introduced. Where L=4 and C=5, the fused representation is obtained in the following way: ; in, This indicates a projection operation. .

[0056] During training FBAMPSM, parameters are updated according to a composite loss function combining Dice loss and cross-entropy loss (CE loss) to improve overall performance. CE loss provides pixel-level supervision, facilitating accurate boundary delineation and stable gradient propagation. Dice loss is particularly effective in handling class imbalance, which is common in medical image segmentation when the foreground-to-background ratio is highly skewed. By leveraging their complementary strengths, the composite loss ensures stable optimization and robust segmentation. An adaptive weighting mechanism allows for flexible control over the contribution of each component, achieving accurate boundary prediction while maintaining robustness to class imbalance. Formally, the composite loss function is defined as follows: ; in, and These represent hyperparameters, which have different values ​​on different datasets. For example, the value is 1 on the Glas and ISIC2018 datasets, and 0.7 and 0.3 on the ACDC dataset, respectively. Indicates compound loss. Represents the Dice loss function. This represents the CE loss function.

[0057] To verify the effectiveness of the medical image segmentation model (FBAMPSM) in this embodiment of the invention, experiments were conducted on the GlaS dataset, ISIC2018 dataset, and ACDC dataset, and a systematic comparison was made with existing medical image segmentation models.

[0058] The GlaS dataset contains 165 microscopic images of H&E-stained colon tissue sections, covering both benign and malignant tumor samples. Each image includes pixel-level annotations of glandular structures provided by a professional pathologist. The image resolution is set to 224×224 pixels, and the dataset is divided into a training set of 85 images and a test set of 80 images.

[0059] The ISIC2018 dataset contains 2594 images, encompassing clinical photographs and dermoscopic images, with lesion masks annotated by dermatologists. To ensure consistency, each image was resized to 224×224 pixels. Consistent with I2U-Net, this dataset is divided into training, validation, and test sets in a 70 / 10 / 20 ratio.

[0060] The ACDC dataset contains cardiac MRI scans from 100 participants, including healthy individuals and patients with right ventricular (RV), left ventricular (LV), and myocardial (Myo) lesions. To normalize the input size, all slices were resized to 224×224 pixels. The dataset was divided into 70 training samples, 10 validation samples, and 20 test samples.

[0061] Five-fold cross-validation was used in the experiment to comprehensively evaluate the model's generalization ability, and the segmentation quality was evaluated using three key metrics (Dice coefficient, mIoU, and HD95). The Dice coefficient quantifies the similarity between the segmentation result and the corresponding ground truth label. mIoU (mean intersection-union ratio) measures the average overlap between the predicted segmentation mask and the ground truth mask. Both the Dice coefficient and mIoU values ​​range from 0 to 1; higher values ​​indicate a closer similarity to the ground truth label, implying higher accuracy and reliability. In contrast, HD95 (95% Hausdorff distance) quantifies the maximum boundary difference between the predicted segmentation and the ground truth label; lower values ​​indicate a better boundary fit. The specific calculations for these metrics are shown below: ; ; ; in, This represents the number of pixels that the model correctly predicted as the target region. This indicates the number of pixels that the model incorrectly predicted as the target region. This indicates the number of pixels that the model incorrectly predicted as background. y Represents the true value. Indicates the predicted segmentation result. and This represents the one-way Hausdorff distance from the true value to the predicted result and from the predicted segmentation result to the true value.

[0062] The experiments were implemented using the PyTorch framework, and all training and testing were performed on an NVIDIA RTX4080 GPU equipped with 16GB of video memory. For optimization, the AdamW algorithm was used with an initial learning rate of 0.001 to accelerate convergence and improve generalization performance. Table 1 lists the hyperparameters used in the experiments.

[0063] Table 1 FBAMPSM Training Parameter Information Table As shown in Table 2, FBAMPSM in this embodiment of the invention demonstrates outstanding performance on the ACDC dataset, surpassing all comparative models with an average Dice coefficient of 92.30%. FBAMPSM's superior performance stems from its long-range dependency modeling capability based on the Mamba architecture, effectively handling spatial constraints between different cardiac structures and ensuring anatomical consistency in segmentation results. Furthermore, the multi-modal scanning mechanism enhances feature representation by integrating spatial information from multiple perspectives, which is particularly advantageous for complex cardiac geometries. In terms of computational efficiency, FBAMPSM requires only 9.10 GFLOPs, significantly lower than the comparable HResFormer (131.70 GFLOPs) and D-LKA Net (78.01 GFLOPs), showcasing an excellent efficiency-accuracy trade-off.

[0064] Table 2 Comparison results of each model on the ACDC dataset Figure 8 This section presents qualitative visualizations of FBAMPSM compared to other models on the ACDC dataset. The first four rows of visualizations demonstrate FBAMPSM's superior performance in cardiac multi-structure segmentation, accurately segmenting the right ventricle (orange), myocardium (red), and left ventricle (yellow) with clear segmentation boundaries that closely approximate the ground truth. FBAMPSM exhibits a significant advantage in recognizing complex cardiac geometry, avoiding the irregular shapes or blurred boundaries common in other models. Even with relatively small cardiac structures or low contrast, FBAMPSM maintains good segmentation accuracy and stability, with the continuity and integrity of the myocardial region significantly outperforming comparative models such as UNet-50 and AttenUNet. FBAMPSM's segmentation results are the closest to the ground truth among all models. Compared to the severe oversegmentation of AttenUNet and UNet-50, FBAMPSM's boundary localization is more accurate, fully demonstrating its robustness and superiority even under challenging conditions.

[0065] As shown in Table 3, FBAMPSM performs exceptionally well on the ISIC2018 skin lesion segmentation dataset. The core challenge in skin lesion segmentation lies in the high diversity of lesion morphology and the ambiguity of boundaries, which is fundamentally different from the segmentation of regular organs. FBAMPSM overcomes the limitations of fixed scanning mechanisms in capturing complex boundaries by flexibly adapting to diverse and irregular lesion geometries through diverse spatial traversal patterns. Furthermore, its adaptive feature fusion mechanism dynamically optimizes the segmentation strategy based on the visual features of individual lesions, effectively alleviating common challenges such as low contrast and unclear boundary gradients. Simultaneously, FBAMPSM, with 61.38M parameters, surpasses CFATransUNet (66.57M) and TransUNet (105.28M), which have more parameters, while its computational cost of 9.09 GFLOPs is significantly lower than MFCNet's 153.67 GFLOPs, demonstrating the efficiency of its model design.

[0066] Table 3. Comparison results of each model on the ISIC18 dataset. Figure 11 The visualizations show partial segmentation results of various models on the ISIC2018 dataset, with red indicating oversegmentation and blue indicating undersegmentation. The first three rows of visualizations demonstrate that FBAMPSM exhibits strong performance across various lesion types, accurately depicting large, irregular, small, and complex lesions. FBAMPSM's predictions closely match the ground truth labels, with precise boundary localization and excellent preservation of structural details, consistently outperforming other models. Compared to the severe shape distortions produced by AttenUNet and DCSAUNet, and the widespread oversegmentation in U-Net-based models, FBAMPSM maintains relatively high accuracy while preserving morphological integrity. These results highlight the robustness and overall superiority of FBAMPSM in complex skin lesion segmentation tasks.

[0067] As shown in Table 4, FBAMPSM achieves a Dice coefficient of 91.84% on the GlaS dataset, exceeding MMUNet (91.17%) by 0.67 percentage points. More importantly, FBAMPSM's HD95 is 4.86 mm, indicating a significant improvement in boundary accuracy. These results highlight the effectiveness of FBAMPSM in accurate boundary localization. FBAMPSM's superior performance stems primarily from two design components: a multi-modal scanning strategy that captures the typical complex tubular and annular geometries of glandular structures, and a frequency- and boundary-aware hybrid attention mechanism that enhances multi-frequency feature extraction in H&E-stained images and improves boundary delineation. Furthermore, FBAMPSM requires only 61.38M parameters and 9.09 GFLOPs, achieving the highest computational efficiency.

[0068] Table 4. Comparison results of each model on the GlaS dataset Figure 12 This paper presents the qualitative results of glandular segmentation for each model on the GlaS dataset, highlighting the superior performance of the FBAMPSM framework, especially in handling complex pathological tissue structures. In the standard case (first few rows), FBAMPSM accurately segments glandular structures with clear and well-defined boundaries and morphology, consistently outperforming CFANet, CSCAUNet, and MFCNet. In the more challenging case (last two rows), UCTransNet and TransUNet exhibit significant oversegmentation, while FBAMPSM maintains high accuracy by accurately depicting glandular boundaries (without expanding the segmented region). This consistent performance across simple and complex scenarios demonstrates FBAMPSM's powerful feature extraction capabilities, accurate boundary discrimination, and excellent generalization ability.

[0069] To verify the independent contributions and synergistic effects of the core components in FBAMPSM, systematic ablation experiments were conducted on three benchmark datasets: GlaS, ISIC2018, and ACDC. Specifically, five progressive model variants were designed to quantitatively evaluate the individual effects and combined effects of the multimodal scanning Mamba module (MPSM module), the Sobel operator-based boundary-aware attention (SBAA) mechanism, and the multi-scale frequency domain attention (MFDA) mechanism. The Baseline model contains only forward and reverse scanning strategies, while the other model variants add the corresponding modules to the Baseline model.

[0070] As shown in Table 5, the comparison between bidirectional scanning and five-mode scanning verifies the effectiveness of multimodal spatial traversal. By capturing complex spatial dependencies, the integrated multimodal scanning method exhibits significant advantages over traditional bidirectional scanning. The inclusion of a Sobel-based boundary-aware attention (SBAA) module significantly improves segmentation accuracy, confirming the effectiveness of the Sobel-based mechanism in accurate lesion boundary localization. The corresponding reduction in HD95 further highlights the contribution of SBAA to boundary accuracy. The multi-scale frequency domain attention (MFDA) module achieves stable performance improvements on all datasets, indicating that the MFDA mechanism effectively enhances the model's multi-scale feature representation capability through adaptive frequency domain weight learning. These advantages are particularly evident in medical images with complex multi-scale structures. The complete FBAMPSM model, by integrating all components, achieves optimal performance on all datasets, showing continuous improvement across multiple metrics compared to the bidirectional scanning baseline. These results fully demonstrate the significant advantages of the fully integrated technical framework.

[0071] Table 5. Quantitative study of ablation of different modules Figure 13 The visualization of segmentation results from the quantitative study of ablation of different modules is shown, with red areas representing oversegmentation and blue areas representing undersegmentation. Visual comparisons of the ablation experiments reveal that the complete model (e) exhibits the best segmentation quality across all three datasets, with the fewest oversegmentation (red areas) and undersegmentation (blue areas) errors. Particularly in glandular boundary processing on the GlaS dataset and lesion contouring on the ISIC2018 dataset, the complete model significantly reduces boundary errors compared to other configurations. On the ACDC dataset, the segmentation results for each cardiac structure are also closer to the true labels, fully demonstrating the effectiveness of the collaborative work of all components.

[0072] To comprehensively evaluate the incremental impact of different scanning strategies in the MPSMamba module, progressive ablation experiments were conducted on three datasets. These experiments systematically evaluated the performance changes from bidirectional scanning to a full five-mode configuration. Base represents the baseline model, an MPSMamba configuration using only forward and backward scanning strategies.

[0073] As shown in Table 6, each newly added scanning mode consistently improved segmentation performance, validating the effectiveness of multimodal spatial modeling. Specifically, helical scanning, by capturing rotationally invariant and centrosymmetric features through radial traversal, increased the Dice coefficient on the GlaS dataset from 90.79% to 91.30%, and reduced HD95 from 7.01 mm to 6.26 mm. Adding zigzag scanning, by alternating diagonals to enhance horizontal and vertical continuity, further improved the Dice coefficient on the GlaS dataset to 91.65%, and HD95 to 5.65 mm. Block scanning, by capturing fine-grained texture through local region processing, achieved the best overall performance, with a Dice coefficient of 91.84% and HD95 of 4.86 mm on the GlaS dataset. The impact of the newly added scanning modes on the models was similar on the ISIC2018 and ACDC datasets. These improvements reflect a complementary design of forward and reverse scanning (for row and column traversal), spiral scanning (for radial feature modeling), zigzag scanning (for diagonal enhancement), and block scanning (for local perception), which together enable the MPSMamba module to effectively model multi-dimensional spatial dependencies.

[0074] Table 6. Quantitative Study of the Progressive Scan Strategy of the MPSMamba Module Figure 14 The visualization shows a quantitative experiment of the progressive scanning strategy of the MPSMamba module, with oversegmentation indicated by red. On the GlaS dataset, oversegmentation (red areas) decreases and segmentation becomes more accurate as the number of scanning modes increases. On the ISIC2018 dataset, the improvement is significant, gradually transitioning from oversegmentation in configurations (a)-(c) to accurate boundary delineation in configuration (d), demonstrating the ability of multi-modal scanning to capture complex lesion morphologies. On the ACDC dataset, the intermediate configurations exhibit varying degrees of undersegmentation near the cardiac boundary, but this undersegmentation decreases with the increase in modes, and configuration (d) produces results closest to the true labels.

[0075] To evaluate the effectiveness of different fusion strategies in the MPSMamba module, the output features of five scanning modes were fused, and comparative experiments were conducted. The results are summarized in Table 7. Three strategies were considered in the experiments: (1) Direct addition fusion: Performs element-wise addition on the output; (2) Channel splicing and fusion (Concat): Features are spliced ​​along the channel dimension and then dimensionality is reduced through a linear layer; (3) Weighted Fusion: The proposed adaptive weighting mechanism with learnable parameters is adopted.

[0076] Experimental results show that the weighted fusion strategy consistently outperforms additive fusion and splicing fusion on all datasets. On the GlaS dataset, weighted fusion increases the Dice coefficient to 91.84% (0.72% improvement compared to additive fusion and 0.27% improvement compared to splicing fusion) and reduces HD95 to 4.86 mm (32.78% reduction compared to additive fusion and 33.06% reduction compared to splicing fusion), indicating improved boundary accuracy. On the ISIC2018 dataset, its Dice coefficient reaches 91.03% and mIoU reaches 84.79%, surpassing both additive and splicing fusion. On the ACDC dataset, the average Dice coefficient reaches 92.30%, exceeding both additive and splicing fusion, with the most significant improvement in right ventricular segmentation (91.22%, 1.05% improvement compared to additive fusion and 0.66% improvement compared to splicing fusion), highlighting its effectiveness in modeling complex spatial structures. The adaptive weighted fusion dynamically adjusts the feature contribution based on the image content and significantly reduces HD95, verifying its ability to integrate multi-spatial-dimensional boundary information.

[0077] Table 7. Quantitative Comparative Study of Different Fusion Strategies The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any embodiment that achieves the technical effects of the present invention by the same or equivalent means should fall within the protection scope of the present invention. Within the protection scope of the present invention, various modifications and variations can be made to the technical solutions and / or implementation methods.

Claims

1. A method for training a medical image segmentation model based on multi-modal scanning Mamba, characterized in that, Includes the following steps: A medical image segmentation model is constructed, comprising an encoder, a skip connection layer, and a decoder, wherein the decoder includes a multi-modal scanning Mamba module; The medical image segmentation model is trained using a medical image dataset. In the multi-modal scanning Mamba module, forward scanning, backward scanning, spiral scanning, zigzag scanning, and block scanning are used to sequence the feature maps. The feature sequences are processed by selecting a state-space model and feature fusion is performed based on learnable fusion weights. The parameters of the medical image segmentation model are updated based on the target loss; The skip connection layer includes an FBHA module, which applies multi-scale frequency domain attention and boundary-aware attention based on the Sobel operator to the input feature map, then concatenates it with the input feature map, then performs gated residual connection with the input feature map, and finally performs convolution to obtain the output feature map of the FBHA module. The application of multi-scale frequency domain attention includes: constructing a Gaussian pyramid based on the input feature map, calculating a frequency pyramid, and feeding the features of the frequency pyramid into the frequency enhancement module for frequency domain attention learning, and then feeding them into the gated multi-scale feature fusion module for multi-scale feature fusion; wherein, in the frequency enhancement module, the input feature map is sequentially subjected to convolution, batch normalization, ReLU activation, convolution, and Sigmoid activation, and then multiplied element-wise with the input feature map to obtain the output feature map of the frequency enhancement module; in the gated multi-scale feature fusion module, the output feature map of the frequency enhancement module is upsampled and concatenated along the channel dimension, and then sequentially subjected to convolution, batch normalization, ReLU activation, and convolution to obtain a multi-scale fused feature map; then the multi-scale fused feature map is gated residually connected to the input feature map of the FBHA module; The application of boundary-aware attention based on the Sobel operator includes: converting the input feature map into a grayscale image, using the Sobel operator to calculate the gradients in the horizontal and vertical directions, concatenating the gradient feature map with the input feature map, and then calculating the boundary attention weights; furthermore, sequentially performing convolution, ReLU activation, convolution, and Sigmoid activation on the input feature map to obtain a predicted boundary map; and obtaining a boundary enhancement feature map based on the boundary attention weights and the predicted boundary map.

2. The method according to claim 1, characterized in that, In the multi-mode scanning Mamba module, the input feature map is layer-normalized and then processed by the MPSMamba module. The output feature map of the MPSMamba module is then residually connected with the input feature map. The intermediate feature map after residual connection is layer-normalized and then processed by a multilayer perceptron. The output feature map of the multilayer perceptron is then residually connected with the intermediate feature map to obtain the output feature map of the multi-mode scanning Mamba module.

3. The method according to claim 2, characterized in that, In the MPSMamba module, the input feature map is layer-normalized to obtain a layer-normalized feature map. The layer-normalized feature map is then processed sequentially through a linear layer, a depthwise separable convolutional layer, the MPSSM module, and a layer-normalized layer. Element-wise multiplication is then performed between the layer-normalized feature map and the feature map activated by the linear layer and the SiLU function to obtain an enhanced feature map. The enhanced feature map is then processed by the linear layer and residually concatenated with the input feature map to obtain the output feature map of the MPSMamba module.

4. The method according to claim 3, characterized in that, In the MPSSM module, the input feature map of the MPSSM module is serialized by forward scanning, backward scanning, spiral scanning, sawtooth scanning and block scanning respectively to obtain feature sequence. After layer normalization, the feature sequence is processed by selecting state space model and feature fusion is performed according to learnable fusion weights. The fused feature map after feature fusion is linearly projected to obtain the output feature map of the MPSSM module.

5. A medical image segmentation method, characterized in that, Includes the following steps: The medical image is input into the medical image segmentation model, and the medical image segmentation model outputs the segmentation result; The medical image segmentation model is obtained by the medical image segmentation model training method based on multimodal scanning Mamba as described in any of claims 1-4.

6. A medical image segmentation device, characterized in that, include: At least one processor; and memory that is communicatively connected to at least one processor; The memory stores instructions that, when executed by at least one processor, implement the medical image segmentation method of claim 5.

7. A computer-readable storage medium, characterized in that, The system stores instructions that, when executed by a processor, implement the medical image segmentation method of claim 5.

Images