Cascade multi-scale convolution and modal enhancement brain tumor segmentation method based on Mamba architecture

Abstract

The invention discloses a cascade multi-scale convolution and modal enhancement brain tumor segmentation method based on a Mama framework, and relates to the technical field of brain tumors. According to the method, an MCME-UNet model integrated with an MCMS module is provided, through a hierarchical feature extraction mechanism and a multi-modal feature fusion strategy optimized by an MEM module, the segmentation precision is remarkably improved while the calculation efficiency is kept, and particularly, an EEM module innovatively applies Sobel operator to be connected with residual errors, so that the tumor boundary continuity and the segmentation accuracy are synchronously improved; moreover, a novel training normal form of a focus Tversky loss function is introduced, so that the problem of class imbalance is effectively solved, the sensitivity of the model to a small tumor region is enhanced, a smoother segmentation boundary can be generated, and the stability and accuracy of a segmentation result are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of brain tumor technology, and specifically to a brain tumor segmentation method based on a cascaded multi-scale convolution and modality enhancement architecture of Mamba. Background Art

[0002] Brain tumors are a type of disease that seriously threatens human life and health, especially malignant brain tumors, which are characterized by strong invasiveness, high recurrence rate, and short patient survival.

[0003] In recent years, Mamba-based brain tumor segmentation networks have outperformed traditional CNN and Transformer models in terms of computational efficiency and global information modeling. For example, Vossough et al. compared the performance of nnU-Net and DeepMedic on multimodal MRI data, focusing on improving the models' segmentation accuracy and generalization capabilities in specific populations. However, they did not systematically address issues such as local feature extraction and boundary detail processing. Ma et al. proposed a dual Transformer architecture that integrates local and global information and introduces an attention supervision mechanism, effectively enhancing local feature extraction and overall segmentation accuracy. However, this approach still lacks sufficient consideration for handling class imbalance and controlling model complexity. Similarly, Zarenia's research primarily focused on optimizing local features and boundary details, but insufficiently considered class imbalance and model generalization. Yang et al. leveraged the advantages of the Mamba network, combining it with the UNet architecture. While maintaining model lightweight, they optimized feature extraction through the Mamba module and mitigated feature redundancy using the SD-SSM and a carefully designed SD-Conv architecture. The resulting architecture demonstrated strong generalization performance, but they did not propose a specific strategy for optimizing boundary detail.

[0004] These limitations are mainly reflected in insufficient local feature extraction, poor boundary detail processing, category imbalance, and generalization ability. Brain tumors are closely related to high mortality. In the treatment of this disease, the use of magnetic resonance imaging (MRI) to achieve accurate segmentation of brain tumors is particularly critical. Although existing segmentation technologies have their own advantages, they still have significant limitations: the local receptive field of convolutional neural networks (CNNs) is limited, and the computational cost of Transformer-based networks is high. Although the emerging Mamba network can alleviate some computational bottlenecks, it is difficult to balance the continuity of tumor boundaries and the refinement of local features - especially under the heterogeneity and noise interference of multimodal MRI data, its generalization ability is often limited;

[0005] Therefore, a new solution to the above problems needs to be proposed. Summary of the Invention

[0006] The purpose of the present invention is to provide a brain tumor segmentation method based on the Mamba architecture using cascaded multi-scale convolution and modality enhancement, aiming to enhance the ability to capture details of brain tumor images and solve the technical problems raised in the background technology.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a brain tumor segmentation method based on cascaded multi-scale convolution and modality enhancement of the Mamba architecture, comprising at least the following steps:

[0008] S1: Based on the Mamba framework, Mamba structure UNet, cascaded multi-scale convolution and modality enhancement, an MCME-UNet model is proposed. The MCME-UNet model is established on the basis of the state space model. The MCME-UNet model includes a Mamba enhanced cascaded multi-scale convolution module, a modality enhancement module and an edge enhancement module, and adopts a focal Tversky loss function. The Mamba enhanced cascaded multi-scale convolution module is the MCMS module, the modality enhancement module is the MEM module, and the edge enhancement module is the EEM module.

[0009] S2: The MCMS module simultaneously captures local details and global context features through a hierarchical feature extraction mechanism, effectively compensating for the shortcomings of the Mamba structure in local feature modeling. The MCMS module is based on the predicted segmentation result framework of the convolutional layer and adopts a cascaded multi-scale convolution structure containing 3×3, 5×5, and 7×7 convolution kernels. Cascaded multi-scale convolutions are integrated in the decoding path to enhance detail perception and global context understanding capabilities. A hierarchical encoder is used that integrates three-way spatial Mamba blocks and cascaded multi-scale convolution modules.

[0010] S3: Adding the MEM module at the top of the encoder hierarchy, using dual data streams and contrastive learning to ensure balanced and comprehensive feature information across scales. The MEM module optimizes the multimodal feature fusion process through innovative designs such as channel dimension adjustment and dual data stream contrastive learning.

[0011] S4: The global multi-scale features of the encoder are connected to the decoder to achieve feature reuse and enhanced by adding EEM modules at different encoding stages. The EEM module uses residual connections and the Sobel operator to emphasize edge details in the feature map and assists in accurate segmentation by learning the mapping relationship between edge features and true labels. The encoder extracts features hierarchically through alternating global-local processing stages, while the decoder gradually restores the space through explicit edge guidance. This significantly improves the continuity of tumor boundary segmentation while maintaining computational efficiency.

[0012] S5: Finally, through the collaborative optimization of the EEM module and the focal Tversky loss function, the MCME-UNet model can not only utilize multi-scale features and edge prior information in the decoding stage, but also enhance the segmentation ability of tumor boundaries and small lesions by dynamically adjusting the loss weights, effectively alleviating the class imbalance problem and enhancing the focus on small tumor areas.

[0013] Furthermore, the MCMS module is used to improve the model's contextual modeling capabilities and the robustness and generalization performance of feature expression in complex scenarios. The MCMS module aims to fully extract and adaptively integrate multi-scale features, thereby enhancing the model's ability to recognize and segment diverse tumor morphologies.

[0014] The MCMS module effectively captures rich contextual information from different scales by constructing a multi-scale feature pyramid. At the same time, it introduces the Mamba structure to dynamically model and enhance the extracted multi-level features, further improving the feature discrimination capability.

[0015] The MCMS module adopts a cascaded multi-scale design, which is called CMS. It gradually expands the receptive field through a hierarchical feature progressive architecture and integrates multi-scale information. Each layer extracts features from the output of the previous level and fuses shallow detail features with deep semantic features through residual connections or skip connections. This progressive feature extraction method can effectively model multi-scale contextual information from local to global, enhancing the model's ability to represent complex structures.

[0016] At the same time, with the help of the cross-level feature fusion mechanism, the MCMS module captures key area details at different resolutions, thereby improving the model's sensitivity to target boundaries and global semantic understanding. It not only significantly enhances the hierarchical representation capability of feature expression, but also achieves efficient utilization of multi-scale features through gradual refinement.

[0017] Furthermore, the CMS layer in the MCME-UNet network model integrates convolutional kernels of multiple sizes, including 3×3, 5×5, and 7×7, to accurately adapt to the feature extraction of multimodal brain MRI images. Each kernel size achieves full-scale feature capture at different spatial levels.

[0018] The 3×3 core is better at focusing on fine details than the 5×5 and 7×7 cores;

[0019] The 5×5 kernel is more accurate than the 3×3 and 7×7 kernels in capturing mid-level features;

[0020] The 7×7 kernel can cover a wider context area than the small kernel, thus achieving comprehensive feature extraction across scales;

[0021] The CMS description formula is as follows:

[0022]

[0023] Among them, x represents an input feature map with a dimension of (B, C, H, W). For a given H, W and C represent the height, width and number of channels respectively, and the initial scale feature Extracted from the stem layer; convolution normalization function F k×k Perform operations with a kernel size of k×k; Represents the output feature map of each layer, where k represents the convolution kernel size and n represents the number of iterations in the convolution sequence of the kernel size;

[0024] The output of each layer will be passed as input to the subsequent convolutional layer with the same kernel size, and the output of the last layer will be passed to the convolutional layer with a smaller kernel size. After the final 3×3 convolution, the operation sequence will converge to generate the final output feature map output;

[0025] The configuration sets a=1, indicating that 7×7ConvBN performs only a single iteration; b=1, indicating that 5×5ConvBN performs a single operation; and c=3, indicating that three 3×3ConvBN operations are performed consecutively.

[0026] Furthermore, the MCMS module aims to optimize the integration of semantic information and spatial features more effectively;

[0027] The MEM module is placed between the encoder and the decoder, and by synchronously adjusting the channel dimensions of high-level semantic features and low-level spatial features, the application of the MEM module includes at least the following steps:

[0028] Point-by-point convolution is used to compress the channel dimension of low-level spatial features to 3 / 4 of the original size, and high-level semantic features are first adjusted to the same spatial size as the low-level features through bilinear interpolation;

[0029] Then, point-by-point convolution is used to reduce the channel dimension to half of the original one. This process increases the information density of each channel and enhances the feature expression capability.

[0030] Finally, the adjusted features are merged and passed through another point-wise convolution to generate the fused feature output.

[0031] Furthermore, the EEM module is used to reduce computational complexity while enhancing edge information in the feature map. The EEM module aims to learn the mapping relationship between feature map edge information and true labels, thereby improving segmentation accuracy.

[0032] When the EEM module processes an input feature layer of size H×W×C:

[0033] First, the feature map is denoised through the average pooling layer, and then the Sobel layer is used to detect edge features through bidirectional convolution. The output is processed by multiple convolution layers and finally restored to the original size through shape reconstruction. The process is as follows:

[0034]

[0035] Among them, X, is the output of EEM, Represents the sobel layer, σ represents the double convolution layer, AvgPool represents the average pooling of the input X, and Reshape represents adjusting the shape of the data;

[0036] Each of the multiple convolutional layers includes a convolution operation and a ReLU activation.

[0037] Furthermore, the core unit of the EEM module is the Sobel layer, which is used to carry the Sobel operator. The Sobel operator is a discrete differential operator consisting of two 3×3 filters. The Sobel operator performs a convolution operation on the input image to obtain the gradient of the image in different directions, and then extracts the image edge information through threshold screening. The Sobel operator not only has a smoothing effect on noise, but also produces better edge detection results and provides more accurate edge information.

[0038] The calculation process of the Sobel operator is described as follows:

[0039]

[0040] Among them, A is the input of the Sobel layer, G x is the gradient value of the image in the x direction, G y is the gradient value of the image in the y direction, and G is the final gradient value.

[0041] Furthermore, the focal Tversky loss function is defined as follows:

[0042]

[0043] Among them, the value range of γ is [1,3], X represents the predicted segmentation area, Y represents the real labeled area, |X∩Y| represents the intersection of the two areas, |XY| corresponds to false positive, that is, the model misclassifies the negative area as positive, |YX| corresponds to false negative, that is, the model misses the real positive area,

[0044] When γ>1, the loss function will strengthen the focus on misclassified predictions;

[0045] The parameters α and β are used to control the penalty weights of false positives and false negatives, respectively, and satisfy the constraint condition of α + β = 1.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. This paper proposes an MCME-UNet model that integrates the MCMS module. Through a hierarchical feature extraction mechanism and a multimodal feature fusion strategy optimized by the MEM module, it significantly improves segmentation accuracy while maintaining computational efficiency. In particular, the EEM module innovatively applies the Sobel operator and residual connection to simultaneously improve tumor boundary continuity and segmentation accuracy.

[0048] 2. This invention introduces a new training paradigm using the focal Tversky loss function, which not only effectively addresses the class imbalance problem and enhances the model's sensitivity to small tumor areas, but also generates smoother segmentation boundaries, significantly improving the stability and accuracy of segmentation results.

[0049] 3. Systematic experimental verification of the present invention on the BraTS2019 and BraTS2023 datasets shows that MCME-UNet is significantly superior to existing methods in terms of segmentation effect and consistency. Its Dice coefficient reaches 92.11%, the HD value is reduced to 2.58, and only 21.16M parameters are required, showing excellent clinical practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 This is a schematic diagram of the overall architecture of MCME-UNet of the present invention;

[0052] Figure 2 This is a schematic diagram of the MCMS module architecture of the present invention;

[0053] Figure 3 Schematic diagram of the structure of the serialized multi-scale convolutional layer with residual connections in the CMS module of the present invention;

[0054] Figure 4 This is a schematic diagram of the framework structure of the MEM of the present invention;

[0055] Figure 5 Schematic diagram of the structure of the edge enhancement module of the present invention;

[0056] Figure 6 This is a schematic diagram of the focal Tversky loss of the present invention;

[0057] Figure 7Schematic diagram of the evolution of DSC values ​​on the BraTS2019 and BraTS2023 validation subsets of the present invention;

[0058] Figure 8 Schematic diagram comparing the segmentation effects of the present invention on BraTS2023;

[0059] Figure 9 Schematic diagram of the comparative loss metric across models on the training and validation datasets of the present invention;

[0060] Figure 10 It is a visualization diagram of feature mapping in the feature extraction process of the present invention. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0062] See also Figure 1 , a brain tumor segmentation method based on cascaded multi-scale convolution and modality enhancement of Mamba architecture, comprising at least the following steps:

[0063] S1: Based on the Mamba framework, Mamba-structured UNet, cascaded multi-scale convolution, and modality enhancement, we propose the MCME-UNet model. Built on a state-space model, the MCME-UNet model leverages Mamba's advanced modeling capabilities to improve performance and effectively address the limitations of the basic Mamba model in brain tumor segmentation tasks. The MCME-UNet model includes a Mamba-enhanced cascaded multi-scale convolution module, a modality enhancement module, and an edge enhancement module, and employs a focal Tversky loss function. The Mamba-enhanced cascaded multi-scale convolution module is the MCMS module, the modality enhancement module is the MEM module, and the edge enhancement module is the EEM module.

[0064] S2: The MCMS module simultaneously captures local details and global context features through a hierarchical feature extraction mechanism, effectively compensating for the shortcomings of the Mamba structure in local feature modeling. The MCMS module is based on the predicted segmentation result framework of the convolutional layer and adopts a cascaded multi-scale convolution structure containing 3×3, 5×5, and 7×7 convolution kernels. Cascaded multi-scale convolutions are integrated in the decoding path to enhance detail perception and global context understanding capabilities. A hierarchical encoder is used that integrates three-way spatial Mamba blocks and cascaded multi-scale convolution modules.

[0065] S3: The MEM module is added to the top of the encoder hierarchy, using dual data streams and contrastive learning to ensure balanced and comprehensive feature information across scales. The MEM module optimizes the multimodal feature fusion process through innovative designs such as channel dimension adjustment and dual data stream contrastive learning, significantly reducing the sensitivity of the Mamba architecture to multimodal data redundancy and feature imbalance.

[0066] S4: The global multi-scale features of the encoder are connected to the decoder to achieve feature reuse and enhanced by adding EEM modules at different encoding stages. The EEM module uses residual connections and the Sobel operator to emphasize edge details in feature maps and assists in accurate segmentation by learning the mapping relationship between edge features and true labels. The encoder extracts features hierarchically through alternating global and local processing stages, while the decoder gradually restores the space through explicit edge guidance. This significantly improves the continuity of tumor boundary segmentation while maintaining computational efficiency, successfully overcoming Mamba's shortcomings in capturing edge details.

[0067] S5: Finally, through the collaborative optimization of the EEM module and the focal Tversky loss function, the MCME-UNet model can not only utilize multi-scale features and edge prior information in the decoding stage, but also enhance the segmentation ability of tumor boundaries and small lesions by dynamically adjusting the loss weights, effectively alleviating the class imbalance problem and enhancing the focus on small tumor areas.

[0068] The MCMS module is used to improve the model's contextual modeling capabilities in complex scenarios and the robustness and generalization of feature expression. The MCMS module aims to fully extract and adaptively integrate multi-scale features, thereby enhancing the model's ability to recognize and segment diverse tumor morphologies.

[0069] The MCMS module effectively captures rich contextual information from different scales by constructing a multi-scale feature pyramid. At the same time, it introduces the Mamba structure to dynamically model and enhance the extracted multi-level features, further improving feature discrimination capabilities.

[0070] This design has achieved a key breakthrough based on the traditional module. Multi-scale convolution has powerful local detail modeling capabilities, while the Mamba mechanism has excellent global semantic modeling capabilities and efficient long-range dependency capture capabilities. The advantage of Mamba comes from its design concept based on the structured state space model (S6). By introducing a state transfer mechanism that depends on the input data, it significantly alleviates the problems of high memory usage and gradient disappearance in the traditional state space model, while achieving computational efficiency that is linearly related to the length of the input sequence. This synergistic fusion of local and global feature modeling mechanisms enables the MCMS module to construct a more expressive and robust multi-scale feature representation while maintaining efficient computing. Ultimately, this module not only improves the model's perception of fine-grained structures in complex backgrounds, but also significantly enhances its segmentation stability and generalization capabilities in tumor areas with different deformations. The specific design structure is as follows. Figure 2 shown.

[0071] The MCMS module adopts a cascaded multi-scale design. This design gradually expands the receptive field through a hierarchical feature progression architecture, integrating multi-scale information. Each layer extracts features from the previous layer's output and fuses shallow detail features with deep semantic features through residual or skip connections. This progressive feature extraction method effectively models multi-scale contextual information from local to global scales, enhancing the model's ability to represent complex structures.

[0072] At the same time, with the help of the cross-level feature fusion mechanism, the MCMS module captures key area details at different resolutions, thereby improving the model's sensitivity to target boundaries and global semantic understanding. It not only significantly enhances the hierarchical representation capability of feature expression, but also achieves efficient utilization of multi-scale features through gradual refinement.

[0073] The CMS layer in the MCME-UNet network model uses convolutional kernels of various sizes, including 3×3, 5×5, and 7×7, to accurately adapt to the feature extraction of multimodal brain MRI images. Each kernel size achieves full-scale feature capture at different spatial levels.

[0074] The 3×3 core is better at focusing on fine details than the 5×5 and 7×7 cores;

[0075] The 5×5 kernel is more accurate than the 3×3 and 7×7 kernels in capturing mid-level features;

[0076] The 7×7 kernel can cover a wider context area than the small kernel, thus achieving comprehensive feature extraction across scales;

[0077] Figure 3 The CMS structure is shown, showing the feature processing flow from 7×7 convolution to 5×5 convolution to multiple 3×3 convolutions.

[0078] This cascaded multi-scale approach enriches the neural network's ability to discern complex details and broader patterns within the brain. Compared to a single large-scale kernel, the CMS module significantly reduces the computational load and facilitates efficient and robust feature extraction, thereby formalizing the integration of these changes into a unified analytical framework.

[0079] The CMS description formula is as follows:

[0080]

[0081] Among them, x represents an input feature map with a dimension of (B, C, H, W). For a given H, W and C represent the height, width and number of channels respectively, and the initial scale feature Extracted from the stem layer; convolution normalization function F k×k Perform operations with a kernel size of k×k; Represents the output feature map of each layer, where k represents the convolution kernel size and n represents the number of iterations in the convolution sequence of the kernel size;

[0082] The output of each layer will be passed as input to the subsequent convolutional layer with the same kernel size, and the output of the last layer will be passed to the convolutional layer with a smaller kernel size. After the final 3×3 convolution, the operation sequence will converge to generate the final output feature map output;

[0083] The configuration sets a=1, indicating that 7×7ConvBN performs only a single iteration; b=1, indicating that 5×5ConvBN performs a single operation; and c=3, indicating that three 3×3ConvBN operations are performed consecutively.

[0084] The MCMS module aims to optimize the integration of semantic information and spatial features more effectively;

[0085] In traditional UNet models and their variants, feature fusion is typically achieved through skip connections connecting the encoder and decoder paths. However, this simple interaction often fails to fully exploit the rich semantic information of deep features or the fine spatial details of shallow features, and may introduce unnecessary computational overhead due to the inclusion of redundant data. To overcome these limitations, we propose a Modality Enhancement Module (MEM) to more effectively optimize the integration of semantic information and spatial features.

[0086] See Figure 4 Through point-by-point convolution operations, the low-level spatial features have their channel number reduced to 3 / 4 of their original size. At the same time, through bilinear interpolation operations, the high-level semantic features are adjusted to the same size as the low-level semantic features;

[0087] The MEM module is placed between the encoder and decoder. By synchronously adjusting the channel dimensions of high-level semantic features and low-level spatial features, the application of the MEM module includes at least the following steps:

[0088] Point-by-point convolution is used to compress the channel dimension of low-level spatial features to 3 / 4 of the original size, and high-level semantic features are first adjusted to the same spatial size as the low-level features through bilinear interpolation;

[0089] Then, point-by-point convolution is used to reduce the channel dimension to half of the original one. This process increases the information density of each channel and enhances the feature expression capability.

[0090] Finally, the adjusted features are merged and passed through another point-wise convolution to generate the fused feature output.

[0091] Set the size to H×W×C in The low-level spatial feature X is defined as the output feature map Y, and the formula of the output feature map Y is as follows:

[0092]

[0093] Where W is the size C out ×C in The weight matrix of b is the size C out The bias vector of [i, j, l] and [k] are indices;

[0094] In the MEM module, point-by-point convolution operations are used to reduce the feature channel dimension, compressing the number of channels of low-level spatial features to 3 / 4 of the original size, while halving the number of channels of high-level features;

[0095] Bilinear interpolation is a common method for feature map size adjustment. Figure X The spatial dimensions of (size H×W) are adjusted to the output feature map Y (size H′×W′);

[0096] For the value of any position (x, y) in the output feature map Y, based on the input feature Figure X The weighted average of the four nearest neighbor points is used to determine the output feature map Y at position (x, y, c):

[0097] Y[x,y,c]=(1-dx)×(1-dy)×X[x floor ,y floor ,c]

[0098] +dx×(1-dy)×X[x ceil ,y floor ,c]

[0099] +(1-dx)×dy×X[xfloor ,y ceil ,c]

[0100] +dx×dy×X[x ceil ,y ceil ,c]

[0101] at the same time,

[0102]

[0103]

[0104] dx=x in -x floor ,dy=y in -y floor

[0105] This bilinear interpolation operation is used to adjust the spatial dimensions of the high-level semantic features to match the low-level spatial features, thereby ensuring that the subsequent fusion and processing dimensions are aligned. low\_proc ) and high-level semantic features (size H×W×C high\_proc ) are fused through splicing operations, and further feature integration is optimized using point-by-point convolution. The merged features obtained after splicing are Figure X merged The dimensions are H×W×(C low\_proc +C high\_proc ), the final fusion output Y fused (including C fused The calculation formula of channel is as follows:

[0106]

[0107] Where W is the size of C fused ×(C low\_proc +C high\_proc ) matrix, representing the weight; b is a matrix of size C fused A vector representing the bias.

[0108] The Modality Enhancement Module (MEM) effectively integrates low-level and high-level features, effectively alleviating the semantic mismatch between low-level and high-level features and the coarse multimodal fusion problems in traditional UNet skip connections. This approach enables the model to simultaneously capture rich spatial details and semantic information in images, significantly improving overall performance and feature representation capabilities.

[0109] The EEM module is used to reduce computational complexity while enhancing edge information in feature maps. Considering that the final segmentation result is a binary mask of a brain tumor, the EEM module aims to learn the mapping relationship between feature map edge information and true labels, thereby improving segmentation accuracy.

[0110] like Figure 5 As shown, when the EEM module processes the input feature layer of size H×W×C:

[0111] First, the feature map is denoised through the average pooling layer, and then the Sobel layer is used to detect edge features through bidirectional convolution. The output is processed by multiple convolution layers and finally restored to the original size through shape reconstruction. The process is as follows:

[0112]

[0113] Among them, X, is the output of EEM, Represents the sobel layer, σ represents the double convolution layer, AvgPool represents the average pooling of the input X, and Reshape represents adjusting the shape of the data;

[0114] Each layer of the multiple convolutional layers contains convolution operations and ReLU activations.

[0115] The core unit of the EEM module is the Sobel layer, which is used to carry the Sobel operator. As one of the most important operators in image edge detection methods, the Sobel operator is a discrete differential operator consisting of two 3×3 filters. The Sobel operator performs a convolution operation on the input image to obtain the gradient of the image in different directions, and then extracts the image edge information through threshold screening. The Sobel operator not only has a smoothing effect on noise, but also produces better edge detection results and provides more accurate edge information.

[0116] The calculation process of the Sobel operator is described as follows:

[0117]

[0118] Among them, A is the input of the Sobel layer, G x is the gradient value of the image in the x direction, G y is the gradient value of the image in the y direction, and G is the final gradient value.

[0119] In the multimodal brain tumor image segmentation task, the tumor region typically occupies only a very small portion of the image, while the background region covers a larger area. The traditional cross-entropy loss function is affected by class imbalance, which can cause the model to favor background regions over tumor regions. In contrast, the focal Tversky loss function directly calculates the proportion of overlapping regions, allowing the model to focus more on segmenting small regions, resulting in smoother segmentation boundaries. This function not only helps the decoder better recover spatial details but also provides stable gradient updates, significantly improving training stability and segmentation performance.

[0120] The focal Tversky loss function is defined as follows:

[0121]

[0122] The value range of γ is [1,3]. In practical applications, when the pixel is misclassified but the Tversky exponent is high, the focal Tversky loss function is basically unaffected; on the contrary, if the focal Tversky exponent is small and the pixel is misclassified, the loss value will drop significantly.

[0123] X represents the predicted segmentation area, Y represents the real labeled area, |X∩Y| represents the intersection of the two areas, |XY| corresponds to false positive (FP), that is, the model misclassifies the negative area as positive, |YX| corresponds to false negative (FN), that is, the model misses the real positive area.

[0124] When γ>1, the loss function will strengthen the focus on misclassified predictions, but as the classification accuracy improves and approaches convergence, it will produce over-suppression. Figure 6 As shown in Figure 1, as the value of γ increases, the Tversky exponent value will be mapped to a flatter focal Tversky loss function curve area. Experiments show that when γ = 4 / 3, the model achieves the best performance;

[0125] The parameters α and β are used to control the penalty weights for false positives (FP) and false negatives (FN), respectively, and satisfy the constraint of α + β = 1. When α = β = 0.5, the focal Tversky loss function degenerates into the Dice loss function. A minimal constant ∈ is introduced in the formula to avoid division by zero errors. This loss function provides a more flexible method for solving the problem of class imbalance by applying differentiated penalty weights to false positives and false negatives. Through the above function design, it can ensure that the model adapts to the overall shape and position characteristics of the segmented area while ensuring the accuracy of the prediction of each pixel.

[0126] Based on the above embodiment, the following experimental verification is proposed:

[0127] Dataset

[0128] This paper uses BraTS2019 and BraTS2023 datasets to verify the effectiveness of the model. All data samples have been aligned, skull stripped and 1mm 3 The images are normalized to isotropic resolution and stored in the NIfTI (.nii.gz) format. These datasets provide detailed segmentation annotations of brain tumors, dividing them into three main regions: the whole tumor, the tumor core, and the enhancing tumor. Figure 7 The training process of the model on two datasets is shown.

[0129] The BRATS dataset is one of the most commonly used benchmark datasets in brain tumor segmentation research. It contains MRI scans of four modalities: T1 contrast-enhanced (T1CE), T2 weighted (T2), and fluid-attenuated inversion recovery (FLAIR). Each modality provides different imaging features that contribute to a comprehensive understanding of brain tumors and their accurate segmentation. As an open competition platform, BRATS has brought together a large number of researchers and has rich community support and benchmarking systems. The dataset has the following significant features: (1) high-quality expert annotations; (2) unified data formats and standards; (3) higher requirements for deep learning models - they need to have strong feature expression and generalization capabilities, while addressing challenges such as data imbalance, computing resource limitations, and multimodal fusion.

[0130] BraTS2019: The training data consists of MRI scans from 335 glioma patients, each with segmentation labels manually annotated by experts. Annotated tumor regions include enhancing tumor (ET), necrotic and non-enhancing tumor core (TC), and peritumoral edema (ED). The test dataset includes 125 cases whose ground truth labels are not publicly available. This dataset includes both high-grade gliomas (HGG) and low-grade gliomas (LGG), providing high-quality annotations for model development and validation.

[0131] BraTS2023: The number of training cases has been increased to 1,251 patients, with a more diverse data source to enhance model generalization. The annotation scheme has also changed, now labeling tumor regions as enhancing tumor (ET), tumor core (TC), and whole tumor (WT). The test dataset contains approximately 530 cases, with 219 cases used for validation.

[0132] Evaluation Metrics

[0133] When evaluating model performance, three key indicators are mainly relied upon: Dice similarity coefficient (DSC), sensitivity (Sensitivity) and Hausdorff distance (HD). DSC is an indicator used to evaluate the degree of overlap between binary segmentation results and the true value. DSC is a commonly used indicator to measure the performance of segmentation models, especially for medical image segmentation tasks. Its value range is 0 to 1, and the larger the value, the closer the segmentation result is to the true value. The Dice similarity coefficient is used by almost all studies to evaluate the performance of segmentation models. It is defined as follows:

[0134]

[0135] Here, A represents the set of segmentation results, and B represents the set of ground truth values. This formula is a commonly used metric for measuring segmentation model performance, particularly for medical image segmentation tasks. Its value range is between 0 and 1. A larger value indicates that the segmentation result is closer to the ground truth.

[0136] Sensitivity, also known as recall, is defined as the ratio of the number of pixels correctly segmented as tumors to the number of all tumor pixels in the true value. Its calculation formula is defined as:

[0137]

[0138] In addition, HD is used to measure the maximum distance between two point sets and is often used to evaluate the worst-case boundary distance between the segmentation result and the ground truth. Its calculation formula is defined as:

[0139]

[0140] S and T are two non-empty point sets, d(s,t) represents the basic distance between points s and t, sup and inf represent the supremum (minimum upper bound) and infimum (maximum lower bound), respectively, and HD can quantify the distance of the point in the segmentation result that deviates farthest from the true boundary, which is very useful for evaluating the performance of the model in the worst case.

[0141] Experimental results

[0142] On the divided BraTS2023 dataset, such as Figure 8 As shown in the figure, we conducted comparative experiments on several UNet and Transformer networks. The visualization results show that MCME-UNet can more accurately capture the true contours of the tumor region in multiple slice dimensions (axial, sagittal, and coronal). Specifically, compared with the labeled image (label), it fits the tumor boundary more closely and demonstrates greater robustness in identifying small target areas. Taking the samples in the third row of the figure as an example, MCME-UNet successfully segmented the small red lesion area, while other methods showed significant underdetection. In addition, it maintains good recovery capabilities for the overall morphology and internal structural details of the tumor region in multiple samples, demonstrating strong multi-scale perception and semantic modeling capabilities. Experiments show that our proposed MCME-UNet network performs better on this dataset.

[0143] Based on the MCME-UNet model, we constructed an edge enhancement module based on residual connections and incorporated it into each encoder stage. This module is capable of learning topologically structured features without significantly increasing computational complexity. As shown in Table 1, we compare the floating-point operations (FLOPs) and parameter counts of different models. On the BraTS2023 dataset, our model achieves the lowest FLOPs (11G) and parameter count (21.16M).

[0144] Table 1: Comparison of method parameters and FLOPs

[0145]

[0146] Note: FLOPs stands for floating-point operations. Parameters represent adjustable variables or weights within the model, learned during training, to dictate its behavior and effectiveness. Bold values ​​indicate optimal values.

[0147] Our model sets a new benchmark for computational efficiency, requiring only 21.16 million parameters and 9G

[0148] This optimized architecture significantly reduces resource requirements while maintaining high performance, making it particularly suitable for deployment in computing-constrained environments without sacrificing performance quality.

[0149] In the comprehensive evaluation of neural network architectures, the changing trend of training / validation loss is the key analysis indicator. Figure 9 We compared the performance curves of architectures such as DenseNet, UNet3P, TransBTS, UNetR, and Segmamba. Our model achieved a dual advantage—achieving the lowest training loss (0.08) and validation loss (0.11). Through epoch-by-epoch loss visualization, these empirical data strongly demonstrate that our model has faster convergence and stronger generalization ability than existing methods. Quantization traces further demonstrate the effectiveness of our architecture in extracting hierarchical feature representations and reducing the risk of overfitting.

[0150] We also conducted a detailed evaluation of several mainstream brain tumor segmentation algorithms, using the evaluation metrics described in Section 4.2. Our model performed exceptionally well on BraTS2019, achieving Dice coefficients of 0.86, 0.88, and 0.84 for the whole tumor region (WT), tumor core (TC), and enhancing tumor (ET), respectively, outperforming other compared methods (see Table 2). Notably, our model exhibited a slight decrease in sensitivity for the whole tumor (WT) and enhancing tumor (ET) regions, but demonstrated superior detection capabilities in other tumor regions.

[0151] Table 2: Results of Dice, HD, and Sensitivity indicators of various algorithms on the BraTS2019 validation set.

[0152]

[0153] Note: The bold part indicates the optimal value.

[0154] The experimental results on the BraTS2023 dataset further verified the segmentation performance advantage of MCME-UNet. We compared the models of three types of network architectures: CNN, Transformer, and Mamba. The detailed results are shown in Table 3. MCME-UNet performed outstandingly in both the Dice coefficient (92.11%) and Hausdorff distance (HD = 2.58), significantly outperforming the current state-of-the-art methods such as SwinUNetR (88.23%) and Hi-Mamba (90.06%), indicating that it has better overall segmentation accuracy and pixel category prediction capabilities. In addition, the HD index of MCME-UNet far exceeds that of traditional UNetR (5.49) and the recent Segmamba (3.56), proving that it can better capture the boundaries and detailed features of the target area in the segmentation task.

[0155] Table 3: Three quantitative comparison models of BraTS2023

[0156]

[0157] Note: The bold part indicates the optimal value.

[0158] In addition, we visualize the feature maps during training ( Figure 10 , different colors indicate the model's area of ​​interest, and the darker the color, the higher the attention weight), showing how the model identifies the key features of tumor segmentation layer by layer. The first row shows, from left to right, the multi-level features extracted from the input image during the forward propagation of the model: the first few layers capture low-level texture and edge information, such as brain tissue contours, tumor edges, etc.; as the network deepens, the feature map becomes more abstract and can focus on higher-level semantic areas. The second row shows the process of feature fusion and upsampling, which gradually restores the abstract features to the original space to achieve more refined spatial positioning capabilities. This process effectively retains the discriminative features of key tumor areas, and ultimately enables the model to complete accurate segmentation.

[0159] Ablation experiments

[0160] To deeply analyze the contribution of each network component, we respectively analyze the cascaded multi-scale (CMS) module, modal enhancement (ME) module, edge enhancement (EE) module and L Focal-TverskyWe systematically studied the loss function. By gradually introducing various modules and comparing performance changes, we verified the independent impact of each component on the final segmentation effect. The experimental results are shown in Table 4. Based on the data in the table, we can draw the following conclusions:

[0161] Edge enhancement (EE) strengthens the model's ability to identify complex tumor morphologies by enhancing edge information. Experiments show that while the overall segmentation accuracy remains largely unchanged when the EE module is added, significant deficiencies in tumor boundary prediction (especially in the enhanced detail processing of tumor regions) are observed. This demonstrates the unique value of the EE module in improving edge prediction accuracy.

[0162] The introduction of the Cascaded Multi-Scale (CMS) module enables the model to more flexibly adapt to the actual morphological characteristics of tumors, particularly excelling in identifying irregular and complex tumor shapes. Ablation experiments show that when the CMS module is missing, the average DSC value drops from 87.50 to 86.42, and the HD value increases from 2.60 to 2.68. The model's performance significantly decreases when processing tumor regions of varying shapes, demonstrating the module's irreplaceable role in enhancing model adaptability and segmentation accuracy.

[0163] The introduction of the Modal Enhancement (ME) module significantly improves the model's ability to distinguish tumors from non-tumor tissue, particularly in clarifying tumor boundaries and enhancing the recognition of internal tumor structures. Ablation experiments show that the addition of this module significantly improves average DSC by 1.55% and decreases HD by 0.1, demonstrating the ME module's significant value in improving model recognition and segmentation accuracy.

[0164] Loss function L Foca-Tversky The component is a key element in optimizing the model to capture tumor edge details, especially in predicting the boundaries of small tumors. This component effectively improves the model's ability to recognize complex tumor morphologies by emphasizing the continuity of the boundary. Experimental data show that when the L Focal-Tversky After adding the component, the average DSC increased to 89.39 and the HD value decreased to 2.48. This verifies the regulatory value of this loss function component in improving edge prediction accuracy.

[0165] Table 4: Ablation experiments of MCME-UNet on BraTS2019 and BraTS2023 datasets, the impact of adding CM, ME, EE and Loss

[0166]

[0167] Note: Here, DSC and HD values ​​are taken as the average of the segmentation, WT, TC and ET are added together, and the bold part is the optimal value.

[0168] To verify the impact of convolution kernel size on the model's feature extraction capabilities, we conducted comparative experiments using convolution kernels of different scales. The study found that the selection of convolution kernel size in the MCMS module significantly affects the model's feature extraction performance—convolution kernels of different scales differ significantly in capturing spatial structure and texture information. As shown in Table 5, we conducted comparative experiments using three multi-scale convolution combinations to determine the optimal convolution scale. The experimental results show that the convolution kernel combination of (3, 5, 7) achieves the best segmentation performance: large-scale convolution kernels (7×7) excel at extracting macroscopic structural features, medium-scale convolution kernels (5×5) balance receptive field and detail preservation, and small-scale convolution kernels (3×3) effectively capture fine-grained texture features. This multi-scale collaborative mechanism significantly improves the overall performance of brain tumor segmentation.

[0169] Table 5: Experimental results on the effects of three different sizes of convolution kernels on multi-scale feature extraction

[0170]

[0171] In summary:

[0172] This paper proposes a brain tumor segmentation MCME-UNet model based on the Mamba framework, Mamba-structured UNet, cascaded multi-scale convolution, and modality enhancement. The MCMS module uses multi-scale convolution and the Mamba mechanism to extract and dynamically enhance multi-level features, constructing a feature pyramid to capture contextual information. MEM optimizes the fusion of high-level semantic features and low-level spatial features between the encoder and decoder, while EEM utilizes residual connections and the Sobel operator to enhance edge details. Furthermore, the model uses a focal Tversky loss function to alleviate class imbalance, thereby improving small region segmentation.

[0173] This paper proposes the MCME-UNet model—a novel brain tumor segmentation framework that integrates Mamba-structured state-space models (SSMs), cascaded multi-scale convolutions, and modality enhancement modules. This model employs a dual-strategy design: layered multi-scale convolution kernels are used to extract detailed features, global dependencies are modeled using the Mamba architecture, and a focal Tversky loss function is introduced to address class imbalance. This combination improves segmentation accuracy while reducing computational complexity. The cascaded multi-scale (CMS) module captures features at different spatial levels, enabling the model to simultaneously identify fine structures and macroscopic patterns. The modality enhancement module (MEM) optimizes multimodal feature fusion, and the edge enhancement module (EEM) refines boundary details using the Sobel operator and residual connections.

[0174] Our model demonstrates exceptional performance on the BraTS2019 and BraTS2023 datasets, achieving a mean Dice coefficient of 92.11% and an HD value of 2.58 on BraTS2023, surpassing other methods. Requiring only 21.16M parameters, this model boasts the best parameter efficiency among similar methods, demonstrating its potential for efficient deployment in resource-constrained environments. Its significant advantages in improving the accuracy and efficiency of brain tumor segmentation provide an innovative solution for medical image analysis.

[0175] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A brain tumor segmentation method based on cascaded multi-scale convolution and modality enhancement based on the Mamba architecture, characterized by: At least the following steps are included: S1: Based on the Mamba framework, Mamba structure UNet, cascaded multi-scale convolution and modality enhancement, an MCME-UNet model is proposed. The MCME-UNet model is established on the basis of the state space model. The MCME-UNet model includes a Mamba enhanced cascaded multi-scale convolution module, a modality enhancement module and an edge enhancement module, and adopts a focal Tversky loss function. The Mamba enhanced cascaded multi-scale convolution module is the MCMS module, the modality enhancement module is the MEM module, and the edge enhancement module is the EEM module. S2: The MCMS module simultaneously captures local details and global context features through a hierarchical feature extraction mechanism, effectively compensating for the shortcomings of the Mamba structure in local feature modeling. The MCMS module is based on the predicted segmentation result framework of the convolutional layer and adopts a cascaded multi-scale convolution structure containing 3×3, 5×5, and 7×7 convolution kernels. Cascaded multi-scale convolutions are integrated in the decoding path to enhance detail perception and global context understanding capabilities. A hierarchical encoder is used that integrates three-way spatial Mamba blocks and cascaded multi-scale convolution modules. S3: Adding the MEM module at the top of the encoder hierarchy, using dual data streams and contrastive learning to ensure balanced and comprehensive feature information across scales. The MEM module optimizes the multimodal feature fusion process through innovative designs such as channel dimension adjustment and dual data stream contrastive learning. S4: The global multi-scale features of the encoder are connected to the decoder to achieve feature reuse and enhanced by adding EEM modules at different encoding stages. The EEM module uses residual connections and the Sobel operator to emphasize edge details in the feature map and assists in accurate segmentation by learning the mapping relationship between edge features and true labels. The encoder extracts features hierarchically through alternating global-local processing stages, while the decoder gradually restores the space through explicit edge guidance. This significantly improves the continuity of tumor boundary segmentation while maintaining computational efficiency. S5: Finally, through the collaborative optimization of the EEM module and the focal Tversky loss function, the MCME-UNet model can not only utilize multi-scale features and edge prior information in the decoding stage, but also enhance the segmentation ability of tumor boundaries and small lesions by dynamically adjusting the loss weights, effectively alleviating the class imbalance problem and enhancing the focus on small tumor areas.

2. The brain tumor segmentation method based on the Mamba architecture cascaded multi-scale convolution and modality enhancement according to claim 1 is characterized by: The MCMS module is used to improve the model's contextual modeling capabilities and the robustness and generalization performance of feature expression in complex scenarios. The MCMS module aims to fully extract and adaptively integrate multi-scale features, thereby enhancing the model's ability to recognize and segment diverse tumor morphologies. The MCMS module effectively captures rich contextual information from different scales by constructing a multi-scale feature pyramid. At the same time, it introduces the Mamba structure to dynamically model and enhance the extracted multi-level features, further improving the feature discrimination capability. The MCMS module adopts a cascaded multi-scale design, which is called CMS. It gradually expands the receptive field through a hierarchical feature progressive architecture and integrates multi-scale information. Each layer extracts features from the output of the previous level and fuses shallow detail features with deep semantic features through residual connections or skip connections. This progressive feature extraction method can effectively model multi-scale contextual information from local to global, enhancing the model's ability to represent complex structures. At the same time, with the help of the cross-level feature fusion mechanism, the MCMS module captures key area details at different resolutions, thereby improving the model's sensitivity to target boundaries and global semantic understanding. It not only significantly enhances the hierarchical representation capability of feature expression, but also achieves efficient utilization of multi-scale features through gradual refinement.

3. The brain tumor segmentation method based on the Mamba architecture cascaded multi-scale convolution and modality enhancement according to claim 2 is characterized by: The CMS layer in the MCME-UNet network model uses 3×3, 5×5, and 7×7 convolution kernels to accurately adapt to the feature extraction of multimodal brain MRI images. Each kernel size achieves full-scale feature capture at different spatial levels. The 3×3 core is better at focusing on fine details than the 5×5 and 7×7 cores; The 5×5 kernel is more accurate than the 3×3 and 7×7 kernels in capturing mid-level features; The 7×7 kernel can cover a wider context area than the small kernel, thus achieving comprehensive feature extraction across scales; The CMS description formula is as follows: Among them, x represents an input feature map with a dimension of (B, C, H, W). For a given H, W and C represent the height, width and number of channels respectively, and the initial scale feature Extracted from the stem layer; convolution normalization function F k×k Perform operations with a kernel size of k×k; Represents the output feature map of each layer, where k represents the convolution kernel size and n represents the number of iterations in the convolution sequence of the kernel size; The output of each layer will be passed as input to the subsequent convolutional layer with the same kernel size, and the output of the last layer will be passed to the convolutional layer with a smaller kernel size. After the final 3×3 convolution, the operation sequence will converge to generate the final output feature map output; The configuration sets a=1, indicating that 7×7ConvBN performs only a single iteration; b=1, indicating that 5×5ConvBN performs a single operation; and c=3, indicating that three 3×3ConvBN operations are performed consecutively.

4. The brain tumor segmentation method based on the Mamba architecture cascaded multi-scale convolution and modality enhancement according to claim 1 is characterized by: The MCMS module aims to optimize the integration of semantic information and spatial features more effectively; The MEM module is placed between the encoder and the decoder, and by synchronously adjusting the channel dimensions of high-level semantic features and low-level spatial features, the application of the MEM module includes at least the following steps: Point-by-point convolution is used to compress the channel dimension of low-level spatial features to 3 / 4 of the original size, and high-level semantic features are first adjusted to the same spatial size as the low-level features through bilinear interpolation; Then, point-by-point convolution is used to reduce the channel dimension to half of the original one. This process increases the information density of each channel and enhances the feature expression capability. Finally, the adjusted features are merged and passed through another point-wise convolution to generate the fused feature output.

5. The brain tumor segmentation method based on the Mamba architecture cascaded multi-scale convolution and modality enhancement according to claim 1 is characterized by: The EEM module is used to reduce computational complexity while enhancing edge information in the feature map. The EEM module aims to learn the mapping relationship between feature map edge information and true labels, thereby improving segmentation accuracy. When the EEM module processes an input feature layer of size H×W×C: First, the feature map is denoised through the average pooling layer, and then the Sobel layer is used to detect edge features through bidirectional convolution. The output is processed by multiple convolution layers and finally restored to the original size through shape reconstruction. The process is as follows: Among them, X 、 For the output of EEM, represents the sobel layer, σ represents the double convolution layer, and AvgPool represents the input X 、 Average pooling, Reshape means adjusting the shape of the data; Each of the multiple convolutional layers includes a convolution operation and a ReLU activation.

6. The brain tumor segmentation method based on the Mamba architecture cascaded multi-scale convolution and modality enhancement according to claim 5 is characterized by: The core unit of the EEM module is the Sobel layer, which is equipped with the Sobel operator. The Sobel operator is a discrete differential operator consisting of two 3×3 filters. The Sobel operator performs a convolution operation on the input image to obtain the gradient of the image in different directions, and then extracts the image edge information through threshold screening. The Sobel operator not only has a smoothing effect on noise, but also produces better edge detection results and provides more accurate edge information. The calculation process of the Sobel operator is described as follows: Among them, A is the input of the Sobel layer, G x is the gradient value of the image in the x direction, G y is the gradient value of the image in the y direction, and G is the final gradient value.

7. The brain tumor segmentation method based on the Mamba architecture cascaded multi-scale convolution and modality enhancement according to claim 1, characterized in that: The focal Tversky loss function is defined as follows: The value range of γ is [1,3], X represents the predicted segmentation area, Y represents the true labeled area, |X∩Y| represents the intersection of the two areas, |XY| corresponds to a false positive, that is, the model mistakenly judges the negative area as positive, and |YX| corresponds to a false negative, that is, the model misses the true positive area; When γ>1, the loss function will strengthen the focus on misclassified predictions; The parameters α and β are used to control the penalty weights of false positives and false negatives, respectively, and satisfy the constraint condition of α + β = 1.

Images