U-net brain tumor segmentation method fusing vmamba and cbam

Abstract

The application discloses a U-Net brain tumor segmentation method fusing VMamba and CBAM, and belongs to the field of medical image processing. Brain tumor medical images are collected and preprocessed to construct a training set; a brain tumor medical image segmentation model is constructed based on a U-Net model; a Dice-cross entropy-Focal compound total loss function is used to supervise and train the brain tumor medical image segmentation model; and a brain tumor medical image segmentation model that has completed training can obtain a high-precision tumor region segmentation result. The application realizes the collaborative optimization of local details and global semantics, improves the integrity and fineness of brain tumor segmentation, and provides reliable image evidence for clinical diagnosis and surgical planning.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and deep learning, specifically relating to a U-Net brain tumor segmentation method that integrates VMamba and CBAM. Background Technology

[0002] Currently, brain tumor segmentation mostly uses traditional U-Net and its improved models. Among them, traditional U-Net is the mainstream segmentation framework, but it has obvious technical defects in practical applications and is difficult to meet the requirements of accurate segmentation: First, the encoder uses a pure convolutional module with a limited receptive field, which cannot effectively model the long-distance spatial dependence between brain tumors and distant brain regions, and is prone to missing global lesion features; Second, the bottleneck layer design is simple, which leads to the inability to effectively integrate the multimodal features of the encoder, resulting in the semantic association of the global tumor region being broken, and ultimately the problem of incomplete tumor region segmentation; Third, during the upsampling process of the decoder, feature details are easily lost, and there is a lack of targeted feature selection mechanism, resulting in blurred tumor boundary segmentation; Fourth, cross-scale feature fusion only uses a simple feature concatenation method without screening and weighting high and low level features, which introduces a lot of redundant information, resulting in the loss of edge details, an increase in false positive regions, and problems such as missegmentation and missed segmentation.

[0003] Furthermore, existing improved models mostly rely solely on convolution or a single attention mechanism, failing to achieve coordinated optimization of global semantic modeling and local detail capture. This makes it difficult to balance segmentation accuracy and efficiency, and they cannot adapt to the characteristics of brain tumors, such as irregular morphology, strong lesion heterogeneity, and easy confusion with normal brain tissue. Therefore, there is an urgent need for a targeted improved segmentation method to address the shortcomings of the existing technologies. Summary of the Invention

[0004] To address the shortcomings of existing technologies, a U-Net brain tumor segmentation method integrating VMamba and CBAM is proposed. It is simple to use, has good segmentation results, and achieves a dual improvement in the completeness and precision of brain tumor segmentation results, meeting the requirements for accurate segmentation of segmented regions.

[0005] Technical Solution: To achieve the above technical objectives, this invention discloses a U-Net brain tumor segmentation method integrating VMamba and CBAM, the steps of which are as follows: S1. Collect and preprocess medical images of brain tumors to construct a training set; S2. Constructing a brain tumor medical image segmentation model: Conv-VMamba hybrid modules are embedded throughout the encoder, bottleneck layer, and decoder of the U-Net model, and a differentiated attention CBAM is designed: the encoder uses only CBAM channel attention to enhance tumor-related channels, the decoder uses only CBAM spatial attention to focus on the tumor edge, and the bottleneck layer and skip connections use full CBAM channel-spatial dual attention to achieve dual enhancement of local and global features; abandoning the direct stitching of U-Net, the attention-guided cross-scale feature fusion module first performs double weighting of the shallow features of the encoder with CBAM, and then stitches them together after matching with the feature dimensions of the decoder. S3. The brain tumor medical image segmentation model is trained under supervision using the Dice-cross-entropy-Focal composite total loss function. Once the brain tumor medical image segmentation model has been trained, high-precision tumor region segmentation results can be obtained.

[0006] Furthermore, the specific steps for preprocessing medical images of brain tumors are as follows: The input brain tumor medical images are resampled to a uniform size, and the images are normalized to standardize the data. Feature enhancement is then performed using contrast enhancement to amplify the grayscale differences between the tumor region and normal brain tissue and background regions, strengthening the edge and texture features of the tumor region and improving its distinguishability. The pixel values ​​of the brain tumor medical images are normalized using the following formula: , In the formula These are the normalized pixel values. Original medical images of brain tumors at pixel locations grayscale value at that location This represents the global grayscale mean of a single brain tumor medical image that is currently awaiting normalization. This represents the global grayscale standard deviation of a single brain tumor medical image that is currently awaiting normalization.

[0007] Furthermore, in the brain tumor medical image segmentation model, VMamba and U-Net models are deeply integrated: Conv-VMamba hybrid modules are embedded throughout the encoder, bottleneck layer, and decoder, and differentiated CBAM is designed according to the functional requirements of different modules of the model: the encoder uses only CBAM channel attention to enhance tumor-related channels, the decoder uses only CBAM spatial attention to focus on the tumor edge, and the bottleneck layer and skip connections use full CBAM channel-spatial dual attention to achieve dual enhancement of local and global features; abandoning the direct stitching of traditional U-Net, the attention-guided cross-scale feature fusion module first performs double weighting of shallow features of the encoder through CBAM, and then stitches them together after matching with the feature dimensions of the decoder, which effectively alleviates the semantic gap between the encoder and decoder, integrates local tumor details and global semantics, and improves segmentation accuracy.

[0008] Furthermore, the working process of the brain tumor medical image segmentation model is as follows: The encoder is a local-global dual-modal encoder: it adopts the CBAM channel attention enhancement mechanism, and uses a 3×3×3 three-dimensional convolution and Conv-VMamba hybrid module to encode the features of the input brain tumor medical image. While capturing local details such as tumor edges and textures, it models the long-distance semantic association between tumor tissue and distant brain regions, and filters tumor-related feature channels through channel attention to suppress background noise. The bottleneck layer is a VMamba-CBAM dual-enhancement bottleneck layer: it adopts the complete CBAM channel-space dual attention mechanism to perform global long-distance dependency modeling and dual feature weighting on the deepest encoded features output by the local-global dual-modal encoder. While strengthening the global semantic association of tumor tissue, it accurately focuses on the core area of ​​the tumor and further suppresses background interference. The decoder is a global semantic preservation feature recovery decoder: the input of the first layer is the deep global enhancement feature output from the bottleneck layer, and the input of subsequent layers is the fused feature output from the cross-scale feature fusion module of the previous layer; it performs upsampling on the input features through transposed convolution to restore the spatial size of the feature map, eliminates upsampling artifacts and refines texture through 3×3×3 convolution, maintains global long-distance semantic association through the Conv-VMamba module to avoid semantic loss caused by upsampling, embeds the CBAM spatial attention module to focus on the tumor edge region and generate spatial enhancement features of this layer, and finally inputs the generated spatial enhancement features into the attention-guided cross-scale feature fusion module of this layer to complete the feature transfer within the layer; The fusion module is an attention-guided cross-scale feature fusion module: the shallow features output by the encoder at the corresponding level are subjected to channel-spatial dual weighting by the complete CBAM module to generate attention-guided skip connection features; after dimensional matching of the skip connection features with the spatial enhancement features output by the decoder at the current level, they are spliced ​​and fused along the channel dimension to output the cross-scale fusion features of this level; the fusion features of non-last layers are passed to the decoder feature recovery module of the next level, and the fusion features of the last layer are passed to the output and iterative optimization module, so as to effectively alleviate the semantic gap between high and low level features and significantly improve the accuracy and edge integrity of tumor region segmentation.

[0009] Furthermore, the specific process by which the encoder obtains encoded features containing multi-scale semantic information is as follows: First, a 3×3×3 convolution is used to capture local details such as tumor edges and textures to obtain local features. ; Then, local features are processed through the VMamba module. Perform long-distance semantic association modeling: , In the formula The specific steps are as follows: first, select local features The dimension is matched to the input dimension of the VMamba state space, as shown in the formula. , For linear projection layers, The intermediate feature tensor after linear projection represents the features after dimensional adaptation, used for subsequent selective scanning; then, scanning is performed along the spatial dimensions of the feature map to model global long-range dependencies. , The global feature tensor output by the selective scanning module is the feature after modeling long-distance association, carrying global long-distance semantics; These are gating parameters used to control the intensity of information updates; Let be the parameter matrix of the state space, which controls the state evolution, input mapping, and output mapping, respectively. This is a bias term used to adjust the final output global semantic features. This allows us to capture long-distance semantic associations between tumor tissue and distant brain regions, thus addressing the issues of irregular morphology and discontinuous edges in brain tumor lesions. Finally, the results of the selective scanning are projected back onto the original feature dimensions to obtain the global semantic features. , Then, global semantic features Embedded CBAM channel attention Weighting: , In the formula, ⊗ represents channel-by-channel multiplication; Let be the channel attention weight generation function, denoted as The implementation steps are as follows: first, process the input global semantic features... Corresponding 3D feature tensor Perform global pooling, where For global average pooling, The process involves global max pooling; then, the resulting 1D feature vector is fed into global average pooling. and global max pooling A shared two-layer fully connected network, in which , For a two-layer fully connected network, the trainable weight matrix is... Used for feature dimensionality reduction, Used for feature dimensionality enhancement; finally, channel attention weights are generated through the Sigmoid activation function to strengthen tumor-related channels and suppress background noise interference, thus solving the problems of strong background interference and missed segmentation of small lesions in brain tumor segmentation. Finally, downsampling is performed using a 3×3×3 convolution with a stride of 2, according to the formula: To obtain multi-scale coding features This provides multi-scale lesion features for subsequent decoder upsampling to restore lesion resolution and attention-guided cross-scale fusion, ultimately achieving precise segmentation of brain tumor lesion areas.

[0010] Furthermore, the specific working process of the VMamba-CBAM dual-enhancement bottleneck layer is as follows: First, the multi-scale encoded features output by the local-global dual-modal encoder are processed by concatenating the Conv-VMamba module. The process is further refined to model the long-distance dependency between tumor lesions and normal brain tissue, outputting the global features of the bottleneck layer. Specifically: First ,Again ;in, The multi-scale encoded features output by the local-global dual-modal encoder include local features of tumor lesions and normal brain tissue. A 3×3×3 three-dimensional convolution is used as a preprocessing tool to enhance local tumor features, extract tumor edges and texture details, and prepare for long-distance modeling. The local feature tensor after convolution enhancement; Using the VMamba state-space model as a core tool, it employs a selective scan operator to globally scan along the feature map space dimension, modeling the long-distance semantic dependencies between tumor lesions and normal brain tissue. Ultimately, it outputs global features of the bottleneck layer that carry long-distance correlation information. This addresses the technical challenges of irregular lesion morphology, discontinuous edges, and missed segmentation in brain tumor segmentation, providing global semantic feature support for subsequent accurate lesion segmentation. Global characteristics of the bottleneck layer output by the Conv-VMamba module through the complete CBAM module. Perform dual weighting of channel and space: , In the formula, For the refined feature tensor; ⊗ represents the global feature tensor of the bottleneck layer; ⊗ represents element-wise multiplication. Let be the channel attention weight generation function, denoted as ,in, The Sigmoid activation function is used to normalize feature values ​​to the [0,1] interval and generate channel attention weights. , For a two-layer fully connected network, the trainable weight matrix is... Used for feature dimensionality reduction, Used for feature dimensionality enhancement; ReLU activation function is used to introduce nonlinearity and enhance the network's feature representation ability; For global average pooling, global average pooling is performed on the input features to extract global statistical information in the channel dimension; For global max pooling, global max pooling is performed on the input features to extract significant feature information in the channel dimension; (⋅) is the spatial attention weight generation function, expressed as: ,in, This is an average pooling operation, which performs spatial dimension average pooling on the input features; This is a max pooling operation; it performs spatial dimension max pooling on the input features. A 3×3×3 three-dimensional convolutional layer is used to fuse pooling features and generate spatial attention weights; This is a channel-dimensional concatenation operation used to concatenate and fuse features from average pooling and max pooling at the channel dimension. The input feature tensor contains channel attention. Input Global features of the bottleneck layer output by the Conv-VMamba module Spatial attention Input (⋅) These are intermediate feature tensors after channel weighting, and all are 3D feature tensors after model processing. Global characteristics of the bottleneck layer output by the Conv-VMamba concatenated module through the complete CBAM module The process involves both channel and spatial weighting, specifically: During the channel attention weighting stage, the attention is weighted by... Global average pooling and global max pooling are performed in the spatial dimension, followed by a fully connected layer and Sigmoid activation to generate channel attention weights. Perform channel weighting to obtain intermediate features For the spatial attention weighting stage, this intermediate feature is used. As input, average pooling and max pooling are performed along the channel dimension, followed by 3×3×3 convolution for dimensionality reduction and Sigmoid activation to generate spatial attention weights for intermediate features. Perform spatial weighting; Finally, through dual weighting of channel-space attention within the complete CBAM attention framework, highly discriminative global semantic features are output. This provides feature support for the precise segmentation of brain tumor lesions in the future.

[0011] Furthermore, the feature recovery process of the global semantic-preserving encoder is as follows: First, high-recognition global semantic features are processed using transposed convolution. Perform a 2x upsampling to restore the feature map size: , In the formula, It is a 2×2×2 three-dimensional transposed convolutional layer used to perform upsampling on global semantic features and restore the spatial size of the feature map; The features obtained after transposed convolution and upsampling are used for subsequent detail refinement; then, a 3×3×3 convolution is used to eliminate upsampling artifacts and refine the texture, resulting in enhanced detail features. The VMamba module maintains global semantic associations, avoids semantic loss due to upsampling, and outputs decoder features. Finally, the decoder features output by the VMamba module are... Embedded CBAM spatial attention focuses on the tumor margin region: , In the formula, The decoder characteristics output by the VMamba module; (⋅) is the spatial attention weight generation function; for The spatial enhancement tensor after CBAM spatial attention weighting carries the enhanced features after focusing on the tumor edge; this process strengthens the spatial response of the tumor edge region through spatial attention weighting, suppresses background noise, and ultimately improves the segmentation accuracy of brain tumor lesion edges.

[0012] Furthermore, the attention-guided cross-scale feature fusion process is as follows: Shallow features of the corresponding layer output of the local-global dual-modal encoder Processing: Shallow features By applying channel and spatial weighting to the complete CBAM module, we obtain the attention-guided skip connectivity feature tensor. : , In the formula, This is a channel attention weight generation function, used to generate channel attention weights, enhance tumor-related feature channels, and suppress background noise; (⋅) is the spatial attention weight generation function, used to generate spatial attention weights and focus on the spatial response of the tumor region; ⊗ is the element-wise multiplication, used to weight and fuse the attention weight tensor with the original feature tensor to achieve feature enhancement; Attention-guided jump connection feature tensor Spatial augmentation features of the current level of the decoder After performing dimensional matching and concatenation, cross-scale fused features are obtained. : , In the formula, This is a channel-level splicing operation, responsible for splicing the treaty-linked features after the shallow features of the encoder have been weighted by CBAM. Spatial augmentation features corresponding to the output of the decoder layer ,in, The encoder carries the local details of the tumor. The tumor global semantic information recovered by the decoder is carried out. After dimensional matching and concatenation, the final output retains a feature map that combines the local details of the encoder and the global semantics of the decoder.

[0013] Furthermore, the output and iterative optimization process is as follows: the fused features output by the attention-guided cross-scale feature fusion module of the last layer are... Inputting into the Softmax layer generates the probability distribution of the tumor region: the following formula is used to calculate each spatial coordinate in the image. The probability of a pixel belonging to a tumor category is calculated. The probability values ​​of all pixels together constitute the probability distribution of the tumor region in the entire brain tumor medical image. Regions with high probability correspond to tumor lesions, while regions with low probability correspond to normal brain tissue. This achieves probabilistic prediction of tumor regions and provides a probabilistic basis for subsequent lesion segmentation. , In the formula, C represents the number of categories in the brain tumor segmentation task. This is an index for the category dimension, corresponding to different category channels output by the Softmax layer; coordinates The predicted probability that a pixel belongs to the tumor category ranges from [0,1]. The closer the value is to 1, the higher the probability that the pixel belongs to the tumor region. This is an exponential operation used for probability normalization calculations in Softmax.

[0014] Furthermore, the Dice-cross-entropy-Focal composite total loss function Iterative optimization of the brain tumor medical image segmentation model: The fused features output by the last layer attention-guided cross-scale feature fusion module are used. The probability distribution of the tumor region at the pixel level is generated through a Softmax layer, and the total loss function is used. The difference between the value and the true tumor label is calculated, and this difference drives the model to backpropagate and update the parameters to complete iterative optimization. Total loss function as follows: , , , , in This represents the Dice loss function, which measures the overlap between the predicted tumor region and the actual tumor region, avoiding missed lesion classification and improving the integrity of region segmentation. This represents the cross-entropy loss function, used to measure the accuracy of pixel-level classification, optimize segmentation details, and improve pixel classification accuracy. This represents the Focal loss function, which is used to focus on hard-to-classify samples and solve the problems of class imbalance and missed classification of lesions. To train the hyperparameter weight coefficients, they are used to balance the proportions of Dice loss, cross-entropy loss, and Focal loss in the total loss; coordinates The predicted probability that a pixel belongs to the tumor category ranges from [0,1]. The closer the value is to 1, the higher the probability that the pixel belongs to the tumor region. coordinates The true label of each pixel (1 represents the tumor area, 0 represents normal brain tissue). pass , and The three types of losses complement each other and work synergistically. While ensuring the overlap of tumor regions and the accuracy of pixel classification, the learning weights for difficult sample regions such as tumor edges and small lesions are strengthened, which further improves the accuracy and robustness of the segmentation results until the brain tumor medical image segmentation model converges.

[0015] A computer device includes a processor and a memory, the processor being electrically connected to the memory, the memory being used to store instructions and data, and the processor being used to execute the U-Net brain tumor segmentation method merging VMamba and CBAM as described in any one of claims 1 to 8.

[0016] Beneficial Effects: The U-Net brain tumor segmentation method based on VMamba and attention enhancement provided in this application can encode features of the input image through a local-global dual-modal encoder, modeling the long-distance semantic association between tumor tissue and distant brain regions; it can perform global long-distance dependency modeling and feature weighting of encoded features through a VMamba-CBAM dual-enhancement bottleneck layer, strengthening the global semantic association of tumor tissue; it can filter and weight multi-scale features output by the encoder through an attention-guided cross-scale feature fusion module, alleviating the semantic gap between high- and low-level features; and it can upsample and restore fused features through an attention-guided VMamba decoder, preserving tumor edge details and reducing semantic information loss during the restoration process. This application achieves accurate segmentation of brain tumors through the synergistic optimization of VMamba and differentiated CBAM attention mechanisms, providing more reliable image evidence for clinical diagnosis and surgical planning. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the brain tumor medical image segmentation model architecture provided in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the U-Net brain tumor segmentation method that integrates VMamba and CBAM according to the present invention. Detailed Implementation

[0019] The embodiments of the present invention will be further described below with reference to the accompanying drawings: like Figure 1 As shown, this invention discloses a brain tumor medical image segmentation model. A Conv-VMamba hybrid module is embedded throughout the encoder, bottleneck layer, and decoder of the U-Net model, and a differentiated attention CBAM is designed: the encoder uses only CBAM channel attention to enhance tumor-related channels, the decoder uses only CBAM spatial attention to focus on the tumor edges, and the bottleneck layer and skip connections use full CBAM channel-spatial dual attention to achieve dual enhancement of local and global features; abandoning the direct stitching of U-Net, the attention-guided cross-scale feature fusion module first performs double weighting of the encoder's shallow features using CBAM, and then stitches them together after matching with the decoder's feature dimensions. The local-global bimodal encoder replaces traditional convolutional blocks with a 3×3×3 3D convolutional module and a Conv-VMamba hybrid module to model long-range semantic associations between tumor tissue and distant brain regions, and outputs multi-scale shallow features at each downsampling level. A VMamba-CBAM dual-enhancement bottleneck layer performs global long-range dependency modeling and CBAM channel-space dual weighting on the deepest encoded features output by the encoder, outputting deep global enhanced features, which are then passed to the top layer of the decoder as the initial input for hierarchical upsampling. An attention-guided cross-scale feature fusion module performs channel-space dual weighting on the shallow features of the corresponding encoder level using a complete CBAM module, generating attention-guided skip connection features; these features are then combined with the current layer of the decoder. After matching the spatial enhancement features at each level, they are concatenated along the channel dimension to output the cross-scale fusion features of this level. The fusion features that are not in the last layer are passed to the feature recovery module of the next layer decoder. The first layer of the global semantic preservation decoder is the bottleneck layer enhancement feature, and the subsequent layers are the fusion features output by the cross-scale feature fusion module of the previous layer. The decoder performs transposed convolution upsampling, 3×3×3 convolution thinning, Conv-VMamba to maintain global semantics, and CBAM spatial attention to focus on the tumor edge to generate spatial enhancement features, which are input to the cross-scale feature fusion module of this level. Finally, the cross-scale feature fusion module guided by the last layer of attention outputs accurate detail fusion features, which are input to the output and iterative optimization module composed of a Softmax probability generation layer, a Dice-cross-entropy-Focal composite total loss function, and a backpropagation parameter update mechanism to complete brain tumor segmentation.

[0020] like Figure 2 As shown, a U-Net brain tumor segmentation method integrating VMamba and CBAM is described below: Step 1: Input Sample Construction.

[0021] First, the input image is resampled to a uniform size H×W to eliminate size differences between different scanning devices; then, the image pixel values ​​are normalized using formula (1): (1), In equation (1), For the original image at pixel location The gray value at that location, μ is the global mean of the image at that modality, and σ is the global standard deviation. The values ​​are normalized pixel values; finally, the sample set is expanded by random flipping, rotation, and other methods.

[0022] Step 2: Local-Global Bimodal Coding.

[0023] First, a 3×3×3 convolution is used to capture local details such as tumor edges and textures to obtain local features. ; Then through the VMamba module For long-distance semantic association modeling, the formula is (2): (2), In formula (2) The specific steps are as follows: First, input features The input dimension projected onto the state space is given by the formula: , This is a linear projection layer used to match the input dimension of the state space. Then, it scans along the spatial dimensions of the feature map to model long-range dependencies. , These are gating parameters used to control the intensity of information updates; These are parameter matrices in the state space, controlling state evolution, input mapping, and output mapping, respectively. This is a bias term used to adjust the final output. Finally, the results of the selective scan are projected back onto the original feature dimensions to obtain the global semantic features. Re-embedded CBAM channel attention pairs The weighted average is calculated using formula (3): (3), In equation (3), ⊗ represents channel-by-channel multiplication; Let be the channel attention weight generation function, denoted as The implementation steps are as follows: first, global pooling is performed on the input feature 3D map; then, the pooled vector is fed into a shared two-layer fully connected network; and finally, channel attention weights are generated through the Sigmoid activation function.

[0024] Finally, downsampling is performed using a 3×3×3 convolution with a stride of 2, according to the formula... Obtain multi-scale coding features .

[0025] Step 3: VMamba-CBAM dual-enhancement bottleneck layer processing.

[0026] First, the long-range dependency between the tumor and the whole brain is further modeled by concatenating Conv-VMamba modules, and the output features are then generated. Specifically: First ,Again .

[0027] Finally, channel and spatial weighting is performed using the complete CBAM module, and the formula is (4): (4) In equation (4), (⋅) is the spatial attention weight generation function, expressed as: The implementation steps are as follows: For the input feature map, perform average pooling and max pooling in the channel dimension to obtain two H×W×D feature maps. Concatenate these two feature maps by channel, then reduce the dimension through a 3×3×3 convolutional layer, and finally generate spatial attention weights for each position through the Sigmoid activation function.

[0028] Finally, through dual weighting of channel and spatial attention in the CBAM module, highly recognizable global semantic features are output. .

[0029] Step 4: Global semantic-preserving decoder feature recovery.

[0030] First, through transposed convolution pairs To restore the feature map size by performing a 2x upsampling, the formula is (5): (5), Then, a 3×3×3 convolution is used to eliminate upsampling artifacts and refine the texture to obtain the features. Then, global semantic associations are maintained through the VMamba module, and features are output. Finally, CBAM spatial attention is embedded to focus on the tumor margin region, as shown in formula (6): (6), Step 5: Attention-guided cross-scale feature fusion.

[0031] First, the corresponding shallow features output by the encoder are processed, and then weighted both channel and spatially using a complete CBAM module to obtain the features. The formula is (7): (7), Finally Features of the current level of the decoder After performing dimensional matching and concatenation, the formula is (8): (8), In equation (8), This is a channel-level stitching operation, responsible for stitching. and The final output retains a feature map that combines the local details of the encoder and the global semantics of the decoder.

[0032] Step Six: Output and Iterative Optimization.

[0033] First, the final fusion features will be... The softmax layer is input to generate the probability distribution of the tumor region, as shown in the formula: (9), In equation (9), C is the number of categories. For pixels, Predicted probability of belonging to a tumor category.

[0034] Finally, to simultaneously consider regional overlap, pixel classification accuracy, and focusing on difficult samples, formula (10) is adopted: (10) In equation (10), , , ;in, For pixels, Predicted probability of belonging to a tumor category. This is the actual label for that pixel.

[0035] This method deeply integrates VMamba and U-Net: Conv-VMamba hybrid modules are embedded throughout the encoder, bottleneck layer, and decoder, breaking through the receptive field limitations of traditional convolution, accurately modeling the long-distance semantic association between brain tumors and the whole brain, and solving the problem of weak global semantic modeling capabilities of traditional U-Net.

[0036] Differentiated CBAM attention architecture design: To meet the functional requirements of different modules of the model, a differentiated CBAM is designed: the encoder uses only CBAM channel attention to enhance tumor-related channels, the decoder uses only CBAM spatial attention to focus on the tumor edge, and the bottleneck layer and skip connections use full CBAM channel-spatial dual attention to achieve dual enhancement of local and global features, which is different from the single general reuse of traditional CBAM.

[0037] Attention-guided cross-scale feature fusion: Abandoning the direct concatenation of traditional U-Net, the attention-guided cross-scale feature fusion module first performs double weighting of the shallow features of the encoder using CBAM, and then concatenates them after matching with the feature dimensions of the decoder. This effectively alleviates the semantic gap between the encoder and decoder, integrates local tumor details with global semantics, and improves segmentation accuracy.

[0038] The Dice-CE-Focal composite total loss function integrates Dice loss, cross-entropy loss, and Focal loss. By balancing the proportions of these three factors through trainable weights α, β, and γ, it simultaneously considers region integrity, classification accuracy, and learning from difficult samples, thus solving the technical problems of class imbalance and lesion omission in brain tumor segmentation.

[0039] This application does not merely perform single-dimensional optimization, but rather conducts a collaborative optimization of global semantic modeling and local detail capture. Addressing the limitation of traditional convolutional models, which can only capture local neighborhood information, this application designs a Conv-VMamba hybrid module embedded in the entire encoder, bottleneck layer, and decoder process. It uses 3×3×3 three-dimensional convolution to capture fine local features such as brain tumor edges and textures, while leveraging VMamba's selective spatial scanning mechanism to model the global long-distance semantic association between tumor tissue and distant brain regions. This achieves a dual balance between local detail capture and global semantic modeling in brain tumor segmentation, overcoming the technical shortcomings of traditional pure convolutional modules, which have limited receptive fields and cannot simultaneously capture both local and global features.

[0040] Furthermore, compared to the previous single and universal configuration of attention mechanisms, this application has carried out differentiated customization of attention mechanisms, so that the attention enhancement strategies of each module can be accurately adapted to their own feature processing goals, effectively avoiding ineffective enhancement and feature redundancy, and maximizing the feature selection and weighting role of attention mechanisms.

[0041] The advantages of this application in brain tumor segmentation are as follows: Brain tumor medical images are 3D spatial structures, and different scanning devices can lead to differences in image size and grayscale intensity. Traditional 2D convolution is prone to losing interlayer correlation information, and non-standardized images result in weak model generalization ability, making it unable to adapt to diverse clinical scanning data. This application has made two targeted designs: First, the entire model process adopts 3×3×3 three-dimensional convolution to adapt to the spatial structure of 3D medical images of brain tumors, accurately capture the interlayer correlation features of tumors, and avoid the spatial information loss problem of 2D convolution; Second, a standardized 3D image preprocessing workflow is designed, which first resamples the input images to a uniform size, performs global grayscale normalization on pixel values ​​to eliminate size and intensity differences between different scanning devices, and then improves the distinction between tumors and background through contrast enhancement and randomly flips / rotates to expand the sample set, so that the model can adapt to clinical scanning data of brain tumors with different devices and different morphologies.

[0042] Brain tumor medical imaging is characterized by its cross-brain region distribution and highly irregular morphology. Traditional convolution can only capture local information and cannot model the long-distance semantic relationship between the tumor and distant brain regions, easily leading to incomplete lesion segmentation and global semantic fragmentation. This application designs a Conv-VMamba hybrid module and embeds the encoder, bottleneck layer, and decoder throughout the process. It uses 3D convolution to capture local fine features such as tumor edges and textures, while relying on VMamba's selective spatial scanning mechanism to globally scan along the feature map spatial dimension and model the long-distance semantic dependency between tumor tissue and distant brain regions. This breaks through the receptive field limitation of traditional convolution, allowing brain tumor lesions distributed across regions to form a complete global semantic relationship, solving the core problems of incomplete segmentation and semantic fragmentation from the source of feature encoding.

[0043] The background of brain tumor medical imaging is complex normal brain tissue, and tumor features are easily submerged by background noise. Furthermore, tumor edges are blurred and difficult to distinguish from normal brain tissue boundaries, leading to strong background interference and blurred edge segmentation. This application addresses these issues by implementing a differentiated CBAM attention design tailored to the different feature extraction needs and hierarchical semantic characteristics of each module. The encoder embeds only channel attention to accurately select tumor-related feature channels and strongly suppress background noise from normal brain tissue, preventing tumor features from being submerged. The decoder embeds only spatial attention to focus on the spatial location information of the tumor edge, enhance the feature response of the edge region, and accurately distinguish the boundary between the tumor and normal brain tissue, solving the problem of blurred edge segmentation. The bottleneck layer and skip connections embed complete CBAM channel-spatial dual attention to achieve dual enhancement of tumor features, further improving the distinguishability between tumor features and the background.

[0044] The segmentation of brain tumor medical images suffers from significant class imbalance, with difficult-to-classify samples such as micro-lesions and tumor infiltration areas easily overlooked. Furthermore, a semantic gap exists between the shallow details of the encoder and the deep semantic features of the decoder; direct concatenation can introduce redundant information, leading to missed classification of micro-lesions and an increase in false positives. This application addresses this issue from two specific aspects: First, it designs an attention-guided cross-scale feature fusion module, abandoning the traditional direct concatenation method. It first performs CBAM channel-spatial dual-weighted filtering on the shallow features of the encoder, retaining effective features such as micro-lesions and edge details while filtering out redundant background information. Then, it matches these features with the corresponding level of deep semantic features from the decoder before concatenation, effectively mitigating the semantic gap between high and low-level features. Second, it constructs a Dice-cross-entropy-Focal composite total loss function. Focal loss assigns higher learning weights to difficult-to-classify samples such as micro-lesions and infiltration areas of brain tumors, while Dice loss ensures the integrity of region segmentation, and cross-entropy loss optimizes pixel classification accuracy. This addresses the problems of missed classification of difficult samples and class imbalance from a training supervision perspective.

[0045] The core architecture of this model is a general medical image segmentation optimization framework. For example, in the Conv-VMamba hybrid module, the general design of convolution capturing local details and VMamba modeling global semantics aims to solve the common problem of difficulty in simultaneously considering local and global features in medical image segmentation, rather than being a module specifically for brain tumors. Another example is the differentiated CBAM attention architecture in this application, which is a general design. Its core logic is to match different attention dimensions according to the different feature extraction requirements and hierarchical semantic characteristics of each module in the model. The CBAM attention mechanism used in this application has the core function of channel attention: filtering target lesion-related features and suppressing background interference; and spatial attention: focusing on lesion edges and enhancing boundary discrimination, which is also applicable to all irregularly shaped and blurred-edge solid tumors / lesions. The core function of the complete CBAM attention mechanism is dual enhancement of global features and mitigation of the cross-scale semantic gap, which is a common optimization direction for all U-Net-like segmentation models. When segmentation is required for lesions in other locations, the U-shaped structure, module combination method, and embedding logic of this model do not need to be modified. Only the convolution kernel size needs to be fine-tuned according to the image morphology (2D / 3D) of the target lesion, or the scanning window size of VMamba needs to be fine-tuned according to the lesion size, etc., to adapt to the segmentation needs of lesions in other locations such as lung cancer and liver cancer.

Claims

1. A U-Net brain tumor segmentation method integrating VMamba and CBAM, characterized in that, The steps are as follows: S1. Collect and preprocess medical images of brain tumors to construct a training set; S2. Constructing a brain tumor medical image segmentation model: Conv-VMamba hybrid modules are embedded throughout the encoder, bottleneck layer, and decoder of the U-Net model, and a differentiated attention CBAM is designed: the encoder uses only CBAM channel attention to enhance tumor-related channels, the decoder uses only CBAM spatial attention to focus on the tumor edge, and the bottleneck layer and skip connections use full CBAM channel-spatial dual attention to achieve dual enhancement of local and global features; abandoning the direct stitching of U-Net, the attention-guided cross-scale feature fusion module first performs double weighting of the shallow features of the encoder with CBAM, and then stitches them together after matching with the feature dimensions of the decoder. S3. The brain tumor medical image segmentation model is trained under supervision using the Dice-cross-entropy-Focal composite total loss function. Once the brain tumor medical image segmentation model has been trained, high-precision tumor region segmentation results can be obtained.

2. The U-Net brain tumor segmentation method integrating VMamba and CBAM according to claim 1, characterized in that, The specific steps for preprocessing medical images of brain tumors are as follows: The input brain tumor medical images are resampled to a uniform size, and the images are normalized to standardize the data. Feature enhancement is then performed using contrast enhancement to amplify the grayscale differences between the tumor region and normal brain tissue and background regions, strengthening the edge and texture features of the tumor region and improving its distinguishability. The pixel values ​​of the brain tumor medical images are normalized using the following formula: ， In the formula These are the normalized pixel values. Original medical images of brain tumors at pixel locations grayscale value at that location This represents the global grayscale mean of a single brain tumor medical image that is currently awaiting normalization. This represents the global grayscale standard deviation of a single brain tumor medical image that is currently awaiting normalization.

3. The U-Net brain tumor segmentation method integrating VMamba and CBAM according to claim 2, characterized in that, In the brain tumor medical image segmentation model, VMamba and U-Net models are deeply integrated: Conv-VMamba hybrid modules are embedded throughout the encoder, bottleneck layer, and decoder, and differentiated CBAM is designed according to the functional requirements of different modules: the encoder uses only CBAM channel attention to enhance tumor-related channels, the decoder uses only CBAM spatial attention to focus on the tumor edge, and the bottleneck layer and skip connections use full CBAM channel-spatial dual attention to achieve dual enhancement of local and global features; abandoning the direct stitching of traditional U-Net, the attention-guided cross-scale feature fusion module first performs double weighting of shallow features of the encoder using CBAM, and then matches them with the feature dimensions of the decoder before stitching, which effectively alleviates the semantic gap between the encoder and decoder, integrates local tumor details and global semantics, and improves segmentation accuracy.

4. The U-Net brain tumor segmentation method integrating VMamba and CBAM according to claim 3, characterized in that, The working process of the brain tumor medical image segmentation model is as follows: The encoder is a local-global dual-modal encoder: it adopts the CBAM channel attention enhancement mechanism, and uses a 3×3×3 three-dimensional convolution and Conv-VMamba hybrid module to encode the features of the input brain tumor medical image. While capturing local details such as tumor edges and textures, it models the long-distance semantic association between tumor tissue and distant brain regions, and filters tumor-related feature channels through channel attention to suppress background noise. The bottleneck layer is a VMamba-CBAM dual-enhancement bottleneck layer: it adopts the complete CBAM channel-space dual attention mechanism to perform global long-distance dependency modeling and dual feature weighting on the deepest encoded features output by the local-global dual-modal encoder. While strengthening the global semantic association of tumor tissue, it accurately focuses on the core area of ​​the tumor and further suppresses background interference. The decoder is a global semantic preservation feature recovery decoder: the input of the first layer is the deep global enhancement feature output by the bottleneck layer, and the input of the subsequent layers is the fused feature output by the cross-scale feature fusion module of the previous layer; It performs upsampling on the input features through transposed convolution to restore the spatial size of the feature map. It eliminates upsampling artifacts and refines texture through 3×3×3 convolution. It maintains global long-distance semantic association through the Conv-VMamba module to avoid semantic loss caused by upsampling. It embeds the CBAM spatial attention module to focus on the tumor edge region and generate spatial enhancement features at this level. Finally, the generated spatial enhancement features are input to the attention-guided cross-scale feature fusion module at this level to complete the feature transfer within the level. The fusion module is an attention-guided cross-scale feature fusion module: the shallow features output by the corresponding level of the encoder are subjected to channel-space dual weighting by the complete CBAM module to generate attention-guided skip connection features; After dimensional matching of the skip connection feature with the spatial enhancement feature output by the current level of the decoder, the feature is spliced ​​and fused along the channel dimension to output the cross-scale fusion feature of this level. The fusion features from non-last layers are passed to the next level decoder feature recovery module, while the fusion features from the last layer are passed to the output and iterative optimization module. This effectively alleviates the semantic gap between high and low level features and significantly improves the accuracy and edge integrity of tumor region segmentation.

5. The U-Net brain tumor segmentation method integrating VMamba and CBAM according to claim 3, characterized in that, The specific process by which the encoder obtains encoded features containing multi-scale semantic information is as follows: First, a 3×3×3 convolution is used to capture local details such as tumor edges and textures to obtain local features. ; Then, local features are processed through the VMamba module. Perform long-distance semantic association modeling: ， In the formula The specific steps are as follows: first, select local features The dimension is matched to the input dimension of the VMamba state space, as shown in the formula. , For linear projection layers, The intermediate feature tensor after linear projection represents the features after dimensional adaptation, used for subsequent selective scanning; then, scanning is performed along the spatial dimensions of the feature map to model global long-range dependencies. , The global feature tensor output by the selective scanning module is the feature after modeling long-distance association, carrying global long-distance semantics; These are gating parameters used to control the intensity of information updates; Let be the parameter matrix of the state space, which controls the state evolution, input mapping, and output mapping, respectively. This is a bias term used to adjust the final output global semantic features. This allows us to capture long-distance semantic associations between tumor tissue and distant brain regions, thus addressing the issues of irregular morphology and discontinuous edges in brain tumor lesions. Finally, the results of the selective scanning are projected back onto the original feature dimensions to obtain the global semantic features. , Then, global semantic features Embedded CBAM channel attention Weighting: ， In the formula, ⊗ represents channel-by-channel multiplication; Let be the channel attention weight generation function, denoted as The implementation steps are as follows: first, process the input global semantic features... Corresponding 3D feature tensor Perform global pooling, where For global average pooling, The process involves global max pooling; then, the resulting 1D feature vector is fed into global average pooling. and global max pooling A shared two-layer fully connected network, in which , For a two-layer fully connected network, the trainable weight matrix is... Used for feature dimensionality reduction, Used for feature dimensionality enhancement; finally, channel attention weights are generated through the Sigmoid activation function to strengthen tumor-related channels and suppress background noise interference, thus solving the problems of strong background interference and missed segmentation of small lesions in brain tumor segmentation. Finally, downsampling is performed using a 3×3×3 convolution with a stride of 2, according to the formula: To obtain multi-scale coding features This provides multi-scale lesion features for subsequent decoder upsampling to restore lesion resolution and attention-guided cross-scale fusion, ultimately achieving precise segmentation of brain tumor lesion areas.

6. The U-Net brain tumor segmentation method integrating VMamba and CBAM according to claim 5, characterized in that, The specific working process of the VMamba-CBAM dual-enhancement bottleneck layer is as follows: First, the multi-scale encoded features output by the local-global dual-modal encoder are processed by concatenating the Conv-VMamba module. The process is further refined to model the long-distance dependency between tumor lesions and normal brain tissue, outputting the global features of the bottleneck layer. Specifically: First ,Again ;in, The multi-scale encoded features output by the local-global dual-modal encoder include local features of tumor lesions and normal brain tissue. A 3×3×3 three-dimensional convolution is used as a preprocessing tool to enhance local tumor features, extract tumor edges and texture details, and prepare for long-distance modeling. The local feature tensor after convolution enhancement; Using the VMamba state-space model as a core tool, it employs a selective scan operator to globally scan along the feature map space dimension, modeling the long-distance semantic dependencies between tumor lesions and normal brain tissue. Ultimately, it outputs global features of the bottleneck layer that carry long-distance correlation information. This addresses the technical challenges of irregular lesion morphology, discontinuous edges, and missed segmentation in brain tumor segmentation, providing global semantic feature support for subsequent accurate lesion segmentation. Global characteristics of the bottleneck layer output by the Conv-VMamba module through the complete CBAM module. Perform dual weighting of channel and space: ， In the formula, For the refined feature tensor; ⊗ represents the global feature tensor of the bottleneck layer; ⊗ represents element-wise multiplication. Let be the channel attention weight generation function, denoted as ， in, The Sigmoid activation function is used to normalize feature values ​​to the [0,1] interval and generate channel attention weights. , For a two-layer fully connected network, the trainable weight matrix is... Used for feature dimensionality reduction, Used for feature dimensionality enhancement; ReLU activation function is used to introduce nonlinearity and enhance the network's feature representation ability; For global average pooling, global average pooling is performed on the input features to extract global statistical information in the channel dimension; For global max pooling, global max pooling is performed on the input features to extract significant feature information in the channel dimension; (⋅) is the spatial attention weight generation function, expressed as: ， in, This is an average pooling operation, which performs spatial dimension average pooling on the input features; This is a max pooling operation; it performs spatial dimension max pooling on the input features. A 3×3×3 three-dimensional convolutional layer is used to fuse pooling features and generate spatial attention weights; This is a channel-dimensional concatenation operation used to concatenate and fuse features from average pooling and max pooling at the channel dimension. The input feature tensor contains channel attention. Input Global features of the bottleneck layer output by the Conv-VMamba module Spatial attention Input (⋅) These are intermediate feature tensors after channel weighting, and all are 3D feature tensors after model processing. Global characteristics of the bottleneck layer output by the Conv-VMamba concatenated module through the complete CBAM module The process involves both channel and spatial weighting, specifically: During the channel attention weighting stage, the attention is weighted by... Global average pooling and global max pooling are performed in the spatial dimension, followed by a fully connected layer and Sigmoid activation to generate channel attention weights. Perform channel weighting to obtain intermediate features For the spatial attention weighting stage, this intermediate feature is used. As input, average pooling and max pooling are performed along the channel dimension, followed by 3×3×3 convolution for dimensionality reduction and Sigmoid activation to generate spatial attention weights for intermediate features. Perform spatial weighting; Finally, through dual weighting of channel-space attention within the complete CBAM attention framework, highly discriminative global semantic features are output. This provides feature support for the precise segmentation of brain tumor lesions in the future.

7. The U-Net brain tumor segmentation method integrating VMamba and CBAM according to claim 6, characterized in that, The feature recovery process of a global semantic-preserving encoder is as follows: First, high-recognition global semantic features are processed using transposed convolution. Perform a 2x upsampling to restore the feature map size: ， In the formula, It is a 2×2×2 three-dimensional transposed convolutional layer used to perform upsampling on global semantic features and restore the spatial size of the feature map; The features obtained after transposed convolution and upsampling are used for subsequent detail refinement; then, a 3×3×3 convolution is used to eliminate upsampling artifacts and refine the texture, resulting in enhanced detail features. The VMamba module maintains global semantic associations, avoids semantic loss due to upsampling, and outputs decoder features. Finally, the decoder features output by the VMamba module are... Embedded CBAM spatial attention focuses on the tumor margin region: ， In the formula, The decoder characteristics output by the VMamba module; (⋅) is the spatial attention weight generation function; for The spatial enhancement tensor after CBAM spatial attention weighting carries the enhancement features after focusing on the tumor edge; This process enhances the spatial response of the tumor margin region through spatial attention weighting, suppresses background noise, and ultimately improves the segmentation accuracy of brain tumor lesion margins.

8. The U-Net brain tumor segmentation method integrating VMamba and CBAM according to claim 7, characterized in that, The attention-guided cross-scale feature fusion process is as follows: Shallow features of the corresponding layer output of the local-global dual-modal encoder Processing: Shallow features By applying channel and spatial weighting to the complete CBAM module, we obtain the attention-guided skip connectivity feature tensor. : ， In the formula, This is a channel attention weight generation function, used to generate channel attention weights, enhance tumor-related feature channels, and suppress background noise; (⋅) is the spatial attention weight generation function, used to generate spatial attention weights and focus on the spatial response of the tumor region; ⊗ is the element-wise multiplication, used to weight and fuse the attention weight tensor with the original feature tensor to achieve feature enhancement; Attention-guided jump connection feature tensor Spatial augmentation features of the current level of the decoder After performing dimensional matching and concatenation, cross-scale fused features are obtained. : ， In the formula, This is a channel-level splicing operation, responsible for splicing the treaty-linked features after the shallow features of the encoder have been weighted by CBAM. Spatial augmentation features corresponding to the output of the decoder layer ,in, The encoder carries the local details of the tumor. The tumor global semantic information recovered by the decoder is carried out. After dimensional matching and concatenation, the final output retains a feature map that combines the local details of the encoder and the global semantics of the decoder.

9. The U-Net brain tumor segmentation method integrating VMamba and CBAM according to claim 8, characterized in that, The output and iterative optimization process is as follows: the fused features output by the last attention-guided cross-scale feature fusion module are... Inputting into the Softmax layer generates the probability distribution of the tumor region: the following formula is used to calculate each spatial coordinate in the image. The probability of a pixel belonging to a tumor category is calculated. The probability values ​​of all pixels together constitute the probability distribution of the tumor region in the entire brain tumor medical image. Regions with high probability correspond to tumor lesions, while regions with low probability correspond to normal brain tissue. This achieves probabilistic prediction of tumor regions and provides a probabilistic basis for subsequent lesion segmentation. ， In the formula, C represents the number of categories in the brain tumor segmentation task. This is an index for the category dimension, corresponding to different category channels output by the Softmax layer; coordinates The predicted probability that a pixel belongs to the tumor category ranges from [0,1]. The closer the value is to 1, the higher the probability that the pixel belongs to the tumor region. This is an exponential operation used for probability normalization calculations in Softmax.

10. The U-Net brain tumor segmentation method integrating VMamba and CBAM according to claim 9, characterized in that, Dice-cross-entropy-Focal composite total loss function Iterative optimization of the brain tumor medical image segmentation model: The fused features output by the last layer attention-guided cross-scale feature fusion module are used. The probability distribution of the tumor region at the pixel level is generated through a Softmax layer, and the total loss function is used. The difference between the value and the true tumor label is calculated, and this difference drives the model to backpropagate and update the parameters to complete iterative optimization. Total loss function as follows: ， ， ， ， in This represents the Dice loss function, which measures the overlap between the predicted tumor region and the actual tumor region, avoiding missed lesion classification and improving the integrity of region segmentation. This represents the cross-entropy loss function, used to measure the accuracy of pixel-level classification, optimize segmentation details, and improve pixel classification accuracy. This represents the Focal loss function, which is used to focus on hard-to-classify samples and solve the problems of class imbalance and missed classification of lesions. To train the hyperparameter weight coefficients, they are used to balance the proportions of Dice loss, cross-entropy loss, and Focal loss in the total loss; coordinates The predicted probability that a pixel belongs to the tumor category ranges from [0,1]. The closer the value is to 1, the higher the probability that the pixel belongs to the tumor region. coordinates The true label of each pixel (1 represents the tumor area, 0 represents normal brain tissue). pass , and The three types of losses complement each other and work synergistically. While ensuring the overlap of tumor regions and the accuracy of pixel classification, the learning weights for difficult sample regions such as tumor edges and small lesions are strengthened, which further improves the accuracy and robustness of the segmentation results until the brain tumor medical image segmentation model converges.

Images