A brain tumor image segmentation method and system

Abstract

The present application relates to a kind of brain tumor image segmentation method and system, belong to deep learning technical field, brain tumor image segmentation method is based on the brain tumor segmentation network of construction to brain tumor MRI image is segmented, brain tumor segmentation network includes encoder, decoder;Based on the feature extraction of the obtained brain tumor MRI image of encoder, obtain multiscale feature, wherein, the sub-encoder of encoder includes Conv-VMamba fusion module, Conv-VMamba fusion module is used to extract the local feature and global feature of brain tumor MRI image in parallel, and the local feature and global feature are spliced and channel mix wash, based on the decoding of multiscale feature, generate brain tumor image segmentation result, wherein, the sub-decoder of decoder includes dynamic channel sampling module, Conv-VMamba fusion module, improve the segmentation precision of brain tumor image.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method and system for brain tumor image segmentation. Background Technology

[0002] The accurate diagnosis and treatment planning of brain tumors depend heavily on the accurate segmentation of the lesion area. Magnetic resonance imaging (MRI) is currently the main imaging method for clinical diagnosis of brain tumors. However, due to the high heterogeneity of different brain tumors, such as gliomas, meningiomas, and pituitary tumors, in terms of shape, size, and location, and the fact that tumor boundaries are often indistinct and have low contrast with the surrounding normal brain tissue, automated segmentation faces a huge challenge.

[0003] Existing technologies mainly rely on U-Net and its variants based on convolutional neural networks (CNNs). Although CNNs perform well in extracting local texture features, they are limited by the local receptive field of the convolutional kernel, making it difficult to effectively capture long-distance global contextual information. This leads to insufficient understanding of the overall structure of large tumors or misjudgment of surrounding tissues. The introduced Transformer architecture solves the global modeling problem through a self-attention mechanism, but the computational complexity increases quadratically with image resolution, resulting in a heavy computational burden. In addition, existing U-Net skip connections usually use simple feature concatenation, ignoring the semantic differences between features at different levels, which easily introduces redundant noise and affects the segmentation accuracy of small lesions (such as pituitary tumors). Summary of the Invention

[0004] In view of this, it is necessary to provide a brain tumor image segmentation method and system to solve the technical problem of low segmentation accuracy of brain tumor images.

[0005] To address the aforementioned problems, in a first aspect, the present invention provides a brain tumor image segmentation method for segmenting brain tumor MRI images based on a constructed brain tumor segmentation network, wherein the brain tumor segmentation network includes an encoder and a decoder. The brain tumor image segmentation method includes: Based on the encoder, feature extraction is performed on the acquired brain tumor MRI image to obtain multi-scale features. The sub-encoder of the encoder includes a Conv-VMamba fusion module. The Conv-VMamba fusion module is used to extract local and global features of the brain tumor MRI image in parallel, and to stitch and shuffle the local and global features. The multi-scale features are decoded based on the decoder to generate brain tumor image segmentation results. The sub-decoder of the decoder includes a dynamic channel sampling module and a Conv-VMamba fusion module. The dynamic channel sampling module is used to concatenate the upsampled features and the multi-scale features output by the sub-encoder at the same level, and to filter the concatenated features. The Conv-VMamba fusion module is used to extract the filtered features to obtain brain tumor image segmentation results.

[0006] In one possible implementation, the encoder includes a first sub-encoder, a second sub-encoder, a third sub-encoder, a fourth sub-encoder, and a fifth sub-encoder. The network structures of the first, second, third, and fourth sub-encoders are identical. The first sub-encoder includes a Conv-VMamba fusion module and a downsampling layer. The fifth sub-encoder includes stacked Conv-VMamba fusion modules. The step of extracting features from the acquired brain tumor MRI image based on the encoder to obtain multi-scale features includes: Based on the first sub-encoder, the Conv-VMamba fusion module extracts features from the brain tumor MRI image to obtain first-scale features; The first scale features are downsampled based on the downsampling layer of the first sub-encoder, and the second scale features are obtained by feature extraction based on the Conv-VMamba fusion module of the second sub-encoder. The second-scale features are downsampled based on the downsampling layer of the second sub-encoder, and the third-scale features are extracted based on the Conv-VMamba fusion module of the third sub-encoder. The third-scale features are downsampled based on the downsampling layer of the third sub-encoder, and the fourth-scale features are extracted based on the Conv-VMamba fusion module of the fourth sub-encoder. The fourth-scale features are downsampled based on the downsampling layer of the fourth sub-encoder, and the fourth-scale features are extracted based on the Conv-VMamba fusion module of the fifth sub-encoder to obtain global semantic features.

[0007] In one possible implementation, the network structure of the Conv-VMamba fusion module is a parallel convolutional module and a VMamba module; the Conv-VMamba fusion module based on the first sub-encoder performs feature extraction on the brain tumor MRI image to obtain first-scale features, including: The brain tumor MRI image is divided to obtain a first feature subset and a second feature subset; Based on the convolution module, feature extraction is performed on the first feature subset to obtain local features; Based on the VMamba module, feature extraction is performed on the second feature subset to obtain global features; The local and global features are spliced ​​together, and the spliced ​​features are then subjected to channel shuffling to obtain the first-scale features.

[0008] In one possible implementation, the convolutional module includes a depthwise convolutional layer, a layer normalization layer, a first convolutional layer, a Gaussian error linear unit, and a second convolutional layer; the step of extracting features from the first feature subset based on the convolutional module to obtain local features includes: Based on the deep convolutional layer, spatial filtering is performed on the first feature subset to obtain spatially filtered features; The spatially filtered features are normalized based on the layer normalization. Based on the first convolutional layer, the normalized features are expanded by channel extension to obtain the expanded features; The extended features are subjected to nonlinear processing based on the Gaussian error linear unit; Based on the second convolutional layer, channel compression is performed on the nonlinearly processed features to obtain local features.

[0009] In one possible implementation, the network structure of the VMamba module is a dual residual structure, which includes a first-level residual mapping and a second-level residual mapping. The first-level residual mapping includes layer normalization and a two-dimensional selective scanning mechanism module. The second-level residual mapping includes layer normalization and a feedforward neural network layer. The two-dimensional selective scanning mechanism module includes a first linear layer, a deep convolutional layer, an activation function, layer normalization, and a second linear layer.

[0010] In one possible implementation, the decoder includes a first sub-decoder, a second sub-decoder, a third sub-decoder, and a fourth sub-decoder. The network structures of the first sub-decoder, the second sub-decoder, the third sub-decoder, and the fourth sub-decoder are identical. The first sub-decoder includes an upsampling layer, a dynamic channel sampling module, and a Conv-VMamba fusion module. The step of decoding the multi-scale features based on the decoder to generate brain tumor image segmentation results includes: The global semantic features are upsampled based on the upsampling layer of the first sub-decoder. The upsampled global semantic features and the fourth scale features are concatenated based on the dynamic channel sampling module of the first sub-decoder. The concatenated features are then filtered. The filtered features are extracted based on the Conv-VMamba fusion module of the first sub-decoder to obtain the first decoding features. The first decoding feature is upsampled based on the upsampling layer of the second sub-decoder. The first decoding feature and the third scale feature are concatenated based on the dynamic channel sampling module of the second sub-decoder. The concatenated features are then filtered. The second decoding feature is obtained by feature extraction based on the Conv-VMamba fusion module of the second sub-decoder. The second decoding feature is upsampled based on the upsampling layer of the third sub-decoder. The upsampled second decoding feature and the second scale feature are concatenated based on the dynamic channel sampling module of the third sub-decoder. The concatenated features are then filtered. The filtered features are extracted based on the Conv-VMamba fusion module of the third sub-decoder to obtain the third decoding feature. The third decoding feature is upsampled based on the upsampling layer of the fourth sub-decoder. The upsampled third decoding feature and the first scale feature are concatenated based on the dynamic channel sampling module of the fourth sub-decoder. The concatenated feature is then filtered. The filtered feature is extracted based on the Conv-VMamba fusion module of the fourth sub-decoder to obtain the fourth decoding feature. The fourth decoded feature is classified based on the classifier to obtain a binarized segmentation mask, and the brain tumor image segmentation result is obtained based on the binarized segmentation mask.

[0011] In one possible implementation, the dynamic channel sampling module includes a probability predictor; the dynamic channel sampling module based on the first sub-decoder concatenates the upsampled global semantic features and the fourth-scale features, and filters the concatenated features, including: The upsampled global semantic features and the fourth-scale features are concatenated, and the concatenated features are channel compressed to obtain a global descriptor vector. Based on the probability predictor, a sampling probability vector is generated by probabilistic prediction of the global descriptor vector. The concatenated features are filtered based on the sampling probability vector and the preset sampling probability threshold.

[0012] In one possible implementation, the step of filtering the concatenated features based on the sampling probability vector and a preset sampling probability threshold includes: The concatenated features are divided to obtain multiple feature subsets; Obtain the sampling probability vector corresponding to each feature subset. When the sampling probability vector is less than or equal to the sampling probability threshold, perform a maximum value operation on the feature subset. When the sampling probability vector is greater than the sampling probability threshold, the feature subset is averaged. Aggregate the feature subsets after the maximum value operation and the feature subsets after the average value operation.

[0013] In one possible implementation, the loss function of the brain tumor segmentation network is: , in, The total loss of the brain tumor segmentation network. For background category weights, For tumor category weights, For cross-entropy loss, This is a loss for Dice.

[0014] In a second aspect, the present invention also provides a brain tumor image segmentation system for segmenting brain tumor MRI images based on a constructed brain tumor segmentation network, wherein the brain tumor segmentation network includes an encoder and a decoder. The brain tumor image segmentation system includes: The feature extraction module is used to extract features from the acquired brain tumor MRI image based on the encoder to obtain multi-scale features. The sub-encoder of the encoder includes a Conv-VMamba fusion module. The Conv-VMamba fusion module is used to extract local features and global features of the brain tumor MRI image in parallel, and to stitch and shuffle the local features and global features. The image segmentation module is used to decode the multi-scale features based on the decoder to generate brain tumor image segmentation results. The decoder includes a dynamic channel sampling module and a Conv-VMamba fusion module. The dynamic channel sampling module is used to concatenate the upsampled features and the multi-scale features output by the sub-encoder at the same level, and to filter the concatenated features. The Conv-VMamba fusion module is used to extract the filtered features to obtain brain tumor image segmentation results.

[0015] The beneficial effects of this invention are as follows: Based on the encoder, feature extraction is performed on the acquired brain tumor MRI image to obtain multi-scale features. The encoder's sub-encoder includes a Conv-VMamba fusion module, which is used to extract local and global features of the brain tumor MRI image in parallel, and to stitch and shuffle the local and global features. The Conv-VMamba fusion module adopts a parallel dual-branch design. By extracting local and global features through the Conv-VMamba fusion module, the fine characterization of the tumor edge is ensured. Through the stitching and shuffling of global and local features, efficient fusion of the two feature paths is achieved, significantly improving segmentation accuracy while effectively controlling computational resource consumption. Based on the decoder, the multi-scale features are decoded to generate brain tumor images. The brain tumor image segmentation result includes a sub-decoder comprising a dynamic channel sampling module and a Conv-VMamba fusion module. The dynamic channel sampling module is used to stitch together the upsampled features and multi-scale features output by the sub-encoder at the same level, and to filter the stitched features. The Conv-VMamba fusion module is used to extract the filtered features to obtain the brain tumor image segmentation result. By filtering multi-scale features through the dynamic channel sampling module, the brain tumor segmentation network can dynamically adjust its focus based on the real-time feature distribution of the input slice, thereby significantly improving the ability to identify lesions with blurred boundaries and small tumors (such as pituitary tumors). Through feature extraction by the Conv-VMamba fusion module and feature filtering by the dynamic channel sampling module, the segmentation accuracy of the brain tumor image is improved. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of an embodiment of the brain tumor image segmentation method provided by the present invention; Figure 2 A schematic diagram of the brain tumor segmentation network in the brain tumor image segmentation method provided by the present invention; Figure 3 A schematic diagram of the Conv-VMamba fusion module in the brain tumor image segmentation method provided by the present invention; Figure 4 This is a schematic diagram of the dynamic channel sampling module in the brain tumor image segmentation method provided by the present invention. Figure 5A schematic diagram showing the segmentation results of three tumor subtypes in the brain tumor image segmentation method provided by the present invention; Figure 6 A schematic diagram of the segmentation results of a representative case of brain tumor using the brain tumor image segmentation method provided by the present invention; Figure 7 This is a schematic diagram of an embodiment of the brain tumor image segmentation system provided by the present invention. Detailed Implementation

[0018] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0019] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0020] A specific embodiment of the present invention discloses a brain tumor image segmentation method. This method is used to segment brain tumor MRI images based on a constructed brain tumor segmentation network. The brain tumor segmentation network includes an encoder and a decoder, such as... Figure 1 As shown, the brain tumor image segmentation method includes: S101. Based on the encoder, feature extraction is performed on the acquired brain tumor MRI image to obtain multi-scale features. The encoder's sub-encoder includes a Conv-VMamba fusion module, which is used to extract local and global features of the brain tumor MRI image in parallel, and then stitches and shuffles the local and global features.

[0021] It should be noted that the brain tumor segmentation network is a DCV-UNet network. The DCV-UNet network includes an encoder, a dynamic channel sampling module (DCS), and a decoder. The encoder and decoder include a Conv-VMamba fusion module, which includes parallel convolutional modules and VMamba modules. The convolutional operators of the Conv-VMamba fusion module extract local features, ensuring fine characterization of tumor edges. At the same time, the linear complexity of the visual state space model (VMamba) module is used to capture global features, which significantly improves segmentation accuracy while effectively controlling the consumption of computational resources.

[0022] S102. Based on the decoder, multi-scale features are decoded to generate brain tumor image segmentation results. The decoder's sub-decoder includes a dynamic channel sampling module and a Conv-VMamba fusion module. The dynamic channel sampling module is used to stitch together the upsampled features and the multi-scale features output by the sub-encoder at the same level, and to filter the stitched features. The Conv-VMamba fusion module is used to extract the filtered features to obtain brain tumor image segmentation results.

[0023] It should be noted that by using the dynamic channel sampling module to filter multi-scale features, the brain tumor segmentation network can dynamically adjust its focus based on the real-time feature distribution of the input slices, thereby significantly improving its ability to identify lesions with blurred boundaries and small tumors (such as pituitary tumors).

[0024] In some embodiments, in step S101, feature extraction is performed on the acquired brain tumor MRI image based on the encoder to obtain multi-scale features. The encoder's sub-encoder includes a Conv-VMamba fusion module, which is used to extract local and global features of the brain tumor MRI image in parallel, and to stitch and perform channel mixing on the local and global features. For a schematic diagram of the tumor segmentation network, please refer to [link to schematic diagram]. Figure 2 ,like Figure 2 As shown, the brain tumor segmentation network includes an encoder and a decoder. The encoder includes a first sub-encoder, a second sub-encoder, a third sub-encoder, a fourth sub-encoder, and a fifth sub-encoder. The network structures of the first, second, third, and fourth sub-encoders are identical. The first sub-encoder includes a Conv-VMamba fusion module and a downsampling layer. The network structure of the first sub-encoder consists of stacked Conv-VMamba fusion modules and downsampling layers. The stacked Conv-VMamba fusion module consists of two Conv-VMamba fusion modules, each with identical structure and function. The fifth sub-encoder includes a Conv-VMamba fusion module, specifically a stacked Conv-VMamba fusion module. Its first, second, third, and fourth sub-encoders correspond to the four encoding stages of the encoder, respectively. For a schematic diagram of the Conv-VMamba fusion module structure, please refer to [link to schematic diagram]. Figure 3 ,like Figure 3 As shown, the network structure of the Conv-VMamba fusion module consists of parallel convolutional modules (ConvBlock) and VMamba modules (VMambaBlock). The convolutional modules include deep convolutional layers (DWConv). Layer normalization (LN), first convolutional layer ( conv), Gaussian error linear unit (GULU), and second convolutional layer ( The Conv-VMamba fusion module, consisting of a layer normalization (LN) layer, a two-dimensional selective scanning mechanism (SS2D Block), and a feedforward neural network (FFN) layer, extracts features from the acquired brain tumor MRI images based on the encoder to obtain multi-scale features. Specifically, it acquires at least one brain tumor MRI image, which can be sourced from hospital imaging equipment, medical imaging databases, or public datasets. The image type includes at least T1-weighted enhancement sequences and can be extended to multi-modal MRI images such as T1, T2, T1ce, and FLAIR. Based on the Conv-VMamba fusion module of the first sub-encoder, it extracts features from the brain tumor MRI image to obtain first-scale features. Specifically, it divides the brain tumor MRI image to obtain a first feature subset and a second feature subset. For the input brain tumor MRI image, it performs image processing through the Stem layer to obtain the input feature map. The input feature map is divided into two parts based on the channel dimension, namely the first feature subset. and the second feature subset After obtaining the first and second feature subsets, the convolutional module and VMamba module of the Conv-VMamba fusion module are used to process the first and second feature subsets respectively. The convolutional module extracts features from the first feature subset to obtain local features. The processing steps of the convolutional module are as follows: spatial filtering is performed on the first feature subset based on a depthwise convolutional layer to obtain spatially filtered features; layer normalization is applied to the spatially filtered features; channel expansion is performed on the normalized features based on the first convolutional layer to obtain expanded features; nonlinear processing is applied to the expanded features based on Gaussian error linear units; and channel compression is performed on the nonlinearly processed features based on the second convolutional layer to obtain local features. First, a convolutional kernel with a size of... The depthwise convolution (DWConv) layer, compared to the conventional one... convolution, The convolutional kernel can provide a larger receptive field, effectively capturing the structured information of the tumor region edge, while maintaining a low parameter count by utilizing the characteristics of depthwise convolution. The processing procedure of the depthwise convolutional layer is as follows: , The DWConv operator performs independent spatial filtering on each input channel, thereby reducing coupling interference between different channels while preserving spatial details. After the output of the deep convolutional layer, it abandons the traditional batch normalization (BN) and instead introduces layer normalization (LN). LN is used because brain tumor MRI images typically have high feature resolution, and during training, memory limitations often restrict the use of small batches. LN does not rely on batch statistics and can significantly improve the gradient stability of the model with small training samples. The calculation process of LN is as follows: , in, and These are the mean and variance of the feature map across all neurons in the layer normalization process, respectively. and To enable learnable scaling and translation parameters and further enhance the nonlinear expressive power of features, a variant of the inverse residual structure is introduced. This inverse residual structure comprises a first convolutional layer, a Gaussian error linear unit, and a second convolutional layer. It first passes through a... Pointwise convolution, where the first convolutional layer expands the channel dimension to four times the original dimension, i.e., from... Expand to Within the extended space, the Gaussian Error Linear Unit (GELU) is applied as the activation function for nonlinear processing. The GELU function, by introducing stochastic regularization, improves the smoothness of the model when processing subtle pixel grayscale variations in brain tumor images. Its GELU function is: , Finally, the branch road adopts another one. The convolutional layer, or second convolutional layer, reprojects the expanded feature map back to the original dimensions. This ensures that the output tensor has the exact same geometric dimensions as the output of the VMambaBlock branch, providing standardized feature inputs for subsequent concatenation and channel shuffling. Finally, the ConvBlock branch of the convolutional module outputs local features. Local features are represented as: , By combining operators consisting of large kernel convolution, layer normalization, and high-magnification channel expansion of deep convolutional layers, the convolution module ConvBlock can significantly improve the channel representation capability of feature maps while focusing on the details of lesion edges, laying a local feature foundation for the accurate localization of subsequent multimodal lesions. Simultaneously, feature extraction is performed on the second feature subset based on the VMamba module to obtain global features. The network structure of the VMamba module is a dual residual structure, such as... Figure 3 As shown, the dual residual structure includes a first-level residual mapping and a second-level residual mapping. The first-level residual mapping includes layer normalization (LN) and a two-dimensional selective scanning mechanism module (SS2D Block). The two-dimensional selective scanning mechanism module includes a first linear projection layer (linera), a deep convolutional layer, an activation function (SiLU), a two-dimensional selective scanning mechanism (SS2D), layer normalization (LN), and a second linear projection layer (linear). The second-level residual mapping includes layer normalization (LN) and a feedforward neural network layer (FFN). To ensure the stability of deep feature flow and prevent gradient vanishing, the VMamba module adopts a precise dual residual topology. In the first-level residual mapping, the second feature subset is normalized by layer normalization (LN), and then processed by the two-dimensional selective scanning mechanism module. The resulting features are added and merged with the second feature subset to obtain the intermediate feature map Z, which is as follows: , In the second-level residual mapping, the intermediate feature map Z undergoes layer normalization and feedforward neural network layers for global feature enhancement and dimensionality fine-tuning. Finally, the second-level residual connections generate the final output matrix, i.e., the global feature map. Its global features are: , To address the problem that convolution operations cannot effectively model long-range dependencies and to overcome the inherent limitations of convolution operators in long-range dependency modeling, the Visual State Space (VMamba) module introduces a two-dimensional selective scanning mechanism, SS2D. To process two-dimensional images, SS2D unfolds the input feature map into a one-dimensional sequence along four specific directions (top left to bottom right, bottom right to top left, top right to bottom left, and bottom left to top right). For each sequence, the discrete state space model is run independently, and then the output sequences from the four directions are merged to restore the two-dimensional feature map. This SS2D mechanism enables the model to obtain the contextual information of the entire image with linear computational complexity, effectively distinguishing tumors from normal brain tissue. Discretization of its state-space model: Introducing time-scale parameters using the zero-order preserve method (ZOH). This transforms the continuous state-space model (SSH) into a discrete form to adapt it to digital image processing. The continuous state-space model, when processing input visual sequences, utilizes internal latent state variables... Describe the input sequence To output sequence The evolutionary mapping, its continuous state-space model is: , , in, The transition matrix characterizes the evolution of the system state. and To enable the continuous state-space model to handle discrete pixels, a discretization transformation is performed using the Zero-Order Hold (ZOH) rule, introducing a time-scale parameter, which serves as the projection parameter. , will continuous parameters , Convert to discrete parameters , : , , After processing by the VMamba module, the output is a feature matrix with the same dimensions as the convolutional branch. After obtaining local and global features, the local and global features are concatenated, and the concatenated features are then subjected to channel shuffling to obtain the first-scale feature, the size of which is [size missing]. The local features output by the ConvBlock branch Global characteristics output with VMambaBlock branch The data is stitched together along the channel dimension to restore it. Channel, then, execution group size The channel shuffle operation disrupts the channel order, forcing local feature channels to interact with global feature channels in subsequent layers, thereby generating more expressive fused features. After obtaining the first-scale features, the downsampling layer of the first sub-encoder downsamples the first-scale features, and the Conv-VMamba fusion module of the second sub-encoder extracts features from the downsampled first-scale features to obtain the second-scale features, the size of which is... The second-scale features are downsampled using the downsampling layer of the second sub-encoder, and then feature extraction is performed on the downsampled second-scale features using the Conv-VMamba fusion module of the third sub-encoder to obtain the third-scale features. The size of the third-scale features is... The downsampling layer based on the third sub-encoder downsamples the third-scale features, and the Conv-VMamba fusion module based on the fourth sub-encoder extracts features from the downsampled third-scale features to obtain fourth-scale features. The size of the fourth-scale features is... The fourth-scale features are downsampled using the downsampling layer of the fourth sub-encoder, and then the fourth-scale features are extracted using the Conv-VMamba fusion module of the fifth sub-encoder to obtain global semantic features. The size of the global semantic features is... ; Brain tumor images are processed sequentially using the first, second, third, fourth, and fifth sub-encoders. As the network depth increases, the spatial size of the feature maps is halved sequentially (from...). Down to ), while the number of channels doubles sequentially (from Increase to At the deepest level of the encoder, in the fifth sub-encoder, two Conv-VMamba fusion modules are deployed to capture the highest-level global semantic features.

[0025] In some embodiments, in step S102, multi-scale features are decoded based on the decoder to generate brain tumor image segmentation results. The decoder's sub-decoders include a dynamic channel sampling module and a Conv-VMamba fusion module. The dynamic channel sampling module is used to concatenate the upsampled features and the multi-scale features output by the sub-encoder at the same level, and to filter the concatenated features. The Conv-VMamba fusion module is used to extract the filtered features to obtain brain tumor image segmentation results, such as... Figure 2 As shown, the decoder includes a first sub-decoder, a second sub-decoder, a third sub-decoder, and a fourth sub-decoder. The network structures of the first sub-decoder, the second sub-decoder, the third sub-decoder, and the fourth sub-decoder are consistent. The first sub-decoder includes an upsampling layer, a dynamic channel sampling module (DCS), and a Conv-VMamba fusion module. The network structure of the first sub-decoder consists of an upsampling layer, a dynamic channel sampling module, and stacked Conv-VMamba fusion modules. The dynamic channel sampling module (DCS) is embedded in the skip connection and is located between the corresponding layers of the encoder and the decoder. After obtaining the first-scale features, second-scale features, third-scale features, fourth-scale features, and global semantic features, the multi-scale features output by the encoder are obtained. The global semantic features are upsampled using the upsampling layer of the first sub-decoder. The upsampled global semantic features and the fourth-scale features are then concatenated using the dynamic channel sampling module of the first sub-decoder, and the concatenated features are filtered. The Conv-VMamba fusion module of the first sub-decoder extracts features from the filtered features to obtain the first decoded features. Similarly, the first decoded features are upsampled using the upsampling layer of the second sub-decoder. The upsampled first decoded features and the third-scale features are then concatenated using the dynamic channel sampling module of the second sub-decoder, and the concatenated features are filtered. The Conv-VMamba fusion module of the second sub-decoder extracts features from the filtered features to obtain the second decoded features. The process involves: 1) Upsampling the second decoded feature using the upsampling layer of the third sub-decoder; 2) Concatenating the upsampled second decoded feature with the second-scale feature using the dynamic channel sampling module of the third sub-decoder; 3) Filtering the concatenated feature; 4) Extracting the selected feature using the Conv-VMamba fusion module of the fourth sub-decoder; 5) Classifying the fourth decoded feature using a classifier to obtain a binarized segmentation mask; and 6) Obtaining the brain tumor image segmentation result based on the binarized segmentation mask. Please refer to the structural diagram of its dynamic channel sampling module. Figure 4 ,like Figure 4 As shown, the dynamic channel sampling module includes a probability predictor. The probability predictor includes a linear transformation and a sigmoid activation function. Taking the dynamic channel sampling module in the first sub-decoder as an example, the dynamic channel sampling module concatenates the global semantic features and the fourth-scale features, and then filters the concatenated features. The filtering process is as follows: channel compression is performed on the concatenated features to obtain a global descriptor vector; probability prediction is performed on the global descriptor vector based on the probability predictor to generate a sampling probability vector; the concatenated features are filtered based on the sampling probability vector and a preset sampling probability threshold, and channel compression is performed on the multi-scale features to obtain a global descriptor vector; that is, the L1 norm is used to compress each channel of the input feature map into a scalar to form a global descriptor vector z. The input features can be the features obtained by concatenating the global semantic features and the fourth-scale features, and its global descriptor vector is: , Through a learnable probability predictor Generates a sampling probability vector p; the probability predictor includes a linear transformation and a sigmoid activation function. The sampling probability vector is: , , The concatenated features are divided into multiple feature subsets. A sampling probability vector is obtained for each feature subset. When the sampling probability vector is less than or equal to a sampling probability threshold, a maximum value operation is performed on the feature subset; when the sampling probability vector is greater than the sampling probability threshold, an average value operation is performed on the feature subset. The feature subsets after the maximum value operation and the feature subsets after the average value operation are aggregated, and the concatenated features, i.e., the input features, are divided into multiple subsets. Based on the comparison between the predicted sampling probability and the preset sampling probability threshold, an adaptive sampling strategy is selected. The sampling strategy is as follows: , in, For sampling and measurement, For the first A subset The sampling probability threshold is used. When the probability is low, the maximum value operation (Max) is used to extract significant features; when the probability is high, the average value operation (Avg) is used to preserve the overall information. Finally, all processed subsets are aggregated to obtain the output feature map.

[0026] To address the severe imbalance between background and tumor pixel ratios in brain tumor segmentation, and to simultaneously optimize pixel-level classification accuracy and region overlap, a weighted loss function is adopted. This weighted loss function consists of cross-entropy loss and Dice loss. The loss function of the brain tumor segmentation network is as follows: , in, The total loss of the brain tumor segmentation network. For background category weights, For tumor category weights, For cross-entropy loss, To utilize Dice loss and emphasize the importance of the tumor region, the weights of the background category are adjusted. Set the weight to 0.2 to determine the weight of the tumor category. Set it to 0.8.

[0027] The brain tumor MRI image is input into the model. The brain tumor segmentation network extracts features through the encoder, filters features through the DCS module, and restores resolution through the decoder, outputting a binarized segmentation mask. This enables automatic segmentation of glioma, meningioma, or pituitary adenoma regions. See the schematic diagrams of the segmentation results for the three tumor subtypes on the Figshare dataset. Figure 5 ,like Figure 5 As shown, the rows from top to bottom represent gliomas, meningiomas, and pituitary tumors, respectively; the columns from left to right correspond to the original image, the ground truth, and the prediction results of DCV-UNet. In the glioma sample, the DCV-UNet model can accurately identify the entire tumor region and completely preserve its irregular boundary structure. In the meningioma sample, the DCV-UNet model can clearly distinguish the tumor region from the surrounding normal brain tissue, reducing background missegmentation. In the pituitary tumor sample, it can still achieve stable identification for small and low-contrast tumor regions. For a schematic diagram of the segmentation results of representative brain tumor cases on the BraTS2020 dataset, please refer to [link to relevant documentation]. Figure 6 ,like Figure 6 As shown, taking three representative brain tumor cases as examples, the columns from left to right are: original image, ground truth, and DCV-UNet prediction results. The prediction results include the whole tumor, enhanced tumor, and tumor core. In the ground truth and predicted images of the samples, red represents necrotic or non-enhanced tumor core (NCR), green represents peritumoral edema (ED), and purple represents enhanced tumor (ET). By comparing with the ground truth annotation results, it can be found that the DCV-UNet model can obtain segmentation results that are highly consistent with the ground truth annotation in all three tumor regions: for the whole tumor region (WT), the model can cover the entire tumor area relatively completely; for the tumor core region (TC), the model can accurately capture the main internal structures of the tumor; for the enhanced tumor region (ET), even when the area is small and the boundary is complex, the model can still achieve relatively accurate localization.

[0028] Before segmenting brain tumor MRI images using a brain tumor segmentation network, the network is trained. During training, the AdamW optimizer is used to update the parameters of the brain tumor segmentation network, with an initial learning rate of [value missing]. Weight decay is set to After training for 100 epochs with a batch size of 16, the model with the best results is tested using a test set. Once the training and testing results of the brain tumor segmentation network meet expectations, the trained brain tumor segmentation network is saved. When applying the network, the saved brain tumor segmentation network is directly loaded, and the MRI image to be segmented is input into the trained DCV-UNet network. The network outputs a segmentation mask with the same spatial size as the input image, thereby achieving automatic segmentation of the brain tumor region.

[0029] In summary, the brain tumor image segmentation method provided by this invention segments brain tumor MRI images based on a constructed brain tumor segmentation network, which includes an encoder and a decoder. The encoder extracts features from the acquired brain tumor MRI images to obtain multi-scale features. The encoder's sub-encoder includes a Conv-VMamba fusion module, which extracts local and global features of the brain tumor MRI image in parallel, and concatenates and performs channel shuffling on the local and global features. The decoder decodes the multi-scale features to generate brain tumor image segmentation results. The decoder's sub-decoder includes a dynamic channel sampling module and a Conv-VMamba fusion module. The dynamic channel sampling module concatenates the upsampled features with the multi-scale features output by the sub-encoder at the same level, and filters the concatenated features. The Conv-VMamba fusion module extracts the filtered features to obtain brain tumor image segmentation results, thus improving the segmentation accuracy of brain tumor images.

[0030] To better implement the brain tumor image segmentation method in this embodiment of the invention, based on the brain tumor image segmentation method, correspondingly, as follows: Figure 7 As shown, this embodiment of the invention also provides a brain tumor image segmentation system for segmenting brain tumor MRI images based on a constructed brain tumor segmentation network, the brain tumor segmentation network including an encoder and a decoder; Brain tumor image segmentation system 700, including: The feature extraction module 701 is used to extract features from the acquired brain tumor MRI image based on the encoder to obtain multi-scale features. The sub-encoder of the encoder includes a Conv-VMamba fusion module. The Conv-VMamba fusion module is used to extract local and global features of the brain tumor MRI image in parallel, and to stitch and shuffle the local and global features. Image segmentation module 702 is used to decode multi-scale features based on the decoder to generate brain tumor image segmentation results. The decoder includes a dynamic channel sampling module and a Conv-VMamba fusion module. The dynamic channel sampling module is used to concatenate the upsampled features and the multi-scale features output by the sub-encoder at the same level, and to filter the concatenated features. The Conv-VMamba fusion module is used to extract the filtered features to obtain brain tumor image segmentation results.

[0031] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A brain tumor image segmentation method, characterized in that, This is used to segment brain tumor MRI images based on a constructed brain tumor segmentation network, which includes an encoder and a decoder; The brain tumor image segmentation method includes: Based on the encoder, feature extraction is performed on the acquired brain tumor MRI image to obtain multi-scale features. The sub-encoder of the encoder includes a Conv-VMamba fusion module. The Conv-VMamba fusion module is used to extract local and global features of the brain tumor MRI image in parallel, and to stitch and shuffle the local and global features. The multi-scale features are decoded based on the decoder to generate brain tumor image segmentation results. The sub-decoder of the decoder includes a dynamic channel sampling module and a Conv-VMamba fusion module. The dynamic channel sampling module is used to concatenate the upsampled features and the multi-scale features output by the sub-encoder at the same level, and to filter the concatenated features. The Conv-VMamba fusion module is used to extract the filtered features to obtain brain tumor image segmentation results.

2. The brain tumor image segmentation method according to claim 1, characterized in that, The encoder includes a first sub-encoder, a second sub-encoder, a third sub-encoder, a fourth sub-encoder, and a fifth sub-encoder. The network structures of the first, second, third, and fourth sub-encoders are identical. The first sub-encoder includes a Conv-VMamba fusion module and a downsampling layer. The fifth sub-encoder includes stacked Conv-VMamba fusion modules. The feature extraction of the acquired brain tumor MRI image based on the encoder to obtain multi-scale features includes: Based on the Conv-VMamba fusion module of the first sub-encoder, feature extraction is performed on the brain tumor MRI image to obtain first-scale features; The first scale features are downsampled based on the downsampling layer of the first sub-encoder, and the second scale features are obtained by extracting features from the downsampled first scale features based on the Conv-VMamba fusion module of the second sub-encoder. The second-scale features are downsampled based on the downsampling layer of the second sub-encoder, and the third-scale features are extracted based on the Conv-VMamba fusion module of the third sub-encoder. The third-scale features are downsampled based on the downsampling layer of the third sub-encoder, and the fourth-scale features are extracted based on the Conv-VMamba fusion module of the fourth sub-encoder. The fourth-scale features are downsampled based on the downsampling layer of the fourth sub-encoder, and the fourth-scale features are extracted based on the Conv-VMamba fusion module of the fifth sub-encoder to obtain global semantic features.

3. The brain tumor image segmentation method according to claim 2, characterized in that, The network structure of the Conv-VMamba fusion module consists of parallel convolutional modules and VMamba modules; based on the first sub-encoder, the Conv-VMamba fusion module extracts features from the brain tumor MRI image to obtain first-scale features, including: The brain tumor MRI image is divided to obtain a first feature subset and a second feature subset; Based on the convolution module, feature extraction is performed on the first feature subset to obtain local features; Based on the VMamba module, feature extraction is performed on the second feature subset to obtain global features; The local and global features are spliced ​​together, and the spliced ​​features are then subjected to channel shuffling to obtain the first-scale features.

4. The brain tumor image segmentation method according to claim 3, characterized in that, The convolutional module includes a depthwise convolutional layer, a layer normalization layer, a first convolutional layer, a Gaussian error linear unit, and a second convolutional layer; the feature extraction based on the convolutional module to obtain local features includes: Based on the deep convolutional layer, spatial filtering is performed on the first feature subset to obtain spatially filtered features; The spatially filtered features are normalized based on the layer normalization. Based on the first convolutional layer, the normalized features are expanded by channel extension to obtain the expanded features; The extended features are subjected to nonlinear processing based on the Gaussian error linear unit; Based on the second convolutional layer, channel compression is performed on the nonlinearly processed features to obtain local features.

5. The brain tumor image segmentation method according to claim 3, characterized in that, The network structure of the VMamba module is a dual residual structure, which includes a first-level residual mapping and a second-level residual mapping. The first-level residual mapping includes a layer normalization and a two-dimensional selective scanning mechanism module. The second-level residual mapping includes a layer normalization and a feedforward neural network layer. The two-dimensional selective scanning mechanism module includes a first linear projection layer, a depthwise convolutional layer, an activation function, a two-dimensional selective scanning mechanism, a layer normalization, and a second linear projection layer.

6. The brain tumor image segmentation method according to claim 2, characterized in that, The decoder includes a first sub-decoder, a second sub-decoder, a third sub-decoder, and a fourth sub-decoder. The network structures of the first sub-decoder, the second sub-decoder, the third sub-decoder, and the fourth sub-decoder are consistent. The first sub-decoder includes an upsampling layer, a dynamic channel sampling module, and a Conv-VMamba fusion module. The process of decoding the multi-scale features based on the decoder to generate brain tumor image segmentation results includes: The global semantic features are upsampled based on the upsampling layer of the first sub-decoder. The upsampled global semantic features and the fourth scale features are concatenated based on the dynamic channel sampling module of the first sub-decoder. The concatenated features are then filtered. The filtered features are extracted based on the Conv-VMamba fusion module of the first sub-decoder to obtain the first decoding features. The first decoding feature is upsampled based on the upsampling layer of the second sub-decoder. The first decoding feature and the third scale feature are concatenated based on the dynamic channel sampling module of the second sub-decoder. The concatenated features are then filtered. The second decoding feature is obtained by feature extraction based on the Conv-VMamba fusion module of the second sub-decoder. The second decoding feature is upsampled based on the upsampling layer of the third sub-decoder. The upsampled second decoding feature and the second scale feature are concatenated based on the dynamic channel sampling module of the third sub-decoder. The concatenated features are then filtered. The filtered features are extracted based on the Conv-VMamba fusion module of the third sub-decoder to obtain the third decoding feature. The third decoding feature is upsampled based on the upsampling layer of the fourth sub-decoder. The upsampled third decoding feature and the first scale feature are concatenated based on the dynamic channel sampling module of the fourth sub-decoder. The concatenated feature is then filtered. The filtered feature is extracted based on the Conv-VMamba fusion module of the fourth sub-decoder to obtain the fourth decoding feature. The fourth decoded feature is classified based on the classifier to obtain a binarized segmentation mask, and the brain tumor image segmentation result is obtained based on the binarized segmentation mask.

7. The brain tumor image segmentation method according to claim 6, characterized in that, The dynamic channel sampling module includes a probability predictor; the dynamic channel sampling module based on the first sub-decoder concatenates the upsampled global semantic features and the fourth-scale features, and filters the concatenated features, including: The upsampled global semantic features and the fourth-scale features are concatenated, and the concatenated features are channel compressed to obtain a global descriptor vector. Based on the probability predictor, a sampling probability vector is generated by probabilistic prediction of the global descriptor vector. The concatenated features are filtered based on the sampling probability vector and the preset sampling probability threshold.

8. The brain tumor image segmentation method according to claim 7, characterized in that, The step of filtering the concatenated features based on the sampling probability vector and a preset sampling probability threshold includes: The concatenated features are divided to obtain multiple feature subsets; Obtain the sampling probability vector corresponding to each feature subset. When the sampling probability vector is less than or equal to the sampling probability threshold, perform a maximum value operation on the feature subset. When the sampling probability vector is greater than the sampling probability threshold, the feature subset is averaged. Aggregate the feature subsets after the maximum value operation and the feature subsets after the average value operation.

9. The brain tumor image segmentation method according to claim 1, characterized in that, The loss function of the brain tumor segmentation network is: ， in, The total loss of the brain tumor segmentation network. For background category weights, For tumor category weights, For cross-entropy loss, This is a loss for Dice.

10. A brain tumor image segmentation system, characterized in that, This is used to segment brain tumor MRI images based on a constructed brain tumor segmentation network, which includes an encoder, a dynamic channel sampling module, and a decoder; The brain tumor image segmentation system includes: The feature extraction module is used to extract features from the acquired brain tumor MRI image based on the encoder to obtain multi-scale features. The sub-encoder of the encoder includes a Conv-VMamba fusion module. The Conv-VMamba fusion module is used to extract local features and global features of the brain tumor MRI image in parallel, and to stitch and shuffle the local features and global features. The image segmentation module is used to decode the multi-scale features based on the decoder to generate brain tumor image segmentation results. The decoder includes a dynamic channel sampling module and a Conv-VMamba fusion module. The dynamic channel sampling module is used to concatenate the upsampled features and the multi-scale features output by the sub-encoder at the same level, and to filter the concatenated features. The Conv-VMamba fusion module is used to extract the filtered features to obtain brain tumor image segmentation results.

Images