A multi-modal magnetic resonance image segmentation method based on MSBA-Net

Abstract

The present application relates to the technical field of medical image processing and computer-aided diagnosis, and particularly relates to a multi-modal magnetic resonance image segmentation method based on MSBA-Net, aiming at significantly reducing the model parameter quantity and improving the training and inference efficiency under the premise of ensuring the segmentation accuracy. The method comprises the following steps: acquiring multi-modal brain tumor magnetic resonance images and preprocessing to obtain preprocessed images; inputting the preprocessed images into a compact 3D segmentation network MSBA-Net to extract multi-scale deep semantic features; based on the multi-scale deep semantic features, outputting a preliminary segmentation prediction map through a decoder, and outputting an auxiliary segmentation prediction map at four resolution levels by using a deep supervision mechanism; constructing a boundary-guided hybrid loss function, training the compact 3D segmentation network MSBA-Net by using the boundary-guided hybrid loss function, and segmenting the test images in the test stage to output the segmentation masks of the whole tumor region, the tumor core region and the enhanced tumor region of the brain tumor.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and computer-aided diagnosis, and in particular to a multimodal magnetic resonance image segmentation method based on MSBA-Net. Background Technology

[0002] Gliomas are the most common primary malignant brain tumors, and magnetic resonance imaging (MRI) is the core imaging tool for brain tumor diagnosis, surgical planning, and radiotherapy evaluation. Multimodal MRI images (including T1, T1ce, T2, and FLAIR modalities) can provide anatomical and physiological information about the tumor. Accurate segmentation of the entire tumor region, the tumor core region, and the enhanced tumor region is crucial for assessing tumor burden and treatment response. However, manual segmentation is time-consuming and suffers from poor inter-observer consistency, thus automated segmentation methods have urgent clinical value. In recent years, deep learning models, especially 3D UNet and its variants, have been widely used in brain tumor segmentation tasks. For example, Chinese patent CN115115838B discloses a medical image segmentation method based on feature reuse and attention gating. This method introduces an attention gating between the encoder and decoder and sets up an ASPP module in the network to expand the receptive field, achieving good segmentation results.

[0003] While existing technologies have improved segmentation accuracy to some extent, they still have the following shortcomings: First, most high-performance models (such as nnFormer, SwinUNETR V2, and other Transformer architectures) have a huge number of parameters (usually exceeding 60 million), resulting in high GPU memory consumption and significant inference latency, making them difficult to deploy in resource-constrained clinical environments. Second, existing models are insufficient in handling the extreme imbalance of tumor regions and the problem of blurred boundaries, especially in the tumor core region and the boundary segmentation accuracy of enhanced tumor regions. Third, the lack of joint optimization of deep supervision and boundary guidance for multi-resolution features during training leads to slow model convergence and discontinuous segmentation edges. In addition, although patents such as CN115115838B use attention gates and ASPP, they do not integrate channel attention mechanisms with deep supervision, nor do they design dedicated loss functions for tumor sub-region boundaries, making it difficult to simultaneously achieve high accuracy, lightweight design, and efficient training. Summary of the Invention

[0004] The purpose of this invention is to provide a multimodal magnetic resonance image segmentation method based on MSBA-Net, which aims to significantly reduce the number of model parameters, improve training and inference efficiency, and enhance tumor boundary segmentation effect while ensuring segmentation accuracy.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: The present invention provides a multimodal magnetic resonance image segmentation method based on MSBA-Net, comprising: S1: acquiring multimodal brain tumor magnetic resonance images and performing preprocessing to obtain preprocessed images, wherein the multimodal brain tumor magnetic resonance images include T1 modality images, T1ce modality images, T2 modality images and FLAIR modality images; S2: constructing a compact 3D segmentation network MSBA-Net, inputting the preprocessed images into the compact 3D segmentation network MSBA-Net, and extracting multi-scale deep semantic features; the compact 3D segmentation network MSBA-Net includes an encoder, a bottleneck layer and a decoder, wherein the encoder consists of four residual squeezing-excitation modules, each residual squeezing-excitation module is integrated after residual connection, and the bottleneck layer is... The neck layer employs a 3D voided spatial pyramid pooling module, using multiple voided convolutional branches with different voiding rates and a global average pooling branch in parallel. Attention-gated skip connections are introduced between the decoder and encoder. S3: Based on the multi-scale deep semantic features extracted in step S2, the decoder outputs a preliminary segmentation prediction map, and a deep supervision mechanism is used to output auxiliary segmentation prediction maps at four resolution levels. S4: A boundary-guided hybrid loss function is constructed, which combines Dice loss, weighted cross-entropy loss, and boundary-aware loss. The compact 3D segmentation network MSBA-Net is trained using the boundary-guided hybrid loss function, and the test image is segmented during the testing phase, outputting a segmentation mask for the entire tumor region, the tumor core region, and the enhanced tumor region of the brain tumor.

[0006] In step S2, each residual squeezing-excitation module further includes two consecutive 3×3×3 convolutional layers, group normalization operation, and ReLU activation function; the squeezing-excitation module obtains channel descriptors through global average pooling, and then generates channel weights through two fully connected layers and the Sigmoid function to adaptively calibrate the feature channels.

[0007] In step S2, the 3D void space pyramid pooling module of the bottleneck layer uses four 3×3×3 void convolution branches with void ratios of 1, 2, 4 and 6 in parallel and one global average pooling branch. The outputs of all branches are concatenated along the channel dimension and then fused through a 1×1×1 convolutional layer to capture multi-scale contextual information.

[0008] In step S2, attention gating uses the gating signal of the decoder to filter the skip connection features of the encoder and passes the feature map after background noise suppression to the decoder. Attention gating calculates attention coefficients through an additive attention mechanism, specifically including: using the upsampled features of the decoder as a gating signal after passing through a 1×1×1 convolutional layer and a ReLU activation function, adding it to the skip connection features of the corresponding level of the encoder, and then generating attention coefficients by sequentially passing through a ReLU activation function, a 1×1×1 convolutional layer, and a Sigmoid function. The attention coefficients are then multiplied element-wise with the skip connection features to obtain a weighted feature map.

[0009] In step S3, the deep supervision mechanism adds auxiliary segmentation heads to four resolution levels. The four resolution levels include one-eighth of the original resolution level, one-quarter of the original resolution level, one-half of the original resolution level, and the original resolution level. Each auxiliary segmentation head consists of a 1×1×1 convolutional layer and an upsampling layer. The feature map of the corresponding resolution is extracted from the decoder, processed by the auxiliary segmentation head, and then output as an auxiliary segmentation prediction map.

[0010] In step S4, the boundary-aware loss function extracts boundary voxel masks for the entire tumor region, the tumor core region, and the enhanced tumor region of the brain tumor. The extraction method is as follows: morphological erosion and dilation operations are sequentially performed on the ground truth mask of each sub-region. The result of the dilation operation is subtracted from the result of the erosion operation to obtain the boundary voxel mask of that sub-region. Then, the union of the boundary voxel masks of the three sub-regions is taken as the final boundary voxel mask, denoted as . Only at the boundary voxel mask The complement of the Dice coefficients is calculated above, and the formula for calculating the boundary-aware loss is as follows: in, To predict probabilities, This is a real label.

[0011] In step S4, the formula for calculating the total loss function of the boundary-guided hybrid loss function is: in, The primary output loss is the loss from the initial segmentation prediction map. The four deep-supervised auxiliary losses are the losses for the auxiliary segmentation prediction maps. To assist in loss weighting; Main output loss The calculation formula is: in, For Dice's loss, For weighted cross-entropy loss, For boundary-aware loss, , , These are the corresponding weighting coefficients.

[0012] In step S4, before training the compact 3D segmentation network MSBA-Net, a mixed training batch is constructed using the MixUp data augmentation strategy. Specifically, this includes: for two random samples within a batch... and Generate mixed samples : The mixing coefficient λ follows a Beta distribution with parameter α=1.0, i.e., λ∼Beta(1.0, 1.0), such that λ is approximately uniformly distributed on [0,1].

[0013] In step S4, when training the compact 3D segmentation network MSBA-Net, PyTorch's automatic mixed precision training is used, FP16 is used for forward propagation and gradient calculation, and gradient scaling is used to prevent numerical underflow; multi-process data prefetching is set in DataLoader, with num_workers set to 4 and prefetch_factor set to 2; and all preprocessed training data is cached in memory.

[0014] In step S4, during the testing phase, when segmenting the test image, K preset geometric transformations are applied to each test sample. Geometric transformations include identity transformations, single-axis flips, and dual-axis flips; the transformed samples are then input into the compact 3D segmentation network MSBA-Net to obtain the predicted probability map. Apply inverse transform to the prediction probability map Transform it back to the original space; average and fuse the probability maps after K inverse transformations, using the following formula: Through the The final segmentation result is obtained by taking argmax, and the segmentation mask of the entire tumor region, the segmentation mask of the tumor core region, and the segmentation mask of the enhanced tumor region are output.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The compact 3D segmentation network MSBANet provided in this invention achieves high-precision segmentation with only about 45 million parameters through lightweight design of the residual squeezing excitation module in the encoder, multi-scale feature extraction of the 3D void space pyramid pooling module in the bottleneck layer, and attention-gated skip connections in the decoder. This significantly reduces the number of parameters and memory usage. Furthermore, through engineering optimizations such as mixed-precision training, multi-process data prefetching, and full-data memory caching, the training cycle is greatly shortened. In addition, this invention designs a boundary-guided hybrid loss function. The boundary-aware loss accurately extracts the boundary voxel masks of the entire tumor region, the tumor core region, and the enhanced tumor region through morphological erosion and dilation operations, and calculates the complement of the Dice coefficients only on the boundary voxels. This allows the model to focus on the fuzzy boundaries between tumor sub-regions and the boundary between tumor and normal tissue during training. Combining the Dice loss and weighted cross-entropy loss, voxel-level classification accuracy and boundary continuity are synergistically optimized.

[0016] 2. This invention sets auxiliary segmentation heads at four resolution levels (1 / 8, 1 / 4, 1 / 2, and original resolution) of the decoder, applying supervisory signals to the feature maps at each level. This deep supervision mechanism not only alleviates the gradient vanishing problem and accelerates network convergence, but also enables the model to learn more discriminative feature representations at different scales. During the training phase, a MixUp data augmentation strategy is introduced, sampling mixing coefficients through a Beta distribution (α=1.0) to perform linear interpolation on the image and labels, effectively alleviating the extreme imbalance of tumor region categories and suppressing overfitting. This invention can output segmentation masks for the entire tumor region (edema, necrosis, non-enhanced core), the tumor core region (necrosis, non-enhanced core, enhanced core), and three key sub-regions of the enhanced tumor region in a single output, providing comprehensive and detailed quantitative data for clinical surgical planning, radiotherapy target delineation, and efficacy evaluation. Attached Figure Description

[0017] Figure 1 This is a flowchart of a multimodal magnetic resonance image segmentation method based on MSBA-Net provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating a performance experiment comparison provided in an embodiment of this application; Figure 3 This is a schematic diagram showing the overlay effect of the original view of each modal image and the corresponding segmentation mask generated by this method, provided in an embodiment of this application. Figure 4 This is a schematic diagram of the experimental results of training and inference efficiency provided in an embodiment of this application; Figure 5 This is a schematic diagram of the results of an ablation experiment provided in an embodiment of this application; Figure 6This is a schematic diagram of a t-test statistical analysis provided in an embodiment of this application; Figure 7 This is a schematic diagram of a typical case segmentation result provided in an embodiment of this application; Figure 8 This is a 3D stereoscopic rendering provided for an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] This application provides a multimodal magnetic resonance image segmentation method based on MSBA-Net. By constructing a compact 3D segmentation network MSBA-Net, it integrates channel attention, multi-scale hollow spatial pyramid pooling, and attention gating mechanisms. Combined with a boundary-guided hybrid loss function and engineered training and inference optimization, it achieves accurate segmentation of the whole tumor region (WT), tumor core region (TC), and enhancing tumor region (ET) of brain tumors. For example,... Figure 1 As shown. The method includes: S1: Acquire multimodal brain tumor magnetic resonance images and perform preprocessing to obtain preprocessed images.

[0020] Multimodal MRI images of brain tumors include T1 modal images, T1ce modal images (T1 contrast enhancement modal), T2 modal images, and FLAIR modal images (fluid attenuation inversion recovery modal).

[0021] As one possible implementation, preprocessing operations include modality-specific Z-score normalization, random pruning, online data augmentation, and building training batches.

[0022] In some embodiments, the Medical Segmentation Decathlon (MSD) Task01_BrainTumour dataset is used, which contains 484 multimodal 3D MRI images of the brain. Each image contains four modalities: T1 modality, T1ce modality, T2 modality, and FLAIR modality, with a uniform image size of 240×240×155 voxels.

[0023] First, modality-specific Z-score normalization is performed, with normalization applied separately for each mode. The calculation formula is as follows: in, Original voxel values, This represents the mean value of all voxels in this mode. This represents the standard deviation of all voxels in this mode. After normalization, the data for each mode follow a standard normal distribution with a mean of 0 and a standard deviation of 1, in order to eliminate dimensional differences between different modes.

[0024] Due to the large size of the original images (240×240×155), to reduce GPU memory usage and increase sample diversity, sub-cubes of 128×128×128 voxels were randomly cropped from each modality image. If the tumor region was close to the edge, a translation strategy was used during cropping to ensure that the cropped block contained the tumor region. During each training iteration, the cropped images were dynamically augmented with random operations, including: random rotation (angle range ±10°), random scaling (scale range 0.9~1.1), random elastic deformation (deformation field standard deviation 4 voxels, grid spacing 20 voxels), and random Gaussian noise (mean 0, standard deviation 0.01). All augmentation operations applied the same spatial transformation to the image and its corresponding label mask within the same training batch to maintain correspondence. The preprocessed images were organized into batches, with each batch containing 4 samples. To address class imbalance, an oversampling strategy was employed to ensure that samples containing the tumor core region and augmented tumor regions were selected into the batch with a higher probability.

[0025] S2: Construct a compact 3D segmentation network MSBA-Net, input the preprocessed image into the compact 3D segmentation network MSBA-Net, and extract multi-scale deep semantic features.

[0026] As one possible implementation, the compact 3D segmentation network MSBA-Net includes an encoder, a bottleneck layer, and a decoder. The encoder consists of four residual squeeze-and-excitation modules (SE modules). Each SE module is integrated after a residual connection. The bottleneck layer uses 3D atrous spatial pyramid pooling (3D-ASPP) and uses multiple dilated convolution branches with different dilation rates and a global average pooling branch in parallel. Attention-gated skip connections are introduced between the decoder and the encoder.

[0027] As one possible implementation, in step S2, each residual squeezing-excitation module further includes two consecutive 3×3×3 convolutional layers, group normalization operation, and ReLU activation function; the squeezing-excitation module obtains channel descriptors through global average pooling, and then generates channel weights through two fully connected layers and the Sigmoid function to adaptively calibrate the feature channels.

[0028] For example, the structure of each residual SE module is as follows: the input feature map is sequentially passed through two consecutive 3×3×3 convolutional layers. Group normalization and ReLU activation are performed after each convolutional layer. Group normalization divides the channels into several groups, and the mean and standard deviation of each group are calculated independently using the following formula: in, and These represent the mean and variance of each channel within each group. It is a minimal constant (usually taken as) To prevent division by zero, the output feature maps of the two convolutional layers are added to the input feature map (after adjusting the number of channels through a 1×1×1 convolution) to form a residual connection. Then, a squeeze-and-excitation module (SE module) is integrated after the residual connection.

[0029] For example, the execution process of the SE module includes squeezing, activation, and recalibration. Squeezing: The spatial dimension of the feature map is compressed to 1×1×1 using Global Average Pooling (GAP) to obtain the channel descriptors. Let the input feature map size be... ,in These represent spatial depth, height, width, and number of channels, respectively. The global average pooling output is... The length is obtained as The vector. Excitation: Channel descriptors are sequentially input into two fully connected layers (FC layers). The first layer reduces the number of channels to... ( To achieve the reduction ratio (16 in this embodiment), after ReLU activation, the second layer is restored to... Then, the channel weights are generated using the Sigmoid function. ,in For ReLU function, This is the Sigmoid function. Recalibration: Multiply the channel weights channel-by-channel with the original feature map to obtain the calibrated feature map. Through the above process, the encoder can adaptively highlight important feature channels and suppress irrelevant channels.

[0030] As one possible implementation, in step S2, the 3D void space pyramid pooling module of the bottleneck layer uses four 3×3×3 void convolution branches with void ratios of 1, 2, 4 and 6 in parallel and a global average pooling branch. The outputs of all branches are concatenated along the channel dimension and then fused through a 1×1×1 convolutional layer to capture multi-scale contextual information.

[0031] As one possible implementation, in step S2, attention gating uses the gating signal of the decoder to filter the skip connection features of the encoder and passes the feature map after background noise suppression to the decoder. Attention gating calculates attention coefficients through an additive attention mechanism, specifically including: using the upsampled features of the decoder as a gating signal after passing through a 1×1×1 convolutional layer and a ReLU activation function, adding it to the skip connection features of the corresponding level of the encoder, and then generating attention coefficients by sequentially passing through a ReLU activation function, a 1×1×1 convolutional layer, and a Sigmoid function. The attention coefficients are then multiplied element-wise with the skip connection features to obtain a weighted feature map.

[0032] For example, in the first There are several decoder levels, and the skip connection feature of the corresponding encoder level is given by... (size is) The upsampling features of the decoder are (size is) ,generally Attention gating calculates the attention coefficient through an additive attention mechanism, with the following steps: First of all Perform a 1×1×1 convolution and ReLU activation, and adjust its channel number to match the... The same, the gating signal is obtained. .right Also perform a 1×1×1 convolution (without changing the number of channels) to obtain Then calculate the additive attention score: in, , These are the parameters of the first 1×1×1 convolution layer. It is the ReLU activation function; , These are the parameters for the second 1×1×1 convolution. Use the Sigmoid activation function. Output the attention coefficients. Space dimensions and Similarly, each voxel corresponds to a weight value between 0 and 1. Finally, the attention coefficients are multiplied element-wise with the original skip connection features to obtain the weighted feature map: .in This represents element-wise multiplication. Through this mechanism, the compact 3D segmentation network MSBA-Net can automatically learn to suppress background and irrelevant regions, highlighting tumor-related features.

[0033] S3: Based on the multi-scale deep semantic features extracted in step S2, the decoder outputs a preliminary segmentation prediction map, and the deep supervision mechanism is used to output auxiliary segmentation prediction maps at four resolution levels.

[0034] As one possible implementation, in step S3, the deep supervision mechanism adds auxiliary segmentation heads at four resolution levels: one-eighth of the original resolution, one-quarter of the original resolution, one-half of the original resolution, and the original resolution. Each auxiliary segmentation head consists of a 1×1×1 convolutional layer and an upsampling layer. The 1×1×1 convolutional layer compresses the number of feature map channels to the number of classification categories, and the upsampling layer (using trilinear interpolation) enlarges the feature map to the original image size, thus outputting an auxiliary segmentation prediction map of the same size as the initial segmentation prediction map. The feature map of the corresponding resolution is extracted from the decoder, processed by the auxiliary segmentation head, and then output as the auxiliary segmentation prediction map.

[0035] S4: Construct a boundary-guided hybrid loss function, which combines Dice loss, weighted cross-entropy loss, and boundary-aware loss. Use the boundary-guided hybrid loss function to train the compact 3D segmentation network MSBA-Net and segment test images during the testing phase to output segmentation masks for the entire tumor region, the tumor core region, and the enhanced tumor region of the brain tumor.

[0036] As one possible implementation, in step S4, the boundary-aware loss extracts boundary voxel masks for the entire tumor region, the tumor core region, and the enhanced tumor region of the brain tumor. The extraction method is as follows: for each sub-region (the entire tumor region, the tumor core region, and the enhanced tumor region), the ground truth label mask G (a binary mask, where 1 represents the voxel of the sub-region and 0 represents the background) is sequentially subjected to morphological erosion and dilation operations. The result of the dilation operation is subtracted from the result of the erosion operation to obtain the boundary voxel mask of the sub-region. Then, the union of the boundary voxel masks of the three sub-regions is taken as the final boundary voxel mask, denoted as . Only at the boundary voxel mask The complement of the Dice coefficients is calculated above, and the formula for calculating the boundary-aware loss is as follows: in, To predict probabilities, Let $i$ be the ground truth label and $i$ be the voxel index. The boundary-aware loss is essentially the complement of the Dice coefficients. The loss is 0 when the predicted probability is completely consistent with the ground truth label on the boundary voxel; and 1 when they are completely inconsistent. This loss function guides the compact 3D segmentation network MSBA-Net to focus on the boundaries of tumor sub-regions.

[0037] For example, a spherical structural element with a radius of 2 voxels is selected. Let the result of the expansion operation be... The corrosion operation results are Then the boundary voxel mask of this sub-region The dilation result is subtracted from the erosion result to obtain a binary mask containing only boundary voxels. The union of the boundary voxel masks of the three sub-regions is then used to obtain the final boundary voxel mask. .

[0038] In step S4, the formula for calculating the total loss function of the boundary-guided hybrid loss function is: in, The primary output loss is the loss from the initial segmentation prediction map. (i=1, 2, 3, 4) represent the four deep-supervised auxiliary losses, i.e., the losses of the four auxiliary segmentation prediction maps. To assist in loss weighting; Main output loss The calculation formula is: in, For Dice's loss, For weighted cross-entropy loss, For boundary-aware loss, , , These are the corresponding weighting coefficients.

[0039] As one possible implementation, in step S4, before training the compact 3D segmentation network MSBA-Net, a mixed training batch is constructed using the MixUp data augmentation strategy. MixUp is a linear interpolation data augmentation method, specifically including: for two random samples within a batch... and Generate mixed samples : The mixing coefficient λ follows a Beta distribution with parameter α = 1.0, i.e., λ∼Beta(1.0, 1.0), and the probability density function of the Beta distribution is: ,when When [condition], the distribution degenerates into a uniform distribution on [0,1]. This means [the distribution degenerates into a uniform distribution on [0,1]]. Approximately uniform sampling between 0 and 1 allows for diverse weight allocation in the mixed samples. A mixed training batch is constructed using the MixUp data augmentation strategy, enabling the compact 3D segmentation network MSBA-Net to learn the linear relationship between two samples, improving generalization ability and suppressing overfitting.

[0040] As one possible implementation, in step S4, when training the compact 3D segmentation network MSBA-Net, PyTorch's automatic mixed-precision training is used. FP16 (half-precision floating-point, 16-bit floating-point format) is used for forward propagation and gradient calculation, while FP32 (single-precision floating-point, 32-bit floating-point format) is used for parameter updates to maintain numerical stability. Gradient scaling is employed, multiplying the loss value by a scaling factor, and then dividing the gradient by the same scaling factor after backpropagation to prevent numerical underflow.

[0041] In DataLoader, multi-process data prefetching is configured by setting the `num_workers` parameter to 4 and the `prefetch_factor` parameter to 2. This enables four child processes to load data in parallel, with each child process prefetching two batches of data into memory, thereby reducing CPU-GPU data transfer latency. All preprocessed training data is then cached in memory to avoid repeatedly reading data from disk during training, significantly improving training speed.

[0042] In the testing phase, based on the pre-trained compact 3D segmentation network MSBA-Net, as a possible implementation, in step S4, when segmenting the test image, K preset geometric transformations are applied to each test sample. Geometric transformations include identity transformations, single-axis flips (rotations around the z-axis, y-axis, and x-axis), and two-axis flips. The transformed samples are then input into the compact 3D segmentation network MSBA-Net to obtain the predicted probability map. Apply inverse transform to the prediction probability map Transform it back to the original spatial coordinate system to obtain The probability maps after K inverse transformations are averaged and fused, and the calculation formula is as follows: Probability map after average fusion Take the maximum value along the category dimension, that is, by... The final segmentation result is obtained by taking argmax, and the segmentation mask of the entire tumor region, the segmentation mask of the tumor core region, and the segmentation mask of the enhanced tumor region are output.

[0043] To verify the effectiveness of the method provided in this application, the applicant conducted extensive experiments on the Medical Segmentation Decathlon (MSD) Task01_BrainTumour dataset. This dataset contains 484 multimodal 3D MRI images of the brain, and the Dice Similarity Coefficient (DSC) and 95% Hausdorff Distance (HD95) were used as evaluation metrics.

[0044] For example, refer to Figure 2 This application compares MSBA-Net with representative methods on the MSD Task01 BrainTumour dataset. These methods include classic models (3D U-Net, Attention U-Net, nnU-Net) and state-of-the-art models (nnFormer and SwinUNETR V2). The performance metrics of each comparison model are cited from the original paper or official implementation. Experimental results show that the method (MSBA) provided in this application achieves the best WT / TCDice value (0.918 / 0.885), which is better than nnFormer and SwinUNETR V2. Its ET Dice value is also excellent (0.838, while nnFormer and SwinUNETR V2 are 0.841 and 0.835, respectively). The method provided in this application requires only 45 million parameters—27.7% less than SwinUNETR V2 (62.8 million parameters) and 34.2% less than nnFormer (68.4 million parameters)—fully demonstrating its good balance between accuracy and efficiency.

[0045] For example, refer to Figure 3 The method provided in this application embodiment can accurately segment the three sub-regions of the tumor (the entire tumor region WT, the tumor core region TC, and the enhanced tumor region ET) under different modal inputs (T1, T1ce, T2, and FLAIR). Figure 3 The original views of each modality image are shown overlaid with the corresponding segmentation masks generated by this method, demonstrating the effective fusion capability of this method for multimodal information.

[0046] To evaluate the practical benefits of the engineering-optimized process of the method provided in this application, training and inference efficiencies were compared under the same hardware conditions. For example, refer to... Figure 4The comprehensively optimized process reduced the training cycle time from 199 seconds to 59 seconds, achieving a 3.4x speedup. During the inference phase, although TTA increased the single inference time from 0.57 seconds to 0.70 seconds, it significantly improved segmentation robustness (an average 2.1% increase in Dice value). Disabling TTA reduced the inference time to 0.09 seconds, a 6.3x improvement over the baseline, making it perfectly suitable for real-time applications.

[0047] To verify the necessity of the core components of the method provided in this application, embodiments of this application designed ablation experiments and performed statistical analysis using paired t-tests. For example, ... Figure 5 and Figure 6 As shown in the figure. Ablation experiments show that the complete model (with a residual extrusion excitation module, a 3D void space pyramid pooling module, attention-gated skip connections, boundary-aware loss, and the MixUp data augmentation strategy) improves the average Dice coefficient by 2.22% compared to the baseline model. MixUp contributes the most (1.54% improvement), followed by boundary-aware loss (1.01% improvement). Statistical significance tests (paired t-tests) show that the improvement of the complete model compared to the baseline model is highly significant (p < 0.01).

[0048] For example, refer to Figure 7 , Figure 7 Three typical cases (brats 422, brats 417, and brats 446) were selected for a visual comparison of the segmentation results. Figure 7 The first row contains the actual labels, the second row contains the segmentation results of this invention, and the third row contains a heatmap of the segmentation error. Figure 7 It can be seen that the method provided in this application embodiment can still maintain a high degree of segmentation integrity in complex cases with irregular tumor morphology and blurred boundaries.

[0049] Reference Figure 8 The segmentation mask output by the method provided in this application, after being reconstructed in three dimensions, can clearly present the three-dimensional morphology, spatial location, and adjacent relationship between the brain tumor and the surrounding brain tissue in three-dimensional space. Figure 8 The image shows 3D renderings of three tumor sub-regions (green for the entire tumor region, yellow for the tumor core region, and red for the enhanced tumor region). These results can be directly used for clinical surgical planning, radiotherapy target delineation, and efficacy evaluation.

[0050] In the description of this specification, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0051] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multimodal magnetic resonance image segmentation method based on MSBA-Net, characterized in that, include: S1: Acquire multimodal brain tumor MRI images and preprocess them to obtain preprocessed images. The multimodal brain tumor MRI images include T1 modality images, T1ce modality images, T2 modality images, and FLAIR modality images. S2: Construct a compact 3D segmentation network MSBA-Net. Input the preprocessed images into the compact 3D segmentation network MSBA-Net to extract multi-scale deep semantic features. The compact 3D segmentation network MSBA-Net includes an encoder, a bottleneck layer, and a decoder. The encoder consists of four residual squeeze-excitation modules. Each residual squeeze-excitation module is integrated after residual connections. The bottleneck layer adopts a 3D voided spatial pyramid pooling module and uses multiple voided convolutional branches with different voiding rates and a global average pooling branch in parallel. An attention-gated skip connection is introduced between the decoder and the encoder. S3: Based on the multi-scale deep semantic features extracted in step S2, the decoder outputs a preliminary segmentation prediction map, and the deep supervision mechanism outputs auxiliary segmentation prediction maps at four resolution levels; S4: Construct a boundary-guided hybrid loss function, which combines Dice loss, weighted cross-entropy loss, and boundary-aware loss. The boundary-guided hybrid loss function is used to train the compact 3D segmentation network MSBA-Net, and the test image is segmented during the testing phase to output a segmentation mask for the entire tumor region, the tumor core region, and the enhanced tumor region of the brain tumor.

2. The multimodal magnetic resonance image segmentation method based on MSBA-Net according to claim 1, characterized in that, In step S2, each residual squeezing-excitation module further includes two consecutive 3×3×3 convolutional layers, group normalization operation, and ReLU activation function; the squeezing-excitation module obtains channel descriptors through global average pooling, and then generates channel weights through two fully connected layers and the Sigmoid function to adaptively calibrate the feature channels.

3. The multimodal magnetic resonance image segmentation method based on MSBA-Net according to claim 1, characterized in that, In step S2, the 3D void space pyramid pooling module of the bottleneck layer uses four 3×3×3 void convolution branches with void ratios of 1, 2, 4 and 6 in parallel and one global average pooling branch. The outputs of all branches are concatenated along the channel dimension and then fused through a 1×1×1 convolution layer to capture multi-scale contextual information.

4. The multimodal magnetic resonance image segmentation method based on MSBA-Net according to claim 1, characterized in that, In step S2, the attention gating uses the gating signal of the decoder to filter the skip connection features of the encoder and transmits the feature map after background noise suppression to the decoder. The attention gating calculates attention coefficients through an additive attention mechanism, specifically including: using the upsampled features of the decoder as a gating signal after passing through a 1×1×1 convolutional layer and a ReLU activation function, adding it to the skip connection features of the corresponding layer of the encoder, and then generating attention coefficients by sequentially passing through a ReLU activation function, a 1×1×1 convolutional layer, and a Sigmoid function. The attention coefficients are then multiplied element-wise with the skip connection features to obtain a weighted feature map.

5. The multimodal magnetic resonance image segmentation method based on MSBA-Net according to claim 1, characterized in that, In step S3, the deep supervision mechanism adds auxiliary segmentation heads to four resolution levels, including an eighth-to-one original resolution level, a quarter-to-one original resolution level, a half-to-one original resolution level, and an original resolution level. Each auxiliary segmentation head consists of a 1×1×1 convolutional layer and an upsampling layer. The feature map of the corresponding resolution is extracted from the decoder, processed by the auxiliary segmentation head, and then output as an auxiliary segmentation prediction map.

6. The multimodal magnetic resonance image segmentation method based on MSBA-Net according to claim 1, characterized in that, In step S4, the boundary-aware loss method extracts boundary voxel masks for the entire tumor region, the tumor core region, and the enhanced tumor region of the brain tumor. The extraction method is as follows: morphological erosion and dilation operations are sequentially performed on the ground truth mask of each sub-region. The result of the dilation operation is subtracted from the result of the erosion operation to obtain the boundary voxel mask of that sub-region. Then, the union of the boundary voxel masks of the three sub-regions is taken as the final boundary voxel mask, denoted as . Only on the boundary voxel mask The complement of the Dice coefficients is calculated above, and the formula for calculating the boundary-aware loss is as follows: in, To predict probabilities, This is a real label.

7. The multimodal magnetic resonance image segmentation method based on MSBA-Net according to claim 1, characterized in that, In step S4, the formula for calculating the total loss function of the boundary-guided hybrid loss function is as follows: in, The primary output loss is the loss of the initial segmentation prediction map. The four deep-supervised auxiliary losses are the losses of the auxiliary segmentation prediction maps. To assist in loss weighting; The main output loss The calculation formula is: in, For Dice's loss, For weighted cross-entropy loss, For boundary-aware loss, , , These are the corresponding weighting coefficients.

8. The multimodal magnetic resonance image segmentation method based on MSBA-Net according to claim 1, characterized in that, In step S4, before training the compact 3D segmentation network MSBA-Net, the MixUp data augmentation strategy is used to construct mixed training batches. Specifically, this includes: for two random samples within a batch... and Generate mixed samples : The mixing coefficient λ follows a Beta distribution with parameter α=1.0, i.e., λ∼Beta(1.0, 1.0), such that λ is approximately uniformly distributed on [0, 1].

9. The multimodal magnetic resonance image segmentation method based on MSBA-Net according to claim 1, characterized in that, In step S4, when training the compact 3D segmentation network MSBA-Net, PyTorch's automatic mixed precision training is used, FP16 is used for forward propagation and gradient calculation, and gradient scaling is used to prevent numerical underflow; multi-process data prefetching is set in DataLoader, with num_workers set to 4 and prefetch_factor set to 2; and all preprocessed training data is cached in memory.

10. The multimodal magnetic resonance image segmentation method based on MSBA-Net according to claim 1, characterized in that, In step S4, during the testing phase, when segmenting the test image, K preset geometric transformations are applied to each test sample. The geometric transformations include identity transformations, single-axis flips, and dual-axis flips; the transformed samples are input into the compact 3D segmentation network MSBA-Net to obtain the predicted probability map. Apply inverse transform to the predicted probability map Transform it back to the original space; average and fuse the probability maps after K inverse transformations, using the following formula: Through the The final segmentation result is obtained by taking argmax, and the segmentation mask of the entire tumor region, the segmentation mask of the tumor core region, and the segmentation mask of the enhanced tumor region are output.

Images