MRI (Magnetic Resonance Imaging) brain tumor segmentation method based on multi-scale attention mechanism and secondary segmentation strategy

Abstract

The invention discloses an MRI (Magnetic Resonance Imaging) brain tumor segmentation method based on a multi-scale attention mechanism and a secondary segmentation strategy, and belongs to the field of semantic segmentation. The method comprises the following steps: firstly, acquiring MRI brain tumor data and preprocessing the MRI brain tumor data; secondly, designing a 3D U-Net model, and combining a multi-scale attention mechanism and depth separable convolution to extract features; then, secondary segmentation guided by a cascade strategy is introduced; and finally, predicting by using the optimal model, storing and verifying a result, and calculating and comparing evaluation indexes. A depth separable convolution module is used for replacing a traditional convolution module, so that the parameter and calculation cost is reduced, and the segmentation precision is kept. In addition, a multi-scale attention fusion module is introduced to the first two layers of the encoder, the ability of capturing shallow information is enhanced, and important features are further extracted. A cascade attention mechanism is introduced into a decoder, and the segmentation capability of a small target area is improved. And training is carried out through a BraTS2020 data set, and the result shows that the optimized network significantly improves the MRI brain tumor segmentation accuracy.

Description

Technical Field

[0001] The present invention relates to a deep neural network, which is an MRI brain tumor segmentation method based on a multi-scale attention mechanism and a secondary segmentation strategy, and belongs to the field of semantic segmentation. Background Art

[0002] MRI brain tumor segmentation is a critical medical image analysis task that aims to help doctors accurately identify and locate abnormal brain tissue through precise image processing technology. Brain tumors are tumors that grow abnormally in brain tissue, which seriously threaten the function of the nervous system, are difficult to treat, and have a high mortality rate. Traditional manual segmentation methods are not only time-consuming and labor-intensive, but also prone to errors, so the development of automated segmentation technology is particularly important.

[0003] Magnetic resonance imaging (MRI), as the main non-invasive brain tumor imaging technology, can generate high-resolution, non-destructive multi-modal brain images, such as Flair, T1-weighted, T1ce and T2-weighted modalities. Each modality provides different tissue contrast and information, providing doctors with rich diagnostic evidence, but it faces the challenges of complex information, large volume and difficulty in interpretation.

[0004] In recent years, medical image analysis technology based on deep learning has made significant progress, especially U-Net and its improved versions. U-Net effectively captures local and global features through the encoder-decoder structure and performs well in medical image segmentation tasks. The introduction of mechanisms such as dilated convolution and multi-scale feature fusion further enhances the model's ability to understand subtle structures and large-scale contextual information, improving the accuracy and robustness of segmentation. Summary of the invention

[0005] According to the current technical difficulties, in order to solve the problems of blurred segmentation boundaries and poor attention to smaller targets in brain tumor segmentation, the present invention provides an MRI brain tumor segmentation method based on a multi-scale attention mechanism and a secondary segmentation strategy. The method effectively extracts shallow information in the model and refines the boundaries and structures, and has good segmentation performance. The specific scheme of the present invention is as follows:

[0006] The MRI brain tumor segmentation method based on multi-scale attention mechanism and secondary segmentation strategy includes the following steps:

[0007] S1: Acquire MRI brain tumor data and perform data preprocessing;

[0008] S2: A 3D U-Net model combining multi-scale attention mechanism and 3D depthwise separable convolution for feature extraction;

[0009] S3: Introducing a segmentation method guided by a cascade strategy for secondary segmentation;

[0010] S4: Use the optimal model to make predictions, save and verify the results online, calculate the evaluation indicators, and finally compare the results.

[0011] Furthermore, the dataset used in the experiment in step S1 is the BraTS2020 glioma public dataset provided by the International Medical Image Computing and Computer-Assisted Intervention Association (MICCAI). By preprocessing the brain tumor dataset, especially the multimodal MRI images, noise signals such as grayscale unevenness and abnormal points can be effectively removed, which not only improves the image quality, but also helps to accurately identify and segment the background, necrotic tumor core area, edema area around the tumor, and enhanced tumor area, thereby improving the accuracy and reliability of segmentation.

[0012] Furthermore, the improvement on the basic U-Net structure in step S2 is as follows:

[0013] S21, using 3D depth-separable convolutional modules to replace the traditional modules in the original U-Net network;

[0014] S22. A multi-scale attention mechanism combining local detail attention module, different scale context information processing and global attention module is designed and introduced into the first two layers of U-Net network encoder.

[0015] The present invention is improved on the basis of the U-Net network structure, focusing on optimizing the design of its encoding and decoding stages. In view of the problem that the traditional U-Net has insufficient parameter efficiency and spatial information capture ability when processing three-dimensional medical images, a 3D deep separable convolution module is used to replace the traditional module in the original U-Net network. A multi-scale attention fusion module designed by us is introduced in the first and second layers of the U-Net encoder. The module adds the input tensor and its corresponding residual tensor, and then captures and enhances feature information at different levels through local detail attention, contextual information processing at different scales, and global attention submodules. Finally, the Sigmoid activation function is used to generate weight coefficients, and these weights are applied to the input tensor and residual tensor to generate the final output feature tensor. The whole process effectively improves the segmentation accuracy and perception ability of the model for medical images such as MRI brain tumors.

[0016] Furthermore, in step S3, a secondary segmentation method guided by a cascade strategy is introduced, and a single network is used to achieve the task segmentation of brain tumors. The attention mechanism is used to refine the small target area in the coarse segmentation result. Two modules are defined: AF and MSACM. First, the AF module implements the cascade attention mechanism, and extracts, normalizes and maps the input feature tensor through two 3D convolutional layers and a Sigmoid activation function, and finally generates the processed feature tensor. Then, the MSACM model uses two AF modules to split the input tensor by channel, and inputs the segmented channels into the AF module for processing, thereby achieving the fine segmentation task.

[0017] Furthermore, in step S4, the model prediction and evaluation uses the best model after training and optimization to predict the validation set, and submits the predicted segmentation results to the official website: https: / / ipp.cbica.upenn.edu / for verification. When evaluating the model performance, the Dice similarity coefficient and Hausdorff_95 distance are used as the main quantitative indicators. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of the complete model structure in the present invention

[0019] Figure 2(a) is a schematic diagram of 3D depth-separable convolution in the present invention

[0020] FIG. 2( b ) is a schematic diagram of a conventional convolution module in the present invention.

[0021] Figure 3 Schematic diagram of the multi-scale attention mechanism module in the present invention

[0022] Figure 4 Schematic diagram of evaluation index in the present invention DETAILED DESCRIPTION

[0023] S1: Acquire MRI brain tumor data and perform data preprocessing

[0024] S11, data acquisition stage:

[0025] This study uses the BraTS2020 dataset, which contains MR images of four modalities (Flair, T1, T1ce, and T2) and segmentation labels manually labeled by experienced neuroradiologists. The labeled data is divided into four parts: enhanced tumor area, peritumoral edema area, necrotic tumor core area, and background. In order to comprehensively evaluate the segmentation effect, these parts are divided into three tasks: the entire tumor area (WT), the tumor core area (TC), and the enhanced tumor area (ET). Among them, WT includes enhanced tumor area, peritumoral edema area, and necrotic tumor core area; TC includes enhanced tumor area and necrotic tumor core area; ET only includes enhanced tumor area.

[0026] S12, data preprocessing stage:

[0027] The brain tumor data was preprocessed to remove noise signals in multimodal MRI brain tumor images. First, the areas other than normal brain tissue in the MR images were removed, and the top and bottom 1% slices that did not contain brain tumors were removed on the depth axis. Subsequently, the original 240×240×155 MR images were cut to a size of 128×128×144. Finally, the MR images of each modality were normalized by subtracting the mean and dividing by the standard deviation of the intensity within the slice to address the grayscale inhomogeneity of the brain tumor images.

[0028] S2. MRI brain tumor segmentation method model based on multi-scale attention mechanism and secondary segmentation strategy

[0029] S21. 3D Depthwise Separable Conv (3DSC). Currently, deep learning algorithms in image segmentation usually increase the network depth by stacking convolutional layers, pooling layers, and upsampling layers to fully extract features from the image. However, as the network depth increases, the computational complexity and number of parameters also increase significantly, resulting in longer training time and prone to problems such as gradient vanishing or gradient exploding. To solve these problems and improve the computational efficiency of the network, this paper introduces a 3D depthwise separable convolution module (Figure 2(a)) to replace the traditional convolution module (Figure 2(b)).

[0030] The 3D depthwise separable convolution module is implemented by decomposing the standard 3D convolution operation into two simpler steps, namely, depthwise convolution and pointwise convolution on each input channel. This method significantly reduces the amount of computation and the number of parameters, while reducing the need for repeated calculations. Pointwise convolution linearly combines the output of depthwise convolution through 1x1x1 3D convolution to achieve information integration between channels. Figure 2(a) shows the structure of depthwise convolution and pointwise convolution. This decomposition strategy not only maintains the ability to extract features, but also greatly reduces the computational overhead.

[0031] By using the 3D depth-separable convolution module, the network can improve feature extraction capabilities without significantly increasing computational complexity, thereby achieving more efficient image segmentation. This module is particularly suitable for processing 3D medical imaging data, such as MRI and CT scans, and can significantly reduce training and inference time while ensuring segmentation accuracy.

[0032] S22, Multi-Scale Attention Feature Fusion 3D Module (MSAFF3D) Figure 3 ), by integrating local attention modules, context submodules of different scales and global attention modules, feature extraction and fusion are realized. The local attention module uses 3D convolution and batch normalization to capture local details; the context submodules of different scales effectively process context information from multiple scales through adaptive average pooling and 3D convolution; the global attention module obtains global information through global average pooling. After fusion, these features are activated by Sigmoid to generate weight coefficients, which are finally applied to the input tensor and residual to generate the final output. Through the fusion of multi-scale features, the MSAFF3D module enhances the model's ability to obtain global information and improves image segmentation performance.

[0033] S3, Cascade Attention Mechanism Segmentation (CAMS) implements the cascade attention mechanism to improve image segmentation performance by integrating the cascade attention mechanism module. The cascade attention mechanism module contains a 3D convolution layer, a 3D batch normalization layer, and a 1x1x1 convolution layer for feature extraction, normalization, and feature mapping. The module gradually extracts features in a cascade manner, generates weight coefficients through Sigmoid activation, applies the weights to the input features and residuals, and generates the final output. The cascade attention mechanism module in the CAMS model helps to enhance the model's ability to extract and fuse features while maintaining computational overhead, thereby improving segmentation performance.

[0034] Step 3: Model training

[0035] The model was trained on the training set of the BraTS2020 dataset using the PyTorch framework on the CUDA 12.1 architecture platform. The AdamW optimizer was used for model structure optimization, with an initial learning rate of 0.001, a momentum of 0.9, a batch size of 2, and a training iteration number (eopch) of 200. During the training process, I used the CuDNN neural network computing library to accelerate the operation, and used cosine annealing as the learning rate scheduler.

[0036] Model prediction and evaluation indicators in S4; input the BraTS2020 validation set into the network, use the trained optimal model for segmentation prediction, save the segmentation prediction map of each patient, then enter the official website, upload the segmentation result map of the validation set for online verification, and finally obtain the prediction results and compare the results.

[0037] The main purpose of the evaluation index is to measure the performance of the proposed solution. In this paper, we mainly focus on two types of indicators: Hausdorff distance and Dice similarity coefficient.

[0038] (1) Hausdorff distance

[0039] Hausdorff distance is a mathematical concept that measures the distance between two point sets. It is usually used to compare the similarity or difference between two sets (such as predicted segmentation results and true labels). In the field of medical image processing, especially in segmentation tasks, Hausdorff distance is often used to measure the maximum distance between the predicted segmentation edge and the true label edge. A smaller Hausdorff distance value indicates that the edge of the predicted result is closer to the edge of the true label, and vice versa, it indicates that there is a large difference between the predicted result and the true label. Hausdorff_95 is an adjustment to the Hausdorff distance. By amplifying the calculated result by 0.95 times, it helps to reduce the influence of outliers, thereby more accurately measuring the distance between the edge of the segmentation result and the true label. Its definition formula is as follows.

[0040]

[0041] Where max and min represent the maximum and minimum boundaries of the brain tumor region, respectively. p is a point on the tumor surface P1, which belongs to the result P predicted by the segmentation algorithm. t represents a point on the tumor surface T1, which belongs to the true value map T annotated by the expert. In addition, d(·,·) is a function that calculates the distance between t and p.

[0042] (2) Dice

[0043] Dice is a metric commonly used to evaluate the quality of image segmentation, especially in the field of medical image processing. It measures the similarity between two sets, that is, the similarity between the predicted segmentation results and the true segmentation results. Its definition formula is as follows.

[0044]

[0045] Where ∩ is the logical AND operator, |·| is the area or size of the region, P1 is the region of the segmentation result predicted by the model and T1 represents the actual segmentation region ( Figure 4 ).

[0046] Ablation experiments were conducted on the BraTS2020 dataset. Different modules were added to the U-Net model in turn, and the experiments were performed using the same parameter settings. The specific results are shown in Table 1, where 3DSC represents the 3D depthwise separable convolution module, MSAFF3D represents the multi-scale attention feature fusion 3D module, and CAMS represents the cascaded attention mechanism fine segmentation module.

[0047] Table 1 Comparison of segmentation results of U-Net with different modules added

[0048]

[0049] From Table 1, we can see that first, replacing the ordinary convolution modules in the U-Net encoder and decoder with 3D depth-separable convolution modules, ET, WT and TC increased by 3%, 2% and 2% respectively. On this basis, adding a multi-scale attention feature fusion 3D module to enhance feature representation helps to more effectively capture local details and global context information, and the accuracy is improved more significantly, ET, WT and TC increased by 5%, 2% and 5% respectively. Then, adding a cascaded attention mechanism fine segmentation module at the end of the decoder, ET, WT and TC increased by 3%, 1% and 3% respectively, indicating that the model's perception of complex structures and subtle features has been significantly enhanced, improving the precision and accuracy of medical image segmentation.

[0050] Experimental results show that the improved network structure proposed in this paper includes multiple modules introduced, such as cascaded attention mechanism and secondary segmentation strategy, which significantly improves the segmentation accuracy of MRI brain tumor images. These improvements effectively enhance the model's ability to identify different tumor areas, bringing more accurate and reliable solutions to the field of medical image analysis.

Claims

1. An MRI brain tumor segmentation method based on a multi-scale attention mechanism and a secondary segmentation strategy, characterized in that: The method comprises: S1: Acquire MRI brain tumor data and perform data preprocessing; S2: Design a 3D U-Net model that combines multi-scale attention mechanism and depthwise separable convolution for feature extraction; S3: Introducing a segmentation method guided by a cascade strategy for secondary segmentation; S4: Use the optimal model to make predictions, save and verify the results online, calculate the evaluation indicators, and finally compare the results.

2. The MRI brain tumor segmentation method based on multi-scale attention mechanism and secondary segmentation strategy according to claim 1, characterized in that: In step S1, the experimental data set uses the public glioma data set BraTS2020 provided by the International Medical Image Computing and Computer-Assisted Intervention Association (MICCAI), in which the labeled data is divided into label0: background, label1: necrotic tumor core area, label2: edema area around the tumor, label4: enhanced tumor area; the brain tumor data set is preprocessed to eliminate the noise signal in the multimodal images of brain tumors and reduce their grayscale inhomogeneity.

3. The MRI brain tumor segmentation method based on multi-scale attention mechanism and secondary segmentation strategy according to claim 1, characterized in that: The step S2 is improved on the basic U-Net network structure, specifically: S21, using 3D depth-separable convolutional modules to replace the traditional modules in the original U-Net network; S22. A multi-scale attention mechanism combining local detail attention module, context information processing of different scales and global attention module is designed and introduced into the first two layers of the U-Net network encoder.

4. The MRI brain tumor segmentation method based on multi-scale attention mechanism and secondary segmentation strategy according to claim 3, characterized in that: In step S21, in the encoding and decoding stages of the U-Net model, the traditional convolution module is replaced with a 3D depth-separable convolution module. Using 3D depth-separable convolution can significantly reduce the number of parameters and computational costs while maintaining the model's feature extraction capability and segmentation accuracy, and is particularly suitable for MRI brain tumor segmentation tasks.

5. The MRI brain tumor segmentation method based on multi-scale attention mechanism and secondary segmentation strategy according to claim 3, characterized in that: In step S22, a multi-scale attention fusion module designed by us is introduced into the first and second layers of the U-Net encoder. The module adds the input tensor and its corresponding residual tensor, and then captures and enhances the feature information at different levels through local detail attention, context information processing at different scales, and global attention submodules. Finally, the weight coefficients are generated using the Sigmoid activation function, and these weights are applied to the input tensor and residual tensor to generate the final output feature tensor. The whole process effectively improves the segmentation accuracy and perception ability of the model for medical images such as MRI brain tumors.

6. The MRI brain tumor segmentation method based on multi-scale attention mechanism and secondary segmentation strategy according to claim 1, characterized in that: In step S3, a single network is used to implement the task segmentation of brain tumors. The attention mechanism is used to refine the small target area in the rough segmentation result. Two modules are defined: AF and CAMS. First, the AF module implements the cascade attention mechanism. Through two 3D convolutional layers and a Sigmoid activation function, the input feature tensor is extracted, normalized and mapped, and finally the processed feature tensor is generated. Then, the CAMS model uses two AF modules to split the input tensor by channel, and inputs the segmented channels into the AF module for processing, thereby achieving the fine segmentation task.

7. The MRI brain tumor segmentation method based on multi-scale attention mechanism and secondary segmentation strategy according to claim 1, characterized in that: In step S4, after the model is built, the model is trained using the BraTS2020 training set, and the best model in the training process is saved for prediction.

8. The MRI brain tumor segmentation method based on multi-scale attention mechanism and secondary segmentation strategy according to claim 8, characterized in that: The best model trained on the BraTS2020 validation set is used for prediction, and then various evaluation indicators are obtained through online verification, and the experimental results are compared and analyzed.

Images