A Brain Tumor Segmentation Method for MRI Images Based on Improved 3D-UNet

Abstract

The present invention discloses a method for segmenting brain tumors in MRI images based on an improved 3D-UNet, which specifically includes: 1) preprocessing the brain MRI images in the dataset to obtain a training set; 2) constructing an improved 3D-UNet framework by combining dilated convolution, channel attention mechanism, and residual convolution; 3) training the improved 3D-UNet, the brain tumor segmentation network, with the training set data; 4) importing the test data of the brain MRI image to be segmented into the brain tumor segmentation network to obtain the segmented result. The present invention provides an automatic, accurate, and repeatable tumor segmentation algorithm, which can effectively solve the problems that the traditional manual segmentation method is time-consuming and overly dependent on the subjective experience of experts, and has a higher segmentation accuracy than the method for segmenting brain MRI images based on UNet. At the same time, the method proposed by the present invention can be further applied to other medical MRI image segmentation tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and is applicable to brain MRI image segmentation, and can be used in fields such as medical image segmentation and brain tumor recognition. Background Art

[0002] Brain tumors are abnormal tissue growths that can lead to increased intracranial pressure and damage to the central nervous system, thus endangering the patient's life. Reliably detecting and segmenting brain tumors from magnetic resonance images can assist in surgical planning and treatment evaluation in medical diagnosis. Currently, most brain tumors are manually segmented by medical experts, which is time-consuming and overly dependent on the subjective experience of experts. Computer-aided tumor segmentation plays an increasingly important role in modern medical analysis. However, due to the large spatial and structural variability of brain tumors, as well as the overlap between the tumor gray intensity range and the healthy tissue gray intensity range, traditional machine learning methods are still unable to accurately segment brain tumors from magnetic resonance images. Therefore, developing an automatic, accurate, and repeatable tumor segmentation algorithm remains a challenging task. Summary of the Invention

[0003] In view of the deficiencies of the prior art, the present invention provides a method for segmenting brain tumors in MRI images based on an improved 3D-UNet. This method can effectively solve the problems of the traditional manual segmentation method being time-consuming and overly dependent on the subjective experience of experts, and has a higher segmentation accuracy than the method for segmenting brain MRI images based on UNet. At the same time, the method proposed by the present invention can be further applied to other medical MRI image segmentation tasks.

[0004] In order to achieve the above-mentioned invention purpose, the technical solution adopted by the present invention is as follows:

[0005] 1. Slice the three-dimensional MRI image along the z-axis, label it in sequence, and then remove the skull part. The specific method is as follows:

[0006] 1) Start a straight-line search from the middle position on the leftmost side of the sliced image to the right.

[0007] 2) Stop the search after finding the first non-zero pixel point and use it as the initial coordinate for skull removal.

[0008] 3) Start searching from the initial coordinate and use the region growing algorithm to search for the complete skull image and mark it.

[0009] 4) Delete the marked pixel points.

[0010] After the removal is completed, reassemble the sliced images according to the labels into a three-dimensional MRI image of the brain after skull removal.

[0011] 2. Search for all non-zero pixel points in the image and mark their positions. Use the maximum and minimum values among all the marked positions as the upper and lower boundaries of the image, and delete the invalid pixel points outside the boundaries.

[0012] 3. Use the nearest neighbor interpolation algorithm to resize the image resolution to 120*120*80.

[0013] 4. Perform Max-min normalization on all pixels.

[0014] 5. Randomly divide the processed three-dimensional MRI image into a training set and a validation set, with a ratio of 9:1 between the two.

[0015] 6. Use a splicing module to splice the images of four modalities, namely T1, T2, T1CE, and Flair, of the MRI image in channels and then input them into the 3D-UNet network.

[0016] 7. Modify the sampling module in the original network so that the sampling module at each level includes a main branch composed of, in sequence, a 3*3*3 convolutional kernel with a stride of 2, a ReLU activation function, a 3*3*3 convolutional kernel with a stride of 1, and a ReLU activation function layer, and a residual branch composed of, in sequence, a 1*1*1 convolutional kernel with a stride of 2 and a ReLU activation function layer. The two branches share the same input, and the outputs of the two branches are simultaneously input into a superimposing module for addition and finally output to the residual convolutional module at the next level.

[0017] 8. Replace the bridging part of the original network with three parallel modules that accept the input simultaneously. The structures of the three modules are all composed of, in sequence, a 3*3*3 convolutional kernel with a stride of 1, a ReLU activation function layer, and a BatchNormalization layer. The difference is that the dilation rates of the three convolutional kernels are 3, 6, and 9 respectively. The outputs of the three parallel modules are simultaneously input into a splicing module, and the output is spliced by channels and then output as a dilated feature extraction module.

[0018] 9. Upsample the convolutional result of the subsequent stage in the downsampling process and splice it with the upper layer result and input it into the splicing module in the decoding part of the network.

[0019] 10. Perform global average pooling with an output of 1*1*1 on all input channels to obtain the feature values of each channel. Pass through a fully connected layer 1, a ReLU layer, a fully connected layer 2, and normalization in sequence to obtain the weights of each channel. Sort according to the channel weight sizes, discard the channels with lower weights, and only retain the required number of channels.

[0020] 11. Import the obtained training data into the improved 3D-UNet network for training to obtain a brain tumor segmentation network.

[0021] 12. Import the brain MRI image data to be segmented into the trained segmentation network to obtain the segmented tumor location mask.

[0022] 13. Combine the obtained tumor location mask with the original brain MRI image to obtain the final segmentation result. Description of the Drawings

[0023] Figure 1 It is a flowchart of a method for segmenting brain tumors in MRI images based on an improved 3D-UNet of the present invention.

[0024] Figure 2 It is a flowchart of the residual convolution module.

[0025] Figure 3 It is a flowchart of the dilated feature extraction module.

[0026] Figure 4 It is a flowchart of the channel attention module.

[0027] Figure 5 It is the brain tumor segmentation result. Detailed Embodiments

[0028] The following describes the detailed embodiments of the present invention to facilitate those skilled in the art of the present technology to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the detailed embodiments. For those of ordinary skill in the art of the present technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concept of the present invention are within the scope of protection.

[0029] Preprocess the BraTS dataset obtained from Kaggle, specifically including integrating different types of tumors in the obtained dataset, placing the labeled tumor information and the corresponding brain MRI images in a unified folder and renaming them, deleting the edema area in the labeled information, and merging the enhanced tumor area and the necrosis area as the target area for tumor segmentation.

[0030] Since the data resolution provided by the BraTS dataset is 240*240*155 and the skull part has been removed in advance, the skull removal work is carried out here.

[0031] Adjust the data resolution to 120*120*80 and normalize the data.

[0032] Statistically generate the required file list for the preprocessed dataset and randomly select 1 / 10 as the test set for training.

[0033] Reference Figure 1 ,Figure 1 The flowchart of the brain tumor segmentation method for MRI images based on the improved 3D-UNet is shown. The required network is constructed, which consists of five parts: input, output, upsampling, downsampling, and bridging part.

[0034] The input part mainly reads the previously generated file list, inputs four-modal MRI images into the model simultaneously, processes them as four independent channels, inputs them into a 3*3*3 convolutional kernel with a stride of 1 for channel number adjustment, adjusts the channel number to 16, then inputs it into the ReLU layer, a 3*3*3 convolutional layer with a stride of 1, and after the ReLU layer, inputs it into the downsampling part.

[0035] The downsampling part is divided into four levels of downsampling in total. All four levels are residual convolution modules, and its specific structure is as Figure 2 shown. Among them, on the left are a 3*3*3 convolutional kernel with a stride of 2, ReLU, a 3*3*3 convolutional kernel with a stride of 1, ReLU. Each layer maintains the image resolution from decreasing due to the edge position by filling empty pixels. On the right is a 1*1*1 convolutional kernel with a stride of 2, ReLU. Finally, the two sides are superimposed by pixel-by-pixel addition and output to the next layer and reserved as the input information of the skip layer. After each layer of downsampling of the input image, the number of pixels will be reduced to 1 / 8 of the original, but the number of feature channels in the convolutional part will double. After four levels of downsampling, 128 feature channels with a resolution of 30*30*20 are obtained and input into the bridging part.

[0036] The bridging part is an atrous convolution module, which is specifically composed of three parts. The structures of these three parts are all composed of 1 3*3*3 convolutional kernel with a stride of 1, a ReLU activation function layer, and a BatchNormalization layer in sequence. The difference is that the dilation rates of the three convolutional kernels are 3, 6, and 9 respectively. The outputs of the three parallel modules and the input of the atrous module are simultaneously input into a splicing module. After splicing the outputs by channel, 512 feature channels with a resolution of 30*30*20 are output.

[0037] The output feature channels are input into the upsampling part, which is specifically composed of a channel attention part, a feature decoding part, a resolution improvement part, and a skip connection part in four loops. The channel attention part specifically selectively retains the bar feature channels output by the upper layer, only retaining the channels with higher feature extraction efficiency. The specific method is to perform global average pooling with an output of 1*1*1 on all input channels to obtain the feature values of each channel, and then obtain the weights of each channel through a fully connected layer 1, a ReLU layer, a fully connected layer 2, and normalization connected in sequence. Sort the channels according to the channel weight size, discard the channels with lower weights, and only retain the required 128 feature channels. The feature decoding part consists of a 3*3*3 convolution with a stride of 1, a ReLU, a 3*3*3 convolution with a stride of 1, and a ReLU. The resolution improvement part is performed through transposed convolution, and the length, width, and height of each layer are doubled. The skip connection part is to perform channel-wise splicing of the output of the same level in the downsampling part and the upsampling of the next-level output.

[0038] The output module consists of a BatchNormalization layer, a ReLU layer, a 1*1*1 convolution kernel with a stride of 1, and a Softmax activation layer connected in sequence.

[0039] Use binary cross-entropy loss as the loss function to train and tune the parameters of the new network model on the training set, and finally use the validation set to verify the model effect.

[0040] After testing, the improved 3D-UNet network provided by this patent has the following effects in brain tumor segmentation Figure 5 As shown, the dice accuracy of the segmentation result can reach 92%, while the dice accuracy of the segmentation result of the unimproved U-Net network as a comparison is 89%. The experimental results show that the segmentation accuracy of the improved 3D-UNet network has a certain improvement.

[0041] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A method for segmenting brain tumors in MRI images based on an improved 3D-UNet, characterized in that, It includes the following steps: Step 1: Preprocess the obtained 3D MRI image with marker information, which specifically includes the following contents: 1) Slice the 3D MRI image along the z-axis, label it, and then remove the skull part according to the following method: a) Start a linear search from the middle position of the leftmost side of the sliced image to the right; b) Stop the search after finding the first non-zero pixel point and use it as the initial coordinate for skull removal; c) Start searching from the initial coordinate, use the region growing algorithm to search for the complete skull image and label it; d) Delete the labeled pixel points; After the removal is completed, reassemble the sliced images according to the labels into a 3D MRI image of the brain after skull removal; 2) Search for all non-zero pixel points in the image and mark their positions, use the maximum and minimum values among all the marked positions as the upper and lower boundaries of the image, and delete the invalid pixel points outside the boundaries; 3) Use the nearest neighbor interpolation algorithm to adjust the image resolution size to 120*120*80; 4) Perform Max-min normalization on all pixels; 5) Randomly divide the processed 3D MRI image into a training set and a validation set, and the ratio between the two is 9:1 Step 2: Construct an improved 3D-UNet network, which specifically includes the following contents: 1) Use a splicing module to splice the images of four modalities, namely T1, T2, T1CE, and Flair of the MRI image in channels and then input them into the 3D-UNet network; 2) Replace the sampling module of the 3D-UNet network with a residual convolution module; 3) Replace the bridging part of the 3D-UNet network with a dilated feature extraction module; 4) Upsample the convolution result of the latter stage in the downsampling process and splice it with the upper layer result and then input it into the splicing module in the decoding part of the network; 5) Improve the channel selection in the decoding part using a channel attention module; Step 3: Use the preprocessed training set data to train the improved 3D-UNet network to obtain a brain tumor segmentation network; Step 4: Import the brain MRI image to be segmented into the brain tumor segmentation network, and then obtain the brain tumor segmentation result after processing by the brain tumor segmentation network.

2. The MRI image brain tumor segmentation method based on the improved 3D-UNet according to claim 1, characterized in that, For the second step in step two, the specific structure of the residual convolution module is: The sampling module at each level includes: a main branch composed of 1 3*3*3 convolution kernel with a stride of 2, 1 ReLU activation function, 1 3*3*3 convolution kernel with a stride of 1, and 1 ReLU activation function layer in sequence, and a residual branch composed of 1 1*1*1 convolution kernel with a stride of 2 and 1 ReLU activation function layer in sequence. The two branches share the same input, and the outputs of the two branches are simultaneously input into a superimposing module for addition and finally output to the next level.

3. A method for segmenting brain tumors in MRI images based on an improved 3D-UNet, characterized in that, For the third step in step two, the specific structure of the dilated feature extraction module part is: Three parallel modules are used to receive the input simultaneously. The structure of each of the three modules consists of a 3*3*3 convolutional kernel with a stride of 1, a ReLU activation function layer, and a Batch Normalization layer in sequence. The difference lies in that the dilation rates of the three convolutional kernels are 3, 6, and 9 respectively. The outputs of the three parallel modules are simultaneously input into a splicing module, and then spliced by channels and output.

4. A method for segmenting brain tumors in MRI images based on an improved 3D-UNet, characterized in that, In step 5 of step 2, the specific structure of the channel attention module is as follows: Global average pooling with an output of 1*1*1 is performed on all input channels to obtain the eigenvalue of each channel. It passes through the connected fully connected layer 1, ReLU layer, fully connected layer 2, and normalization processing in sequence to obtain the weight of each channel, sorts according to the channel weight size, discards the channels with lower weights, and only retains the required number of channels.

5. A method for segmenting brain tumors in MRI images based on an improved 3D-UNet, characterized in that, In step 2, the specific structure of the improved 3D-UNet network is as follows: The input splicing module is connected to four residual convolutional modules for downsampling, followed by a dilated feature extraction module, and then connected to an upsampling module composed of a tandem module, a channel attention module, and a residual convolutional module for four layers. In the downsampling part, each residual convolutional module is connected to the tandem module at the same level and input into the upper-level tandem module after upsampling, and finally connected to the output module.

Images