A lightweight brain tumor segmentation method and system based on V-Net
By improving the V-Net network structure and combining group normalization, depthwise separable convolution, and SE attention mechanism, the problem of 2D networks losing 3D contextual information is solved, achieving efficient brain tumor segmentation, which is suitable for clinical diagnosis and treatment.
Patent Information
- Application Number
- CN202311301482.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-10
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-10-10
AI Technical Summary
Existing 2D convolutional network methods lose important contextual information from 3D images in brain tumor image segmentation, and are computationally intensive and slow to train, making them difficult to meet the needs of clinical diagnosis.
A lightweight brain tumor segmentation method based on V-Net is adopted. The method improves the network structure by preprocessing the image, constructing a V-Net network, using group normalization and depthwise separable convolution, adding SE attention mechanism, and training with a hybrid loss function BCEDice Loss. Skip connections are also used to improve segmentation accuracy and speed.
While reducing computational load and accelerating training speed, it maintains high segmentation accuracy, improves the performance of brain tumor segmentation, and is suitable for clinical diagnosis and treatment.
Smart Images

Figure CN117422871B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of deep learning computer vision and medical image processing technology, and particularly relates to a lightweight brain tumor segmentation method and system based on V-Net. Background Technology
[0002] Brain tumors are among the most serious diseases threatening patients' lives. They are abnormal clusters of cells growing within the skull, with irregular shapes and unpredictable sizes, capable of appearing anywhere in the brain. They can cause severe dysfunction of the nervous system, making them a life-threatening tumor. Gliomas are the most common type of intracranial brain tumor, characterized by high incidence, high recurrence rate, high mortality rate, and low cure rate. Based on the degree of invasion and patient prognosis, gliomas can be classified into high-grade gliomas (HGG) and low-grade gliomas (LGG). High-grade gliomas have a higher mortality rate, while low-grade gliomas develop more slowly. Therefore, early diagnosis and timely treatment of low-grade brain tumors are of great significance in increasing patients' chances of survival, improving their quality of life, and prolonging their lives.
[0003] Magnetic Resonance Imaging (MRI) uses the magnetic resonance phenomenon to acquire and reconstruct information about the human body. Compared to CT imaging, MRI can obtain higher contrast brain images. As a non-invasive technique, MRI is harmless to the human body and not only provides complete, non-invasive images without cranial imaging, but also has excellent soft tissue contrast, displaying higher resolution tissue structures. It is widely used in clinical medical diagnosis and treatment.
[0004] With the development of deep learning and the improvement of related hardware, deep learning methods have been successfully applied to the field of medical imaging. Deep learning models have overcome the time-consuming and labor-intensive nature of manual segmentation. Currently, deep learning for brain tumor image segmentation mainly includes two methods: 2D convolutional networks and 3D convolutional networks. 2D networks mainly include FCN, U-Net, and U-Net++. In the 2D method, the constructed 3D MRI volume is decomposed into many 2D slices. Each of these slices is passed to the segmentation model, which generates a segmentation for each slice. Then, the 2D slices are recombined to form the segmented 3D volume. The disadvantage of this method is that important contextual information in the 3D image is lost.
[0005] In summary, providing a MRI image segmentation method based on 3D convolutional networks is a problem that urgently needs to be solved. Summary of the Invention
[0006] In view of this, the present invention discloses a lightweight brain tumor segmentation method and system based on V-Net to obtain a network model with high training accuracy and fast training speed.
[0007] The technical solution of this invention: A lightweight brain tumor segmentation method based on V-Net, comprising:
[0008] Step 1: Image preprocessing to obtain an image that conforms to the network structure;
[0009] Step 2: Construct the V-Net network; the V-Net network consists of an encoder and a decoder;
[0010] The encoder is used to extract features from the original input image, and the decoder is used to restore the extracted features to the segmentation result.
[0011] The encoder consists of multiple DownTransition modules, and the decoder consists of multiple UpTransition modules; the encoder and decoder are connected via jump connections.
[0012] The DownTransition module is used to gradually reduce the size of the feature map and gradually increase the number of feature channels, so that the encoder can capture information at different levels.
[0013] The UpTransition module is used to gradually restore the size of the feature map to the same as the input image and reduce the number of feature channels;
[0014] The skip connection is used to connect the feature map of the encoder with the feature map of the corresponding layer of the decoder, so that the network can transmit information better.
[0015] Step 3: Improve the V-Net network;
[0016] Change batch normalization to group normalization in the V-Net network; replace ordinary convolutions in the network with depthwise separable convolutions;
[0017] Add a Squeeze-and-ExcitationAttention mechanism to the encoder part of the network;
[0018] Step 4: Train the improved network using the hybrid loss function BCEDice Loss; select the best network model based on the training results;
[0019] Step 5: Input the preprocessed image data from Step 1 into the optimal network model to obtain the segmentation result;
[0020] Step 6: Post-process the segmentation results.
[0021] Specifically, step 1 image preprocessing includes: standardizing, cropping and segmenting the brain tumor MRI dataset in sequence.
[0022] Specifically, the hybrid loss function BCEDice Loss combines the binary cross-entropy loss and the Dice coefficient loss, and calculates the final loss through a linear combination.
[0023] The formula for calculating the binary cross-entropy loss is as follows:
[0024]
[0025] Where N represents the number of samples, y pred (i) y is the probability predicted by the model that sample i belongs to a positive example. true (i) is the actual label of the sample, with a value of 0 or 1; log represents the natural logarithm.
[0026] The formula for calculating the Dice coefficient loss function is as follows:
[0027]
[0028] Where N represents the number of samples, y pred (i) y is the probability predicted by the model that sample i belongs to a positive example. true (i) This is the actual label of the sample, with a value of 0 or 1;
[0029] Therefore, the formula for calculating the hybrid loss function BCEDice Loss is:
[0030]
[0031] Both α and β take values of 0.5.
[0032] Specifically, the post-processing in step 5 includes noise removal, hole filling, and smoothing of segmentation boundaries.
[0033] This invention also provides a lightweight brain tumor segmentation system based on V-Net, comprising:
[0034] Image preprocessing module: used to preprocess the image dataset to be segmented to obtain images that conform to the network structure;
[0035] Network building block: Used to build V-Net networks;
[0036] V-Net Network Improvement Module: Used to reduce network parameters and improve network training speed while maintaining high accuracy;
[0037] Network training module: used to train the improved network using the hybrid loss function BCEDice Loss;
[0038] Post-processing module: Used for further processing of the segmentation results.
[0039] This invention proposes a lightweight brain tumor segmentation method and system based on V-Net. This method greatly reduces the amount of computation while accelerating the training speed and shortening the training time, while maintaining high training accuracy and exhibiting good segmentation performance. It has a very positive significance for clinicians' diagnosis and patients' treatment.
[0040] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit the disclosure of the present invention. Attached Figure Description
[0041] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0043] Figure 1 These are four modal diagrams of BraTs provided in the embodiments of this invention;
[0044] Figure 2 This is a preprocessed data block diagram provided in the embodiments of the present invention;
[0045] Figure 3 This is a structural diagram of batch normalization and group normalization provided in the embodiments of the present invention;
[0046] Figure 4 This is a diagram of a typical convolution process provided in an embodiment of the present invention;
[0047] Figure 5 This is a diagram of the depth-separable convolution process provided in the embodiments of the present invention;
[0048] Figure 6 This is a flowchart of the SE attention mechanism algorithm provided in the embodiments of the present invention;
[0049] Figure 7This is a network structure diagram provided in the embodiments disclosed in this invention;
[0050] Figure 8 This is a comparison diagram of the segmentation effect provided by the embodiments disclosed in this invention. Detailed Implementation
[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of systems consistent with some aspects of the invention as detailed in the appended claims.
[0052] 3D convolutional network methods mainly include 3D U-Net and V-Net. Ideally, in 3D methods, the entire MRI volume is input into the segmentation model to obtain a 3D segmentation result of the entire MRI. However, due to hardware and other resource limitations, the MRI image is usually cut into several blocks, which are then fed into the segmentation model and finally aggregated to form a segmentation map of the entire volume. This method can capture certain contextual information.
[0053] This implementation scheme first provides a lightweight brain tumor segmentation method based on V-Net, including:
[0054] Step 1: Image preprocessing to obtain an image that conforms to the network structure;
[0055] Specifically, brain tumor MRI data were preprocessed by standardizing, cropping, and segmenting the BraTS2018 dataset. This ensured that the processed images met the requirements of the network structure.
[0056] In this implementation scheme: The experiments used the BraTs2018 and BraTs2019 datasets. The training and validation sets for this invention's experiments came from the BraTs2018 dataset. The training set contained 210 high-grade glioma patient samples and 75 low-grade glioma patient samples, while the validation set contained 66 unlabeled patient samples. Each sample in the training set contained brain MRI images of four different modalities (T1, T2, T1ce, Flair) and corresponding ground truth labeled images. However, since BraTs only publishes the training set data and not the test set data, splitting a portion of the training set data as the test set would lead to overfitting due to insufficient training data, resulting in poor network generalization ability. To address the issue of insufficient data, we selected the portion of the BraTs2019 training set that was larger than that in the BraTs2018 training set as the test set. This portion included 49 high-grade glioma patient samples and 1 low-grade glioma patient sample.
[0057] The preprocessing steps mainly include:
[0058] (1) Manually add 5 black slices. Add 3 black slices in front of the four modal images (155,240,240) and the corresponding Mask (155,240,240), and add 2 black slices behind them. Change the image size to (160,240,240).
[0059] (2) Standardization. BraTS used four MR images of the T1, T2, flair and T1ce sequences. These four sequences are images of different modalities, so the image contrast is also different. Therefore, the z-score method is used to standardize each modal image separately, subtracting the mean and dividing by the standard deviation.
[0060] (3) Cropping. In BraTS MR images, the gray areas represent the brain region, while the black areas represent the background. The background information constitutes a large proportion of the entire image, and it does not contribute to segmentation. From a doctor's perspective, MR images automatically filter out this background information, focusing attention on the brain region. Therefore, removing the background information around the brain region is necessary. Cropping also reduces the size of the network parameters, improving network performance. Here, we cropped the original image (160, 240, 240) to (160, 160, 160).
[0061] (4) Block processing: Due to the limitations of memory and graphics card resources, the image cannot be completely input into the network, so it is necessary to process the image and the corresponding Mask into blocks. The size of the cropped image and Mask is (160, 160, 160). In the experiment of this invention, it was divided into 5 blocks of size (32, 160, 160) along the Axial direction.
[0062] (5) Merge and save the data. Merge the four normalized and segmented modalities into four channels, and save the shape as (32, 160, 160, 4) with dtype as float64. Then, for the corresponding Mask, which is also segmented, merge the three labels into three nested sub-regions, and finally merge them into three channels, namely WT, TC, and ET, with values of 0 or 1, and save the shape as (32, 160, 160, 3) with dtype as uint8.
[0063] Step 2: Construct the V-Net network; V-Net consists of an encoder and a decoder, whose functions are to extract features from the original input image and to restore the extracted feature maps to the segmentation results, respectively. V-Net uses 3D convolution to process 3D volumetric data. In addition, V-Net introduces skip connections, which connect the feature maps of the encoder to the corresponding layer feature maps of the decoder, enabling the network to better preserve and transmit detailed information. The encoder part of V-Net consists of multiple DownTransition modules. Each DownTransition module includes a downsampling operation (usually using a 3D convolution with a stride of 2), a batch normalization, a ReLU activation, and one or more convolution operations (usually 5×5×5 convolution kernels), as well as an optional Dropout layer. The role of DownTransition is to gradually reduce the size of the feature map and gradually increase the number of feature channels in order to capture information at different levels in the encoder. The V-Net decoder consists of multiple UpTransition modules. Each UpTransition module includes an upsampling operation (typically using a transposed convolution with a stride of 2 or bilinear interpolation), a normalization process, a ReLU activation function, and one or more convolution operations. The decoder's role is to progressively restore the feature map to the same size as the input image and reduce the number of feature channels. V-Net also introduces skip connections between the encoder and decoder. By connecting the encoder's feature maps to the feature maps of corresponding layers in the decoder, the network can better transmit information. These skip connections allow the network to access low-level feature information from the encoder during decoding, thus helping the network to better restore details and boundaries.
[0064] Step 3: Improve the V-Net network; replace batch normalization (BN) with group normalization (GN). Traditional V-Net networks use batch normalization (BN) to process images. BN calculates the N, H, and W values for each channel individually, which speeds up training and enhances the network's generalization ability. However, the accuracy of BN is significantly affected by the batch size N. Smaller batch sizes result in less representative mean and variance, leading to a higher error rate. Increasing the batch size places higher demands on computer memory. Since medical images typically have large datasets, batch sizes are generally small. In this case, the calculation of group normalization (GN) is unaffected by batch size, and its accuracy remains stable regardless of the batch size. Therefore, group normalization (GN) is more suitable for medical image segmentation experiments. Thus, in this invention, the batch normalization in the V-Net network is modified to be more suitable for this experiment using group normalization (GN). The basic idea of GN is as follows: first divide the features extracted from a certain layer into G groups, then normalize the features in each group, and finally merge the normalized data of the G groups.
[0065] By replacing ordinary convolutions in the network with depthwise separable convolutions and adding a Squeeze-and-Excitation (SE) attention mechanism to the encoder part of the network, the network can maintain high accuracy while reducing a large number of parameters and improving training speed.
[0066] Depthwise separable convolution splits a single convolutional kernel into two independent kernels, primarily involving two steps: depthwise convolution and pointwise convolution. Depthwise convolution performs convolution on each channel of the input feature map separately. Pointwise convolution uses a 1×1 kernel to perform convolution on all channels. As shown in the figure: Assume the size of the input feature map is D. K ×D K ×M, the kernel size is D F ×D F The computational complexity of convolving the feature map with the kernel is D ×M×N. F ×D F ×M×N×D K ×D K The kernel size of depthwise convolution is D. F ×D F The computational cost of performing a convolution operation between the feature map and the depthwise convolution is D × 1 × M. F ×D F ×M×D K ×D K The kernel size of pointwise convolution is 1×1×M×N, and the computational cost of convolving the feature map with pointwise convolution is M×N×D. K ×D K By implementing this through these two steps, its computational complexity can be expressed as D. F ×D F ×M×D K ×D K +M×N×D K ×D K The computational cost of ordinary convolution is D. F ×D F ×M×N×D K ×D K Compared to the former, depthwise separable convolution significantly reduces computational cost. Therefore, replacing ordinary convolution with depthwise separable convolution in convolutional units can greatly improve the computational speed of network models.
[0067] A Squeeze-and-Excitation (SE) Attention mechanism is added after the convolution of each layer in the encoder part of the V-Net network.
[0068] Depthwise separable convolutions reduce the number of parameters and computational cost to some extent, but they can sometimes limit the network's feature representation capabilities, potentially failing to fully capture the complex features of the data and thus affecting model performance. SEAttention, on the other hand, dynamically adjusts the importance of features between channels, allowing the network to focus more on important features and thus enhancing feature representation. Furthermore, SEAttention helps the network better capture key information in the image, thereby improving model performance. The basic idea of SEAttention is to learn the weights of each channel through global pooling, which mainly includes two steps: Squeeze and Excitation. Specifically, Ftr, given an input feature map X, generates a feature map U through the Ftr operation. In the Squeeze(Fsq(·)) step, the feature map is subjected to full-set average pooling, generating a 1×1×C vector, with each channel represented by a numerical value. The Excitation(Fsq) step is completed through two fully connected layers, generating the required weight information using weights W, which are learned and used to indicate the relevance of the features we need to model. In Fscale, the weight vector S generated in the third step is applied to the feature map U to obtain the desired feature map X', whose size is exactly the same as feature map U. The SE module does not change the size of the feature map. By combining the Squeeze and Excitation steps, SE Attention rescales the feature map of each channel using the learned weights. Channels with higher weights receive more attention, while channels with lower weights are suppressed. In this way, SE Attention helps the network focus on features meaningful to the task, thereby improving network performance. Moreover, SE has a small number of parameters, which does not affect the lightweight performance of the network.
[0069] Step 4: Train the improved network using the hybrid loss function BCEDice Loss;
[0070] Brain tumor segmentation is a highly challenging task, primarily due to the fact that it typically involves multiple classes with significantly different distributions in the image, leading to class imbalance. This imbalance causes models to tend to predict larger classes while neglecting smaller ones, impacting segmentation accuracy. Furthermore, the shape and size of brain tumors vary greatly among different patients and cases, making it difficult for models to capture all shape details and boundaries, especially for smaller or poorly defined tumors. BCEDice Loss, which combines binary cross-entropy loss and Dice coefficient loss through a linear combination, comprehensively considers pixel-level classification accuracy (via binary cross-entropy) and the overlap of segmentation results (via Dice coefficient). In class-imbalanced brain tumor segmentation, BCEDice Loss better balances these two losses, thereby improving model performance.
[0071] The binary cross-entropy loss function is a commonly used loss function for binary classification tasks, measuring the difference between the predicted probability and the true label. In brain tumor segmentation tasks, pixel-level segmentation is a binary classification problem, meaning each pixel belongs to either the tumor region or a non-tumor region. Therefore, using binary cross-entropy loss can help the network learn the correct pixel classification. Its calculation formula is as follows:
[0072]
[0073] Where N represents the number of samples, y pred (i) y is the probability predicted by the model that sample i belongs to a positive example. true (i) This is the actual label of the sample, with a value of 0 or 1. log represents the natural logarithm.
[0074] The Dice coefficient loss function: The Dice coefficient is a metric for evaluating the similarity between two sets, measuring the degree of overlap between the predicted segmentation and the actual segmentation. In brain tumor segmentation tasks, the Dice coefficient is widely used to measure segmentation accuracy, especially effective for imbalanced class tasks. Its formula is as follows:
[0075]
[0076] Where N represents the number of samples, y pred (i) y is the probability predicted by the model that sample i belongs to a positive example. true (i) This is the actual label of the sample, with a value of 0 or 1.
[0077] The final loss function is:
[0078]
[0079] In this invention, both α and β are 0.5.
[0080] Post-processing of the segmentation results:
[0081] In brain tumor segmentation tasks, VNet networks typically undergo post-processing to further optimize the segmentation results. The goal of post-processing is to perform operations on the segmentation mask output by the model, including noise removal, hole filling, and smoothing segmentation boundaries, thereby obtaining more accurate segmentation results.
[0082] Based on the training results above, the best network model was selected. The following table compares the segmentation performance of various algorithms on the BraTS dataset:
[0083]
[0084] This invention randomly splits the training set into five equal parts and performs five-fold cross-validation, as shown in Table 1. The average Dice values of the algorithm in this invention reach 90.10%, 90.10%, and 89.42% in the intact tumor region, the enhanced tumor region, and the tumor core region, respectively. NVDLMED in the table is the first-place algorithm in the BraTS2018 competition. As can be seen from the table, the algorithm of this invention reduces the computational cost and parameter count by 12.5 times and 32.1 times respectively compared to V-Net, and by 21.8 times and 28.6 times respectively compared to NVDLMED, with significant reductions in both parameter count and computational cost. However, in terms of computational accuracy, the algorithm of this invention is slightly lower than V-Net by 1.89% in the intact tumor region (WT), but higher than V-Net by 1.71% and 0.52% in the enhanced tumor region (ET) and the tumor core region (TC), respectively, and higher than NVDLMED by 8.72% and 3.46% respectively. In summary, the algorithm of this invention can maintain high training accuracy while greatly reducing the amount of computation, speeding up training, and shortening training time. It has good segmentation performance and is of great significance for clinicians' diagnosis and patient treatment.
[0085] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these changes and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A lightweight brain tumor segmentation method based on V-Net, comprising the following steps: Step 1: Image preprocessing to obtain an image that conforms to the network structure; Step 2: Construct the V-Net network; The V-Net network consists of an encoder and a decoder; The encoder is used to extract features from the original input image, and the decoder is used to restore the extracted features to the segmentation result. The encoder consists of multiple DownTransition modules, and the decoder consists of multiple UpTransition modules; the feature maps are concatenated between the encoder and decoder through skip connections. The DownTransition module is used to gradually reduce the size of the feature map and gradually increase the number of feature channels, so that the encoder can capture information at different levels. The UpTransition module is used to gradually restore the size of the feature map to the same as the input image and reduce the number of feature channels; The skip connection is used to connect the feature map of the encoder with the feature map of the corresponding layer of the decoder, so that the network can transmit information better. Step 3: Improve the V-Net network; Change batch normalization to group normalization in the V-Net network; replace ordinary convolutions in the network with depthwise separable convolutions; Add a Squeeze-and-ExcitationAttention mechanism to the encoder part of the network; Step 4: Train the improved network using the hybrid loss function BCEDice Loss; select the best network model based on the training results; Step 5: Input the preprocessed image data from Step 1 into the optimal network model to obtain the segmentation result; Step 6: Post-process the segmentation results.
2. The lightweight brain tumor segmentation method based on V-Net according to claim 1, characterized in that, The image preprocessing in step 1 includes: standardizing, cropping, and segmenting the brain tumor MRI dataset sequentially.
3. The lightweight brain tumor segmentation method based on V-Net according to claim 1, characterized in that, The hybrid loss function BCEDice Loss combines the binary cross-entropy loss and the Dice coefficient loss, and calculates the final loss through a linear combination. The formula for calculating the binary cross-entropy loss is as follows: Where N represents the number of samples, y pred (i) y is the probability predicted by the model that sample i belongs to a positive example. true (i) is the actual label of the sample, with a value of 0 or 1; log represents the natural logarithm. The formula for calculating the Dice coefficient loss function is as follows: Where N represents the number of samples, y pred (i) y is the probability predicted by the model that sample i belongs to a positive example. true (i) This is the actual label of the sample, with a value of 0 or 1; Therefore, the formula for calculating the hybrid loss function BCEDice Loss is: Both α and β take values of 0.
5.
4. The lightweight brain tumor segmentation method based on V-Net according to claim 1, characterized in that, The post-processing in step 5 includes noise removal, hole filling, and smoothing of segmentation boundaries.
5. A lightweight brain tumor segmentation system based on V-Net, characterized in that, include Image preprocessing module: used to preprocess the image dataset to be segmented to obtain images that conform to the network structure; A network construction module is used to construct a V-Net network; the V-Net network consists of an encoder and a decoder; the encoder is used to extract features from the original input image, and the decoder is used to restore the extracted features to the segmentation result; wherein, the encoder consists of multiple DownTransition modules, and the decoder consists of multiple UpTransition modules; the feature maps are concatenated between the encoder and the decoder through skip connections; The DownTransition module is used to gradually reduce the size of the feature map and gradually increase the number of feature channels, enabling the encoder to capture information at different levels; the UpTransition module is used to gradually restore the size of the feature map to the same as the input image and reduce the number of feature channels; the skip connection is used to connect the feature map of the encoder with the feature map of the corresponding layer of the decoder, enabling the network to better transmit information; The network improvement module is used to improve the V-Net network by changing batch normalization to group normalization; replacing ordinary convolutions in the V-Net network with depthwise separable convolutions; and adding a Squeeze-and-Excitation Attention mechanism to the encoder part of the V-Net network. The network training module is used to train the improved network using the hybrid loss function BCEDice Loss and select the best network model based on the training results. The segmentation module is used to input the image data preprocessed by the image preprocessing module into the optimal network model to obtain the segmentation result; The post-processing module is used to perform post-processing on the segmentation results output by the segmentation module.
Citation Information
Patent Citations
Lightweight brain tumor segmentation method based on depth separable convolution
CN112686905A
Breast mass segmentation method based on multi-modal feature fusion Vnet
CN114549558A
