A brain tumor MRI segmentation method based on cross-stage local network structure

Abstract

The application discloses a brain tumor MRI segmentation method based on a cross-stage local network structure, and steps are as follows: step 1, 3D image data set is acquired; step 2, 2D image set of effective pixels in the 3D image is obtained; step 3, an optimized U-Net model is constructed, and the optimization process is specifically as follows: the first convolutional layer of each image compression path block in the image compression path in the U-Net model is a cross-stage local network structure, and / or the second convolutional layer of each image expansion path block in the image expansion path is a cross-stage local network structure; a loss function of the U-Net model is obtained by weighted summation of a generalized dice loss function and a weighted cross-entropy loss function; step 4, the U-Net model is trained through the effective pixels of the image; and step 5, real-time image segmentation is carried out by using the trained U-Net model. The application can effectively reduce the possibility of replication in the information integration process, reduce the information loss in the up-sampling, and can be used for small-scale tumor segmentation.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to a brain tumor MRI segmentation method based on a cross-stage local network structure. Background Technology

[0002] Brain tumors are abnormal cells that grow in the brain or skull. Medically, they are classified into benign and malignant brain tumors based on their growth characteristics and impact on the body, with the latter having a higher incidence and greater harm. Because gliomas of different differentiation grades are often accompanied by varying degrees, types, and degrees of peritumoral edema, and their boundaries with normal tissue are often blurred, surgeons struggle to determine the extent of the lesion and the surgical area, severely impacting surgical quality and patient prognosis. Therefore, confirming the tumor location and identifying the entire tumor based on imaging data before surgery helps reduce unnecessary tissue damage during surgery and maximizes the protection of neurological function. Since the magnetic field signal of brain tumor tissue is almost identical to that of normal human brain tissue, and contrast-enhanced MRI can clearly distinguish malignant tumors, contrast-enhanced MRI is currently one of the main methods for examining brain tumor patients. It effectively provides information about the shape, size, and location of the brain tumor, which helps improve clinicians' diagnostic capabilities for brain tumors. The ultimate goal of brain tumor image analysis is to extract important clinical information and diagnostic features from patients and embed them into multidimensional image data. This enables relevant guidance and monitoring after disease detection, followed by the implementation of interventions, ultimately leading to corresponding clinical diagnoses, staging, and treatment methods, which has significant practical implications. Due to the numerous advantages of using MRI images in brain tumor diagnosis, computer-aided diagnosis and treatment of brain tumors has become one of the most popular research topics in the field of medical imaging.

[0003] In recent years, due to the powerful automatic feature extraction capabilities of deep neural networks in supervised learning, their application in brain tumor segmentation has become a research hotspot in computer vision tasks. Over the past few years, various deep neural network models for computer vision tasks have been proposed, such as VGGNet, GoogLeNet, ResNet, and DenseNet. These deep neural network models have provided new ideas for solving the problem of MRI brain image segmentation, greatly promoting the development of deep learning-based brain tumor diagnosis methods. Deep learning-based image segmentation methods are mainly divided into two types: convolutional neural networks (CNNs) and fully convolutional neural networks (FCNs). Brain tumor segmentation methods based on CNNs mainly employ small-scale image patch classification methods to design MRI brain tumor segmentation networks that incorporate single-path and multi-path approaches. Compared to single-path CNNs, multi-path CNNs can acquire and summarize various information from different paths. Havaei et al. proposed a novel CNN architecture, constructing a multi-path network and using convolutional kernels of different sizes to extract important contextual features for analyzing brain tumor-related information in 2D MRI images. Zhao et al. used a similar convolutional neural network (CNN) method to automatically extract lesions from 2D MRI glioma images, overcoming the drawbacks of manual processing such as time consumption and poor repeatability. Moeskops et al. used three convolutional paths with different receptive fields to obtain more detailed features and analyze spatial consistency for brain tumor segmentation. However, since the CNN architecture is based on small image patches, it requires a large amount of computation and lacks spatial continuity for 3D region slice training, resulting in low efficiency and unsatisfactory segmentation results. Fully convolutional neural networks (WCNNs), based on the encoder-decoder concept and performing pixel-by-pixel computation, not only improve image segmentation efficiency but also solve the spatial continuity problem, making it one of the important methods for image semantic segmentation tasks. Ronneberger et al. proposed a U-Net architecture suitable for various medical image segmentation problems. This architecture is based on the concept of encoder and decoder paths, using the encoder path to extract contextual features and the decoder path to ensure accurate location tracking. The lateral connections between the two are used to reduce information loss during downsampling. Because U-Net is very suitable for multimodal medical image segmentation tasks with limited data, it has become the mainstream algorithm in the field of brain tumor segmentation. However, the U-Net model suffers from significant information loss due to continuous convolution and nonlinear transformations during upsampling and downsampling, resulting in a need to improve segmentation accuracy. Furthermore, the computational cost of 3D networks is too high, making widespread adoption difficult. Therefore, further research is needed on lightweight segmentation networks. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a brain tumor MRI segmentation method based on a cross-stage local network structure, which solves the technical problems of limited information expression capacity, significant information loss, image category imbalance, and low segmentation accuracy in existing technologies.

[0005] This invention provides a brain tumor MRI segmentation method based on cross-stage local network structure, the steps of which are as follows:

[0006] Step 1: Obtain the 3D image dataset;

[0007] Step 2: Preprocess the 3D image dataset to obtain a set of 2D images containing the effective pixels of the 3D images;

[0008] Step 3: Construct the optimized U-Net model. The optimization process is as follows:

[0009] In the U-Net model, the first convolutional layer of each image compression path block is a cross-stage local network structure, and / or...

[0010] In the U-Net model, the second convolutional layer of each image expansion path block in the image expansion path is a cross-stage local network structure;

[0011] The loss function of the U-Net model is obtained by weighted summation of the generalized dice loss function and the weighted cross-entropy loss function;

[0012] Step 4: Train the optimized U-Net model using the effective pixels of the image;

[0013] Step 5: Use the trained U-Net model for real-time image segmentation.

[0014] Furthermore, step 2 includes the following process:

[0015] Step 21: Normalize the 3D image dataset to obtain the effective pixels of the 3D image;

[0016] Step 22: Crop the effective pixels of the 3D image to obtain several 2D images with a pixel size of 128×128.

[0017] Furthermore, the model strategy for the cross-stage local network structure is: a strategy of direct output after the transition.

[0018] Furthermore, the model strategy for the cross-stage local network structure is: a strategy of transitioning first, then concatenating and then adding another transition.

[0019] Furthermore, in step 3, the loss function of the U-Net model is:

[0020] L = LGDL +L WCE

[0021] in,

[0022] In the formula, L WCE L represents the weighted cross-entropy loss function. GDL Let Q represent the generalized dice loss function, c represent the label index, and ω represent the total number of labels. c f represents the weight assigned to the c-th label. c The pixel values ​​of the segmented two-dimensional image, l c It is the actual value of the pixels in a two-dimensional image.

[0023] Furthermore, in step 4, when training the optimized U-Net model, a 5-fold cross-validation method is used to perform gold standard verification on the segmentation results of the optimized U-Net model on the training set.

[0024] The beneficial effects of this invention are:

[0025] (1) To address the significant information loss caused by continuous convolution in upsampling and downsampling of traditional U-shaped networks, a cross-stage local network structure is used in upsampling and downsampling of U-shaped networks to maximize the difference in gradient combination. Through cross-stage splitting and merging strategies, the possibility of duplication during information integration can be effectively reduced, ensuring that gradient flow is not reused, thereby reducing information loss in upsampling, enhancing the feature extraction capability of the model, and effectively solving the problem of difficulty in small-scale tumor segmentation of traditional U-shaped networks.

[0026] (2) To address the class imbalance problem in brain tumor segmentation tasks, this invention combines two loss functions, GDL and WCE. The GDL loss function is responsible for predicting the segmentation region, while the WCE loss function is used to classify tissue cells, thereby reducing the impact of the huge difference in the occupancy rate of different categories of brain tumor MRI on the segmentation accuracy.

[0027] (3) To address the difficulty of training on small datasets of brain tumors, this invention employs a 5-fold cross-validation method to divide the dataset and perform gold standard validation during the training process. This avoids the limitations and specificities of fixed dataset division and effectively reduces the risk of model overfitting. Attached Figure Description

[0028] The features and advantages of the invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the invention in any way. In the drawings:

[0029] Figure 1 The sample image of the dataset selected in the specific embodiment of the present invention;

[0030] Figure 2 This is a diagram of the convolutional neural network structure incorporated into a specific embodiment of the present invention.

[0031] Figure 3 This is a cross-stage local network structure diagram in a specific embodiment of the present invention;

[0032] Figure 4 This is a comparison chart of the segmentation results of various models on a brain tumor MRI dataset. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] The present invention will be further illustrated below with reference to specific embodiments. Those skilled in the art should understand that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Modifications to the present invention in various equivalent forms all fall within the scope defined by the appended claims.

[0035] This invention provides a brain tumor MRI segmentation method based on cross-stage local network structures, comprising the following steps:

[0036] Step 1: Dataset Preparation: The datasets used in this invention are BraTS 2018 and BraTS 2019. There are four categories of data labels: tumor enhancement area, edema area, necrotic area, and healthy tissue. Segmentation results are evaluated using three metrics: overall (WT), core (TC), and enhancement (ET). WT includes tumor enhancement area, edema area, and necrotic area; TC includes tumor enhancement area and necrotic area; and ET includes tumor enhancement area.

[0037] Specifically:

[0038] Step 11: Select Training and Validation Datasets: The training and validation datasets are derived from BraTS 2018 and BraTS 2019. The BraTS 2018 dataset is a collection of brain tumor images from 285 glioma patients, with the training set containing images of 210 patients with high-grade gliomas (HGG) and 75 patients with low-grade gliomas (LGG). In addition, the validation dataset includes images of 66 patients with unknown grades. BraTS 2019 has a training set containing 335 glioma cases, of which 259 are high-grade gliomas (HGG) and the remaining 76 are low-grade gliomas (LGG). The validation set for BraTS 2019 has also been expanded to include 125 cases with unknown grades. Furthermore, each image is 240×240×155 pixels, and each case in the training set includes four image modalities: Flair, T1, T1ce, T2, and expert-provided manual segmentation results.

[0039] like Figure 1 As shown, each patient's data consists of five images, from left to right: four MRI modes (Flair, T1, T1CE, T2) and real labels manually segmented by experts for comparison.

[0040] Step 12: Validation Result Return: The validation set provided in BraTS 2018 and BraTS 2019 does not provide expert manual segmentation results as the true labels. Therefore, the model trained in the training set is used to segment the validation set, and the segmentation results are then uploaded to the BraTS official website for evaluation by the official team, who will return the validation set segmentation accuracy.

[0041] Step 2: Data Preprocessing Method Design: This invention uses multimodal 3D MRI brain tumor images, with each raw 3D MRI image data being 240×240×155 pixels. Since the images contain valid information and much useless background, the raw data is first normalized, extreme data is eliminated, regularized, and denoised. Subsequently, the processed 3D images are sliced ​​into 2D images, and 128×128 effective pixels are extracted as model input.

[0042] Specifically:

[0043] Step 21: Normalization: The normalization formula is expressed as:

[0044]

[0045] Where x is the input 3D image, x p It is the processed, normalized image. σ is the average value of the input image, and σ is the standard deviation of the input image.

[0046] Step 22: Data Cropping: In brain tumor MRI, healthy tissue occupies an average area of ​​98.46%, while edema, enhanced tumors, and non-enhanced tumors occupy 1.02%, 0.29%, and 0.23%, respectively. Therefore, the 3D image is first cropped into 3D image blocks according to an appropriate ratio to eliminate irrelevant background and reduce computational load. Since this invention uses a 2D convolutional neural network framework, each 3D image block is sliced ​​into 2D images, and 128×128 effective pixels are extracted as model input.

[0047] Step 3: Cross-stage local network structure design: This invention applies cross-stage local modules to the encoder and decoder of the U-Net model respectively, and adopts a combined loss function, combining the generalized dice loss and weighted cross-entropy loss functions to solve the class imbalance problem of MRI brain tumor images. Finally, it is integrated into an optimized U-Net model, namely the CSPU-Net model, and the Dice coefficient is used as the evaluation index of the model.

[0048] Specifically:

[0049] Step 31: Model Structure Design: CSPU-Net maintains the traditional encoder-decoder architecture, which consists of an image compression path (encoder) and an image expansion path (decoder). The encoder and decoder structures each have three blocks. Each block on the encoder side consists of two convolutional layers. The first convolutional layer uses a Cross-Stage Local Network (CSP) module to replace the traditional 3×3 convolutional structure, and the second convolutional layer replaces the max-pooling operation in the traditional U-Net downsampling to reduce information loss, avoid gradient vanishing, and accelerate network fusion. Each block on the decoder side consists of upsampling plus convolution and a CSP module. In the upsampling layer, the image size is doubled using bilinear interpolation with a stride of 2, and then the number of channels is halved by a convolutional layer with half the number of kernels, completing the upsampling process. Simultaneously, to increase nonlinearity and model convergence speed, batch normalization and nonlinear activation function operations are performed after each convolutional layer of the model. In the last decoder block of the model, an additional PReLU activation function and a 1×1 convolution using the SoftMax activation function are added to map the multi-channel features to the corresponding categories. The formulas for the PReLU and SoftMax activation functions are as follows.

[0050]

[0051]

[0052] In formula (2), i represents different channels, and a iThis is a parameter that needs to be learned, and its value is updated using a dynamic update method. For the i-th channel, a i The values ​​are the same.

[0053] Figure 2 This is the overall architecture of CSPU-Net, where the left side represents the image compression path and the right side represents the image expansion path.

[0054] Step 32: Cross-Stage Local Network Module Design: The main purpose of the CSP module is to divide the feature mapping of the base layer into two parts, split the gradient flow so that the gradient flow propagates through different network paths, and then merge the two parts through a horizontal hierarchical structure. By alternating concatenation and transition operations, the propagated gradient information can have significant correlation differences. This invention uses two fusion methods for the CSP module: one with transition followed by concatenation and then transition, and the other with concatenation after transition and direct output.

[0055] This module first performs a 3×3 convolution operation on the input data and then splits the network structure into two paths. Path A is directly concatenated with the output of path B, forming a structure similar to a residual network. Path B first passes through a 3×3 convolutional layer with half the number of kernels, thus halving the number of channels and reducing the number of parameters. It then splits into two paths again, concatenating the output of the 3×3 convolutional layer with the input of the previous layer. The overall idea of ​​this method is to maximize the difference in gradient combinations. Through cross-stage splitting and merging strategies, it can effectively reduce the possibility of duplication during information integration. The CSP module calculation formula is as follows.

[0056] p(A)=BN(F(x,ω i (4)

[0057] p(B)=T([F(p(A),ω) i t D, F(F(p(AD, ω) i i ), ω i i (5)

[0058] csp(x)=BN([p(A),p(B)]) (6)

[0059] In the formula, x is the input image, and F(x, ω) i This indicates a 3×3 convolution operation on the input image, with ω kernels. i ω i i Represents ω iHalf of the convolutional kernels are used, T represents the transition layer operation, and [] represents the concatenation operation. p(A) represents the output of path A, p(B) represents the output of path B, and csp(x) represents the final output of this module. Each 3×3 convolutional layer in the model is followed by a BN layer and a LeakyReLU activation function to address the aliasing problem in the gradient direction during backpropagation. The BN layer formula is as follows:

[0060]

[0061] In the formula, x i For [x i (1) x i (2) , ..., x i (m) The row vector is formed by ], where m represents the number of samples fed into the training each time, and x i (b) This represents the value of the i-th input node in this layer when the b-th sample is input in the batch. μ(x) i ) and σ(x i ) 2 ε represents the mean and variance of the row vector, respectively. ε is used to prevent the local minima introduced by division by zero, and γ and β are the scale and shift parameters to be learned.

[0062] Figure 3 This is the cross-stage local network structure used in this invention, where Con represents a concatenation operation, which involves splicing together the features extracted by multiple convolutional feature extraction frameworks along the channel dimension.

[0063] Step 33: Loss Function Design: To address the class imbalance problem, this invention selects two loss functions: Generalized loss and Weighted cross-entropy (WCE) to better evaluate model performance. The calculation formulas for the two loss functions are as follows:

[0064]

[0065]

[0066] L WCE L represents the weighted cross-entropy (WCE) loss function, a good solution to the multi-class imbalance problem, which can reduce the discrepancy between training samples and evaluation metrics, making the network prioritize the class with a smaller sample share. GDLrepresents the General Dice loss function, which is commonly used for segmenting small objects in medical images, allowing the model to focus on samples that are difficult to learn. The weights of each label are corrected by the reciprocal of its volume, thereby reducing the correlation between region size and Dice score. Where L... GDL L is responsible for predicting segmentation regions. WCE Used for classifying tissue cells. In the formula, Q represents the total number of labels (4 in the brain tumor segmentation dataset), c represents the label number, and ω... c This represents the weight assigned to the c-th label. c The pixel values ​​of the segmented two-dimensional image, l c These are the true values ​​of pixels in a two-dimensional image. The complete loss function L is expressed as:

[0067] L = L GDL +L WCE (10)

[0068] Step 3.4 Evaluation Metric Design: For the performance evaluation of the proposed model, this invention uses the Dice coefficient (DSC) as the evaluation metric for the segmentation network. The Dice coefficient is mainly used to measure the similarity between two sets, with a value between 0 and 1. Currently, the Dice coefficient is widely used to evaluate the performance of brain tumor segmentation tasks. The formula for calculating the Dice coefficient is as follows:

[0069]

[0070] In the formula, TP represents the total number of pixels correctly classified as brain tumors by the deep learning model, and FP represents the number of pixels misclassified. TN and FN represent the number of pixels correctly classified as healthy tissue and the number of pixels misclassified as healthy tissue, respectively.

[0071] Step 4: Model Training: During model training, the training set is randomly divided into 8:2 ratios using the 5-fold cross-validation method. Appropriate hyperparameters are set based on factors such as data volume, data characteristics, and hardware conditions, and necessary data is recorded.

[0072] Specifically:

[0073] Step 41: Experimental environment: The deep learning framework used in the model of this invention is Keras 2.2.4 with Tensorflow as the backend, and the hardware environment is a cluster with 32GB of memory and TESLAV100 GPU.

[0074] Step 42: Parameter Settings: The model uses stochastic gradient descent (SGD) as the optimizer, with an initial learning rate of 0.085, momentum of 0.95, and weight decay of 5e-6. The model input consists of four MRI modal images (Flair, T1, T1CE, T2) and manually segmented labeled images by experts. The batch size is set to 10, and the training run is 5 epochs.

[0075] Step 43: Import the dataset and fit it to the model: After performing the preprocessing operation in step 2 on the selected dataset, import the processed dataset into a U-shaped network that integrates cross-stage local network structures to complete the fitting of the target dataset, train and save the model, and evaluate the model using the 5-fold cross-validation method to avoid overfitting and improve segmentation accuracy.

[0076] Step 5: Model Validation: Use the trained CSPU-Net model to segment the validation set, visualize the segmentation results, and obtain the model's segmentation accuracy by comparing it with the gold standard data.

[0077] Figure 4 From left to right, the images are labeled images manually annotated by professional physicians, segmentation results of U-Net, ResU-Net, and CSPUNet proposed in this invention. It can be seen that compared with other methods, the segmentation results of the proposed method are closer to the real labels.

[0078] This invention introduces a cross-stage local network structure into the brain tumor segmentation task. Through a splitting and merging strategy, the number of gradient paths is doubled, mitigating the drawbacks of using explicit feature mapping replication for connections, thereby enhancing the model's feature learning ability and improving brain tumor segmentation accuracy. This invention uses the BraTs18 and 2019 datasets for training, and uses the trained model to predict on the validation set. Comparison with other 2D segmentation schemes shows a significant improvement in accuracy. The comparison results are shown in Table 1 (Evaluation Results of the BraTs2018 Validation Set) and Table 2 (Evaluation Results of the BraTs2019 Validation Set).

[0079]

[0080] Table 1

[0081]

[0082] Table 2.

[0083] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A brain tumor MRI segmentation method based on cross-stage local network structure, characterized in that, Includes the following steps: Step 1: Obtain the 3D image dataset; Step 2: Preprocess the 3D image dataset to obtain a set of 2D images containing the effective pixels of the 3D images; Step 3: Construct the optimized U-Net model. The optimization process is as follows: In the U-Net model, the first convolutional layer of each image compression path block is a cross-stage local network structure, and / or... In the U-Net model, the second convolutional layer of each image expansion path block in the image expansion path is a cross-stage local network structure; The loss function of the U-Net model is obtained by weighted summation of the generalized dice loss function and the weighted cross-entropy loss function; Step 4: Train the optimized U-Net model using the effective pixels of the image; Step 5: Use the trained U-Net model for real-time image segmentation.

2. The brain tumor MRI segmentation method based on cross-stage local network structure as described in claim 1, characterized in that, Step 2 includes the following process: Step 21: Normalize the 3D image dataset to obtain the effective pixels of the 3D image; Step 22: Crop the effective pixels of the 3D image to obtain several 2D images with a pixel size of 128×128.

3. The brain tumor MRI segmentation method based on cross-stage local network structure as described in claim 1, characterized in that, The model strategy for the cross-stage local network structure in step 3 is: a strategy of direct output after transition through concatenation.

4. The brain tumor MRI segmentation method based on cross-stage local network structure as described in claim 1, characterized in that, The model strategy for the cross-stage local network structure in step 3 is: a strategy of transitioning first, then concatenating and adding another transition.

5. The brain tumor MRI segmentation method based on cross-stage local network structure as described in any one of claims 1, 3, and 4, characterized in that, In step 3, the loss function of the U-Net model is: L=L GDL +L WCE in, In the formula, L WCE L represents the weighted cross-entropy loss function. GDL Let Q represent the generalized dice loss function, c represent the label index, and ω represent the total number of labels. c f represents the weight assigned to the c-th label. c The pixel values ​​of the segmented two-dimensional image, l c It is the actual value of the pixels in a two-dimensional image.

6. The brain tumor MRI segmentation method based on cross-stage local network structure as described in claim 1, characterized in that, In step 4, when training the optimized U-Net model, a 5-fold cross-validation method is used to perform gold standard verification on the segmentation results of the optimized U-Net model on the training set.

Images