Improved brain tumor image segmentation algorithm based on generative adversarial network model

Abstract

The invention discloses an improved brain tumor image segmentation algorithm based on a generative adversarial network model. The problem of inaccurate segmentation of tumor edges and detail regions is solved. According to the method, a generative adversarial network is improved, and cross-scale jump links are introduced for a generator part of a model so as to better capture context information and extract more complete edge detail features; in a decoder stage, position information and a channel relation are captured in combination with a coordinate attention mechanism, so that the network is focused on tumor features, and irrelevant features are inhibited at the same time; and the objective function combining L1 loss and Dice loss is used to solve the problem of sample category imbalance, and the accuracy and robustness of the brain tumor MR image are significantly improved. Experimental results show that the improved network has a better segmentation effect than other typical brain tumor segmentation methods.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and in particular to a method and system for automatic segmentation of brain tumor MRI images based on a generative adversarial network, which is suitable for the segmentation and diagnosis of multimodal brain tumor images. Background Art

[0002] Brain diseases often have high rates of disability and mortality, posing a serious threat to human health. Brain tumors are caused by abnormal cells that grow and reproduce irregularly within the brain, of which gliomas are the most severe. Glioma can directly damage the nervous system and trigger a series of complications, making it one of the most invasive tumors. Since non-invasive magnetic resonance imaging (MRI) is more efficient than computed tomography (CT) and positron emission tomography (PET), MRI medical images are widely used in image segmentation. Currently, brain tumor segmentation mainly relies on manual segmentation by radiologists, which is time-consuming and greatly influenced by personal experience. Traditional image segmentation methods (such as threshold segmentation and region growing) have limitations when processing brain tumor MRI images, especially for tumor regions with blurred edges and large grayscale differences. Therefore, developing an accurate and reliable automatic brain tumor segmentation method is of great clinical significance.

[0003] Early AI-based segmentation algorithms primarily relied on machine learning methods. Discriminative models such as support vector machines and random forests have made significant contributions to improving the accuracy of brain MRI image segmentation. However, these traditional machine learning models still require manual selection of image features, requiring high engineering and specialized knowledge. In recent years, deep learning-based image segmentation methods (such as convolutional neural networks (CNNs) and U-Net) have made significant progress in brain tumor segmentation.

[0004] However, existing methods still have the following problems:

[0005] 1. Edge information is lost during downsampling;

[0006] 2. Inaccurate positioning of the tumor area;

[0007] 3. Unbalanced category distribution leads to poor segmentation results.

[0008] To solve the above problems, this paper proposes an improved brain tumor image segmentation algorithm based on the generative adversarial network model. By introducing cross-scale links, coordinate attention mechanism and hybrid loss function, the segmentation accuracy and robustness are significantly improved. Summary of the Invention

[0009] The purpose of this invention is to provide an improved generative adversarial network based on cross-scale links and coordinate attention mechanism for automatic segmentation of brain tumor MRI images, to solve the problems of edge information loss, class imbalance and training instability in the prior art, and to improve segmentation accuracy and robustness.

[0010] To achieve the above object, the technical solution adopted by the present invention is:

[0011] An improved brain tumor image segmentation algorithm based on the generative adversarial network model has the following steps:

[0012] Step 1: Data preprocessing: Offset field correction, image normalization, and slice cropping are performed on brain tumor MR3D images to obtain slice images of four modalities. The grayscale images of the three modalities (Flair, T2, and T1ce) are then channel-concatenated. The preprocessed three-channel images can be used as the training and testing datasets.

[0013] Step 2: Build an improved generative adversarial network model, introduce a cross-scale link module and coordinate attention mechanism into the U-shaped generative network, and mask the predicted segmentation map and the target segmentation map with the original image respectively, and then input them into the discriminative network;

[0014] Step 3: Improve the loss function and use a hybrid loss function that combines L1 loss and Dice loss as the objective function for model training;

[0015] Step 4: Use the improved generative adversarial network model for training to obtain the segmentation results of brain tumor MR images.

[0016] 1. The specific process in step 1 is as follows:

[0017] 1) The 3D images of each modality are normalized using the z-score method. Before normalization, the 3DMR images are first corrected for offset field to make the image grayscale more uniform, and then normalized using the z-score method:

[0018]

[0019] Where x is the sample data, σ is the sample standard deviation, and μ is the sample mean;

[0020] 2) The background occupies a large proportion of the entire image. To remove the invalid area, the MRI image with a size of 240×240×155 was cropped to 200×160×155, and axial shearing was performed to obtain 2D slices of the four modalities, and then the image without foreground was removed;

[0021] 3) The slices of FLAIR, T1ce and T2 modalities are spliced by RGB channel, and the obtained three-channel image is used as the network input.

[0022] 2. The specific process in step 2 is as follows:

[0023] 1) Build an improved generation network

[0024] The generative network uses an improved U-Net structure with a multi-layer encoder-decoder structure, in which each decoder layer fuses small-scale and same-scale feature maps from the encoder with large-scale feature maps from the decoder. Cross-scale links not only allow the decoder's feature maps to capture shallow and deep semantic features at all scales, but also reduce the number of model parameters, making the model more concise. To alleviate the gradient vanishing or gradient exploding problems caused by deepening the layers and reduce the overfitting of deep networks, convolution operations are replaced by residual blocks. The residual blocks contain three convolutional layers: 1×1, 3×3, and 1×1, respectively. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. The input image is added to the original input after passing through these three convolution operations to obtain the output.

[0025] During the upsampling process of the decoder, detail information is easily lost, resulting in blurred or inaccurate segmentation boundaries. Therefore, a coordinate attention mechanism is added before upsampling. The coordinate attention mechanism suppresses unimportant channels and enhances the weight of high-frequency detail features, helping the network better restore the edges and details of the target, thereby improving the accuracy and robustness of the segmentation results.

[0026] 2) Build an improved discriminant network

[0027] The network structure of the discriminator is similar to the decoder in the generator. To reduce the loss of edge information during the convolution process, the image is padded in terms of height H and width W according to the size of the convolution kernel, and padding is added to the convolution layer to make the input dimension of the convolution layer consistent with the output dimension; 3 channels represent 3 modes. The discriminator enhances its ability to capture the edges and details of the segmented area by extracting multi-level features, further improving the segmentation accuracy.

[0028] 3. The specific process in step 3 is as follows:

[0029] The training adopts a loss function that is a mixture of multi-scale loss and Dice Loss:

[0030]

[0031] Among them, l mae Refers to the mean absolute error MAE or L1 distance, x n is the original input image; S(xn ) represents the predicted segmentation map generated by the segmentation network, y n is the real label map, Represents the pixel-by-pixel multiplication of each channel of the original image and the predicted segmentation map, Represents the pixel-by-pixel multiplication of each channel of the original image and the true label map, f c Indicates that the discriminator network performs multi-level feature extraction on the input image;

[0032] Specific process of calculating multi-scale L1 loss:

[0033] The predicted segmentation map and the true segmentation map are masked with the original image and input into the discriminant network at the same time. In the discriminant network, the feature maps output by multiple levels are spliced to obtain the predicted output and the true output, and then the error between the predicted output and the true output is calculated:

[0034]

[0035] n represents the number of samples, y i represents the true value of the i-th sample, represents the predicted value of the i-th sample, Represents the absolute error between the predicted value and the true value of the i-th sample;

[0036] The specific process of calculating Dice Loss:

[0037] Dice Loss is defined based on the Dice coefficient. The Dice coefficient is a set similarity measurement function, usually used to calculate the similarity between two samples. Its value range is [0, 1]. Dice Loss defined based on the Dice coefficient:

[0038]

[0039] Where |y∩y'| is the intersection of y and y', |y| and |y'| respectively represent the number of elements of y and y'; smooth adds a Laplace smoothing to Dice Loss, and its value is generally set to 1e-5.

[0040] 4. The specific process in step 4 is as follows:

[0041] The generator S and discriminator C in the model are trained in an alternating manner. First, the gradient calculated from the loss function is used to fix S and train C. Then, the gradient calculated from the same loss function passed from C to S is used to fix C and train S. Finally, the optimal segmentation result is obtained; the segmentation result is softmaxed, and the category corresponding to the maximum probability in the output vector is found. The four-channel result is converted into a single-channel label image with 0, 1, 2, and 3, and then the label is restored to the corresponding grayscale value to obtain the brain tumor segmentation result image.

[0042] The improved generative adversarial network model proposed in this invention has the following beneficial effects:

[0043] 1) Improved segmentation accuracy: Through cross-scale links and coordinate attention mechanisms, the network's ability to capture brain tumor edges and details is enhanced, significantly improving segmentation accuracy.

[0044] 2) Solving the problem of class imbalance: The hybrid loss function is used to effectively solve the problem of unbalanced class distribution in brain tumor MR images and improve the segmentation performance of the model.

[0045] 3) Training stability: By combining multi-scale L1 loss and Dice loss, the instability problem in the training process of the generative adversarial network is solved, and the convergence speed and stability of the model are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 : Algorithm flow chart of the present invention;

[0047] Figure 2 : The improved generative adversarial network model of the present invention;

[0048] Figure 3 : Schematic diagram of the impact of the hybrid loss function on the Dice coefficient;

[0049] Figure 4 : Comparison of segmentation results of different models. DETAILED DESCRIPTION

[0050] The process of the improved brain tumor image segmentation algorithm based on the generative adversarial network model is as follows: Figure 1 As shown in the figure, this method achieves segmentation of the entire brain tumor, the tumor core, and the enhanced tumor core. First, the preprocessed multimodal brain tumor image data is spliced to obtain a three-channel image. Then, an improved generative adversarial network model is constructed and trained on the training set. Finally, the model is tested on the test set and the segmentation results are evaluated.

[0051] 1. Data Preprocessing

[0052] 1) Offset field correction: Use the N4 algorithm to perform offset field correction on MRI images to eliminate grayscale inhomogeneity in the images;

[0053] 2) Image normalization: Since the image contrast of different modalities is different, the z-score method is used to normalize the image. The formula is:

[0054]

[0055] Where x is the sample data, σ is the sample standard deviation, and μ is the sample mean;

[0056] 3) 3D slice cropping: To remove invalid areas, the 3D MRI images with a size of 240 × 240 × 155 were cropped to 200 × 160 × 155 and axially sheared to obtain 2D slices of the four modalities, and then the non-foreground images were removed;

[0057] 4) Channel fusion: The slices of FLAIR, T1ce, and T2 modalities are spliced according to RGB channels, and the resulting 3×200×160 image data is used as the network input.

[0058] 2. Build an improved generative adversarial network model

[0059] 1) Build a generative network

[0060] The generative network structure has a multi-layer encoder-decoder structure, so that each decoder layer integrates the small-scale and same-scale feature maps from the encoder and the large-scale feature maps from the decoder; the cross-scale jump link not only allows the decoder's feature map to capture the shallow semantic features and deep semantic features of the full scale, but also reduces the number of model parameters, making the model more concise; in order to alleviate the gradient vanishing or gradient exploding problems caused by deepening the layer, and to reduce the overfitting phenomenon of the deep network, the convolution operation is replaced by a residual block, which contains three convolutional layers of 1×1, 3×3 and 1×1 respectively. Each convolutional layer is followed by a BN layer and a ReLU activation function. The input image is added to the original input after passing through these three convolution operations to obtain the output.

[0061] During the upsampling process of the decoder, detail information is easily lost, resulting in blurred or inaccurate segmentation boundaries. Therefore, a coordinate attention mechanism is added before upsampling. The coordinate attention mechanism suppresses unimportant channels and enhances the weight of high-frequency detail features, helping the network to better restore the edges and details of the target, thereby improving the accuracy and robustness of the segmentation results.

[0062] 2) Build a discriminant network

[0063] The discriminant network structure is similar to the decoder in the generator. In order to reduce the loss of edge information during the convolution process, the image is padded in terms of height H and width W according to the size of the convolution kernel, and padding is added to the convolution layer to make the input dimension of the convolution layer consistent with the output dimension; the discriminator enhances the ability to capture the edges and details of the segmented area by extracting multi-level features, thereby further improving the segmentation accuracy.

[0064] 3. Improve the loss function

[0065] The training adopts a loss function that is a mixture of multi-scale loss and Dice Loss:

[0066]

[0067] Among them, l mae Refers to the mean absolute error MAE or L1 distance, x n is the original input image, S(x n ) represents the predicted segmentation map generated by the segmentation network, y n is the real label map, Represents the pixel-by-pixel multiplication of each channel of the original image and the predicted segmentation map, Represents the pixel-by-pixel multiplication of each channel of the original image and the true label map, f c It indicates that the discriminator network performs multi-level feature extraction on the input image.

[0068] Specific process of calculating multi-scale L1 loss:

[0069] The predicted segmentation map and the true segmentation map are masked with the original image and input into the discriminant network at the same time. In the discriminant network, the feature maps output by multiple levels are spliced to obtain the predicted output and the true output, and then the error between the predicted output and the true output is calculated:

[0070]

[0071] n represents the number of samples, y i represents the true value of the i-th sample, represents the predicted value of the i-th sample, It represents the absolute error between the predicted value and the true value of the i-th sample.

[0072] The specific process of calculating Dice Loss:

[0073] First, let's understand the definition of the Dice coefficient. The Dice coefficient is a set similarity measurement function, usually used to calculate the similarity between two samples. Its value range is [0, 1]. The Dice Loss defined by the Dice coefficient is:

[0074]

[0075] Where |y∩y'| is the intersection of y and y', |y| and |y'| respectively represent the number of elements of y and y', and smooth adds a Laplace smoothing to Dice Loss, which is generally set to 1e-5.

[0076] Multi-scale L1 loss can be calculated by using these hierarchical features, namely pixel-level features, low-level features, and mid-level features, to capture long-range and short-range spatial relationships between pixels. Multi-class segmentation tasks often suffer from class imbalance, such as a smaller foreground than background, large differences in the size of segmented objects, and large variations in the proportions of different classes. Dice Loss can effectively address this problem.

[0077] 4. Train the model

[0078] The overall structure of the model is shown in the figure Figure 2 As shown in the figure, the dataset is input into the generative network, and the predicted segmentation map is obtained after network segmentation. The predicted segmentation map and the target segmentation map are then masked with the original image respectively, and finally input into the discriminative network to extract multi-level features to calculate the hybrid loss.

[0079] The generator S and discriminator C in the model are trained in an alternating manner. First, the gradient calculated from the loss function is used to fix S and train C. Then, the gradient calculated from the same loss function passed from C to S is used to fix C and train S. Finally, the optimal segmentation result is obtained.

[0080] 5. Evaluation and Analysis of Experimental Results

[0081] The Dice coefficient, precision and sensitivity are used as evaluation indicators. The higher the value of the evaluation indicator, the better the segmentation performance of the model. The present invention evaluates the model by using a hybrid function and not using a hybrid loss term, such as Figure 3 As shown in Figure 3, the use of the hybrid loss function can significantly improve the Dice coefficient of brain tumor segmentation. Experiments show that the hybrid loss function proposed in this invention can effectively improve the accuracy of brain tumor MR image segmentation.

[0082] The segmentation results were then quantitatively analyzed using three metrics. As can be seen in Table 1, in experiments on the same dataset, the Dice coefficient, accuracy, and sensitivity of the proposed method are all improved compared to the baseline method. Each improved module has a beneficial impact on the segmentation results.

[0083]

[0084] Table 1

[0085]

[0086] Table 2

[0087] In order to verify the superiority of the improved model, the present invention selected the existing mainstream segmentation networks Unet, AttentionUnet, Unet++ and SegAN for comparative experiments on the same data set. The comparison results show that the improved model has a great advantage in multi-category segmentation of brain tumors. The experimental results are shown in Table 2. Compared with other networks, the average Dice coefficient, average precision and average sensitivity of the three brain tumor segmentation categories of the present invention are improved. The comparison of segmentation results of different network models is shown in the figure below. Figure 4 As shown in the comparison figure, it can be seen that the algorithm of the present invention is more accurate in edge segmentation of the three categories of brain tumors.

Claims

1. An improved brain tumor image segmentation algorithm based on a generative adversarial network model, comprising the following steps: Step 1: Data preprocessing: Offset field correction, image normalization, and slice cropping are performed on brain tumor MR3D images to obtain slice images of four modalities. The grayscale images of the three modalities (Flair, T2, and T1ce) are then channel-concatenated. The preprocessed three-channel images can be used as the training and testing datasets. Step 2: Build an improved generative adversarial network model, introduce a cross-scale link module and coordinate attention mechanism into the U-shaped generative network, and mask the predicted segmentation map and the target segmentation map with the original image respectively, and then input them into the discriminative network; Step 3: Improve the loss function and use a hybrid loss function that combines L1 loss and Dice loss as the objective function for model training; Step 4: Use the improved generative adversarial network model for training to obtain the segmentation results of brain tumor MR images.

2. The improved brain tumor image segmentation algorithm based on the generative adversarial network model according to claim 1 is characterized in that: The specific process in step 1 is as follows: 1) The 3D images of each modality are normalized using the z-score method. Before normalization, the 3DMR images are first corrected for offset field to make the image grayscale more uniform, and then normalized using the z-score method: Where x is the sample data, σ is the sample standard deviation, and μ is the sample mean; 2) The background occupies a large proportion of the entire image. To remove the invalid area, the MRI image with a size of 240×240×155 was cropped to 200×160×155, and axial shearing was performed to obtain 2D slices of the four modalities, and then the image without foreground was removed; 3) The slices of FLAIR, T1ce and T2 modalities are spliced by RGB channel, and the obtained three-channel image is used as the network input.

3. The improved brain tumor image segmentation algorithm based on the generative adversarial network model according to claim 1 is characterized in that: The specific process in step 2 is as follows: 1) Build an improved generation network The generative network uses an improved U-Net structure with a multi-layer encoder-decoder structure, in which each decoder layer fuses small-scale and same-scale feature maps from the encoder with large-scale feature maps from the decoder. Cross-scale links not only allow the decoder's feature maps to capture shallow and deep semantic features at all scales, but also reduce the number of model parameters, making the model more concise. To alleviate the gradient vanishing or gradient exploding problems caused by deepening the layers and reduce the overfitting of deep networks, convolution operations are replaced by residual blocks. The residual blocks contain three convolutional layers: 1×1, 3×3, and 1×1, respectively. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. The input image is added to the original input after passing through these three convolution operations to obtain the output. During the upsampling process of the decoder, detail information is easily lost, resulting in blurred or inaccurate segmentation boundaries. Therefore, a coordinate attention mechanism is added before upsampling. The coordinate attention mechanism suppresses unimportant channels and enhances the weight of high-frequency detail features, helping the network better restore the edges and details of the target, thereby improving the accuracy and robustness of the segmentation results. 2) Build an improved discriminant network The network structure of the discriminator is similar to the decoder in the generator. To reduce the loss of edge information during the convolution process, the image is padded in terms of height H and width W according to the size of the convolution kernel, and padding is added to the convolution layer to make the input dimension of the convolution layer consistent with the output dimension; 3 channels represent 3 modes. The discriminator enhances its ability to capture the edges and details of the segmented area by extracting multi-level features, further improving the segmentation accuracy.

4. The improved brain tumor image segmentation algorithm based on the generative adversarial network model according to claim 1 is characterized in that: The specific process in step 3 is as follows: The training adopts a loss function that is a mixture of multi-scale loss and Dice Loss: Among them, l mae Refers to the mean absolute error MAE or L1 distance, x n is the original input image, S(x n ) represents the predicted segmentation map generated by the segmentation network, y n is the real label map, Represents the pixel-by-pixel multiplication of each channel of the original image and the predicted segmentation map, Represents the pixel-by-pixel multiplication of each channel of the original image and the true label map, f c Indicates that the discriminator network performs multi-level feature extraction on the input image; Specific process of calculating multi-scale L1 loss: The predicted segmentation map and the true segmentation map are masked with the original image and input into the discriminant network at the same time. In the discriminant network, the feature maps output by multiple levels are spliced to obtain the predicted output and the true output, and then the error between the predicted output and the true output is calculated: n represents the number of samples, y i represents the true value of the i-th sample, represents the predicted value of the i-th sample, Represents the absolute error between the predicted value and the true value of the i-th sample; The specific process of calculating Dice Loss: Dice Loss is defined based on the Dice coefficient. The Dice coefficient is a set similarity measurement function, usually used to calculate the similarity between two samples. Its value range is [0, 1]. Dice Loss defined based on the Dice coefficient: Where |y∩y'| is the intersection of y and y', |y| and |y'| respectively represent the number of elements of y and y', and smooth adds a Laplace smoothing to Dice Loss, which is generally set to 1e-5.

5. The improved brain tumor image segmentation algorithm based on the generative adversarial network model according to claim 1 is characterized in that: The specific process in step 4 is as follows: The generator S and discriminator C in the model are trained in an alternating manner. First, the gradient calculated from the loss function is used to fix S and train C. Then, the gradient calculated from the same loss function passed from C to S is used to fix C and train S. Finally, the optimal segmentation result is obtained; the segmentation result is softmaxed, and the category corresponding to the maximum probability in the output vector is found. The four-channel result is converted into a single-channel label image with 0, 1, 2, and 3, and then the label is restored to the corresponding grayscale value to obtain the brain tumor segmentation result image.

Images