Medical image segmentation method based on improved Dice loss function

Abstract

The invention relates to a medical image segmentation method based on an improved Dice loss function, and belongs to the field of image processing, and the method comprises the steps: obtaining a data set, and dividing the data set into a training set and a verification set according to a proportion; training the Resl8-Unet model through the training set, solving the gradient imbalance problem of a difficult sample in the model training process through an SDL-V1 loss function, and solving the gradient imbalance problem of a difficult pixel region of the same image in the model training process through an SDL-V2 loss function, so as to obtain a final trained model; and segmenting the medical image through the finally trained model. The defect that a traditional Dice loss function easily ignores learning of difficult samples and learning of difficult pixels is overcome, the model can better pay attention to learning of the difficult samples and the difficult pixels in the training process, the model can learn more features, and therefore the performance of the network model is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a medical image segmentation method based on an improved Dice loss function. Background Art

[0002] In recent years, with the development of deep learning, medical image segmentation has received more and more attention. Unlike other semantic segmentation tasks, medical image segmentation datasets are often more scarce, and using fewer datasets to achieve better segmentation effects has become a challenging task. Currently, for various complex medical image segmentation tasks, some people have proposed to achieve this by improving the loss function.

[0003] For deep neural networks, it is very important to choose a suitable loss function. The loss functions currently used in medical image segmentation tasks can be roughly divided into four categories: 1. Distribution-based loss function; 2. Region-based loss function; 3. Boundary-based loss function; 4. Hybrid loss function. Region-based loss functions mainly include Intersection over Union loss, Dice loss, Tversky loss, etc. Due to its connection with the Dice similarity coefficient indicator, Dice loss is the most commonly used function in this category. It approximately measures the degree of overlap between the predicted segmentation result and the true segmentation, and clearly distinguishes the instance pixels from the background pixels when returning the gradient. Therefore, unlike the distribution-based loss function, the images segmented by this type of function are complete connected graphs with specific shapes. Figure 1 The gradient changes of Dice loss for samples and pixels during training are shown, indicating the shortcomings of Dice loss in ignoring difficult samples and difficult pixels. Due to structural defects, other region-based loss functions have similar shortcomings as Dice loss. The hybrid loss function attempts to use two different types of loss functions together in the hope that they can play their respective advantages. A more popular approach is to weightedly combine cross entropy loss and Dice loss. This combination can usually combine the advantages of the two types of loss functions and alleviate the shortcomings mentioned above to a certain extent. However, the above problem is still not solved, and the distribution-based loss and region-based loss are not equally sensitive to the image resolution of the dataset, so the hyperparameters are difficult to adjust. In short, this hybrid approach still needs to be improved.

[0004] Although the Dice loss function has performed well in practical applications, it also has its own limitations: 1) Figure 1As shown in the left figure, the gradient value of the Dice loss function with respect to the confidence function (logits) increases as the Dice similarity coefficient increases, while the ideal loss function should focus on more difficult samples (samples with lower Dice similarity coefficients) when the Dice similarity coefficient is high. Figure 1 As shown in the right figure, Dice loss pays equal attention to pixels with lower predicted values ​​and pixels with higher predicted values. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a medical image segmentation method based on an improved Dice loss function to solve the shortcomings of the prior art.

[0006] The object of the present invention is achieved by the following technical solution: a medical image segmentation method based on an improved Dice loss function, the segmentation method comprising:

[0007] Step 1: Obtain the data set and divide it into training set and validation set in proportion;

[0008] Step 2: Train the Resl8-Unet model using the training set, and use the SDL-V1 loss function to solve the gradient imbalance problem of difficult and easy samples during model training, and use the SDL-V2 loss function to solve the gradient imbalance problem of difficult and easy pixel areas of the same image during model training, and finally obtain the trained model;

[0009] Step 3: Segment the medical image using the final trained model.

[0010] The construction process of the SDL-V1 loss function includes the following:

[0011] During the training process, let the model output be x, and after the sigmoid function, it is x s Then, after the Dice loss function f(x s );

[0012] The gradient generated by the loss function in the image includes two parts: the instance and the background. The two parts of the gradient are displayed separately from the perspective of a certain pixel point, and the new Dice loss function is obtained as follows: And the two parts of the gradient are and Among them I c Represents the sum of the predicted values ​​of some pixels of the instance, O c Represents the sum of the predicted values ​​of the background pixels, G c represents the sum of the true labels, and Represent the absolute values ​​of the gradients of the instance part and the background part respectively, ∈ is a constant;

[0013] In the original DSC C Adding the exponent γ makes the gradient of the difficult sample after differentiation account for a larger proportion, so that when calculating the gradient, multiply γDSC by the original basis C γ-1 , so that the difficult samples will have a large coefficient on the original gradient, so that the loss function pays more attention to the difficult samples, and then the SDL-V1 loss function is SDLV1 C =∑ C (1-DSC C γ ), γ is a hyperparameter in the interval (0,1), DSC C is the Dice similarity coefficient.

[0014] The construction process of the SDL-V2 loss function includes the following:

[0015] During the training process, let the model output be x, and after the sigmoid function, it is x s Then, after the Dice loss function f(x s );

[0016] The gradient generated by the loss function in the image includes two parts: instance and background. The gradients of the two parts are and and Represents the absolute value of the gradient of the instance part and the background part, p ic represents the prediction of the instance part, p oc Represents the prediction of the background part. When studying the gradient difference of different pixels of the same image at the same time, and is regarded as a constant, that is, the difference between different pixels is calculated by p ic *(1-p ic ) and p oc *(1-p oc )Decide;

[0017] By using the cross entropy loss CE C The coefficient is multiplied by the coefficient in real time And N keeps it consistent with the Dice coefficient. By adjusting the added hyperparameter γ, the loss function's attention to difficult and easy pixels is adjusted. The two-part gradient when Dice loss and cross entropy loss are used together is obtained as follows: and And the SDL-V2 loss function is Among them, ∑C (1-DSC C γ ) is the SDL-V1 loss function, α is used to adjust the loss function's attention to difficult pixels, N is the total number of pixels in the image, include and The absolute value of the gradient of the instance part and the background part.

[0018] The Resl8-Unet model includes an encoder and a decoder. The decoder consists of a series of convolutional layers, pooling layers, and residual blocks for extracting features from an input image. The encoder consists of a series of upsampling layers and convolutional layers for restoring the feature map extracted by the encoder to the size of the original input image.

[0019] The present invention has the following advantages: a medical image segmentation method based on an improved Dice loss function improves the shortcomings of the traditional Dice loss function that easily ignores the learning of difficult samples and the learning of difficult pixels, so that the model can better focus on the learning of difficult samples and difficult pixels during the training process, allowing the model to learn more features, thereby improving the performance of the network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of the gradient changes of samples and pixels during the training process of traditional Dice loss;

[0021] Figure 2 This is a schematic diagram of the Res18-Unet network model structure;

[0022] Figure 3 A comparison of some segmentation results on the liver tumor CT dataset. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application provided below in conjunction with the drawings is not intended to limit the scope of protection of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application. The present invention is further described below in conjunction with the drawings.

[0024] The present invention specifically relates to a medical image segmentation method based on an improved Dice loss function. The present invention improves the original Dice loss function and proposes a more efficient and ambitious loss function: Sharp Dice loss. The loss function has two versions. The first improvement method focuses on more efficient learning difficulty samples, and the second improvement method focuses on more efficient learning difficulty pixels. The same improvement method can be applied to other loss functions such as intersection-over-union loss and Tversky loss. Specifically, it includes the following contents:

[0025] Step 1: Obtain the data set and divide it into training set and validation set in proportion;

[0026] Step 2: Train the Resl8-Unet model using the training set, and use the SDL-V1 loss function to solve the gradient imbalance problem of difficult and easy samples during model training, and use the SDL-V2 loss function to solve the gradient imbalance problem of difficult and easy pixel areas of the same image during model training, and finally obtain the trained model;

[0027] Step 3: Segment the medical image using the final trained model.

[0028] Furthermore, the liver tumor dataset used for training in the present invention includes multiple tumor types, including hepatocellular carcinoma, intrahepatic cholangiocarcinoma, and metastatic liver cancer, etc. The imaging phases include ordinary CT scan, portal vein phase, and arterial phase, and include slices of two different thicknesses, 2 mm and 5 mm. The dataset contains 62,335 slices from 511 patients. The present invention divides the dataset into a training set and a validation set by the ratio of 7 to 3 patients, with 352 patients in the training set and 159 patients in the validation set.

[0029] Furthermore, if Figure 2 As shown, the training of the present invention uses the Res18-Unet model, which is an improvement on the traditional Unet. The model is divided into two main parts: encoder and decoder. The decoder is a classic ResNet-18 deep convolutional neural network, consisting of a series of convolutional layers, pooling layers, and residual blocks. This part is used to extract features from the input image. The encoder is composed of Unet, which is a classic network for image segmentation. It includes symmetrical encoder and decoder parts. The decoder part is symmetrical with the encoder. It consists of a series of upsampling layers and convolutional layers, which are used to restore the feature map extracted by the encoder to the size of the original input image.

[0030] Furthermore, the present invention proposes Sharp Dice Loss (hereinafter referred to as SDL), which has two versions. For the problem of unbalanced gradients of difficult and easy samples in the model training process, SDL-V1 version is proposed, and for the problem of unbalanced gradients of difficult and easy pixel areas of the same image in the model training process, SDL-V2 version is proposed. In order to introduce the proposed SDL, the Dice loss function is first introduced, and the gradient provided to the network by the Dice loss during the gradient descent process is studied, and the two shortcomings of the Dice loss are specifically analyzed.

[0031] In the medical field, Dice coefficient (DSC) is widely used to evaluate segmentation tasks. The expression of Dice loss (DL) is as follows, where g ic ∈{0, 1} and p ic ∈[0, 1] represents the true label and the predicted label respectively. The total number of pixels in the image is represented by N, and ∈ is used to prevent division by 0.

[0032]

[0033] DL C =Σ C (1-DSC C );

[0034] SDL-V1 loss function:

[0035] During the training process, assuming that the model output is x, after the sigmoid function, it is x s Then, after the Dice loss function f(x s ). Here we discuss the loss function for x s The gradient of the loss in the image can be divided into two parts, instance (in) and background (out). Here, the two parts of the gradient are shown from the perspective of a certain pixel. The Dice loss of the new formula and the two parts of the gradient are shown as follows:

[0036]

[0037]

[0038]

[0039] Among them I c Represents the sum of the predicted values ​​of some pixels of the instance, O c Represents the sum of the predicted values ​​of the background pixels, G c represents the sum of the true labels, and Represents the absolute value of the gradient of the instance part and the background part respectively.

[0040] Tracking the gradient as the DSC changes during training Figure 1 As shown in the left figure, the gradient value of the sample increases with the increase of DSC. This shows that the model has a larger gradient for the learned samples, that is, the simple samples, than for the difficult samples, that is, the Dice loss will pay attention to the images that are easy to train and ignore the images that are difficult to train.

[0041] In order to solve the above problem, the present invention suppresses the gradient of the sample with a good prediction value, that is, the simple sample, as shown below, by C Adding the exponent γ to the gradient of the difficult sample after differentiation makes it account for a larger proportion. γ is a hyperparameter in the interval (0,1). In this way, when calculating the gradient, γDSC will be multiplied by the original one. C γ-1 In this way, the difficult samples will have a larger coefficient on the original gradient, so that the loss function can pay more attention to the difficult samples. C is the Dice similarity coefficient.

[0042] SDLV1 C =∑ C (1-DSC C γ )(γ∈(0,1));

[0043] SDL-V2 loss function:

[0044] like Figure 1 As shown in the right figure, during training, the relationship between gradients and pixels exposes the second possible problem of Dice loss, that is, Dice loss gives the same attention to simple pixels and difficult pixels, ignoring pixels that are difficult to train.

[0045] During the training process, assuming that the model output is x, after the sigmoid function, it is x s Then, after the Dice loss function f(x s ), based on the previous content, here we discuss the gradient of the loss function with respect to x, which is also divided into the instance part and the background part. The two parts of the gradient are shown below. ic represents the prediction of the instance part, p oc Represents the prediction of the background part.

[0046]

[0047]

[0048] When studying the gradient differences of different pixels in the same image at the same time, we can temporarily and is regarded as a constant, that is, the difference between different pixels is calculated by p c *(1-p c ) decides, c *(1-p c ) including p ic *(1-p ic ) and p oc *(1-p oc ).

[0049] In order to solve the above problem, the general approach is to combine the Dice loss function with the cross entropy loss function, but at this time the gradient of the Dice loss As the DSC of the image changes during training, the coefficient in the cross entropy is a constant about the total number of pixels N. In most cases, the gradient provided by the cross entropy loss is much smaller than the gradient provided by the Dice loss in large-sized images. Therefore, the present invention proposes a new method to solve the above problem, that is, to adjust the coefficient of the cross entropy loss. By dividing the cross entropy loss CE C Real-time multiplication factor And N makes it always consistent with the Dice coefficient, and then the attention of the loss function to difficult and easy pixels can be adjusted by adjusting the added hyperparameter γ. The following formula shows the gradient when Dice loss is used in conjunction with cross entropy loss (only the example part is shown) and the V2 version proposed by the present invention, where α is used to adjust the attention of the loss function to difficult pixels.

[0050]

[0051]

[0052] The present invention conducted four experiments on six datasets, namely, a liver tumor CT dataset provided by a hospital, a public abdominal dataset, a public liver dataset, a public colon polyp endoscopy dataset, and a thyroid nodule ultrasound dataset. The evaluation indicators used in the experiments were DSC, Jaccard Similarity Coefficient (Jaccard), False Negative Rate (FNR), and False Positive Rate (FPR).

[0053] In order to verify the effectiveness of the present invention, the present invention compares the proposed Sharp Dice loss with other loss functions on the liver tumor CT dataset. Compared with the Dice loss, the SDLV1 and SDLV2 of the present invention are improved by 2.22% and 2.15% on DSC, 2.51% and 2.4% on Jaccard, and the FNR is reduced by 1.76% and 0.37%; compared with the Dice+CE mixed loss, the SDLV1+CE mixed loss of the present invention is improved by 2.15% on DSC, 2.64% on Jaccard, the FPR is reduced by 0.03%, and the FNR is reduced by 0.54%. In comparison with other losses, the SDLV1+CE mixed loss of the present invention achieved the best overall performance. The improved loss function of the present invention achieved the best performance in each indicator, and some segmentation results are shown in Figure 2. Figure 3 As shown, the effectiveness of the loss function of the present invention is verified.

[0054] In order to verify the generalization of the loss function proposed in the present invention and its effectiveness for mixed data sets, the present invention simultaneously conducted liver segmentation experiments on public abdominal data sets and public liver data sets, and extended the improvement of the Dice loss of the present invention to the two region-based loss functions of IOU loss and Tversky loss. The following results can be obtained: 1) Compared with IOU loss, the Sharp IOU of the present invention improves DSC by 0.91%; compared with Dice loss, the SDLV1 and SDLV2 of the present invention improve DSC by 0.48% and 0.73% respectively; compared with Dice+CE loss, the SDLV+CE of the present invention improves DSC by 0.74%; compared with Tversky, the Sharp Tversky of the present invention improves DSC by 4.02%; compared with Tversky+CE, the Sharp Tversky+CE of the present invention improves DSC by 1.21%; 2) The improved Sharp Tversky of the present invention achieves the best comprehensive performance. This experiment verifies the scalability of the improved method of the present invention for region-based loss functions, and also verifies the superiority of the improved method of the present invention for mixed data sets.

[0055] In order to verify the robustness of the present invention for different image resolutions, the present invention conducted experiments on two thyroid nodule ultrasound data sets, and the experimental results showed that the SDLV1+CE of the present invention achieved the best results, verifying the robustness of the present invention for complex data sets.

[0056] In order to verify the robustness of the present invention for different models, the present invention uses two deep neural networks to train the public colon polyp endoscopy dataset at the same time. The experimental results show that SDLV2 achieves the best results on both models at the same time, verifying the robustness of the present invention for different models.

[0057] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used for various other combinations, modifications and improvements, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art do not deviate from the spirit and scope of the present invention, and should be within the scope of protection of the claims attached to the present invention.

Claims

1. A medical image segmentation method based on an improved Dice loss function, characterized in that: The segmentation method comprises: Step 1: Obtain the data set and divide it into training set and validation set in proportion; Step 2: Train the Resl8-Unet model using the training set, and use the SDL-V1 loss function to solve the gradient imbalance problem of difficult and easy samples during model training, and use the SDL-V2 loss function to solve the gradient imbalance problem of difficult and easy pixel areas of the same image during model training, and finally obtain the trained model; Step 3: Segment the medical image using the final trained model.

2. The medical image segmentation method based on the improved Dice loss function according to claim 1, characterized in that: The construction process of the SDL-V1 loss function includes the following: During the training process, let the model output be x, and after the sigmoid function, it is x s Then, after the Dice loss function f(x s ); The gradient generated by the loss function in the image includes two parts: the instance and the background. The two parts of the gradient are displayed separately from the perspective of a certain pixel point, and the new Dice loss function is obtained as follows: And the two parts of the gradient are and Among them I c Represents the sum of the predicted values ​​of some pixels of the instance, O c Represents the sum of the predicted values ​​of the background pixels, G c represents the sum of the true labels, and Represent the absolute values ​​of the gradients of the instance part and the background part respectively, ∈ is a constant; In the original DSC C Adding the exponent γ makes the gradient of the difficult sample after differentiation account for a larger proportion, so that when calculating the gradient, multiply γDSC by the original basis C γ-1 , so that the difficult samples will have a large coefficient on the original gradient, so that the loss function pays more attention to the difficult samples, and then the SDL-V1 loss function is SDLVt C =∑ C (t-DSC C γ ), γ is a hyperparameter in the interval (0,1), DSC C is the Dice similarity coefficient.

3. The medical image segmentation method based on the improved Dice loss function according to claim 1, characterized in that: The construction process of the SDL-V2 loss function includes the following: During the training process, let the model output be x, and after the sigmoid function, it is x s Then, after the Dice loss function f(x s ); The gradient generated by the loss function in the image includes two parts: instance and background. The gradients of the two parts are and and Represents the absolute value of the gradient of the instance part and the background part, p ic represents the prediction of the instance part, p oc Represents the prediction of the background part. When studying the gradient difference of different pixels of the same image at the same time, and is regarded as a constant, that is, the difference between different pixels is calculated by p ic *(tp ic ) and p oc *(tp oc )Decide; By using the cross entropy loss CE C The coefficient is multiplied by the coefficient in real time And N keeps it consistent with the Dice coefficient. By adjusting the added hyperparameter γ, the loss function's attention to difficult and easy pixels is adjusted. The two-part gradient when Dice loss and cross entropy loss are used together is obtained as follows: and And the SDL-V2 loss function is Among them, ∑ C (t-DSC C γ ) is the SDL-V1 loss function, α is used to adjust the loss function's attention to difficult pixels, N is the total number of pixels in the image, include and The absolute value of the gradient of the instance part and the background part.

4. The medical image segmentation method based on the improved Dice loss function according to claim 1, characterized in that: The Resl8-Unet model includes an encoder and a decoder, wherein the decoder is composed of a series of convolutional layers, pooling layers, and residual blocks for extracting features from input images; The encoder consists of a series of upsampling layers and convolutional layers to restore the feature maps extracted by the encoder to the size of the original input image.

Images