Retinal vessel segmentation method fusing unet and edge detection model

Abstract

The application relates to a retinal blood vessel segmentation method combining a UNet and an edge detection model. Based on the UNet network, common convolution of the UNet is replaced by mixed hole convolution, the receptive field is increased, and more features are extracted; on the other hand, the VGG-16 of the HED is replaced by the encoder part of the UNet, an edge detection method is used, the outline of the blood vessel is displayed, and finally the output of the HED and the output of the UNet segmentation after ASSP are combined to obtain the final result. The application can improve the segmentation accuracy of the retinal blood vessel.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image segmentation, and particularly relates to a retinal blood vessel segmentation method fusing a UNet and an edge detection model. BACKGROUND

[0002] The retinal blood vessels of the fundus provide a lot of information about the health of the eye and even the whole body. Retinal blood vessels of different structures can be used to diagnose certain diseases, such as hypertension or diabetes. Hypertensive retinopathy (HR) is caused by the elevated arterial blood pressure or stenosis of blood vessels of patients with long-term hypertension, and diabetic retinopathy (DR) is caused by diabetes and may be accompanied by vascular swelling. Therefore, retinal blood vessel segmentation is an important step in diagnosing certain diseases, and accurate segmentation of retinal blood vessels is of great significance. At present, there are mainly two methods for segmenting retinal blood vessels: manual segmentation of retinal blood vessels and deep learning-based methods. After retinal blood vessel segmentation, doctors can diagnose some diseases of patients according to the segmented blood vessel structure. However, manual segmentation not only requires professionals to have very high professional ability, but also is a time-consuming and laborious work. Deep learning-based methods not only save manpower and time, but also have great guarantee in accuracy. Retinal blood vessel segmentation is the basis of eye disease detection, and the accuracy of blood vessel segmentation directly affects the accuracy of subsequent disease diagnosis.

[0003] In the photographed retinal fundus image, the retinal blood vessels are similar in color to the background and have low contrast, which makes it difficult to identify, especially the microvessel part, which is more difficult to distinguish from the background. Moreover, the number of public retinal fundus image datasets is very small at present, so it is necessary to use data enhancement methods such as random rotation, adding Gaussian noise, color jittering, and horizontal, vertical and diagonal flipping to increase the number of datasets, and some of these methods can improve the contrast between blood vessels and the background, thereby improving the segmentation accuracy.

[0004] The challenges of retinal fundus blood vessel segmentation are: (1) the small number of retinal datasets; (2) the blood vessels and the background have no obvious difference, and it is very difficult to find each blood vessel without introducing too many false positives; (3) the connectivity of blood vessels: the connected blood vessels may be segmented into two segments due to external factors such as light. SUMMARY

[0005] The purpose of the present application is to improve the segmentation accuracy of retinal blood vessels, and to provide a retinal blood vessel segmentation method fusing a UNet and an edge detection model.

[0006] To achieve the above objectives, the technical solution of the present invention is: a retinal vessel segmentation method that integrates UNet and edge detection models. Based on the UNet network, the ordinary convolution of UNet is replaced with hybrid dilated convolution to increase the receptive field and extract more features. On the other hand, the encoder part of UNet is used to replace the VGG-16 of HED to display the contour of the blood vessels. Finally, the final result is obtained by combining the output of HED and the output of UNet segmentation after ASPP.

[0007] Compared with the prior art, the present invention has the following beneficial effects:

[0008] To improve the accuracy of vessel segmentation in acquired retinal images, especially the segmentation accuracy of microvessels at the edges, this invention proposes a UNet retinal vessel segmentation method combined with edge detection. This invention is primarily based on UNet. First, it replaces the ordinary convolution in UNet with hybrid dilated convolution to increase the receptive field size and extract more image features. Atrous Spatial Pyramid Pooling (ASPP) is then applied to the segmented output. Simultaneously, the encoder part of UNet is used to replace the original VGG-16 in Holistically-nested edge detection (HED), aiming to find more microvessel edges and improve the accuracy of microvessel segmentation. Finally, the output of HED and the segmented output are combined to obtain the final segmentation result. Attached Figure Description

[0009] Figure 1 This is a flowchart of the algorithm of the present invention.

[0010] Figure 2 It is a hollow convolutional block structure.

[0011] Figure 3 This is a diagram showing the DRIVE segmentation results.

[0012] Figure 4 The segmentation result for CHASE_DB1.

[0013] Figure 5 middle: Figure 5 (a) The test set for DRIVE; Figure 5 (b) Graph showing the algorithm results of this invention; Figure 5 (c) Results without using HED; Figure 5 (d) Results of UNet; Figure 5 (e) Manual ideal segmentation result.

[0014] Figure 6 middle: Figure 6 (a) The test set for CHASE_DB1; Figure 6(b) Graph showing the algorithm results of this invention; Figure 6 (c) Results without using HED; Figure 6 (d) Results of UNet; Figure 6 (e) Manual ideal segmentation result. Detailed Implementation

[0015] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0016] This invention discloses a retinal vessel segmentation method that integrates UNet and edge detection models. Based on the UNet network, the ordinary convolution of UNet is replaced with hybrid dilated convolution to increase the receptive field and extract more features. On the other hand, the encoder part of UNet is used to replace the VGG-16 of HED to reveal the contour of the blood vessels. Finally, the final result is obtained by combining the output of HED and the output of UNet segmentation after ASPP.

[0017] The following is a detailed implementation process of the present invention.

[0018] like Figure 1 As shown, to improve the segmentation accuracy of microvessels, this invention proposes a UNet retinal vessel segmentation method combined with edge detection. The algorithm is primarily based on the UNet network, replacing ordinary convolutions with hybrid dilated convolutions to increase the receptive field and extract more features. Furthermore, the encoder portion of the UNet replaces the VGG-16 edge detection method (HED), using edge detection to reveal the vessel contours. Finally, the edge detection output and the UNet segmentation output after Atrous Spatial Pyramid Pooling (ASSP) are combined to obtain the final result. Experimental results on the DRIVE and CHASE-DB1 public datasets demonstrate that the algorithm improves the segmentation accuracy of retinal vessels.

[0019] 1. Dentular convolution block

[0020] The model in this invention is primarily based on the UNet network, adhering to the encoder-decoder structure. To acquire more image features, hybrid dilated convolutions replace ordinary convolutions in the UNet network. This not only increases the receptive field and captures more features but also avoids some image information loss and loss of information continuity caused by consecutively using dilated convolutions with the same dilation rate. Furthermore, modifications were made to the basic UNet structure, Conv-BN-ReLU, by adding Dropout between BN and ReLU, as shown below. Figure 2As shown, Dropout effectively avoids the problem of overfitting. More importantly, it ensures that some neurons do not necessarily appear in every dropout, and weight updates no longer depend on the combined effect of hidden nodes with fixed relationships. This prevents certain features from being effective only under other features, making the model learn more robust features.

[0021] 2. Holistically nested edge detection

[0022] To compensate for the loss of image information caused by upsampling, Holistically-nested edge detection (HED) is used to delineate the boundaries of retinal vessels, thereby improving the network's segmentation accuracy in microvessels. To combine HED and UNet, this invention utilizes the encoder portion of UNet as the backbone network of HED. HED employs multi-scale features, and the fused HED branch consists of four boundary branches merged through a concatenation operation. The dimension of each boundary branch is the same as the original input image. The size of each output is the same as the original input image.

[0023] 3. Feature Decoding Block

[0024] In the encoder section, features of the input image are extracted to generate higher-level feature information. The decoder then decodes the encoder's results using upsampling to restore the image's initial resolution. Upsampling, whether using deconvolution or bilinear interpolation, results in information loss. To improve classification at the end, ASPP is used in the final part of the decoder. ASPP utilizes multi-scale features by using multiple dilated convolutions with varying ratios and adds the output to the HED result.

[0025] 4. Loss Function

[0026] Cross-entropy describes the distance between the actual output (probability) and the expected output (probability); that is, the smaller the cross-entropy value, the closer the two probability distributions are. Since classification tasks often involve binary classification, and retinal vessel segmentation is essentially a binary classification problem, there is a special case of cross-entropy: binary cross-entropy, which is defined as follows:

[0027]

[0028] Where y i Represents the true labels in the training set. This indicates the predicted category of pixel i, and N represents the total number of pixels.

[0029] To evaluate the performance of the retinal vessel segmentation algorithm, we conducted experiments on two public datasets, DRIVE and CHASE-DB1. The DRIVE dataset contains 40 fundus images, each with a resolution of 565×584, and the ground truth for each image is provided by a doctor. Our algorithm is first qualitatively compared with five existing medical image segmentation algorithms (UNet, RU-Net, R2U-Net, SD-UNet, and IterNet) in terms of segmentation accuracy. Then, we quantitatively compare the six algorithms on two datasets using four common segmentation metrics. These four metrics are accuracy (ACC), sensitivity (SE), specificity (SP), and area under the ROC curve (AUC), defined as follows:

[0030]

[0031]

[0032]

[0033] In this model, TP and TN represent positive samples predicted as positive and negative samples predicted as negative, respectively, while FP and FN represent negative samples predicted as positive and positive samples predicted as negative, respectively. All three segmentation metrics range from 0 to 1. Higher values ​​for each metric indicate better segmentation performance. All experiments were conducted on a laptop with an NVIDIA 3060 GPU and 6GB of VRAM.

[0034] 5. Qualitative comparison

[0035] To qualitatively compare the segmentation performance of different retinal vessel segmentation methods, we conducted two sets of experiments, performing vessel segmentation on four retinal fundus images. The segmentation results are shown below. Figure 3 and Figure 4 As shown.

[0036] Figure 3 and Figure 4 The image shows the blood vessel segmentation results of the UNet method and the method of this invention on the same eight retinal fundus images. From top to bottom, the image displays the original image, the UNet segmentation result, the segmentation result without HED, the segmentation result of the algorithm of this invention, and the manually optimized segmentation result. Figure 5 neutralization Figure 6 The results provide a more detailed comparison between the UNet method and the method of this invention on the DRIVE and CHASE_DB1 datasets. Figure 5In this invention, the algorithm does not exhibit the error that UNet cannot segment microvessels, while... Figure 6 In this invention, the algorithm does not, like UNet, sever a complete blood vessel. (Summary) Figure 5 and Figure 6 The comparison results show that the algorithm of this invention performs better than the UNet method in the segmentation of retinal microvessels.

[0037] Table 1 compares the segmentation results on the DRIVE dataset.

[0038] ACC SE SP AUC UNet 0.9651 0.7626 0.9847 0.9623 RU-Net 0.9556 0.7751 0.9816 0.9782 R2U-Net 0.9556 0.7792 0.9813 0.9784 SD-UNet 0.9674 0.7891 0.9848 0.9836 IterNet 0.9574 0.7791 0.9831 0.9813 Invention algorithm 0.9694 0.8009 0.9858 0.9858

[0039] Table 2 compares the segmentation results on the CHASE_DB1 dataset.

[0040] ACC SE SP AUC UNet 0.9738 0.7491 0.9890 0.9701 RU-Net 0.9622 0.7459 0.9836 0.9803 R2U-Net 0.9634 0.7756 0.9820 0.9815 SD-UNet 0.9738 0.7559 0.9900 0.9872 IterNet 0.9760 0.7969 0.9881 0.9899 Invention algorithm 0.9752 0.8094 0.9865 0.9882

[0041] 6. Quantitative comparison

[0042] To quantitatively compare the segmentation accuracy of the six segmentation methods, we tested them on 20 test datasets from DRIVE and 8 test datasets from CHASE_DB1, using four segmentation accuracy metrics: Accuracy, Sensitivity, Specificity, and Area Under Curve. Tables 1 and 2 show the quantitative evaluation results of retinal fundus vessel segmentation on the two datasets, with the best metric value in each column highlighted in bold. In the DRIVE dataset, as shown in Table 1, our algorithm achieved the best segmentation performance in Accuracy, Sensitivity, Specificity, and AUC. While it didn't significantly outperform the other five algorithms in Accuracy, Specificity, and AUC, it significantly surpassed the second-best algorithm in Sensitivity. In the CHASE_DB1 dataset, as shown in Table 2, our algorithm performed similarly to SD-UNet and IterNet, excelling in each of the four evaluation metrics. Overall, our proposed algorithm outperforms the other five algorithms.

[0043] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for retinal vessel segmentation integrating UNet and edge detection models, characterized in that, Based on the UNet network, the ordinary convolutions of UNet are replaced with hybrid dilated convolutions to increase the receptive field and extract more features. On the other hand, the encoder part of UNet is used to replace the VGG-16 of HED to reveal the contours of blood vessels. Finally, the HED output and the UNet segmentation output after ASPP are combined to obtain the final result. The replacement of the VGG-16 of HED with the encoder part of UNet means that the encoder part of UNet is used as the backbone network of HED. HED adopts multi-scale features. The fused HED branch consists of four boundary branches merged by a splicing operation. The dimension of each boundary branch is the same as that of the original input image, and the size of each output is the same as that of the original input image.

2. The retinal vessel segmentation method fusing UNet and edge detection models according to claim 1, characterized in that, The UNet network adopts an encoder-decoder structure.

3. The retinal vessel segmentation method fusing UNet and edge detection models according to claim 1, characterized in that, Furthermore, modifications were made to the basic structure of the UNet network, Conv-BN-ReLU, by adding Dropout between BN and ReLU.

4. The retinal vessel segmentation method fusing UNet and edge detection models according to claim 1, characterized in that, The output of UNet segmentation after ASPP is obtained by applying ASPP after the UNet segmentation output. ASPP utilizes multi-scale features by using multiple dilated convolutions with different ratios and adds the output to the output of HED.

5. The retinal vessel segmentation method integrating UNet and edge detection models according to claim 1, characterized in that, The loss function used in this method is binary cross-entropy, defined as follows: Where y i Represents the true labels in the training set. This indicates the predicted category of pixel i, and N represents the total number of pixels.

Images