A global fusion dual-channel retinal vessel segmentation method

Abstract

The present invention discloses a globally fused dual-channel retinal vessel segmentation method, which first obtains a retinal vessel image dataset and preprocesses the original image and label. Secondly, the preprocessed image is resampled into multiple images of different sizes. The multiple resampled images of different sizes are then input into a parallel small vessel enhancement network and a segmentation network in multiple stages to obtain a retinal vessel segmentation result image. Finally, a loss function is designed, and the loss is calculated between the retinal vessel segmentation result image and the label, and training and testing are performed. The present invention can effectively segment retinal vessels and achieve better accuracy and sensitivity than common retinal vessel segmentation networks.

Description

Technical Field

[0001] The present invention belongs to the field of medical image segmentation, and in particular relates to a global fusion dual-channel retinal blood vessel segmentation method. Background Art

[0002] The human eye is composed of the cornea, pupil, iris, vitreous humor, and retina. Ophthalmic diseases such as diabetic retinopathy (DR), glaucoma, and age-related macular degeneration can easily alter the width, bifurcation, and tortuosity of retinal blood vessels, leading to chronic insufficient blood supply to the eyes and even blindness. The anatomical structure of retinal vessels contains critical information that helps ophthalmologists detect and diagnose a range of retinal diseases. In recent years, computer-aided diagnosis (CAD) systems have been widely used to automatically segment and analyze retinal vascular structures. CAD not only bridges the gap between the growing demand for diagnostics and the limited ability of ophthalmologists to process the ever-increasing number of retinal images, but also enables clinicians to make accurate and efficient decisions. This effectively helps doctors screen for and monitor corresponding ophthalmic diseases.

[0003] However, due to the complexity of fundus images, accurate segmentation of retinal vessels has always been a challenging task, which has a significant impact on the accurate detection of retinal vessels. Since the acquired retinal images are affected by the complex vascular structure, low contrast, uneven illumination, and pathological noise, it is difficult to accurately segment small blood vessels during the segmentation process. Factors such as complex backgrounds, fundus exudates, and hemangiomas contribute to the complexity and low contrast of blood vessels.

[0004] In order to obtain the morphological changes of retinal blood vessels, various algorithms have been proposed and improved to enhance detection capabilities. Retinal blood vessel labels are generally manually annotated by professional medical personnel, which is time-consuming and labor-intensive, and requires certain professional qualities from the annotating medical personnel. In order to reduce the workload of doctors and the errors caused by manual labeling, many model-based methods combined with image processing algorithms have been applied to the image analysis of fundus images. In retinal blood vessel image segmentation, the most widely used network architecture is U-Net. Although U-net and networks based on U-net have performed well in retinal blood vessel segmentation tasks, they still have certain shortcomings in accurate image segmentation due to shortcomings such as information loss and insufficient processing of local context features. Summary of the Invention

[0005] Based on the above analysis, this paper proposes a globally fused dual-channel retinal vessel segmentation method to provide an automatic end-to-end fundus image vessel segmentation solution. This method addresses the following issues in current retinal vessel image segmentation methods: a single segmentation network cannot accurately segment vessels of varying thickness; small vessels have low contrast, making direct segmentation of these vessels prone to mis-segmentation and missed segmentation; and direct input of multi-scale images into the segmentation network lacks channel and spatial location information.

[0006] In order to solve the above problems, the present invention provides a global fusion dual-channel retinal vessel segmentation method. The overall segmentation scheme includes two networks: vessel enhancement and vessel segmentation. The steps are as follows:

[0007] Step 1: Obtain a retinal vascular image dataset and preprocess the original images and labels. The processing process includes cropping, random rotation, and contrast enhancement.

[0008] Step 2: resample the preprocessed image into multiple images of different sizes. The largest image size after resampling is n×n, followed by And so on to get all the resampled images.

[0009] Step 3: The resampled images of different sizes are input into the parallel small blood vessel enhancement network and segmentation network in multiple stages to obtain the retinal blood vessel segmentation result map.

[0010] Step 4: Design a loss function, calculate the loss between the retinal blood vessel segmentation result image and the label, perform training, and test.

[0011] Furthermore, the specific process of step three is as follows:

[0012] 3.1, multiple preprocessed images of different sizes are input into multiple parallel small blood vessel enhancement networks to obtain multiple enhanced feature maps of different sizes.

[0013] The small blood vessel enhancement network uses a Frangi filter based on the Hessian matrix; an optimizer is used to reduce noise in the image enhanced by the Frangi filter.

[0014] 3.2. A variable convolution module is added before each convolution layer in the U-Net encoder to build a segmentation network. The variable convolution module consists of sequentially connected convolution layers, pooling layers, and variable convolutions; variable convolution helps capture the global context information of multi-stage input multi-scale images and detail-enhanced images.

[0015] The preprocessed images of different sizes are input into different stages of the encoder respectively and fused with the output feature maps of the previous stage; the enhanced feature maps of different sizes output by the small blood vessel enhancement network are fused with the corresponding feature maps fused at different stages of the encoder and the outputs of different stages of the decoder, and used as the input of the next stage; in the transition stage between the encoder and the decoder, a residual connection of hybrid attention constructed by combining spatial attention and channel attention is added to solve the problem of channel spatial position information loss that may occur when multiple images are fused in the encoder stage.

[0016] 3.3, the decoder outputs images at multiple stages, uses linear interpolation to restore them to the image size of n×n, and uses the image output of the last stage of the decoder as the retinal blood vessel segmentation result image.

[0017] Furthermore, the loss function described in step 4 is: the multiple images output after linear interpolation are respectively compared with the labels using the cross entropy loss function to calculate the loss, and the average is added to obtain the total loss function.

[0018] Beneficial effects of the present invention:

[0019] The proposed method can effectively segment retinal vessels. It utilizes variable convolution to capture global contextual information from multi-stage input, multi-scale images and detail-enhanced images, and utilizes residual connections with mixed attention to integrate spatial position information from multiple fused images at the encoder stage. The proposed segmentation network can effectively segment retinal vessels and achieves superior accuracy and sensitivity compared to conventional retinal vessel segmentation networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The segmentation network structure used in the retinal blood vessel segmentation network of the present invention;

[0021] Figure 2 This is a structural diagram of a variable convolution module of the retinal vessel segmentation network of the present invention;

[0022] Figure 3 This is a structural diagram of the attention module of the retinal blood vessel segmentation network of the present invention;

[0023] Figure 4 This is a feature integration comparison diagram of the retinal blood vessel segmentation network of the present invention;

[0024] Figure 5 This is a comparison chart of the segmentation results of the present invention and the true labels. DETAILED DESCRIPTION

[0025] The present invention is further described in conjunction with a specific implementation process. A global fusion dual-channel retinal vessel segmentation method is specifically implemented as follows:

[0026] Step 1: Obtain a retinal vascular image dataset and preprocess the original images and labels. The processing process includes cropping, random rotation, and random GANMA contrast enhancement.

[0027] Step 2: Resample the preprocessed image to 512×512, 256×256, 128×128, and 64×64 sizes.

[0028] Step 3: The four resampled images of different sizes are input into the parallel small blood vessel enhancement network and segmentation network in four stages to obtain the retinal blood vessel segmentation result image.

[0029] 3.1, after preprocessing, four images of different sizes are input into four parallel small blood vessel enhancement networks to obtain four enhanced feature maps of different sizes.

[0030] The small blood vessel enhancement network uses a Frangi filter based on the Hessian matrix, and uses an optimizer to reduce noise in the image enhanced by the Frangi filter.

[0031] 3.2, a variable convolution module is added before each convolution layer in the U-Net encoder to build a segmentation network. The variable convolution module consists of sequentially connected convolution layers, pooling layers and variable convolutions.

[0032] The four preprocessed images of different sizes are input into the four different stages of the encoder respectively and fused with the output feature maps of the previous stage; the four enhanced feature maps of different sizes output by the small blood vessel enhancement network are fused with the corresponding feature maps fused at different stages of the encoder and the outputs of different stages of the decoder, and used as the input of the next stage; in the transition stage between the encoder and the decoder, a residual connection of hybrid attention constructed by combining spatial attention and channel attention is added.

[0033] 3.3, the four stages of the decoder output images respectively, and use linear interpolation to restore them to the image size of 512×512. The image output by the last stage of the decoder is used as the retinal blood vessel segmentation result image.

[0034] Step 4: Design a loss function, calculate the loss between the retinal blood vessel segmentation result image and the label, perform training, and test.

[0035] Example:

[0036] The overall segmentation process of this method is as follows Figure 1The datasets used are DRIVE, STARE, and CHASEDB1.

[0037] The DRIVE dataset contains 40 three-channel color images, each measuring 768 × 584 pixels. Twenty images were used as training, and 20 as testing. The STARE dataset contains 20 three-channel color images, each measuring 700 × 605 pixels. A leave-one-out cross-validation approach was used, where each image was tested against a classification model trained on the other 19 images. The CHASEDB1 dataset contains 28 three-channel color images, each measuring 960 × 999 pixels. The first 20 images were used for training, and the remaining 8 images were used for testing. At the beginning of the experiment, all retinal vascular images were resampled to 512 × 512 × 3 to ensure consistency.

[0038] (1) Enhanced network of small blood vessels:

[0039] The input of the network is a grayscale multi-scale image, and the Frangi filter distinguishes blood vessels from the background based on the parameters in the Hessian matrix. The idea behind the Hessian matrix eigenvalue analysis is to extract the main direction at the location where the local second-order derivative structure can be decomposed in the image. Since this method can directly give the direction of minimum curvature (along the blood vessel), it avoids the use of multiple filters in multiple directions. The Hessian matrix pS of the 2D image at a scaled location is:

[0040]

[0041] The parameters of the Hessian matrix are learnable parameters (α, β) set in the neural network, and α and β are continuously updated and optimized according to the network to achieve the optimal parameters. Normalization is then used to reduce noise in the image after the filter enhancement.

[0042] (2) Segmentation network

[0043] To meet the network's multi-size input requirements, the original retinal vascular images and labels were resampled to 512×512, 256×256, 128×128, and 64×64, respectively. The resampled images and labels were used for neural network training. Furthermore, to enhance the network's generalization capabilities, data augmentation techniques were applied to the images and corresponding labels, including random rotations within the range [-20°, 20°] with a probability of 0.5 and gamma contrast enhancement within the range [0.5, 2].

[0044] This paper improves on the basis of U-Net and uses a feasible variable convolution module to replace the original convolution layer in the encoder stage. The variable convolution adds an offset to the sampling position of the standard convolution operation, so that the convolution kernel can pay more attention to the blood vessel part during the training process. Figure 2 , which helps to capture the global context information of multi-scale images of multi-stage input and detail enhanced images. Add a residual connection of mixed attention at the transition stage between the encoder and decoder. The mixed attention consists of interdependent channel attention and spatial attention based on matrix multiplication. Figure 3 In this way, the network can enhance the contrast of features in different channels and capture the spatial dependency of features, giving the model better recognition capabilities, thereby integrating information of different dimensions at different positions of the feature map to solve the problem of channel spatial position information loss that may occur when fusing images multiple times in the encoder stage.

[0045] Each of the four decoder stages outputs a segmentation image, which is then restored to its original 512×512 size using linear interpolation. Vessel segmentation is treated as a pixel-by-pixel classification task, generating a probability map at each pixel. The output of the final decoder stage serves as the overall segmentation image for the network. The outputs of the remaining stages provide additional supervision for network training. Losses are calculated by combining each output with the label. The overall loss function is the average of the sum of the losses at each output.

[0046] (3) Loss function

[0047] During training, the overall loss function used is the average of the loss functions of each classifier:

[0048]

[0049] Where M is the number of decoder stages, here it is 4, L cross-entropy represents the cross entropy loss function, which is defined as follows:

[0050]

[0051] where y i Represents the predicted value of class i, y′ i represents the true label.

[0052] (4) Evaluation indicators

[0053] In order to evaluate the effectiveness of the method of the present invention, the following indicators were used for evaluation. Sensitivity, specificity, area under the receiver operating characteristic curve (AUC) and accuracy. Sensitivity (Se) reflects the proportion of actual vascular pixels correctly identified as blood vessels to all actual vascular pixels, while specificity (Sp) reflects the proportion of actual non-vascular pixels correctly identified as non-vascular to all actual non-vascular pixels. AUC is the area under the receiver operating characteristic curve (ROC curve). AUC reflects the ability to distinguish between blood vessels and non-vasculars. Accuracy (Acc) is the proportion of correct classifications within the field of view. Definitions are as follows:

[0054]

[0055]

[0056]

[0057] Where TP, FN, TN, and FP represent true positive, false negative, true negative, and false positive, respectively, and only pixels within the field of view are counted.

[0058] (5) Results

[0059] The proposed method was evaluated on the DRIVE, STARE and CHASEDB1 datasets based on AUC, Acc, Se and Sp. Figure 5 Partial segmentation results of the present invention on three public data are shown. Figure 4 The segmentation results of the proposed method are compared with those of DF-Net. It can be observed that the segmentation map after feature integration is significantly refined, retaining useful information for vessel segmentation. At the same time, other tissues or features unrelated to vessel segmentation, such as exudates, hemangiomas, and microaneurysms (boxed in the figure), are effectively filtered out.

[0060] To fully demonstrate the effectiveness of the segmentation network used in this paper, the segmentation results of this method are compared with those of commonly used networks in the field of retinal vessel segmentation and tested using the aforementioned evaluation metrics. The comparison results are shown in Table 1. Table 1 shows that the proposed network structure has certain advantages in the task of retinal vessel segmentation.

[0061] Table 1 Comparison of segmentation performance of different networks

[0062]

Claims

1. A global fusion dual-channel retinal vessel segmentation method, characterized in that: The following steps are involved: Step 1: Obtain a retinal vascular image dataset and preprocess the original images and labels; Step 2: resample the preprocessed image into multiple images of different sizes; Step 3: The resampled images of different sizes are fed into the parallel small vessel enhancement network and segmentation network in multiple stages to obtain the retinal vessel segmentation results. The specific process is as follows: 3.

1. Input the preprocessed images of different sizes into multiple parallel small blood vessel enhancement networks to obtain multiple enhanced feature maps of different sizes. The small blood vessel enhancement network uses a Frangi filter based on the Hessian matrix; 3.2, a variable convolution module is added before each convolution layer in the U-Net encoder to build a segmentation network. The variable convolution module consists of a sequentially connected convolution layer, a pooling layer, and a variable convolution layer; The preprocessed images of different sizes are fed into different stages of the encoder and fused with the output feature maps of the previous stage. The enhanced feature maps of different sizes output by the small blood vessel enhancement network are fused with the corresponding feature maps fused at different stages of the encoder and the outputs of different stages of the decoder, and used as the input for the next stage. At the transition stage between the encoder and decoder, a residual connection with hybrid attention constructed by combining spatial attention and channel attention is added; 3.3, output images at multiple stages of the decoder, use linear interpolation to restore them to the image size of n×n, and use the image output of the last stage of the decoder as the retinal blood vessel segmentation result image; Step 4: Design a loss function, calculate the loss between the retinal blood vessel segmentation result image and the label, perform training, and test.

2. The global fusion dual-channel retinal vessel segmentation method according to claim 1, characterized in that: The preprocessing in step 1 includes cropping, random rotation and contrast enhancement.

3. The global fusion dual-channel retinal vessel segmentation method according to claim 1, characterized in that: The largest image size after resampling in step 2 is n×n, followed by And so on to get all the resampled images.

4. The global fusion dual-channel retinal vessel segmentation method according to claim 3, characterized in that: Step 3.1 also includes using the optimizer to denoise the image after Frangi filter enhancement.

5. The global fusion dual-channel retinal vessel segmentation method according to claim 4, characterized in that: The loss function described in step 4 is: the multiple images output after linear interpolation are respectively compared with the labels using the cross entropy loss function to calculate the loss, and the average is added to obtain the total loss function.

Images