A method for optic disc segmentation based on convolutional neural network

Abstract

This invention discloses a convolutional neural network-based optic disc segmentation method, characterized by comprising the following steps: 1) defining a TU-Net network; 2) building a U-Net network; 3) locating the optic disc based on the output annotated image; 4) preprocessing the image; 5) building an AU-Net network; 6) setting a training strategy; 7) setting a loss function; 8) training the network and updating parameters; 9) post-processing the output image; and 10) setting evaluation criteria. This method is simple to implement, highly universal, and can reduce the impact of noise in fundus images.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and image processing, and in particular to a convolutional neural network-based optic disc segmentation method. Background Art

[0002] Retinal image features are complex, and color fundus images often have problems with uneven illumination and central reflection of blood vessels, which makes the contrast between the optic disc boundary and the background low. Existing methods are often affected by external noise, resulting in inaccurate optic disc segmentation. Therefore, it is very important to study an accurate and efficient optic disc segmentation method.

[0003] Since 2010, deep learning has achieved tremendous success in the field of computer vision, with breakthroughs in image semantic segmentation methods based on convolutional neural networks (CNNs). Compared to human observation, CNNs can automatically extract deeper features from images, giving them a clear advantage in image segmentation. Among deep learning-based image segmentation algorithms, U-Net has played a significant role in the field of medical image segmentation. Numerous researchers have proposed numerous U-Net variants based on the U-Net architecture. The literature IEEE transactions on medical imaging, 2019.38(10): p.2281-2292 proposed CE-Net: using multi-branch convolution to extract features of different receptive fields, and migrating ResNet in the encoder to speed up training; Zhuang, J. proposed LadderNet: adding skip connections between each pair of adjacent decoders and each level of decoder branches, and using weight sharing to reduce the number of parameters, but the computational cost was not reduced; Li, H., et al. proposed a pyramid attention network: combining attention mechanism and spatial pyramid to extract accurate features; Al-Bander, B., et al proposed a joint segmentation method for optic disc and optic cup combining a fully convolutional network and a DenseNet network. This method can well segment the optic disc area, but has the disadvantage of long training time. Summary of the Invention

[0004] The present invention aims to address the shortcomings of the prior art and provide a convolutional neural network-based optic disc segmentation method. This method is simple to implement, has strong universality, and can reduce the impact of fundus image noise.

[0005] The technical solution for achieving the purpose of the present invention is:

[0006] A convolutional neural network-based optic disc segmentation method comprises the following steps:

[0007] 1) Define TU-Net network: TU-Net network consists of U-Net network and AU-Net network;

[0008] 2) Building a U-Net network: The network model structure is U-shaped and symmetrical, with an encoding path and a decoding path. The encoding path consists of four Down-Conv modules, each of which has two convolutional layers (Conv) consisting of 3×3 convolutions and ReLU linear activation functions, and a 2×2 maximum pooling layer for downsampling. The decoding path consists of four Up-Conv modules, each of which consists of a deconvolution layer for upsampling and two convolution layers. Each Down-Conv module and Up-Conv module are connected by a skip connection layer. The skip connection layer enables the U-Net network to fuse low-level features and high-level features, so that the final output feature map contains features from different levels of the image, thereby improving the segmentation accuracy of the model. The role of the U-Net network: The U-Net network can roughly locate the position of the optic disc. Using the U-Net network, a segmentation map of the same size as the original image can be obtained.

[0009] 3) Locate the optic disc based on the output labeled image: A feature image is obtained through the U-Net network. This feature image roughly determines the optic disc area. Edge detection is performed on this feature image, and the coordinates of the region center are calculated based on the obtained boundary information. The center coordinates are the approximate position of the optic disc in the original retinal image.

[0010] 4) Cropping the original retinal image based on the coordinate information obtained in step 3) and preprocessing the image, including:

[0011] 4-1) Cropping: Based on the optic disc location information obtained in step 3), a 200x200 pixel image is cropped, with the center of the image being the optic disc area;

[0012] 4-2) Image preprocessing: Figure 4 As shown, first, the color fundus image of the extracted B channel is enhanced using the limited contrast adaptive histogram equalization CLAHE algorithm. In order to better eliminate the influence of the central blood vessels and background noise in the color fundus image, the enhanced image is processed using the morphological closing operation. Finally, the output image and the annotated image corresponding to the input image obtained by cropping in step 4-1) are subjected to polar coordinate transformation.

[0013] 5) Constructing the AU-Net network: The AU-Net network consists of four Res-Conv convolutional blocks in the decoding path and four Up-Conv modules in the encoding path. The Up-Conv module is consistent with the decoding module in the U-Net network in step 1). Compared with the classic U-Net network, the AU-Net uses the Res-Conv convolutional module instead of the traditional encoding module in the encoding path. The Res-Conv module adopts the ResNet network concept, allowing the network to make better use of the learned features. Each Res-Conv module and Up-Conv module are connected by a skip connection layer. The skip connection layer enables the network to fuse low-level features and high-level features, so that the final output feature map contains features from different levels of the image, thereby improving the segmentation accuracy of the model. AU-Net uses attention gates (AGs) in the skip connection. AGs were first proposed by Oktay, O., et al. AGs can suppress feature responses in irrelevant background areas while retaining position information, making the optic disc features more prominent. This allows AU-Net to improve the model's sensitivity and prediction accuracy while maintaining computational efficiency.

[0014] 6) Set the training strategy: Use the SGD optimization algorithm, set the batch size to 4, the training period to 200, and the initial learning rate to 0.001. The AU-Net network can achieve pixel-level segmentation of the input image and accurately segment the optic disc area.

[0015] 7) Setting the loss function: The loss function combining cross entropy and Dice loss is used as the loss function for network training. The optic disc segmentation problem can be regarded as a binary classification problem of pixels, so this paper adopts the cross entropy loss function. The Dice loss function is often used to calculate the similarity between two samples. In order to improve the segmentation accuracy as a whole, the Dice loss function is introduced as part of the loss function. The cross entropy function is shown in formula (1):

[0016]

[0017] where y i is the label of the i-th pixel, the disc is 1, the background is 0, and p i represents the probability that the i-th pixel is predicted to be a optic disc,

[0018] The Dice loss function is shown in formula (2):

[0019]

[0020] Where |A| and |B| represent the number of ground truth and predicted masks respectively. The loss function used in this technical solution is shown in formula (3):

[0021] L Loss =L Cross-Entropy +L Dice (3),

[0022] 8) Train the network and update parameters: Train the AU-Net network according to the training strategy set in step 6). The AU-Net network uses the backpropagation algorithm to update the weights and biases in the AU-Net network, and uses the loss function to dynamically maintain the parameters during the training process.

[0023] 9) Post-processing the output image: First, the output predicted image needs to be inversely transformed into a Cartesian coordinate system, and then the output predicted image size is restored to the original image size based on the positioning information;

[0024] 10) Setting evaluation criteria: In order to evaluate the segmentation effect of the network model, this technical solution uses accuracy Acc, specificity Spe, sensitivity Se and F1-Score for evaluation. Their calculation formulas are shown in formula (4), formula (5), formula (6), formula (7) and formula (8) respectively:

[0025]

[0026]

[0027]

[0028]

[0029]

[0030] TP, TN, FP, and FN represent true positives, true negatives, false positives, and false negatives, respectively. TP represents the number of pixels predicted to be the optic disc and actually are the optic disc; TN represents the number of pixels predicted to be the background and actually are the background; FP represents the number of pixels predicted to be the optic disc and actually are the background; and FN represents the number of pixels predicted to be the background and actually are the optic disc.

[0031] 11) Evaluating the network model: Compare the optic disc segmentation image predicted by this technical solution with the annotated image in the data set in step 1), and evaluate the performance of the model according to the evaluation criteria.

[0032] This technical solution is built on the U-Net neural network framework and provides a new image segmentation network TU-Net. TU-Net consists of a front-end U-Net network and a back-end AU-Net network. The U-Net network is a pre-trained network that can roughly segment the optic disc position. Through the U-Net network, a 200x200 image with the optic disc as the central area can be obtained. After image preprocessing, it enters the AU-Net network. The AU-Net network can accurately segment the optic disc area and output a 200x200 segmented image. Finally, the final optic disc segmentation image is restored using the previous positioning information. AU-Net uses the Res-Conv module provided by this technical solution to replace the original two convolution modules in the downsampling process: Specifically, the Res-Conv module provided by this technical solution will first perform two 1x1 convolutions and batch normalizations on the input feature map A, 1 3x3 convolution, batch normalization and ReLU operations are performed, and the obtained feature map is added to A and then a ReLU operation is performed to obtain feature map B; feature map B is added to feature map B after a 3x3 convolution and batch normalization, and a ReLU operation is performed to obtain feature map C; feature map C enters the next layer after a 2x2 downsampling. Compared with other modules, the Res-Conv module adds a jump connection between the input and output. The combination of the original feature map and the convolved feature map can effectively reduce feature loss and solve the network degradation phenomenon that occurs as the network deepens. In the jump connection part of the original U-Net network, this technical solution provides the use of attention gates AGs. The use of AGs can suppress feature responses in irrelevant background areas while retaining position information, making the optic disc features more prominent, thereby allowing AU-Net to improve the sensitivity and prediction accuracy of the model while maintaining computational efficiency.

[0033] The advantages of this technical solution are:

[0034] 1. This technical solution uses U-Net to locate the optic disc area, which can effectively eliminate the noise impact of non-optic disc areas;

[0035] 2. This technical solution uses the AU-Net network: AU-Net uses the Res-Conv convolution module in the encoding path instead of the traditional encoding module. The Res-Conv module is designed based on the ResNet network concept, allowing the network to make better use of learned features. Each Res-Conv module and Up-Conv module are connected by a skip connection layer. The skip connection layer enables the network to fuse low-level features with high-level features, so that the final output feature map contains features from different levels of the image, thereby improving the model's segmentation accuracy. AU-Net introduces attention gates (AGs) on the skip connection. Using AGs, it can suppress feature responses in irrelevant background areas while retaining position information, making the optic disc features more prominent. This allows AU-Net to improve the model's sensitivity and prediction accuracy while maintaining computational efficiency.

[0036] 3. This technical solution adopts single channel extraction, polar coordinate transformation and CLAHE image preprocessing methods, which can effectively improve the contrast between the optic disc area and the background, thereby improving the accuracy of optic disc segmentation.

[0037] This method is simple to implement, has strong universality, and can reduce the impact of fundus image noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Schematic diagram of the TU-Net network structure in the embodiment;

[0039] Figure 2 Schematic diagram of the AU-Net network structure in the embodiment;

[0040] Figure 3 Schematic diagram of the Res-Conv module structure in the embodiment;

[0041] Figure 4 Schematic diagram of image preprocessing results in the embodiment; DETAILED DESCRIPTION

[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments, but the present invention is not limited thereto.

[0043] Example:

[0044] A convolutional neural network-based optic disc segmentation method comprises the following steps:

[0045] 1) Define the TU-Net network: Figure 1 As shown, the TU-Net network consists of the U-Net network and the AU-Net network;

[0046] 2) Build a U-Net network: The network model structure is U-shaped and symmetrical, with an encoding path and a decoding path. The encoding path consists of four Down-Conv modules, each of which has two convolutional layers Conv consisting of 3×3 convolution and ReLU linear activation functions and a 2×2 maximum pooling layer for downsampling. The decoding path consists of four Up-Conv modules, each of which consists of a deconvolution layer for upsampling and two convolution layers. Each Down-Conv module and Up-Conv module are connected by a skip connection layer. The skip connection layer enables the U-Net network to fuse low-level features and high-level features, so that the final output feature map contains features at different levels of the image, thereby improving the segmentation accuracy of the model; The role of the U-Net network: The U-Net network can roughly locate the position of the optic disc, and the U-Net network can obtain a segmentation map of the same size as the original image;

[0047] 3) Locate the optic disc based on the output labeled image: A feature image is obtained through the U-Net network. This feature image roughly determines the optic disc area. Edge detection is performed on this feature image, and the coordinates of the region center are calculated based on the obtained boundary information. These coordinates are the approximate location of the optic disc in the original retinal image.

[0048] 4) Cropping the original retinal image based on the coordinate information obtained in step 3) and preprocessing the image, including:

[0049] 4-1) Cropping: Based on the optic disc location information obtained in step 3), a 200x200 pixel image is cropped, with the center of the image being the optic disc area;

[0050] 4-2) Image Preprocessing: First, the color fundus image extracted from the B channel is enhanced using the contrast-constrained adaptive histogram equalization (CLAHE) algorithm. To better eliminate the effects of central blood vessels and background noise in the color fundus image, the enhanced image is processed using a morphological closing operation. Finally, the output image and the annotated image corresponding to the input image obtained by cropping in step 4-1) are subjected to polar coordinate transformation.

[0051] 5) Construct AU-Net network: Figure 2 As shown in Figure 2, the AU-Net network consists of four Res-Conv convolution blocks forming a decoding path and four Up-Conv modules forming an encoding path. The Up-Conv module is consistent with the decoding module in the U-Net network in step 1). Compared with the classic U-Net network, the AU-Net uses the Res-Conv convolution module in the encoding path instead of the traditional encoding module. The Res-Conv module is as follows: Figure 3As shown in the figure, the Res-Conv module adopts the ResNet network concept, allowing the network to make better use of the learned features. Each Res-Conv module and Up-Conv module are connected by a skip connection layer. The skip connection layer enables the network to fuse low-level features and high-level features, so that the final output feature map contains features at different levels of the image, thereby improving the segmentation accuracy of the model. AU-Net uses attention gates AGs on the skip connection. AGs were first proposed by Oktay, O., et al. AGs can suppress feature responses in irrelevant background areas while retaining position information, making the optic disc features more prominent. This allows AU-Net to improve the sensitivity and prediction accuracy of the model while maintaining computational efficiency.

[0052] 6) Set the training strategy: Use the SGD optimization algorithm, set the batch size to 4, the training period to 200, and the initial learning rate to 0.001. The AU-Net network can achieve pixel-level segmentation of the input image and accurately segment the optic disc area.

[0053] 7) Setting the loss function: The loss function combining cross entropy and Dice loss is used as the loss function for network training. The optic disc segmentation problem can be regarded as a binary classification problem of pixels, so this paper adopts the cross entropy loss function. The Dice loss function is often used to calculate the similarity between two samples. In order to improve the segmentation accuracy as a whole, the Dice loss function is introduced as part of the loss function. The cross entropy function is shown in formula (1):

[0054]

[0055] where y i is the label of the i-th pixel, the disc is 1, the background is 0, and p i represents the probability that the i-th pixel is predicted to be a optic disc,

[0056] The Dice loss function is shown in formula (2):

[0057]

[0058] Where |A| and |B| represent the number of ground truth and predicted masks respectively. The loss function used in this example is shown in formula (3):

[0059] L Loss =L Cross-Entropy +L Dice (3),

[0060] 8) Train the network and update parameters: Train the AU-Net network according to the training strategy set in step 6). The AU-Net network uses the backpropagation algorithm to update the weights and biases in the AU-Net network, and uses the loss function to dynamically maintain the parameters during the training process.

[0061] 9) Post-processing the output image: First, the output predicted image needs to be inversely transformed into a Cartesian coordinate system, and then the output predicted image size is restored to the original image size based on the positioning information;

[0062] 10) Setting evaluation criteria: In order to evaluate the segmentation effect of the network model, this example uses accuracy Acc, specificity Spe, sensitivity Se and F1-Score for evaluation. Their calculation formulas are shown in formula (4), formula (5), formula (6), formula (7) and formula (8) respectively:

[0063]

[0064]

[0065]

[0066]

[0067]

[0068] TP, TN, FP, and FN represent true positives, true negatives, false positives, and false negatives, respectively. TP represents the number of pixels predicted to be the optic disc and actually are the optic disc; TN represents the number of pixels predicted to be the background and actually are the background; FP represents the number of pixels predicted to be the optic disc and actually are the background; and FN represents the number of pixels predicted to be the background and actually are the optic disc.

[0069] 11) Evaluate the network model: Compare the optic disc segmentation image predicted by the method in this example with the annotated image in the dataset in step 1) and evaluate the performance of the model according to the evaluation criteria.

Claims

1. A method for optic disc segmentation based on convolutional neural network, characterized in that: The steps include: 1) Define TU-Net network: TU-Net network consists of U-Net network and AU-Net network; 2) Building a U-Net network: The network model has a U-shaped symmetrical structure with an encoding path and a decoding path. The encoding path consists of four Down-Conv modules, each of which has two Conv convolution layers consisting of 3×3 convolutions and ReLU linear activation functions, and a 2×2 max pooling layer for downsampling. The decoding path consists of four Up-Conv modules, each of which consists of a deconvolution layer for upsampling and two convolution layers. Each Down-Conv module and Up-Conv module is connected by a skip connection layer. The skip connection layer enables the U-Net network to fuse low-level features with high-level features, so that the final output feature map contains features from different levels of the original input image. 3) Locate the optic disc based on the output annotated image: A feature image is obtained through the U-Net network. This feature image roughly determines the optic disc area. Edge detection is performed on this feature image. The center coordinates of the area are calculated based on the obtained boundary information. The center coordinates are the approximate position of the optic disc in the original retinal image. 4) Cropping the retinal image and the annotated image corresponding to the image in the data set according to the coordinate information obtained in step 3) and preprocessing the image, including: 4-1) Cropping: Based on the optic disc location information obtained in step 3), a 200x200 pixel image is cropped, with the center of the image being the optic disc area; 4-2) Image Preprocessing: First, the color fundus image extracted from the B channel is enhanced using the contrast-constrained adaptive histogram equalization (CLAHE) algorithm. The enhanced image is then processed using a morphological closing operation. Finally, the output image and the annotated image corresponding to the input image obtained by cropping in step 4-1) are subjected to polar coordinate transformation. The image preprocessing process is shown in Figure 4. 5) Build the AU-Net network: The AU-Net network consists of four Res-Conv convolutional blocks in the decoding path and four Up-Conv modules in the encoding path. The Up-Conv modules are the same as the decoding modules in the U-Net network in step 1). In the encoding path, AU-Net uses Res-Conv convolutional modules instead of traditional encoding modules. The Res-Conv modules adopt the ResNet network concept. Each Res-Conv module and Up-Conv module are connected by a skip connection layer. The skip connection layer enables the network to fuse low-level features with high-level features, so that the final output feature map contains features from different levels of the image. AU-Net uses attention gates (AGs) in the skip connections. 6) Set the training strategy: Use the SGD optimization algorithm, set the batch size to 4, the training period to 200, and the initial learning rate to 0.001; 7) Setting the loss function: A loss function combining cross entropy and Dice loss is used as the loss function for network training. The optic disc segmentation problem is regarded as a binary classification problem of pixels. The Dice loss function is introduced as part of the loss function. The cross entropy function is shown in formula (1): where y i is the label of the i-th pixel, the disc is 1, the background is 0, and p i represents the probability that the i-th pixel is predicted to be a optic disc, and the Dice loss function is shown in formula (2): Where |A| and |B| represent the number of ground truth and predicted masks respectively, and the loss function is shown in formula (3): 8) Train the network and update parameters: Train the AU-Net network according to the training strategy set in step 6). The AU-Net network uses the backpropagation algorithm to update the weights and biases in the AU-Net network, and uses the loss function to dynamically maintain the parameters during the training process. 9) Post-processing the output image: First, the output predicted image needs to be inversely transformed into a Cartesian coordinate system, and then the output predicted image size is restored to the original image size based on the positioning information; 10) Setting evaluation criteria: The accuracy Acc, specificity Spe, sensitivity Se and F1-Score are used to evaluate the segmentation effect of the network model. Their calculation formulas are shown in formula (4), formula (5), formula (6), formula (7) and formula (8) respectively: TP, TN, FP, and FN represent true positives, true negatives, false positives, and false negatives, respectively. TP represents the number of pixels predicted to be the optic disc and actually are the optic disc; TN represents the number of pixels predicted to be the background and actually are the background; FP represents the number of pixels predicted to be the optic disc and actually are the background; and FN represents the number of pixels predicted to be the background and actually are the optic disc. 11) Evaluating the network model: Compare the optic disc segmentation image predicted by this technical solution with the annotated image in the data set in step 1), and evaluate the performance of the model according to the evaluation criteria.

Images