A method for joint segmentation of optic cup and disc in retinal fundus images

Abstract

The present invention discloses a method for jointly segmenting the optic cup and optic disc in retinal fundus images, belonging to the field of medical image processing. The present invention combines the elliptical morphological features of the optic cup and optic disc to construct a two-stage optic cup and optic disc joint segmentation network model from the perspective of ellipse detection. In the first stage, the optic cup is located within the optic disc region, and the Paired-Box RPN is introduced to achieve coupled detection of the minimum bounding rectangle of the optic cup and optic disc. In the second stage, the five parameters of the ellipse in the bounding box region are learned. Based on the elliptical morphological features and spatial geometric constraints of the optic cup and optic disc, the optic cup and optic disc are jointly segmented using the optic cup and optic disc joint segmentation network model. This method can solve the problem of uneven segmentation edges and achieve accurate optic cup and optic disc segmentation.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a method for jointly segmenting optic cup and optic disc in retinal fundus images. Background Art

[0002] In color fundus images, such as Figure 1 As shown, the optic disc appears as an elliptical area in the box, the optic cup is a bright central depression of variable size on the optic disc, and the vertical cup-to-disc ratio (vCDR) is the ratio of the vertical diameter of the optic cup (VCD) to the vertical diameter of the optic disc (VDD).

[0003] Because manual segmentation of the optic cup and optic disc is extremely time-consuming, many automatic optic cup and optic disc segmentation methods have been proposed. Existing optic cup and optic disc segmentation algorithms can be divided into two categories: traditional segmentation algorithms and neural network segmentation algorithms.

[0004] Traditional segmentation algorithms typically extract features based on color, shape, gradient information, and the curvature of blood vessels near the cup and disc. They use thresholds to determine the boundary and then post-process the boundary using methods such as ellipse fitting to obtain a more accurate cup-disc boundary. While simple and easy to use, traditional segmentation algorithms utilize only a small amount of local information, are easily affected by pathological areas, lesion occlusion, and image contrast quality, and have low robustness.

[0005] In recent years, researchers have introduced deep learning-based methods to optic disc and cup segmentation. The most common approach is to treat the segmentation problem as a pixel-level classification task. Unified Networks (UNets) have been widely used in medical image processing due to their ability to achieve good results with relatively small amounts of training data. For example, Fu et al. proposed MNet, which incorporates a multi-scale image pyramid input and multi-scale output to learn discriminative features, thereby improving the performance of optic cup and disc segmentation. MNet also uses polar coordinate transformation to transform the internal and external geometric constraints of the cup and disc into a hierarchical structure for segmentation. Furthermore, researchers have used other anatomical information from the fundus, such as the location of the macula, as auxiliary tasks to improve the accuracy of cup and disc segmentation. Compared to traditional segmentation methods, deep learning-based methods have achieved better segmentation results. However, existing methods fail to fully exploit the prior knowledge of the elliptical morphology of the optic cup and disc, resulting in rough segmentation boundaries and prone to missegmentation in the presence of numerous lesions and atrophic arcs. Furthermore, these methods often require complex post-processing operations such as ellipse fitting and maximum connected domain selection to improve segmentation accuracy, making it difficult for existing methods to achieve accurate segmentation results. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for jointly segmenting the optic cup and optic disc in retinal fundus images, which can solve the problem of uneven segmentation edges and achieve accurate segmentation of the optic cup and optic disc.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] A method for jointly segmenting an optic cup and optic disc in a retinal fundus image, comprising:

[0009] Based on elliptical morphological features and spatial geometric constraints, a network model for joint segmentation of the optic cup and optic disc is constructed. The network model comprises two stages: in the first stage, a paired-box RPN network is used to perform coupled detection of the minimum bounding rectangle of the optic cup and optic disc, taking into account the spatial geometric constraint that the optic cup is located within the optic disc area; in the second stage, elliptical parameters of the bounding rectangle area are determined; and finally, the optic cup and optic disc segmentation result is output based on the minimum bounding rectangle of the optic cup and optic disc detected by coupled detection in the first stage and the elliptical parameters of the second stage.

[0010] Obtaining a retinal color fundus image sample set and the ellipse fitting parameters of the optic cup and optic disc in each retinal color fundus image sample;

[0011] training the optic cup and optic disc joint segmentation network model according to the retinal color fundus image sample set and the ellipse fitting parameters of the optic cup and optic disc in each retinal color fundus image sample to obtain an optimal optic cup and optic disc segmentation network model;

[0012] The retinal color fundus image to be detected is input into the optimal optic cup and optic disc segmentation network model, and the optic cup and optic disc segmentation result image is output.

[0013] Optionally, the optic cup and optic disc joint segmentation network model includes: a Faster RCNN front-end network, an edge attention module, a Paired-Box RPN network, a first RoI-Align pooling layer, a second RoI-Align pooling layer, a rectangular box predictor, an ellipse predictor, and post-processing operations;

[0014] The Faster RCNN front-end network uses ResNet50-FPN as the backbone network, and the ResNet50 in the ResNet50-FPN is used to extract feature maps from the input retinal color fundus images;

[0015] The edge attention module uses the intermediate output feature layer of the C2-C5 stage of ResNet50 in ResNet50-FPN as input, unifies the channels to 256 through a 3×3 convolution layer, and uses a 1×1 convolution layer and a sigmoid function to map the boundary features to the edge map. Subsequently, all the scales of the edge features are resized to 1 / 8 of the input retinal color fundus image, and all the rescaled edge features are concatenated together. Finally, the concatenated edge features are resized and output;

[0016] The FPN in the ResNet50-FPN is used to element-wise add the edge features output by the edge attention module to each feature layer of the feature map extracted by ResNet50, and output a multi-layer feature layer;

[0017] The Paired-Box RPN network, the first RoI-Align pooling layer, and the rectangle predictor constitute the first stage;

[0018] The Paired-Box RPN network is used to perform coupled detection on the minimum bounding rectangle of the optic cup and optic disc based on multiple feature layers to obtain paired candidate boxes of the optic cup and optic disc; the paired candidate boxes include positive sample paired candidate boxes and negative sample paired candidate boxes; the first RoI-Align pooling layer is used to pool the paired candidate boxes of the optic cup and optic disc into a optic cup feature layer and an optic disc feature layer; the rectangle predictor uses two fully connected layers to perform classification and regression prediction on the optic cup feature layer and the optic disc feature layer, and outputs the coordinate value of the minimum bounding rectangle of the optic cup and its confidence level, and the coordinate value of the minimum bounding rectangle of the optic disc and its confidence level;

[0019] The second RoI-Align pooling layer and the ellipse predictor constitute the second stage;

[0020] The Paired-Box RPN network is also used to input the output positive sample paired candidate boxes into the FPN in ResNet50-FPN, and output a feature layer containing the candidate box position information;

[0021] The second RoI-Align pooling layer is used to expand the rectangular candidate box area in the feature layer containing the candidate box position information into a square, and output a corrected feature layer;

[0022] The ellipse predictor uses two fully connected layers to predict the classification and regression prediction quantities based on the modified feature layer and output the predicted ellipse fitting parameters;

[0023] The post-processing operation is used to output the optic cup and optic disc segmentation result according to the confidence of the minimum bounding rectangle box of the optic cup, the confidence of the minimum bounding rectangle box of the optic disc and the predicted ellipse fitting parameters.

[0024] Optionally, the output layer of the Paired-Box RPN network contains a generator 8-dimensional vector (R d ,R c ) has a regression branch and a classification branch S; where R d and R c The 4-dimensional minimum rectangular box regression vectors (f x ,f y ,f w ,f h ) and (v x ,v y ,v w ,v h ); S is the 2D vector after Softmax normalization (S + ,S - ), S + and S - Represent the probability that the anchor box belongs to the foreground and background respectively; the anchor box is the preset optic cup and optic disc border;

[0025] The process of assigning anchor box positive samples to the Paired-Box RPN network is as follows: if the anchor box A satisfies the IoU (A, D) ≥ α1 with the optic disc and the IoF (A, C) ≥ α2 with the optic cup, then the anchor box A is considered to be a positive sample match; where Q = (D, C) is the rectangular annotation box pair consisting of the rectangular bounding boxes of the optic disc and the optic cup, α1 and α2 are the anchor box positive sample assignment thresholds respectively;

[0026] The Paired-Box RPN network selects balanced negative samples as follows: for each negative sample sampling layer, first randomly sample from the cup-disc pairs whose two IoUs both meet the negative sample sampling threshold; then randomly sample from the cup-disc pairs whose one of the two IoUs meets the negative sample sampling threshold;

[0027] The matching process of the Paired-Box RPN network for the positive sample candidate region is as follows: the loss calculation is performed on the positive and negative samples, so that the Paired-Box RPN simultaneously generates the candidate region pair of the optic cup and optic disc X = (P d ,P c ); If the candidate region pair X=(P d ,P c ) satisfies IoU(P d ,D)≥γ1 and IoF(P c ,C)≥β2, it is considered as a positive sample candidate area; where β1 and β2 are the positive sample allocation thresholds for the optic disc and optic cup candidate areas respectively.

[0028] Optionally, expanding the rectangular candidate box area in the feature layer containing the candidate box position information into a square includes:

[0029] Expand outward along the two longer sides of the rectangular candidate box area so that the shorter side is equal to the longer side, and fill the extended part with 0.

[0030] Optionally, the ellipse fitting parameters include: center point coordinates (c x ,c y ), the major semi-axis a, the minor semi-axis b and the rotation angle θ;

[0031] Among them, θ∈(-π / 2,π / 2] represents the angle from the positive x-axis to the major axis of the ellipse; the width of the axis-aligned minimum bounding rectangle of the ellipse Height is The minimum bounding rectangle of an ellipse is represented by (c x -w / 2,c y -h / 2,c x +w / 2,c y +h / 2).

[0032] Optionally, the calculation formula for the predicted ellipse fitting parameters is:

[0033]

[0034] Among them, (P x1 ,P x2 ,P y1 ,P y2 ) is the coordinate representation of a square candidate region in the modified feature layer, (c' x ,c' y ) is the predicted center point coordinate, a' is the predicted major semi-axis, b' is the predicted minor semi-axis, θ' is the predicted rotation angle, P l To correct the diagonal length of a square candidate region in the feature layer,

[0035] Optionally, the total loss function of the optic cup and optic disc joint segmentation network model is

[0036] Where λ1, λ2 and λ3 are the first, second and third weighted weights respectively;

[0037] is the loss function of the Paired-Box RPN network, is the binary cross entropy classification loss for distinguishing background and foreground, and Represent the regression loss of the optic disc and optic cup with the corresponding anchor box, and Smooth L1 loss is used in both cases;

[0038] is the network head loss function, represents the classification loss, represents the rectangular candidate box regression loss, represents the ellipse parameter regression loss, γ1 and γ2 are the fourth and fifth weighted weights respectively;

[0039] is the marginal loss function, e i is the binary edge label generated by the optic cup and optic disc annotation mask, is the edge prediction probability obtained by the edge attention module.

[0040] Optionally, the ellipse fitting parameters are characterized as a two-dimensional Gaussian distribution for parameter coupling, and the ellipse fitting parameters and the two-dimensional Gaussian distribution are The conversion relationship is: μ=(c x ,c y ) T ,Σ=RΛR T ; where μ is the mean of the variable, Σ is the covariance matrix,

[0041] Prediction ellipse With target ellipse Ellipse parameter regression loss Expressed as in,

[0042] Optionally, the acquiring of the retinal color fundus image sample set and the ellipse fitting parameters of the optic cup and optic disc in each retinal color fundus image sample specifically includes:

[0043] Obtain a known set of retinal color fundus image samples;

[0044] According to the gold standard mask of the optic cup and optic disc annotated in each retinal fundus image sample in the retinal color fundus image sample set, the edge extraction and ellipse fitting of the optic cup and optic disc in each retinal fundus image sample are performed to obtain the ellipse fitting parameters of the optic cup and optic disc in each retinal color fundus image sample.

[0045] Optionally, the step of obtaining a retinal color fundus image sample set and ellipse fitting parameters of the optic cup and disc in each retinal color fundus image sample further includes:

[0046] Preprocessing the retinal fundus image samples in the retinal color fundus image sample set to increase the number and diversity of samples in the retinal color fundus image sample set; the preprocessing includes horizontal and vertical flipping with a random probability of 0.5, rotation of a random angle of 0-180, scaling at a random scale of 0.8-1.2, and random color dithering, maintaining the aspect ratio of the image and scaling the shortest side to 800 resolution and the longest side not exceeding 1333 resolution;

[0047] The ellipse fitting parameters of the optic cup and optic disc in the preprocessed retinal fundus image samples are adjusted accordingly.

[0048] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0049] The present invention discloses a method for jointly segmenting the optic cup and optic disc in retinal fundus images. Combining the elliptical morphological features of the optic cup and optic disc, a two-stage optic cup and optic disc joint segmentation network model is constructed from the perspective of ellipse detection. In the first stage, a paired-box RPN is introduced to achieve coupled detection of the minimum bounding rectangular box of the optic cup and optic disc, taking into account the spatial geometric constraint that the optic cup is located inside the optic disc area. In the second stage, five parameters of the ellipse in the bounding box area are learned. Furthermore, based on the elliptical morphological features and spatial geometric constraints of the optic cup and optic disc, the optic cup and optic disc are jointly segmented using the optic cup and optic disc joint segmentation network model. This method can solve the problem of uneven segmentation edges and achieve accurate segmentation of the optic cup and optic disc. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 This is the existing original color fundus diagram;

[0052] Figure 2 A flowchart of a method for jointly segmenting the optic cup and optic disc in a retinal fundus image provided by an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of a method for jointly segmenting the optic cup and optic disc in a retinal fundus image provided by an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of the edge attention module structure provided by an embodiment of the present invention;

[0055] Figure 5Schematic diagram of a Paired-Box RPN network provided by an embodiment of the present invention;

[0056] Figure 6 A schematic diagram of correcting a rectangular feature candidate region provided by an embodiment of the present invention;

[0057] Figure 7 A schematic diagram of an ellipse representation of the optic cup and disc provided in an embodiment of the present invention;

[0058] Figure 8 A schematic diagram of a loss function provided by an embodiment of the present invention;

[0059] Figure 9 This is a diagram showing the effect of joint segmentation of the optic cup and optic disc provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0061] The purpose of the present invention is to provide a method for jointly segmenting the optic cup and optic disc in retinal fundus images, which can solve the problem of uneven segmentation edges and achieve accurate segmentation of the optic cup and optic disc.

[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] The embodiment of the present invention provides a method for jointly segmenting the optic cup and optic disc in a retinal fundus image. Figure 2 Shown, including:

[0064] Step S1, constructing a network model for joint segmentation of the optic cup and optic disc based on elliptical morphological features and spatial geometric constraints; the network model for joint segmentation of the optic cup and optic disc includes two stages: in the first stage, a paired-box RPN network is used to perform coupled detection of the minimum bounding rectangle of the optic cup and optic disc in combination with the spatial geometric constraint that the optic cup is located inside the optic disc area; in the second stage, elliptical parameters of the bounding rectangle area are determined; and finally, a optic cup and optic disc segmentation result is output based on the minimum bounding rectangle of the optic cup and optic disc detected by coupled detection in the first stage and the elliptical parameters of the second stage.

[0065] like Figure 3As shown in the figure, the network model uses ResNet50-FPN as the backbone network and Faster RCNN as the front-end network, which includes two stages: the first stage combines the spatial geometric constraint that the optic cup is located inside the optic disc area, introduces Paired-BoxRPN to realize the coupled detection of the minimum bounding rectangle of the optic cup and optic disc, and the second stage learns the five parameters of the ellipse of the bounding box area to achieve accurate segmentation of the optic cup and optic disc.

[0066] In order to improve the distinction between the optic cup and optic disc in the skeleton network feature map, an edge extraction module is introduced in the ResNet50-FPN part to extract the edge features of the optic cup and optic disc, and the edge features are used as important guidance for the aggregation of contextual semantic features. The specific implementation is as follows:

[0067] like Figure 4 As shown, the edge attention module uses the intermediate output feature layer of the C2-C5 stage of ResNet50 as input, unifies the channels to 256 through a 3×3 convolutional layer, and uses a 1×1 convolutional layer and a sigmoid function to map the boundary features to the edge map. All edge features are then resized to 1 / 8 the size of the input image and concatenated. Finally, they are resized to the corresponding size and element-wise added to each feature layer of the FPN to enhance the edge features of the feature map. Furthermore, an edge loss is added to provide deep supervision of all edge feature maps. The resulting multi-layer feature layer is input to the Paired-Box RPN to extract rectangular candidate boxes for the optic cup and disc.

[0068] like Figure 5 As shown in Figure 2, the specific framework of Paired-Box RPN adopts the RPN of FasterR-CNN. The difference is the output layer. For the same anchor box, the output layer of Paired-BoxRPN contains a generator that generates an 8-dimensional vector (R d ,R c ) has a regression branch and a classification branch S, where R d and R c The 4-dimensional minimum rectangular box regression vectors (f x ,f y ,f w ,f h ) and (v x ,v y ,v w ,v h ); S is the 2D vector after Softmax normalization (S + ,S -), representing the probability that the anchor box belongs to the foreground or background. The regression branch simultaneously predicts the offsets of the two minimum rectangular boxes of the optic disc and the optic cup. Anchor boxes are essentially predefined bounding boxes, a set of preset bounding boxes. During training, training samples are constructed based on the offsets of the actual bounding box positions relative to the preset bounding boxes. This is equivalent to first roughly "framing" the target at the possible locations with the preset bounding boxes, and then making adjustments based on these preset bounding boxes.

[0069] For the assignment of anchor box positive samples, the rectangular bounding boxes of the optic disc and the optic cup are combined into a rectangular annotation box pair Q = (D, C). For an anchor box A, if it satisfies the IoU (A, D) ≥ α1 with the optic disc and the IoF (A, C) ≥ α2 with the optic cup, then the anchor box is considered to be a positive sample match, where α1 and α2 are the thresholds for assigning positive anchor box samples. Therefore, the loss calculation is performed on the positive and negative samples, so that the Paired-Box RPN generates the candidate region pair of the optic cup and optic disc at the same time X = (P d ,P c ). In order to obtain high-quality positive and negative sample candidate region pairs to obtain stable and accurate prediction, for the annotation box pair Q=(D,C), if the candidate region pair X=(P d ,P c ) satisfies IoU(P d ,D)≥β1 and IoF(P c ,C)≥β2, it is considered a positive sample, where β1 and β2 are the positive sample allocation thresholds for the candidate regions of the optic disc and cup, respectively. For the selection of balanced negative samples, modifications are made based on IoU-balanced sampling: for each negative sample sampling layer, first, random sampling is performed from the optic cup and disc pairs whose two IoUs both meet the negative sample sampling threshold; then, random sampling is performed from the optic cup and disc pairs whose one of the two IoUs meets the negative sample sampling threshold. The optic cup and optic disc candidate regions of the selected positive and negative samples are respectively input into the RoiAlign layer and pooled into a 256-dimensional 7×7 feature layer. The fully connected network is used to perform classification and regression prediction to fine-tune the rectangular candidate boxes, and loss calculation is performed to update the model parameters.

[0070] The previous paragraph includes two parts of positive sample judgment. The first positive sample judgment is to set a threshold based on the proposed IoU(a,b) and IoF(a,b) to determine whether the anchor box is a positive sample; the second positive sample judgment is to set a threshold based on the IoU(a,b) and IoF(a,b) to determine whether the initially predicted candidate area is a positive sample. The difference between the two positive sample judgment (also called matching) standards is the different thresholds.

[0071] In the second stage, the ellipse parameters of the optic cup and optic disc are predicted in the fine-tuned positive sample candidate area. Since each feature area also contains the ellipse direction information. If the rectangular candidate area feature map output by the ellipse RPN is directly input into the RoiAlign layer, the rectangular feature area is adjusted to a square of fixed size, which will cause the distortion of the ellipse angle and make the prediction of the original ellipse rotation angle information unstable. Therefore, the rectangular candidate area is corrected. Figure 6 As shown in the figure, in order to maintain the rotation angle information within the feature area and introduce as few background features as possible, the rectangular candidate area feature map is expanded into a square, the longer side of the rectangular candidate area feature map is selected as the side length of the circumscribed axis aligned square, and the extended part is filled with 0.

[0072] The ellipse predictor is designed using a fully connected network to predict the classification and regression prediction quantities. The center point of the candidate region and the center point of the ellipse are regarded as the same point. Therefore, the ellipse predictor only needs to perform the ellipse major axis offset δa, the minor axis offset δb and the rotation angle prediction value δθ.

[0073] Therefore, for a square candidate region P = (P x1 ,P x2 ,P y1 ,P y2 ), the predicted ellipse parameters are calculated as follows:

[0074]

[0075] Among them, P l is the diagonal length of the expanded square candidate region P,

[0076] Step S2: Obtain a set of retinal color fundus image samples and the ellipse fitting parameters of the optic cup and optic disc in each retinal color fundus image sample.

[0077] A known dataset of retinal fundus images was obtained. Edge extraction and ellipse fitting were performed on the gold-standard mask of the optic cup and optic disc annotated by an ophthalmologist. Ellipse fitting parameters for the retinal color fundus image and its corresponding optic cup and optic disc were obtained. The "gold-standard mask," technically known as the 'ground truth mask,' is essentially the result of a segmentation of the optic cup and optic disc annotations performed by an ophthalmologist on the fundus images in the dataset. Edge extraction and ellipse fitting were implemented using the cv2.findContours and cv2.fitEllipse functions in the OpenCV image processing library.

[0078] In specific implementation, the specific ellipse parameter representation includes: Figure 7 As shown, the optic cup and optic disc are represented by the center point coordinates (cx ,c y ), the major semi-axis a and the minor semi-axis b (a ≥ b), and the rotation angle θ are used to characterize the ellipse, where θ∈(-π / 2,π / 2] represents the angle from the positive semi-axis of the x-axis to the major axis of the ellipse, with clockwise rotation being positive and counterclockwise rotation being negative. The minimum bounding box width of the axis-aligned ellipse Height is The minimum bounding box of the ellipse is expressed as (c x -w / 2,c y -h / 2,c x +w / 2,c y +h / 2).

[0079] Step S3, training the optic cup and optic disc joint segmentation network model according to the retinal color fundus image sample set and the ellipse fitting parameters of the optic cup and optic disc in each retinal color fundus image sample to obtain the optimal optic cup and optic disc segmentation network model.

[0080] The preprocessed training images are fed into the network for training, and the parameters are iteratively updated to obtain the optimal model parameters. Specifically, the training is performed using the following steps:

[0081] The loss function of the model includes three types of losses: Paired-Box RPN loss Network head loss and marginal loss

[0082] Paired-Box RPN loss for: in is the binary cross entropy classification loss for distinguishing background and foreground, and Represent the regression losses of the optic disc and optic cup with the corresponding anchor boxes, respectively, using Smooth L1 loss.

[0083] like Figure 8 As shown, the network head loss It consists of three parts: classification loss Ellipse rectangle candidate box regression loss and ellipse parameter regression loss in and Same as in Faster RCNN. In order to solve the problem that the ellipse prediction accuracy drops sharply due to a slight offset of one parameter when optimizing the five parameters of the ellipse independently, the five parameters of the ellipse are represented as a two-dimensional Gaussian distribution for parameter coupling. The conversion relationship is: μ=(cx ,c y ) T ,Σ=RΛR T ,in The predicted ellipse With target ellipse Elliptic loss function Expressed as in Therefore, the network head loss Expressed as Where γ1 and γ2 are weighted weights.

[0084] For the edge attention module, binary cross entropy loss is used for deep supervision, and edge loss Expressed as where e i is the binary edge label generated by the optic cup and optic disc annotation mask; is the edge prediction probability obtained by the edge attention module.

[0085] The total loss function is Where λ1, λ2 and λ3 are weighted weights respectively.

[0086] The network parameters are updated through back propagation, and the network model is trained repeatedly until the model converges.

[0087] Step S4: inputting the retinal color fundus image to be detected into the optimal optic cup and optic disc segmentation network model, and outputting an optic cup and optic disc segmentation result map.

[0088] For the test set / color fundus image to be tested, the network model and the corresponding optimal model parameters are used for detection to obtain the optic cup and optic disc segmentation results. Figure 9 shown.

[0089] The present invention combines the elliptical morphological features of the optic cup and disc to construct a two-stage network model from the perspective of ellipse detection for joint segmentation of the optic cup and disc. In the first stage, the spatial geometric constraint of the optic cup located within the optic disc region is combined with the introduction of a paired-box RPN to achieve coupled detection of the minimum bounding rectangle of the optic cup and disc. In the second stage, the five parameters of the ellipse in the bounding box region are learned, the ellipse is represented as a two-dimensional Gaussian distribution, and the five parameters are coupled. The Kullback-Leibler divergence loss function is introduced for loss supervision to achieve accurate segmentation of the optic cup and disc. In addition, the network introduces an edge attention module to extract edge features of the optic cup and disc to improve the discrimination between the two, and uses edge features as an important guide for context aggregation to improve the accuracy of the optic cup and disc. The method of the present invention explicitly utilizes the elliptical morphological features and spatial geometric constraints of the optic cup and disc to solve the problem of uneven segmentation edges.

[0090] This method explicitly utilizes the elliptical morphological characteristics and spatial geometric constraints of the optic cup and disc to segment the optic cup and disc. This segmentation allows for the deriving of the vertical cup-to-disc ratio, a key diagnostic parameter for glaucoma. This enables high-precision segmentation and glaucoma diagnosis, even in the presence of numerous lesions and atrophic arcs adjacent to the optic disc. The vertical cup-to-disc ratio is the ratio of the vertical diameter of the optic cup to the vertical diameter of the optic disc.

[0091] The characteristics of the combined optic cup and optic disc segmentation method based on elliptical morphological features and spatial geometric constraints established in the present invention are as follows:

[0092] 1. This paper combines the elliptical morphological features of the optic cup and optic disc, redefines the optic cup and optic disc segmentation task from the perspective of ellipse detection, and explicitly uses morphological features to detect the optic cup and optic disc and directly measure glaucoma diagnostic indicators. In addition, the ellipse is represented as a two-dimensional Gaussian distribution, the five parameters of the ellipse are coupled, and the Kullback-Leibler divergence loss is introduced. Achieve accurate prediction of ellipses.

[0093] 2. Taking into account the spatial positional constraint of the optic cup within the optic disc, this paper proposes a paired-box RPN for simultaneous detection of the optic cup and disc. Coupling the optic cup and disc in fundus images provides mutual support for detection, making the final prediction more reliable, especially in the presence of lesions and atrophic arcs.

[0094] 3. The present invention introduces an edge attention module to extract the edge features of the optic cup and optic disc to improve the distinction between the two, and uses the edge features as an important guide for context feature aggregation to improve the detection accuracy of the optic cup and optic disc.

[0095] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0096] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A method for jointly segmenting the optic cup and optic disc in a retinal fundus image, characterized in that: include: Based on elliptical morphological features and spatial geometric constraints, a network model for joint segmentation of the optic cup and optic disc is constructed. The network model comprises two stages: in the first stage, a paired-box RPN network is used to perform coupled detection of the minimum bounding rectangle of the optic cup and optic disc, taking into account the spatial geometric constraint that the optic cup is located within the optic disc area; in the second stage, elliptical parameters of the bounding rectangle area are determined; and finally, the optic cup and optic disc segmentation result is output based on the minimum bounding rectangle of the optic cup and optic disc detected by coupled detection in the first stage and the elliptical parameters of the second stage. Obtaining a retinal color fundus image sample set and the ellipse fitting parameters of the optic cup and optic disc in each retinal color fundus image sample; training the optic cup and optic disc joint segmentation network model according to the retinal color fundus image sample set and the ellipse fitting parameters of the optic cup and optic disc in each retinal color fundus image sample to obtain an optimal optic cup and optic disc segmentation network model; Inputting the retinal color fundus image to be detected into the optimal optic cup and optic disc segmentation network model, and outputting the optic cup and optic disc segmentation result image; The optic cup and optic disc joint segmentation network model includes: Faster RCNN front-end network, edge attention module, Paired-Box RPN network, first RoI-Align pooling layer, second RoI-Align pooling layer, rectangle predictor, ellipse predictor and post-processing operation; The Faster RCNN front-end network uses ResNet50-FPN as the backbone network, and the ResNet50 in the ResNet50-FPN is used to extract feature maps from the input retinal color fundus images; The edge attention module uses the intermediate output feature layer of the C2-C5 stage of ResNet50 in ResNet50-FPN as input, unifies the channels to 256 through a 3×3 convolution layer, and uses a 1×1 convolution layer and a sigmoid function to map the boundary features to the edge map. Subsequently, all the scales of the edge features are resized to 1 / 8 of the input retinal color fundus image, and all the rescaled edge features are concatenated together. Finally, the concatenated edge features are resized and output; The FPN in the ResNet50-FPN is used to element-wise add the edge features output by the edge attention module to each feature layer of the feature map extracted by ResNet50, and output a multi-layer feature layer; The Paired-Box RPN network, the first RoI-Align pooling layer, and the rectangle predictor constitute the first stage; The Paired-Box RPN network is used to perform coupled detection on the minimum bounding rectangle of the optic cup and optic disc based on multiple feature layers to obtain paired candidate boxes of the optic cup and optic disc; the paired candidate boxes include positive sample paired candidate boxes and negative sample paired candidate boxes; the first RoI-Align pooling layer is used to pool the paired candidate boxes of the optic cup and optic disc into a optic cup feature layer and an optic disc feature layer; the rectangle predictor uses two fully connected layers to perform classification and regression prediction on the optic cup feature layer and the optic disc feature layer, and outputs the coordinate value of the minimum bounding rectangle of the optic cup and its confidence level, and the coordinate value of the minimum bounding rectangle of the optic disc and its confidence level; The second RoI-Align pooling layer and the ellipse predictor constitute the second stage; The Paired-Box RPN network is also used to input the output positive sample paired candidate boxes into the FPN in ResNet50-FPN, and output a feature layer containing the candidate box position information; The second RoI-Align layer is used to expand the rectangular candidate box area in the feature layer containing the candidate box position information into a square, and output a corrected feature layer; The ellipse predictor uses two fully connected layers to predict the classification and regression prediction quantities based on the modified feature layer and output the predicted ellipse fitting parameters; The post-processing operation is used to output the optic cup and optic disc segmentation result according to the confidence of the minimum bounding rectangle box of the optic cup, the confidence of the minimum bounding rectangle box of the optic disc and the predicted ellipse fitting parameters.

2. The method for joint segmentation of optic cup and optic disc in retinal fundus images according to claim 1, characterized in that: The output layer of the Paired-Box RPN network contains a generated 8-dimensional vector (R d ,R c ) has a regression branch and a classification branch S; where R d and R c The 4-dimensional minimum rectangular box regression vectors (f x ,f y ,f w ,f h ) and (v x ,v y ,v w ,v h ); S is the 2D vector after Softmax normalization (S + ,S - ), S + and S - Represent the probability that the anchor box belongs to the foreground and background respectively; the anchor box is the preset optic cup and optic disc border; The process of assigning anchor box positive samples to the Paired-Box RPN network is as follows: if the anchor box A satisfies the IoU (A, D) ≥ α1 with the optic disc and the IoF (A, C) ≥ α2 with the optic cup, then the anchor box A is considered to be a positive sample match; where Q = (D, C) is the rectangular annotation box pair consisting of the rectangular bounding boxes of the optic disc and the optic cup, α1 and α2 are the anchor box positive sample assignment thresholds respectively; The Paired-Box RPN network selects balanced negative samples as follows: for each negative sample sampling layer, first randomly sample from the cup-disc pairs whose two IoUs both meet the negative sample sampling threshold; then randomly sample from the cup-disc pairs whose one of the two IoUs meets the negative sample sampling threshold; The matching process of the Paired-Box RPN network for the positive sample candidate region is as follows: the loss calculation is performed on the positive and negative samples, so that the Paired-Box RPN simultaneously generates the candidate region pair of the optic cup and optic disc X = (P d , P c ); If the candidate region pair X=(P d , P c ) satisfies IoU(P d , D)≥β1 and IoF(P c , C)≥β2, it is considered as a positive sample candidate area; where β1 and β2 are the positive sample allocation thresholds for the optic disc and optic cup candidate areas respectively.

3. The method for joint segmentation of optic cup and optic disc in retinal fundus images according to claim 1, characterized in that: Expand the rectangular candidate box area in the feature layer containing the candidate box position information into a square, specifically including: Expand outward along the two longer sides of the rectangular candidate box area so that the shorter side is equal to the longer side, and fill the extended part with 0.

4. The method for joint segmentation of optic cup and optic disc in retinal fundus images according to claim 1, characterized in that: The ellipse fitting parameters include: center point coordinates (c x ,c y ), the major semi-axis a, the minor semi-axis b and the rotation angle θ; Among them, θ∈(-π / 2,π / 2] represents the angle from the positive x-axis to the major axis of the ellipse; the width of the axis-aligned minimum bounding rectangle of the ellipse Height is The minimum bounding rectangle of an ellipse is represented by (c x -w / 2,c y -h / 2,c x +w / 2,c y +h / 2).

5. The method for joint segmentation of optic cup and optic disc in retinal fundus images according to claim 4, characterized in that: The calculation formula for the predicted ellipse fitting parameters is: Among them, (P x1 ,P x2 ,P y1 ,P y2 ) is the coordinate representation of a square candidate region in the modified feature layer, (c' x ,c' y ) is the predicted center point coordinate, a' is the predicted major semi-axis, b' is the predicted minor semi-axis, θ' is the predicted rotation angle, P l To correct the diagonal length of a square candidate region in the feature layer, 6. The method for joint segmentation of optic cup and optic disc in retinal fundus images according to claim 4, characterized in that: The total loss function of the optic cup and optic disc joint segmentation network model is Where λ1, λ2 and λ3 are the first, second and third weighted weights respectively; is the loss function of the Paired-Box RPN network, is the binary cross entropy classification loss for distinguishing background and foreground, and Represent the regression loss of the optic disc and optic cup with the corresponding anchor box, and Smooth L1 loss is used in both cases; is the network head loss function, represents the classification loss, represents the rectangular candidate box regression loss, represents the ellipse parameter regression loss, γ1 and γ2 are the fourth and fifth weighted weights respectively; is the marginal loss function, e i is the binary edge label generated by the optic cup and optic disc annotation mask, is the edge prediction probability obtained by the edge attention module.

7. The method for joint segmentation of optic cup and optic disc in retinal fundus images according to claim 6, characterized in that: The ellipse fitting parameters are characterized as a two-dimensional Gaussian distribution for parameter coupling, and the ellipse fitting parameters and the two-dimensional Gaussian distribution are characterized as a two-dimensional Gaussian distribution for parameter coupling. The conversion relationship is: μ=(c x , c y ) T ,Σ=RΛR T ; where μ is the mean of the variable, Σ is the covariance matrix, Prediction ellipse With target ellipse Ellipse parameter regression loss Expressed as in, 8. The method for joint segmentation of optic cup and optic disc in retinal fundus images according to claim 1, characterized in that: The obtaining of the retinal color fundus image sample set and the ellipse fitting parameters of the optic cup and optic disc in each retinal color fundus image sample specifically includes: Obtain a known set of retinal color fundus image samples; According to the gold standard mask of the optic cup and optic disc annotated in each retinal fundus image sample in the retinal color fundus image sample set, the edge extraction and ellipse fitting of the optic cup and optic disc in each retinal fundus image sample are performed to obtain the ellipse fitting parameters of the optic cup and optic disc in each retinal color fundus image sample.

9. The method for joint segmentation of optic cup and optic disc in retinal fundus images according to claim 1, characterized in that: The method further comprises: obtaining a retinal color fundus image sample set and an ellipse fitting parameter of the optic cup and optic disc in each retinal color fundus image sample; Preprocessing the retinal fundus image samples in the retinal color fundus image sample set to increase the number and diversity of samples in the retinal color fundus image sample set; the preprocessing includes horizontal and vertical flipping with a random probability of 0.5, rotation of a random angle of 0-180, scaling at a random scale of 0.8-1.2, and random color dithering, maintaining the aspect ratio of the image and scaling the shortest side to 800 resolution and the longest side not exceeding 1333 resolution; The ellipse fitting parameters of the optic cup and optic disc in the preprocessed retinal fundus image samples are adjusted accordingly.

Images