An eye fundus image retinal blood vessel segmentation method based on a graph convolutional neural network

By using a graph convolutional neural network-based method, and employing multi-level feature representation and low-rank matrix completion, the problems of difficulty in segmenting small blood vessels and insufficient utilization of high-level features in retinal vessel segmentation of fundus images are solved, achieving higher accuracy and more stable vessel segmentation results.

CN116862928BActive Publication Date: 2026-04-21QINGDAO UNIV OF SCI & TECH
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
QINGDAO UNIV OF SCI & TECH
Filing Date
2023-07-12
Publication Date
2026-04-21

AI Technical Summary

Technical Problem

Existing methods for segmenting retinal vessels in fundus images have difficulty segmenting small vessels and maintaining the contrast between vessels and the background. They also fail to fully utilize high-level features and ignore the relationships between different features, which affects the robustness and consistency of the segmentation.

Method used

A graph-based convolutional neural network approach is adopted, which reconstructs avascular fundus images by using a multi-level feature representation module, graph-based high-level and low-level convolutional feature aggregation modules, and low-rank matrix completion method, thereby enhancing the foreground-background contrast. Furthermore, the graph structure is used to propagate information and explore the relationships between features at different levels.

Benefits of technology

It improves the accuracy and robustness of blood vessel segmentation, reduces the omission of small blood vessels and background interference, enhances the stability and generalization ability of the segmentation algorithm, and adapts to different data distributions and task requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116862928B_ABST
    Figure CN116862928B_ABST
Patent Text Reader

Abstract

The application provides a fundus image blood vessel segmentation method based on a graph convolutional neural network. Feature information of a fundus image is extracted through a feature extraction network to establish a four-layer multi-scale feature representation network. Feature maps of low three layers are taken as inputs of a low-level convolutional feature aggregation module of a graph, and feature maps of high three layers are taken as inputs of a high-level convolutional feature aggregation module of the graph. Pixels on the feature maps are taken as nodes of the graph, and relationships between the pixels are taken as edges. The high-level convolutional feature aggregation module reconstructs a non-blood vessel retina image by using more semantic information, and the low-level convolutional feature aggregation module segments blood vessels by using more detailed information. The application also obtains a non-blood vessel fundus image through a matrix filling method of a low-rank matrix, takes the non-blood vessel fundus image as a label to train the high-level convolutional feature aggregation module based on the graph, enhances contrast between foreground and background, and improves segmentation precision of the algorithm.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of deep learning and image processing, and in particular to a method for segmenting retinal vessels in fundus images. Background Technology

[0002] The eye is one of the most important organs for the human body to perceive external information. Visual information accounts for 70% of the information processed by the brain; therefore, research on the structure, function, and physiology of the eye has been a continuous focus. The fundus is the only part of the body where arteries, veins, and capillaries can be directly and closely observed with the naked eye. These blood vessels can reflect the dynamics of the body's blood circulation and overall health; many systemic diseases can be reflected in the fundus. The retina, as a sensor that responds to light, transmits impulses from both eyes to the brain via the optic nerve. Its complex structure has at least nine layers and plays a crucial role in vision. Therefore, research on the retina continues. Various retinal diseases caused by dysfunction of different retinal layers can be diagnosed by ophthalmologists analyzing fundus photographs.

[0003] With the widespread application of artificial intelligence technology, deep learning has become a fast and efficient method for processing medical images. Naturally, computer-aided diagnosis (CAD) algorithms for the eye also utilize deep learning. Many eye diseases involve the retina, and fundus images can be divided into vascular and non-vascular areas. By training a network model on a dataset to segment blood vessels in retinal fundus images, efficiency can be improved, assisting doctors in making effective diagnoses.

[0004] Retinal vessel segmentation is a challenging semantic segmentation problem. The blood vessels in fundus images have a tree-like topology, and the terminal vessels are small and difficult to segment completely. The fundus image itself is orange-red, and the blood vessels are red, resulting in a weak contrast between the foreground and background, which also adds to the difficulty of vessel segmentation.

[0005] To achieve retinal vessel segmentation, some algorithms combine traditional methods with deep learning, employing deep neural networks (DNNs) as trainable feature extractors and incorporating features from non-end-to-end methods. Other methods apply convolutional neural networks to retinal vessel segmentation. Still others use fully convolutional networks to classify pixels in fundus images. Most of these methods employ an encoder-decoder architecture, where the encoder module encodes information and the decoder module decodes it, without carefully considering the intrinsic relationships between multiple layers during the encoding process.

[0006] [1] Liu Yipeng, Rui Xue, Jiang Li, Wang Haixia, Chen Peng, Liang Ronghua. A method for segmenting blood vessels in fundus images based on multi-scale features of fully convolutional neural networks [P]. Zhejiang Province: CN111242949B, 2021-06-18. This method uses a fully convolutional neural network, which uses an encoder-decoder fully convolutional network structure to segment blood vessels. For fully convolutional neural networks, due to the local perception capability of convolution, the color information of fundus images is not fully utilized, which will result in the loss of some contrast and discrimination between blood vessels and background. At the same time, the extraction of local features of fundus images is not sufficient. Usually, only global spatial pyramid pooling or upsampling layers are used to restore resolution, which will inevitably ignore some small and tortuous blood vessel structures.

[0007] [2] The invention disclosed in application number CN202210392336.1 is a method for segmenting blood vessels in fundus images based on multi-scale channel fusion and spatial activation. The proposed model consists of three modules: a hybrid convolution module, a channel-coordinated attention fusion module, and a global spatial activation module. First, the hybrid convolution module is used to extract deep features of blood vessels; then, the channel-coordinated attention fusion module interacts with the information of the target at different scales and channels; finally, the global spatial activation module aggregates features to achieve fundus blood vessel segmentation. Although this method uses an attention mechanism to enhance the channel and spatial information of the image and improve the segmentation accuracy of positive samples, it mainly focuses on low-level features of the image, such as edges and textures, without making full use of high-level features, such as shape and semantics. This may lead to confusion and omission of some details and background. At the same time, this method does not consider the relationship and mutual influence between different features, such as context and contrast, which greatly affects the robustness and consistency of segmentation. Summary of the Invention

[0008] To address the aforementioned shortcomings of existing retinal vessel segmentation methods for fundus images, this invention proposes a retinal vessel segmentation method for fundus images based on graph convolutional neural networks, which employs the following technical solution: A retinal vessel segmentation method for fundus images based on graph convolutional neural networks, comprising:

[0009] Step A: Establish a multi-level feature representation module, using ResNet as the backbone network for visual feature extraction, and build a four-layer multi-scale feature map from bottom to top;

[0010] Step B: Establish a graph-based high-level convolutional feature aggregation module. The last three layers of the multi-level feature representation module in Step A are used as the input of the graph-based high-level convolutional feature aggregation module. Each layer achieves the same resolution and number of channels. After graph initialization, features are aggregated, nodes are updated, features are stitched together and deconvolved to reconstruct avascular fundus images.

[0011] Step C: Establish a graph-based low-level convolutional feature aggregation module. Take the lower three layers of the multi-level feature representation module in Step A as the input of the graph-based low-level convolutional feature aggregation module, and ensure that each layer achieves the same resolution and number of channels.

[0012] Step D: Output the segmentation results.

[0013] Furthermore, between step C and step D, the method further includes: acquiring an avascular fundus image, reconstructing the avascular fundus image using a low-rank completion method, and inputting it back into the network.

[0014] Further, in step B, graph nodes V = {v} are constructed on the extracted feature map pixels. i |i=1,2,3....,n}, from node v i to node v j The edge e ij Defined as: e ij =conv((v i -v j )⊙v j ), where ⊙ is a channel-wise concatenation product operation. Similarly, from node v j to node v i The edge e ji Defined as: e ji =conv((v j -v i )⊙v i The characteristics of the current node are aggregated from its neighboring nodes, that is:

[0015]

[0016] in, Represents the nonlinear activation function ReLU, where K is the number of iterations.

[0017] Residual joins are used to update node embeddings. Each node updates its embedding based on its own information and that of its surrounding nodes. This process involves T rounds of information propagation and aggregation, i.e.:

[0018]

[0019] Furthermore, the low-rank completion method includes:

[0020] The fundus image to be processed is This is an image of the fundus after removing blood vessels, H t (·) represents a delayed embedding operation, for the input E, and M after one H t After the operation (·), we get:

[0021]

[0022]

[0023] The obtained E H It is a 2N-order Hank tensor, and the low-rank matrix is ​​repaired as follows:

[0024]

[0025]

[0026]

[0027]

[0028]

[0029]

[0030] in, This represents the inversion of the MDT operation. For the factor matrix, For the kernel tensor, the initial value is B. n =1.

[0031] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0032] This invention proposes a graph convolutional neural network (GNN) method for vascular segmentation in fundus images. The model consists of three modules: a multi-level feature extraction module, a graph-based high-level convolutional feature aggregation module, and a graph-based low-level convolutional feature aggregation module. First, feature information from the fundus image is extracted using a feature extraction network, establishing a four-layer multi-scale feature representation network. The lower three layers have higher resolution feature maps containing more details and serve as input to the graph-based low-level convolutional feature aggregation module; the higher three layers have lower resolution feature maps containing more semantic information and serve as input to the graph-based high-level convolutional feature aggregation module. Next, the graph-based high-level convolutional feature aggregation module and the graph-based high-level convolutional feature aggregation module are established, respectively. Pixels on the feature maps are treated as nodes in the graph, and the relationships between pixels are treated as edges. The high-level convolutional feature aggregation module reconstructs the avascular retinal image using more semantic information, while the low-level convolutional feature aggregation module segments the blood vessels using more detailed information. Furthermore, this invention also uses a matrix filling method with low-rank matrices to obtain avascular fundus images, which are used as labels to train the graph-based high-level convolutional feature aggregation module, enhancing the contrast between foreground and background and improving the segmentation accuracy of the algorithm. Attached Figure Description

[0033] Figure 1 This is a schematic flowchart of the retinal vessel segmentation method for fundus images according to the present invention.

[0034] Figure 2 This is the training branch framework diagram of the present invention.

[0035] Figure 3 This is a framework diagram of the test branch and training branch of this invention.

[0036] Figure 4 It contains fundus images with and without blood vessels.

[0037] Figure 5 This is an example of retinal vessel segmentation results.

[0038] Figure 6 This is a comparison chart of experimental results from multiple methods. Detailed Implementation

[0039] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0040] refer to Figure 1 This embodiment of the retinal vessel segmentation method based on graph convolutional neural networks for fundus images includes:

[0041] The first step is to establish a multi-level feature representation module.

[0042] refer to Figure 2 This application employs ResNet as the backbone network for visual feature extraction, aiming to extract image features to build a multi-level feature representation module, which serves as the input for subsequent modules. This application establishes four layers of feature maps at different scales from bottom to top, exploring the context and dependency information across feature representations.

[0043] The second step is a graph-based high-level convolutional feature aggregation module.

[0044] The last three layers of the multi-level feature representation module obtained in the previous step are used as input to this module, and interpolation and convolution are performed respectively to obtain the same number of channels and the same resolution. Then, these features are used for graph node initialization and graph edge initialization. After graph initialization, features are aggregated and nodes are updated. Finally, the features are concatenated and deconvolved to reconstruct the avascular fundus image.

[0045] The third step is a graph-based low-level convolutional feature aggregation module.

[0046] The lower three layers of the multi-level feature representation module are taken as input, and each layer is interpolated or convolved to the same resolution and number of channels. These three layers have a higher resolution and contain more details. This module is designed for blood vessel segmentation and therefore requires more detailed features.

[0047] This application utilizes a graph-based convolutional feature aggregation network to propagate information and achieve retinal vessel segmentation. Features are transformed into graph nodes, and the information-transferring ability of graphs is leveraged to explore the relationships between feature nodes at different levels. Using graphs to propagate representational information and extracting local and global features from images through graph structure expands the receptive field while avoiding the loss of local location information, thereby improving the accuracy and robustness of vessel segmentation. Furthermore, it can model long-distance dependencies between pixels in the image, capturing richer contextual information and effectively addressing ambiguity and uncertainty issues in vessel segmentation.

[0048] The fourth step is the acquisition of avascular fundus images.

[0049] Blood vessels in fundus images are very small targets, occupying a small area of ​​the image and having a strong similarity to the background, making them difficult to segment and remove using traditional image processing methods. Furthermore, blood vessels in fundus images exhibit different shapes, thicknesses, curvatures, and branching structures, thus requiring a method that can adapt to different vascular characteristics to acquire avascular fundus images.

[0050] To improve the accuracy of the blood vessel segmentation algorithm, reconstructed avascular fundus images were incorporated into the network training. (Reference) Figure 3 This invention uses the original fundus image as input, with the blood vessels as the foreground and the remaining image as the background. The fundus image is treated as a low-rank matrix, and image completion using this low-rank matrix is ​​used to reconstruct avascular fundus images. This image is then fed back into the network, providing clearer retinal structure information, which improves segmentation accuracy and robustness, reduces interference from blood vessels on other retinal features, and further enhances segmentation efficiency and stability. Furthermore, it strengthens the contrast between foreground and background regions throughout the network, improving the accuracy of blood vessel segmentation algorithms.

[0051] Step 5: Obtain the segmentation results

[0052] Following the graph-based low-level convolutional feature aggregation module, the final segmentation result is output by continuously aggregating features, updating node information, and outputting the final segmentation result.

[0053] This application utilizes a graph-based convolutional feature aggregation network to propagate information and achieve retinal vessel segmentation. Features are transformed into graph nodes, leveraging the information-transferring capabilities of graphs to explore relationships between feature nodes at different levels. Using graphs to propagate representation information allows for the exploration of dependencies between multiple layers. The graph-based low-level convolutional feature aggregation module fully utilizes boundary information for vessel segmentation.

[0054] The following section provides a detailed introduction with specific application examples.

[0055] The first step is multi-level feature extraction.

[0056] Fundus images typically include the optic disc (optic nerve), blood vessels, retinal tissue, and choroid. This application uses a multi-level feature extraction network to reconstruct fundus images.

[0057] The original fundus image dataset is defined as follows:

[0058] FundusSet = {E i ∈R H×W ,i=1…N} (1)

[0059] Among them, E i Let H represent the i-th fundus image sample (Eye Sample). Each sample in the dataset is a two-dimensional image, and H×W is the spatial resolution. R represents the three-dimensional vector space.

[0060] In this embodiment, a feature set {L} is output using the ResNet feature extraction backbone network. i The algorithm contains four feature image layers of different sizes, L2 to L5, arranged from bottom to top. Each layer uses interpolation to adjust the resolution of cross-layer features from bottom to top, with a reduction rate of 0.5 times that of the previous layer. The number of channels for L2, L3, L4, and L5 are 256, 512, 1024, and 2056, respectively. Figure 2 and Figure 3 The parameters W and H represent the width and height of the input retinal image, respectively. i The following formula is used for iterative calculation:

[0061] L i+1 =conv(interp(L) i )), i = 2, 3, 4

[0062] Where interp represents the interpolation operation and conv represents a 3×3 convolution. The extracted basic feature representation L i The input is fed into the next step of the graph convolutional network to explore the context and dependency information of the multi-level feature representations.

[0063] The second step is a graph-based high-level convolutional feature aggregation module.

[0064] The multi-level feature representations obtained in the previous step, including L3, L4, and L5, will be input into the graph-based high-level convolutional feature aggregation module. Before this, it is necessary to unify the size and channel dimensions of these three feature representations. This process is represented as follows:

[0065]

[0066]

[0067] The new feature representation obtained after normalizing the high-level feature representation and inputting it into the high-level convolutional feature aggregation module is as follows:

[0068]

[0069] See step four for a detailed explanation of the graph network construction process.

[0070] The third step is a graph-based low-level convolutional feature aggregation module.

[0071] Similar to the previous module, the graph-based low-level convolutional feature aggregation module takes the lower three layers L2, L3, and L4 as input. Likewise, each of them needs to be normalized in terms of size and channel dimensions before being input into the network module.

[0072] The normalization process is represented as follows:

[0073]

[0074]

[0075] By interpolating and convolution, we obtain the same channel dimension of 64 and the same resolution w / 8×h / 8. Therefore, the new input to the low-level convolutional feature aggregation module is represented as follows:

[0076] The lower-level three-layer features L2, L3, and L4 used in this step have a higher resolution (w / 8×h / 8) than the previous step's (W / 16×H / 16) features. This higher resolution means more detail in the feature maps. The low-level convolutional feature aggregation module is designed for segmenting blood vessels, therefore it requires more detail than the high-level convolutional feature aggregation module. The lower-resolution L2, L3, and L4 features provide better semantic information for the high-level convolutional feature aggregation module to reconstruct avascular fundus images, although they contain less high-frequency information than blood vessels.

[0077] The fourth step is to construct a graph-based network module.

[0078] Given a graph Graph = (V, E), where V represents the set of nodes in the graph and E represents the set of edges in the graph. Graph nodes {vi |v i ∈V} is related to the initial features, and the edges of the graph {e ij |e ij ∈E} represents a pair of nodes (v i v j The relationship between ) . For example Figure 2 As shown, the inputs L3, L4, and L5 of the high-level convolutional feature aggregation module are standardized to 64×w / 16×h / 16 through interpolation and convolution operations. A crucial question is how to assign initial feature vectors to each node and edge so that the convolutional operation can effectively learn the patterns and structure in the graph. Nodes and edges may require different initialization methods depending on the type and source of the graph, to adapt to the data distribution and noise in the graph. The following section describes how to utilize existing features to construct the graph and update the nodes and edges.

[0079] First, graph construction requires initializing the graph nodes and the edges between them. Graph nodes originate from feature maps at different levels, while edges can be obtained by constructing connections between nodes once the nodes are determined. In a graph, information for each node can be updated from its neighbors to explore the relationships and dependencies between nodes. Directly using pixels from the input fundus image as graph nodes would significantly increase the computational burden. Therefore, graph nodes V = {v...} are constructed on the extracted feature map pixels. i |i=1,2,3....,n}. Select a pixel from L3, denoted as p. 30 Its size is clearly 64×1. Similarly, take p from L4 and L5 respectively. 40 p 50 So p 30 p 40 p 50 This forms a node set V, which comes from different feature maps. Features from different levels have different relevance, and this information is used to explore the dependencies between nodes at different levels of features. In addition, the graph structure allows messages to propagate between nodes, and all nodes can be updated during iterations with other nodes.

[0080] After establishing the nodes, edges are further constructed using these nodes. In Graph Neural Networks (GNNs), graph edges represent the relationships between graph nodes; a larger edge weight indicates a stronger relationship between the nodes at both ends of the edge. From node v... i to node v j The edge e ij Defined as:

[0081] e ij =conv((v i -v j )⊙v j )

[0082] Here, ⊙ represents a channel-wise concatenation operation, and conv is a 1×1 convolution operation. Similarly, from node v... j to node v i The edge e ji Defined as:

[0083] e ji =conv((v j -v i )⊙v i )

[0084] This allows us to create edges between node pairs to describe the connection between them.

[0085] After initializing the nodes and edges of the graph, the node features need to be updated during the iteration process. This is to enable information propagation and exchange between graph nodes, and to update the features of the current node. It is aggregated from its neighboring nodes. This process can be represented as:

[0086]

[0087] in, Representing the nonlinear activation function ReLU, which transforms edges into connectable weights, K is the number of iterations. Considering the continuity of the vascular structure, the relationships between nodes can be constructed using their neighboring nodes. This application uses a residual connection to update node embeddings. The residual connection can alleviate gradient vanishing and gradient exploding, thus enabling the training of deep networks. Each node can be updated based on its own information and that of its surrounding nodes. After T rounds of information propagation and aggregation, i.e.:

[0088]

[0089] Using equation (5), the output node v i It is obtained by connecting the two parts on the right channel by channel and then convolving them with a 1×1 convolution.

[0090] In the high-level convolutional feature aggregation module, feature layers L3, L4, and L5 are aggregated during the iteration process. During the aggregation process, the graph nodes p located on these layers can be iteratively updated. 30 p 40 p 50 .

[0091] The network process is as follows:

[0092]

[0093] Step 5: Reconstruction of avascular fundus images.

[0094] In normal fundus images, blood vessels occupy a relatively small proportion of the image; these vessels can be considered as "noise," and the remaining fundus image is the image to be restored. Removing the blood vessels yields the remaining fundus image without them. From a matrix filling perspective, this involves recovering the background of all regions, assuming the background information of non-foreground areas is known. Therefore, in this step, this application constructs a vessel-free fundus image through low-rank matrix image filling.

[0095] The fundus image to be processed is Where R is the set of all real numbers, E is an N-order tensor. It is an image mask after removing blood vessels from the fundus image. Then, the Hankel matrix is ​​constructed using the Multiway Delay-embedding Transform (MDT) method.

[0096] For vector x = (x1, x2, ..., xn) T )∈R T The delay embedding with a delay window size of t is defined as:

[0097]

[0098] Where H t (·) represents a delayed embedding operation, where the terms on the antidiagonal are identical; such a matrix is ​​called a Hank matrix. Here, we take T = 7 and t = 3.

[0099] For the input tensor E, and the blood vessel mask M after one H t After the operation (·), we get:

[0100]

[0101]

[0102] The obtained E H It is a 2N order Hankel tensor. The subsequent low-rank matrix repair is based on E. H Tucker decomposition (TKD) is used for repairing low-rank matrices. The Tucker decomposition contains 2N factor matrices. and a 2N-order kernel tensor Initial value B n =1, by gradually increasing B n The value of B is calculated until the error is less than the threshold to obtain the optimal B. n Then I got it. The process is as follows:

[0103]

[0104]

[0105]

[0106]

[0107] The final tensor is obtained by reversing the MDT, as follows:

[0108]

[0109]

[0110] in, This represents the inversion of the MDT operation.

[0111] like Figure 4 As shown, two types of images are obtained: avascular fundus images and vascular fundus images. The avascular fundus images contain only background information and no vascular information, and have the same brightness and contrast, making them easier for the algorithm to process. They can be used as background images, i.e., as templates or reference images, to help the computer algorithm better separate blood vessels in the fundus images. This invention, by simultaneously using avascular and vascular fundus images, can reduce errors and improve the robustness of the algorithm, achieving good segmentation results even in complex image scenes.

[0112] Step 6: Loss function.

[0113] In the third step, the graph-based low-level convolutional feature aggregation module is mainly used for blood vessel segmentation in fundus images. This step utilizes the Dice loss function and the Cross Entropy (CELoss) loss function. Dice loss is a widely used loss function in medical image segmentation, named after the Dice coefficient. The Dice coefficient is a metric used to evaluate the similarity between two samples; a larger value indicates greater similarity. The Dice coefficient can be defined as follows:

[0114]

[0115] Among them, y i This represents the label value of pixel i. Let N represent the predicted value of pixel i, and N be the number of pixels. Therefore, DiceLoss can be defined as:

[0116]

[0117] CE loss is a commonly used loss function in classification and segmentation tasks, and its definition can be expressed as:

[0118]

[0119] Where y represents the label. This represents the predicted value.

[0120] In the second step, the low-level convolutional feature aggregation module based on graphs, the main purpose is to reconstruct the retinal fundus image. This module uses Mean Squared Error (MSE) loss, defined as follows:

[0121]

[0122] Where N is the number of pixels and y is the label value. These are predicted values.

[0123] Figure 5 This is an example of retinal vessel segmentation results obtained using the method of this invention at different shooting angles. When the camera shoots from left to right along the pupil center line, the image shown in the figure is displayed. The distribution of blood vessels differs at different angles, and the segmentation difficulty also varies. The left and right images have more and finer blood vessel branches, making segmentation more challenging. The results demonstrate that the method of this invention achieves excellent segmentation results regardless of the image type. Figure 6 The figure shows a comparison of experimental results with various methods. As can be seen, the method of this invention significantly improves the continuity of blood vessel segmentation, such as at the junctions between blood vessels or in thinner vessels, compared to SegNet, Attention-UNet, and UNet. This is because the present invention fully considers the relationship and dependency between nodes and surrounding nodes, aggregating more detailed information from low-level features. Since blood vessels are inherently continuous, the present invention explores non-local contextual relevance, minimizing unnecessary predictions. For example, SegNet and UNet produce numerous missegments, with many non-blood vessel areas, including the edge of the eyeball, being segmented as blood vessels. The method of this invention incorporates less noise.

[0124] This invention employs an end-to-end network, reducing interference and errors in intermediate processes, and improving network efficiency and accuracy. Simultaneously, it better adapts to different data distributions and task requirements, enhancing the network's generalization ability and robustness. Furthermore, this invention uses two branches to obtain segmented images and avascular fundus images from the original image. The output fundus image helps improve segmentation accuracy and robustness, as it contains more background and semantic information, which can be used to reconstruct avascular retinal images. The output fundus image also facilitates subsequent analysis and applications, such as determining vascular geometric features, arterial / venous ratios, blood flow analysis, image quality assessment, image registration, and synthesis.

[0125] The embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for retinal blood vessel segmentation of fundus images based on graph convolutional neural network, characterized in that, include: Step A: Establish a multi-level feature representation module, using ResNet as the backbone network for visual feature extraction, and build a four-layer multi-scale feature map from bottom to top; Step B: Establish a graph-based high-level convolutional feature aggregation module The last three layers of the multi-level feature representation module in step A are used as input to the graph-based high-level convolutional feature aggregation module. Each layer achieves the same resolution and number of channels. After graph initialization, features are aggregated, nodes are updated, and features are concatenated and deconvolved to obtain avascular fundus images. The avascular fundus images are then reconstructed using the low-rank completion method and fed back into the network. The low-rank completion method includes: The fundus image to be processed is , is an image of the fundus image after blood vessels are removed, represents a one-delay embedding operation, for the input E, and M after one operation, the following is obtained: ; ; obtained is a Hankel tensor of order 2N, and the low-rank matrix is repaired as follows: ; ; ; wherein represents a reverse MDT operation, is a factor matrix, is a core tensor, initial value = 1; Step C: Establish a graph-based low-level convolutional feature aggregation module The lower three layers of the multi-level feature representation module in step A are used as the input of the graph-based low-level convolutional feature aggregation module, with each layer achieving the same resolution and number of channels. Step D: Output the segmentation results.

2. The fundus image retinal blood vessel segmentation method based on the graph convolutional neural network according to claim 1, characterized in that, The step B, constructing graph nodes on the extracted feature image pixels from node to node is defined as: where is a channel-wise concatenation operation; similarly, the edge from node to node is defined as: ; the features of the current node are aggregated from its neighbor nodes, i.e.:​​ ; wherein, represents a non-linear activation function RELU, K is the number of iterations.

3. The fundus image retinal blood vessel segmentation method based on the graph convolutional neural network according to claim 2, characterized in that, Residual joins are used to update node embeddings. Each node updates its embedding based on its own information and that of its surrounding nodes. This process involves T rounds of information propagation and aggregation, i.e.: 。

Citation Information

Patent Citations

  • Fundus image blood vessel segmentation method based on full convolutional neural network multi-scale features

    CN111242949A

  • Eye fundus image blood vessel segmentation method based on multi-scale channel fusion and space activation

    CN116071411A