A color fundus image blood vessel segmentation method and device
By fusing residual self-attention features into a UNet model, and combining residual global context module and pyramid average pooling, the problem of incomplete segmentation of small blood vessels in fundus vessel segmentation is solved, achieving high resolution and clear vessel segmentation results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UNIV OF ELECTRONICS SCI & TECH OF CHINA
- Filing Date
- 2023-10-17
- Publication Date
- 2026-07-31
AI Technical Summary
Existing technologies struggle to effectively segment small blood vessels in areas with low contrast or at the edges of the fundus, and they also exhibit overfitting, resulting in incomplete and unclear segmentation.
A residual self-attention feature fusion UNet model is adopted. By using residual connections and self-attention feature fusion between the encoder and decoder, the effective vascular information of the encoder is preserved and useless information is filtered out. The residual global context module and pyramid average pooling technique are used to enhance the feature extraction and segmentation effect.
It achieves high resolution, high continuity, more completeness and clearer details in blood vessel segmentation, solves the problem of blood vessel segmentation in edge and low contrast areas, and improves the accuracy and detail of segmentation.
Smart Images
Figure CN117611603B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image processing, and in particular to a method and apparatus for segmenting blood vessels in color fundus images. Background Technology
[0002] The relationship between retinal vascular morphology and disease has been extensively studied. These studies have shown that the morphology and changes of retinal vessels can provide clinicians with important information about patients' health status, thereby aiding in the early diagnosis and treatment of various diseases. The following are some aspects of the role of retinal vessels: As a disease indicator: The morphology and changes of retinal vessels can serve as indicators for predicting and diagnosing various diseases, such as diabetic retinopathy, hypertension, cardiovascular disease, and Alzheimer's disease. Reflecting overall health status: The morphology and changes of retinal vessels reflect the state of the systemic microcirculation because retinal vessels share similar structures and functions with the systemic arterial system; Monitoring disease progression: The morphology and changes of retinal vessels can be used to monitor the progression of diseases.
[0003] Although retinal blood vessels can effectively prevent major human diseases, their complex structure and intricate distribution make manual annotation extremely difficult and time-consuming. These problems prevent many patients from receiving timely detection and treatment in the early stages of disease. Automated blood vessel segmentation can free ophthalmologists from tedious blood vessel annotation work, allowing more patients in need to receive timely treatment. The morphology and structure of retinal blood vessels are very complex, and the differences in brightness and contrast between retinal blood vessels and surrounding tissues are small. Therefore, obtaining high-resolution, accurate, and complete blood vessel segmentation results is very challenging.
[0004] In recent years, research in image segmentation has been increasing. In terms of fundus vessel segmentation algorithms, they can be broadly divided into unsupervised and supervised algorithms. Unsupervised methods achieve image segmentation by automatically learning features and patterns in the data, without requiring prior knowledge or labeled data. These methods mainly include matching filtering, vessel tracking, morphological transformation, and model-based algorithms. Neto et al. developed a self-contained algorithm based on mathematical morphology, spatial dependence, and curvature. Zhao et al. proposed an infinitely active contour model utilizing mixed region information. Roychowdhury et al. proposed an adaptive thresholding method for iterative vessel segmentation. Yin et al. proposed a segmentation method using the Hessian matrix and threshold entropy, employing post-processing to eliminate noise and light reflection from the vessel centerline. Supervised segmentation methods use standard vessel images to train a classifier to distinguish whether pixels are vessels. Roychowdhury et al. reduced the number of pixels classified by eliminating major vessels detected in common regions after thresholding of high-pass filtered images, thus saving time. Orlando et al. proposed a discriminative training segmentation model based on a fully connected conditional random field, which can better distinguish desired structures than methods based on local neighborhoods.
[0005] Deep learning-based methods can automatically extract and classify features without requiring manual design and adjustment of features and classifiers. They offer high segmentation accuracy and robustness, and are increasingly widely used in medical image processing. In recent years, numerous deep learning algorithms have emerged for retinal vessel segmentation. Wang et al. proposed a novel error attention refining network (ERA-Net), which learns and predicts potential errors in a two-stage manner for effective retinal vessel segmentation. Li et al. proposed a lightweight convolutional neural network incorporating attention mechanisms. The entire network structure consists of a basic U-Net and an attention module, which captures global information and enhances features by incorporating it into the feature fusion process. Zhang et al. proposed a novel multi-module cascaded U-shaped network that applies dilated convolution and multi-kernel pooling techniques to retinal vessel segmentation. Saini et al. proposed a multi-scale, multi-level attention embedding CNN architecture ((M)SLAe-Net), which extracts features at multiple scales and levels, enabling the model to comprehensively extract both local and global features. Zhang et al. proposed a pyramid U-Net that uses pyramid-level aggregation blocks (PSABs) for both the encoder and decoder to aggregate features at multiple different levels. In this way, coarse-to-fine contextual information is shared and aggregated within each block, thereby improving vessel segmentation performance. Karaali et al. proposed a novel deep learning method that combines residual dense network modules and residual compression activation modules, effectively improving the segmentation of small vessels. Guo et al. used dropblocks to prevent network overfitting and embedded a spatial attention module at the connection point between the encoder and decoder. Analysis of these studies reveals that current deep learning-based fundus vessel segmentation methods mainly use the UNET architecture and add modules to assist the network in acquiring features better. However, they all simply connect encoder and decoder features via skip connections, ignoring the relationship between the encoder and decoder features. The decoder needs to supplement the effective information lost during continuous downsampling in the encoding process. While simple skip connections can provide this effective information, they also contain a large amount of redundant information. Therefore, how to enable the decoder to rationally select encoder features is crucial for improving vessel segmentation performance. Summary of the Invention
[0006] The purpose of this invention is to provide a method and system for segmenting blood vessels in color fundus images, which can distinguish small blood vessels from the background in areas with low contrast, such as edges, solve the overfitting phenomenon, and segment more complete blood vessels.
[0007] The objective of this invention is achieved through the following technical solution:
[0008] A method for segmenting color fundus images, comprising the following steps:
[0009] Step 1: Obtain a dataset of color fundus images;
[0010] Step 2: Preprocess the color fundus image dataset to obtain a preprocessed color fundus image dataset;
[0011] Step 3: Construct a residual self-attention feature fusion UNet model;
[0012] Step 4: Train the residual self-attention feature fusion UNet model to obtain the trained residual self-attention feature fusion UNet model;
[0013] Step 5: Input the preprocessed color fundus image dataset into the trained residual self-attention feature fusion UNet model for training to obtain the color fundus blood vessel segmentation result.
[0014] In the above technical solution, the preprocessing of the color fundus image dataset to obtain the preprocessed color fundus image dataset specifically includes:
[0015] The color fundus image dataset is subjected to edge filling, CLAHE algorithm enhancement, rotation and vertical flip enhancement to obtain the processed color fundus image dataset.
[0016] In the above technical solution, the residual self-attention feature fusion UNet model specifically includes:
[0017] An encoder structure model is used to extract the most representative and discriminative high-level semantic information features of images. The implementation of the encoder structure model consists of three steps, each of which includes an RGCTB structure and a downsampling structure. The RGCTB structure adds a GC block between two convolutional blocks to preserve important information in the space and channels. The downsampling process includes convolution with a kernel of 3 and a stride of 2, normalization layer, and ReLU activation layer operations to reduce the image size and increase the receptive field.
[0018] The decoder structure model recovers the initial image information. The decoder structure model includes three steps, each of which includes an RGCTB and an upsampling operation. The upsampling process includes transposed convolution with a stride of 2, a normalization layer, and a ReLU activation layer operation.
[0019] A self-attention feature fusion model fuses features from the decoder and encoder. The self-attention feature fusion model includes the following steps: the decoder feature input is used as a value tensor, and the encoder features are used as queries and keys. Then, by calculating the similarity between the queries and keys, an attention weight vector is obtained. This vector is applied to the value tensor to generate a weighted sum to obtain the dependencies at different positions.
[0020] In the above technical solution, in the encoder structure model and decoder structure model, for the input feature F After passing through the first convolutional block with a kernel size of 3 and a stride of 1, we obtain... In the encoder and decoder, C is equal to C′ / 2 and 2C respectively, where C represents the number of feature channels, and F... i The definition for obtaining is as follows:
[0021] F i =ReLu(BN(Conv) 3×3 (F))) (1)
[0022] ReLU() means activation using the ReLU function, BN() means batch normalization, and Conv 3×3 () means a convolution operation with a kernel size of 3×3 and a stride of 1;
[0023] Then, the output F of the first convolutional block i As input to the GC block, it first undergoes a 3×3 convolution and a reshape operation to obtain... The definition is as follows:
[0024]
[0025] Reshape() means to change the size of the image;
[0026] After performing a softmax operation on it to obtain the weights, and then F i Reshaping yields the C×HW feature map, and pairwise computation is performed to obtain the attention features.
[0027]
[0028] Reshape() means to change the image size, and softmax() means to activate using the softmax function;
[0029] Then, the obtained attention features F gThe feature map is obtained by performing a transform operation on the input, followed by a 3×3 convolutional layer 1, Layer Norm, ReLU, and a 3×3 convolutional layer 2. Defined as:
[0030] F c =Conv 3×3 (ReLU(LN(Conv 3×3 (F g (4)
[0031] LN() means performing layer normalization, which will normalize F c As a residual term, it is added to the input feature F. i The final output of the GC block is obtained through residual connections.
[0032] F o =F c +F i (5)
[0033] The + operation represents the summation operation between broadcast elements;
[0034] GC block output F o After passing through convolution block 2, the input feature F is adjusted for the number of channels using a 1×1 convolution and then added as a residual term to the output of convolution block 2 to obtain the final output of RGCTB.
[0035] F out =ReLu(BN(Conv) 3×3 (F o )))+Conv 1×1 (F) (6).
[0036] In the above technical solution, the self-attention feature fusion model includes decoder features. and encoder features The query is obtained by performing 3×3 convolution and reshape operations respectively. key value Their channel number C′=4, respectively for and Pyramid average pooling was used, with an output size of (1, 3, 6, 8). and The size changed from C′×HW to C′×S, where:
[0037]
[0038] The calculation of cross-attention between query and key is described as follows:
[0039]
[0040] Then, After performing the softmax operation, the weights are obtained and then compared with... Attention features are obtained by performing pairwise computation.
[0041]
[0042] Finally, the number of channel features is adjusted back from C′ to C using a 1×1 convolution, and then combined with the decoder features F. d The final feature fusion result is obtained by performing residual connections.
[0043]
[0044] in, Residual joins representing element-wise addition.
[0045] In the above technical solution, accuracy, sensitivity, specificity, and the area under the receiver operating characteristic curve are used as evaluation indicators to evaluate the results of color fundus vessel segmentation.
[0046] The present invention also provides a color fundus vessel segmentation device, characterized in that it comprises:
[0047] The color fundus image dataset acquisition module is used to acquire color fundus image datasets;
[0048] A color fundus image dataset preprocessing module is used to preprocess the color fundus image dataset to obtain a processed color fundus image dataset;
[0049] The model building module is used to build a residual self-attention feature fusion UNet model;
[0050] The model training module is used to train the residual self-attention feature fusion UNet model to obtain the trained residual self-attention feature fusion UNet model.
[0051] The color fundus vessel segmentation result determination module is used to input the processed color fundus image dataset into the trained residual self-attention feature fusion UNet model to obtain the color fundus vessel segmentation result.
[0052] In the above-mentioned apparatus, the method for processing a color fundus image dataset is characterized in that a color fundus image dataset preprocessing unit is used to perform edge filling, CLAHE algorithm enhancement, rotation and vertical flip enhancement processing on the color fundus image dataset to obtain a processed color fundus image dataset.
[0053] In the above-mentioned device, the self-attention feature fusion UNet model specifically includes:
[0054] The encoder structure model is used to design an encoder structure model to extract the most representative and discriminative high-level semantic information features of the image. The design of the encoder structure model consists of three steps, each of which includes an RGCTB and a downsampling operation. The downsampling process includes convolution with a kernel of 4 and a stride of 2, a normalization layer, and a ReLU activation layer.
[0055] A decoder structure model is used to design a decoder structure model to recover the initial image information. The decoder structure model includes three steps, each of which includes an RGCTB and an upsampling operation. The upsampling process includes transposed convolution with a stride of 2, a normalization layer, and a ReLU activation layer operation.
[0056] The self-attention feature fusion unit fuses features from the decoder and encoder. The self-attention feature fusion model includes the following steps: the decoder feature input is used as a value tensor, and the encoder features are used as queries and keys. Then, by calculating the similarity between the queries and keys, an attention weight vector is obtained. This vector is applied to the value tensor to generate a weighted sum to obtain the dependencies at different positions.
[0057] In the aforementioned device, the encoder structure model and decoder structure model, for input features After passing through the first convolutional block with a kernel size of 3 and a stride of 1, we obtain... In the encoder and decoder, C is equal to C′ / 2 and 2C respectively, where C represents the number of feature channels, and F... i The definition for obtaining is as follows:
[0058] F i =ReLu(BN(Conv) 3×3 (F))) (1)
[0059] ReLU() means activation using the ReLU function, BN() means batch normalization, and Conv 3×3 () means a convolution operation with a kernel size of 3×3 and a stride of 1;
[0060] Then, the output F of the first convolutional blocki As input to the GC block, it first undergoes a 3×3 convolution and a reshape operation to obtain... The definition is as follows:
[0061]
[0062] The meaning of Reshape() is to change the image size.
[0063] After performing a softmax operation on it to obtain the weights, and then F i Reshaping yields the C×HW feature map, and pairwise computation is performed to obtain the attention features.
[0064]
[0065] Reshape() means to change the image size, and softmax() means to activate using the softmax function;
[0066] Then, the obtained attention features F g The feature map is obtained by performing a transform operation on the input, followed by a 3×3 convolutional layer 1, Layer Norm, ReLU, and a 3×3 convolutional layer 2. Defined as:
[0067] F c =Conv 3×3 (ReLU(LN(Conv 3×3 (F g (4)
[0068] F c As a residual term, it is added to the input feature F. i The final output of the GC block is obtained through residual connections.
[0069] F o =F c +F i (5)
[0071] The + operation represents the summation operation between broadcast elements;
[0072] GC block output F o After passing through convolution block 2, the input feature F is adjusted for the number of channels using a 1×1 convolution and then added as a residual term to the output of convolution block 2 to obtain the final output of RGCTB.
[0073] F out =ReLu(BN(Conv) 3×3 (F o )))+Conv 1×1 (F) (6).
[0074] Because the present invention employs the above-mentioned technical means, it has the following beneficial effects:
[0075] This invention provides a method for blood vessel segmentation in color fundus images. The method includes: acquiring a color fundus image dataset; processing the color fundus image dataset to obtain a processed color fundus image dataset; constructing a residual self-attention feature fusion UNet model; training the residual self-attention feature fusion UNet model to obtain a trained residual self-attention feature fusion UNet model; and inputting the processed color fundus image dataset into the trained residual self-attention feature fusion UNet model to obtain the blood vessel segmentation result of the color fundus image. By employing residual self-attention feature fusion, effective blood vessel information in the encoder can be effectively selected, while useless lesion and tissue information can be filtered out, achieving the desired effect. Figure 7 As shown, by employing a residual global context module, information about small blood vessels can be effectively preserved during feature extraction, solving the problem of difficult segmentation of blood vessels located at edges, in low-contrast, and with uneven brightness. This achieves high-resolution, high-continuity, more complete, and more detailed blood vessel segmentation results, such as... Figure 8 As shown. Attached Figure Description
[0076] Figure 1 A diagram of the Residual Global Context Module (RGCTB);
[0077] Figure 2 Here is a diagram of the pyramid pooling structure;
[0078] Figure 3 Here is a structural diagram of the Feature Fusion Module (FSTB);
[0079] Figure 4 This is a diagram of the SF-UNet network architecture.
[0080] Figure 5 A visualization of the overall segmentation effect;
[0081] Figure 6 This is a comparison image of the segmentation results;
[0082] Figure 7 The image shows a comparison of the performance of the Feature Fusion Module (FSTB).
[0083] Figure 8A comparison chart showing the effects of the Residual Global Context Module (RGCTB);
[0084] Figure 9 This is a flowchart of the color fundus image blood vessel segmentation method of the present invention. Detailed Implementation
[0085] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.
[0086] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.
[0087] This invention provides a method for blood vessel segmentation in color fundus images, the method comprising the following steps:
[0088] Step 1: Obtain the retinal image dataset;
[0089] Step 2: Preprocess the color fundus image dataset to obtain a preprocessed color fundus image dataset;
[0090] Step 3: Construct a residual self-attention feature fusion UNet model;
[0091] Step 4: Train the residual self-attention feature fusion UNet model to obtain the trained residual self-attention feature fusion UNet model;
[0092] Step 5: Input the preprocessed color fundus image dataset into the trained residual self-attention feature fusion UNet model for training to obtain the color fundus blood vessel segmentation result.
[0093] Furthermore, the process of processing the color fundus image dataset to obtain the processed retinal image dataset is characterized by performing edge filling, CLAHE algorithm enhancement, rotation and vertical flip enhancement processing on the color fundus image dataset to obtain the processed color fundus image dataset.
[0094] Furthermore, the residual self-attention feature fusion UNet model is an encoder-decoder structure model.
[0095] Furthermore, the self-attention feature fusion UNet model is characterized by: an encoder structure model design unit, used to design an encoder structure model to extract the most representative and discriminative high-level semantic information features of the image; the encoder structure model design consists of three steps, each including an RGCTB and a downsampling operation; the downsampling process includes a convolution with a kernel of 4 and a stride of 2, a normalization layer, and a ReLU activation layer; a decoder structure model design unit, used to design a decoder structure model to recover the initial image information; the decoder structure model includes three steps, each including an RGCTB and an upsampling operation; the upsampling process includes a transposed convolution with a stride of 2, a normalization layer, and a ReLU activation layer; and a self-attention feature fusion unit, fusing the features of the decoder and encoder; the self-attention feature fusion model includes the following steps, wherein the decoder feature input is used as a value tensor, and the encoder features are used as queries and keys. Then, by calculating the similarity between the queries and keys, an attention weight vector is obtained, which is applied to the value tensor to generate a weighted sum to obtain the dependencies at different positions.
[0096] Furthermore, the accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve are used as evaluation indicators to evaluate the color fundus vessel segmentation results.
[0097] like Figure 1 As shown, this invention proposes a residual self-attention feature fusion UNet model method, which specifically includes the following steps:
[0098] Step 1: Obtain the retinal image dataset, which includes DRIVE, CHASE DB1, and
[0099] STARE, details are shown in Table 1.
[0100] Table 1. Detailed information for the three datasets: DRIVE, CHASE DB1, and STARE.
[0101]
[0102] Step 2: Process the retinal image dataset to obtain the processed retinal image dataset.
[0103] Specifically, this includes performing edge padding, CLAHE algorithm enhancement, rotation, and vertical flip enhancement on the retinal image dataset to obtain the processed retinal image dataset.
[0104] Step 3: Construct the residual self-attention feature fusion UNet model. The residual self-attention feature fusion UNet model is an encoder-decoder structure model, specifically including:
[0105] Design an encoder structure model to extract the most representative and discriminative high-level semantic information features of the image. The design of the encoder structure model consists of three steps, each of which includes an RGCTB and a downsampling operation. The downsampling process includes convolution with a kernel of 4 and a stride of 2, a normalization layer, and a ReLU activation layer.
[0106] Design a decoder structure model to recover the initial image information. The decoder structure model includes three steps, each of which includes an RGCTB and an upsampling operation. The upsampling process includes transposed convolution with a stride of 2, a normalization layer, and a ReLU activation layer operation.
[0107] For capturing long-range dependencies and performing global top-to-bottom modeling, traditional convolution is inefficient and difficult to achieve because a convolution operation models local pixel relationships. Although the receptive field of neurons in the latter half of the network can be increased by continuously deepening the network, achieving a similar effect to global modeling, this crude approach of deepening the network has three drawbacks: (1) it is not sophisticated enough, with a large number of parameters and computational load; (2) the deeper the network, the greater the optimization difficulty; and (3) it cannot fully represent the global context and has a maximum distance limitation.
[0108] To better aggregate spatial information, we propose RGCTB, such as... Figure 2 As shown. For input features After the first convolutional block, we get In the encoder and decoder, C is equal to C′ / 2 and 2C, respectively. F i The definition for obtaining is as follows:
[0109] F i =ReLu(BN(Conv) 3×3 (F))) (1)
[0110] Then, the output F of convolution block 1 i As input to the GC block, it first undergoes a 3×3 convolution and a reshape operation to obtain... The definition is as follows:
[0111]
[0112] After performing a softmax operation on it to obtain the weights, and then F i Reshaping yields the C×HW feature map, and pairwise computation is performed to obtain the attention features.
[0113]
[0114] Then, we obtain the attention features F g The feature map is obtained by performing a transform operation on the input, followed by a 3×3 convolutional layer 1, Layer Norm, ReLU, and a 3×3 convolutional layer 2. Defined as:
[0115] F c =Conv 3×3 (ReLU(LN(Conv 3×3 (F g (4)
[0116] F c As a residual term, it is added to the input feature F. i The final output of the GC block is obtained through residual connections.
[0117] F o =F c +F i (5)
[0118] The + operation represents the summation operation between broadcast elements.
[0119] GC block output F o After passing through convolution block 2, the input feature F is adjusted for the number of channels using a 1×1 convolution and then added as a residual term to the output of convolution block 2 to obtain the final output of RGCTB.
[0120] F out =ReLu(BN(Conv) 3×3 (F o )))+Conv 1×1 (F) (6)
[0121] Design a self-attention feature fusion unit to fuse features from the decoder and encoder. The self-attention feature fusion model includes the following steps: the decoder feature input is used as a value tensor, and the encoder features are used as queries and keys. Then, by calculating the similarity between the queries and keys, an attention weight vector is obtained. This vector is applied to the value tensor to generate a weighted sum to obtain dependencies at different positions.
[0122] To better select features in the skip-level structure, we designed a self-attention feature fusion unit, the structure of which is as follows: Figure 3 As shown. Decoder features and encoder features The query is obtained by performing 3×3 convolution and reshape operations respectively. key value Their channel count C′ = 4. To reduce computational load and memory usage while preserving as much information as possible, we respectively... and Pyramid average pooling was used, such as Figure 2 As shown, the average output size of pyramid pooling is (1, 3, 6, 8). Therefore, and The size changed from C′×HW to C′×S, where:
[0123]
[0124] The calculation of cross-attention between query and key is described as follows:
[0125]
[0126] Then, After performing the softmax operation, the weights are obtained and then compared with... Attention features are obtained by performing pairwise computation.
[0127]
[0128] Finally, the number of channel features is adjusted back from C′ to C using a 1×1 convolution, and then combined with the decoder features F. d The final feature fusion result is obtained by performing residual connections.
[0129]
[0130] in, Residual joins representing element-wise addition.
[0131] Step 4: Train the residual self-attention feature fusion UNet model to obtain the trained residual self-attention feature fusion UNet model.
[0132] During the training phase, the learning rate was set to 0.001, the batch size was set to 2, the Adam algorithm was selected as the optimization method, and the binary cross-entropy was used as the loss function.
[0133] Step 5: The retinal vessel segmentation results are evaluated using accuracy, sensitivity, specificity, and the area under the receiver operating characteristic (ROC) curve as evaluation metrics. The above steps quantitatively evaluate the performance of the residual self-attention feature fusion UNet model. The definitions of accuracy (ACC), sensitivity (SEN), specificity (SPE), and the area under the ROC curve (AUC) are as follows:
[0134]
[0135]
[0136]
[0137] Table 2 shows the results of other methods on the DRIVE dataset:
[0138]
[0139] Table 3 shows the results of other methods on the CHASE-DB1 dataset:
[0140]
[0141] Table 4 shows the results of other methods on the STARE dataset:
[0142]
[0143] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for segmenting a color fundus image, characterized in that, The method includes the following steps: Step 1: Obtain a dataset of color fundus images; Step 2: Preprocess the color fundus image dataset to obtain a preprocessed color fundus image dataset; Step 3: Construct a residual self-attention feature fusion UNet model; Step 4: Train the residual self-attention feature fusion UNet model to obtain the trained residual self-attention feature fusion UNet model; Step 5: Input the preprocessed color fundus image dataset into the trained residual self-attention feature fusion UNet model for training to obtain the color fundus blood vessel segmentation result; The residual self-attention feature fusion UNet model specifically includes: An encoder structure model is used to extract the most representative and discriminative high-level semantic information features of images. The implementation of the encoder structure model consists of three steps, each of which includes an RGCTB structure and a downsampling structure. The RGCTB structure adds a global context block (GC block) to two convolutional blocks to preserve important information in space and channels. The downsampling process includes convolution with a kernel of 3 and a stride of 2, normalization layer, and ReLU activation layer operations to reduce the image size and increase the receptive field. The decoder structure model recovers the initial image information. The decoder structure model includes three steps, each of which includes an RGCTB and an upsampling operation. The upsampling process includes transposed convolution with a stride of 2, a normalization layer, and a ReLU activation layer operation. A self-attention feature fusion model fuses features from the decoder and encoder. The self-attention feature fusion model includes the following steps: the decoder feature input is used as a value tensor, and the encoder features are used as queries and keys. Then, by calculating the similarity between the queries and keys, an attention weight vector is obtained. This vector is applied to the value tensor to generate a weighted sum to obtain the dependencies at different positions. In the encoder and decoder structural models, for input features After passing through the first convolutional block with a kernel size of 3 and a stride of 1, we obtain... In the encoder and decoder, They are respectively equal to and ,in The meaning is the number of feature channels. The definition for obtaining is as follows: (1) in This means activating using the ReLU function. The meaning is to standardize. This means a convolution operation with a kernel size of 3×3 and a stride of 1. Then, the output of the first convolutional block As input to a GC block, it first passes through a... Convolution and reshape operations yield The definition is as follows: (2) in, It means to change the image size; After performing a softmax operation on it to obtain the weights, and then... reshape obtained The attention features are obtained by pairwise computation of the feature maps. : (3) It means to change the image size. This means using the softmax function for activation; Then, the obtained attention features The input is used for the transform operation, after which... Convolutional layer 1, layernorm, ReLU and Feature map obtained from convolutional layer 2 Defined as: (4) The meaning is to perform layer normalization, and... As a residual term, it is added to the input features. The final output of the GC block is obtained through residual connections. : (5) The + operation represents the summation operation between broadcast elements; GC block output After passing through convolution block 2, the input features are... pass After adjusting the number of channels in the convolution, the residual term is added to the output of convolution block 2 to obtain the final output of RGCTB. : (6)。 2. The color fundus image segmentation method according to claim 1, characterized in that, The preprocessing of the color fundus image dataset to obtain the preprocessed color fundus image dataset specifically includes: The color fundus image dataset is subjected to edge filling, CLAHE algorithm enhancement, rotation and vertical flip enhancement to obtain the processed color fundus image dataset.
3. The color fundus image segmentation method according to claim 2, characterized in that, Self-attention feature fusion model: Decoder features and encoder features Passing through Convolution and reshape operations yield the query ,key ,value Their channel number , respectively and Pyramid average pooling was used, with an output size of (1, 3, 6, 8). and Size by It became ,in: (7) The calculation of cross-attention between query and key is described as follows: (8) Then, After performing the softmax operation, the weights are obtained and then compared with... Attention features are obtained by performing pairwise computation. : (9) Finally, through Convolution reduces the number of channel features from Adjust back and decoder features The final feature fusion result is obtained by performing residual connections. : (10) in, Residual joins representing element-wise addition.
4. The color fundus image segmentation method according to claim 1, characterized in that, The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve were used as evaluation indicators to evaluate the color fundus vessel segmentation results.
5. A color retinal vascular segmentation device, characterized in that, include: The color fundus image dataset acquisition module is used to acquire color fundus image datasets; A color fundus image dataset preprocessing module is used to preprocess the color fundus image dataset to obtain a processed color fundus image dataset; The model building module is used to build a residual self-attention feature fusion UNet model; The model training module is used to train the residual self-attention feature fusion UNet model to obtain the trained residual self-attention feature fusion UNet model. The color fundus vessel segmentation result determination module is used to input the processed color fundus image dataset into the trained residual self-attention feature fusion UNet model to obtain the color fundus vessel segmentation result; The self-attention feature fusion UNet model specifically includes: The encoder structure model is used to design an encoder structure model to extract the most representative and discriminative high-level semantic information features of the image. The design of the encoder structure model consists of three steps, each of which includes an RGCTB and a downsampling operation. The downsampling process includes convolution with a kernel of 4 and a stride of 2, a normalization layer, and a ReLU activation layer. A decoder structure model is used to design a decoder structure model to recover the initial image information. The decoder structure model includes three steps, each of which includes an RGCTB and an upsampling operation. The upsampling process includes transposed convolution with a stride of 2, a normalization layer, and a ReLU activation layer operation. The self-attention feature fusion unit fuses features from the decoder and encoder. The self-attention feature fusion model includes the following steps: the decoder feature input is used as a value tensor, and the encoder features are used as queries and keys. Then, by calculating the similarity between the queries and keys, an attention weight vector is obtained. This vector is applied to the value tensor to generate a weighted sum to obtain the dependencies at different positions. In the encoder and decoder structural models, for input features After passing through the first convolutional block with a kernel size of 3 and a stride of 1, we obtain... In the encoder and decoder, They are respectively equal to and ,in The meaning is the number of feature channels. The definition for obtaining is as follows: (1) in This means activating using the ReLU function. The meaning is to standardize. This means a convolution operation with a kernel size of 3×3 and a stride of 1. Then, the output of the first convolutional block As input to a GC block, it first passes through a... Convolution and reshape operations yield The definition is as follows: (2) in, It means changing the image size. After performing a softmax operation on it to obtain the weights, and then... reshape obtained The attention features are obtained by pairwise computation of the feature maps. : (3) It means to change the image size. This means using the softmax function for activation; Then, the obtained attention features The input is used for the transform operation, after which... Convolutional layer 1, layernorm, ReLU and Feature map obtained from convolutional layer 2 Defined as: (4) Will As a residual term, it is added to the input features. The final output of the GC block is obtained through residual connections. : (5) Where + represents the summation operation; GC block output After passing through convolution block 2, the input features are... pass After adjusting the number of channels in the convolution, the residual term is added to the output of convolution block 2 to obtain the final output of RGCTB. : (6)。 6. A color fundus vascular segmentation device according to claim 5, characterized in that, The color fundus image dataset preprocessing module is used to perform edge filling, CLAHE algorithm enhancement, rotation and vertical flip enhancement processing on the color fundus image dataset to obtain the processed color fundus image dataset.