Retinal vessel segmentation method and system based on multi-level attention feature fusion

Abstract

The invention provides a retinal vessel segmentation method and system based on multilevel attention feature fusion. The method is specially designed for realizing an accurate retinal vessel segmentation task. Firstly, data is divided into blocks mainly comprising coarse blood vessels and fine blood vessels in a preprocessing part; and data enhancement is performed on the block mainly comprising the fine blood vessels, so that the proportion of the two types of samples tends to be balanced, and the recognition performance of the fine blood vessels is improved. Then, a multi-level attention feature fusion module is provided, and by establishing attention interaction between deep semantic features and shallow spatial details, blood vessel features of different scales are effectively captured, so that the segmentation ability of thick and thin blood vessels is cooperatively enhanced. In order to further enhance the continuity of the vascular structure and the accuracy of the boundary, the multi-constraint guide loss is designed, and the segmentation performance of the network on the microvessels and the fuzzy boundary is optimized by constraining the vascular skeleton and edge information at the same time.

Description

Technical Field

[0001] This invention proposes a method and system for retinal vessel segmentation based on multi-level attention feature fusion, which relates to the field of image processing technology. Background Technology

[0002] During the acquisition process, retinal images are easily affected by factors such as uneven lighting, noise pollution, and low contrast, which can lead to blurred blood vessel edges and structural breaks, especially in the microvascular area, which is even more difficult to extract accurately.

[0003] Retinal vessels, as key structures in fundus images, require precise segmentation for quantitative disease assessment and treatment decisions. However, retinal images are susceptible to interference from uneven lighting, noise contamination, and low contrast during acquisition, leading to blurred vessel edges and structural breaks, especially in the microvascular region, which is particularly difficult to extract accurately. Traditional segmentation methods often perform unpredictably when faced with complex structures and diverse image quality, failing to meet clinical application needs. In recent years, with the rapid development of deep learning technology in medical image processing, constructing automatic segmentation models with strong robustness and structure awareness has gradually become a research hotspot for improving the accuracy of retinal image analysis. Generally speaking, retinal vessel segmentation methods can be broadly divided into two categories: unsupervised segmentation methods and supervised segmentation methods. Unsupervised segmentation methods explore implicit patterns and regularities in unlabeled data without relying on manual annotation. Common methods include morphological methods, filtering, vessel tracking, and thresholding. These methods are simple to implement, low-cost, and highly interpretable, making them suitable for scenarios with insufficient samples. However, due to their reliance on low-level features, they are susceptible to noise interference and struggle to handle complex structures and high-level semantics, resulting in limited segmentation accuracy and robustness. In contrast, supervised segmentation methods rely on large amounts of labeled image data, learning vascular structural features in images by constructing training sets, and often combining machine learning or deep learning techniques for modeling. Early supervised methods mainly used traditional classifiers such as Support Vector Machines (SVM) and Random Forests to extract handcrafted features such as color, texture, and edges for pixel-level classification. With the development of deep neural networks, the emergence of the U-Net architecture has driven the development of medical image segmentation, improving the speed and accuracy of retinal vessel segmentation and better assisting doctors in diagnosing ophthalmic diseases. After the introduction of U-Net, a series of U-Net variants were introduced, which can learn complex spatial structures and contextual information in images end-to-end, improving segmentation accuracy and generalization ability. For example, through feature extraction and feature embedding mechanisms, combined with depth magnification strategies, the accuracy of feature representation and the suppression of interference information are effectively improved, and high-precision segmentation mask generation is achieved by utilizing dense multi-scale feature fusion. However, problems such as weak fine vessel recognition ability and discontinuity still exist. The following are the main challenges currently facing retinal vessel segmentation.

[0004] First, a significant class imbalance exists between coarse and fine blood vessels in fundus images, with fine vessels occupying a smaller proportion of pixels. This causes the model to tend to learn the discriminative features of coarse vessels during training, thus affecting the recognition and segmentation performance of fine vessels. Second, traditional encoder-decoder structures often cannot simultaneously consider both superficial detail sensitivity and deep semantic discriminative ability during feature extraction. This leads to the gradual weakening of fine vessels during stepwise downsampling and fusion, affecting the integrity of the overall vascular structure. Furthermore, fundus images often contain complex background information such as noise interference, insufficient contrast, and various pathological features, which can easily lead to blurred vascular boundaries and structural inconsistencies. This increases the difficulty of extracting and reconstructing continuous vascular structures, thus limiting the model's robustness and generalization ability in practical clinical applications. Summary of the Invention

[0005] In view of this, in order to fill the gaps and deficiencies of the existing technology, this invention proposes a retinal vessel segmentation method and system based on multi-level attention feature fusion, which can effectively solve problems such as class imbalance, difficulty in identifying fine vessels and blurred boundaries, and achieve high-precision retinal vessel segmentation.

[0006] This invention proposes a method and system for retinal vessel segmentation based on multi-level attention feature fusion, comprising the following:

[0007] This invention proposes a retinal vessel segmentation method based on multi-level attention feature fusion, characterized by the following:

[0008] Step S1: Preprocess the dataset containing images of the target retinal vessels, then establish an automatic classification method for thick and thin blood vessel blocks and classify the preprocessed data into blocks of thick blood vessels and blocks of thin blood vessels.

[0009] Step S2: Establish a multi-level attention feature fusion module, which includes adding a multi-level attention feature fusion module between the encoder and decoder to combine spatial detail information in shallow features with semantic context information in deep features; wherein the encoder extracts multi-scale features from the input target retinal blood vessel image, the multi-level attention feature fusion module fuses and enhances deep features and shallow features, and then the decoder extracts feature maps and generates the final fusion result of deep and shallow features;

[0010] Step S3: Establish a multi-constraint guided loss function to enhance the connectivity and boundary clarity of the vascular skeleton;

[0011] Step S4: Establish an evaluation mechanism and verify it.

[0012] Further, step S1 includes the following:

[0013] Step S11: Preprocessing the dataset containing retinal vessel images includes: selecting the retinal vessel image in the green channel of the RGB three-color channel as the target retinal vessel image; and then using contrast-limited adaptive histogram equalization and gamma correction on the target retinal vessel image to enhance image contrast.

[0014] Furthermore, step S1 also includes the following:

[0015] Step S12: Establish an automatic method for classifying blood vessel blocks of different sizes, including the following:

[0016] Step S121: Extract the vascular skeleton from the target retinal vascular image using the skeleton line algorithm, denoted as S; where the skeleton can effectively preserve the centerline structure of the blood vessel while removing redundant edge information;

[0017] Step S122: Based on the skeleton extraction, the blood vessel width R at the skeleton point is calculated using distance transformation; where the distance transformation assigns a value to each pixel, which represents its Euclidean distance to the nearest background pixel;

[0018] Step S123: Automatic binary classification of blood vessel width using the Otsu threshold method yields the following results:

[0019] T=Otsu(R)

[0020] Where T is the optimal width threshold for classifying blood vessel thickness, and Sr represents the set of blood vessel widths corresponding to all skeleton points;

[0021] Next, the total width of all skeleton points is calculated and divided into the sum of capillary widths R according to a threshold. thin The sum of the widths of the thickest blood vessels, R thick The following content was obtained:

[0022]

[0023] Finally, classification is based on the following rules: If R thin >R thick If the target retinal vascular image is considered to have blocks of fine blood vessels, then the blocks are considered to have blocks of thick blood vessels.

[0024] Further, step S2 includes the following:

[0025] Step S21: Based on U-Net, a multi-level attention feature fusion module is added between the encoder and the decoder; the encoder extracts multi-scale features from the input target retinal blood vessel image, the multi-level attention feature fusion module fuses and enhances the multi-scale features, and then the decoder uses the enhanced features to generate the final segmentation result.

[0026] Step S22: The multi-level attention feature fusion module includes a deep feature fusion module and a shallow feature fusion module;

[0027] Step S221: The deep feature fusion module preserves the semantic information of deep features. The establishment of the deep feature fusion module includes the following:

[0028] Feature maps d4 and d3 are upsampled and then convolved with 3×3 to ensure they have consistent spatial dimensional parameters, including size and number of channels. Then, an SE attention mechanism is introduced for element-wise addition and fusion to generate a channel-dimensional feature weight map M1. This weight map M1 is then applied to both feature maps d4 and d3, and the two weighted feature maps are concatenated along the channel dimension. A 3×3 convolution operation is then used to further extract the deep semantic information from the fused feature maps, resulting in a deep feature fusion module. The SE attention mechanism includes global average pooling, a fully connected layer, ReLU activation function, and a sigmoid function.

[0029] The expressions for the deep feature fusion module include the following:

[0030] M1=SE(UpConv(d4)+UpConv(d3))

[0031]

[0032] Among them, Conv 3×3 () represents a 3×3 convolution operation; SE() refers to the channel attention operation; M1 is the feature weight map generated by SE attention; UpConv() represents an upsampling and convolution operation; d3 and d4 are feature maps of deep features; Out DFI This represents the output of the deep feature fusion module;

[0033] Step S222: The shallow feature fusion module is used to enhance the expression of local details and edge information in shallow features and improve the segmentation sensitivity of the vascular region. The establishment of the shallow feature fusion module includes the following:

[0034] Feature map dec2 is upsampled and subjected to a 1×1 convolution to ensure that feature maps d2 and d1 have consistent spatial dimensional parameters, including spatial size and number of channels. Further, based on a lightweight attention-guided mechanism, intermediate features are obtained through 1×1 convolution, batch normalization, and ReLU activation. These intermediate features are then used to generate an attention weight map M2 via a sigmoid function. The M2 weight map adaptively emphasizes the responses of detailed regions such as edges and textures. M2 is then applied to feature maps d2 and d1 respectively to selectively enhance shallow features. Finally, the weighted features are concatenated along the channel dimension, and fusion information is further extracted through a 3×3 convolution to obtain the shallow feature fusion module. The expression for the shallow feature fusion module includes the following:

[0035] M2=σ(BN(Conv 1×1 (ReLU(UpConv(d2)+d1))))

[0036]

[0037] Where M2 is the feature weight map, Concat(·) represents the concatenation operation, and ReLU is the activation function;

[0038] It is an element-wise multiplication operation, where d1 and d2 are feature maps of shallow features; Out SFI This is the output of the shallow feature fusion module.

[0039] Furthermore, step S2 also includes the following:

[0040] Step S23: Using a fusion graph neural network module built based on a graph attention network, the deep feature fusion module and the shallow feature fusion module are fused together, and finally the deep and shallow feature fusion module is output; the process of fusing the deep feature fusion module and the shallow feature fusion module includes the following:

[0041] A graph attention network is introduced; deep and shallow features are concatenated in the channel dimension, and feature compression and enhancement are performed through convolution and SE attention mechanism. Then, the features are mapped to the graph structure, and edge index is constructed based on 4-adjacency relationship. Adaptive feature aggregation is performed on the graph structure through GAT. GAT can automatically learn attention weights based on the similarity between different nodes, thereby mining more complex spatial dependencies and structural constraints between regions in the image.

[0042] The expression for fusing the deep feature fusion module and the shallow feature fusion module includes the following:

[0043] Out DS =Concat(OutDFI Out SFI )

[0044] Out = GAT(Conv) 3×3 (Out DS ))

[0045] Out DS This indicates that the outputs of the deep feature fusion module and the shallow feature fusion module are concatenated; GAT() represents the graph convolution operation; Out is the output of the deep feature fusion module. DS The final result is obtained by performing a 3×3 convolution operation on the basis of graph convolution operation.

[0046] Further, step S3 includes the following:

[0047] Step S31: Establish a multi-constraint guided loss function to optimize the output of the multi-level attention feature fusion module, including the following:

[0048] A multi-constraint guided loss function based on skeleton extraction and Canny edge computation is adopted, wherein the multi-constraint guided loss function includes a combination of BCE and Dice loss;

[0049] The edge details and centerline features of the target retinal vessel image are captured and enhanced. The expression of the multi-constraint guided loss function, which combines skeleton extraction and the Canny edge operator, includes the following:

[0050]

[0051] Among them, L Dice It is the Dice loss function, L BCE It is the BCE loss function, L BD It is a loss function that combines the BCE loss function and the Dice loss function; Y refers to the true label. This is the segmentation map of the model, with a λ value of 0.5;

[0052] Step S32: Perform vascular skeletonization on the vascular annotation map of the target retinal vessel image after edge detail and centerline feature capture and enhancement, and segment the vascular annotation map; the vascular skeletonization adopts a thinning algorithm, including the following:

[0053]

[0054] Where L1 is the skeleton-guided loss; P SD This represents the model prediction graph after dilation and skeletonization; Y SD A diagram showing the expanded, skeletalized blood vessels;

[0055] Step S33: By performing Canny edge operator operations on the annotated blood vessel map after the blood vessel skeletonization process, the contour of the blood vessel boundary is obtained; wherein the Canny edge operator includes the following:

[0056]

[0057] Where L2 is the edge-guided loss; P E (i) represents the value of the i-th pixel in the edge map obtained after edge extraction, Y E (i) indicates the actual pixel value of the edge labeled at the same position; N is the number of pixels in the image patch;

[0058] Step S33: The multi-constraint guided loss function includes the following:

[0059] L MCG =αL1+βL2+L BD

[0060] Among them, L MCG It is a multi-constraint guided loss function; the value of α is 0.5 and the value of β is 0.5.

[0061] Further, step S4 includes the following:

[0062] Step S41: Establish an evaluation mechanism, including using accuracy (Acc), sensitivity (Sen), specificity (Spe), F1 score (F1), area under the ROC curve (AUC), and Matthews correlation coefficient (MCC) to assess retinal vessel segmentation. The relevant parameters include the following:

[0063]

[0064] In this context, TP is the number of pixels that correctly predict blood vessels, TN is the number of pixels that correctly predict the background, FP is the number of pixels that incorrectly predict the background as blood vessels, and FN is the number of pixels that incorrectly predict blood vessels as background. The true positive rate (TPR) is the same as the sensitivity, and the false positive rate (FPR) is the proportion of pixels that are incorrectly predicted as blood vessels to those that are actually background.

[0065] According to a second aspect of the present invention, a retinal vessel segmentation system based on multi-level attention feature fusion is proposed for performing a retinal vessel segmentation method based on multi-level attention feature fusion as described in any one of the claims of the present invention; characterized in that the retinal vessel segmentation system based on multi-level attention feature fusion includes the following:

[0066] The data preprocessing module is used to preprocess datasets containing retinal vessel images;

[0067] A multi-level attention feature fusion module is used to effectively fuse deep and shallow features to improve the accuracy of blood vessel segmentation;

[0068] A multi-constraint guided loss module was established to enhance the connectivity and boundary clarity of the vascular skeleton.

[0069] The evaluation mechanism module is used to evaluate the effectiveness of retinal vessel segmentation.

[0070] According to a third aspect of the present invention, the present invention provides a retinal vessel segmentation system based on multi-level attention feature fusion, comprising an electronic device, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements a retinal vessel segmentation method based on multi-level attention feature fusion as described in any one of the present invention.

[0071] According to a fourth aspect of the present invention, the present invention provides a retinal vessel segmentation system based on multi-level attention feature fusion, comprising a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements a retinal vessel segmentation method based on multi-level attention feature fusion as described in any one of the present invention.

[0072] In this invention, the multi-level attention feature fusion module is represented as: Multi-levelAttentionGuidedFusion,MAGF.

[0073] MAFI-Net stands for:

[0074] A Multi-level Attention Feature Interaction Network for Retinal Vessel Segmentation represents a retinal vessel segmentation network based on multi-level attention feature interaction.

[0075] CLAHE indicates contrast-limited adaptive histogram equalization.

[0076] DFI stands for Deep Feature Fusion. SFI stands for Shallow Feature Fusion.

[0077] U-Net represents an end-to-end image segmentation model based on a convolutional neural network (CNN).

[0078] Multi-Constraint Guided Loss (MCGL) represents the multi-constraint guided loss function.

[0079] The present invention has the following advantages:

[0080] 1. The MAFI-Net network is proposed, which effectively alleviates the sample imbalance problem by automatically classifying blood vessel blocks of different sizes, and achieves accurate blood vessel segmentation.

[0081] 2. A multi-level attention feature fusion module (MAGF) was designed, which uses attention to guide the adaptive fusion of deep semantic features and shallow spatial features, taking into account both the global structure and local details of blood vessels, and effectively improving the segmentation ability of microvessels and branches.

[0082] 3. A skeleton and edge detection joint guidance mechanism (SEG) is introduced to enhance the model's perception of microvessels and blurred boundaries by strengthening the information of the vessel centerline and edges, thereby improving the coherence of the segmentation results and the clarity of the edges.

[0083] 4. Extensive experiments were conducted on three public datasets, DRIVE, STARE, and CHASEDB1, to verify the effectiveness of the present invention. Attached Figure Description

[0084] Figure 1 This is a flowchart of the steps of the present invention.

[0085] Figure 2 This is a schematic diagram of the MAFI-Net framework proposed in this invention.

[0086] Figure 3 This is a schematic diagram of the MAGF module proposed in this invention.

[0087] Figure 4 This is a schematic diagram of the fundus image preprocessing process of the present invention.

[0088] Figure 5 This is a flowchart of the classification algorithm for thick and thin blood vessel blocks in this invention.

[0089] Figure 6 This is a schematic diagram of the classification results of blood vessel-dominant image patches in the dataset of this invention.

[0090] Figure 7 This is a comparison diagram of the visualization results of the dataset of the present invention. Figure 1 .

[0091] Figure 8 This is a schematic diagram illustrating an example of fine blood vessel segmentation according to the present invention.

[0092] Figure 9 This is a schematic diagram illustrating an example of the continuity of blood vessel segmentation according to the present invention.

[0093] Figure 10 This is a comparison diagram of the visualization results of the dataset of the present invention. Figure 2 . Detailed Implementation

[0094] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0095] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0096] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0097] like Figures 1 to 10 As shown, this invention proposes a method and system for retinal vessel segmentation based on multi-level attention feature fusion, including the following:

[0098] like Figure 1 As shown, this invention proposes a retinal vessel segmentation method based on multi-level attention feature fusion, characterized by the following:

[0099] Step S1: Preprocess the dataset containing images of the target retinal vessels, then establish an automatic classification method for thick and thin blood vessel blocks and classify the preprocessed data into blocks of thick blood vessels and blocks of thin blood vessels.

[0100] Step S2: Establish a multi-level attention feature fusion module, which includes adding a multi-level attention feature fusion module between the encoder and decoder to combine spatial detail information in shallow features with semantic context information in deep features; wherein the encoder extracts multi-scale features from the input target retinal blood vessel image, the multi-level attention feature fusion module fuses and enhances deep features and shallow features, and then the decoder extracts feature maps and generates the final fusion result of deep and shallow features;

[0101] Step S3: Establish a multi-constraint guided loss function to enhance the connectivity and boundary clarity of the vascular skeleton;

[0102] Step S4: Establish an evaluation mechanism and verify it.

[0103] Further, step S1 includes the following:

[0104] Step S11: Preprocessing the dataset containing retinal vessel images includes: selecting the retinal vessel image in the green channel of the RGB three-color channel as the target retinal vessel image; and then using contrast-limited adaptive histogram equalization and gamma correction on the target retinal vessel image to enhance image contrast.

[0105] Furthermore, step S1 also includes the following:

[0106] Step S12: Establish an automatic method for classifying blood vessel blocks of different sizes, including the following:

[0107] Step S121: Extract the vascular skeleton from the target retinal vascular image using the skeleton line algorithm, denoted as S; where the skeleton can effectively preserve the centerline structure of the blood vessel while removing redundant edge information;

[0108] Step S122: Based on the skeleton extraction, the blood vessel width R at the skeleton point is calculated using distance transformation; where the distance transformation assigns a value to each pixel, which represents its Euclidean distance to the nearest background pixel;

[0109] Step S123: Automatic binary classification of blood vessel width using the Otsu threshold method yields the following results:

[0110] T=Otsu(R)

[0111] Where T is the optimal width threshold for classifying blood vessel thickness, and Sr represents the set of blood vessel widths corresponding to all skeleton points;

[0112] Next, the total width of all skeleton points is calculated and divided into the sum of capillary widths R according to a threshold. thin The sum of the widths of the thickest blood vessels, R thick The following content was obtained:

[0113]

[0114] Finally, classification is based on the following rules: If R thin >R thick If the target retinal vascular image is considered to have blocks of fine blood vessels, then the blocks are considered to have blocks of thick blood vessels.

[0115] Further, step S2 includes the following:

[0116] Step S21: Based on U-Net, a multi-level attention feature fusion module is added between the encoder and the decoder; the encoder extracts multi-scale features from the input target retinal blood vessel image, the multi-level attention feature fusion module fuses and enhances the multi-scale features, and then the decoder uses the enhanced features to generate the final segmentation result.

[0117] Step S22: The multi-level attention feature fusion module includes a deep feature fusion module and a shallow feature fusion module;

[0118] Step S221: The deep feature fusion module preserves the semantic information of deep features. The establishment of the deep feature fusion module includes the following:

[0119] Feature maps d4 and d3 are upsampled and then convolved with 3×3 to ensure consistency in spatial dimensional parameters, including spatial size and number of channels. Then, an SE attention mechanism is introduced for element-wise addition and fusion to generate a channel-dimensional feature weight map M1. This weight map M1 is then applied to both feature maps d4 and d3, and the two weighted feature maps are concatenated along the channel dimension. A 3×3 convolution operation is then used to further extract the deep semantic information from the fused feature maps, resulting in a deep feature fusion module. The SE attention mechanism includes global average pooling, a fully connected layer, ReLU as the activation function, and a sigmoid function.

[0120] The expressions for the deep feature fusion module include the following:

[0121] M1=SE(UpConv(d4)+UpConv(d3))

[0122]

[0123] Among them, Conv 3×3 () represents a 3×3 convolution operation; SE() refers to the channel attention operation; M1 is the feature weight map generated by SE attention; UpConv() represents an upsampling and convolution operation; d3 and d4 are feature maps of deep features; Out DFI This represents the output of the deep feature fusion module;

[0124] Step S222: The shallow feature fusion module is used to enhance the expression of local details and edge information in shallow features and improve the segmentation sensitivity of the vascular region. The establishment of the shallow feature fusion module includes the following:

[0125] Feature map dec2 is upsampled and subjected to a 1×1 convolution to ensure that feature maps d2 and d1 have consistent spatial dimensional parameters, including spatial size and number of channels. Further, based on a lightweight attention-guided mechanism, intermediate features are obtained through 1×1 convolution, batch normalization, and ReLU activation. These intermediate features are then used to generate an attention weight map M2 via a sigmoid function. The M2 weight map adaptively emphasizes the responses of detailed regions such as edges and textures. M2 is then applied to feature maps d2 and d1 respectively to selectively enhance shallow features. Finally, the weighted features are concatenated along the channel dimension, and fusion information is further extracted through a 3×3 convolution to obtain the shallow feature fusion module. The expression for the shallow feature fusion module includes the following:

[0126] M2=σ(BN(Conv 1×1 (ReLU(UpConv(d2)+d1))))

[0127]

[0128] Where M2 is the feature weight map, Concat(·) represents the concatenation operation, and ReLU is the activation function;

[0129] It is an element-wise multiplication operation, where d1 and d2 are feature maps of shallow features; Out SFI This is the output of the shallow feature fusion module.

[0130] Furthermore, step S2 also includes the following:

[0131] Step S23: Using a fusion graph neural network module built based on a graph attention network, the deep feature fusion module and the shallow feature fusion module are fused together, and finally the deep and shallow feature fusion module is output; the process of fusing the deep feature fusion module and the shallow feature fusion module includes the following:

[0132] A graph attention network is introduced; deep and shallow features are concatenated in the channel dimension, and feature compression and enhancement are performed through convolution and SE attention mechanism. Then, the features are mapped to the graph structure, and edge index is constructed based on 4-adjacency relationship. Adaptive feature aggregation is performed on the graph structure through GAT. GAT can automatically learn attention weights based on the similarity between different nodes, thereby mining more complex spatial dependencies and structural constraints between regions in the image.

[0133] The expression for fusing the deep feature fusion module and the shallow feature fusion module includes the following:

[0134] Out DS =Concat(OutDFF Out SFF )

[0135] Out = GAT(Conv) 3×3 (Out DS ))

[0136] Out DS This indicates that the outputs of the deep feature fusion module and the shallow feature fusion module are concatenated; GAT() represents the graph convolution operation; Out is the output of the deep feature fusion module. DS The final result is obtained by performing a 3×3 convolution operation on the basis of graph convolution operation.

[0137] In one embodiment of the present invention, the construction process of the multi-level attention feature fusion module is as follows: Figures 2 to 3 As shown.

[0138] Further, step S3 includes the following:

[0139] Step S31: Establish a multi-constraint guided loss function to optimize the output of the multi-level attention feature fusion module, including the following:

[0140] A multi-constraint guided loss function based on skeleton extraction and Canny edge computation is adopted, wherein the multi-constraint guided loss function includes a combination of BCE and Dice loss;

[0141] The edge details and centerline features of the target retinal vessel image are captured and enhanced. The expression of the multi-constraint guided loss function, which combines skeleton extraction and the Canny edge operator, includes the following:

[0142]

[0143] Among them, L Dice It is the Dice loss function, L BCE It is the BCE loss function, L BD It is a loss function that combines the BCE loss function and the Dice loss function; Y refers to the true label. This is the segmentation map of the model, with a λ value of 0.5;

[0144] Step S32: Perform vascular skeletonization on the vascular annotation map of the target retinal vessel image after edge detail and centerline feature capture and enhancement, and segment the vascular annotation map; the vascular skeletonization adopts a thinning algorithm, including the following:

[0145]

[0146] Where L1 is the skeleton-guided loss; PSD This represents the model prediction graph after dilation and skeletonization; Y SD A diagram showing the expanded, skeletalized blood vessels;

[0147] Step S33: By performing Canny edge operator operations on the annotated blood vessel map after the blood vessel skeletonization process, the contour of the blood vessel boundary is obtained; wherein the Canny edge operator includes the following:

[0148]

[0149] Where L2 is the edge-guided loss; P E (i) represents the value of the i-th pixel in the edge map obtained after edge extraction, Y E (i) indicates the actual pixel value of the edge labeled at the same position; N is the number of pixels in the image patch;

[0150] Step S33: The multi-constraint guided loss function includes the following:

[0151] L MCG =αL1+βL2+L BD

[0152] Among them, L MCG Multiple constraint guided loss function; α value is 0.5, β value is 0.5.

[0153] Further, step S4 includes the following:

[0154] Step S41: Establish an evaluation mechanism, including using accuracy (Acc), sensitivity (Sen), specificity (Spe), F1 score (F1), area under the ROC curve (AUC), and Matthews correlation coefficient (MCC) to assess retinal vessel segmentation. The relevant parameters include the following:

[0155]

[0156] In this context, TP is the number of pixels that correctly predict blood vessels, TN is the number of pixels that correctly predict the background, FP is the number of pixels that incorrectly predict the background as blood vessels, and FN is the number of pixels that incorrectly predict blood vessels as background. The true positive rate (TPR) is the same as the sensitivity, and the false positive rate (FPR) is the proportion of pixels that are incorrectly predicted as blood vessels to those that are actually background.

[0157] According to a second aspect of the present invention, a retinal vessel segmentation system based on multi-level attention feature fusion is proposed for performing a retinal vessel segmentation method based on multi-level attention feature fusion as described in any one of the claims of the present invention; characterized in that the retinal vessel segmentation system based on multi-level attention feature fusion includes the following:

[0158] The data preprocessing module is used to preprocess datasets containing retinal vessel images;

[0159] A multi-level attention feature fusion module is used to effectively fuse deep and shallow features to improve the accuracy of blood vessel segmentation;

[0160] A multi-constraint guided loss module was established to enhance the connectivity and boundary clarity of the vascular skeleton.

[0161] The evaluation mechanism module is used to evaluate the effectiveness of retinal vessel segmentation.

[0162] According to a third aspect of the present invention, the present invention provides a retinal vessel segmentation system based on multi-level attention feature fusion, comprising an electronic device, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements a retinal vessel segmentation method based on multi-level attention feature fusion as described in any one of the present invention.

[0163] According to a fourth aspect of the present invention, the present invention provides a retinal vessel segmentation system based on multi-level attention feature fusion, comprising a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements a retinal vessel segmentation method based on multi-level attention feature fusion as described in any one of the present invention.

[0164] In addition to the above, the present invention also has related embodiments, including the following:

[0165] In one embodiment of the present invention, by Figure 4 It is known that in the RGB three channels, the green channel of the fundus image shows the most prominent vascular structure, making it more suitable for retinal vessel segmentation. Besides enhancing contrast by selecting the green channel, this invention also utilizes Contrast-Limited Adaptive Histogram Equalization (CLAHE) and gamma correction with a gamma value of 1.2 to provide the model with high-contrast and high-quality images. Fundus images are difficult to acquire and label, resulting in a limited number of labeled fundus image datasets. Therefore, data augmentation methods, such as image segmentation and data augmentation, are used to expand the dataset and prevent overfitting during model training.

[0166] In one embodiment of the present invention, in deep learning, as the number of network layers increases and convolutional operations are continuously stacked, the receptive field of the feature map gradually expands, and the overlap of receptive regions between pixels also increases, leading to a certain degree of redundancy in feature information. However, these deep features often contain richer global semantic information, which is beneficial for capturing the structural features of medium to large targets. In contrast, shallow features have stronger spatial resolution and retain more local detail information, thus having an advantage in detecting small targets (such as small blood vessels). Based on the complementarity of deep and shallow features in terms of spatial detail and semantic information, effectively fusing the two helps to maintain the integrity of the overall blood vessel structure while avoiding the loss of information about small blood vessels, thereby improving the accuracy of blood vessel segmentation.

[0167] To address this, this invention proposes a Multi-level Attention Guided Fusion (MAGF) module. To more effectively combine spatial detail information from shallow features with semantic context information from deep features, this invention introduces a Graph Attention Network (GAT) mechanism. It includes Deep Feature Fusion (DFI), Shallow Feature Fusion (SFI), and a final fusion of deep and shallow features, with the overall structure as follows: Figure 2 As shown, during the decoder stage, the model extracts four sets of feature maps from different layers: d4, d3, d2, and d1, where d4 and d3 are deep features, and d2 and d1 are shallow features.

[0168] like Figure 3 As shown, in the Deep Feature Fusion (DFI) section, the Deep Feature Fusion (DFI) module established by this invention includes the following:

[0169] Feature maps d4 and d3 are upsampled and then subjected to 3×3 convolutions to adjust their size and number of channels, ensuring they have consistent spatial dimensions. Element-wise addition is then performed for fusion, and finally, two consecutive 3×3 convolutions are used to extract the deep semantic information from the fused features, resulting in the Deep Feature Fusion (DFI) module. The entire DFI module preserves the semantic information of the deep features while avoiding redundancy caused by direct concatenation.

[0170] In the Shallow Feature Fusion (SFI) section, this invention introduces an attention-guided mechanism to enhance the expressive power of shallow details. Figure 3 As shown, in the Shallow Feature Fusion (SFI) section, the Shallow Feature Fusion (SFI) module established in this invention includes the following:

[0171] Upsampling and a 1×1 convolution operation are performed on feature map d2 to resize it to the same size as feature map d1. Then, feature maps d2 and d1 are summed, and a feature weight map M is generated using a 1×1 convolution, batch normalization (BN), ReLU, and a sigmoid activation function (σ). M is used to highlight local details. The feature weight map M is multiplied by both feature maps d2 and d1, and the weighted feature map obtained by multiplying M by feature map d2 is calculated. Figure 2 The weighted features are obtained by multiplying the feature weight map M by the feature map d1 respectively. Figure 1 The modules concatenate the data along the channel dimension and further extract fused features through double convolution. Finally, they reweight the fused features along the channel and spatial dimensions to obtain the final shallow feature fusion (SFI). This module further enhances the expressive power of the target region.

[0172] Furthermore, to uncover the relationships between shallow and deep features, the results of OutDFI and OutSFI are fused element-wise to obtain OutDS. Subsequently, to enhance the spatial modeling capability of the fused features, a Fusion GNN Module constructed using a Graph Attention Network (GAT) is introduced. The process of introducing the Fusion GNN Module using the Graph Attention Network (GAT) includes the following: First, shallow and deep features are concatenated along the channel dimension, and feature compression and enhancement are performed through convolution and SE attention. Then, the features are mapped onto a graph structure, and edge indices are constructed based on 4-adjacency relationships. GAT performs adaptive feature aggregation on the graph structure, ultimately obtaining the module for fusing shallow and deep features. GAT can automatically learn attention weights based on the similarity between different nodes, thereby uncovering more complex spatial dependencies and structural constraints between regions in the image.

[0173] This fusion approach not only preserves deep semantic information and shallow spatial details, but also captures cross-regional contextual information through graph structure, significantly improving the completeness and fine-grained representation of the segmentation.

[0174] In one embodiment of the present invention, in retinal images, the edge details and centerline features of blood vessels, especially microvessels, play a crucial role in the segmentation task. To more effectively capture this structural information, the present invention further introduces a guiding module combining skeleton extraction and the Canny edge operator, based on the traditional BCE and Dice combined loss, to simultaneously constrain the learning of region, edge, and centerline features, thereby improving the network's ability to accurately represent blood vessels.

[0175] In one embodiment of the present invention, in fundus images, microvessels account for only about 23% of the total number of vascular pixels, while the remaining about 77% are composed of thicker main vessels. Therefore, during training, the model tends to prioritize learning the features of main vessels, resulting in insufficient representation of small vessels. Achieving accurate segmentation of small vessels is considered one of the key challenges in retinal vascular analysis. The vascular skeleton can effectively highlight the centerline structure of the vessels, retaining their main trunk and topological features while removing redundant edge information. The present invention uses a classic thinning algorithm to skeletonize the ground truth (GT) to obtain a fine centerline representation. This skeleton map can be used as both a supervision signal and a feature guidance signal, and its main functions are reflected in the following three aspects: (1) enhancing the network's sensitivity to small vascular structures; (2) maintaining the connectivity and orientation information of the vessels; and (3) avoiding overfitting of the network to irrelevant textures.

[0176] In one embodiment of the present invention, in order to verify the effectiveness of the method of the present invention, comprehensive experiments were conducted on three retinal vessel segmentation datasets: DRIVE, STARE, and CHASE_DB1.

[0177] In one embodiment of the present invention, the DRIVE dataset contains 40 retinal images, which are divided into a training set and a test set, each containing 20 images. Each image has a resolution of 565*584 pixels, and the blood vessel regions of the images have been manually labeled. The test set of this dataset provides blood vessel annotation files (ground truth) corresponding to the images annotated by two schemes. The present invention selects the ground truth of the first scheme as the evaluation criterion.

[0178] The STARE dataset contains 20 retinal images, each measuring 700×605 pixels. Since half of the images in this dataset are images of diseased fundus tissue, this dataset was chosen to test the robustness of the model of this invention. Following the DRIVE dataset, this invention selects the first 10 images from the dataset as the training set and the remaining 10 as the test set.

[0179] The CHASE_DB1 dataset contains 28 fundus images, consisting of fundus images from the left and right eyes of 14 samples. Each image is 999×960 pixels. This dataset can be used to test the accuracy of the model in retinal image segmentation for young children. Following most studies, the last 8 images are used for testing, while the remaining images are used as the training set. Similar to the DRIVE dataset, each image in this dataset has two labels; this invention uses the first label in the experiments.

[0180] An evaluation mechanism was established, including the use of accuracy (Acc), sensitivity (Sen), specificity (Spe), F1 score (F1), area under the ROC curve (AUC), and Matthews correlation coefficient (MCC) to assess retinal vessel segmentation. The relevant parameters include the following:

[0181]

[0182] In this context, TP is the number of pixels that correctly predict blood vessels, TN is the number of pixels that correctly predict the background, FP is the number of pixels that incorrectly predict the background as blood vessels, and FN is the number of pixels that incorrectly predict blood vessels as background. The true positive rate (TPR) is the same as the sensitivity, and the false positive rate (FRP) is the proportion of pixels that are incorrectly predicted as blood vessels to those that are actually background.

[0183] In one embodiment of the present invention, the experimental environment was as follows: the experiment was conducted on an NVIDIA 3090 GPU with 24GB of memory, using Python 3.8.19 CUDA 12.4 as the programming language. The proposed method was trained and tested on a PyTorch platform of version 2.4.0.

[0184] Parameter settings: The batch size is set to 16, and the network model will undergo 150 training iterations to learn vascular features. This invention utilizes the Adam optimizer to update the network parameters, setting the initial learning rate to 1e-4 and the weight decay value to 1e-5 to prevent overfitting. Batch normalization and ReLU are applied sequentially after each convolutional layer to accelerate model convergence and address the gradient vanishing problem. Finally, the probability map obtained from the model is converted to binary form using a threshold of 0.5.

[0185] During training, data augmentation techniques were applied, such as flipping, rotating, and adding Gaussian noise.

[0186] In one embodiment of the present invention, to verify the effectiveness of the proposed method, MAFI-Net was compared with several representative segmentation models, including U-Net, EDAE-Net, DEFFA-UNet, UKAN, and AttUKAN. Experiments were conducted on three widely used public datasets (DRIVE, STARE, and CHASE_DB1), and the segmentation performance of each method was systematically evaluated using six commonly used evaluation metrics (AUC, F1, Acc, Sen, Spe, and MCC). Tables 1, 2, and 3 present a comparison of the performance evaluation results of each method on the DRIVE, STARE, and CHASE_DB1 datasets, respectively. As shown in Table 1, on the DRIVE dataset, the proposed method achieves the best performance in AUC (0.9759); F1 (0.8115) and Sen (0.8003) rank second, only behind AttUKAN; Acc (0.9677) and MCC (0.7952) rank third, while Spe (0.9839) ranks fourth. Table 2 shows that on the STARE dataset, the proposed method achieves the best results in AUC (0.9761), F1 (0.8114), Acc (0.9748), Sen (0.7853), and MCC (0.8017), with only Spe (0.9893) ranking fourth. As shown in Table 3, on the CHASE_DB1 dataset, our method performs best in AUC (0.9828) and Sen (0.8250); it ranks third in F1 (0.8050), Acc (0.9748), and MCC (0.7923), while Spe (0.9850) also ranks fourth. A higher Sen indicates that the model can identify more small blood vessels and boundary pixels. Therefore, this result fully demonstrates MAFI-Net's strong ability in microvascular structure extraction. The experimental results show that the proposed method outperforms existing competing models in overall performance, especially in AUC and Sen. The significant improvement in AUC and Sen indicates that this model has a clear advantage in reducing missed blood vessels (especially abnormal and small vessels), thus effectively improving the diagnostic reliability of medical image analysis. Although slightly lower than some other methods in some metrics, MAFI-Net consistently leads in AUC and achieves a significant improvement of nearly 10% in Sen.

[0187] Table 1. Experimental Results of the DRIVE Dataset

[0188] Method AUC F1 Acc Sen Spe MCC UNet 0.9534 0.8058 0.9678 0.7681 0.9872 0.7909 EDAE-Net 0.9702 0.8106 0.9674 0.7872 0.9860 0.7970 DEFFA-UNet 0.9651 0.7759 0.9596 0.7977 0.9749 0.7559 UKAN 0.9529 0.7171 0.9428 0.7889 0.9537 0.6953 AttUKAN 0.9749 0.8209 0.9693 0.8074 0.9850 0.8055 MAFI-Net 0.9759 0.8115 0.9677 0.8003 0.9839 0.7952

[0189] Table 2 shows the experimental results of the STARE dataset.

[0190] Method AUC F1 Acc Sen Spe MCC UNet 0.9529 0.7933 0.9743 0.7468 0.9906 0.7861 EDAE-Net 0.9419 0.7813 0.9739 0.7272 0.9914 0.7759 DEFFA-UNet 0.9553 0.7929 0.9727 0.7704 0.9882 0.7853 UKAN 0.9064 0.6217 0.9456 0.6786 0.9646 0.5994 AttUKAN 0.9535 0.7845 0.9743 0.7292 0.9917 0.7807 MAFI-Net 0.9761 0.8114 0.9756 0.7853 0.9893 0.8017

[0191] Table 3 shows the experimental results for the CHASE_DB1 dataset.

[0192] Method AUC F1 Acc Sen Spe MCC UNet 0.9591 0.7938 0.9744 0.7840 0.9873 0.7810 EDAE-Net 0.9823 0.8083 0.9756 0.8152 0.9865 0.7955 DEFFA-UNet 0.9688 0.8143 0.9761 0.8250 0.9863 0.8025 UKAN 0.9416 0.7226 0.9603 0.8240 0.9693 0.7105 AttUKAN 0.9776 0.8024 0.9748 0.8106 0.9841 0.7897 MAFI-Net 0.9828 0.8050 0.9748 0.8250 0.9850 0.7923

[0193] Figure 7 The segmentation results of MAFI-Net on different samples are shown. It is evident that this method accurately depicts vascular structures, especially excelling in the extraction of small vessels, while effectively reducing missed vessel detection. Visualization results show that U-Net produces more red error regions compared to the proposed method, indicating its limitations in handling different vascular structures. EDAE-Net improves its ability to extract local edge information by introducing a multi-dimensional attention enhancement module. However, compared to the method of this invention, it still exhibits more red error regions in low-contrast and high-contrast areas, indicating insufficient adaptability to vascular structures in complex backgrounds. This demonstrates that MAFI-Net of this invention is more effective in extracting vessels under low-contrast conditions. DEFFA-UNet achieves feature selection and stability enhancement through a feature filtering fusion module. However, as... Figure 7 As shown, its segmentation performance for thick and thin blood vessels is still affected by class imbalance, exhibiting more significant missed detections than the method of this invention. UKAN improves U-Net by using KAN layers to enhance feature modeling capabilities, but from... Figure 7 As can be seen, it still exhibits significant missegmentation in the background region (misclassifying the background as blood vessels), while MAFI-Net demonstrates greater robustness in this regard. AttUKAN enhances model sensitivity by suppressing irrelevant feature activation and improves interpretability through nonlinear modeling of KAN blocks. However, compared to the method of this invention, its segmentation results still show more discontinuities in vascular structures. This further highlights MAFI-Net's advantage in maintaining vascular coherence and validates its effectiveness in complex retinal image segmentation tasks.

[0194] To verify the effectiveness of the model in fine vessel segmentation, this invention... Figure 8 The figure presents representative comparative results for fine vascular regions. The first row shows that existing methods often struggle to identify minute differences in fine vascular structures; the second and third rows further demonstrate their tendency to miss extremely fine vessels in low-contrast regions. In contrast, MAFI-Net exhibits a significant advantage in small vessel extraction, capable of more completely restoring fine vascular structures. As indicated by the yellow arrows in the figure, the method of this invention consistently outperforms existing technologies in terms of vascular integrity, fully demonstrating its superior ability to capture fine vascular details.

[0195] The high continuity of vascular structure is one of the core advantages of MAFI-Net. Figure 9 This characteristic is visually demonstrated, proving that the method of this invention significantly outperforms existing technologies in preserving the integrity of vascular pathways. This enhanced continuity is crucial for the accurate reconstruction of complex vascular networks in image analysis, helping to avoid structural misjudgments caused by fracture prediction.

[0196] To verify the effectiveness of the proposed Preprocess, MCGL, and MAGF modules, a systematic ablation experiment was conducted on the DRIVE dataset. This section uses U-Net as the baseline model to evaluate the contribution of each module to the performance improvement. The relevant quantification results are summarized in Table IV. Figure 10 Furthermore, intuitive visual comparisons are provided.

[0197] The benefits of the Preprocess module: Figure 10 (e) clearly demonstrates the effectiveness of integrating the Preprocess module into the baseline model. Notably, compared to the original U-Net, the U-Net+Preprocess combination exhibits stronger segmentation capabilities in fine blood vessel regions, and is particularly sensitive and robust in the identification of tiny blood vessel pixels, thereby effectively improving the overall prediction quality.

[0198] The examination of the information shown in Table 4 validated the integration of the Preprocess module into the baseline system. Specifically, the integration resulted in an increase in AUC / F1 / Acc / Sen / MCC scores, increasing by 2.25%, 0.48%, 0.02%, 1.96%, and 0.4% from 0.9534 / 0.8058 / 0.9668 / 0.7681 / 0.7909, respectively. The significant increases in AUC and Sen scores indicate an effective mitigation of missed detections, particularly in the capillary region, validating the module's crucial role in enhancing the separability of early features.

[0199] Effectiveness of MCGL: To evaluate the effect of Multi-Constraint Guided Loss (MCGL), this invention integrates it into the U-Net+Preprocess model (U-Net+Preprocess+MCGL). Figure 10 As shown in (f), the introduction of MCGL significantly improved the integrity of the vessel edge and effectively suppressed interference in local areas. The quantitative results in Table IV further demonstrate that all indicators were improved after integrating MCGL, with the F1 score reaching the highest value, fully proving the effectiveness of MCGL in enhancing edge recognition capabilities.

[0200] The validity of the MAGF module is shown in Table IV:

[0201] Integrating the MAGF module into U-Net+Preprocess+MCGL (U-Net+Preprocess+MCGL+MAGF) demonstrates its impact on segmentation effectiveness. This addition optimizes the metrics for AUC, Acc, Sen, and MCC. Figure 10 (g) provides further visualization examples, demonstrating a significant advantage, particularly in depicting minute blood vessels. Furthermore, from Figure 10 (g) The comparison of the second, third and fourth rows shows that the segmentation results have been significantly enhanced in terms of vascular continuity, which verifies the effectiveness of MAGF in fusing deep semantic features with shallow spatial details and improving structural integrity.

[0202] It is worth noting that the Spe metric decreased slightly compared to the baseline model, reflecting the trade-off between Sen and Spe. Overall, the quantitative metrics and visualization results clearly demonstrate that each component in the network makes a real contribution, and that the model achieves optimal vessel segmentation through the synergistic effect of all modules.

[0203] Table 4 Summary of Quantitative Results

[0204] Method AUC F1 Acc Sen Spe MCC U-Net 0.9534 0.8058 0.9668 0.7681 0.9872 0.7909 +Preprocess 0.9675 0.8106 0.9670 0.7877 0.9855 0.7949 +MCGL 0.9743 0.8117 0.9672 0.7887 0.9856 0.7951 +MAGF 0.9759 0.8115 0.9677 0.8003 0.9839 0.7952

[0205] In one embodiment of the present invention, Figure 2 This invention presents the MAFI-Net framework. The network consists of three parts: (a) a preprocessing module, which classifies image patches into regions dominated by thick or thin blood vessels; (b) a multi-level attention feature interaction backbone network, which captures rich semantic and spatial detail information; and (c) a multi-constraint guided loss (MCGL), which enhances the consistency between the vascular skeleton and edges, thereby improving structural continuity and boundary accuracy.

[0206] In one embodiment of the present invention, Figure 2 This is a schematic diagram of the MAFI-Net framework proposed in this invention. The network consists of three parts: (a) a preprocessing module, used to classify image patches into regions dominated by thick or thin blood vessels; (b) a multi-level attention feature interaction backbone network, used to capture rich semantic and spatial detail information; and (c) a multi-constraint guided loss (MCGL), used to enhance the consistency between the vascular skeleton and edges, thereby improving structural continuity and boundary accuracy.

[0207] In one embodiment of the present invention, Figure 3 This is a schematic diagram of the MAGF module proposed in this invention.

[0208] In one embodiment of the present invention, Figure 4This is the fundus image preprocessing flow of the present invention. From left to right, it shows: the original color fundus image, grayscale images of the red, blue, and green channels, the CLAHE processing result, and the Gamma correction result.

[0209] In one embodiment of the present invention, Figure 5 This is a flowchart of the classification algorithm for thick and thin blood vessel blocks in this invention.

[0210] In one embodiment of the present invention, Figure 6 This presents the classification results of blood vessel-dominant image patches in the three datasets of this invention. From top to bottom, the first two rows show image patches dominated by fine blood vessels and their corresponding ground truth labels, while the last two rows show image patches dominated by thick blood vessels and their corresponding ground truth labels. Arranged by columns, the first three columns correspond to the DRIVE dataset, the middle three columns to the STARE dataset, and the last three columns to the CHASEDB1 dataset.

[0211] In one embodiment of the present invention, Figure 7 This is a comparison of visualization results from the DRIVE, STARE, and CHASE_DB1 datasets presented in this invention. The three datasets are arranged from top to bottom, with each dataset occupying two rows. Red, green, and blue pixels represent false negatives (FN), true positives (TP), and false positives (FP), respectively.

[0212] In one embodiment of the present invention, Figure 8 This is an example of fine blood vessel segmentation according to the present invention. From top to bottom, the comparison results are shown on the DRIVE, STARE, and CHASE_DB1 datasets.

[0213] In one embodiment of the present invention, Figure 9 This is an example of the continuity of blood vessel segmentation in this invention.

[0214] The comparison results are shown from top to bottom on the DRIVE, STARE, and CHASE_DB1 datasets.

[0215] In one embodiment of the present invention, Figure 10 The visualization results from the DRIVE, STARE, and CHASE_DB1 datasets of this invention are compared, corresponding to the three datasets from top to bottom. (a) Fundus image, (b) Image patch, (c) Ground truth (GT), (d) Baseline model result, (e) Baseline model + preprocessing, (f) Baseline model + preprocessing + MCGL, (g) Baseline model + preprocessing + MCGL + MAGF.

[0216] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A retinal blood vessel segmentation method based on multi-level attention feature fusion, characterized in that, Comprise the following contents: Step S1: preprocessing the data set containing the target retinal blood vessel image, and then establishing an automatic classification of thick and thin blood vessel block method and classifying the preprocessed data into thick blood vessel blocks and thin blood vessel blocks; Step S2: establish a multi-level attention feature fusion module, including adding a multi-level attention feature fusion module between the encoder and the decoder for combining spatial detail information in the shallow feature and semantic context information in the deep feature; Wherein the encoder extracts multi-scale features from the input target retinal blood vessel image, the multi-level attention feature fusion module fuses and enhances the deep feature and the shallow feature, and then the decoder extracts the feature map and generates the final fusion result of the deep and shallow features; Step S3: establish a multi-constraint guided loss function to strengthen the connectivity of the blood vessel skeleton and the boundary definition; Step S4: establish an evaluation mechanism and verify.

2. The retinal blood vessel segmentation method based on multi-level attention feature fusion according to claim 1, characterized in that, Step S1 includes the following contents: Step S11: preprocessing the data set containing the retinal blood vessel image includes: in the RGB three color channels, selecting the retinal blood vessel image in the green channel as the target retinal blood vessel image; then using contrast limited adaptive histogram equalization and gamma correction on the target retinal blood vessel image to enhance the image contrast.

3. The retinal blood vessel segmentation method based on multi-level attention feature fusion according to claim 2, characterized in that, Step S1 also includes the following contents: Step S12: establish an automatic classification of thick and thin blood vessel block method, including the following contents: Step S121: extract the blood vessel skeleton from the target retinal blood vessel image using the skeleton line algorithm, denoted as S; wherein the skeleton can effectively retain the center line structure of the blood vessel, while removing redundant edge information; Step S122: on the basis of skeleton extraction, calculate the blood vessel width R at the skeleton point using distance transformation; wherein distance transformation assigns a value to each pixel, which represents the Euclidean distance to the nearest background pixel; Step S123: use Otsu threshold method to automatically classify the blood vessel width to obtain the following contents: T = Otsu(R) Wherein, T is the best width threshold for classifying blood vessel thickness, and Sr represents the blood vessel width set corresponding to all skeleton points; Next, the total width of all skeleton points is counted, and divided into the total width of thin blood vessels R and the total width of thick blood vessels R according to a threshold value thin and the total width of thick blood vessels R thick , as follows: Finally, the blocks are classified according to the following rule: if R thin >R thick then the block of the target retinal blood vessel image is considered to be a block of thin blood vessels, and vice versa.

4. The retinal blood vessel segmentation method based on multi-level attention feature fusion according to claim 1, characterized in that, Step S2 includes the following contents: Step S21: based on the U-Net structure, add a multi-level attention feature fusion module between the encoder and the decoder; Wherein the encoder extracts multi-scale features from the input target retinal blood vessel image, the multi-level attention feature fusion module fuses and enhances the multi-scale features, and then the decoder generates the final segmentation result using the enhanced features; Step S22: the multi-level attention feature fusion module includes a deep feature fusion module and a shallow feature fusion module; Step S221: the deep feature fusion module retains the semantic information of the deep feature, and the establishment of the deep feature fusion module includes the following contents: The feature map d4 and the feature map d3 are respectively subjected to upsampling and 3*3 convolution operation, so that the feature map d4 and the feature map d3 are consistent in spatial dimension parameters including spatial size and channel number; then the SE attention mechanism is introduced for element-wise addition fusion to generate a channel-dimension feature weight map M1; then the weight map M1 is respectively applied to the feature map d4 and the feature map d3, and the two weighted feature maps are spliced in the channel dimension, and then 3*3 convolution operation is further performed to extract the fused deep semantic information, to obtain a deep feature fusion module; The SE attention mechanism includes global average pooling, a fully connected layer, a ReLU activation function, a fully connected layer and a Sigmoid function; The expression of the deep feature fusion module includes the following contents: M1 = SE (UpConv (d4) + UpConv (d3)) wherein, Conv 3×3 is a 3x3 convolution operation, SE() refers to a channel attention operation, M1 is a feature weight map generated by SE attention; UpConv() represents an up-sampling plus convolution operation, d3 and d4 are feature maps of deep features; Out DFI represents the output result of the deep feature fusion module; Step S222: The shallow feature fusion module is used to strengthen the expression of local details and edge information in the shallow feature and improve the segmentation sensitivity of the blood vessel region, and the establishment of the shallow feature fusion module includes the following contents: The feature map d2 is subjected to upsampling and 1*1 convolution operation, so that the feature map d2 and the feature map d1 are consistent in spatial dimension parameters including spatial size and channel number; further, on the basis of the lightweight attention guiding mechanism, an intermediate feature is obtained through 1*1 convolution, batch normalization and ReLU activation function, and then a Sigmoid function is used to generate an attention weight map M2, wherein the M2 weight map can adaptively emphasize the response of edge and texture detail regions; then M2 is respectively applied to the feature map d2 and the feature map d1 to realize selective strengthening of the shallow feature; finally, the weighted features are spliced in the channel dimension, and 3*3 convolution is further performed to extract the fused information, to obtain a shallow feature fusion module; wherein the expression of the shallow feature fusion module includes the following contents: M2 = σ(BN(Conv 1×1 (ReLU(UpConv(d2) + d1)))) Wherein, M2 is a feature weight map, and Concat(·) represents a splicing operation, and ReLU is an activation function; is an element-wise multiplication operation, d1 and d2 are feature maps of shallow features; Out SFI is the output result of the shallow feature fusion module.

5. The retinal blood vessel segmentation method based on multi-level attention feature fusion according to claim 4, characterized in that, Step S2 further includes the following contents: Step S23: A fusion graph neural network module constructed based on a graph attention network is used to fuse the deep feature fusion module and the shallow feature fusion module, and finally a deep and shallow feature fusion module is output; wherein the process of fusing the deep feature fusion module and the shallow feature fusion module includes the following contents: A graph attention network is introduced; the deep and shallow features are spliced in the channel dimension, and then convolution and SE attention mechanism are used for feature compression and enhancement; then the features are mapped to a graph structure, and an edge index is constructed based on 4-adjacent relationship, and adaptive feature aggregation is performed on the graph structure through GAT; Wherein, GAT can automatically learn attention weights according to the similarity between different nodes, so as to mine more complex spatial dependency and structural constraints between regions in the image; The expression of fusing the deep feature fusion module and the shallow feature fusion module includes the following contents: Out DS = Concat(Out DFI ,Out SFI ) Out = GAT(Conv 3×3 (Out DS )) Out DS denotes the output result of the deep feature fusion module and the output result of the shallow feature fusion module is spliced, GAT() denotes a graph convolution operation; Out is the final result obtained by completing a 3x3 convolution operation on the basis of Out DS and then performing a graph convolution operation.

6. The method of claim 1, wherein the method is based on multi-level attention feature fusion. Step S3 includes the following contents: Step S31: a multi-constraint guided loss function is established to optimize the results output by the multi-level attention feature fusion module, including the following contents: A multi-constraint guided loss function based on skeleton extraction combined with Canny edge operation is adopted, wherein the multi-constraint guided loss function includes a combination loss of BCE and Dice; The edge details and centerline features of the target retinal blood vessel image are captured and strengthened, wherein the expression of the multi-constraint guided loss function based on skeleton extraction combined with Canny edge operator includes the following contents: wherein L Dice is a Dice loss function, L BCE is a BCE loss function, L BD is a loss function combining the BCE loss function and the Dice loss function; Y refers to a real label, is a segmentation map of the model, and λ is 0.5; Step S32: the blood vessel skeletonization processing is performed on the blood vessel annotation image of the target retinal blood vessel image whose edge details and centerline features have been captured and strengthened, and the segmentation of the blood vessel annotation image is realized; wherein the blood vessel skeletonization adopts a thinning algorithm, including the following contents: where L1 is a skeleton-guided loss; P SD represents an inflated skeletonized model prediction map; Y SD represents an inflated skeletonized vessel annotation map; Step S33: the Canny edge operator is operated on the blood vessel annotation image after the blood vessel skeletonization processing, and the contour of the blood vessel boundary is obtained; wherein the Canny edge operator includes the following contents: wherein L2 is an edge guidance loss; P E (i) represents the i-th pixel value in the edge map obtained after edge extraction of the prediction result, Y E (i) is then represents the edge pixel value of the true label at the same position; N is the number of pixels of the image block; Step S33: the multi-constraint guided loss function includes the following contents: L MCG = aL1+ bL2+ L BD wherein, L MCG is the multiple constraint guidance loss function; the value of a is 0.5 and the value of β is 0.

5.

7. The method of claim 1, wherein the method is based on multi-level attention feature fusion. Step S4 includes the following contents: Step S41: an evaluation mechanism is established, including the use of accuracy Acc, sensitivity Sen, specificity Spe, F1 score F1, area under ROC curve AUC and Matthew correlation coefficient MCC to evaluate the retinal blood vessel segmentation, wherein the related parameters include the following contents: Wherein, TP is the number of pixel points correctly predicted as blood vessels, TN is the number of pixel points correctly predicted as background, FP represents the number of pixel points incorrectly predicted as blood vessels, and FN represents the number of pixel points incorrectly predicted as background, the true positive rate TPR is the same as the sensitivity, and the false positive rate FPR is the proportion of incorrectly predicted blood vessels to the actual background.

8. A retinal blood vessel segmentation system based on multi-level attention feature fusion, configured to perform a retinal blood vessel segmentation method based on multi-level attention feature fusion according to any one of claims 1 to 7; characterized in that, The retinal blood vessel segmentation system based on multi-level attention feature fusion includes the following contents: A data preprocessing module for preprocessing a data set containing a retinal blood vessel image; A multi-level attention feature fusion module for fusing deep features and shallow features to improve the accuracy of blood vessel segmentation; A multi-constraint guided loss function is established to strengthen the connectivity of blood vessel skeleton and the clarity of boundary; An evaluation mechanism module for evaluating the effect of retinal blood vessel segmentation. 9.A retinal blood vessel segmentation system based on multi-level attention feature fusion, comprising an electronic device, wherein the electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and characterized in that, The processor executes the computer program to realize the retinal blood vessel segmentation method based on multi-level attention feature fusion as claimed in any one of claims 1 to 7.

10. A retinal blood vessel segmentation system based on multi-level attention feature fusion, comprising a computer readable storage medium, the computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to realize the retinal blood vessel segmentation method based on multi-level attention feature fusion as claimed in any one of claims 1 to 7.

Images