Retinal vessel segmentation method based on feature enhancement and multi-scale perceptual feature fusion

Abstract

The invention discloses a retinal vessel segmentation method based on feature enhancement and multi-scale perceptual feature fusion, and aims to improve the precision and robustness of retinal vessel segmentation. The method comprises the following steps: firstly, constructing a feature enhancement network to improve image quality and perform feature enhancement, and then performing processing after feature enhancement; then, on the basis of the U-Net network, a designed dense upsampling convolution module, a parallel scale convolution module, a multi-scale perception feature fusion module and multi-scale cavity convolution jump connection are combined to construct a multi-scale perception feature fusion retinal vessel segmentation network, and a processed data set is used for training and testing; and finally, carrying out segmentation performance comparison with a classic segmentation network. The method is excellent in performance in a retinal blood vessel segmentation task, shows the advantages in the aspects of blood vessel feature extraction and fusion, has better performance in detail feature segmentation, and can realize high-precision segmentation of the retinal blood vessel in a complex scene.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a retinal blood vessel segmentation method based on feature enhancement and multi-scale perception feature fusion. Background Art

[0002] Accurate segmentation of retinal blood vessels is of great significance in the diagnosis and screening of ophthalmic diseases, especially in the early detection of diseases such as diabetic retinopathy, glaucoma, and hypertensive retinopathy, where detailed analysis of vascular structure is crucial. However, retinal images have complex backgrounds, low contrast, and noise interference, making accurate segmentation of blood vessels a challenging task.

[0003] Traditional retinal blood vessel segmentation methods mainly include classical image processing techniques such as edge detection, threshold segmentation, and region growing. Although these methods can achieve certain results in some cases, they have limited segmentation capabilities for complex backgrounds, small blood vessels, and lesion areas, and are difficult to meet the high-precision requirements of clinical diagnosis.

[0004] In recent years, the rapid development of deep learning technology has provided new ideas for retinal vessel segmentation. The segmentation method based on convolutional neural network (CNN) can achieve relatively excellent segmentation effect in complex scenes by training large-scale labeled data. However, the existing deep learning methods still have certain limitations in processing multi-scale vascular structures, extracting small vascular features, and dealing with complex backgrounds. Therefore, how to combine feature enhancement and multi-scale perception feature fusion technology to further improve the accuracy and robustness of retinal vessel segmentation has become a hot issue in current research. Summary of the invention

[0005] Aiming at the shortcomings of existing retinal blood vessel segmentation methods in processing multi-scale blood vessel structures, extracting small blood vessel features and coping with complex backgrounds, the present invention proposes a retinal blood vessel segmentation method based on feature enhancement and multi-scale perception feature fusion, comprising the following steps:

[0006] S1: Construct a feature enhancement network and perform feature enhancement:

[0007] Use two mirror-symmetric generative adversarial networks (GANs) to form a ring network to build a feature enhancement network. The network architecture consists of two generators (G1 and G2) and two discriminators (D1 and D2);

[0008] Generator G1 generates fake high-quality images from corresponding low-quality images, while generator G2 generates fake low-quality images from high-quality images. At the same time, when generating fake images, discriminator D1 is used to distinguish between real low-quality images and fake low-quality images, and discriminator D2 is used to distinguish between real high-quality images and fake high-quality images. In this way, the model converts low-quality images into high-quality images and inputs the images into the feature enhancement network for feature enhancement;

[0009] S2: Perform secondary feature enhancement on the image after feature enhancement in step S1;

[0010] S3: Divide the training set and test set and perform data augmentation on the divided training set to increase the number of images and enhance the diversity of data;

[0011] S4: Construction of retinal vessel segmentation network based on multi-scale perception feature fusion:

[0012] A retinal vascular segmentation network with multi-scale perceptual feature fusion is constructed by combining the designed dense upsampling convolution module, parallel scale convolution module, multi-scale perceptual feature fusion module and multi-scale hole convolution jump connection. The parallel scale convolution module is used to enhance the expression ability and reusability of vascular features, the multi-scale perceptual feature fusion module is used to perform multi-scale convolution operations on the bottom-level feature information, the multi-scale hole convolution jump connection is used to connect the encoder and decoder of the network, and the dense upsampling convolution module is used in the decoder to achieve efficient upsampling.

[0013] S5: loss function selection;

[0014] S6: Training and testing the constructed segmentation network:

[0015] During the experiment, the network parameters were continuously adjusted to obtain the optimal training network model, and then the model was tested and the performance was evaluated to obtain the final retinal vessel segmentation network model;

[0016] S7: Comparison of segmentation performance with classic segmentation networks:

[0017] The final retinal vessel segmentation network model obtained in step S6 was experimentally compared with six classic segmentation networks, namely U-Net, R2U-Net, SA-Unet, FR-Unet, Genetic U-Net, and MFI-Net, and the performance of all network models was evaluated to further verify the performance superiority of the method of the present invention.

[0018] In step S1, the generator construction process of the feature enhancement network is as follows:

[0019] Generator design: Generators G1 and G2 are fully convolutional networks, consisting of 3 downsampling, 9 residual blocks and 3 upsampling blocks; each downsampling block consists of a convolutional layer, instance normalization and ReLU activation. The number of channels in the convolutional layer is 64, 128 and 256, respectively, and the convolution kernel size and stride are (7×7), (3×3), (3×3) and (1×1), (2×2), (2×2), respectively; the number of channels in the convolutional layer in the residual block is 256, the convolution kernel size is (3×3), and the stride is (1×1); each upsampling block consists of a transposed convolutional layer, instance normalization and ReLU activation. The number of channels in the transposed convolutional layer is 128, 64 and 3, respectively, and the convolution kernel size and stride are (3×3), (3×3), (7×7) and (2×2), (2×2), (1×1), respectively. The output of the last upsampling block is then activated by tanh to obtain the final result.

[0020] In step S1, the generator loss function selection process of the feature enhancement network is as follows:

[0021] Generator loss function selection: According to cycle consistency, the image generated from one domain to another should be convertible back to the original domain without significant changes. The expression of the selected cycle consistency loss function is:

[0022] L cyc (X,G2(Y),Y,G1(X))=E X [||X-G2(Y)|| 1 ]+E Y [||Y-G1(X)|| 1 ]

[0023] Converting X to Y (i.e., Y=G1(X)) and then back to X (i.e., X=G2(Y)) is called forward cycle consistency, and converting Y to X (i.e., X=G2(Y)) and then back to Y (i.e., Y=G1(X)) is called reverse cycle consistency. This loss ensures that the image after two generator conversions can be as close to the original image as possible, thereby ensuring the stability and consistency of the inter-domain conversion.

[0024] In step S1, the discriminator construction process of the feature enhancement network is as follows:

[0025] Discriminator design: Discriminators D1 and D2 consist of 5 downsampling blocks and an output convolutional layer; each downsampling block consists of a convolutional layer, instance normalization and LeakyReLU activation. The number of channels in the convolutional layers is 64, 128, 256, 512, and 512, respectively. The convolution kernel size is (4×4), and the stride size is (2×2), (2×2), (2×2), (2×2), and (1×1), respectively; the number of channels in the output convolutional layer is 1, the convolution kernel size is (4×4), and the stride size is (1×1).

[0026] In step S1, the discriminator loss function selection process of the feature enhancement network is as follows:

[0027] Discriminator loss function selection: D1 and D2 use mean square error to determine whether these images are real or fake. The expression of the loss function is:

[0028] L XGY (D1,X,G1(X))=E X [log(D1(X))]+E G1(X) [log(1-D1(G1(X)))]

[0029] L YGX (D2,Y,G2(Y))=E Y [log(D2(Y))]+E G2(Y) ]log(1-D2(G2(Y)))]

[0030] Generator G1 converts a given low-quality image X into a high-quality image Y (i.e., Y=G1(X)), while generator G2 converts a high-quality image Y into a low-quality image X (i.e., X=G2(Y)); afterwards, discriminator D1 receives two images X and G2(Y), and discriminator D2 receives Y and G1(X) as input; through this loss, the discriminator can better distinguish between real images and fake images, thereby pushing the generator to generate more realistic images.

[0031] The processing of step S2 is as follows:

[0032] S2.1: Use multi-scale grayscale transformation and Group Whitening operation to convert the RGB image into a single-channel grayscale image and filter out some noise;

[0033] S2.2: LIME (Low-light Image Enhancement) is used for image enhancement to highlight the features of retinal blood vessels;

[0034] S2.3: Use structured contrast enhancement to preserve the details of the retinal image while improving the overall contrast.

[0035] S2.4: Gradient-Based Feature Scaling uses image gradient information to guide feature scaling, ensuring that important edges and detail areas are not blurred during the scaling process.

[0036] In step S3, the MedAug library and FastaiAugmentation method are used to augment the training set; the Local Feature Perturbation method is used to perform slight perturbations or noise injections in local areas of the retinal image to improve the robustness of the model to local features; and the Color Augmentation Libraries are used to perform diversity adjustment to enrich the features of the training samples.

[0037] In step S4, the specific construction process of each module is as follows:

[0038] Design a parallel scale convolution module, cascade the 3×3 convolution layer, DenseBlock, DropBlock (block size is set to 7), BN layer and ShiLU activation layer to form the main branch, and calculate it as follows:

[0039] F(x i )=ShiLU(BN(DP(DB(Conv 3×3 (x i )))))

[0040] At the same time, the 1×1 convolutional layer, the BN layer, and the ShiLU activation layer are cascaded to form a bypass branch, which is calculated as follows:

[0041] f(x i )=ShiLU(BN(Conv 1×1 (x i )))

[0042] Finally, the outputs of the main branch and the bypass branch are feature fused:

[0043] x o =F(x i )+f(x i )

[0044] Where: x i represents the input, f(x i ) indicates bypass calculation, F(x i ) indicates the main branch calculation, x o Indicates output;

[0045] A multi-scale perception feature fusion module is designed, which connects four interval convolutions in cascade, with interval rate r = [1, 2, 3, 4] and convolution kernel size of 3 × 3. Each branch also participates in the final feature fusion process through residual connection. The output after feature fusion passes through a 1 × 1 convolution layer, BN layer and ShiLU activation, and finally performs channel splicing with the original input feature.

[0046] A multi-scale dilated convolution jump connection is designed. The input retinal vascular features are first processed by a 3×3 convolution layer, a BN layer, and a ShiLU activation, and then processed by a multi-scale dilated convolution with a cascade dilation rate r=[1,3,5] and a convolution kernel of 1×1 as follows:

[0047]

[0048] Where: P(x) represents the preliminary processing, D(x) represents the output fusion of the three paths;

[0049] Finally, the output is fused with the original features input through the residual structure through a 1×1 convolution layer, a BN layer, and a SwLU activation.

[0050] The expressions of ShiLU activation in each module are as follows:

[0051] ShiLU(x)=α×ReLU(x)+β

[0052] Where: x is the value of the input feature, α and β are trainable parameters, and the optimal values ​​are determined through training in the experiment;

[0053] The expression of SwLU activation in multi-scale dilated convolution jump connection is as follows:

[0054]

[0055] Where: x is the value of the input feature, and the SwLU activation performs well in handling smooth transitions while preserving nonlinear features.

[0056] In step S4, the overall network uses a 4-layer encoding and decoding structure. In the encoding stage, the first layer of the encoder is downsampled through two layers of parallel scale convolution modules and a maximum pooling layer to generate 32 feature maps, which are output to the second layer and downsampled through two layers of parallel scale convolution modules and a maximum pooling layer to generate 64 feature maps, and then output to the third layer and downsampled through two layers of parallel scale convolution modules and a maximum pooling layer to generate 128 feature maps, and finally output to the multi-scale perception feature fusion module for processing. The multi-scale perception feature fusion module performs multi-scale convolution operations on the feature information of the bottom layer through a cascaded dilated convolution layer composed of dilated convolutions with different dilated rates, making full use of the feature information of the bottom layer of the network. In the decoding stage, each layer of the decoder corresponds to each layer of the encoder, and the convolution block is the same as the convolution block in the encoder. The only difference is that the input channel and the output channel are reversed. In the third layer, the output of the multi-scale perception feature fusion module contains 256 feature maps, which are connected to the 128 feature maps of the multi-scale dilated convolution jump connection, and the features are upsampled through two layers of parallel scale convolution modules and dense upsampling convolution modules to generate 128 feature maps. In the second layer, the 128 feature maps are connected to the 64 feature maps of the multi-scale dilated convolution jump connection, and the features are upsampled through two layers of parallel scale convolution modules and dense upsampling convolution modules to generate 64 feature maps. In the final decoding layer, the 64 feature maps are connected to the 32 feature maps of the multi-scale dilated convolution jump connection, and 32 feature maps are generated through two layers of parallel scale convolution modules, and then the final retinal vascular segmentation image is generated through a 1×1 convolution layer and a Sigmoid activation function.

[0057] In step S5, since the non-vascular pixels account for a large proportion, resulting in an imbalance between vascular and non-vascular pixels, the Hybrid Focal Loss (HFLoss) is selected as the loss function of the model, which improves the Dice and cross-entropy-based component losses and combines adjustable parameters to handle output imbalance and focal parameters to handle input imbalance;

[0058] The hybrid focal loss replaces the Dice loss with the Focal Tversky loss and the cross entropy loss with the Focal loss. The expression of the hybrid focal loss is as follows:

[0059] HFLoss=λ×FLoss+(1-λ)×FTLoss

[0060] FLoss(pt)=α×(1-pt) γ ×BCELoss(p,y)

[0061]

[0062] Wherein: HFLoss is the hybrid focal loss, and the relative weights of the two component losses are determined by adjusting the value of λ; FLoss(pt) is the Focal loss, which is parameterized by α and γ, respectively controlling the class weight and weight reduction degree of easy-to-classify pixels. When γ = 0, it is simplified to the binary cross entropy loss; FTLoss is the Focal Tversky loss, where γ < 1 increases the attention to more difficult samples. When γ = 1, the Focal Tversky loss is simplified to the Tversky loss. The hybrid focal loss performs outstandingly in dealing with data imbalance and enhancing the model's ability to capture microvessels, and effectively improves the segmentation accuracy of the model while ensuring training stability.

[0063] In step S6, stochastic gradient descent (SGD) is used as the optimizer to optimize the model training. The initial learning rate is set to 0.001, the momentum factor is set to 0.9, the weight decay coefficient is set to 0.0001, and the cosine learning rate decay strategy is used. The batch size of the DRIVE dataset is set to 4, the batch size of the CHASE_DB1 dataset is set to 2, and the number of training iterations is set to 150.

[0064] The advancements of the present invention:

[0065] Firstly, the present invention enhances the features of the input image by constructing a feature enhancement network, converting low-quality images into high-quality images so that the model can be better trained and optimized;

[0066] Secondly, the present invention processes the feature-enhanced retinal image through multi-scale grayscale transformation, Group Whitening, LIME (Low-light Image Enhancement), structured contrast enhancement and feature scaling processing method based on image gradient, so that the processed image has better contrast, more obvious distinction between blood vessels and background, clearer blood vessel features, and is more suitable for feature extraction;

[0067] Then, based on the U-Net network, the present invention combines dense upsampling convolution modules, parallel scale convolution modules, multi-scale perception feature fusion modules, and multi-scale hole convolution jump connections to construct a retinal vessel segmentation network based on multi-scale perception feature fusion. The parallel scale convolution module and the dense upsampling convolution module realize feature enhancement of small blood vessels and complex structures in retinal images, improving the model's ability to capture subtle blood vessel features. The multi-scale perception feature fusion module fuses feature information of different scales, improving the model's ability to process blood vessels of different sizes. The multi-scale hole convolution jump connection adapts to the thickness changes of blood vessels through multi-scale convolution kernels, further improving the robustness and accuracy of the model;

[0068] Finally, the present invention selects Hybrid Focal Loss (HFLoss) as the loss function of the model to overcome the problem of imbalance in the number of blood vessels and background pixels and improve the performance of the model in the retinal blood vessel segmentation task. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is a flow chart of the method of the present invention;

[0070] Figure 2 is a structural diagram of the feature enhancement network of the present invention;

[0071] Figure 3 is a structural diagram of the generators (G1 and G2) in the feature enhancement network of the present invention;

[0072] Figure 4 It is a structural diagram of the discriminators (D1 and D2) in the feature enhancement network of the present invention;

[0073] Figure 5 is a structural diagram of the retinal blood vessel segmentation network of the present invention;

[0074] Figure 6 It is a structural diagram of the parallel scale convolution module in the retinal blood vessel segmentation network of the present invention;

[0075] Figure 7 It is a structural diagram of a multi-scale perception feature fusion module in a retinal blood vessel segmentation network of the present invention;

[0076] Figure 8 It is a structural diagram of multi-scale hole convolution jump connection in the retinal blood vessel segmentation network of the present invention;

[0077] Fig. 9 It is the segmentation result diagram of the present invention on the DRIVE dataset;

[0078] Fig.10 This is the segmentation result diagram of the present invention on the CHASE_DB1 dataset. DETAILED DESCRIPTION

[0079] The present invention proposes a retinal blood vessel segmentation method based on feature enhancement and multi-scale perception feature fusion. The method realizes high-precision retinal blood vessel segmentation by preprocessing the retinal image, performing feature enhancement, multi-scale perception feature fusion and segmentation processing. Figure 1 The present invention is further described below in conjunction with the accompanying drawings and embodiments:

[0080] Step 1, dataset selection. The embodiment uses two public retinal image datasets, DRIVE and CHASE_DB1, to train and test the proposed method;

[0081] Step 2: Build a feature enhancement network and perform feature enhancement;

[0082] Step 3: Perform secondary feature enhancement on the image after feature enhancement in step 2;

[0083] Step 4: Divide the training set and test set and perform data enhancement on the divided training set;

[0084] Step 5: Construct a retinal vessel segmentation network based on multi-scale perception feature fusion;

[0085] Step 6: Loss function selection;

[0086] Step 7: Train and test the constructed segmentation network;

[0087] Step 8: Compare the segmentation performance with the classic segmentation network. The embodiment uses five commonly used retinal image segmentation evaluation indicators, namely, accuracy, F1-scores, specificity, recall, and AUC (Area Under Curve) to evaluate the segmentation performance of the proposed method, which are defined as follows:

[0088]

[0089]

[0090]

[0091]

[0092] Where: TP refers to the number of pixels correctly identified and segmented as blood vessels, that is, true positives; TN refers to the number of pixels correctly identified as background and not incorrectly segmented as blood vessels, that is, true negatives; FP refers to the situation where background pixels are mistakenly identified as blood vessels, that is, false positives; FN refers to the number of pixels that should belong to the blood vessel area but are ignored by the model or misjudged as background, that is, false negatives; AUC is the area under the ROC curve and the coordinate axis. The closer the AUC value is to 1, the better the performance of the model in distinguishing blood vessels from the background, and the more accurately it can determine the ownership of each pixel, thereby generating a more accurate and reliable blood vessel segmentation map.

[0093] In step 2, the generator construction process of the feature enhancement network is as follows:

[0094] Generator design: Generators G1 and G2 are fully convolutional networks, consisting of 3 downsampling, 9 residual blocks and 3 upsampling blocks; each downsampling block consists of a convolutional layer, instance normalization and ReLU activation. The number of channels in the convolutional layer is 64, 128 and 256, respectively, and the convolution kernel size and stride are (7×7), (3×3), (3×3) and (1×1), (2×2), (2×2), respectively; the number of channels in the convolutional layer in the residual block is 256, the convolution kernel size is (3×3), and the stride is (1×1); each upsampling block consists of a transposed convolutional layer, instance normalization and ReLU activation. The number of channels in the transposed convolutional layer is 128, 64 and 3, respectively, and the convolution kernel size and stride are (3×3), (3×3), (7×7) and (2×2), (2×2), (1×1), respectively. The output of the last upsampling block is then activated by tanh to obtain the final result.

[0095] In step 2, the generator loss function selection process of the feature enhancement network is as follows:

[0096] Generator loss function selection: According to cycle consistency, the image generated from one domain to another should be convertible back to the original domain without significant changes. The expression of the selected cycle consistency loss function is:

[0097] L cyc (X,G2(Y),Y,G1(X))=E X [||X-G2(Y)|| 1 ]+E Y [||Y-G1(X)|| 1 ]

[0098] Converting X to Y (i.e., Y=G1(X)) and then back to X (i.e., X=G2(Y)) is called forward cycle consistency, and converting Y to X (i.e., X=G2(Y)) and then back to Y (i.e., Y=G1(X)) is called reverse cycle consistency. This loss ensures that the image after two generator conversions can be as close to the original image as possible, thereby ensuring the stability and consistency of the inter-domain conversion.

[0099] In step 2, the discriminator construction process of the feature enhancement network is as follows:

[0100] Discriminator design: Discriminators D1 and D2 consist of 5 downsampling blocks and an output convolutional layer; each downsampling block consists of a convolutional layer, instance normalization and LeakyReLU activation. The number of channels in the convolutional layers is 64, 128, 256, 512, and 512, respectively. The convolution kernel size is (4×4), and the stride size is (2×2), (2×2), (2×2), (2×2), and (1×1), respectively; the number of channels in the output convolutional layer is 1, the convolution kernel size is (4×4), and the stride size is (1×1).

[0101] In step 2, the discriminator loss function selection process of the feature enhancement network is as follows:

[0102] Discriminator loss function selection: D1 and D2 use mean square error to determine whether these images are real or fake. The expression of the loss function is:

[0103] L XGY (D1,X,G1(X))=E X [log(D1(X))]+E G1(X) [log(1-D1(G1(X)))]

[0104] L YGX (D2,Y,G2(Y))=E Y [log(D2(Y))]+E G2(Y) [log(1-D2(G2(Y)))]

[0105] Generator G1 converts a given low-quality image X into a high-quality image Y (i.e., Y=G1(X)), while generator G2 converts a high-quality image Y into a low-quality image X (i.e., X=G2(Y)); afterwards, discriminator D1 receives two images X and G2(Y), and discriminator D2 receives Y and G1(X) as input; through this loss, the discriminator can better distinguish between real images and fake images, thereby pushing the generator to generate more realistic images.

[0106] The processing of step 3 is as follows:

[0107] S3.1: Use multi-scale grayscale transformation and Group Whitening operation to convert the RGB image into a single-channel grayscale image and filter out some noise;

[0108] S3.2: LIME (Low-light Image Enhancement) is used for image enhancement to highlight the features of retinal blood vessels;

[0109] S3.3: Use structured contrast enhancement to preserve the details of retinal images while improving the overall contrast.

[0110] S3.4: Gradient-Based Feature Scaling uses image gradient information to guide feature scaling, ensuring that important edges and detail areas are not blurred during the scaling process.

[0111] In step 4, the MedAug library and FastaiAugmentation method are used to augment the training set. The Local Feature Perturbation method is used to perform slight perturbations or noise injections in local areas of the retinal image to improve the model's robustness to local features. The Color Augmentation Libraries are then used to perform diversity adjustments to enrich the features of the training samples.

[0112] In step 5, the specific construction process of each module is as follows:

[0113] Design a parallel scale convolution module, cascade the 3×3 convolution layer, DenseBlock, DropBlock (block size is set to 7), BN layer and ShiLU activation layer to form the main branch, and calculate it as follows:

[0114] F(x i )=ShiLU(BN(DP(DB(Conv 3×3 (x i )))))

[0115] At the same time, the 1×1 convolutional layer, the BN layer, and the ShiLU activation layer are cascaded to form a bypass branch, which is calculated as follows:

[0116] f(x i )=ShiLU(BN(Conv 1×1 (x i )))

[0117] Finally, the outputs of the main branch and the bypass branch are feature fused:

[0118] x o =F(x i )+f(x i )

[0119] Where: x i represents the input, f(x i ) indicates bypass calculation, F(x i ) indicates the main branch calculation, x o Indicates output;

[0120] A multi-scale perception feature fusion module is designed, which connects four interval convolutions in cascade, with interval rate r = [1, 2, 3, 4] and convolution kernel size of 3 × 3. Each branch also participates in the final feature fusion process through residual connection. The output after feature fusion passes through a 1 × 1 convolution layer, BN layer and ShiLU activation, and finally performs channel splicing with the original input feature.

[0121] A multi-scale dilated convolution jump connection is designed. The input retinal vascular features are first processed by a 3×3 convolution layer, a BN layer, and a ShiLU activation, and then processed by a multi-scale dilated convolution with a cascade dilation rate r=[1,3,5] and a convolution kernel of 1×1 as follows:

[0122]

[0123] Where: P(x) represents the preliminary processing, D(x) represents the output fusion of the three paths;

[0124] Finally, the output is fused with the original features input through the residual structure through a 1×1 convolution layer, a BN layer, and a SwLU activation.

[0125] The expressions of ShiLU activation in each module are as follows:

[0126] ShiLU(x)=α×ReLU(x)+β

[0127] Where: x is the value of the input feature, α and β are trainable parameters, and the optimal values ​​are determined through training in the experiment;

[0128] The expression of SwLU activation in multi-scale dilated convolution jump connection is as follows:

[0129]

[0130] Where: x is the value of the input feature, and the SwLU activation performs well in handling smooth transitions while preserving nonlinear features.

[0131] In step 5, the overall network uses a 4-layer encoding and decoding structure. In the encoding stage, the first layer of the encoder is downsampled through two layers of parallel scale convolution modules and a maximum pooling layer to generate 32 feature maps, which are output to the second layer and downsampled through two layers of parallel scale convolution modules and a maximum pooling layer to generate 64 feature maps, and then output to the third layer and downsampled through two layers of parallel scale convolution modules and a maximum pooling layer to generate 128 feature maps, and finally output to the multi-scale perception feature fusion module for processing. The multi-scale perception feature fusion module performs multi-scale convolution operations on the bottom-level feature information through a cascaded dilated convolution layer composed of dilated convolutions with different dilation rates, making full use of the feature information at the bottom of the network. In the decoding stage, each layer of the decoder corresponds to each layer of the encoder, and the convolution block is the same as the convolution block in the encoder. The only difference is that the input channel and the output channel are reversed. In the third layer, the output of the multi-scale perception feature fusion module contains 256 feature maps, which are connected to the 128 feature maps of the multi-scale dilated convolution jump connection, and the features are upsampled through two layers of parallel scale convolution modules and dense upsampling convolution modules to generate 128 feature maps. In the second layer, the 128 feature maps are connected to the 64 feature maps of the multi-scale dilated convolution jump connection, and the features are upsampled through two layers of parallel scale convolution modules and dense upsampling convolution modules to generate 64 feature maps. In the final decoding layer, the 64 feature maps are connected to the 32 feature maps of the multi-scale dilated convolution jump connection, and 32 feature maps are generated through two layers of parallel scale convolution modules, and then the final retinal vascular segmentation image is generated through a 1×1 convolution layer and a Sigmoid activation function.

[0132] In step 6, since the proportion of non-vascular pixels is large, resulting in an imbalance between vascular and non-vascular pixels, the Hybrid Focal Loss (HFLoss) is selected as the loss function of the model, which improves the Dice and cross-entropy-based component losses and combines adjustable parameters to handle output imbalance and focal parameters to handle input imbalance;

[0133] The hybrid focal loss replaces the Dice loss with the Focal Tversky loss and the cross entropy loss with the Focal loss. The expression of the hybrid focal loss is as follows:

[0134] HFLoss=λ×FLoss+(1-λ)×FTLoss

[0135] FLoss(pt)=α×(1-pt) γ ×BCELoss(p,y)

[0136]

[0137] Wherein: HFLoss is the hybrid focal loss, and the relative weights of the two component losses are determined by adjusting the value of λ; FLoss(pt) is the Focal loss, which is parameterized by α and γ, respectively controlling the class weight and weight reduction degree of easy-to-classify pixels. When γ = 0, it is simplified to the binary cross entropy loss; FTLoss is the Focal Tversky loss, where γ < 1 increases the attention to more difficult samples. When γ = 1, the Focal Tversky loss is simplified to the Tversky loss. The hybrid focal loss performs outstandingly in dealing with data imbalance and enhancing the model's ability to capture microvessels, and effectively improves the segmentation accuracy of the model while ensuring training stability.

[0138] In step 7, stochastic gradient descent (SGD) is used as the optimizer to optimize the model training. The initial learning rate is set to 0.001, the momentum factor is set to 0.9, the weight decay coefficient is set to 0.0001, and the cosine learning rate decay strategy is used. The batch size of the DRIVE dataset is set to 4, the batch size of the CHASE_DB1 dataset is set to 2, and the number of training iterations is set to 150.

[0139] In step 8, the final retinal vessel segmentation network model obtained in step 7 is experimentally compared with six classic segmentation networks: U-Net, R2U-Net, SA-Unet, FR-Unet, Genetic U-Net, and MFI-Net on the DRIVE and CHASE_DB1 datasets, and the performance of all network models is evaluated. The evaluation results are shown in Table 1:

[0140] Table 1 Comparison of segmentation performance of different segmentation methods

[0141]

[0142] As shown in Table 1, on the DRIVE dataset, the proposed method achieved better results in all evaluation indicators, especially in the Recall and Specificity indicators, which were improved by 0.0734 and 0.0087 respectively compared with U-Net, which not only shows that the proposed method has a higher ability in accurately identifying retinal vascular structures, but also proves its accuracy in distinguishing blood vessels from background areas; on the CHASE_DB1 dataset, although the proposed method is slightly lower than the SA-Unet algorithm in the F1-Score indicator, it also performs well in the other evaluation indicators, especially in the Accuracy and AUC indicators, which are improved by 0.017 and 0.0111 respectively compared with U-Net, which further proves that the proposed method can still stably and accurately identify retinal blood vessels in complex image environments, effectively resist the interference of background noise, thereby improving the overall accuracy and reliability of blood vessel segmentation;

[0143] Fig. 9 and Fig.10 The segmentation results of the present invention on the DRIVE dataset and the CHASE_DB1 dataset are shown in the first column. The first column is the original retinal image, the second column is the gold standard, that is, the accurate blood vessel segmentation result diagram, and the third column is the segmentation result diagram of the method of the present invention. Compared with the gold standard, the method of the present invention can accurately segment the vascular structure in the retinal image, including the main blood vessels and microvessels, retain most of the detail information, and clearly display the distribution and morphology of the vascular network. In addition, the method of the present invention can still achieve excellent performance in the face of retinal images with more complex vascular structures and pathological conditions on the two datasets, accurately distinguishing blood vessels from the background, reflecting the good robustness and generalization ability of the present invention.

[0144] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A retinal vessel segmentation method based on feature enhancement and multi-scale perception feature fusion, It is characterized in that The following steps are involved: S1： Construct a feature enhancement network and perform feature enhancement: Two mirror-symmetric generative adversarial networks (GANs) are used to form a ring network to construct a feature enhancement network. The network architecture consists of two generators (G1 and G2) and two discriminators (D1 and D2); Generator G1 generates fake high-quality images from corresponding low-quality images, while Generator G2 generates fake low-quality images from high-quality images. At the same time, when generating fake images, discriminator D1 is used to distinguish between real low-quality images and fake low-quality images, and discriminator D2 is used to distinguish between real high-quality images and fake high-quality images. In this way, the model converts low-quality images into high-quality images and inputs the images into the feature enhancement network for feature enhancement; S2: Perform secondary feature enhancement on the feature-enhanced image; S3: training set, test set division and data augmentation for the divided training set; S4: Construction of retinal vessel segmentation network based on multi-scale perception feature fusion: A retinal vascular segmentation network with multi-scale perceptual feature fusion is constructed by combining the designed dense upsampling convolution module, parallel scale convolution module, multi-scale perceptual feature fusion module and multi-scale hole convolution jump connection. The parallel scale convolution module is used to enhance the expression ability and reusability of vascular features, the multi-scale perceptual feature fusion module is used to perform multi-scale convolution operations on the underlying feature information, the multi-scale hole convolution jump connection is used to connect the encoder and decoder of the network, and the dense upsampling convolution module is used in the decoder to achieve efficient upsampling. S5: loss function selection; S6: Training and testing the constructed segmentation network; S7: Comparison of segmentation performance with classic segmentation networks: The final retinal vessel segmentation network model obtained in step S6 was experimentally compared with six classic segmentation networks: U-Net, R2U-Net, SA-Unet, FR-Unet, Genetic U-Net, and MFI-Net to further verify the performance superiority of this method.

2. The retinal blood vessel segmentation method based on feature enhancement and multi-scale perception feature fusion according to claim 1, It is characterized in that The generator construction process of the feature enhancement network in step S1 is as follows: Generator design: Generators G1 and G2 are fully convolutional networks, consisting of 3 downsampling, 9 residual blocks and 3 upsampling blocks; each downsampling block consists of a convolutional layer, instance normalization and ReLU activation. The number of channels in the convolutional layer is 64, 128 and 256, respectively, and the convolution kernel size and stride are (7×7), (3×3), (3×3) and (1×1), (2×2), (2×2), respectively; the number of channels in the convolutional layer in the residual block is 256, the convolution kernel size is (3×3), and the stride is (1×1); each upsampling block consists of a transposed convolutional layer, instance normalization and ReLU activation. The number of channels in the transposed convolutional layer is 128, 64 and 3, respectively, and the convolution kernel size and stride are (3×3), (3×3), (7×7) and (2×2), (2×2), (1×1), respectively. The output of the last upsampling block is then activated by tanh to obtain the final result.

3. The retinal blood vessel segmentation method based on feature enhancement and multi-scale perception feature fusion according to claim 1, It is characterized in that The generator loss function selection process of the feature enhancement network in step S1 is as follows: Generator loss function selection: According to cycle consistency, the image generated from one domain to another should be convertible back to the original domain without significant changes. The expression of the selected cycle consistency loss function is: L cyc (X,G2(Y),Y,G1(X))=E X [||X-G2(Y)|| 1 ]+E Y [||Y-G1(X)|| 1 ] Converting X to Y (i.e., Y=G1(X)) and then back to X (i.e., X=G2(Y)) is called forward cycle consistency, and converting Y to X (i.e., X=G2(Y)) and then back to Y (i.e., Y=G1(X)) is called reverse cycle consistency. This loss ensures that the image after two generator conversions can be as close to the original image as possible, thereby ensuring the stability and consistency of the inter-domain conversion.

4. The retinal blood vessel segmentation method based on feature enhancement and multi-scale perception feature fusion according to claim 1, It is characterized in that The discriminator design and discriminator loss function selection process of the feature enhancement network in step S1 are as follows: Discriminator design: The discriminators D1 and D2 consist of 5 downsampling blocks and an output convolutional layer; each downsampling block consists of a convolutional layer, instance normalization, and LeakyReLU activation. The number of channels in the convolutional layer is 64, 128, 256, 512, and 512, respectively. The convolution kernel size is (4×4), and the stride is (2×2), (2×2), (2×2), (2×2), and (1×1), respectively; the number of channels in the output convolutional layer is 1, the convolution kernel size is (4×4), and the stride is (1×1); Discriminator loss function selection: D1 and D2 use mean square error to determine whether these images are real or fake. The expression of the loss function is: L XGY (D1,X,G1(X))=E X [log(D1(X))]+E G1(X) [log(1-D1(G1(X)))] L YGX (D2,Y,G2(Y))=E Y [log(D2(Y))]+E G2(Y) [log(1-D2(G2(Y)))] Generator G1 converts a given low-quality image X into a high-quality image Y (i.e., Y=G1(X)), while generator G2 converts a high-quality image Y into a low-quality image X (i.e., X=G2(Y)); thereafter, discriminator D1 receives two images X and G2(Y), and discriminator D2 receives Y and G1(X) as input.

5. The retinal blood vessel segmentation method based on feature enhancement and multi-scale perception feature fusion according to claim 1, It is characterized in that The processing process of step S2 is as follows: S2.1: Use multi-scale grayscale transformation and Group Whitening operation to convert the RGB image into a single-channel grayscale image and filter out some noise; S2.2: Use LIME (Low-light Image Enhancement) to enhance the image and highlight the features of retinal blood vessels; S2.3: Use structured contrast enhancement to preserve the details of the retinal image while improving the overall contrast. S2.4: Gradient-Based Feature Scaling uses image gradient information to guide feature scaling, ensuring that important edges and detail areas are not blurred during the scaling process.

6. The retinal blood vessel segmentation method based on feature enhancement and multi-scale perception feature fusion according to claim 1, It is characterized in that In step S3, the MedAug library and Fastai Augmentation method are used to augment the training set; the Local Feature Perturbation method is used to perform slight perturbations or noise injections in local areas of the retinal image to improve the robustness of the model to local features; and the ColorAugmentationLibraries method is used to perform diversity adjustment to enrich the features of the training samples.

7. The retinal blood vessel segmentation method based on feature enhancement and multi-scale perception feature fusion according to claim 1, It is characterized in that The specific construction process of each module in step S4 is as follows: Design a parallel scale convolution module, cascade the 3×3 convolution layer, DenseBlock, DropBlock (block size is set to 7), BN layer and ShiLU activation layer to form the main branch, and calculate it as follows: F(x i )=ShiLU(BN(DP(DB(Conv 3×3 (x i ))))) At the same time, the 1×1 convolutional layer, the BN layer, and the ShiLU activation layer are cascaded to form a bypass branch, which is calculated as follows: f(x i )=ShiLU(BN(Conv 1×1 (x i ))) Finally, the outputs of the main branch and the bypass branch are feature fused: x o =F(x i )+f(x i ) Where: x i represents the input, f(x i ) indicates bypass calculation, F(x i ) indicates the main branch calculation, x o Indicates output; A multi-scale perception feature fusion module is designed, which connects four interval convolutions in cascade, with interval rate r = [1, 2, 3, 4] and convolution kernel size of 3 × 3. Each branch also participates in the final feature fusion process through residual connection. The output after feature fusion passes through a 1 × 1 convolution layer, BN layer and ShiLU activation, and finally performs channel splicing with the original input feature. A multi-scale dilated convolution jump connection is designed. The input retinal vascular features are first processed by a 3×3 convolution layer, a BN layer, and a ShiLU activation, and then processed by a multi-scale dilated convolution with a cascade dilation rate r=[1,3,5] and a convolution kernel of 1×1 as follows: Where: P(x) represents the preliminary processing, D(x) represents the output fusion of the three paths; Finally, the output is fused with the original features input through the residual structure through a 1×1 convolution layer, a BN layer, and a SwLU activation.

8. The retinal blood vessel segmentation method based on feature enhancement and multi-scale perception feature fusion according to claim 1, It is characterized in that In step S5, since non-vascular pixels account for a large proportion, resulting in an imbalance between vascular and non-vascular pixels, a hybrid focal loss (HFLoss) is selected as the loss function of the model, which improves Dice and cross-entropy-based component losses, combining adjustable parameters to handle output imbalance and focal parameters to handle input imbalance; The hybrid focal loss replaces the Dice loss with the Focal Tversky loss and the cross entropy loss with the Focal loss. The expression of the hybrid focal loss is as follows: HFLoss=λ×FLoss+(1-λ)×FTLoss FLoss(pt)=α×(1-pt) γ ×BCELoss(p,y) Wherein: HFLoss is the hybrid focal loss, and the relative weights of the two component losses are determined by adjusting the value of λ; FLoss(pt) is the Focal loss, which is parameterized by α and γ, respectively controlling the class weight and weight reduction degree of easy-to-classify pixels. When γ = 0, it is simplified to the binary cross entropy loss; FTLoss is the Focal Tversky loss, where γ < 1 increases the attention to more difficult samples. When γ = 1, the Focal Tversky loss is simplified to the Tversky loss. The hybrid focal loss performs outstandingly in dealing with data imbalance and enhancing the model's ability to capture microvessels, and effectively improves the segmentation accuracy of the model while ensuring training stability.

9. The retinal blood vessel segmentation method based on feature enhancement and multi-scale perception feature fusion according to claim 7, It is characterized in that The expressions of ShiLU activation in each module are as follows: ShiLU(x)=α×ReLU(x)+β Where: x is the value of the input feature, α and β are trainable parameters, and the optimal values ​​are determined through training in the experiment; The expression of SwLU activation in multi-scale dilated convolution jump connection is as follows: Where: x is the value of the input feature, and the SwLU activation performs well in handling smooth transitions while preserving nonlinear features.

Images