A lightweight retinal blood vessel segmentation method based on MSLV-Net

Abstract

The application discloses a light coding enhancement and decoding sharpening network MSLV-Net segmentation method and system for retinal blood vessel segmentation, an electronic device and a storage medium. First, the green channel of a color fundus image is extracted, CLAHE enhancement, padding, normalization and sliding window cutting preprocessing are performed; a three-layer light U-shaped network based on a deep separable convolution is constructed, a light blood vessel enhancement block LVB is embedded at the coding end, micro-vessel topology is reserved through cross-scale hollow convolution and coordinate attention, a multi-scale learnable adaptive sharpening module MS-LAS is embedded at the decoding end, and multi-scale smooth difference and adaptive weight fusion are used to repair edge blur and fracture. The network parameter is only 0.435M, and the sensitivity, F1 and other indexes on the DRIVE, CHASE_DB1 and STARE are better than those of LadderNet, SA-UNet and FR-UNet, and the network can be deployed on an embedded portable fundus screening device.

Description

Technical Field

[0001] This invention relates to the fields of medical fundus image segmentation and lightweight deep learning model deployment technology, and in particular to a lightweight retinal vessel segmentation method, system, electronic device and computer-readable storage medium. Background Technology

[0002] Changes in the morphology and diameter continuity of retinal vessels are core evidence for the screening, diagnosis, follow-up assessment of systemic chronic diseases and ophthalmic diseases such as diabetic retinopathy, glaucoma, and hypertensive retinopathy. Manually annotating vessels pixel by pixel in clinical practice is time-consuming and laborious, and the segmentation results are unstable due to factors such as physician experience, image noise, and lesion occlusion. Therefore, deep learning-based automatic segmentation algorithms have become the mainstream solution for clinical image interpretation.

[0003] Encoder-decoder networks, exemplified by UNet, have been widely applied to retinal vessel segmentation tasks by fusing deep and shallow features through skip connections. However, existing solutions suffer from three core drawbacks:

[0004] 1. The segmentation of tiny microvessels is very difficult: the scale of blood vessels in fundus images is extremely wide, with coarse main blood vessels and tiny capillaries at the terminal. Moreover, the contrast between microvessels and the background is low, and they are easily affected by imaging noise and interference from lesions. During the network downsampling process, it is very easy to lose the spatial details of fine blood vessels, resulting in missed detection of microvessels and blood vessel rupture.

[0005] 2. Poor edge reconstruction in the decoding stage: Standard UNet relies solely on skip connections to supplement shallow details. After multiple downsampling and upsampling, the high-frequency information of the blood vessel boundary continues to decay, resulting in blurred edges and discontinuous small branches in the segmentation output.

[0006] 3. It is difficult to balance lightweight model with segmentation accuracy: High-precision segmentation networks stack a large number of standard convolutions, resulting in huge parameter and computational costs, long inference time, and inability to adapt to resource-constrained scenarios such as mobile fundus cameras and embedded portable diagnostic devices; existing lightweight networks reduce feature extraction capabilities after simplifying the backbone, further exacerbating the defects in fine blood vessel segmentation.

[0007] Existing improvement schemes optimize retinal vessel segmentation from a single dimension, focusing on multi-scale modeling, attention mechanisms, and lightweight convolution. They lack a collaborative design for enhancing fine vessels at the encoding end and sharpening edges at the decoding end, failing to simultaneously improve microvessel recall and vessel boundary clarity under extremely low parameter constraints. Therefore, a retinal vessel segmentation scheme that balances lightweight deployment, fine vessel feature enhancement, and adaptive edge repair during decoding is urgently needed. Summary of the Invention

[0008] A lightweight retinal vessel segmentation method, comprising:

[0009] S1: Acquire color fundus images of the target eye, perform standardized preprocessing, and output single-channel enhanced fundus images;

[0010] S2: Construct a lightweight retinal vessel segmentation network MSLV-Net; MSLV-Net is a three-layer U-shaped encoder-decoder architecture, which includes an input feature extraction layer, a three-level encoder, a bottleneck layer, a three-level decoder, and an output layer; skip connections are set between the encoder and the decoder; all backbone units of the network are constructed using depthwise separable convolutions; each of the three-level encoders embeds a lightweight vessel enhancement block (LVB) to enhance the feature response of slender microvessels during the downsampling stage; each of the three-level decoders embeds a multi-scale learnable adaptive sharpening module (MS-LAS) to adaptively repair vessel edges and supplement high-frequency details during the upsampling stage.

[0011] S3: Input the preprocessed single-channel fundus enhancement image into the MSLV-Net network, and output the retinal binary vessel segmentation map through end-to-end inference.

[0012] The S1 standardization preprocessing step specifically includes: extracting the green channel component of the color fundus image, using CLAHE to limit contrast adaptive histogram equalization to improve the local contrast of microvessels; zero-filling the image edges and uniformly scaling them to a standard square size; using min-max normalization to map pixel values ​​to the [0,1] interval; during the training phase, a 48×48 sliding window with a stride of 6 is used to expand the samples by overlapping blocks, while during the testing phase, no blocks are used and the entire image is directly input into the network.

[0013] The lightweight vascular enhancement block (LVB) is a two-stage, multi-scale feature enhancement structure, and the processing flow includes:

[0014] (1) First stage feature enhancement: The input features are divided into the main direct transmission branch and the auxiliary downsampling branch; after downsampling, the auxiliary branch uses 3×3 hole depth convolution with dilation rate r=2 to extract multi-scale blood vessel texture, and completes channel mapping through 1×1 point convolution. It also incorporates coordinate attention CA to model the horizontal and vertical blood vessel structures, and enhances the directional response of slender blood vessels; the auxiliary branch features are upsampled to restore the original image scale, and the initial fusion is completed by adding the main branch features element by element.

[0015] (2) Second stage feature compensation: After the initial fusion features are reorganized by 1×1 convolution, auxiliary branches are separated again. The branches are first upsampled to expand the resolution, and the hole depth convolution, point convolution, and coordinate attention module are reused to refine the details of blood vessel bifurcation and terminal. Then, they are downsampled to restore the scale of the main branch and fused with the main branch features element by element for the second time to output LVB enhanced features.

[0016] The MS-LAS ​​processing flow of the multi-scale learnable adaptive sharpening module includes:

[0017] (1) Multi-scale smoothing extraction of low-frequency features: The input features are smoothed by 3×3, 5×5 and 7×7 depth convolutions respectively to obtain three sets of low-frequency smooth features with different receptive fields;

[0018] (2) Extraction of high-frequency detail residuals by difference: The original input features are subtracted from the corresponding scale smoothed features pixel by pixel to extract the edge detail residuals corresponding to coarse blood vessels, medium blood vessels, and fine blood vessels respectively. ;

[0019] (3) Adaptive weight fusion details: Set three sets of learnable scale fusion parameters Adaptive weights for each scale of detail are obtained through Softmax normalization. The weighted summation and fusion of residuals at all scales yields the fused detail features. ;

[0020] (4) Residual Enhancement Output: Introduce a globally learnable enhancement coefficient α to superimpose the fused high-frequency details onto the original input features in the form of residuals, and output the decoded features with improved edge sharpness.

[0021] The MSLV-Net network training and construction steps include: selecting three publicly available retinal fundus datasets—DRIVE, CHASE_DB1, and STARE—and performing image enhancement and sliding window block expansion according to the preprocessing workflow to construct training and testing sets; setting the network input channel to 1 and output channel to 1, and using the Adam optimizer and binary cross-entropy (BCE) loss function for supervised training; and setting the initial learning rate. Weight decay coefficient The model is trained using a cosine annealing learning rate scheduling strategy; the training batch size is 16, and the total number of iterations is 50; the training is completed on an RTX3090 GPU with 24GB of video memory, and the model weight file is saved to obtain a lightweight retinal vessel segmentation model.

[0022] The MSLV-Net network has only 0.435M parameters, 18.47G FLOPs, and a single fundus image inference time of 24.86ms on a GPU. The backbone uses depthwise separable convolution to split spatial convolution and channel convolution, which greatly reduces computational overhead and is suitable for deployment on resource-constrained clinical devices such as embedded and mobile devices.

[0023] The present invention also provides a lightweight retinal vessel segmentation system for implementing the lightweight retinal vessel segmentation method described in any one of the preceding claims, comprising:

[0024] The image acquisition and preprocessing unit is used to acquire clinical color fundus images, perform standardized preprocessing procedures, and output normalized single-channel enhanced fundus images.

[0025] A lightweight network training unit is used to build the MSLV-Net segmentation network, load the publicly available fundus datasets DRIVE, CHASE_DB1, and STARE to complete model training, and store the trained weight files;

[0026] The vessel segmentation inference unit loads the trained MSLV-Net weights, receives the preprocessed fundus image input, and outputs refined binary segmentation results of retinal vessels for clinical image interpretation and auxiliary diagnosis.

[0027] The present invention also provides an electronic device, including a memory and a processor; the memory stores a computer-executable program; the processor executes the computer-executable program stored in the memory to implement the lightweight retinal vessel segmentation method described in any of the preceding claims.

[0028] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the lightweight retinal vessel segmentation method described in any of the preceding claims.

[0029] This invention, based on the UNet architecture, employs depthwise separable convolutions to construct a lightweight U-shaped backbone. It collaborates with the LVB module at the encoder and the MS-LAS ​​module at the decoder for optimized performance, specifically addressing issues such as missed detection of microvessels, blurred vessel boundaries, and high model deployment costs in fundus images. The LVB module, through two-stage cross-scale dilated convolutions and coordinate attention, preserves microvessel topological details during downsampling, significantly improving the detection rate of small vessels with only a small increase in parameters. The MS-LAS ​​module utilizes multi-scale smooth difference extraction to extract high-frequency edge information of vessels and adaptively fuses details at different scales using learnable weights, correcting vessel breaks and edge blurring caused by upsampling. The combined use of LVB and MS-LAS ​​creates complementary gains, resulting in significant improvements in key metrics such as model sensitivity, F1 score, and accuracy on three major public fundus datasets: DRIVE, CHASE_DB1, and STARE. With only 0.435M parameters, the computational load and inference speed are superior to most similar segmentation networks. It can be deployed in resource-constrained devices such as portable fundus screening equipment and mobile terminals. With the green channel extraction and CLAHE contrast enhancement preprocessing, it can effectively adapt to complex clinical fundus images with lesion occlusion, low contrast, and imaging noise, and has good clinical practical value. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the overall network architecture of MSLV-Net of the present invention;

[0031] Figure 2 This is a schematic diagram of the lightweight vascular enhancement block LVB module structure of the present invention;

[0032] Figure 3 This is a schematic diagram of the MS-LAS ​​processing logic of the multi-scale learnable adaptive sharpening module of the present invention;

[0033] Figure 4 This is a visualization comparison of the segmentation of MSLV-Net in this invention with existing methods on three datasets; Detailed Implementation

[0034] Fundus image standardization preprocessing steps,

[0035] Step 1: Channel extraction: Only the green channel is retained in the acquired color fundus image, as the green channel has the highest contrast between blood vessels and background; Step 2: Contrast enhancement: CLAHE-limited contrast adaptive histogram equalization is performed on the green channel image to brighten low-contrast microvessels.

[0036] Step 3: Size standardization: Zero padding is applied to the image edges. Images from the DRIVE dataset are uniformly padded to 592×592, images from the CHASE_DB1 dataset are uniformly padded to 1008×1008, and images from the STARE dataset are uniformly padded to 728×728.

[0037] Step 4: Pixel normalization: Map the image pixel values ​​to the [0,1] range using a minimum-maximum scaling method;

[0038] Step 5: Sample augmentation: During the training phase, a 48×48 sliding window with a step size of 6 is used to complete overlapping slicing, generating a large number of local image blocks and corresponding blood vessel labels; during the testing phase, no slicing operation is performed, and the entire image is directly input into the network.

[0039] Steps to build a lightweight segmentation network using MSLV-Net

[0040] Step 1: Basic backbone construction: A three-layer U-shaped encoder-decoder basic architecture is adopted. All feature extraction units use depthwise separable convolutions instead of standard convolutions. The network is set up with an input layer, a three-level encoder, a bottleneck layer, a three-level decoder, and an output layer in sequence. Skip connections are set between the encoder and the corresponding decoder to pass shallow detailed features.

[0041] Step 2: Encoder module deployment: A lightweight LVB vascular enhancement block is embedded in each of the three encoders. After the image is initially convolved to extract shallow texture, it is fed into the encoder for downsampling at each level. Before each downsampling, the LVB module completes cross-scale fine vascular enhancement.

[0042] LVB execution steps:

[0043] Step 1: The input features are divided into the main branch direct transmission branch and the auxiliary downsampling branch. After downsampling, the auxiliary branch uses 3×3 hole depth convolution with dilation rate of 2, 1×1 point convolution and coordinate attention to enhance the blood vessel features. The enhanced features are then upsampled back to the original scale and added to the main branch features element by element.

[0044] Step 2: The fused features are adjusted by 1×1 convolution to adjust the channel dimensions and then the auxiliary branches are separated again. The branches are first upsampled to amplify the resolution and refine the terminal and bifurcation features of blood vessels, and then downsampled to restore the scale of the main branch. They are then fused with the main branch features element by element for the second time, and the output enhanced features are sent to the next layer of downsampling.

[0045] Step 3: Bottleneck layer construction: Located between the encoder and decoder, it only uses depthwise separable convolution to aggregate global high-level semantic features, without adding any additional functional modules;

[0046] Step 4: Decoder module deployment: The three-level decoder upsamples step by step to restore the image resolution. After each level decoder completes the same-scale encoded feature skip fusion, it is connected to the MS-LAS ​​multi-scale adaptive sharpening module to repair the blood vessel edges.

[0047] MS-LAS ​​execution steps:

[0048] Step 1: Apply 3×3, 5×5, and 7×7 depth convolutions to smooth the input features, respectively, to obtain three sets of low-frequency smoothed feature maps;

[0049] Step 2: Subtract the original input features from the three sets of smooth features pixel by pixel to extract multi-scale edge detail residuals;

[0050] Step 3: Set learnable scale fusion parameters, generate adaptive weights through Softmax normalization, and then weight and fuse all scale detail residuals;

[0051] Step 4: Introduce globally learnable enhancement coefficients to superimpose the fused high-frequency details onto the original input features as residuals, and output the edge-repaired features;

[0052] Step 5: Output layer construction: The output features of the last-stage decoder are processed by 1×1 convolution and Sigmoid activation to output a single-channel blood vessel segmentation probability map. A threshold of 0.5 is set to complete binarization, and the final blood vessel segmentation result is obtained.

[0053] Model training configuration steps,

[0054] Step 1: Hardware environment configuration: Use an RTX 3090 24GB GPU, Ubuntu system, and build a training environment based on Python 3.8 and the PyTorch deep learning framework;

[0055] Step 2: Dataset Splitting: For the DRIVE dataset, 20 images are selected as the training set and 20 images as the test set; for the CHASE_DB1 dataset, 20 images are selected as the training set and 8 images as the test set; for the STARE dataset, 15 images are selected as the training set and 5 images as the test set.

[0056] Step 3: Training parameter configuration: Set the network input channel to 1, the output channel to 1, select the Adam optimizer, and the initial learning rate... Weight decay The training is performed using a cosine annealing learning rate decay strategy; the batch size is set to 16, and the total number of training rounds is 50; the loss function is binary cross-entropy (BCE).

[0057] Step 4: Model saving: After completing all iterations of training, save the network weight file to obtain a lightweight retinal vessel segmentation model.

[0058] Image reasoning and segmentation steps,

[0059] Step 1: Perform preprocessing operations on the fundus image to be tested according to the standardized preprocessing procedure;

[0060] Step 2: Load the weight file of the trained MSLV-Net model;

[0061] Step 3: Input the preprocessed single-channel fundus image into the network and output a blood vessel segmentation probability map;

[0062] Step 4: Binarize the probability map with a threshold of 0.5 to obtain a complete retinal vessel segmentation image, which can be used for physician image reading, quantitative measurement of blood vessels, and screening of fundus lesions.

[0063] Multi-model comparison experiment steps,

[0064] Step 1: Select three mainstream retinal segmentation networks, LadderNet, SA-UNet, and FR-UNet, as comparison models;

[0065] Step 2: Complete the training on the DRIVE, CHASE_DB1, and STARE datasets, and calculate the accuracy metrics of each model (Se, Sp, F1, Acc, AUC). Also, calculate the number of model parameters, FLOPs, and GPU inference time per image.

[0066] Step 3: Compare the overall performance of each model from three dimensions: segmentation accuracy, model lightweightness, and inference speed.

[0067] Operating steps of the retinal vessel segmentation system

[0068] Step 1: Image acquisition and preprocessing unit operation: Connect to the fundus camera to acquire color fundus images, automatically perform a full set of preprocessing operations such as green channel extraction, CLAHE enhancement, normalization, and size filling, and output a standardized single-channel image;

[0069] Step 2: Network training unit operation: Load the DRIVE / CHASE_DB1 / STARE dataset, automatically complete sliding window block expansion, MSLV-Net network training, and weight saving, and support importing custom clinical datasets for fine-tuning;

[0070] Step 3: Blood vessel segmentation inference unit operation: Load the trained model weights, input the preprocessed fundus image in real time, output a visual image of blood vessel segmentation, automatically calculate Se, F1, and blood vessel pixel ratio quantification indicators, display blood vessel branches and terminal details, and assist doctors in judging fundus lesions.

[0071] The electronic device executes the following steps: the internal memory of the electronic device stores the computer program corresponding to the segmentation method of the present invention, and the device processor executes the following steps in sequence when running the program: fundus image preprocessing, MSLV-Net network loading, image forward inference, and output of blood vessel segmentation results; the device types include portable fundus screening instruments, desktop diagnostic computers, and mobile tablets.

[0072] The steps for using a computer-readable storage medium include storing an executable computer program within the storage medium, connecting the medium to an electronic device equipped with a processor, and having the processor read and run the program within the storage medium to perform all the aforementioned retinal vessel segmentation processing steps; the storage medium may include a USB flash drive, a solid-state drive, or a cloud storage server.

[0073] The above steps are only used to explain the core technical solution of the present invention and are not intended to limit the scope of protection of the present invention. Based on the ideas of the present invention, those skilled in the art can make conventional adjustments to the module convolution size, dilation rate, and training hyperparameters, and such equivalent modifications all fall within the scope of protection of the present invention.

Claims

1. A lightweight retinal vessel segmentation method, MSLV-Net, characterized in that, include: A color fundus image to be tested is acquired and standardized preprocessing is performed to obtain a single-channel enhanced fundus image. A lightweight encoding-enhancing-decoding-sharpening retinal vessel segmentation network MSLV-Net is constructed. The segmentation network is a three-layer U-shaped encoder-decoder architecture, including an input feature extraction layer, a three-level encoder, a bottleneck layer, a three-level decoder, and an output layer. Skip connections are set between the encoder and the decoder. The encoder embeds a lightweight vessel enhancement block LVB, and each level of the decoder embeds a multi-scale learnable adaptive sharpening module MS-LAS. The backbone of the network is built entirely using depthwise separable convolutions. The preprocessed single-channel enhanced fundus image is input into the MSLV-Net network, and an end-to-end binary retinal vessel segmentation map is output.

2. The lightweight retinal vessel segmentation method according to claim 1, characterized in that, The standardized preprocessing steps specifically include: extracting the green channel component of the fundus color image, using CLAHE to limit contrast adaptive histogram equalization to enhance the local contrast of blood vessels; performing edge zero-filling on the enhanced image, and uniformly scaling the image to a standard square size; finally, performing min-max normalization to map pixel values ​​to the [0,1] interval.

3. The lightweight retinal vessel segmentation method according to claim 1, characterized in that, The lightweight vessel enhancement block (LVB) is a two-stage, cross-scale feature enhancement structure, comprising: input features divided into a main direct transmission branch and an auxiliary downsampling branch. The auxiliary branch extracts vessel features using a 3×3 hole depth convolution with an expansion rate r=2. After 1×1 point convolution channel mapping, it is connected to coordinate attention (CA) to enhance the directional response of slender vessels. Then, it is upsampled back to the original image scale and added element-wise with the main branch to complete the initial feature enhancement. After the initial fused features are channel reorganized by 1×1 convolution, auxiliary branches are split again. The branches are first upsampled to expand the resolution, and then the hole depth convolution, point convolution, and coordinate attention modules are reused to refine the vessel edge features. After downsampling back to the main branch scale, they are fused element-wise a second time to output the LVB enhanced features.

4. The lightweight retinal vessel segmentation method according to claim 1, characterized in that, The processing flow of the multi-scale learnable adaptive sharpening module MS-LAS ​​includes: performing multi-scale smoothing on the input features using 3×3, 5×5, and 7×7 depth convolutions respectively to obtain low-frequency smoothed features at each scale; differentiating the original input features with the corresponding scale smoothed features to extract multi-scale high-frequency blood vessel detail residuals; setting learnable scale fusion parameters, obtaining adaptive weights for details at each scale through Softmax normalization, and weighting and fusing all scale detail residuals; introducing globally learnable enhancement coefficients, superimposing the fused high-frequency details onto the original input features in the form of residuals, and outputting the edge-enhanced decoded features.

5. The lightweight retinal vessel segmentation method according to claim 1, characterized in that, The training and construction steps of the MSLV-Net network include: selecting publicly available retinal fundus datasets from DRIVE, CHASE_DB1, and STARE; processing the images according to the preprocessing procedure of claim 2; using a 48×48 sliding window with a stride of 6 to complete overlapping block expansion of training samples; setting the network input channel to 1 and the output class to 1; using the Adam optimizer and binary cross-entropy (BCE) loss function for supervised training; and setting the initial learning rate... Weight decay The model is equipped with a cosine annealing learning rate scheduling strategy; the training batch size is set to 16, the total number of iterations is 50, and the lightweight retinal vessel segmentation model is obtained after the GPU completes the training.

6. The lightweight retinal vessel segmentation method according to claim 1, characterized in that, The MSLV-Net network has a total parameter count of 0.435M, and the GPU inference time for a single fundus image during the inference phase is less than 25ms. The network backbone uses depthwise separable convolution instead of standard convolution to reduce computational load, with FLOPs of only 18.47G, making it suitable for deployment on resource-constrained clinical devices such as mobile devices and embedded systems.

7. A lightweight retinal vessel segmentation system, characterized in that, A lightweight retinal vessel segmentation method according to any one of claims 1 to 6 is comprising: an image acquisition and preprocessing unit for acquiring color images of the fundus and performing a full set of standardized preprocessing operations as defined in claim 2, outputting a normalized single-channel fundus image; a lightweight network construction and training unit for building an MSLV-Net segmentation network, loading a publicly available fundus dataset to complete model training, and saving the weight file; and a vessel segmentation inference unit for loading the trained MSLV-Net weights, receiving preprocessed image input, and outputting refined retinal vessel segmentation results.

8. An electronic device, characterized in that, It includes a memory and a processor; the memory stores a computer-executable program; the processor executes the computer-executable program stored in the memory to implement the lightweight retinal vessel segmentation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the lightweight retinal vessel segmentation method according to any one of claims 1 to 6.

Images