A lightweight fundus blood vessel image segmentation model based on Mamba

Abstract

The application discloses a kind of light weight fundus blood vessel image segmentation model based on Mamba, comprising the following steps: encoder and decoder include at least four layers SlimPVM Block;Input feature map is gradually down-sampled by maximum pooling operation;CCA Bottleneck module is calculated by calculating cross-channel self-attention in the height and width dimension of feature map;Encoder and decoder are fused by selective jump connection, and the feature is selected by channel attention mechanism by selective jump connection Weighted;Decoder refines the feature after up-sampling by SlimPVM Block, and the model recovers the lost spatial dimension by convolution transpose operation, and combines the encoding features from selective jump connection;The final segmentation mask is generated by a 1x1 convolution layer at last output by decoder, and the segmentation mask can accurately delineate the target region boundary in medical image.Specific to retinal blood vessel segmentation task, reduce model parameters, reduce computing overhead, while keeping or even improving segmentation performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image segmentation, in particular to a lightweight fundus blood vessel image segmentation model based on Mamba. BACKGROUND

[0002] Retinal diseases, including diabetic retinopathy, glaucoma and hypertensive retinopathy, are one of the main causes of visual impairment worldwide. Fundus imaging can capture detailed images of the retina, playing a crucial role in screening and diagnosing these diseases. Medical image segmentation technology plays an important role in retinal image analysis, which can automatically extract key retinal structures such as retinal blood vessels, optic disc and macular area. Accurate segmentation of these anatomical structures is crucial for diagnosing and monitoring retinal diseases.

[0003] With the development of machine learning, machine learning-based retinal blood vessel segmentation methods have gradually emerged. This kind of method usually extracts features from training data, and then uses classifiers such as decision trees, support vector machines (SVM) and random forests for classification. However, the main problem of machine learning methods is that the feature extraction process usually needs to be designed manually, and is sensitive to noise, resulting in weak generalization ability of the model. In recent years, deep learning has made important progress in medical image segmentation, especially methods based on convolutional neural networks (CNN). Although CNN-based models perform well in local feature extraction, their perception range is limited, making it difficult to capture long-distance dependencies and global context information. This is particularly evident in the task of fundus blood vessel segmentation, because the morphological structure of blood vessels is complex, and relying solely on local features is insufficient to accurately describe the entire vascular network. After achieving unprecedented success in natural language tasks, Transformers have been successfully applied to computer vision problems, achieving state-of-the-art results. Although Transformers effectively extract global and long-range dependencies through self-attention mechanisms, they lack in capturing local features and translation invariance. In addition, the computational complexity of self-attention mechanisms significantly increases the number of model parameters, resulting in longer inference and training times.

[0004] The above methods improve the segmentation effect to some extent, but most existing models still have some shortcomings: they often have difficulty processing small blood vessels in complex backgrounds, the large number of model parameters results in high computational overhead, and they perform poorly when dealing with low-contrast images. SUMMARY

[0005] The present application aims to address the above-mentioned problems by providing a lightweight fundus blood vessel image segmentation model based on Mamba, which is specifically designed for retinal blood vessel segmentation tasks, reduces model parameters, reduces computational overhead, and maintains or even improves segmentation performance.

[0006] The technical solutions of the present application are as follows:

[0007] A lightweight fundus blood vessel image segmentation model based on Mamba, comprising the following steps:

[0008] The encoder comprises at least four SlimPVM blocks, and each SlimPVM block comprises a depth separable convolution, a feature extraction module and a parallel visual state space model;

[0009] The input feature map is gradually down-sampled through a max-pooling operation;

[0010] In the bottleneck layer of the network, the CCA bottleneck module calculates the cross-channel self-attention in the height and width dimensions of the feature map;

[0011] The encoder and the decoder perform feature fusion through a selective skip connection, and the selective skip connection selects features through a channel attention mechanism;

[0012] The decoder refines the up-sampled features using a Mamba block, after up-sampling, the model restores the lost spatial dimensions through a convolution transpose operation, and combines the encoded features from the selective skip connection to further improve the reconstruction effect;

[0013] The decoder finally outputs a final segmentation mask through a 1x1 convolution layer, and the segmentation mask can accurately delineate the boundary of the target region in the medical image.

[0014] Further, the SlimPVM block comprises the following steps:

[0015] Local spatial features are extracted through a depth separable convolution layer, and the depth separable convolution layer decomposes a standard convolution into a depth kernel point convolution;

[0016] The features after convolution are processed through batch normalization and a ReLU activation function;

[0017] The feature extraction module extracts global context information through global average pooling, and weights the features through a channel attention mechanism;

[0018] The PVM layer divides the features into multiple sub-channels for parallel processing, each sub-channel is processed through a Mamba state space model, and the Mamba state space model combines a state space matrix and a convolution operation;

[0019] Each sub-channel feature after processing is enhanced through a residual connection, and an adjustment factor is applied to optimize the long-range information acquisition capability, and the sub-channel features are recombined into complete features through a concat operation.

[0020] The Mamba module normalizes the merged features and maps them to the output dimension through a projection operation.

[0021] Furthermore, the feature extraction module is used for multi-scale feature extraction and is deployed at key locations in the encoder and decoder. It fuses features of different scales to regenerate image features and generate an accurate segmentation mask. The feature extraction module includes the following steps:

[0022] The number of channels in the input features is adjusted by using 1×1 convolutional layers, and batch normalization is performed to ensure that the scale of the input features is consistent.

[0023] Global context information is captured through global average pooling operations;

[0024] The attention weights for each channel are learned through two fully connected layers. The first fully connected layer reduces the number of channels in the global feature to 1 / reduction, compressing information and extracting important features. The second fully connected layer restores the number of channels to the original dimension, further optimizing feature selectivity. The channel attention weights are normalized using the Sigmoid function to generate weight coefficients in the range of [0,1].

[0025] The fused features are weighted using a dot product operation, and the final fused features are output.

[0026] Furthermore, the PVM layer is used to enhance the modeling of global context information, including two branches: the first branch includes a linear layer and the SiLU activation function for preliminary feature processing; the second branch includes a linear layer, depthwise convolution, SiLU activation, a state-space model, and LayerNorm operations; specifically, it includes the following steps:

[0027] For the number of channels is Input features The normalization process was performed using LayerNorm.

[0028] The feature is divided into four sub-channels, and the number of channels in each sub-channel is... ;

[0029] Sub-channel features are processed by the VSS module. The VSS module models the spatiotemporal information of the features through deep convolution and state space model, and uses residual connections and adjustment factors to optimize the capture of long-range dependencies.

[0030] The original input features are summed with the processed features through residual connections;

[0031] The four sub-channel features are recombined into a feature with a complete number of channels through a concat operation, and are output mapped through a LayerNorm and a projection layer.

[0032] Further, the calculation formula of the PVM layer is:

[0033] ,

[0034] ,

[0035] ,

[0036] ,

[0037] wherein, represents a LayerNorm operation, is a feature segmentation operation, is a VSS block operation, is an adjustment factor of residual connection, is a feature splicing operation, is a projection operation.

[0038] Further, the selective jump connection specifically includes the following steps:

[0039] The channel number of the input feature is adjusted through a 1x1 convolution to ensure consistency of the channel dimension;

[0040] The distribution of the feature is stabilized through batch normalization;

[0041] The most significant response of each channel is extracted through global maximum pooling;

[0042] After extracting the global feature, a series of 1x1 convolutions are used for dimension reduction processing, and the combination of batch normalization and ReLU activation function enhances the nonlinear expression ability of the feature;

[0043] The channel number is restored to the original dimension through another layer of 1x1 convolution, and finally the Sigmoid activation function is used to generate accurate channel attention weights.

[0044] Further, the CCA Bottleneck module specifically includes the following steps:

[0045] For a feature map , the CCA Bottleneck module generates two feature maps and through two 1x1 convolutions, wherein, , is the channel number after dimension reduction;

[0046] Affinity operation generates attention map ;

[0047] For each position in the feature map , obtain the corresponding feature vector ; extract a set of feature vectors that are in the same row or column as the position from the feature map ; ; ;

[0048] The calculation formula of the affinity operation is:

[0049] ,

[0050] wherein, represents the correlation between the features and , is the correlation matrix; by applying the softmax operation on the channel dimension, the attention map is calculated; is generated by another 1x1 convolution; For each position in the feature map , obtain the corresponding feature vector and set ; collect context information through aggregation operation:

[0051] ,

[0052] wherein, is the feature vector of the output feature map at position , is the scalar value of the corresponding channel and position in the attention map.

[0053] Compared with the existing technology, the beneficial effects of the present application are:

[0054] 1. A lightweight fundus blood vessel image segmentation model based on Mamba, which sets a Feature Extraction Mamba module, an encoder and a decoder, which are one of the core components of the model; SlimPVM Block effectively reduces the amount of calculation by combining depth separable convolution and feature extraction module, while improving the efficiency of feature extraction. In addition, the PVM module introduced in SlimPVM Block further enhances the dynamic feature representation capability, especially in dealing with complex retinal blood vessel structures (such as thin blood vessels and irregularly shaped blood vessels), which can significantly improve the accuracy of segmentation;

[0055] 2. A lightweight fundus blood vessel image segmentation model based on Mamba, in order to better capture the global context information of blood vessels in the retinal image, a CrissCrossAttention module is introduced as the bottleneck layer of the model, and the CCABottleneck constructs a cross global attention mechanism through cross-row and cross-column. This enables the model to aggregate global information from different directions of the image without significantly increasing computational overhead, significantly improving the ability to capture retinal blood vessel details and improving the accuracy of blood vessel boundaries;

[0056] 3. A lightweight fundus blood vessel image segmentation model based on Mamba, a selective skip connection mechanism is set up, which can effectively fuse multi-scale feature information. This design combines channel attention mechanisms to dynamically adjust information flow between different levels, ensuring that low-level features are fully utilized during high-resolution reconstruction. This mechanism enhances the robustness of the model and further improves the accuracy of retinal blood vessel segmentation;

[0057] 4. A lightweight fundus blood vessel image segmentation model based on Mamba, which also reduces the number of model parameters and computational complexity through global lightweight design. During the construction of the model, the performance requirements in actual applications are always considered to ensure that the model can efficiently run in embedded devices or resource-constrained environments while maintaining high-quality segmentation results.

[0058] 5. A lightweight fundus blood vessel image segmentation model based on Mamba, which achieves high-precision segmentation of retinal blood vessels, especially in small blood vessels and complex backgrounds. In a large number of experiments, the model has shown its strong performance and has achieved excellent results on multiple public retinal image datasets. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 Figure 1 is a network architecture diagram of a lightweight fundus blood vessel image segmentation model based on Mamba.

[0060] Figure 2 Figure 2 is a data set in the experiment. DETAILED DESCRIPTION

[0061] It is to be noted that the terms "first" and "second" and similar such relational terms are used solely to distinguish one entity or action from another without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... " does not, without more limitations, exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0062] The features and performance of the present application are described in further detail below in conjunction with embodiments.

[0063] Referring to Figure 1 A lightweight fundus blood vessel image segmentation model based on Mamba includes the following steps:

[0064] The encoder and decoder include at least four SlimPVM blocks, which utilize depth separable convolution to reduce the amount of calculation and the number of parameters while maintaining strong feature extraction capability. The SlimPVM block includes depth separable convolution, a feature extraction module, and a parallel visual state space model to improve the feature expression capability of the model while significantly reducing the amount of calculation and the number of parameters; the SlimPVM block includes the following steps:

[0065] Local spatial features are extracted through a depth separable convolution layer, which decomposes standard convolution into depth kernel point convolution; greatly reducing the number of parameters and computational overhead.

[0066] The features after convolution are passed through batch normalization and a ReLU activation function to enhance non-linear expression capability;

[0067] The feature extraction module extracts global context information through global average pooling and uses a channel attention mechanism to weight the features to highlight key features and suppress irrelevant features;

[0068] The PVM layer divides the features into multiple sub-channels for parallel processing, and each sub-channel is processed through a Mamba state space model, which combines state space matrices and convolution operations to capture long-range dependencies and complex spatial relationships;

[0069] The processed sub-channel features are enhanced through residual connection and optimized for remote information acquisition by applying an adjustment factor. The sub-channel features are recombined into complete features through a concat operation.

[0070] The Mamba module normalizes the merged features and maps them to the output dimension through a projection operation.

[0071] Through this parallel processing method, the Mamba module maintains the same number of overall channels, maximizes performance, and significantly reduces computational overhead. The design of this module effectively balances performance and efficiency, and it has shown excellent performance in complex visual tasks.

[0072] The feature extraction module is used for multi-scale feature extraction and is deployed at key positions in the encoder and decoder to fuse features of different scales, regenerate image features, and generate accurate segmentation masks. The feature extraction module includes the following steps:

[0073] The channel number of the input feature is adjusted through a 1x1 convolutional layer, and batch normalization is performed to ensure the consistency of the input feature scale.

[0074] Global average pooling is used to capture global context information. This global feature extraction helps to strengthen the perception of large-scale regional information, especially in handling complex medical images, which can bring better global information expression.

[0075] Two fully connected layers are used to learn the attention weights of each channel. The first fully connected layer reduces the channel number of the global feature to 1 / reduction of the original. Through dimension reduction, important features are extracted. The ReLU activation function is applied to increase the non-linear expression ability. The second fully connected layer restores the channel number to the original dimension to further optimize feature selectivity. The channel attention weights are normalized by the Sigmoid function to generate weight coefficients in the range of [0, 1]. These weight coefficients are applied to the input feature through channel weighting, achieving the enhancement of important features and the suppression of irrelevant features.

[0076] The fused features are weighted through a dot product operation and output the final fused features.

[0077] The feature extraction module can balance global information and local features while generating more accurate boundary information, providing strong support for subsequent segmentation tasks. The design of this module aims to strengthen the model's focus on key features through channel attention mechanisms and improve the model's perception of the overall structure of the image through the extraction of global context information, thereby achieving higher precision segmentation results.

[0078] The PVM layer is used to enhance the modeling of global context information, including two branches: the first branch includes a linear layer and a SiLU activation function, which is used for preliminary feature processing; the second branch includes a linear layer, a deep convolution, a SiLU activation, a state space model (SSM) and a LayerNorm operation; through multi-level feature extraction and conversion, the model's ability to capture complex visual features is optimized. Specifically, the following steps are included:

[0079] For an input feature with a channel number of , normalization processing is performed by LayerNorm.

[0080] The feature is divided into four sub-channels, each with a channel number of .

[0081] The sub-channel features are processed by the VSS module, which models the spatial and temporal information of the features through deep convolution and state space model, and uses residual connection and adjustment factor to optimize the capture of long-range dependencies.

[0082] The original input feature and the processed feature are added through residual connection to ensure information transmission and enhancement.

[0083] The four sub-channel features are recombined into a feature with complete channel number through concat operation, and then output mapping is performed through LayerNorm and projection layer.

[0084] The calculation formula of the PVM layer is:

[0085] ,

[0086] ,

[0087] ,

[0088] ,

[0089] wherein, represents the LayerNorm operation, is the feature segmentation operation, is the VSS block operation, is the adjustment factor of the residual connection, is the feature concatenation operation, is the projection operation.

[0090] ​By introducing parallel Vision Mamba operations, PVM can effectively improve model performance while keeping the amount of computation and the total number of parameters relatively small. By processing feature sub-channels in parallel, PVM establishes global context information while significantly reducing computational overhead. Finally, the output features of PVM are further fused with the multi-scale features extracted by CNN and Transformer previously, thereby significantly improving the overall performance of the model while keeping the parameter size controllable.

[0091] The input feature map is gradually down-sampled through the max-pooling operation;

[0092] At the bottleneck layer of the network, the CCA Bottleneck module calculates the cross-channel self-attention in the height and width dimensions of the feature map, enabling the model to more effectively capture complex spatial relationships; by collecting context information in the horizontal and vertical directions, the pixel-level feature expression capability is enhanced, thereby improving the semantic segmentation performance of the model. The CCA Bottleneck module specifically includes the following steps:

[0093] For the feature map , the CCA Bottleneck module generates two feature maps and through two 1x1 convolutions, where , is the number of channels after dimension reduction;

[0094] The affinity operation generates an attention map ;

[0095] For each position in the feature map , the corresponding feature vector is obtained; from the feature map , a set of feature vectors in the same row or column as the position is extracted ;

[0096] The calculation formula of the affinity operation is:

[0097] ,

[0098] where represents the correlation between features and , is the correlation matrix; by applying the softmax operation in the channel dimension, the attention map is calculated; through another 1x1 convolution, the feature map is generated; for each position in the feature map obtain the corresponding feature vector and the set Collect context information through aggregation operation:

[0099] ,

[0100] where, is the output feature map the feature vector at position , is the scalar value of the corresponding channel and position in the attention map.

[0101] In this way, the CCA Bottleneck module can selectively aggregate global context information and add it to local features, thereby enhancing the expression ability at the pixel level. This feature representation not only has a wide context view, but also can selectively aggregate context according to the spatial attention map, making it more robust and accurate in semantic segmentation tasks.

[0102] The encoder and decoder fuse features through selective skip connection, which selects features through channel attention mechanism to ensure that the fused features are more discriminative and accurate, thereby reducing the interference of redundant information; and selectively enhances features through channel attention mechanism, thereby improving the accuracy of feature fusion and the overall performance of the model. The selective skip connection (SSC) specifically includes the following steps:

[0103] Adjust the number of channels of the input feature through 1×1 convolution to ensure the consistency of the channel dimension;

[0104] Stabilize the distribution of features through batch normalization to enhance the training stability of the model;

[0105] Global max pooling is used to extract the most significant response of each channel; this operation can focus on the part with the highest information content in the feature map, capture the most representative features, and thus improve the model's perception ability of important areas.

[0106] After extracting global features, a series of 1×1 convolution is used for dimension reduction processing, and the combination of batch normalization and ReLU activation function enhances the non-linear expression ability of the features; not only compresses the information, but also ensures the refinement and strengthening of key features.

[0107] The channel number is restored to the original dimension by another layer of 1x1 convolution, and finally the precise channel attention weight is generated by the Sigmoid activation function. The weight coefficient weights the input features, and then strengthens the important feature regions, and suppresses the irrelevant or secondary information. This selective enhancement mechanism ensures that the information transmission in the skip connection is more accurate, retains the key features, and enhances the model's ability to analyze complex features.

[0108] The decoder refines the up-sampled features using the MambaBlock to ensure that the restored features have high resolution and strong semantic expression ability. After up-sampling, the model restores the lost spatial dimensions through convolution transpose operation, and combines the encoding features from the selective skip connection to further improve the reconstruction effect.

[0109] The decoder finally outputs the final segmentation mask through a 1x1 convolution layer, which can accurately delineate the boundaries of the target region in the medical image.

[0110] Experimental verification:

[0111] Dataset: In order to prove that the method of the present application can meet the requirement of high segmentation accuracy in medical images, training and testing were carried out on the DRIVE, STARE and CHASEDB1 datasets. The DRIVE dataset contains 40 retinal images, divided into training and test sets, each with 20 images. The images were obtained by Canon CR5 non-mydriatic camera, with a resolution of 565x584 pixels. The CHASEDB1 dataset contains 28 fundus images from 14 school-age children, each with a FOV of 30μm and a resolution of 999x960 pixels. The dataset is divided into training and test sets, with the first 20 images used for training and the remaining 8 images used for testing. The STARE dataset consists of 400 retinal images, of which 20 images are used for blood vessel segmentation tasks. These images were captured by TopCon TRV-50 retinal camera based on 35mm film, with an image size of 605x700 pixels. As shown in Figure 2 .

[0112] To address the small sample size of the fundus image datasets and improve the generalization ability of the model, data augmentation was performed on the three datasets to address the issue of insufficient training data. The specific data augmentation is shown in the following table. Techniques such as color jittering, random rotation, scaling, mirror flipping, Gaussian noise addition, sharpening, and affine transformation were used to simulate various changes that may occur during actual medical imaging, such as brightness, contrast, angle, and noise. These changes not only increase the diversity of the data but also improve the robustness of the model when faced with different imaging conditions, ensuring more accurate retinal vessel segmentation.

[0113]

[0114] Evaluation metrics: Five evaluation metrics were used to objectively measure the proposed model and other segmentation networks, including Dice Coefficient (DC), Intersection over Union (IoU), Recall or Sensitivity, Specificity, F1 Score, and Precision. In addition to segmentation performance metrics, we also evaluated several other performance metrics to comprehensively assess the model's performance in terms of resource consumption. Parameters measure the complexity of the model, typically expressed in millions (M). Model size refers to the space occupied by the model in memory, typically measured in megabytes (MB). Computational complexity is measured by the number of floating-point operations (GFLOPs), representing the amount of floating-point computation required for a single inference. Inference time refers to the time required for a single forward propagation, typically measured in milliseconds (ms). Before detailing the five evaluation metrics, some definitions are provided. TP represents true positive, TN represents true negative, FP represents false positive, and FN represents false negative.

[0115] The Dice coefficient is used to measure the overlap between the segmentation results and the true labels, with a larger value indicating better overlap. The definition is as follows:

[0116] ,

[0117] IoU, also known as the Jaccard index, measures the overlap ratio between the predicted segmentation area and the true segmentation area. Its definition is as follows:

[0118] ，

[0119] Precision represents the proportion of cases that are actually positive samples among all predicted positive samples. It is defined as:

[0120] ，

[0121] Recall (Sen) represents the proportion of true positive samples that are detected as positive samples among all true positive samples. It is defined as:

[0122] ，

[0123] Specificity represents the proportion of true negative samples that are predicted as negative samples among all true negative samples. It is defined as:

[0124] .

[0125] Experimental details: The proposed model is based on the PyTorch deep learning framework for training and evaluation. To meet the high computational demand, we use a GEFORCE RTX 3090 graphics processing unit (GPU) with 24 GB of video memory. The initial learning rate is 10−3, the training process is set to 100 epochs, and the batch size is 8. In the model training, the AdamW optimizer is used, and the binary cross-entropy loss function (Binary Cross-Entropy Loss) and weighted IoU loss (Weighted IoU Loss) are combined for optimization.

[0126] Loss function: The loss function is used to evaluate the difference between the predicted value and the true value of the network model. The smaller the value of the loss function, the smaller the difference between the predicted value and the true value of the model, indicating that the performance of the model is better. In this study, we use a structured loss function that combines weighted binary cross-entropy (Weighted Binary Cross-Entropy, WBCE) and weighted intersection over union (Weighted Intersection over Union, WIoU) to more effectively optimize the performance of the model in image segmentation tasks.

[0127] (1) Weighted binary cross-entropy loss (WBCE)

[0128] Binary cross-entropy (Binary Cross Entropy, BCE) loss function is used to measure the difference between the predicted probability and the true label in the binary classification task. Its definition is as follows:

[0129] ，

[0130] where, is the number of pixels, is the true label of the th pixel, is the probability that the model predicts the pixel belongs to the positive class.

[0131] The calculation formula of the weighted binary cross-entropy loss is:

[0132] ，

[0133] (2) Weighted Intersection over Union Loss (WIoU)

[0134] Intersection over Union (IoU) is a commonly used indicator to evaluate the overlap between the segmentation result and the true mask. To further improve the performance of the model on the overall structure, we introduce the Weighted Intersection over Union Loss (WIoU). WIoU emphasizes the importance of target boundary regions by weighting the Intersection over Union.

[0135] The specific calculation process is as follows:

[0136] ，

[0137] ，

[0138] ，

[0139] where, is the probability value of the th pixel predicted by the model, is the true label, is the corresponding weight.

[0140] By combining WBCE and WIoU, the structured loss function not only ensures the accuracy of the model at the pixel level, but also effectively maintains the overall structure and boundary precision of the predicted mask. This loss function enables the model to focus on both local details and global shape during the optimization process, thereby improving the overall performance of the segmentation task.

[0141] Experimental results:

[0142] To evaluate the performance of the model in the retinal vessel segmentation task, a series of comparative experiments were conducted on three datasets with multiple advanced segmentation algorithms using five different evaluation indicators. The specific results are shown in the following table. The model was compared with other models to ensure fair comparison under the same experimental conditions. The model achieved the best results in all evaluation indicators on the DRIVE dataset, including Dice coefficient (DC), intersection over union (IoU), precision, recall, F1 score, and specificity, with scores of 0.7962, 0.6617, 0.8153, 0.8413, 0.7962, and 0.9889, respectively. These results were significantly better than other methods. Specifically, the proposed model improved by 0.0162, 0.0223, 0.0280, 0.0359, and 0.0083 compared to the baseline model. The SCR-Net scored second only to the proposed model in DC and IoU, and performed relatively well among the remaining models. The specific results on the CHASE_DB1 dataset are shown in the following table. The model achieved the best results in all evaluation indicators. Compared to the second-best model, SCR-Net, the model improved by 0.83% in DC, 1.41% in IoU, 4.02% in Precision, 9.37% in Recall, and 0.58% in Specificity. Although META-Unet achieved a high value of 0.8215 in precision, its recall was only 0.6198, failing to match the excellent performance of the proposed model in all indicators. On the STARE dataset, the model also achieved the best results in all evaluation indicators. Compared to the latest model, META-Unet, the model improved by 1.42% in DC, 2.48% in IoU, 0.26% in Precision, 2.91% in Recall, and 1.06% in Specificity. The highest recall value indicates that the model has enhanced ability to identify more blood vessels. The comprehensive quantitative results show that the model leads in all evaluation indicators on the three datasets, demonstrating its strong performance and superior generalization ability in retinal vessel segmentation tasks. Whether in DC, IoU, or Precision and Recall, the model outperforms traditional methods and the latest advanced models, proving its excellent ability in feature extraction and boundary recognition.

[0143]

[0144]

[0145]

[0146] Ablation Study: The ablation study was conducted by systematically removing each component of the encoder and decoder modules and replacing them with a convolutional block of two convolutional layers. The experiment took UNet as the baseline network, and combined the FEM, CCA, and SSC modules in the model.

[0147]

[0148] In the experiment, the FEM module was removed first, and the results showed that the removal of this module led to a significant decrease in performance compared to other experiments, especially in the DC and IOU indicators. This indicates that the FEM module plays a crucial role in improving the overall performance of the model, effectively enhancing the accuracy of the segmentation results through feature fusion. The trend of this module limits the ability of the model to capture global and local features, further proving the important role of this module in improving the receptive field and feature extraction capability of the model. In contrast, the removal of the SSC and CCA Bottleneck has relatively little impact on the performance of the model. After the removal of these two modules, the DC and IOU indicators of the model remain at a high level, indicating that their contribution to the model, although present, is not as significant as that of the FEM. However, it is worth noting that the removal of SSC has also had some impact on the precision and recall of the model, especially in segmentation tasks under complex backgrounds, and its role cannot be ignored. In summary, the ablation experiment results clearly show that FEM is the core driving factor of the model's performance, and its removal will significantly reduce the segmentation accuracy. While SSC and CCA Bottleneck have some contribution to performance improvement, their relative impact is relatively small, but they show significant performance in reducing the number of model parameters.

[0149] The above-described embodiments only express the specific implementation of the present application, and the description is more specific and detailed, but it should not be understood as limiting the protection scope of the present application. It should be noted that for ordinary skilled persons in the art, without departing from the technical concept of the present application, several modifications and improvements can be made, which are within the protection scope of the present application.

Claims

1. A lightweight fundus blood vessel image segmentation model based on Mamba, characterized in that, Includes the following steps: The encoder and decoder include at least four SlimPVM blocks, which include depthwise separable convolutions, feature extraction modules, and parallel visual state space models. The input feature map is progressively downsampled using max pooling. At the bottleneck layer of the network, the CCA Bottleneck module computes cross-channel self-attention in the height and width dimensions of the feature map; The encoder and decoder perform feature fusion through selective skip connections, which use a channel attention mechanism to weight the selection of features. The decoder uses the SlimPVM Block to refine the upsampled features. After upsampling, the model restores the lost spatial dimensions through convolutional transpose and combines the encoded features from selective skip connections to further improve the reconstruction effect. The decoder's final output is passed through a 1x1 convolutional layer to generate the final segmentation mask, which accurately depicts the boundaries of target regions in medical images.

2. The lightweight fundus vessel image segmentation model based on Mamba according to claim 1, characterized in that, The SlimPVM Block includes the following steps: Local spatial features are extracted through depthwise separable convolutional layers, which decompose standard convolution into depthwise convolution kernel point convolutions. The convolutional features are then subjected to batch normalization and the ReLU activation function. The feature extraction module extracts global contextual information through global average pooling and uses channel attention mechanism to weight the features; The PVM layer divides the features into multiple sub-channels for parallel processing. Each sub-channel is processed through the Mamba state space model, which combines a state space matrix and convolution operations. Each sub-channel feature after processing is enhanced through residual connections, and an adjustment factor is applied to optimize the remote information acquisition capability. The sub-channel features are then recombined into complete features through a concat operation. The Mamba module normalizes the merged features and maps them to the output dimension through a projection operation.

3. The lightweight fundus vessel image segmentation model based on Mamba according to claim 2, characterized in that, The feature extraction module is used for multi-scale feature extraction and is deployed at key locations in the encoder and decoder. It fuses features of different scales to regenerate image features and generate an accurate segmentation mask. The feature extraction module includes the following steps: The number of channels in the input features is adjusted by using 1×1 convolutional layers, and batch normalization is performed to ensure that the scale of the input features is consistent. Global context information is captured through global average pooling operations; The attention weights for each channel are learned through two fully connected layers. The first fully connected layer reduces the number of channels in the global feature to 1 / reduction, compressing information and extracting important features. The second fully connected layer restores the number of channels to the original dimension, further optimizing feature selectivity. The channel attention weights are normalized using the Sigmoid function to generate weight coefficients in the range of [0,1]. The fused features are weighted using a dot product operation, and the final fused features are output.

4. A lightweight fundus vessel image segmentation model based on Mamba according to claim 2, characterized in that, The PVM layer is used to enhance the modeling of global context information and includes two branches: the first branch includes a linear layer and the SiLU activation function for preliminary feature processing; the second branch includes a linear layer, depthwise convolution, SiLU activation, a state-space model, and LayerNorm operations; specifically, it includes the following steps: For an input feature X with C channels, it is normalized using LayerNorm. The feature is divided into four sub-channels, and the number of channels in each sub-channel is C / 4; Sub-channel features are processed by the VSS module. The VSS module models the spatiotemporal information of the features through deep convolution and state space model, and uses residual connections and adjustment factors to optimize the capture of long-range dependencies. The original input features are summed with the processed features through residual connections; The four sub-channel features are recombined into features with the full number of channels through the concat operation, and then output mapped through LayerNorm and projection layers.

5. A lightweight fundus vessel image segmentation model based on Mamba according to claim 4, characterized in that, The calculation formula for the PVM layer is: Out=Pro[LN(X out )], Where LN represents LayerNorm operation, Sp is feature segmentation operation, VSS is VSS block operation, θ is the adjustment factor for residual connection, Cat is feature concatenation operation, and Pro is projection operation.

6. A lightweight fundus vessel image segmentation model based on Mamba according to claim 1, characterized in that, The selective skip connection specifically includes the following steps: The number of channels in the input features is adjusted using 1×1 convolutions to ensure consistency in channel dimensions; The distribution of features is stabilized through batch normalization; Global max pooling is used to extract the most significant response for each channel; After extracting global features, dimensionality reduction is performed through a series of 1×1 convolutions. The combination of batch normalization and ReLU activation function enhances the non-linear expressive power of the features. The number of channels is restored to the original dimension by another 1×1 convolution, and finally the precise channel attention weights are generated by the Sigmoid activation function.

7. A lightweight fundus vessel image segmentation model based on Mamba according to claim 1, characterized in that, The CCA Bottleneck module specifically includes the following steps: For feature maps The CCA Bottleneck module generates two feature maps Q and K through two 1×1 convolutions, where, C ′ It is the number of channels after dimensionality reduction; Affinity operations generate attention maps For each position u in the feature map Q, the corresponding feature vector is obtained. Extract the set of feature vectors that are in the same row or column as position u from feature map K. The formula for calculating affinity operations is: Where, d i,u ∈D represents feature Q u With Ω i,u The correlation between them The correlation matrix is ​​used; an attention map A is calculated by applying a softmax operation along the channel dimension; a feature map is generated through another 1×1 convolution. For each position u in V, obtain the corresponding feature vector. and sets Collect context information through aggregation operations: Among them, H′ u To output feature map In the feature vector at position u, A i,u These are the scalar values ​​for the corresponding channels and positions in the attention map.

Images