Light-weight eye fundus blood vessel image segmentation model based on Mama

Abstract

The invention discloses a lightweight eye fundus blood vessel image segmentation model based on Mama. The model comprises the following steps: an encoder and a decoder comprise at least four layers of SlimPVM Blocks; gradually downsampling the input feature map through maximum pooling operation; the CCA Bottleneck module is used for calculating the self-attention of the cross-channel in the height dimension and the width dimension of the feature map; the encoder and the decoder carry out feature fusion through selective jump connection, and the selective jump connection carries out weighted selection on features through a channel attention mechanism; the decoder uses SlimPVM Block to refine the features after up-sampling, the model recovers lost spatial dimensions through convolution transpose operation, and coding features from selective jump connection are combined; and the decoder finally outputs a final segmentation mask generated through a 1 * 1 convolution layer, and the segmentation mask can accurately describe the boundary of the target area in the medical image. The method is specially used for retinal vessel segmentation tasks, model parameters are reduced, calculation overhead is reduced, and meanwhile segmentation performance is kept and even improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to a lightweight fundus vascular image segmentation model based on Mamba. Background Art

[0002] Retinal diseases, including diabetic retinopathy, glaucoma, and hypertensive retinopathy, are among the leading causes of visual impairment worldwide. Fundus imaging, which can capture detailed images of the retina, plays a vital role in screening and diagnosing these diseases. Medical image segmentation technology plays an important role in retinal image analysis, which can automatically extract key retinal structures such as retinal vessels, optic disc, and macular area. Accurate segmentation of these anatomical structures is crucial for diagnosing and monitoring retinal diseases.

[0003] With the development of machine learning, retinal vessel segmentation methods based on machine learning have gradually emerged. Such methods usually extract features from training data and then use classifiers for classification, such as decision trees, support vector machines (SVMs), and random forests. However, the main problem with machine learning methods is that the feature extraction process usually needs to be manually designed and is sensitive to noise, resulting in weak generalization ability of the model. In recent years, deep learning has made important progress in medical image segmentation, especially methods based on convolutional neural networks (CNNs). Although CNN-based models perform well in local feature extraction, their perception range is limited and it is difficult to capture long-range dependencies and global context information. This is particularly evident in the task of fundus vessel segmentation, because the morphological structure of blood vessels is complex and relying solely on local features is not enough to accurately describe the entire vascular network. Following its unprecedented success in natural language tasks, Transformers have been successfully applied to computer vision problems and achieved state-of-the-art results. Although Transformers effectively extract global and long-range dependencies through the self-attention mechanism, they are insufficient in capturing local features and translation invariance. In addition, the computational complexity of the self-attention mechanism significantly increases the number of model parameters, resulting in longer inference and training time.

[0004] The above methods have improved the segmentation effect to a certain extent, but most of the existing models still have some shortcomings: they often have difficulty in processing small blood vessels in complex backgrounds, the model parameters are too large resulting in high computational overhead, and they perform poorly when dealing with low-contrast images. Summary of the invention

[0005] The purpose of the present invention is to provide a lightweight fundus vascular image segmentation model based on Mamba for the above-mentioned existing problems, which is dedicated to retinal vascular segmentation tasks, reduces model parameters, reduces computational overhead, and maintains or even improves segmentation performance.

[0006] The technical solution of the present invention is as follows:

[0007] A lightweight fundus vascular image segmentation model based on Mamba includes the following steps:

[0008] The encoder includes at least four layers of SlimPVM Block, which includes a depth-separable convolution, a feature extraction module, and a parallel visual state space model;

[0009] The input feature map is gradually downsampled through the maximum pooling operation;

[0010] At the bottleneck layer of the network, the CCA Bottleneck module calculates cross-channel self-attention in the height and width dimensions of the feature map;

[0011] The encoder and decoder perform feature fusion through selective skip connections, and the selective skip connections perform weighted feature selection through the channel attention mechanism;

[0012] The decoder uses MambaBlock to refine the upsampled features. After upsampling, the model recovers the lost spatial dimensions through convolution transpose operations and combines the encoded features from the selective skip connection to further improve the reconstruction effect.

[0013] The final output of the decoder is passed through a 1x1 convolutional layer to generate the final segmentation mask, which can accurately depict the boundaries of the target area in the medical image.

[0014] Further, the SlimPVM Block includes the following steps:

[0015] The local spatial features are extracted through the depthwise separable convolutional layer, which decomposes the standard convolution into a depthwise convolution kernel point convolution;

[0016] The convolutional features are passed through batch normalization and ReLU activation function;

[0017] The feature extraction module extracts global context information through global average pooling and uses the channel attention mechanism to weight the features;

[0018] The PVM layer divides the features into multiple sub-channels for parallel processing. Each sub-channel is processed by the Mamba state space model, which is combined by the state space matrix and convolution operation.

[0019] Each processed sub-channel feature is enhanced through residual connection, and the adjustment factor is applied to optimize the long-range information acquisition ability, and the sub-channel features are recombined into complete features through concat operation;

[0020] The Mamba module normalizes the merged features and maps them to the output dimension through a projection operation.

[0021] Furthermore, the feature extraction module is used to extract multi-scale features, is deployed at key positions of the encoder and decoder, fuses features of different scales, regenerates image features, and generates accurate segmentation masks; the feature extraction module includes the following steps:

[0022] The number of channels of the input features is adjusted through a 1×1 convolutional layer, and batch normalization is performed to ensure that the scale of the input features is consistent;

[0023] Capture global context information through global average pooling operation;

[0024] The attention weight of each channel is learned through two fully connected layers. The first fully connected layer reduces the number of channels of the global feature to the original 1 / reduction, compresses the information and extracts important features; the second fully connected layer restores the number of channels to the original dimension to further optimize feature selectivity; the channel attention weight is normalized by the Sigmoid function to generate a weight coefficient in the range of [0,1];

[0025] The fused features are weighted by a dot multiplication operation, and the final fused features are output.

[0026] Furthermore, the PVM layer is used to enhance the modeling of global context information, and includes two branches: the first branch includes a linear layer and a SiLU activation function for preliminary feature processing; the second branch includes a linear layer, a deep convolution, a SiLU activation, a state space model, and a LayerNorm operation; specifically includes the following steps:

[0027] For the input feature X with C channels, it is normalized by LayerNorm;

[0028] The features are divided into four sub-channels, and the number of channels in each sub-channel is C / 4;

[0029] The sub-channel features are processed by the VSS module respectively. The VSS module models the spatiotemporal information of the features through deep convolution and state space model, and uses residual connection and adjustment factor to optimize the capture of long-range dependencies.

[0030] The original input features are added to the processed features through residual connections;

[0031] The four sub-channel features are recombined into features with the full number of channels through the concat operation and are output mapped through the LayerNorm and projection layers.

[0032] Furthermore, the calculation formula of the PVM layer is:

[0033]

[0034] Out=Pro[LN(X out )],

[0035] Among them, LN represents the LayerNorm operation, Sp is the feature segmentation operation, VSS is the VSS block operation, θ is the adjustment factor of the residual connection, Cat is the feature concatenation operation, and Pro is the projection operation.

[0036] Furthermore, the selective jump connection specifically includes the following steps:

[0037] The number of channels of the input features is adjusted through 1×1 convolution to ensure the consistency of channel dimensions;

[0038] Stabilize the distribution of features through batch normalization;

[0039] Global maximum pooling is used to extract the most significant response of each channel;

[0040] After extracting the global features, a series of 1×1 convolutions are used to reduce the dimension, and the combination of batch normalization and ReLU activation function enhances the nonlinear expression ability of the features;

[0041] Another layer of 1×1 convolution is used to restore the number of channels to the original dimension, and finally the Sigmoid activation function is used to generate accurate channel attention weights.

[0042] Furthermore, the CCA Bottleneck module specifically includes the following steps:

[0043] For feature maps The CCA Bottleneck module generates two feature maps Q and K through two 1×1 convolutions, where C ′ is the number of channels after dimensionality reduction;

[0044] Affinity operation generates attention map

[0045] For each position u in the feature map Q, obtain the corresponding feature vector Extract the set of feature vectors in the same row or column as position u from the feature map K

[0046] The calculation formula for affinity operation is:

[0047]

[0048] Among them, di,u ∈D represents the feature Q u With Ω i,u The correlation between is the correlation matrix; the attention map A is calculated by applying a softmax operation on the channel dimension; the feature map is generated by another 1×1 convolution For each position u in V, obtain the corresponding eigenvector And collection Gather contextual information through aggregation operations:

[0049]

[0050] Among them, H′ u Output feature map The eigenvector at position u, A i,u is the scalar value corresponding to the channel and position in the attention map.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. A lightweight fundus vascular image segmentation model based on Mamba, which sets up the Feature Extraction Mamba module, encoder and decoder, which is one of the core components of the model; SlimPVM Block effectively reduces the amount of calculation by combining the deep separable convolution and feature extraction modules, while improving the efficiency of feature extraction. In addition, the PVM module introduced in SlimPVMBlock further enhances the dynamic feature representation capability, especially when dealing with complex retinal vascular structures (such as thin blood vessels and irregularly shaped blood vessels), which can significantly improve the accuracy of segmentation;

[0053] 2. A lightweight fundus vascular image segmentation model based on Mamba. In order to better capture the global contextual information of blood vessels in retinal images, the CrissCrossAttention module is introduced as the bottleneck layer of the model. CCABottleneck constructs a cross-row and cross-column global attention mechanism. This enables the model to aggregate global information from different directions of the image without significantly increasing the computational overhead, significantly improving the ability to capture retinal vascular details and improving the accuracy of vascular boundaries;

[0054] 3. A lightweight fundus vascular image segmentation model based on Mamba, which sets up a selective skip connection mechanism to effectively fuse multi-scale feature information. This design combines the channel attention mechanism to dynamically adjust the information flow between different levels to ensure that low-level features are fully utilized during high-resolution reconstruction; this mechanism enhances the robustness of the model and further improves the accuracy of retinal vascular segmentation;

[0055] 4. A lightweight fundus vascular image segmentation model based on Mamba, which also minimizes the number of model parameters and computational complexity through global lightweight design; in the process of model construction, the performance requirements in actual applications are always taken into consideration to ensure that the model can run efficiently in embedded devices or resource-constrained environments while maintaining high-quality segmentation effects.

[0056] 5. A lightweight fundus vascular image segmentation model based on Mamba achieves high-precision segmentation of retinal vessels, especially in small vessels and complex backgrounds; in a large number of experiments, the model demonstrated its powerful performance and achieved excellent results on multiple public retinal image datasets. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a network architecture diagram of a lightweight fundus vascular image segmentation model based on Mamba.

[0058] Figure 2 The dataset used in the experiment. DETAILED DESCRIPTION

[0059] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0060] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0061] See also Figure 1 , a lightweight fundus vascular image segmentation model based on Mamba, comprising the following steps:

[0062] The encoder and decoder include at least four layers of SlimPVM Block, which use deep separable convolution to reduce the amount of calculation and the number of parameters while maintaining strong feature extraction capabilities. SlimPVM Block includes deep separable convolution, feature extraction module and parallel visual state space model to improve the feature expression ability of the model while significantly reducing the amount of calculation and the number of parameters; SlimPVM Block includes the following steps:

[0063] Local spatial features are extracted through the depthwise separable convolutional layer, which decomposes the standard convolution into a depthwise convolution kernel point convolution, greatly reducing the number of parameters and computational overhead.

[0064] The convolutional features are processed through batch normalization and ReLU activation function to enhance the nonlinear expression ability;

[0065] The feature extraction module extracts global context information through global average pooling and uses the channel attention mechanism to weight the features to highlight key features and suppress irrelevant features;

[0066] The PVM layer divides the features into multiple sub-channels for parallel processing. Each sub-channel is processed by the Mamba state space model. The Mamba state space model combines the state space matrix and convolution operations to capture long-range dependencies and complex spatial relationships.

[0067] Each processed sub-channel feature is enhanced through residual connection, and the adjustment factor is applied to optimize the long-range information acquisition ability, and the sub-channel features are recombined into complete features through concat operation;

[0068] The Mamba module normalizes the merged features and maps them to the output dimension through a projection operation.

[0069] Through this parallel processing method, the Mamba module maintains the overall number of channels unchanged, maximizes performance, and significantly reduces computational overhead. The design of this module effectively balances performance and efficiency, and demonstrates excellent performance in complex visual tasks.

[0070] The feature extraction module is used to extract multi-scale features. It is deployed at the key position of the encoder and decoder to fuse features of different scales, regenerate image features, and generate accurate segmentation masks. The feature extraction module includes the following steps:

[0071] The number of channels of the input features is adjusted through a 1×1 convolutional layer, and batch normalization is performed to ensure that the scale of the input features is consistent;

[0072] Global context information is captured through global average pooling operations; this global feature extraction helps to enhance the perception of information in a large area, especially when processing complex medical images, which can bring better global information expression.

[0073] The attention weight of each channel is learned through two fully connected layers. The first fully connected layer reduces the number of channels of the global feature to the original 1 / reduction. Through dimensionality reduction, the information is compressed and important features are extracted. The ReLU activation function is applied to increase the nonlinear expression ability; the second fully connected layer restores the number of channels to the original dimension to further optimize the feature selectivity; the channel attention weight is normalized by the Sigmoid function to generate weight coefficients in the range of [0,1]; these weight coefficients are applied to the input features in a channel-weighted manner to enhance important features and suppress irrelevant features.

[0074] The fused features are weighted by a dot multiplication operation, and the final fused features are output.

[0075] The feature extraction module can generate more accurate boundary information while ensuring the balance between global information and local features, providing strong support for subsequent segmentation tasks. The design of this module aims to strengthen the model's focus on key features through the channel attention mechanism, and at the same time improve the model's perception of the overall structure of the image through the extraction of global context information, thereby achieving more accurate segmentation results.

[0076] The PVM layer is used to enhance the modeling of global context information. It includes two branches: the first branch includes a linear layer and a SiLU activation function for preliminary feature processing; the second branch includes a linear layer, deep convolution, SiLU activation, a state space model (SSM), and a LayerNorm operation; through multi-level feature extraction and conversion, the model's ability to capture complex visual features is optimized. Specifically, the following steps are included:

[0077] For the input feature X with C channels, it is normalized by LayerNorm;

[0078] The features are divided into four sub-channels, and the number of channels in each sub-channel is C / 4;

[0079] The sub-channel features are processed by the VSS module respectively. The VSS module models the spatiotemporal information of the features through deep convolution and state space model, and uses residual connection and adjustment factor to optimize the capture of long-range dependencies.

[0080] The original input features are added to the processed features through residual connections to ensure the transmission and enhancement of information;

[0081] The four sub-channel features are recombined into features with the full number of channels through the concat operation and are output mapped through the LayerNorm and projection layers.

[0082] The calculation formula of the PVM layer is:

[0083]

[0084]

[0085] Out=Pro[LN(X out )],

[0086] Among them, LN represents the LayerNorm operation, Sp is the feature segmentation operation, VSS is the VSS block operation, θ is the adjustment factor of the residual connection, Cat is the feature concatenation operation, and Pro is the projection operation.

[0087] By introducing parallel Vision Mamba operations, PVM can effectively improve model performance while keeping the amount of computation and total number of parameters relatively small. By processing feature sub-channels in parallel, PVM establishes global context information while significantly reducing computational overhead. Finally, the output features of PVM are further fused with the multi-scale features previously extracted by CNN and Transformer, thereby significantly improving the overall performance of the model while keeping the parameter scale controllable.

[0088] The input feature map is gradually downsampled through the maximum pooling operation;

[0089] At the bottleneck layer of the network, the CCA Bottleneck module calculates cross-channel self-attention in the height and width dimensions of the feature map, enabling the model to capture complex spatial relationships more effectively; by collecting contextual information in the horizontal and vertical directions, it enhances the feature expression ability at the pixel level, thereby improving the semantic segmentation performance of the model. The CCABottleneck module specifically includes the following steps:

[0090] For feature maps The CCA Bottleneck module generates two feature maps Q and K through two 1×1 convolutions, where C ′ is the number of channels after dimensionality reduction;

[0091] Affinity operation generates attention map

[0092] For each position u in the feature map Q, obtain the corresponding feature vector Extract the set of feature vectors in the same row or column as position u from the feature map K

[0093] The calculation formula for affinity operation is:

[0094]

[0095] Among them, d i,u ∈D represents the feature Q u With Ω i,u The correlation between is the correlation matrix; the attention map A is calculated by applying a softmax operation on the channel dimension; the feature map is generated by another 1×1 convolution For each position u in V, obtain the corresponding eigenvector And collection Gather contextual information through aggregation operations:

[0096]

[0097] Among them, H ′ u Output feature map The eigenvector at position u, A i,u is the scalar value corresponding to the channel and position in the attention map.

[0098] In this way, the CCA Bottleneck module is able to selectively aggregate global contextual information and add it to local features, thereby enhancing pixel-level expressiveness. This feature representation not only has a wide contextual field of view, but also enables selective context aggregation based on the spatial attention map, making it more robust and accurate in semantic segmentation tasks.

[0099] The encoder and decoder perform feature fusion through selective skip connections. The selective skip connection selects features by weight through the channel attention mechanism to ensure that the fused features are more discriminative and accurate, thereby reducing the interference of redundant information; the features are selectively enhanced through the channel attention mechanism, thereby improving the accuracy of feature fusion and the overall performance of the model. Selective skip connection (SSC) specifically includes the following steps:

[0100] The number of channels of the input features is adjusted through 1×1 convolution to ensure the consistency of channel dimensions;

[0101] Stabilize the distribution of features through batch normalization and enhance the training stability of the model;

[0102] Global maximum pooling is used to extract the most significant response of each channel; this operation can focus on the part with the highest information content in the feature map, capture the most representative features, and thus improve the model's perception of important areas.

[0103] After extracting the global features, dimensionality reduction is performed through a series of 1×1 convolutions. The combination of batch normalization and ReLU activation function enhances the nonlinear expression ability of the features; it not only compresses the information, but also ensures the extraction and strengthening of key features.

[0104] Another layer of 1×1 convolution is used to restore the number of channels to the original dimension, and finally the Sigmoid activation function is used to generate accurate channel attention weights. The weight coefficients weight the input features, thereby strengthening the important feature areas and suppressing irrelevant or minor information. This selective enhancement mechanism ensures that the information transfer in the jump connection is more accurate, retains key features, and enhances the model's ability to parse complex features.

[0105] The decoder uses MambaBlock to refine the upsampled features to ensure that the recovered features have high resolution and strong semantic expression capabilities. After upsampling, the model recovers the lost spatial dimensions through convolution transpose operations and combines the encoded features from the selective skip connection to further improve the reconstruction effect;

[0106] The final output of the decoder is passed through a 1x1 convolutional layer to generate the final segmentation mask, which can accurately depict the boundaries of the target area in the medical image.

[0107] Experimental verification:

[0108] Dataset: In order to prove that the method of this application can meet the requirements of high segmentation accuracy in medical images, training and testing were carried out on the DRIVE, STARE and CHASEDB1 datasets. The DRIVE dataset contains 40 retinal images, which are divided into training sets and test sets. Each group has 20 images. The images are acquired through a Canon CR5 non-mydriatic camera with a resolution of 565x584 pixels. The CHASEDB1 dataset contains 28 fundus images from 14 school-age children. The FOV of each image is 30μm and the resolution is 999×960 pixels. The dataset is divided into training sets and test sets. The first 20 images are used for training and the remaining 8 are used for testing. The STARE dataset consists of 400 retinal images, of which 20 images are used for vascular segmentation tasks. These images were acquired by the TopCon TRV-50 retinal camera based on 35mm film, and the image size is 605x700 pixels. Figure 2 shown.

[0109] In view of the small sample size of the fundus image dataset, in order to improve the generalization ability of the model and solve the problem of insufficient training data, data enhancement processing was performed on these three datasets. The specific data enhancement is detailed in the table below. We used techniques such as color jitter, random rotation, scaling, mirror flipping, Gaussian noise addition, sharpening, and affine transformation to simulate the various changes that may occur in the actual medical imaging process, such as brightness, contrast, angle, and noise. These changes not only increase the diversity of the data, but also improve the robustness of the model in the face of different imaging conditions, ensuring that it can perform retinal blood vessel segmentation more accurately.

[0110]

[0111] Evaluation Metrics: Five evaluation metrics are used to objectively measure the proposed model and other segmentation networks, including Dice Coefficient (DC), Intersection over Union (IoU), Recall (Recall or Sensitivity), Specificity (Specificity), F1 Score (F1 Score) and Precision (Precision). In addition to the segmentation performance metrics, we also evaluated several other performance evaluation metrics of the model to comprehensively examine the performance of the model in terms of resource consumption. Parameters measure the complexity of the model, usually expressed in millions (M). Model size refers to the space occupied by the model in memory, usually in megabytes (MB). Computational complexity is measured by the number of floating-point operations (GFLOPs), which indicates the number of floating-point calculations required by the model during one inference process. Inference time refers to the time required for the model to perform a forward propagation, usually measured in milliseconds (ms). Before introducing these five evaluation metrics in detail, some definitions are given first. Among them, TP stands for true positive, TN stands for true negative, FP stands for false positive, and FN stands for false negative.

[0112] The Dice coefficient is used to measure the degree of overlap between the segmentation result and the true label. The larger the value, the better the overlap. It is defined as follows:

[0113]

[0114] IoU, also known as the Jaccard index, measures the overlap ratio between the predicted segmentation area and the true segmentation area. It is defined as follows:

[0115]

[0116] Precision refers to the proportion of all cases predicted to be positive samples that are actually positive samples. It is defined as:

[0117]

[0118] Recall (Sen) refers to the proportion of positive samples detected as positive samples among all true positive samples. It is defined as follows:

[0119]

[0120] Specificity refers to the proportion of positive samples predicted as negative samples among all true negative samples. It is defined as:

[0121]

[0122] Experimental details: The proposed model is trained and evaluated based on the PyTorch deep learning framework. To meet the high computational requirements, we used a GEFORCE RTX 3090 graphics processing unit (GPU) with 24GB of video memory. The initial learning rate is 10-3, the training process is set to 100 epochs, and the batch size is 8. In model training, the AdamW optimizer is used, combined with the binary cross-entropy loss function (Binary Cross-Entropy Loss) and the weighted IoULoss loss (Weighted IoULoss) for optimization.

[0123] Loss Function: The loss function is used to evaluate the difference between the predicted value of the network model and the true value. The smaller the value of the loss function, the smaller the difference between the predicted value and the true value of the model, which means that the performance of the model is better. In this study, we adopted a structured loss function that combines weighted binary cross entropy (WBCE) and weighted intersection over union (WIoU) to more effectively optimize the model performance in image segmentation tasks.

[0124] (1) Weighted Binary Cross Entropy Loss (WBCE)

[0125] The Binary Cross Entropy (BCE) loss function is used to measure the difference between the predicted probability and the true label in a binary classification task. It is defined as follows:

[0126]

[0127] Where N is the number of pixels, yi is the true label of the i-th pixel, The model predicts the probability that the pixel belongs to the positive class.

[0128] The weighted binary cross entropy loss is calculated as:

[0129]

[0130] (2) Weighted Intersection over Union Loss (WIoU)

[0131] Intersection over Union (IoU) is a common metric for evaluating the overlap between the segmentation result and the true mask. In order to further improve the performance of the model in the overall structure, we introduced the weighted intersection over union loss (WIoU). WIoU emphasizes the importance of the target boundary area by weighting the intersection over union.

[0132] The specific calculation process is as follows:

[0133]

[0134] Among them, P i is the probability value of the i-th pixel predicted by the model, M i is the true label, W i is the corresponding weight.

[0135] By combining WBCE and WIoU, the structured loss function can not only ensure the accuracy of the model at the pixel level, but also effectively maintain the overall structure and boundary refinement of the predicted mask. This loss function enables the model to focus on both local details and global shapes during the optimization process, thereby improving the overall performance of the segmentation task.

[0136] Experimental results: In order to evaluate the performance of the model in the retinal vessel segmentation task, a series of comparative experiments were conducted on three datasets with a variety of advanced segmentation algorithms using five different evaluation indicators. The specific results are shown in the following table. The model was compared with other models to ensure fair comparison under the same experimental conditions. Among all the evaluation indicators on the DRIVE dataset, the model achieved the best results in Dice coefficient (DC), intersection over union (IoU), precision, recall, F1 score (F1Score) and specificity, reaching 0.7962, 0.6617, 0.8153, 0.8413, 0.7962 and 0.9889 respectively. These results are significantly better than other methods. Specifically, the proposed model is improved by 0.0162, 0.0223, 0.0280, 0.0359 and 0.0083 respectively compared with the baseline model. The score performance of SCR-Net in DC and IoU is second only to the proposed model, and it performs relatively well in the remaining models. The specific results on the CHASE_DB1 dataset are shown in the following table. The model achieved the best results in all evaluation indicators. Compared with the second-best model SCR-Net, the DC increased by 0.83%, the IoU increased by 1.41%, the Precision increased by 4.02%, the Recall increased by 9.37%, and the Specificity increased by 0.58%. Although META-Unet achieved a high value of 0.8215 in precision, its recall rate was only 0.6198, which failed to achieve the excellent performance of the model in this application in all indicators. On the STARE dataset, the model also achieved the best results in all evaluation indicators. Compared with the latest model META-Unet, the model improved by 1.42% in DC, 2.48% in IoU, 0.26% in Precision, 2.91% in Recall, and 1.06% in Specificity. The highest Recall value indicates that the model has enhanced its ability to identify more blood vessels. The comprehensive quantitative results show that the model is in a leading position in all evaluation indicators on the three datasets, demonstrating its strong performance and superior generalization ability in the task of retinal vessel segmentation. Whether in terms of key indicators such as DC, IoU, Precision, and Recall, the model outperforms traditional methods and the latest advanced models, demonstrating its superior capabilities in feature extraction and boundary recognition.

[0138]

[0139]

[0140]

[0141] Ablation study: Ablation study is performed by systematically removing each component of encoder and decoder modules and replacing it with a convolutional block of two convolutional layers. This experiment uses UNet as the baseline network and adds FEM, CCA and SSC modules in combination in the model.

[0142]

[0143] In the experiment, the FEM module was first removed. The results showed that the removal of this module led to a significant decrease in performance compared with other experiments, especially in the DC and IOU indicators. This shows that the FEM module plays a key role in improving the overall performance of the model. It effectively enhances the accuracy of the segmentation results through feature fusion. The trend of this module limits the model's ability to capture global and local features, further proving the important role of this module in improving the model's receptive field and feature extraction capabilities. In contrast, the removal of SSC and CCA Bottleneck has a relatively small impact on the model performance. After these two modules are removed, the DC and IOU indicators of the model remain at a high level, indicating that their contribution to the model, although it exists, is not as significant as FEM. However, it is worth noting that the removal of SSC also has a certain impact on the precision and recall rate of the model, especially in the segmentation task under complex backgrounds, its role cannot be ignored. In summary, the ablation experiment results clearly show that FEM is the core driving factor of the model performance, and its removal will significantly reduce the segmentation accuracy. Although SSC and CCA Bottleneck have a certain contribution to performance improvement, the impact is relatively small, but it is significant in reducing the number of model parameters.

[0144] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A lightweight fundus vascular image segmentation model based on Mamba, characterized in that: The following steps are involved: The encoder and decoder include at least four layers of SlimPVM Block, which includes depthwise separable convolution, feature extraction module and parallel visual state space model; The input feature map is gradually downsampled through the maximum pooling operation; At the bottleneck layer of the network, the CCA Bottleneck module calculates cross-channel self-attention in the height and width dimensions of the feature map; The encoder and decoder perform feature fusion through selective skip connections, and the selective skip connections perform weighted feature selection through the channel attention mechanism; The decoder uses SlimPVM Block to refine the upsampled features. After upsampling, the model recovers the lost spatial dimensions through convolution transpose operations and combines the encoded features from the selective skip connection to further improve the reconstruction effect. The final output of the decoder is passed through a 1x1 convolutional layer to generate the final segmentation mask, which can accurately depict the boundaries of the target area in the medical image.

2. According to the Mamba-based lightweight fundus vascular image segmentation model of claim 1, it is characterized in that: The SlimPVM Block includes the following steps: The local spatial features are extracted through the depthwise separable convolutional layer, which decomposes the standard convolution into a depthwise convolution kernel point convolution; The convolutional features are passed through batch normalization and ReLU activation function; The feature extraction module extracts global context information through global average pooling and uses the channel attention mechanism to weight the features; The PVM layer divides the features into multiple sub-channels for parallel processing. Each sub-channel is processed by the Mamba state space model, which is combined by the state space matrix and convolution operation. Each processed sub-channel feature is enhanced through residual connection, and the adjustment factor is applied to optimize the long-range information acquisition ability, and the sub-channel features are recombined into complete features through concat operation; The Mamba module normalizes the merged features and maps them to the output dimension through a projection operation.

3. The Mamba-based lightweight fundus vascular image segmentation model according to claim 2, characterized in that: The feature extraction module is used to extract multi-scale features and is deployed at key positions of the encoder and decoder to fuse features of different scales, regenerate image features, and generate accurate segmentation masks. The feature extraction module includes the following steps: The number of channels of the input features is adjusted through a 1×1 convolutional layer, and batch normalization is performed to ensure that the scale of the input features is consistent; Capture global context information through global average pooling operation; The attention weight of each channel is learned through two fully connected layers. The first fully connected layer reduces the number of channels of the global feature to the original 1 / reduction, compresses the information and extracts important features; the second fully connected layer restores the number of channels to the original dimension to further optimize feature selectivity; the channel attention weight is normalized by the Sigmoid function to generate a weight coefficient in the range of [0,1]; The fused features are weighted by a dot multiplication operation, and the final fused features are output.

4. The Mamba-based lightweight fundus vascular image segmentation model according to claim 2, characterized in that: The PVM layer is used to enhance the modeling of global context information, and includes two branches: the first branch includes a linear layer and a SiLU activation function for preliminary feature processing; the second branch includes a linear layer, a deep convolution, a SiLU activation, a state space model, and a LayerNorm operation; specifically includes the following steps: For the input feature X with C channels, it is normalized by LayerNorm; The features are divided into four sub-channels, and the number of channels in each sub-channel is C / 4; The sub-channel features are processed by the VSS module respectively. The VSS module models the spatiotemporal information of the features through deep convolution and state space model, and uses residual connection and adjustment factor to optimize the capture of long-range dependencies. The original input features are added to the processed features through residual connections; The four sub-channel features are recombined into features with the full number of channels through the concat operation and are output mapped through the LayerNorm and projection layers.

5. The Mamba-based lightweight fundus vascular image segmentation model according to claim 4, characterized in that: The calculation formula of the PVM layer is: Out=Pro[LN(X out )], Among them, LN represents the LayerNorm operation, Sp is the feature segmentation operation, VSS is the VSS block operation, θ is the adjustment factor of the residual connection, Cat is the feature concatenation operation, and Pro is the projection operation.

6. The Mamba-based lightweight fundus vascular image segmentation model according to claim 1, characterized in that: The selective jump connection specifically comprises the following steps: The number of channels of the input features is adjusted through 1×1 convolution to ensure the consistency of channel dimensions; Stabilize the distribution of features through batch normalization; Global maximum pooling is used to extract the most significant response of each channel; After extracting the global features, a series of 1×1 convolutions are used to reduce the dimension, and the combination of batch normalization and ReLU activation function enhances the nonlinear expression ability of the features; Another layer of 1×1 convolution is used to restore the number of channels to the original dimension, and finally the Sigmoid activation function is used to generate accurate channel attention weights.

7. The Mamba-based lightweight fundus vascular image segmentation model according to claim 1, characterized in that: The CCA Bottleneck module specifically includes the following steps: For feature maps The CCA Bottleneck module generates two feature maps Q and K through two 1×1 convolutions, where C ′ is the number of channels after dimensionality reduction; Affinity operation generates attention map For each position u in the feature map Q, obtain the corresponding feature vector Extract the set of feature vectors in the same row or column as position u from the feature map K The calculation formula for affinity operation is: Among them, d i,u ∈D represents the feature Q u With Ω i,u The correlation between is the correlation matrix; the attention map A is calculated by applying a softmax operation on the channel dimension; the feature map is generated by another 1×1 convolution For each position u in V, obtain the corresponding eigenvector And collection Gather contextual information through aggregation operations: Among them, H′ u Output feature map The eigenvector at position u, A i,u is the scalar value corresponding to the channel and position in the attention map.

Images