Multi-task blood vessel segmentation model construction method based on hybrid encoder

Abstract

The invention discloses a multi-task blood vessel segmentation model construction method based on a hybrid encoder. The encoder is a Swin Transform path multi-layer encoder based on a Mamform module, and the Mamform multi-layer encoder carries out data processing in combination with a W-MSA and an improved Mamba module; the decoder is a multi-task decoder and consists of three independent decoding paths which are respectively used for blood vessel segmentation, contour edge segmentation and blood vessel connectivity segmentation, and each decoding path adopts Swin Transformblock as a construction block and is used for generating feature mapping with the same size as an input image; an attention mechanism is shared, learned features are shared among decoders, the number of model parameters can be reduced, and overfitting is prevented; in the training process, a total loss function is formed through a loss function of a vascular connectivity decoding path, a self-distillation loss function and the like, the total loss function is calculated, model parameters are updated by using a back propagation algorithm, the segmentation performance of the model is gradually optimized, and after training is completed, the model can carry out precision on a vascular image.

Description

Technical Field

[0001] The present invention relates to the technical field of blood vessel image processing, and in particular to a method for constructing a multi-task blood vessel segmentation model based on a hybrid encoder. Background Art

[0002] Early diagnosis and medical intervention are crucial to improving the prognosis of patients with vascular diseases. Clinicians can use vascular imaging technology to observe abnormalities in blood vessels and intervene in the early stages of the disease. However, clinicians need to have sufficient clinical experience to make accurate diagnoses. Deep learning (DL) technology that effectively utilizes data has opened up important new research avenues for medical image analysis.

[0003] The deep learning models currently used for image segmentation tasks are generally based on the U-shaped structure network of encoder-decoder, in which CNN or Vision Transformer is used in the encoder part. "U-Net: Convolutional Networks for Biomedical Image Segmentation" points out that UNet is a convolutional neural network (CNN) architecture for medical image segmentation. It has become a classic model in image segmentation tasks with its excellent segmentation performance and simple structure. UNet is mainly used for pixel-level classification tasks, especially for scenarios that require accurate boundary segmentation, such as medical image analysis. UNet consists of two main parts: encoder (downsampling path) and decoder (upsampling path), as well as skip connections between them. Each convolution layer in the convolutional neural network operates on the vascular image through a small convolution kernel (such as 3x3 or 5x5). Although small convolution kernels can effectively capture local features, small convolution kernels will limit the receptive field, that is, each convolution kernel can only focus on the local area of ​​the image, and cannot directly capture a larger range of global information.

[0004] Although Vision Transformer can capture global image information, it is limited by the attention mechanism, especially when dealing with long sequence modeling, it has high quadratic complexity, which leads to expensive computational overhead when dealing with downstream dense prediction tasks (such as target detection, semantic segmentation, etc.). In addition, in medical imaging, high-resolution images such as whole-slice pathological images are common, where transformer-based models may have overfitting problems and require a large amount of data for effective training.

[0005] Swin Transformer is an improved Transformer variant designed for computer vision tasks, especially for high-resolution images. The Swin Transformer path adopts a hierarchical design, usually with multiple stages. Except for the first stage, which is composed of linear embedding and multiple blocks, each of the remaining stages is composed of patchmerging and multiple blocks. Patch merging is used to reduce the rate of variation of feature maps. Patch partition operation is performed before the first stage to divide the image into small blocks. Swin Transformer Block is composed of Layer Norm, W-MSA, MLP, SW-MSA, etc.

[0006] W-MSA (Window-based Multi-Head Self-Attention) is an important component in visual Transformer (such as SwinTransformer). It improves on the standard self-attention mechanism to adapt to image processing tasks. The input image is divided into multiple non-overlapping windows (for example, each window size is P×P). Within each window, a multi-head self-attention mechanism is used to capture local contextual information. The attention calculation of each head is performed within the window range, which can preserve spatial information and improve efficiency. In order to enhance the expressiveness of the model, the window is sometimes moved after the window is divided, so that the model can handle cross-window dependencies, which can reduce the information loss caused by window division. However, although limiting the window size or stride can reduce the attention calculation cost, it also limits the range of the receptive field, which directly affects the segmentation accuracy.

[0007] Modern SSMs not only model long-range dependencies but also exhibit linear complexity in the input size. In modern SSM-based models, namely the structured state-space sequence model (S4) and Mamba, both rely on a classical continuous system that is constructed via intermediate implicit states h(t)∈R N Maps a one-dimensional input function or sequence to x(t)∈R N To output y(t)∈R N, in particular, Vision Mamba further improves S4 through a selective mechanism, enabling the model to select relevant information in an input-dependent manner. However, the existing Mamba architecture also has limitations that need to be addressed. Since Mamba is an SSM-based architecture, the hidden state h(t) of the Mamba model can only access information from previous and current time steps, which is a token-mixed causal pattern. As an understanding task, image segmentation inherently limits its representation capabilities due to this causal pattern. Although Vim proposed the Cross-Scan Moudle (CSM) to alleviate the lack of global context and spatial understanding, each branch is still limited to the causal pattern, which will always limit its modeling representation capabilities.

[0008] Although the current encoder-decoder based U-structure network performs well in most medical image segmentation tasks, it has certain limitations in vascular image segmentation. The main reasons are: the vascular structure shows obvious morphological changes, including thick blood vessels, thin blood vessels and filamentous retinal vessels; small blood vessels have low contrast, diverse shapes, and lengths of generally less than 10 pixels, and some are even less than 1 pixel; non-vascular structures or lesions (such as the optic disc) may affect the segmentation performance; the presence of artifacts caused by improper data collection brings additional challenges, resulting in the single decoder structure of the deep network being unable to restore the vascular image well.

[0009] Although multi-branch decoders can achieve higher accuracy and more precise segmentation at this stage, in the task of vascular segmentation, missing vascular segments may be pathologically mistaken for stroke or cause drastic changes in the global simulation of blood flow. Therefore, correctly predicted spatially continuous sequences are more important than false correct predictions. A significant drawback of existing multi-decoder networks is that they ignore the consistency of predictions between branches, resulting in suboptimal performance and lack of correlation between branches. Summary of the invention

[0010] The present invention aims to solve the above-mentioned technical problems existing in the prior art and provides a method for constructing a multi-task blood vessel segmentation model based on a hybrid encoder.

[0011] The technical solution of the present invention is: a method for constructing a multi-task blood vessel segmentation model based on a hybrid encoder, which is constructed based on a U-shaped structure network of an encoder-decoder:

[0012] The encoder is a Swin Transformer path multi-layer encoder based on the Mamformer module. The Mamformer module divides the input image block into two information streams after layer normalization. One group of information streams passes through the linear layer, 3×3 depth convolution layer, SS2D block, and normalization layer in sequence, and is added to the other group of information streams that pass through the linear layer and W-MSA in sequence. The merged information stream passes through the linear layer and the input jump connection to obtain the final output of the Mamformer module.

[0013] The decoder is a multi-task decoder, which consists of three decoding paths, which are used for blood vessel segmentation, contour edge segmentation and blood vessel connectivity segmentation respectively. Each decoding path uses Swin Transformer Block and PatchExpanding as building blocks to generate feature maps with the same size as the input image. A shared attention mechanism is set between the Swin Transformer Blocks of the three decoding paths, and finally the outputs of the three decoding paths are spliced ​​along the channel dimension.

[0014] The loss function expression of the vascular connectivity decoding path is as follows:

[0015] L v =1-softclDice

[0016] Among them, L v represents the vascular connectivity loss function, softclDice represents the use of continuous probability values ​​to calculate clDice, and clDice represents the clDice coefficient. The specific expression is as follows:

[0017]

[0018] Among them, V P represents the truth mask, V L represents the predicted segmentation mask, S P and S L Respectively represent from V P ,V L Extract skeleton, Tprec(S P ,V L ) indicates S P ,V L The topological accuracy between L ,V P ) indicates S L ,V P The topological sensitivity between them is expressed as follows:

[0019]

[0020] Among them, |S P ∩V L | indicates S P With V L The number of elements in the intersection, |S P | indicates S P The number of elements of |S L ∩V P | indicates S L With V P The number of elements in the intersection, |S L | indicates S L The number of elements of ;

[0021] A self-distillation loss function is constructed to adjust the segmentation consistency between the three independent decoding paths. The expression of the self-distillation loss function is as follows:

[0022] L sd =1-Dice(sobel(y n ),y e )

[0023] Among them, L sd represents the self-distillation loss function, and Dice represents sobel(y n ) and y e Dice loss between, sobel(y n ) represents the vascular branch prediction y n Sobel operation, y e Represents the vessel contour edge predicted by the decoded path.

[0024] The SS2D block includes a scan extension operation, an S6 block operation and a scan merging operation.

[0025] The overall loss function L of the decoder is total The expression is as follows:

[0026] L total =y n *L n +y e *L e +y v *L v +y sd *L sd

[0027] Among them, L total Represents the overall loss function of the decoder, y n ,y e ,y v ,y sd Respectively represent L n , L e, L v , L sd Corresponding weight, L n represents the semantic vessel mask loss function, L e represents the edge loss function of the blood vessel contour, L v represents the vascular connectivity loss function, L sd represents the self-distillation loss function.

[0028] The present invention is constructed based on a U-shaped structure network of encoder-decoder, including four key modules: Mamformer hybrid encoder, multi-task decoder based on blood vessel, contour and blood vessel connectivity analysis, blood vessel distillation loss and shared parameter mechanism. The Mamformer hybrid encoder integrates the improved Mamba with W-MSA in the feature encoding process, and effectively alleviates the problem of vascular contour changes caused by the processing mode of causal sequences when the Mamba-like model is down-sampled by providing a global image within the window. At the same time, by limiting the attention mechanism to perform calculations only within the window, the computational cost of the Transformer-like model is significantly reduced; the multi-task decoder based on blood vessel, contour and vascular connectivity analysis combines the vascular edge detection method with the vascular connectivity optimization method, maintains the tubular topological structure of the blood vessel during feature decoding, and makes full use of the contour information of the vascular image to avoid changes in the vascular topological structure during upsampling; by adopting a shared attention mechanism, the learned features are shared between decoders, which can reduce the number of model parameters and prevent overfitting; during the training process, the total loss function is formed by the loss function of the vascular connectivity decoding path and the self-distillation loss function, and the model parameters are updated using the back-propagation algorithm to gradually optimize the segmentation performance of the model. After training, the model can accurately segment vascular images. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the framework of an embodiment of the present invention.

[0030] Figure 2 Schematic diagram of the Mamformer framework in an embodiment of the present invention.

[0031] Figure 3 This is a schematic diagram of the SS2D block operation according to an embodiment of the present invention.

[0032] Figure 4 It is a visualization diagram of the segmentation result of the public data set according to an embodiment of the present invention.

[0033] Figure 5 It is a schematic diagram of a local segmentation result extracted from a public data set according to an embodiment of the present invention.

[0034] Figure 6This is a schematic diagram of the SS2D pseudo code of an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention provides a method for constructing a multi-task blood vessel segmentation model based on a hybrid encoder. Figure 1-Figure 6 As shown, it is constructed based on the U-shaped structure network of encoder-decoder:

[0036] The encoder is a multi-layer encoder of the Swin Transformer path based on the Mamformer module. The Mamformer module divides the input image block into two information streams after layer normalization. One group of information streams passes through the linear layer, 3×3 depth convolution layer, SS2D block, and normalization layer in sequence, and is added to the other group of information streams that pass through the linear layer and W-MSA in sequence. The merged information stream passes through the linear layer and the input jump connection to obtain the final output of the Mamformer module.

[0037] The decoder is a multi-task decoder, which consists of three decoding paths, which are used for blood vessel segmentation, contour edge segmentation and blood vessel connectivity segmentation respectively. Each decoding path uses Swin Transformer Block and PatchExpanding as building blocks to generate feature maps with the same size as the input image. A shared attention mechanism is set between the Swin Transformer Blocks of the three decoding paths, and finally the outputs of the three decoding paths are spliced ​​along the channel dimension, and the spliced ​​results are input into a 1×1 convolutional layer.

[0038] The loss function expression of the vascular connectivity decoding path is as follows:

[0039] L v =1-softclDice

[0040] Among them, L v represents the vascular connectivity loss function, softclDice represents the use of continuous probability values ​​to calculate clDice, and clDice represents the clDice coefficient. The specific expression is as follows:

[0041]

[0042] Among them, V P represents the truth mask, V L represents the predicted segmentation mask, S P and S L Respectively represent from V P ,V L Extract skeleton, Tprec(S P ,V L ) indicates S P,V L The topological accuracy between L ,V P ) indicates S L ,V P The topological sensitivity between them is expressed as follows:

[0043]

[0044] Among them, |S P ∩V L | indicates S P With V L The number of elements in the intersection, |S P | indicates S P The number of elements of |S L ∩V P | indicates S L With V P The number of elements in the intersection, |S L | indicates S L The number of elements of ;

[0045] A self-distillation loss function is constructed to adjust the segmentation consistency between the three independent decoding paths. The expression of the self-distillation loss function is as follows:

[0046] L sd =1-Dice(sobel(y n ),y e )

[0047] Among them, L sd represents the self-distillation loss function, and Dice represents sobel(y n 0 and y e Dice loss between, sobel(y n ) represents the vascular branch prediction y n Sobel operation, y e Represents the vessel contour edge predicted by the decoded path.

[0048] The SS2D block includes a scan extension operation, an S6 block operation and a scan merging operation.

[0049] The overall loss function L of the decoder is total The expression is as follows:

[0050] L total =y n *L n +y e *L e +y v *L v +y sd *Lsd

[0051] Among them, L total Represents the overall loss function of the decoder, y n ,y e ,y v ,y sd Respectively represent L n , L e , L v , L sd Corresponding weight, L n represents the semantic vessel mask loss function, L e represents the edge loss function of the blood vessel contour, L v represents the vascular connectivity loss function, L sd represents the self-distillation loss function.

[0052] The present invention proposes an end-to-end vessel segmentation framework (MamVeSeg), such as Figure 1 As shown in the figure. The specific process is described as follows: (1) Feature extraction: First, the vascular image is input into the multi-layer encoder of the Swin Transformer path based on the Mamformer module. The encoder has the ability to perceive the details of vascular morphology with high resolution and effectively captures the global topological structure of the blood vessels. (2) Feature reconstruction: The decoder is a multi-task decoder, which consists of three decoding paths, which are used for vascular segmentation, contour edge segmentation and vascular connectivity segmentation respectively. It can maintain the tubular topological structure of the blood vessels and make full use of the contour information of the vascular image, avoiding the change of the vascular topological structure during the upsampling process, and generating high-quality segmentation results. (3) Container distillation loss: A new self-distillation loss function is introduced to solve the problem of inconsistent prediction results of multi-branch decoders. This function is used to adjust the segmentation consistency between decoders. (4) Efficiency optimization: By adopting a shared attention mechanism, the learned features are shared between decoders, which can reduce the number of model parameters and prevent overfitting. (5) Feature fusion: The outputs of the three decoders are concatenated along the channel dimension, and then a 1×1 convolution layer is applied to the concatenated feature map to obtain the final segmentation result.

[0053] The Mamformer framework of the present invention is as follows Figure 2As shown, the vascular image is preprocessed as the input of the Mamformer module. This segmentation is intended to prevent the loss of fine details in the high-resolution vascular image due to the causal mode of the Mamba module during downsampling, and to prevent the high quadratic complexity generated by the attention module in long sequence modeling. The specific processing steps of the Mamformer module are as follows: the Mamformer module divides the input image block into two information streams after layer normalization, one group of information streams passes through the linear layer, 3×3 deep convolution layer, SS2D block, normalization layer in sequence, and is added to the other group of information streams that pass through the linear layer and W-MSA in sequence. The merged information stream passes through the linear layer and the input jump connection to obtain the final output of the Mamformer module; the present invention maintains the global modeling capability similar to Transformer, while retaining the low complexity and computational requirements similar to Mamba.

[0054] The SS2D block solves the causality problem through three components: scan extension operation, S6 block operation and scan merge operation, e.g. Figure 3 As shown. The present invention chooses to expand the image patch sequence along rows and columns (scan expansion), and then scans in four different directions: top left to bottom right, bottom right to top left, top right to bottom left, bottom left to top right, bottom left to top right. Each sequence is then reconstructed into a single image and merged into a new sequence. The pseudo code of the S6 block is as follows Figure 6 shown.

[0055] Figure 1 The decoding path uses SwinTransformer Block and Patch Expanding upsampling operations to map the high-level features extracted by the encoder back to the original image space and convert it into a feature map of the same size as the input image. The Swin Transformer Blocks of the three decoding paths complete the contour edge detection and vascular connectivity detection through a shared attention mechanism to capture the edge map of the vascular image and ensure the edge details. Moreover, in order to allow the decoding path corresponding to the vascular connectivity decoder to maintain the tubular topology of the blood vessels during the upsampling process, the present invention introduces a new loss function for maintaining vascular connectivity, which is called clDice. Specifically, the present invention considers two binary masks: ground truth mask (V L ) and prediction segmentation mask(V P ), first, from V P and V L Extract the skeleton S P and S L ; Then, the present invention calculates the V L S within PThe fraction is called topological precision or Tprec (S P ; V L ), and vice versa, the present invention obtains the topological sensitivity or Tsens(S L ; V P ), defined as follows:

[0056]

[0057] The present invention observes that Tprec(SP; VL) is prone to false positives in prediction, while measuring Tsens(SL; VP) is prone to false negatives. Therefore, Tprec(SP; VL) is used as the precision of the topology and Tsens(SL; VP) is used as the sensitivity. Since the present invention wants to maximize precision and sensitivity (recall), the present invention defines clDice as the harmonic mean of the two metrics (also called F1 or Dice):

[0058]

[0059] The clDice formula of the present invention is not defined for Tprec=0 and Tsens=0, but can be easily extended with the value 0 continuously.

[0060] The present invention proposes "soft skeletonization", in which iterative minimum pooling and maximum pooling are applied as agents for morphological erosion and dilation. Compared with other volume losses such as Dice and cross entropy, clDice only considers the deformation contraction map of the solid foreground structure. Therefore, clDice in the present invention requires a minimum of new correctly predicted voxels to ensure homotopic equivalence. Along these lines, Dice or cross-entropy can only guarantee homotopic equivalence if every voxel is correctly segmented. On the other hand, clDice can guarantee homotopic equivalence for a wider range of connected voxel combinations, which is intuitively a very desirable property because it makes clDice robust to outliers and noisy segmentation labels. In summary, the loss function of the vascular connectivity decoding path of the present invention is:

[0061] L v =1-softclDice

[0062] Among them, L v represents the vascular connectivity loss function, softclDice represents the use of continuous probability values ​​for clDice, clDice represents the clDice coefficient, V P represents the truth mask, V L represents the predicted segmentation mask, S P and S L Respectively represent from V P ,V LExtract skeleton, Tprec(S P ,V L ) indicates S P ,V L The topological accuracy between L ,V P ) indicates S L ,V P The topological sensitivity between |S P ∩V L | indicates S P With V L The number of elements in the intersection, |S P | indicates S P The number of elements of |S L ∩V P | indicates S L With V P The number of elements in the intersection, |S L | indicates S L The number of elements of , soft means that hard thresholding is not used when calculating clDice. The traditional clDice coefficient is usually used for binary classification problems and is calculated by thresholding the predicted probability into a binary value (0 or 1), while softclDice is calculated directly using the predicted continuous probability value. This method is smoother and allows the model to handle uncertainty in the prediction, thereby performing better when dealing with small samples or fuzzy boundaries. In order to alleviate the problem that the contours generated by the vascular segmentation prediction are inconsistent with the predicted edges, the present invention proposes a new vascular self-distillation loss, denoted as L sd Formally, this regularization is defined as using the Sobel operation (Sobel(y n )) Prediction from blood vessel branches (y n ) and the edge y predicted from the contour edge decoder path e The Dice loss between them. Among them, the blood vessel self-distillation loss L sd The expression L sd =1-Dice(sobel(y n ),y e Finally, the overall loss L of the present invention is calculated as the semantic vessel mask loss (L n ), blood vessel contour edge loss (L e ), loss of vascular connectivity (L v ) and self-distillation loss (L sd ) is the weighted sum of:

[0063] L total =y n *L n +y e *L e +yv *L v +y sd *L sd

[0064] Among them, y n ,y e ,y v The coefficients are set to 0.30, 0.35, and 0.35 respectively. sd Initially set to 1, and decrease by 0.3 every 10 epochs until it reaches 0.4. sd represents the self-distillation loss function, and Dice represents sobel(y n ) and y e Dice loss between, sobel(y n ) represents the vascular branch prediction y n Sobel operation, y e Represents the blood vessel contour edge predicted by the decoding path, y n ,y e ,y v ,y sd Respectively represent L n , L e , L v , L sd Corresponding weight, L n represents the semantic vessel mask loss function, L e represents the edge loss function of the blood vessel contour, L v represents the vascular connectivity loss function, L sd represents the self-distillation loss function;

[0065] Experimental results:

[0066] The present invention (MamVeSeg) is compared with some of the most relevant models including U-Net, R2U-Net, Attention-UNet, Global ConvolutionalNetwork (GCN), Deeplab V3+, Selective Kernel (SK), CBAM, PSPNet, ENet, SegNet, SwinUNet, TransUNet and sgat-net. The performance results on the FIVES dataset are shown in Table 2. MamVeSeg outperforms other methods in 4 of the 6 indicators. Specifically, CBAM with channel and spatial attention achieved the highest sensitivity (93.30%), while MamVeSeg achieved the second highest IoU (83.18%), only 0.29% behind sgat-net. In addition, the specificity of MamVeSeg reached 99.61%, indicating that its ability to extract vascular image features is superior to other models. MamVeSeg has an accuracy of 99.00%, while swin-unet and TransUNet have accuracies of 98.82% and 98.83%, respectively. It is worth noting that swin-unet is entirely based on swin-Transformer, while TransUNet integrates CNN and Transformer architectures. These results highlight the significant performance advantages of the proposed Mamformer encoder. In addition, MamVeSeg is 0.3% higher than sgat-net, which has the second highest AUC. MamVeSeg achieved a higher F1 score than ssat-net and 0.16% higher than sgat-net. Together, these findings demonstrate that MamVeSeg outperforms other state-of-the-art models on several key evaluation metrics. In addition, this paper also evaluates cases of three different diseases and normal eyes to explore its potential medical applications.

[0067] Table 2 shows the performance of these classic methods on three diseases (AMD, DR, GC) and normal eyes. For AMD, MamVeSeg showed excellent specificity (99.59%), accuracy (99.08%), F1 score (93.25%), and AUC (96.01%), outperforming other models, although the sensitivity was slightly lower than TransU-Net (about 0.12%) and IOU (about 0.53%). For DR, although Attention-UNet had the highest recall, SGAT-Net exceeded MamVeSeg by 0.26% and 0.10% in IOU and F1 score, respectively, MamVeSeg performed well in other aspects. For GC disease, MamVeSeg performs best in SP and ACC, and performs second in SE, IOU, F1, and AUC, with a specificity slightly lower than Attention-UNet (about 0.99%), IOU similar to TransU-Net and SGAT-Net (about 0.08%), and F1 and AUC slightly lower than TransU-Net and Deeplab V3+ (about 0.08%). In normal eye cases, MamVeSeg has the highest accuracy (98.76%), AUC (95.16%), F1 score (91.51%), sensitivity (92.38%), and specificity (99.64%), and IOU is slightly lower than SGAT-Net (about 0.24%). Overall, MamVeSeg outperforms similar models, achieves significant results in segmenting normal eyes, and demonstrates competitive performance in areas prone to common errors. In addition, its advantages are also reflected in quantitative indicators such as ACC and specificity.

[0068] Table 1: Comparison results of related deep learning methods on the FIVES dataset

[0069] Method SE(%) SP(%) ACC(%) IOU(%) F1(％) AUC (%) UNet 91.86 99.10 98.66 80.77 88.70 93.00 Attention-UNet 92.73 99.07 98.68 80.73 88.81 92.72 R2UNet 84.52 98.99 98.09 74.65 84.92 92.38 GCN 91.91 99.22 98.79 82.60 90.02 93.99 DeepLabV3+ 87.92 99.33 98.50 80.75 88.56 94.85 SK 89.77 99.12 98.58 79.94 88.35 93.34 CBAM 93.30 99.01 98.67 80.29 88.50 92.26 PSPNet 91.92 99.20 98.78 82.35 89.88 93.96 ENet 89.93 99.22 98.67 81.10 89.09 94.09 SegNet 84.84 98.99 98.13 74.98 85.09 92.44 SwinUNet 92.27 99.22 98.82 82.76 90.13 94.02 TransUNet 91.80 99.28 98.83 83.17 90.37 94.47 SGAT-Net 91.62 99.33 98.86 83.47 90.51 94.67 MamVeSeg (the present invention) 92.62 99.61 99.00 83.18 90.66 94.97

[0070] Table 2: Classification results of top methods on the FIVES dataset

[0071]

[0072] In Table 3, the present invention evaluates the performance of MamVeSeg on coronary artery images, using a training strategy of randomly selecting 40 images for training and the remaining 10 images for testing. The present invention compares U-Net, Attention-UNet, TransUNet, SwinUNet, and SGAT-Net. The results show that MamVeSeg achieves the best sensitivity, specificity, accuracy, F1 score, and AUC in the coronary artery segmentation task, and the IOU is the second best.

[0073] Overall, MamVeSeg outperforms similar models, achieves remarkable results in segmenting normal eyes, and demonstrates competitive performance in areas prone to common errors. In addition, its advantages are also reflected in quantitative indicators such as ACC and specificity.

[0074] Table 3: Algorithm comparison experiment on private coronary artery dataset

[0075]

[0076] For qualitative visualization, Figure 4 Some samples of MamVeSeg segmentation results are shown. MamVeSeg has a higher sensitivity for segmenting thick vessels with descriptive features (such as color) while preserving the anatomical structure. However, its segmentation ability is relatively poor for thin and blurred vessels with fewer pixels. In order to better display the details of the segmentation results, the present invention also plots the segmentation results of the extracted patches on Figure 5 The experimental results show that compared with U-Net and TransUNet, the proposed model accurately segments fine and blurred blood vessels while maintaining the geometric continuity of retinal blood vessels.

[0077] The MamVeSeg of the present invention is compared with U-Net, UNet++, TransUNet, SwinUNet, and SwinUMamba in terms of model complexity and computational efficiency, and the results are shown in Table 4.

[0078] Table 4: Comparison of model complexity

[0079]

[0080] The comparison results in Table 4 show that the MamVeSeg model of the present invention achieves a good balance between model complexity and computational efficiency. Specifically, MamVeSeg shows comparable parameter counts to U-Net and SwinUNamba, and achieves a reduction in FLOPs relative to TransUNet and SwinUNamba.

Claims

1. A method for constructing a multi-task blood vessel segmentation model based on a hybrid encoder is constructed based on an encoder-decoder U-shaped structure network, characterized in that: The encoder is a Swin Transformer path multi-layer encoder based on the Mamformer module. The Mamformer module divides the input image block into two information streams after layer normalization. One group of information streams passes through the linear layer, 3×3 depth convolution layer, SS2D block, and normalization layer in sequence, and is added to the other group of information streams that pass through the linear layer and W-MSA in sequence. The merged information stream passes through the linear layer and the input jump connection to obtain the final output of the Mamformer module. The decoder is a multi-task decoder, which consists of three decoding paths, which are used for blood vessel segmentation, contour edge segmentation and blood vessel connectivity segmentation respectively. Each decoding path uses Swin Transformer block and Patch Expanding as building blocks to generate feature maps with the same size as the input image. A shared attention mechanism is set between the Swin Transformer Blocks of the three decoding paths, and finally the outputs of the three decoding paths are spliced ​​along the channel dimension. The loss function expression of the vascular connectivity decoding path is as follows: L v =1-softclDice Among them, L v represents the vascular connectivity loss function, softclDice represents the use of continuous probability values ​​to calculate clDice, and clDice represents the clDice coefficient. The specific expression is as follows: Among them, V P represents the truth mask, V L represents the predicted segmentation mask, S P and S L Respectively represent from V P , V L Extract skeleton, Tprec(S P , V L ) indicates S P , V L The topological accuracy between L , V P ) indicates S L , V P The topological sensitivity between them is expressed as follows: Among them, |S P ∩V L | indicates S P With V L The number of elements in the intersection, |S P | indicates S P The number of elements of |S L ∩V P | indicates S L With V P The number of elements in the intersection, |S L | indicates S L The number of elements of ; A self-distillation loss function is constructed to adjust the segmentation consistency between the three independent decoding paths. The expression of the self-distillation loss function is as follows: L sd =1-Says(sobel(and n ),and e ) Among them, L sd represents the self-distillation loss function, and Dice represents sobel(y n ) and y e Dice loss between them, sobel(yn) represents the vascular branch prediction y n Sobel operation, y e Represents the vessel contour edge predicted by the decoded path.

2. The method for constructing a multi-task blood vessel segmentation model based on a hybrid encoder according to claim 1, characterized in that The SS2D block includes a scan expansion operation, an S6 block operation, a scan merging operation, and a window attention mechanism at the skip connection.

3. A method for constructing a multi-task blood vessel segmentation model based on a hybrid encoder according to claim 1 or 2, characterized in that The overall loss function L of the decoder is total The expression is as follows: L total =y n *L n +y e *L e +y v *L v +y sd *L sd Among them, L total Represents the overall loss function of the decoder, y n ,y e ,y v ,y sd Respectively represent L n , L e , L v , L sd Corresponding weight, L n represents the semantic vessel mask loss function, L e represents the edge loss function of the blood vessel contour, L v represents the vascular connectivity loss function, L sd represents the self-distillation loss function.

Images