Medical image segmentation model and method based on efficient multi-scale feature self-attention decoder and enhanced jump connection

Abstract

The invention discloses a medical image segmentation model and method based on an efficient multi-scale feature self-attention decoder and enhanced jump connection, a lightweight and efficient multi-scale feature self-attention module is constructed, the module adopts bottleneck convolution in the up and down sampling process of an image, and the image segmentation efficiency is improved. And two self-attention mechanisms are executed in sequence between the two processes. A linear deformable attention guidance module is introduced to enhance feature maps passed through jump connections. An EMFSD decoder formed by combining the two is combined with a pyramid vision converter, and finally a medical image segmentation model (PVT-EMFSD) is formed. The model adopts a hierarchical structure, multi-level features are utilized from a hierarchical visual encoder, and semantic information of a feature map is gradually enhanced by using a U-shaped network architecture. A wide range of experiments on three reference data sets show that the method is superior to an existing method in a medical image segmentation task, and the competitive performance is shown.

Description

Technical Field

[0001] The present invention belongs to the technical field of high-precision medical image segmentation, and specifically relates to a medical image segmentation model and method based on an efficient multi-scale feature self-attention decoder and enhanced skip connections. Background Art

[0002] Image segmentation has been widely used in many fields, especially in medical imaging. As a key foundation for building medical systems, medical image segmentation plays an important role in disease diagnosis and treatment planning. By classifying each pixel in the image, medical image segmentation can effectively identify and accurately locate lesions, tumors, organ structures, and other key areas. This provides doctors with detailed anatomical information to help with disease staging, lesion boundary delineation, and surgical planning. Therefore, generating high-quality segmentation maps is key to promoting the advancement of medical imaging technology and enhancing its application in clinical practice.

[0003] With the rapid development of deep learning technology, medical image segmentation has gradually shifted from traditional manual feature extraction to automatic feature learning. As a representative deep learning method, convolutional neural network (CNN) effectively realizes multi-level feature representation of images through its hierarchical structure, and can automatically capture subtle features and semantic information in images. This feature makes CNN widely used in medical image segmentation tasks, especially the model architecture represented by U-type network, such as UNet [1] It performs well in medical image segmentation, mainly due to its use of skip connections to generate high-resolution segmentation maps and aggregate multi-scale features.

[10] Based on the traditional U-shaped architecture, UNet3+ combines novel designs such as deep supervision and dense skip connections. [4] The introduction of multi-path structure and guided feature learning significantly improves the performance of medical image segmentation. In addition, considering the effectiveness of attention mechanisms in various fields, they are also introduced into the U-network to enhance the segmentation performance. [5] By integrating the self-attention mechanism into the traditional UNet, the challenges of capturing fine features and long-distance dependencies in medical image segmentation are solved. [6] Use adaptive attention to focus on the target area and reduce background noise. PraNet [7] Applying reverse attention gradually focuses on the boundaries, while BRAU-Net++ [8] Adding boundary perception and refined attention modules to U-Net improves the segmentation effect of edges and key areas. [9]Neighborhood fuzzy C-means, multi-scale pyramids, and hybrid attention are combined to address the problem of blurred edges. Although these models have improved performance and pixel-level classification, the high computational cost of the convolutional module makes the model design complex and redundant, limiting its feasibility in practical applications, especially in resource-constrained environments such as mobile devices and edge computing.

[0004] In recent years, visual transformers (ViT)

[10] Due to its excellent ability to capture long-distance dependencies between pixels, it has been widely used in the field of computer vision. In order to more comprehensively capture multi-scale information and enhance the hierarchical representation of features, researchers have developed a hierarchical visual transformer based on ViT. Representative models include Swin

[11] 、PVT [12,13] MERIT

[14] and MetaFormer

[15] These models achieve multi-scale feature extraction through hierarchical structures, significantly enhancing the ability to retain image details. Although the self-attention mechanism has unique advantages in capturing global context, its modeling of local spatial information is relatively weak.

[16] To solve this problem, CASCADE

[17] Researchers introduced local convolutional attention in the decoding stage. Although this method performs well, it also significantly increases the computational resource overhead, making lightweight implementation challenging. In addition, the simplified design of the decoding stage aimed at reducing costs also limits the potential for further performance improvements. Therefore, finding a balance between resource consumption and performance remains an urgent problem to be solved. Summary of the invention

[0005] Traditional high-precision medical image segmentation methods usually require a lot of computing resources. Therefore, it is crucial to develop an efficient and lightweight decoding mechanism. However, the accuracy of medical image segmentation is often limited by the simplified decoder design. How to optimize the decoder architecture while reducing computing resources and maintaining high model accuracy remains a problem that needs to be solved. In response to the above problems, the present invention proposes a medical image segmentation model (PVT-EMFSD model) and method based on an efficient multi-scale feature self-attention decoder and enhanced jump connections, which is specially designed for medical image segmentation, aiming to balance resource consumption and performance, and effectively improve the accuracy and efficiency of segmentation.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is:

[0007] A medical image segmentation model based on an efficient multi-scale feature self-attention decoder and enhanced skip connections is proposed. Firstly, a lightweight and efficient multi-scale feature self-attention module (MFSM) is constructed, which adopts bottleneck convolution in the up-sampling and down-sampling processes of the image respectively, and executes two self-attention mechanisms in sequence between the two processes; secondly, a linear deformable attention guided module (LDAGM) is introduced to enhance the feature map transmitted through the skip connection; finally, the EMFSD decoder is formed by combining MFSM and LDAGM; the EMFSD decoder is combined with the pyramid visual transformer (PVT) to form a medical image segmentation model (PVT-EMFSD), which adopts a hierarchical structure, utilizes multi-level features from the hierarchical visual encoder, and uses the U-shaped network architecture to gradually enhance the semantic information of the feature map.

[0008] The present invention also proposes a medical image segmentation method based on the model, the steps are as follows:

[0009] ①Data collection and preprocessing

[0010] A diagnostic image dataset is obtained, and all images will be uniformly adjusted to a resolution of 224×224 pixels. At the same time, in order to increase data diversity and improve the generalization ability of the model, data enhancement techniques including random rotation and flipping will be applied. These image enhancement operations can effectively expand the training dataset, reduce overfitting, and enhance the robustness of the model in practical applications.

[0011] ②Medical image model architecture

[0012] Medical images are segmented through a lightweight 2D medical image segmentation method with an efficient multi-scale feature self-attention decoder and enhanced skip connections. The model takes a medical image as input, extracts the feature information of the medical image through the encoder, and then restores the feature map to the size of the original image through the decoder, finally generating the classification result of each pixel and outputting an accurate segmentation result map.

[0013] ③Model training and evaluation

[0014] First, the model is trained on the training set to obtain the model weight, and the performance of the model in the segmentation task is evaluated on the validation set; then, the model is finally evaluated by using an unseen test set to ensure that the model has good generalization ability; in this process, this method uses the average Dice coefficient as the main evaluation indicator, and selects the model weight with the best performance on the test set as the final best model; the Dice coefficient is used to measure the overlap between the predicted segmentation result and the true annotation. The higher the value, the better the model segmentation effect. Especially when facing small targets or class imbalance problems, the Dice coefficient can effectively reflect the segmentation accuracy of the model;

[0015] ④Application and reasoning

[0016] Once the model is trained and evaluated, it can be applied to actual medical image segmentation tasks. By inputting new images into the model, the model will output the predicted category of each pixel and generate segmentation results.

[0017] The reasoning process usually includes:

[0018] Image input: input new medical images into the trained model;

[0019] Post-processing: Processing the output of the model, which may include smoothing, thresholding, and removing small areas;

[0020] Visualization: Compare the segmentation results with the original image to show the effect of the model segmentation;

[0021] ⑤Model optimization and tuning

[0022] Increase the dataset: Expand the dataset by obtaining more annotated images or performing data augmentation to improve the generalization ability of the model;

[0023] Improve network architecture: Depending on the complexity of the task, try different network architectures, such as deeper U-nets or more complex convolutional neural networks, to further improve segmentation accuracy.

[0024] Compared with the existing solutions, the beneficial effects of the present invention are:

[0025] 1. The medical image segmentation model (PVT-EMFSD model) proposed in the present invention adopts two unique multi-scale feature attention mechanisms in the multi-scale feature self-attention module, which significantly enhances the representation ability of the feature map. The decoder also introduces a linear deformable convolution attention guidance mechanism to enhance the feature map from the jump connection. This mechanism effectively overcomes the limitations of traditional convolution and better captures the complex semantic relationships in the feature map. By combining the self-attention mechanism with the linear deformable convolution, the PVT-EMFSD model can focus on important areas while showing the advantage of lightweight.

[0026] 2. This paper proposes a new efficient multi-scale feature self-attention module (MFSM), which highlights multi-scale features by suppressing irrelevant areas, while meeting the requirements of lightweight and efficient performance. A novel linear deformable attention guided module (LDAGM) is developed, which enhances feature representation by combining fine-grained features with jump connection features, significantly improves the expressiveness of the model, and improves the flexibility between performance and computational cost. An efficient multi-scale self-attention decoder (EMFSD) is proposed, which can generate more accurate medical image segmentation results at a lower computational cost.

[0027] 3. Experimental results show that the PVT-EMFSD model proposed in this paper significantly improves the performance in 2D medical image segmentation tasks, outperforming the existing state-of-the-art decoders in terms of performance indicators. Compared with the CASCADE decoder, the FLOPs are reduced by 77.36% and the number of parameters is reduced by 81.66%. To demonstrate the effectiveness of this method, extensive experiments were conducted on three benchmark datasets, namely the Synapse multi-organ dataset, the ACDC dataset, and the ISIC2018 dataset. These experiments show that this method outperforms existing methods in medical image segmentation tasks and demonstrates competitive performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is an overview of the PVT-EMFSD model proposed in this paper. X1, X2, X3 and X4 are the feature maps generated by the four stages of the hierarchical encoder. p1, p2, p3 and p4 are the output segmentation maps of the four stages of the decoder. By applying the additive aggregation method to them, the final segmentation result is generated.

[0029] Figure 2 It is the Linear Deformable Attention Guided Module (LDAGM)

[0030] Figure 3 It is a multi-scale feature self-attention module (MFSM)

[0031] Figure 4 Qualitative results on the Synapse multi-organ dataset. (a) Ground Truth (GT), (b) TransUNet, (c) SwinUNet, (d) PVT-CASCDE, (e) TransCASCADE, and (f) PVT-EMFSD.

[0032] Figure 5 Visualization of feature maps before and after introducing LDAGM into the model.

[0033] Figure 6 Visualization of feature maps before and after introducing MFSM into the model. DETAILED DESCRIPTION

[0034] Example 1

[0035] First, the present invention constructs a lightweight and efficient multi-scale feature self-attention module (MFSM). The module adopts bottleneck convolution in the up-sampling and down-sampling processes of the image, and executes two self-attention mechanisms in sequence between the two processes, effectively capturing the detail information from the channel and space, while reducing the computational cost. Secondly, a linear deformable attention guidance module (LDAGM) is introduced to enhance the feature map transmitted through the jump connection. The linear deformable convolution in LDAGM has adjustable parameters and the ability to sample arbitrary shapes, which can effectively capture complex semantic relationships, thereby significantly improving the model performance. Finally, the EMFSD decoder formed by combining MFSM and LDAGM. The EMFSD decoder is combined with a pyramid visual transformer (PVT) to form a PVT-EMFSD model, which adopts a hierarchical structure, utilizes multi-level features from the hierarchical visual encoder, and uses the U-shaped network architecture to gradually enhance the semantic information of the feature map. The following is a detailed description of its construction:

[0036] 1 Overall Architecture

[0037] The PVT-EMFSD model architecture is as follows Figure 1 As shown in the figure, the EMFSD decoder is combined with the PVTv2-B2 encoder based on the standard PVTv2 network. This design achieves model flexibility and efficiency under different computing resources. First, the PVTv2-B2 encoder extracts hierarchical feature maps (X1, X2, X3, and X4) through convolution operations, which effectively captures consistent spatial information and overcomes the limitations of traditional transformer models. These hierarchical feature maps are then fed into the EMFSD decoder, where the LDAGM refines the feature maps through skip connections, and the MFSM module flexibly adjusts the channel reduction rate to effectively control the parameters and computational load while enhancing and optimizing the feature maps. Next, the decoder generates four high-precision segmentation maps through the segmentation head (SH) module, which correspond to each stage of the encoder. Finally, an additive aggregation strategy is used to assemble these segmentation maps into the final segmentation result.

[0038] 2. Efficient Multi-Scale Feature Self-Attention Decoder (EMFSD)

[0039] exist Figure 1 In the present invention, the multi-stage features output by the decoder contain rich global information; however, the local feature information is still insufficient. Therefore, the present invention proposes a novel EMFSD decoder, which enhances the extraction of local information by processing multi-stage features, thereby achieving efficient medical image segmentation capabilities.

[0040] The EMFSD decoder consists of linear deformable attention guided modules (LDAGMs), efficient multi-scale feature self-attention modules (MFSMs), upsampling modules with deep convolution (DWUCBs), and segmentation heads (SHs). MFSMs are used to capture global context features, multi-scale channels, and spatial dependencies; LDAGMs optimize feature maps fused with skip connections by guiding the attention mechanism; DWUCBs are responsible for upsampling feature maps while further enhancing feature details. SHs generate the final segmentation map for each stage.

[0041] In the EMFSD decoder, the present invention gradually utilizes four MFSMs to optimize the four stages of the encoder (i.e. Figure 1 The pyramid feature map extracted by the decoder (X1, X2, X3, X4). DWUCB receives the optimized feature map and performs fine-tuning to enhance the detail performance; SH generates the segmentation map of this stage based on these optimized feature maps. LDAGM further enhances the features of the jump connection by integrating the output from DWUCB and the jump connection feature map. After obtaining four different segmentation maps, the present invention adopts an additive aggregation strategy to fuse them and generate the final segmentation result. Next, each module in the decoder will be introduced in detail:

[0042] 2.1. Linear Deformable Attention Guided Module (LDAGM)

[0043] In traditional methods, such as Attention UNet [6] and CASCADE

[17] , using a k×k convolution structure to process the guidance signal s from the skip connection feature map and the input feature map x (upsampled features). Although these methods effectively enhance the feature map, the inherent limitations of k×k convolutions - due to their local window and fixed sampling shape - make it difficult to capture information from other locations. In addition, the fixed k×k structure of the convolution kernel leads to an increase in the number of parameters, which grows with the square of the kernel size. This lack of flexibility poses challenges in handling diverse datasets and target shapes.

[0044] To solve these problems, this paper introduces a novel linear deformable attention guided module (LDAGM). LDAGM uses linear deformable convolution

[18] , providing an arbitrary number of parameters and flexible sampling capabilities, thereby enhancing the flexibility and optimization potential of the model while maintaining network performance. During training, LDAGM learns to assign higher attention coefficients to relevant features and lower coefficients to irrelevant features, ultimately enhancing feature maps and suppressing irrelevant information. Specifically, LDAGM uses guidance signals extracted from high-level features to control feature maps transmitted through different jump connections, effectively improving the accuracy of medical image segmentation.

[0045] In LDAGM, the attention function Att(·) is first used. The linear deformable convolution LDC is used respectively. s (·) and LDC x (·) processes s and x and merges them by element-wise addition. Subsequently, a 1×1 convolution C(·) is applied to obtain a single-channel feature map and normalized by a batch normalization layer BN(·). Finally, after applying the Sigmoid activation function σ(·), the attention coefficient is obtained. The resulting attention coefficient is multiplied element-wise with the input feature x To enhance relevant features and suppress irrelevant features, thus generating the attention-guiding feature LDAGM(s,x). Figure 2 ) can be expressed by equation (1) and equation (2):

[0046] Att(s,x)=LDC s (s)+LDC x (x) (1)

[0047]

[0048] Due to the use of variable kernel convolution in Att(·), LDAGM can capture effective spatial context information at different computational costs, laying a solid foundation for the flexible application of subsequent models.

[0049] 2.2. Multi-scale Feature Self-Attention Module (MFSM)

[0050] In models based on the U-shaped architecture, it is usually necessary to perform feature enhancement processing on the feature maps generated at each stage before sending them to subsequent operations. Attention mechanisms are widely used in the feature map enhancement process and can usually significantly improve performance. However, effective attention mechanisms are often accompanied by a large amount of computing resource consumption. Therefore, designing an efficient feature map enhancement module is particularly important and has become a problem that needs to be solved urgently.

[0051] To solve this problem, the present invention proposes an efficient multi-scale feature self-attention module (MFSM). The MFSM consists of two bottleneck convolutions (Bottle Conv1(·) and Bottle Conv2(·))) and two multi-scale self-attention modules. In the design of the present invention, the PMFSNet

[23] The idea is to use multi-scale channel self-attention

[23] (MCS(·)) to enhance the representation of feature maps in the channel dimension, and multi-scale spatial self-attention

[23] (MSS(·)) focuses on enhancing the representation in the spatial dimension. The two work together to optimize the representation of the input feature map. For convenience, the processing of feature maps by MCS(·) and MSS(·) is combined into multi-scale feature self-attention (MFS(·)). Bottle Conv1(·) and Bottle Conv2(·) have similar structures. First, batch normalization BN(·) is applied to the feature map, and then the activation function ReLU(·) is used to activate the feature map. Next, a 3×3 depth convolution DWConv is used. 3×3 (·) to enhance the feature map. Finally, a 1×1 standard convolution Conv is applied 1×1 (·) to adjust the number of channels in the feature map. The bottleneck convolution changes the number of channels in the feature map by adjusting the channel compression ratio, thereby achieving flexibility in model scale to adapt to different application environments, while the two multi-scale self-attention modules enhance feature representation in channel and spatial dimensions. The expressions of Bottle Conv1(·), Bottle Conv2(·), MFS(·), and MFSM(·) (see Figure 3 ) are given in equations (3), (4), (5), and (6) respectively:

[0052] Bottle Conv1(x)=Conv 1×1 (DWConv 3×3 (ReLU(BN(x)))) (3)

[0053] Bottle Conv2(x)=Conv 1×1 (DWConv 3×3 (ReLU(BN(x)))) (4)

[0054]

[0055] MFSM(x)=Bottle Conv2(MFS(Bottle Conv1(x))) (6)

[0056] Here Represents an element-wise multiplication operation. By combining bottleneck convolution to reduce the number of channels of the intermediate feature map and multi-scale self-attention modules to enhance the feature map, MFSM surpasses the literature in terms of performance and flexibility.

[17] The convolutional attention module (CAM) proposed in

[15] significantly reduces the computational cost.

[0057] 2.3. Depth-based Convolution Upsampling Convolution Block (DWUCB)

[0058] In the EMFSD decoder, DWUCB uses CASCADE

[17] The purpose of the design is to upsample the feature map of the current stage to ensure that the number of channels and resolution match the feature map in the next jump connection. The calculation formula of DWUCB(·) is shown in formula (7):

[0059] DWUCB(x)=Conv 1×1 (ReLU(BN(DWConv 3×3 (Up 2× (x))))) (7)

[0060] Among them, DWConv 3×3 (·) represents the depth convolution of 3×3 convolution kernel, Conv 1×1 (·) denotes point-wise convolution. BN(·) denotes batch normalization, and ReLU(·) refers to the activation function used.

[0061] 2.4. Split Header (SH)

[0062] In our model, SH is used to generate the final segmentation map from the four stages of the decoder. It uses a 1×1 convolution Conv 1×1 (·) adjusts the number of channels in the feature map. For multi-category datasets, the number of output channels is equal to the number of categories; for binary classification tasks, there is only one output channel. The calculation formula of SH(·) is shown in formula (8):

[0063] SH(x)=Conv 1×1 (x) (8)

[0064] 3 Multi-stage output and loss aggregation

[0065] In the EMFSD decoder of the present invention, a common additive aggregation method is used to combine the four segmentation results p1, p2, p3 and p4 generated by the decoder in four different stages to generate the final segmentation output, as shown in formula (9):

[0066] seg output =αp1+βp2+γp3+ζp4 (9)

[0067] In this case, α, β, γ, and ζ are weights assigned to each prediction head, all set to 1.0. For binary segmentation tasks, the Sigmoid function is used to activate the feature map and generate the final output; for multi-class segmentation tasks, the Softmax function is used to activate the feature map to obtain the final result.

[0068] In our model, we use a joint loss aggregation strategy called mutual task aggregation, just as MERIT

[14] As proposed in .

[0069] Example 2

[0070] Experimental Results and Analysis: Comparative experimental results are obtained on three medical image segmentation datasets, and the results of various ablation experiments are shown.

[0071] In this section, the implementation is described in detail and a comprehensive comparative analysis of the proposed PVT-EMFSD model with a range of state-of-the-art methods is performed. The purpose of this comparison is to demonstrate that the proposed PVT-EMFSD model achieves the best balance between accuracy and computational efficiency. In addition, an ablation study of different modules in the EMFSD decoder and other related aspects is performed to gain a deeper understanding of their respective contributions.

[0072] 1. Dataset

[0073] This paper introduces three datasets used in experiments: Synapse multi-organ dataset, ACDC dataset and ISIC2018 dataset.

[0074] Synapse Multi-Organ Dataset 1 The Synapse multi-organ dataset is a comprehensive medical image segmentation dataset, including CT and MRI images. The dataset contains a total of 30 abdominal CT scans with a total of 3779 axial enhanced slices. Each CT scan consists of 85-198 slices with a resolution of 512×512 pixels. In order to analyze this dataset, this paper uses the TransUNet research [2] A similar approach was used in

[17] . Specifically, the dataset was randomly divided into two different subsets: 18 training scans (including 2,212 axial slices) and 12 validation scans. The main goal of this segmentation task is to accurately delineate eight abdominal organs, namely, aorta (Aorta), gallbladder (GB), left kidney (KL), right kidney (KR), liver, pancreas (PC), spleen (SP), and stomach (SM).

[0075] ACDC Dataset 2 The ACDC challenge collected MRI scans from different patients. The ACDC dataset 2 contains 100 cardiac MRI scan images, each of which consists of three organs: the right ventricle (RV), myocardium (Myo), and left ventricle (LV). [2] The processing method used 70 cases (including 1930 axial slices) for training, 10 cases for validation, and 20 cases for testing.

[0076] ISIC2018 dataset. ISIC2018

[24] The dataset is a dataset for skin disease segmentation, containing 2596 images and corresponding true segmentation masks. In the experiment, the images were resized to 384×384 resolution and then input into the model for segmentation. The data was randomly divided into 80% for training, 10% for validation, and 10% for testing.

[0077] 2. Evaluation Metrics

[0078] For the Synapse multi-organ dataset, three metrics are used to evaluate model performance: DICE, mIoU, and 95% Hausdorff distance (HD95). For the ACDC dataset, only the DICE score is used as the evaluation metric. The ISIC2018 dataset is evaluated using DICE and mIoU. mIoU is obtained by calculating the intersection-over-union ratio of each category and then averaging the intersection-over-union ratios of all categories. DICE score Intersection and Union Distance from HD95 The calculation formulas for are given in equations (10), (11) and (12) respectively:

[0080]

[0081] Among them, Y and They are the true segmentation mask and the predicted segmentation map, respectively.

[0082] 3. Experimental Details

[0083] The network was implemented and experiments were performed using Pytorch 1.11.0. All models were trained on a single GeForce RTX3090 GPU. The PVT-EMFSD model of the present invention uses PVTv2-B2 as the encoder, using weights pre-trained on ImageNet. During training, the AdamW optimizer was used.

[25] , the learning rate and weight decay are both set to 0.0001. For the Synapse multi-organ dataset, the training is set to 300 epochs with a batch size of 6; for the ACDC cardiac organ dataset, it is set to 400 epochs with a batch size of 12; for the ISIC18 skin lesion segmentation dataset, the batch size is 16 and the training epoch is 200. During the training process, the best model of all datasets is saved based on the average DICE score. It is worth noting that for the ISIC18 dataset, the images are resized to 352×352 pixels and gradient clipping (limited to 0.5) is used, while a multi-scale training strategy containing {0.75, 1.0, 1.25} scales is adopted; for the Synapse and ACDC datasets, the images are resized to 224×224 pixels, and data augmentation with random rotation and flipping is applied, while optimizing the combination of cross entropy loss (weighted by 0.3) and DICE loss (weighted by 0.7). In the binary segmentation task, weighted binary cross entropy (BCE) and weighted IoU loss functions are used.

[0084] 4. Experimental Results

[0085] The proposed PVT-EMFSD model is compared with some classic state-of-the-art segmentation models based on CNN and Transformer, and the test datasets used include Synapse multi-organ dataset, ACDC heart organ dataset and ISIC2018 skin lesion segmentation dataset.

[0086] 4.1. Quantitative results of Synapse multi-organ dataset

[0087] Table 1 shows the performance of various CNN-based and transformer-based methods on the Synapse multi-organ segmentation dataset. The average DICE scores of various organs, including aorta, gallbladder (GB), left kidney (KL), right kidney (KR), liver, pancreas (PC), spleen (SP), and stomach (SM), are reported in Table 1. The upward arrow (↑) indicates that higher values ​​are better, while the downward arrow (↓) indicates that lower values ​​are better. The best results are highlighted in bold.

[0088] The proposed PVT-EMFSD model significantly outperforms all state-of-the-art (SOTA) methods in CNN-based and transformer-based 2D medical image segmentation, achieving the best average DICE score of 83.60%.

[17] In comparison, the average DICE score of the proposed PVT-EMFSD model is improved by 2.54%. At the same time, by using bottleneck convolutions in the intermediate processing of MFSM to reduce the number of channels of the feature map, the computational cost is significantly reduced - specifically, FLOPs are reduced by 77.36% and the number of parameters is reduced by 81.66%. In terms of the HD95 distance metric, the proposed PVT-EMFSD model also performs well, reducing 1.84 compared to PVT-CASCADE, indicating that the model of the present invention has higher accuracy in localizing organ boundaries. The lower HD95 score means that the PVT-EMFSD model can provide more accurate boundary detection when dealing with complex anatomical structures, thereby reducing the occurrence of false positives and false negatives. In addition, the PVT-EMFSD model also achieved the highest mIoU score of 74.43%.

[0089] The average DICE scores of individual organ segmentation using the PVT-EMFSD model are also shown. As shown in Table 1, four of the eight organs perform significantly better than the SOTA method, especially in the gallbladder and stomach segmentation tasks, where the PVT-EMFSD model performs well.

[0090] Based on the experimental results, it is concluded that the proposed PVT-EMFSD model achieves higher performance at a lower computational cost. This is due to the fact that MFSM enhances the feature map in both channel and spatial dimensions. In addition, LDAGM further improves the effectiveness of the skip connection feature map while suppressing irrelevant information.

[0091] Table 1. Quantitative results of Synapse multi-organ dataset.

[0092]

[0093] 4.2. Quantitative Results on the ACDC Dataset

[0094] Further experiments are conducted on the ACDC dataset for magnetic resonance imaging using the PVT-EMFSD model. Table 2 shows the performance of various CNN-based and Transformer-based methods on this dataset. In Table 2, the DICE scores for each organ are reported, and the best results are highlighted in bold.

[0095] As can be seen, the PVT-EMFSD model achieves the highest average DICE score of 91.80%, which is 0.34% higher than the previous best lightweight method PVT-CASCADE, while having lower computational cost. Compared with the previous best method TransCASCADE, the PVT-EMFSD model still exceeds it by 0.17%. This slight improvement not only reflects the accuracy advantage of the model, but also demonstrates its effectiveness in fine-grained segmentation tasks. In addition, it is worth noting that the decoder of the PVT-EMFSD model significantly reduces the computational cost compared to the PVT-CASCADE method (see Table 9), making the PVT-EMFSD model more suitable for clinical applications. Furthermore, in the segmentation task of specific organs, the PVT-EMFSD model achieved the best average DICE scores for three key organs, which were 90.36%, 89.25% and 95.79%, respectively, demonstrating its excellent ability to handle complex anatomical structures. These results not only verify the effectiveness of the model, but also highlight its potential value in practical medical image analysis.

[0096] Table 2. Quantitative results on the ACDC dataset.

[0097]

[0098] Quantitative results on the ISIC2018 dataset

[0099] Table 3 lists the average DICE scores and mIoU scores of the PVT-EMFSD model of the present invention and other methods on the ISIC2018 dataset, and the best results are highlighted in bold.

[0100] As shown in Table 3, the PVT-EMFSD model performs exceptionally well, achieving the highest average DICE score of 91.55% and mIoU score of 85.59%. Specifically, the average DICE score of the PVT-EMFSD model is 0.45% higher than that of the PVT-CASCADE method, while the mIoU metric is improved by 0.6%.

[0101] Table 3. Quantitative results on the ISIC2018 dataset.

[0102]

[0103] 4.4. Visualization results of Synapse multi-organ dataset

[0104] Figure 4 The segmentation results of different methods including the PVT-EMFSD model on two sample images are shown. Figure 4 In Figure 2, the segmentation map is overlaid on the original image / slice. White borders are used to highlight areas where most methods predict incorrectly.

[0105] By observing the highlighted areas in the figure, it is obvious that the PVT-EMFSD model shows higher accuracy in organ segmentation than other models while significantly reducing the computational cost. This superior performance is mainly attributed to the multi-scale channel and spatial self-attention modules adopted by the PVT-EMFSD model, which enhances the feature representation in the channel and spatial dimensions. In addition, the use of linear deformable convolution technology enables the model to effectively capture complex semantic relationships, thereby improving the processing of boundaries and details. This design not only improves segmentation accuracy, but also effectively reduces false positives and false negatives, thereby further improving the reliability of diagnosis.

[0106] 5. Ablation Experiment

[0107] In this section, a series of ablation experiments are performed to further study various aspects related to the proposed architecture and experimental setup, such as the impact of different components in the EMFSD decoder, the impact of different channel compression ratios and convolution kernel sizes in LDAGM, and the impact of input resolution.

[0108] 5.1. Effects of different components in EMFSD

[0109] Ablation studies are performed on the Synapse multi-organ dataset to evaluate the effectiveness of different components in the proposed PVT-EMFSD model. All experiments use the same PVTv2-B2 backbone network pre-trained on ImageNet and apply consistent experimental treatments on the Synapse multi-organ dataset. During the experiments, different components of the PVT-EMFSD model are gradually removed and their respective results are compared.

[0110] In Table 4, an additive pooling method is used to combine skip connections, and these results are obtained using an input resolution of 224×224 pixels. The best results are highlighted in bold. As shown in Table 4, the cascade design of the decoder significantly improves the model performance compared to the non-cascaded structure, with an average DICE score increase of 1%. In addition, the introduction of the LDAGM and MFSM modules also has a positive impact on the model performance, improving the average DICE score by 2.31% and 0.14%, respectively. When the two modules are used simultaneously, the model of the present invention achieves the best average DICE score of 83.60%, which is 2.6% higher than when no module is introduced. This shows that the LDAGM and MFSM modules play a vital role in feature fusion and information transmission, thereby enhancing the recognition ability of the model. In addition, Table 4 also shows that with the increase of 0.57G FLOPs and 1.7M parameters, the average DICE score of the PVT-EMFSD model of the present invention is improved by about 3.5%.

[0111] Table 4. Quantitative results of PVT-EMFSD of different components of the PVTv2-B2 encoder on the Synapse multi-organ dataset.

[0112]

[0113] In order to more intuitively demonstrate the importance and role of LDAGM and MFSM in the model, the feature visualization results before and after the introduction of these two modules are compared. Figure 5 and Figure 6 Specifically, Skip1, Skip2, and Skip3 show the comparison of feature maps obtained from shallow to deep layers through skip connections before and after LDAGM processing. Meanwhile, MFSM1, MFSM2, MFSM3, and MFSM4 show the comparison of feature maps aggregated from different resolutions before and after MFSM processing, covering features from shallow to deep layers. The feature maps of all channels were averaged and heatmaps were generated using OpenCV-Python. Figure 5 and Figure 6 It is clearly shown that after the introduction of LDAGM and MFSM, the model's capabilities in pixel classification, recognition, and enhancement are significantly improved.

[0114] 5.2. Impact of the order of MCS and MSS in MFSM and their serial and parallel configuration

[0115] Ablation experiments were conducted to mainly explore the effects of the order of arrangement of MCS and MSS in MFSM and their series and parallel configurations on model performance. The experimental results are detailed in Tables 5 and 6. Both tables use PVTv2-B2 as the encoder to generate results. The best results are highlighted in bold. As shown in Table 5, the arrangement of MCS first and MSS last is better than the arrangement of MSS first and MCS last, and the average DICE score of the former is improved by 0.94%. This shows that the MCS module plays a key role in feature extraction and provides richer information for the subsequent MSS module, thereby improving the overall performance. In Table 6, the series and parallel configurations of MCS and MSS are compared. The results show that the series configuration is significantly better than the parallel configuration, and the average DICE score of the former is improved by 1.13%. This finding shows that in the process of information processing, the series structure can more effectively transmit and integrate features, thereby providing a richer feature representation for the model. Therefore, in the PVT-EMFSD model of the present invention, the series architecture design with MCS first and MSS last is selected.

[0116] Table 5. Comparison of different arrangements of MCS and MSS in MFSM on the Synapse multi-organ dataset.

[0117]

[0118] Table 6. Comparison of segmentation results using serial and parallel configurations in MFSM on the Synapse multi-organ dataset.

[0119]

[0120] 5.3. Effect of different channel compression ratios in MFSD

[0121] In the MFSM module of the present invention, bottleneck convolution is used to effectively reduce the number of channels. Table 7 shows the experimental results of the PVT-EMFSD model at different channel compression ratios. These results are generated using PVTv2-B2 as the encoder, and the best results are highlighted in bold. As can be seen from Table 7, when the channel compression ratio in MFSM is set to 2, the model achieves the best performance with an average DICE score of 83.60%. It is worth noting that when the channel compression ratio is increased to 4, the average DICE score only drops by 0.4%, while the number of parameters in the decoder is reduced by about 40%. When the channel compression ratio is further increased to 8, the average DICE score drops by 1.69%, and the number of parameters in the decoder is reduced by about 53%, while still maintaining good performance.

[0122] Table 7. Comparison of the effects of different channel compression ratios on the Synapse multi-organ dataset in MFSM.

[0123]

[0124] 5.4. Effects of Different Channel Compression Ratios and Kernel Sizes in LDAGM

[0125] Table 8 shows the performance of LDAGM under different channel compression ratios and convolution kernel sizes in the PVT-EMFSD model proposed in this paper. In Table 8, PVTv2-B2 is used as the encoder to generate these results, and the best results are highlighted in bold. It should be emphasized that the combinations presented in Table 8 are some options selected after considering the number of model parameters and FLOPs. Specifically, the shrinkage ratio is set to [4,8,16] and the convolution kernel size is [5,7,7], which means that the channel compression ratio of the intermediate channel of LDAGM is 4, 8 and 16 (from deep to shallow layers), and the convolution kernel size of each stage is 5, 7 and 7, respectively. It can be seen from Table 8 that when LDAGM uses the channel compression ratio configured as [4,4,4] and the convolution kernel size configured as [5,5,5], the model achieves the best performance with an average DICE score of 83.60%. Further analysis shows that the combination of channel compression ratio and convolution kernel size is crucial to model performance. Specifically, the appropriate channel compression ratio setting can effectively reduce the complexity of the model, while the appropriate convolution kernel size helps capture richer feature information. Therefore, this configuration has been established as the standard setting for LDAGM.

[0126] Table 8. Comparison of the effects of different channel compression ratios and convolution kernel sizes on the Synapse multi-organ dataset in LDAGM.

[0127]

[0128] 5.5. Comparison of model decoder and overall complexity

[0129] In Table 9, the total parameters and FLOPs of the encoder backbone and decoder are reported. For both the PVT-CASCADE method and the PVT-EMFSD model, the PVTV2-B2 backbone with an input resolution of 224×224 pixels is used. The best results are highlighted in bold. The table shows that PVT-EMFSD outperforms the current state-of-the-art PVT-CASCADE method in overall performance, with 4.34G FLOPs and 26.56M parameters. This is about 1.5G fewer FLOPs and 8.57M fewer parameters compared to the PVT-CASCADE architecture; since both models use PVTV2-B2 as the encoder, the savings in FLOPs and parameters mainly come from the improved decoder of the present invention. Specifically, the decoder of the present invention reduces FLOPs by 77.36%, the number of parameters by 81.66%, and the average DICE score increases by 2.54%. This significant optimization shows that the decoder proposed in the present invention not only significantly reduces computational resources and costs, but also improves model performance.

[0130] Table 9. Overall computational complexity comparison.

[0131]

[0132] 5.6. Impact of input resolution

[0133] Table 10 shows the segmentation performance of the PVT-EMFSD model at different input resolutions. Experiments were conducted at three input resolutions of 224×224, 384×384, and 512×512 pixels. The results clearly show that the average DICE score increases with the increase in input resolution. This trend suggests that higher input resolutions can provide more detailed information, thereby enhancing the segmentation accuracy of the model. It is worth noting that when the input resolution is set to 512×512 pixels, the best average DICE score of 86.17% is achieved, which is 1.06% higher than 384×384 pixels and 2.57% higher than 224×224 pixels. This result not only reflects the superior performance of the model in processing high-resolution images, but also emphasizes the importance of choosing a suitable input resolution to obtain better segmentation results in practical applications. Further analysis shows that higher input resolutions enable the model to capture the subtle features of objects more effectively, thereby significantly reducing segmentation errors.

[0134] Table 10. Experimental results of PVT-EMFSD with different input resolutions on the Synapse multi-organ dataset.

[0135]

[0136] Example 3

[0137] The specific segmentation method of the medical image segmentation model based on the efficient multi-scale feature self-attention decoder and enhanced skip connection is as follows:

[0138] 1 Data Collection

[0139] Cooperation with medical institutions: Establish cooperative relationships with hospitals and clinics to obtain diagnostic image data. Ensure the legality and compliance of data and comply with relevant privacy protection regulations.

[0140] Public datasets: Use already released medical image datasets, such as the Synapse multi-organ dataset, the ACDC dataset, and the ISIC2018 dataset. These datasets have generally undergone rigorous ethical review and de-identification and are suitable for research and development.

[0141] 2 Data preprocessing

[0142] To meet the model input requirements, all images will be uniformly resized to a resolution of 224×224 pixels. At the same time, in order to increase data diversity and improve the generalization ability of the model, data enhancement techniques such as random rotation and flipping will be applied. These image enhancement operations can effectively expand the training data set, reduce overfitting, and enhance the robustness of the model in practical applications.

[0143] 3 Medical Image Model Architecture

[0144] Medical images are segmented through a lightweight 2D medical image segmentation method with an efficient multi-scale feature self-attention decoder and enhanced jump connections. The model takes a medical image as input, extracts the feature information of the medical image through the encoder, and then restores the feature map to the size of the original image through the decoder, finally generating the classification result of each pixel and outputting an accurate segmentation result map.

[0145] 4 Model training and evaluation

[0146] First, the model is trained on the training set to obtain the model weights, and the performance of the model in the segmentation task is evaluated on the validation set. Subsequently, the model is finally evaluated by using an unseen test set to ensure that the model has good generalization ability. In this process, this method uses the average Dice coefficient as the main evaluation indicator, and selects the model weight that performs best on the test set as the final optimal model. The Dice coefficient is used to measure the degree of overlap between the predicted segmentation result and the true annotation. The higher the value, the better the model segmentation effect. Especially when facing small targets or class imbalance problems, the Dice coefficient can effectively reflect the segmentation accuracy of the model.

[0147] 5 Application and Reasoning

[0148] Once the model is trained and evaluated, it can be applied to actual medical image segmentation tasks. By inputting new images into the model, the model outputs the predicted category for each pixel and generates a segmentation result.

[0149] The reasoning process usually includes:

[0150] Image Input: Input new medical images into the trained model.

[0151] Post-processing: Processing the output results of the model, which may include smoothing, threshold processing, removal of small areas, etc.

[0152] Visualization: Compare the segmentation results with the original image to show the effect of the model segmentation.

[0153] 6. Model optimization and tuning

[0154] Increase the dataset: Expand the dataset by obtaining more annotated images or performing data augmentation to improve the generalization ability of the model.

[0155] Improve network architecture: Depending on the complexity of the task, try different network architectures, such as deeper U-nets or more complex convolutional neural networks, to further improve segmentation accuracy.

[0156] In summary, this paper introduces a novel and efficient multi-scale feature self-attention decoder EMFSD, which aims to optimize the efficiency and performance of medical image segmentation under resource-constrained conditions. The decoder significantly improves the segmentation quality of medical images by aggregating and refining multi-stage features. EMFSD adopts a multi-scale feature self-attention module that can effectively capture channel and spatial information at different scales, thereby improving the performance of medical image segmentation. By utilizing linear deformable convolution in LDAGM, EMFSD not only captures richer location information, but also enhances the segmentation efficiency, enabling the model to better identify complex targets. In experiments on three public medical image segmentation datasets, the PVT-EMFSD model demonstrated strong competitiveness, with an average DICE score surpassing the state-of-the-art PVT-CASCADE method. In addition, compared with the CASCADE decoder, EMFSD shows significant advantages in computational complexity, with 77.36% fewer FLOPs and 81.66% fewer parameters.

[0157] Note: The models, methods or data sets mentioned in this invention come from:

[0158] [1] Ronneberger O, Fischer P, Brox TU-net: Convolutional networks for biomedical image segmentation; proceedings of the Medical image computing and computer-assisted intervention–MICCAI 2015:18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, F, 2015[C]. Springer.

[0159] [2]Chen J,Lu Y,Yu Q,et al.Transunet:Transformers make strong encodersfor medical image segmentation[J].arXiv preprint arXiv:210204306,2021.

[0160] [3]Zhou Z,Rahman Siddiquee M M,Tajbakhsh N,et al.Unet++:A nested u-net architecture for medical image segmentation;proceedings of the DeepLearning in Medical Image Analysis and Multimodal Learning for ClinicalDecision Support:4th International Workshop,DLMIA 2018,and 8th InternationalWorkshop,ML-CDS 2018,Held in Conjunction with MICCAI 2018,Granada,Spain,September 20,2018,Proceedings 4,F,2018[C].Springer.

[0161] [4]Huang H,Lin L,Tong R,et al.Unet 3+:A full-scale connected unet formedical image segmentation;proceedings of the ICASSP 2020-2020 IEEEinternational conference on acoustics,speech and signal processing(ICASSP),F,2020[C].IEEE.

[0162] [5]Guo C,Szemenyei M,Yi Y,et al.Sa-unet:Spatial attention u-net forretinal vessel segmentation;proceedings of the 2020 25th internationalconference on pattern recognition(ICPR),F,2021[C].IEEE.[6]Oktay O.AttentionU-Net:Learning Where to Look for the Pancreas[J].arXiv preprint arXiv:180403999,2018.

[0163] [7]Fan D-P,Ji G-P,Zhou T,et al.Pranet:Parallel reverse attentionnetwork for polyp segmentation;proceedings of the International conference onmedical image computing and computer-assisted intervention,F,2020[C].Springer.

[0164] [8]Lan L,Cai P,Jiang L,et al.BRAU-Net++:U-Shaped Hybrid CNN-Transformer Network for Medical Image Segmentation[J].arXiv preprint arXiv:240100722,2024.

[0165] [9]Zhao X,Xu W.NFMPAtt-Unet:Neighborhood fuzzy c-means multi-scalepyramid hybrid attention unet for medical image segmentation[J].NeuralNetworks,2024,178:106489.

[0166]

[10] Dosovitskiy A.An image is worth 16x16 words:Transformers forimage recognition at scale[J].arXiv preprint arXiv:201011929,2020.

[0167]

[11] Liu Z,Lin Y,Cao Y,et al.Swin transformer:Hierarchical visiontransformer using shifted windows;proceedings of the Proceedings of the IEEE / CVF international conference on computer vision,F,2021[C].

[0168]

[12] Wang W,Xie E,Li X,et al.Pvt v2:Improved baselines with pyramidvision transformer[J].Computational Visual Media,2022,8(3):415-24.

[0169]

[13] Wang W,Xie E,Li X,et al.Pyramid vision transformer:A versatilebackbone for dense prediction without convolutions;proceedings of theProceedings of the IEEE / CVF international conference on computer vision,F,2021[C].

[0170]

[14] Rahman M M,Marculescu R.Multi-scale hierarchical visiontransformer with cascaded attention decoding for medical image segmentation;proceedings of the Medical Imaging with Deep Learning,F,2024[C].PMLR.

[0171]

[15] Yu W,Luo M,Zhou P,et al.Metaformer is actually what you need forvision;proceedings of the Proceedings of the IEEE / CVF conference on computervision and pattern recognition,F,2022[C].

[0172]

[16] Liu Y,Zhang Y,Wang Y,et al.A survey of visual transformers[J].IEEE Transactions on Neural Networks and Learning Systems,2023.

[0173]

[17] Rahman M M,Marculescu R.Medical image segmentation via cascadedattention decoding;proceedings of the Proceedings of the IEEE / CVF WinterConference on Applications of Computer Vision,F,2023[C].

[0174]

[18] Zhang X,Song Y,Song T,et al.LDConv:Linear deformable convolutionfor improving convolutional neural networks[J].Image and Vision Computing,2024,149:105190.

[0175]

[19] Huang X,Deng Z,Li D,et al.Missformer:An effective medical imagesegmentation transformer[J].arXiv preprint arXiv:210907162,2021.

[0176]

[20] Cao H,Wang Y,Chen J,et al.Swin-unet:Unet-like pure transformerfor medical image segmentation;proceedings of the European conference oncomputer vision,F,2022[C].Springer.

[0177]

[21] Wang H,Xie S,Lin L,et al.Mixed transformer u-net for medicalimage segmentation;proceedings of the ICASSP 2022-2022 IEEE internationalconference on acoustics,speech and signal processing(ICASSP),F,2022[C].IEEE.

[0178]

[22] Dong B,Wang W,Fan D-P,et al.Polyp-pvt:Polyp segmentation withpyramid vision transformers[J].arXiv preprint arXiv:210806932,2021.

[0179]

[23] Zhong J,Tian W,Xie Y,et al.PMFSNet:Polarized Multi-scale FeatureSelf-attention Network For Lightweight Medical Image Segmentation[J].arXivpreprint arXiv:240107579,2024.

[0180]

[24] Codella N,Rotemberg V,Tschandl P,et al.Skin lesion analysistoward melanoma detection 2018:A challenge hosted by the international skinimaging collaboration(isic)[J].arXiv preprint arXiv:190203368,2019.

[0181]

[25] Loshchilov I.Decoupled weight decay regularization[J].arXivpreprint arXiv:171105101,2017.

[0182]

[26] Wang J,Huang Q,Tang F,et al.Stepwise feature fusion:Local guidesglobal;proceedings of the International Conference on Medical Image Computingand Computer-Assisted Intervention,F,2022[C].Springer.

[0183]

[27] Lou A,Guan S,Ko H,et al.CaraNet:context axial reverse attentionnetwork for segmentation of small medical objects;proceedings of the MedicalImaging 2022:Image Processing,F,2022[C].SPIE.

[0184]

[28] Zhang Y,Liu H,Hu Q.Transfuse:Fusing transformers and cnns formedical image segmentation;proceedings of the Medical image computing andcomputer assisted intervention–MICCAI 2021:24th international conference,Strasbourg,France,September 27–October 1,2021,proceedings,Part I 24,F,2021[C].Springer.

[0185]

[29] Wang H,Cao P,Wang J,et al.Uctransnet:rethinking the skipconnections in u-net from a channel-wise perspective with transformer;proceedings of the Proceedings of the AAAI conference on artificialintelligence,F,2022[C].

Claims

1. A medical image segmentation model based on efficient multi-scale feature self-attention decoder and enhanced skip connection, characterized in that: Firstly, a lightweight and efficient multi-scale feature self-attention module (MFSM) is constructed, which adopts bottleneck convolution in the up-sampling and down-sampling processes of the image, and executes two self-attention mechanisms in sequence between the two processes; secondly, a linear deformable attention guided module (LDAGM) is introduced to enhance the feature maps transmitted through skip connections; finally, the EMFSD decoder is formed by combining MFSM and LDAGM; the EMFSD decoder is combined with the pyramid visual transformer (PVT) to form a medical image segmentation model (PVT-EMFSD), which adopts a hierarchical structure, utilizes multi-level features from the hierarchical visual encoder, and uses the U-shaped network architecture to gradually enhance the semantic information of the feature maps.

2. The medical image segmentation model based on efficient multi-scale feature self-attention decoder and enhanced skip connection as claimed in claim 1, characterized in that: The proposed EMFSD decoder consists of linear deformable attention guided modules (LDAGMs), efficient multi-scale feature self-attention modules (MFSMs), upsampling modules with deep convolution (DWUCBs) and segmentation heads (SHs); MFSMs are used to capture global context features, multi-scale channels and spatial dependencies; LDAGMs optimize feature maps fused with skip connections by guiding the attention mechanism; DWUCBs are responsible for upsampling feature maps while further enhancing feature details; SHs generate the final segmentation map for each stage.

3. The medical image segmentation model based on efficient multi-scale feature self-attention decoder and enhanced skip connection as claimed in claim 2, characterized in that: In the linear deformable attention guided module (LDAGM), the attention function Att(·) is first used; the linear deformable convolution LDC is used respectively. s (·) and LDC x (·) processes s and x and merges them by element-wise addition; subsequently, a 1×1 convolution C(·) is applied to obtain a single-channel feature map and normalized by a batch normalization layer BN(·); finally, after applying the Sigmoid activation function σ(·), the attention coefficient is obtained; the resulting attention coefficient is multiplied element-wise with the input feature x , to enhance relevant features and suppress irrelevant features, thereby generating attention-guiding features LDAGM(s,x); LDAGM(·), expressed by equations (1) and (2): That(s,x)=LDC s (s)+LDC x (x) (1) Due to the use of variable kernel convolution in Att(·), LDAGM is able to capture effective spatial context information with different computational costs.

4. The medical image segmentation model based on efficient multi-scale feature self-attention decoder and enhanced skip connection as claimed in claim 3, characterized in that: The multi-scale feature self-attention module (MFSM) consists of two bottleneck convolutions (BottleConv1(·) and BottleConv2(·))) and two multi-scale self-attention modules; the processing of feature maps by MCS(·) and MSS(·) is merged into multi-scale feature self-attention (MFS(·)); first, batch normalization BN(·) is applied to the feature map, and then the activation function ReLU(·) is used to activate the feature map; next, a 3×3 depth convolution DWConv is used 3×3 (·) to enhance the feature map; finally, a 1×1 standard convolution Conv is applied 1×1 (·) to adjust the number of channels in the feature map; the bottleneck convolution changes the number of channels in the feature map by adjusting the channel compression ratio, while the two multi-scale self-attention modules enhance the feature representation in the channel and spatial dimensions; The expressions of BottleConv1(·), BottleConv2(·), MFS(·), and MFSM(·) (see Figure 3) are given in Equations (3), (4), (5), and (6), respectively: Bottle Conv1(x)=Conv 1×1 (DWConv 3×3 (ReLU(BN(x)))) (3) Bottle Conv2(x)=Conv 1×1 (DWConv 3×3 (ReLU(BN(x)))) (4) MFSM(x)=Bottle Conv2(MFS(Bottle Conv1(x))) (6) Here Represents an element-by-element multiplication operation; bottleneck convolution is combined to reduce the number of channels of the intermediate feature map, and a multi-scale self-attention module is combined to enhance the feature map.

5. The medical image segmentation model based on efficient multi-scale feature self-attention decoder and enhanced skip connection as claimed in claim 4, characterized in that: The depth-wise convolution-based upsampling convolution block (DWUCB) upsamples the feature map of the current stage to ensure that the number of channels and resolution match the feature map in the next skip connection; the calculation formula of DWUCB(·) is shown in formula (7): DWUCB(x)=Conv 1×1 (ReLU(BN(DWConv 3×3 (Up 2× (x))))) (7) Among them, DWConv 3×3 (·) represents the depth convolution of 3×3 convolution kernel, Conv 1×1 (·) denotes point-wise convolution; BN(·) denotes batch normalization, and ReLU(·) refers to the activation function used.

6. The medical image segmentation model based on efficient multi-scale feature self-attention decoder and enhanced skip connection as claimed in claim 5, characterized in that: The segmentation head (SH) is used to generate the final segmentation map from the four stages of the decoder; it uses a 1×1 convolution Conv 1×1 (·) adjusts the number of channels in the feature map; for multi-class segmentation tasks, the number of output channels is equal to the number of categories; for binary segmentation tasks, there is only one output channel; the calculation formula of SH(·) is shown in formula (8): SH(x)=Conv 1×1 (x) (8)。 7. The medical image segmentation model based on efficient multi-scale feature self-attention decoder and enhanced skip connection as claimed in claim 6, characterized in that: In the EMFSD decoder, the four segmentation results p1, p2, p3 and p4 generated by the decoder in four different stages are combined to generate the final segmentation output, as shown in formula (9): seg output =αp1+βp2+γp3+ζp4 (9) In this case, α, β, γ, and ζ are the weights assigned to each prediction head, all set to 1.0; for the binary segmentation task, the Sigmoid function is used to activate the feature map and generate the final output; for the multi-category task, the Softmax function is used to activate the feature map to obtain the final result.

8. A medical image segmentation method based on the model according to any one of claims 1 to 7, characterized in that: Here are the steps: ①Data collection and preprocessing A diagnostic image dataset is obtained, and all images will be uniformly adjusted to a resolution of 224×224 pixels. At the same time, in order to increase data diversity and improve the generalization ability of the model, data enhancement techniques including random rotation and flipping will be applied. These image enhancement operations can effectively expand the training dataset, reduce overfitting, and enhance the robustness of the model in practical applications. ②Medical image model architecture Medical images are segmented through a lightweight 2D medical image segmentation method with an efficient multi-scale feature self-attention decoder and enhanced skip connections. The model takes a medical image as input, extracts the feature information of the medical image through the encoder, and then restores the feature map to the size of the original image through the decoder, finally generating the classification result of each pixel and outputting an accurate segmentation result map. ③Model training and evaluation First, train the model on the training set to obtain the model weights, and evaluate the performance of the model in the segmentation task on the validation set; Subsequently, the model is finally evaluated by using an unseen test set to ensure that the model has good generalization ability. In this process, the average Dice coefficient is used as the main evaluation indicator, and the model weight with the best performance on the test set is selected as the final best model. The Dice coefficient is used to measure the overlap between the predicted segmentation result and the true annotation. The higher the value, the better the model segmentation effect. Especially when facing small targets or class imbalance problems, the Dice coefficient can effectively reflect the segmentation accuracy of the model. ④Application and reasoning Once the model is trained and evaluated, it can be applied to actual medical image segmentation tasks. By inputting new images into the model, the model will output the predicted category of each pixel and generate segmentation results. The reasoning process usually includes: Image input: input new medical images into the trained model; Post-processing: Processing the output of the model, which may include smoothing, thresholding, and removing small areas; Visualization: Compare the segmentation results with the original image to show the effect of the model segmentation; ⑤Model optimization and tuning Increase the dataset: Expand the dataset by obtaining more annotated images or performing data augmentation to improve the generalization ability of the model; Improve network architecture: Depending on the complexity of the task, try different network architectures, such as deeper U-nets or more complex convolutional neural networks, to further improve segmentation accuracy.

Images