Multi-scale fusion skin lesion image segmentation method based on Mama

Abstract

The invention discloses a Mama-based multi-scale fusion skin lesion image segmentation method, and belongs to the technical field of skin lesion image segmentation, and the method comprises the following steps: S1, obtaining a skin lesion image, carrying out the preprocessing of the skin lesion image, and generating a preprocessed skin lesion image; and S2, inputting the skin lesion image into the lightweight skin lesion segmentation model to generate a skin lesion image segmentation result. According to the invention, a lightweight skin lesion segmentation model is designed for skin lesion image segmentation, the model is provided with an LERM module, and the LERM module combines depth separable convolution with a Mama-based parallel processing module, so that the modeling capability of global and local features is maintained while the parameter quantity of the model is effectively reduced, and the model is more accurate. According to the model, through combination of an LERM module and multi-level jump connection, the modeling capability of the focus with diversified forms is enhanced, and the problem that it is difficult to achieve balance between segmentation precision and calculation efficiency in the prior art is solved.

Description

Technical Field

[0001] This invention belongs to the field of skin lesion image segmentation technology, specifically involving a multi-scale fusion skin lesion image segmentation method based on Mamba. Background Technology

[0002] In skin lesion image segmentation, accurately capturing the lesion region is crucial. CNN-based segmentation models rely on local convolutions for feature extraction, excelling at capturing local texture details. However, due to the limitations of the local receptive field in convolution operations, these models are insufficient in modeling long-range dependencies and global contextual information, easily leading to incomplete boundary issues. Transformers establish long-range dependencies through self-attention mechanisms, improving the model's understanding of global semantic information. However, their self-attention computation is highly complex, resulting in a large number of model parameters and increased computational cost. In skin lesion image segmentation, existing methods generally face the problem that accuracy improvement depends on increased model complexity, making it difficult to achieve a balance between segmentation accuracy and computational efficiency. Summary of the Invention

[0003] To address the aforementioned shortcomings in existing technologies, the Mamba-based multi-scale fusion skin lesion image segmentation method provided by this invention solves the problem that existing technologies struggle to achieve a balance between segmentation accuracy and computational efficiency.

[0004] To achieve the aforementioned objectives, the present invention employs the following technical solution: a multi-scale fusion skin lesion image segmentation method based on Mamba, comprising the following steps: S1. Acquire images of skin lesions, preprocess them, and generate preprocessed images of skin lesions; S2. Input the skin lesion image into the lightweight skin lesion segmentation model to generate the skin lesion image segmentation result; In S2, the lightweight skin lesion segmentation model includes an encoder and a decoder, with the encoder connected to the decoder via skip connections. The encoder includes a first depthwise separable convolution, a second depthwise separable convolution, a third depthwise separable convolution, a first LERM module, a second LERM module, and a third LERM module connected in sequence. The decoder includes a fourth LERM module, a fifth LERM module, a sixth LERM module, a fourth depthwise separable convolution, a fifth depthwise separable convolution, and a sixth depthwise separable convolution connected in sequence. The first depthwise separable convolution is connected to the sixth depthwise separable convolution via a linear skip connection module. The second depthwise separable convolution is connected to the fifth depthwise separable convolution via an edge-enhanced skip connection module. The third depthwise separable convolution is connected to the fourth depthwise separable convolution via an edge-enhanced skip connection module. The first LERM module is connected to the sixth LERM module via an adaptive skip fusion gate module. The second LERM module is connected to the fifth LERM module via an adaptive skip fusion gate module. The third LERM module is connected to the fourth LERM module.

[0005] Furthermore, in S1, the preprocessing method includes a uniform scaling operation.

[0006] Furthermore, the first, second, third, fourth, fifth, and sixth LERM modules have the same structure, each including a lightweight local feature enhancement submodule, a flattening and layer normalization submodule, and a channel segmentation submodule connected in sequence. The output of the channel segmentation submodule is connected to the inputs of the first, second, third, and fourth Mamba submodules, respectively. The outputs of the first, second, third, and fourth Mamba submodules are concatenated and then connected to the first input of element-wise addition. The output of the flattening and layer normalization submodule is connected to the second input of element-wise addition through the residual. The output of element-wise addition is connected in sequence to the regularization and layer normalization submodule and the linear projection submodule.

[0007] The beneficial effects of the above-mentioned further solutions are as follows: This invention proposes a lightweight skin lesion segmentation model, AMF-UNet, which adopts an encoder-decoder symmetrical network structure. In the first three stages of the model, depthwise separable convolutions are used to extract edge and detail information while reducing a large number of redundant parameters. In the last three stages of the model, this invention designs a LERM module, which enhances local feature information through a lightweight local feature enhancement submodule, extracts global features using a block-parallel Mamba submodule, and uses a learnable residual network to adjust the fusion strength of CNN and Mamba. In the skip connection part, three skip connection methods are designed to extract features at different stages. In the shallow stage, a linear skip connection module with 1×1 convolution is used to smoothly transmit edge and texture information to the decoder. In the middle stage, the edge enhancement skip connection module uses depthwise separable convolution to perform lightweight calibration on encoder features and adjust the scale structure space details of this stage. In the deep stage, the adaptive skip fusion gate module uses local attention and channel attention to learn fusion weights, so that the semantic information of the deep encoder and the features of the decoder can achieve consistent compensation. The LERM module adopts a parallel Mamba structure combined with the local enhancement convolution module, which achieves global dependency modeling and fine-grained local texture feature capture with extremely low parameter count.

[0008] Furthermore: the linear jump connection module is equipped with The workflow of the convolutional, linear skip connection module is as follows: through Convolution performs channel compression and linear projection on the input features of the linear skip connection module to generate the output features of the linear skip connection module.

[0009] The beneficial effects of the above further solutions are: the introduction of linear jump connection modules Convolution performs channel compression and linear projection. This operation effectively suppresses irrelevant channel noise while maintaining the integrity of spatial structure information, and maps shallow features to the same channel dimension as the decoder.

[0010] Furthermore: The edge enhancement skip connection module calculates edge enhancement features. E The specific expression is: In the formula, S For the input features of the edge-enhanced skip connection module, For depthwise separable convolution, For group normalization operation, For GELU activation operation, for convolution; Edge features are fused with backbone decoding features to generate the output of the current level of the decoder. The specific expression is: In the formula, Main decoding features To reduce the learningable residual scaling factor, This is an element-wise addition.

[0011] The beneficial effects of the above-mentioned further solutions are as follows: This invention extracts local texture responses through depthwise separable convolution and maintains numerical stability in small-batch scenarios through group normalization. The results after group normalization are then used... Convolution and GELU activation perform linear reconstruction and nonlinear modulation on the edge stream to obtain edge enhancement features. To avoid amplifying noise in edge enhancement while maintaining the supplementation of semantic features, this invention introduces a lightweight learnable residual scaling factor. The edge enhancement features are combined with the lightweight learnable residual scaling factor and added element-wise with the backbone decoding features to achieve the fusion of edge features and backbone decoding features.

[0012] Furthermore: The adaptive skip fusion gate module calculates skip features. The specific expression is: In the formula, For the input features of the adaptive jump fusion gate module, F For spatial attention, M For channel attention, For element-wise multiplication; In the formula, Use the Sigmoid activation function; In the formula, For the first learnable linear transformation, For the second learnable linear transformation, For global average pooling; The skip feature is fused with the backbone decoding feature to generate the output of the current level of the decoder. The specific expression is: In the formula, Main decoding features and For adaptive weights, , , For adaptive fusion gate factor; In the formula, For the first-level linear mapping of the fusion gate, For the second-level linear mapping of the fusion gate, For splicing operations, This is a RELU activation operation.

[0013] The beneficial effects of the above-mentioned further scheme are as follows: the adaptive skip fusion gate module uses spatial attention and channel attention to obtain structurally sensitive and noise-pressed skip features. Furthermore, this invention introduces an adaptive fusion gate factor, which enables the network to dynamically adjust the fusion ratio of skip and decoder according to different image content and noise conditions. During training, the adaptive weight ratio is dynamically optimized. Through this dynamic ratio, the adaptive skip fusion gate module can effectively avoid the common skip-dominant effect and achieve balanced structural and semantic fusion under different noise and contrast conditions.

[0014] Furthermore: In S2, the method for generating the skin lesion image segmentation results is as follows: S21. Input the skin lesion image into the encoder. The skin lesion image is sequentially processed through the first depthwise separable convolution, the second depthwise separable convolution and the third depthwise separable convolution in the shallow feature extraction stage of the encoder to extract shallow features, generating the first shallow feature, the second shallow feature and the third shallow feature respectively. The third shallow feature is sequentially processed through the first LERM module, the second LERM module and the third LERM module in the deep feature extraction stage of the encoder to extract deep features, generating the first deep feature, the second deep feature and the bottom feature respectively. S22. Input the low-level features into the decoder. The low-level features pass through the fourth LERM module, the fifth LERM module and the sixth LERM module of the deep feature decoding stage of the decoder in sequence to generate the first decoded feature, the second decoded feature and the third decoded feature, respectively. Among them, the skip feature is calculated based on the second deep feature by the adaptive skip fusion gate module, the second decoded feature is generated by fusing the skip feature with the first decoded feature by the fifth LERM module, the skip feature is calculated based on the first deep feature by the adaptive skip fusion gate module, and the third decoded feature is generated by fusing the skip feature with the second decoded feature by the sixth LERM module. S23. The third decoded feature is sequentially processed by the fourth, fifth, and sixth depthwise separable convolutions in the shallow feature decoding stage of the decoder to generate the fourth, fifth, and sixth decoded features, respectively. Specifically, the edge enhancement jump connection module calculates the edge enhancement feature based on the third shallow feature, and the fourth depthwise separable convolution fuses the edge enhancement feature with the third decoded feature to generate the fourth decoded feature. The edge enhancement jump connection module calculates the edge enhancement feature based on the second shallow feature, and the fifth depthwise separable convolution fuses the edge enhancement feature with the fourth decoded feature to generate the fifth decoded feature. The linear jump connection module calculates the output feature based on the first shallow feature, and the sixth depthwise separable convolution fuses the output feature with the fifth decoded feature to generate the high-resolution feature. S24. The high-resolution features are mapped to the target category channel through 1×1 convolution, and restored to the resolution of the skin lesion image through bilinear interpolation. The pixel-level segmentation probability map is then activated by Sigmoid and used as the skin lesion image segmentation result.

[0015] The beneficial effects of this invention are as follows: (1) This invention provides a multi-scale fusion skin lesion image segmentation method based on Mamba, and designs a lightweight skin lesion segmentation model AMF-UNet based on Mamba for skin lesion image segmentation. The model is designed with a LERM module, which combines depthwise separable convolution with a parallel processing module based on Mamba. While effectively reducing the number of model parameters, it maintains the ability to model global and local features, solving the problem that it is difficult to achieve a balance between segmentation accuracy and computational efficiency in existing technologies. Furthermore, the model of this invention enhances the ability to model lesions with diverse morphologies by combining the LERM module and multi-level skip connections.

[0016] (2) The present invention designs three types of jump connection modules: linear jump connection module, edge enhancement jump connection module and adaptive jump fusion gate module, to extract multi-level features. The linear jump connection module retains shallow features, the edge enhancement jump connection module enhances structural texture details, and the adaptive jump fusion gate module extracts deep features. Through the multi-level jump connection mechanism of the three jump connection modules, adaptive feature fusion of the lesion area is realized.

[0017] (3) Experimental verification shows that the lightweight skin lesion segmentation model AMF-UNet proposed in this invention focuses on reducing false negatives. This design is of great significance for the clinical diagnosis of skin lesions, effectively avoiding the risk of missed diagnosis and meeting the needs of actual clinical applications. Furthermore, AMF-UNet can generate more accurate and smooth segmentation masks when processing skin lesion images with blurred boundaries, irregular contours, and diverse lesion shapes. Compared with other methods, the model of this invention significantly reduces the error prediction at the edge of the lesion area, and its segmentation results are more consistent with the real annotations.

[0018] (4) Ablation experiments were conducted on the ISIC2017 and ISIC2018 datasets based on the lightweight skin lesion segmentation model AMF-UNet. On the ISIC2017 dataset, the average crossover ratio was improved by 0.97%, the dice coefficient was improved by 0.55%, and the accuracy was improved by 0.19%. The magnitude of the index optimization verified the effectiveness of the improved module in improving the segmentation accuracy of the lesion area. Similarly, on the ISIC2018 dataset, the accuracy was improved by 1.76%, 1.07%, and 0.51% respectively compared with the baseline. The more significant performance gain fully demonstrates that the proposed improvement strategy has excellent adaptability. Attached Figure Description

[0019] Figure 1 This is a flowchart of the Mamba-based multi-scale fusion skin lesion image segmentation method of the present invention.

[0020] Figure 2 Network structure diagram for a lightweight skin lesion segmentation model.

[0021] Figure 3 This is a structural diagram of the LERM module.

[0022] Figure 4 The segmentation results of different models on ISIC2017 and ISIC2018. Detailed Implementation

[0023] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0024] like Figure 1 As shown, in one embodiment of the present invention, a multi-scale fusion skin lesion image segmentation method based on Mamba includes the following steps: S1. Acquire images of skin lesions, preprocess them, and generate preprocessed images of skin lesions; S2. Input the skin lesion image into the lightweight skin lesion segmentation model AMF-UNet (Adaptive Multi-Scale Fusion UNet) to generate the skin lesion image segmentation result.

[0025] In S1, the preprocessing methods include uniform scaling operations.

[0026] In S2, the lightweight skin lesion segmentation model includes an encoder and a decoder, with the encoder connected to the decoder via skip connections. In this embodiment, the lightweight skin lesion segmentation model is as follows: Figure 2 As shown, the encoder includes a first depthwise separable convolution, a second depthwise separable convolution, a third depthwise separable convolution, a first LERM (Local Enhanced Residual Mamba) module, a second LERM module, and a third LERM module connected in sequence. The decoder includes a fourth LERM module, a fifth LERM module, a sixth LERM module, a fourth depthwise separable convolution, a fifth depthwise separable convolution, and a sixth depthwise separable convolution connected in sequence. The first depthwise separable convolution is connected to the sixth depthwise separable convolution via a linear skip connection module. The second depthwise separable convolution is connected to the fifth depthwise separable convolution via an edge-enhanced skip connection module. The third depthwise separable convolution is connected to the fourth depthwise separable convolution via an edge-enhanced skip connection module. The first LERM module is connected to the sixth LERM module via an adaptive skip fusion gate module. The second LERM module is connected to the fifth LERM module via an adaptive skip fusion gate module. The third LERM module is connected to the fourth LERM module.

[0027] In this embodiment, the present invention proposes a lightweight skin lesion segmentation model, AMF-UNet, which adopts an encoder-decoder symmetrical network structure. In the first three stages of the model, depthwise separable convolutions are used to extract edge and detail information while reducing a large number of redundant parameters. In the last three stages, the present invention designs a LERM module, which enhances local feature information through a lightweight local feature enhancement submodule, extracts global features using a block-parallel Mamba submodule, and employs a learnable residual network to adjust the fusion strength of CNN and Mamba. In the skip connection part, three skip connection methods are designed to extract features at different stages. In the shallow stage, a linear skip connection module with 1×1 convolutions is used to smoothly transmit edge and texture information to the decoder; the edge enhancement skip connection module in the middle stage uses depthwise separable convolutions to perform lightweight calibration on the encoder features, adjusting the scale-structure spatial details of this stage; the adaptive skip fusion gate module in the deep stage uses local attention and channel attention to learn fusion weights, enabling consistency compensation between the semantic information of the deep encoder and the features of the decoder.

[0028] like Figure 3 As shown, in this embodiment, the first, second, third, fourth, fifth, and sixth LERM modules have the same structure, each including a lightweight local feature enhancement submodule, a flattening and layer normalization (Fltten+LayerNorm) submodule, and a channel splitting (ChannelSpilt) submodule connected in sequence. The output of the channel splitting submodule is connected to the inputs of the first, second, third, and fourth Mamba submodules, respectively. The outputs of the first, second, third, and fourth Mamba submodules are concatenated and then connected to the first input of element-wise addition. The output of the flattening and layer normalization submodule is connected to the second input of element-wise addition through the residual. The output of element-wise addition is connected in sequence to the regularization and layer normalization (LayerNorm+Dropout) submodule and the linear projection (Linear Projection) submodule.

[0029] In this embodiment, the principle behind the design of the LERM module of the present invention is as follows: To compensate for Mamba's shortcomings in local spatial inductive bias, this embodiment first designs a lightweight local feature enhancement submodule. This submodule enhances the local information of the input features through depthwise separable convolutions and pointwise convolutions. The output of the lightweight local feature enhancement submodule... The expression: In the formula, X For the input of the LERM module, For depthwise separable convolution, for convolution, Activate GELU.

[0030] The output of the lightweight local feature enhancement submodule is flattened into a token sequence and then subjected to layer normalization to ensure consistent feature scale at different locations. The specific expression for this process is as follows: In the formula, For flattening operation, For token sequence, For layer normalization operation, This is the output after the flattening and layer normalization operations.

[0031] Inspired by the concept of channel grouping, features are divided into four sub-blocks along the channel dimension. Each sub-block is modeled for sequence dependencies using an independent Mamba state-space model. After parallel processing, the sub-blocks are concatenated back to the original channel dimension. The specific expression for this process is as follows: In the formula, To reassemble the features back to the original channel dimension, For splicing operations, For the output of the Mamba submodule, This is the first subspace block after the channel is divided. This is the second subspace block after the channel is divided. This is the third subspace block after the channel is divided. This is the fourth subspace block after the channel is divided.

[0032] To address the issue of fixed feature fusion ratios in traditional residual connections, this embodiment introduces a learned residual weight mechanism. By dynamically adjusting the weight ratios of the original features and Mamba submodule features, it enhances the flexibility and adaptability of information fusion during sequence modeling. The output of the learned residual weight mechanism is shown below. The specific expression is: In the formula, These are learnable parameters used to adaptively adjust the residual signal strength, improving training stability. Next, layer normalization and linear projection are used to map the number of channels to the output dimension: In the formula, Y For the output of the LERM module, For regularization operations, This is a linear projection operation.

[0033] The Linear Skip module is equipped with... The workflow of the convolutional, linear skip connection module is as follows: through Convolution performs channel compression and linear projection on the input features of the linear skip connection module to generate the output features of the linear skip connection module.

[0034] In this embodiment, a linear skip connection module is introduced. Convolution performs channel compression and linear projection. This operation effectively suppresses irrelevant channel noise while preserving the integrity of spatial structure information, mapping shallow features to the same channel dimension as the decoder. Subsequently, residual feature fusion is used to add the projected shallow features element-wise with the reconstructed features of the corresponding scale from the decoder, thereby achieving lightweight and stable cross-scale information interaction.

[0035] The Edge-Enhanced Skip module computes edge enhancement features. E The specific expression is: In the formula, S For the input features of the edge-enhanced skip connection module, For depthwise separable convolution, For group normalization operation, For GELU activation operation, for convolution; Edge features are fused with backbone decoding features to generate the output of the current level of the decoder. The specific expression is: In the formula, Main decoding features For lightweight learnable residual scaling factors, where, The initial value is set to 0.5, and the parameter size is adaptively adjusted during training. This is an element-wise addition.

[0036] In this embodiment, for the input features of the edge enhancement skip connection module, the present invention extracts local texture responses through depthwise separable convolution and maintains numerical stability in small-batch scenarios through group normalization (GroupNorm). The results after group normalization are then used... Convolution and GELU activation perform linear reconstruction and nonlinear modulation on the edge flow to obtain edge enhancement features, which are used as the output of the edge enhancement skip connection module. To avoid amplifying noise through edge enhancement while maintaining the supplementation of semantic features, this invention introduces a lightweight learnable residual scaling factor. The edge enhancement features are combined with the lightweight learnable residual scaling factor and added element-wise with the backbone decoding features to achieve the fusion of edge features and backbone decoding features.

[0037] The Adaptive Skip Fusion Gate module utilizes spatial attention. With channel attention Multiplication yields a structure-sensitive and noise-pressed skip feature; the skip feature is then calculated. The specific expression is: In the formula, For the input features of the adaptive jump fusion gate module, F For spatial attention, M For channel attention, For element-wise multiplication; For input features In this embodiment, the edge and local texture responses are extracted and spatial modulation weights are generated to obtain spatial attention. F : In the formula, Use the Sigmoid activation function; Simultaneously with spatial modulation, this embodiment further enhances the semantic representation of skip through a channel attention branch, resulting in channel attention. M : In the formula, For the first learnable linear transformation, For the second learnable linear transformation, For global average pooling, where, This enables the module to learn non-linear channel interactions, thereby more accurately distinguishing channel patterns that are useful for lesion segmentation.

[0038] After obtaining the structure-sensitive and noise-pressed skip features, this embodiment further introduces an adaptive fusion gate factor. This enables the network to dynamically adjust the fusion ratio of skip and decoder based on different image content and noise conditions, and adaptively adjust the fusion gate factor. The specific expression is: In the formula, For the first-level linear mapping of the fusion gate, This is the second-layer linear mapping of the fusion gate. The linear mapping is used to map global features into adaptive weights that control the fusion ratio between skip and decoder. For splicing operations, This is a RELU activation operation.

[0039] Calculating the adaptive fusion gate factor g Then, the skip features are fused with the backbone decoding features to generate the output of the current level of the decoder. The specific expression is: In the formula, Main decoding features and For adaptive weights, , , and During training, it automatically optimizes based on noise levels, boundary strength, and semantic consistency. Through this dynamic matching, the adaptive skip fusion gate module can effectively avoid the common skip-dominant effect and achieve balanced structural and semantic fusion under different noise and contrast conditions.

[0040] In S2, the network parameter settings for the lightweight skin lesion segmentation model are shown in Table 1.

[0041] Table 1 Network Parameter Settings Where H represents the height and W represents the width. The specific method for generating skin lesion image segmentation results is as follows: S21. The skin lesion image is input into the encoder. The skin lesion image is sequentially processed through the first depthwise separable convolution, the second depthwise separable convolution and the third depthwise separable convolution in the shallow feature extraction stage of the encoder to extract shallow features, generating the first shallow feature t1 (H / 2), the second shallow feature t2 (H / 4) and the third shallow feature t3 (H / 8) respectively, where H is the height. The third shallow feature is sequentially processed through the first LERM module, the second LERM module and the third LERM module in the deep feature extraction stage of the encoder to extract deep features, generating the first deep feature t4 (H / 16), the second deep feature t5 (H / 32) and the bottom feature t6 (H / 32) respectively. S22. Input the low-level features into the decoder. The low-level features pass through the fourth LERM module, the fifth LERM module and the sixth LERM module of the deep feature decoding stage of the decoder in sequence to generate the first decoded feature, the second decoded feature and the third decoded feature, respectively. Among them, the skip feature is calculated based on the second deep feature by the adaptive skip fusion gate module, the second decoded feature is generated by fusing the skip feature with the first decoded feature by the fifth LERM module, the skip feature is calculated based on the first deep feature by the adaptive skip fusion gate module, and the third decoded feature is generated by fusing the skip feature with the second decoded feature by the sixth LERM module. S23. The third decoded feature is sequentially processed by the fourth, fifth, and sixth depthwise separable convolutions in the shallow feature decoding stage of the decoder to generate the fourth, fifth, and sixth decoded features, respectively. Specifically, the edge enhancement jump connection module calculates the edge enhancement feature based on the third shallow feature, and the fourth depthwise separable convolution fuses the edge enhancement feature with the third decoded feature to generate the fourth decoded feature. The edge enhancement jump connection module calculates the edge enhancement feature based on the second shallow feature, and the fifth depthwise separable convolution fuses the edge enhancement feature with the fourth decoded feature to generate the fifth decoded feature. The linear jump connection module calculates the output feature based on the first shallow feature, and the sixth depthwise separable convolution fuses the output feature with the fifth decoded feature to generate the high-resolution feature. S24. The high-resolution features are mapped to the target category channel through 1×1 convolution, and restored to the resolution of the input skin lesion image through bilinear interpolation. The output pixel-level segmentation probability map is activated by Sigmoid and used as the skin lesion image segmentation result.

[0042] To evaluate the effectiveness of the lightweight skin lesion segmentation model AMF-UNet proposed in this invention, this embodiment compares the model of this invention with other state-of-the-art models for skin lesion segmentation tasks on the datasets ISIC2017 and ISIC2018. The experimental results are shown in Tables 2 and 3.

[0043] Table 2 Dataset Table 3. Model performance comparison on the ISIC2017 dataset (unit: %) As shown in Table 3, the model of this invention is the lightweight skin lesion segmentation model AMF-UNet. Other advanced models selected are U-shaped network (UNet), Transformer fusion network (TransFuse), Efficient group augmentation U-shaped network (EGE-Unet), Hybrid convolutional Mamba network (HC-Mamba), High-order spatial interaction U-shaped network (MHorUNet), Visual Mamba U-shaped network (VM-Unet), Collaborative multilayer perceptron and convolutional neural network (UCMNetV2), Lightweight visual Mamba U-shaped network (UltraLight VM-UNet), and Lightweight long short-term memory recurrent neural network and Mamba fusion network (XLSTM-VMUNet). At ISIC2017, the AMF-UNet of this invention achieved the highest four evaluation metrics, namely mean intersection-over-union ratio (83.89%), dice coefficient (91.24%), accuracy (96.47%), and recall (90.42%). The model proposed in this invention is slightly lower than some comparative models in terms of specificity. This is because the model focuses on reducing false negatives. This design is of great significance for the clinical diagnosis of skin lesions, can effectively avoid the risk of missed diagnosis, and meets the needs of actual clinical application.

[0044] Table 4 shows the performance comparison results of different models at ISIC2018. The model of this invention exhibits the same advantages, with higher average crossover ratio (82.24%), dice coefficient (90.25%), and accuracy (95.59%) than all the compared models. Compared with the next lower comparison model, the improvements are 0.35%, 0.21%, and 0.09%, respectively, fully verifying the model's competitiveness in accurate segmentation of lesion areas.

[0045] Table 4. Model performance comparison on the ISIC2018 dataset (unit: %) Figure 4 This presentation showcases the visual segmentation comparison results of the model of this invention with several representative state-of-the-art models on different types of skin lesion images. The first two rows correspond to the ISIC2017 dataset, and the last two rows correspond to the ISIC2018 dataset. The visualization results demonstrate that the model of this invention generates more accurate and smoother segmentation masks when processing skin lesion images with blurred boundaries, irregular contours, and diverse lesion shapes. Compared to other methods, the model of this invention significantly reduces misprediction at the edges of lesion regions, and its segmentation results are more consistent with the ground truth annotations.

[0046] To further evaluate the practicality and clinical deployment potential of the proposed model, this embodiment compares its computational complexity and computational cost with existing models, as shown in Table 5. The results show that the proposed model has 0.05M parameters and a computational cost of 0.02G. After refining to three decimal places, the differences in parameter counts among the models become more apparent: the high-efficiency augmented U-shaped network (0.053M), the network combining multilayer perceptrons and convolutional neural networks (0.046M), and the lightweight visual Mamba U-shaped network (0.049M). The proposed model, with 0.047M parameters, reduces computation by 11.32% compared to the high-efficiency augmented U-shaped network, by 4.08% compared to the lightweight visual Mamba U-shaped network, and the difference compared to the network combining multilayer perceptrons and convolutional neural networks is negligible. These results confirm that the proposed model achieves superior lightweight performance through optimized architecture design, providing engineering feasibility for clinical deployment.

[0047] Table 5 Comparison of Parameters and Computational Costs To verify the effectiveness of each module in the proposed improved model, this embodiment conducted systematic ablation experiments on the ISIC2017 and ISIC2018 datasets, and the results are shown in Table 6. This invention employs a phased integration strategy. The experimental setup progresses from retaining only the original components of the baseline model to introducing some improved modules in stages, ultimately achieving a complete replacement of the core components. Compared to the baseline model, the proposed model shows a 0.97% improvement in the average intersection-over-union ratio, a 0.55% improvement in the dice coefficient, and a 0.19% improvement in accuracy on the ISIC2017 dataset. The magnitude of these performance improvements validates the effectiveness of the improved modules in enhancing lesion region segmentation accuracy. Similarly, on the ISIC2018 dataset, improvements of 1.76%, 1.07%, and 0.51% respectively compared to the baseline demonstrate the significant performance gains, indicating that the proposed improvement strategy has excellent adaptability.

[0048] Table 6 Ablation test (unit: %) In the description of this invention, the above are merely preferred embodiments and are not intended to limit the scope of protection of this invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A multi-scale fusion image segmentation method for skin lesions based on Mamba, characterized in that, Includes the following steps: S1. Acquire images of skin lesions, preprocess them, and generate preprocessed images of skin lesions; S2. Input the skin lesion image into the lightweight skin lesion segmentation model to generate the skin lesion image segmentation result; In S2, the lightweight skin lesion segmentation model includes an encoder and a decoder, with the encoder connected to the decoder via skip connections. The encoder includes a first depthwise separable convolution, a second depthwise separable convolution, a third depthwise separable convolution, a first LERM module, a second LERM module, and a third LERM module connected in sequence. The decoder includes a fourth LERM module, a fifth LERM module, a sixth LERM module, a fourth depthwise separable convolution, a fifth depthwise separable convolution, and a sixth depthwise separable convolution connected in sequence. The first depthwise separable convolution is connected to the sixth depthwise separable convolution via a linear skip connection module. The second depthwise separable convolution is connected to the fifth depthwise separable convolution via an edge-enhanced skip connection module. The third depthwise separable convolution is connected to the fourth depthwise separable convolution via an edge-enhanced skip connection module. The first LERM module is connected to the sixth LERM module via an adaptive skip fusion gate module. The second LERM module is connected to the fifth LERM module via an adaptive skip fusion gate module. The third LERM module is connected to the fourth LERM module.

2. The Mamba-based multi-scale fusion skin lesion image segmentation method according to claim 1, characterized in that, In S1, the preprocessing methods include uniform scaling operations.

3. The Mamba-based multi-scale fusion skin lesion image segmentation method according to claim 1, characterized in that, The first, second, third, fourth, fifth, and sixth LERM modules have the same structure, each including a lightweight local feature enhancement submodule, a flattening and layer normalization submodule, and a channel segmentation submodule connected in sequence. The output of the channel segmentation submodule is connected to the inputs of the first, second, third, and fourth Mamba submodules, respectively. The outputs of the first, second, third, and fourth Mamba submodules are concatenated and then connected to the first input of element-wise addition. The output of the flattening and layer normalization submodule is connected to the second input of element-wise addition through the residual. The output of element-wise addition is connected to the regularization and layer normalization submodule and the linear projection submodule in sequence.

4. The Mamba-based multi-scale fusion skin lesion image segmentation method according to claim 3, characterized in that, The linear jump connection module is equipped with The workflow of the convolutional, linear skip connection module is as follows: through Convolution performs channel compression and linear projection on the input features of the linear skip connection module to generate the output features of the linear skip connection module.

5. The Mamba-based multi-scale fusion skin lesion image segmentation method according to claim 3, characterized in that, Edge enhancement skip connection module calculates edge enhancement features E The specific expression is: In the formula, S For the input features of the edge-enhanced skip connection module, For depthwise separable convolution, For group normalization operation, For GELU activation operation, for convolution; Edge features are fused with backbone decoding features to generate the output of the current level of the decoder. The specific expression is: In the formula, Main decoding features To reduce the learningable residual scaling factor, This is an element-wise addition.

6. The Mamba-based multi-scale fusion skin lesion image segmentation method according to claim 5, characterized in that, Adaptive skip fusion gate module calculates skip features The specific expression is: In the formula, For the input features of the adaptive jump fusion gate module, F For spatial attention, M For channel attention, For element-wise multiplication; In the formula, Use the Sigmoid activation function; In the formula, For the first learnable linear transformation, For the second learnable linear transformation, For global average pooling; The skip feature is fused with the backbone decoding feature to generate the output of the current level of the decoder. The specific expression is: In the formula, Main decoding features and For adaptive weights, , , For adaptive fusion gate factor; In the formula, For the first-level linear mapping of the fusion gate, For the second-level linear mapping of the fusion gate, For splicing operations, This is a RELU activation operation.

7. The Mamba-based multi-scale fusion skin lesion image segmentation method according to claim 3, characterized in that, In S2, the specific method for generating skin lesion image segmentation results is as follows: S21. Input the skin lesion image into the encoder. The skin lesion image is sequentially processed through the first depthwise separable convolution, the second depthwise separable convolution and the third depthwise separable convolution in the shallow feature extraction stage of the encoder to extract shallow features, generating the first shallow feature, the second shallow feature and the third shallow feature respectively. The third shallow feature is sequentially processed through the first LERM module, the second LERM module and the third LERM module in the deep feature extraction stage of the encoder to extract deep features, generating the first deep feature, the second deep feature and the bottom feature respectively. S22. Input the low-level features into the decoder. The low-level features pass through the fourth LERM module, the fifth LERM module and the sixth LERM module of the deep feature decoding stage of the decoder in sequence to generate the first decoded feature, the second decoded feature and the third decoded feature, respectively. Among them, the skip feature is calculated based on the second deep feature by the adaptive skip fusion gate module, the second decoded feature is generated by fusing the skip feature with the first decoded feature by the fifth LERM module, the skip feature is calculated based on the first deep feature by the adaptive skip fusion gate module, and the third decoded feature is generated by fusing the skip feature with the second decoded feature by the sixth LERM module. S23. The third decoded feature is sequentially processed by the fourth, fifth, and sixth depthwise separable convolutions in the shallow feature decoding stage of the decoder to generate the fourth, fifth, and sixth decoded features, respectively. Specifically, the edge enhancement jump connection module calculates the edge enhancement feature based on the third shallow feature, and the fourth depthwise separable convolution fuses the edge enhancement feature with the third decoded feature to generate the fourth decoded feature. The edge enhancement jump connection module calculates the edge enhancement feature based on the second shallow feature, and the fifth depthwise separable convolution fuses the edge enhancement feature with the fourth decoded feature to generate the fifth decoded feature. The linear jump connection module calculates the output feature based on the first shallow feature, and the sixth depthwise separable convolution fuses the output feature with the fifth decoded feature to generate the high-resolution feature. S24. The high-resolution features are mapped to the target category channel through 1×1 convolution, and restored to the resolution of the skin lesion image through bilinear interpolation. The pixel-level segmentation probability map is then activated by Sigmoid and used as the skin lesion image segmentation result.

Images