Mama-based edge information and attention mechanism skin focus segmentation method

Abstract

The invention discloses a skin focus segmentation method based on Mama edge information and an attention mechanism, and the method comprises the steps: S1, obtaining a data set, and dividing the data set; s2, connecting an encoder and a decoder in a jumping connection mode, and introducing edge information and an attention mechanism to construct an SMEANet network for skin focus segmentation; training parameters are set, and model training is carried out to obtain an optimal model; and S3, inputting the to-be-detected data into the Mama-based edge information and attention mechanism skin focus segmentation model to realize segmentation, and finally outputting a segmentation result. According to the segmentation method, the encoder and the decoder are connected in a jumping connection mode, edge information and an attention mechanism are introduced, the SMEANet network for skin focus segmentation is constructed, the accuracy of skin focus segmentation is improved, and therefore a more reliable basis is provided for clinical diagnosis.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image segmentation, and particularly relates to a skin lesion segmentation method based on edge information and attention mechanism of Mamba. BACKGROUND

[0002] In recent years, skin cancer has become one of the most common and serious diseases affecting human health worldwide. Although the mortality rate of malignant skin cancer is high, according to incomplete statistics, 95% of patients can reduce the mortality rate through early diagnosis, and the average survival rate is increased by 5 years. Skin lesion segmentation is crucial in medical diagnosis, and through reliable medical image segmentation methods, medical professionals can quickly grasp the shape and detailed features of the patient's lesion area, laying the foundation for more accurate diagnosis and treatment decisions. However, the complexity of skin lesion boundary blur, irregular texture lesions and other complex problems makes it challenging to accurately segment skin lesions, and these complexities make it difficult to distinguish them from normal skin tissue, requiring advanced segmentation techniques to accurately identify and distinguish various skin lesions.

[0003] Currently, for medical image segmentation, especially skin lesion and melanoma image segmentation, most deep semantic-based segmentation methods are based on U-Net network architecture and extended models on top of U-Net architecture. The existing technology has poor effect in shape segmentation and edge segmentation of cancerous skin. Due to the problems of hair covering and light color of lesions in skin, the cancerous skin part cannot be completely segmented. Some researchers first proposed a U-shaped medical image segmentation network. Some researchers proposed a Mamba-based medical image segmentation model VM-UNet, which captures extensive context information through visual state space blocks. Some researchers proposed a Res-Unet residual module, which improves the benchmark of melanoma image segmentation tasks. Some researchers designed a network model, which applies attention modules to the up-sampling stage of the decoder phase, allowing it to fully capture location information while enhancing edge contour features, achieving precise segmentation of target regions. Some researchers designed feature encoding in horizontal and vertical directions to capture location information and strengthen edge information by calculating the offset between location features and input features, improving the model's ability to accurately segment edge contours. However, these networks cannot accurately segment the lesion skin at the cancerous edge part, and the shape of the segmented lesion skin cannot achieve good results for some lesions with similar colors to normal skin.

[0004] Chinese patent document CN119832011A discloses a medical image segmentation method based on wavelet transform and attention mechanism, belonging to the image segmentation technical field of medical image processing. It solves the technical problem of traditional CNN receptive field limitation that cannot capture global features and semantic gap between peer encoder-decoder features. The technical solution is as follows: Step one, pre-process the input polyp medical image; Step two, build an encoder-decoder network based on wavelet transform and attention mechanism; Step three, input the pre-processed polyp image dataset into the encoder-decoder network based on wavelet transform and attention mechanism for training, and retain the optimal model; Step four, after training, input the polyp medical image validation set into the obtained optimal model. The detection model detects the lesion segmentation effect in the polyp medical image validation set. The attention mechanism applied in this technical solution realizes selective transmission and enhancement of information through dynamic weight allocation strategy, making the fusion between features more efficient and also alleviating the semantic gap between features.

[0005] Chinese patent document CN119479021A discloses a lightweight fundus blood vessel image segmentation model based on Mamba, including the following steps: the encoder and decoder include at least four layers of SlimPVM Block; the input feature map is gradually down-sampled through maximum pooling operation; the CCABottleneck module calculates the cross-channel self-attention in the height and width dimensions of the feature map; the encoder and decoder perform feature fusion through selective skip connection, and the selective skip connection selects features through channel attention mechanism; the decoder refines the up-sampled features using SlimPVM Block, the model recovers the lost spatial dimensions through convolution transpose operation, and combines the encoded features from the selective skip connection; the decoder finally outputs a 1x1 convolution layer to generate the final segmentation mask, which can accurately depict the target region boundary in the medical image. Mamba in this technical solution reduces model parameters and reduces computational overhead by splitting channels, but this will cause the segmentation accuracy to decrease, and the CN119479021A model lacks the ability to extract edge information.

[0006] Chinese patent document CN119579627A discloses a prostate MRI image segmentation method based on Mamba Unet, including a Mamba Unet with 6 layers of structure, which is mainly composed of an encoder, a decoder and a jump connection, wherein the shallow feature extraction of the first 3 layers uses a convolution module (Conv Block), each layer includes a standard convolution and a 3x3 convolution kernel and a maximum pooling operation, a novel parallel visual Mamba layer (PVM Layer structure) is introduced in the 4th to 6th layers, the decoder part keeps the same setting as the encoder, in order to extract the edge detail information of the lesion in the MRI image, the 2nd layer output and the 6th layer output of the encoder are introduced into the improved edge-aware module (Parallel Edge-aware Module, PEAM) module to realize effective extraction of shallow boundary information and deep boundary information, then the improved edge feature guidance module (Parallel Edge-guidance Feature Module, PEFM) is introduced to fuse the output of PEAM with the output of each encoder to form a parallel feature fusion structure, the jump connection uses spatial attention bridge (Spatial Attention Bridge, SAB) module and channel attention bridge (Channel Attention Bridge, CAB) module for multi-level and multi-scale information fusion and is connected with the decoder. In this technical solution, edge information is also considered to be extracted, but the global information extracted by Mamba at the deep layer is lacking by splitting the channels. Although PEAM module is used to extract edge information, this method only considers the first layer and the last layer of the encoder, and the model is still lacking in edge information extraction.

[0007] The Chinese patent document CN119785019A discloses a Mamba-based image semantic segmentation method and system, which comprises: inputting an original image into a feature extraction module to extract features, wherein the feature extraction module Mception increases a plurality of Mamba sub-modules on the basis of an Xception network; inputting the result of the first down-sampling of the extracted features into an improved RASPP module to enhance spatial context information fusion, and then summarizing global features through an R-Mamba module to enhance the long-distance dependence relationship of the image; inputting the result of the second down-sampling of the extracted features into an improved RASPP; performing up-sampling on the result of the processing after the second down-sampling to restore the size; splicing and fusing the result, adjusting the number of channels, and then performing up-sampling to restore the image size to output a predicted image. In this technical solution, on the one hand, Mamba is used to extract global information, and on the other hand, an improved RASPP is used to extract local multi-scale information, but this method ignores the importance of local edge information and directly up-samples four times after splicing the extracted feature information together, which easily causes a large amount of feature information to be lost and has a large error with the final correct prediction result.

[0008] However, the above existing segmentation methods cannot completely segment the cancerous skin part, have poor segmentation effect, and the segmentation accuracy still needs to be improved.

[0009] Therefore, there is a need for a skin lesion segmentation method based on Mamba edge information and attention mechanism, which aims to improve the accuracy of skin lesion segmentation and thus provide more reliable basis for clinical diagnosis. SUMMARY

[0010] The technical problem to be solved by the present application is to provide a skin lesion segmentation method based on Mamba edge information and attention mechanism, which adopts a skip connection mode to connect an encoder and a decoder, introduces edge information and attention mechanism, constructs a SMEANet network for skin lesion segmentation, improves the accuracy of skin lesion segmentation, and thus provides more reliable basis for clinical diagnosis.

[0011] To solve the above technical problems, the technical scheme adopted by the present application is that the skin lesion segmentation method based on Mamba edge information and attention mechanism comprises the following steps:

[0012] S1: obtaining a data set and dividing the data set;

[0013] S2: adopting a skip connection mode to connect an encoder and a decoder, introducing edge information and attention mechanism to construct a SMEANet network for skin lesion segmentation; setting training parameters, performing model training, and obtaining an optimal model, i.e., a skin lesion segmentation model based on Mamba edge information and attention mechanism;

[0014] S3: input the to-be-detected data into the skin lesion segmentation model based on the edge information and attention mechanism of Mamba, realize segmentation, and finally output the segmentation result.

[0015] By adopting the technical scheme, the encoder and the decoder are connected in a skip connection manner, the edge information and the attention mechanism are introduced to construct the SMEANet network for skin lesion segmentation, that is, a parallel two-way architecture is adopted to extract overall structure information and edge detail information, then the Mamba model is used to further extract feature information, and then the cross-channel spatial attention module is used to fuse image features, so as to improve the sensitivity of the model to the channel dimension and the spatial dimension, and reduce the semantic gap between the feature maps of the encoder and the decoder. The skin lesion segmentation method based on the edge information and the attention mechanism of Mamba improves the accuracy of skin lesion segmentation, thereby providing a more reliable basis for clinical diagnosis.

[0016] Preferably, in the step S1, the data set is divided into a training set and a validation set in a ratio of 8:2.

[0017] Preferably, in the step S2, the encoder in the SMEANet network for skin lesion segmentation is composed of four stages, including a first editing unit, a second editing unit, a third editing unit and a fourth editing unit, each of the first editing unit, the second editing unit, the third editing unit and the fourth editing unit includes an overall structure information extraction module and an edge detail information extraction module, and a Mamba scanning module and a down-sampling layer; the encoder is used to receive a to-be-segmented image and extract features of the to-be-segmented image; a total of four stages of coding are performed, in the encoder, the feature information of the image is extracted in parallel through the overall structure information extraction module and the edge detail information extraction module to extract overall shape information and edge detail information, respectively, and then the Mamba scanning module is used to further extract features by using the Mamba state space selective scanning mechanism, so that the encoder more fully extracts feature information in each layer of editing unit.

[0018] Preferably, in the step S2, the decoder in the SMEANet network for skin lesion segmentation is used to restore image details and finally output an image segmentation result; the decoder is composed of three stages, including a first decoding unit, a second decoding unit and a third decoding unit,

[0019] The first decoding unit, the second decoding unit and the third decoding unit each include a residual module and an up-sampling layer, and a cross-channel spatial attention module at the skip connection and the up-sampling connection of each layer decoding unit of the decoder. The cross-channel spatial fusion module is introduced to improve the sensitivity of the model to the channel spatial dimension and narrow the semantic gap between the encoder and the decoder feature maps.

[0020] Preferably, the first editing unit is skip-connected with the third decoding unit; the second editing unit is skip-connected with the second decoding unit; and the third editing unit and the fourth editing unit are each skip-connected with the first decoding unit.

[0021] Preferably, the overall structure information extraction module in the encoder in the step S2 is used to map the input feature from the spatial domain to the frequency domain by using the discrete Fourier transform (DFT). After the DFT operation, the low-frequency component is initially located at the four corners of the frequency domain graph, and then is moved to the center of the frequency domain graph to obtain a frequency center spectrum graph.

[0022] Then, a suitable adaptive radius adjustment method is set up in the frequency center spectrum graph to set a cutoff frequency to allow the low-frequency component to pass and hinder the high-frequency component from passing. Then, the frequency domain feature is mapped back to the spatial domain by using the inverse discrete Fourier transform (IDFT) to obtain the overall shape information.

[0023] The formula of the adaptive radius adjustment is as follows:

[0024]

[0025] wherein X mn represents a center spectrum graph, represents a low-frequency component filtered out, (m, n) is a coordinate of an arbitrary point, and r is defined as the distance from (m, n) to (0, 0).

[0026] wherein the formula of the discrete Fourier transform (DFT) and the inverse discrete Fourier transform (IDFT) are as follows:

[0027]

[0028] where m and n are the frequency component coordinates in horizontal and vertical directions respectively, H and W are the height and width of the input signal respectively; h and w are the frequency component indices in vertical and horizontal directions respectively; i is the imaginary unit; f(m, n) represents the result of converting from spatial domain to frequency domain using discrete Fourier transform, and F(h, w) represents the result of converting from frequency domain to spatial domain using inverse discrete Fourier transform. The model extracts overall shape information by introducing an adaptive low-pass filter to enhance the ability of Mamba to extract global information and further improve the segmentation ability of the model. At the same time, the model considers each layer of the encoder, which can more fully extract edge detail information.

[0029] Preferably, the edge detail information extraction module in the encoder of the step S2 is a classic Sobel operator for extracting edge information, which extracts edge profile information in the horizontal direction and the vertical direction of the feature map respectively, and finally performs a summation operation to output the value in the feature map; the edge detail extraction module integrates the extracted gradient information into the convolution operation, so that the model can focus on the edge features of the lesion while learning the features; the input feature map passes through the first convolution module, which adopts 3x3 deep convolution processing to obtain a lesion texture feature map; on the other hand, the input feature information is extracted into horizontal edge features and vertical edge features through horizontal edge detection and vertical edge detection respectively, and the two are added to obtain enhanced feature information, which is further extracted through the second convolution module, the batch normalization layer and the activation layer;

[0030] The formulas for obtaining the horizontal and vertical directions through the Sobel operator are:

[0031]

[0032] where I in is the input feature information, f sobelX represents the extraction of horizontal edge information, f sobelY represents the extraction of vertical edge information, and I sobelXY represents the extracted edge profile information.

[0033] The formula of the edge detail information extraction module is:

[0034] I din = f dc3 (I in );

[0035] I en = I din + I sobelXY ;

[0036] I out = f r (f bn(f dc3 (I en )));

[0037] where f dc3 represents a 3x3 depthwise convolution, f bn represents a Batch Normalization regularization, f r represents a ReLU activation function, I in is the input feature information, I out is the output feature information, I din and I en are intermediate processes.

[0038] Using the Sobel operator can segment more accurate edge contours, and introducing low-frequency information can further improve the ability of Mamba to capture all overall shape information, so that the model can obtain better segmentation performance.

[0039] Preferably, the mutual relationship of the encoder and decoder feature maps in the channel and spatial dimensions is fused by the cross-channel spatial attention module in the step S2, and the specific steps are as follows:

[0040] First, the information from the skip connection and upsampling is pooled and then added through the first convolution module, and then multiplied by the feature information from the skip connection through the sigmoid activation function, thereby fusing the information from the skip connection and upsampling and also strengthening the channel feature information from the skip connection;

[0041] Then, the information from the skip connection and upsampling is added after passing through the second convolution module, batch normalization layer and activation layer, and then passing through the second convolution module, and the feature mapping is multiplied by the information from the upsampling through the sigmoid activation function; in this way, the information from the skip connection and upsampling is fused, and the spatial feature information from the upsampling is also strengthened;

[0042] The formula of the cross-channel spatial attention module is as follows:

[0043] f CAM =f sk ×σ(f1(avgpool(f up ))+f1(avgpool(f sk )));

[0044] f SAM =f up ×σ(f2(f2(f up )+f2(f sk )));

[0045] where f CAM and fSAM respectively represent the channel attention output feature map and the spatial attention output feature map, f sk and f up respectively are the input feature map from the jump connection and the up-sampled input feature map, f1 represents the first convolutional module, f2 represents the second convolutional module, avgpool represents the average pooling; and sigma is the sigmoid activation function.

[0046] The formula of the sigmoid activation function S(x) is:

[0047]

[0048] wherein x represents an input value, which is a scalar, a vector or a matrix, and the sigmoid activation function maps the input value to the interval (0, 1).

[0049] Preferably, the specific step of training the model in step S2 using the training set is:

[0050] S21: Adjust the image size and perform data augmentation by random flipping and random rotation;

[0051] S22: Use the AdamW optimizer with an initial learning rate of 0.0001; set the training parameters and the number of training iterations, input the images of the training set into the constructed model for training, and obtain the model with the minimum loss loss obtained by the loss function as the training model result of the network and save it;

[0052] S23: Input the images of the validation set into the trained model for verification, thereby obtaining the skin lesion segmentation model based on the edge information and attention mechanism of Mamba.

[0053] Data augmentation on the training set not only improves the robustness and generalization ability of the model, but also increases the attention to target details, making the model more adaptable to slight changes in data distribution.

[0054] Preferably, the loss function used in step S22 is the BceDice Loss function, and the formula is:

[0055] BceDiceLoss = BceLoss + DiceLoss;

[0056] BceLoss = -[ylog(p) + (1-y)log(1-p)];

[0057]

[0058] Wherein, y represents a real label, and is 0 or 1; p represents a prediction probability, and is between 0 and 1; |X| represents an actual label of a segmentation graph; |Y| represents a predicted segmentation graph; |X intersection Y| represents an intersection of a predicted value and a label value.

[0059] Compared with the prior art, the application has the beneficial effects that:

[0060] (1) The encoder and the decoder are connected in a skip connection mode, edge information and attention mechanism are introduced, and the SMEANet network for skin lesion segmentation is constructed, that is, parallel two-way architecture is used to extract overall structure information and edge detail information, and then the Mamba model is used to further extract feature information, so that the accuracy of lesion segmentation is improved; based on the unet architecture, the dimensions are restored by layer-by-layer upsampling, the skip connection compensates for the missing information, and the cross-channel spatial attention module arranged at the junction of the skip connection and the upsampling is used to improve the sensitivity of the model to the channel and spatial dimensions;

[0061] (2) The Sobel operator is introduced to extract edge information while extracting local information, which helps the model to segment more accurate edge contours; meanwhile, the cross-channel spatial attention module is introduced to improve the sensitivity of the model to the channel and spatial dimensions, and the semantic gap between the feature maps of the encoder and the decoder is narrowed. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 It is a flowchart of the skin lesion segmentation method based on the edge information and attention mechanism of Mamba of the application;

[0063] Figure 2 It is a structure diagram of the SMEANet network for skin lesion segmentation in the skin lesion segmentation method based on the edge information and attention mechanism of Mamba of the application;

[0064] Figure 3 It is a flowchart of the overall structure information extraction module in the skin lesion segmentation method based on the edge information and attention mechanism of Mamba of the application;

[0065] Figure 4 It is a flowchart of the edge detail information extraction module in the skin lesion segmentation method based on the edge information and attention mechanism of Mamba of the application;

[0066] Figure 5 It is a flowchart of the cross-channel spatial attention module in the skin lesion segmentation method based on the edge information and attention mechanism of Mamba of the application;

[0067] Figure 6A comparison result diagram of segmentation results of each model in the skin lesion segmentation method based on the Mamba-based edge information and attention mechanism of the application on the ISIC2018 dataset is visualized. DETAILED DESCRIPTION

[0068] The embodiments of the application are described in detail below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the application, and cannot be used to limit the protection scope of the application.

[0069] Embodiment: As shown in the figure, the skin lesion segmentation method based on the Mamba-based edge information and attention mechanism specifically includes the following steps: Figure 1

[0070] S1: Obtain a dataset and divide the dataset; in the step S1, the dataset is randomly divided into a training set and a validation set in a ratio of 8:2;

[0071] S2: adopt a skip connection method to connect an encoder and a decoder, introduce edge information and an attention mechanism to construct a SMEANet network for skin lesion segmentation; set training parameters, perform model training, and obtain an optimal model, i.e., a skin lesion segmentation model based on the Mamba-based edge information and attention mechanism;

[0072] As shown in the figure, in the step S2, the encoder in the SMEANet network for skin lesion segmentation is composed of four stages, including a first editing unit, a second editing unit, a third editing unit, and a fourth editing unit, the first editing unit, the second editing unit, the third editing unit, and the fourth editing unit all include an overall structure information extraction module and an edge detail information extraction module, as well as a Mamba scanning module and a down-sampling layer; the encoder is used to receive an image to be segmented and perform feature extraction on the image to be segmented; a total of four stages of encoding, in the encoder, the feature information of the image is extracted in parallel through the overall structure information extraction module and the edge detail information extraction module to extract overall shape information and edge detail information, respectively, and then further feature extraction is performed through the Mamba scanning module using a Mamba state space selective scanning mechanism, so that the encoder more fully extracts feature information in each layer of the editing unit; Figure 2 As shown in the figure, in the step S2, the decoder in the SMEANet network for skin lesion segmentation is used to restore image details and finally output an image segmentation result; the decoder is composed of three stages, including a first decoding unit, a second decoding unit, and a third decoding unit,

[0073] Figure 2

[0074] ​​​The first decoding unit, the second decoding unit, and the third decoding unit each include a residual module and an upsampling layer, as well as a cross-channel spatial attention module at the skip connection and upsampling connection of each decoding unit in the decoder; wherein the cross-channel spatial fusion module is used to improve the model's sensitivity to the channel spatial dimension and narrow the semantic gap between the encoder and decoder feature maps;

[0075] The first editing unit is jump-connected to the third decoding unit; the second editing unit is jump-connected to the second decoding unit; the third editing unit and the fourth editing unit are both jump-connected to the first decoding unit;

[0076] like Figure 3 As shown, the overall structure information extraction module in the encoder in step S2 uses discrete Fourier transform DFT to convert the input feature map from the spatial domain to the frequency domain. After the DFT operation, the low-frequency components are initially located at the four corners of the frequency domain graph, and then moved to the center of the frequency domain graph to obtain a frequency center spectrum graph;

[0077] Then, a suitable adaptive radius adjustment method is established in the frequency center spectrum graph, setting the cutoff frequency to allow low-frequency components to pass through and block the passage of high-frequency components; then, the frequency domain feature map is converted back to the spatial domain using the inverse discrete Fourier transform (IDFT) to obtain the overall shape information;

[0078] The formula for adaptive radius adjustment is expressed as:

[0079]

[0080] Among them, X mn represents the central spectrum, Indicates filtering out the low-frequency components, (m, n) is the coordinate of any point, and r is defined as the distance from (m, n) to (0, 0);

[0081] The formulas for discrete Fourier transform DFT and inverse discrete Fourier transform IDFT are:

[0082]

[0083] Where m and n correspond to the horizontal and vertical frequency component coordinates, respectively; H and W are the height and width of the input signal, respectively; h and w represent the vertical and horizontal frequency component indices, respectively; i is an imaginary unit; f(m, n) represents the result of converting from the spatial domain to the frequency domain using discrete Fourier transform, and F(h, w) represents the result of converting from the frequency domain to the spatial domain using inverse discrete Fourier transform;

[0084] like Figure 4As shown, the edge detail information extraction module in the encoder of the step S2 is a classic Sobel operator for extracting edge information, which extracts edge contour information in the horizontal direction and the vertical direction of the feature map respectively, and finally performs summation operation to output the value in the feature map; the edge detail extraction module integrates the extracted gradient information into the convolution operation, so that the model can focus on the edge features of the lesions while learning the features; the input feature map passes through the first convolution module, and the first convolution module adopts 3x3 deep convolution processing to obtain a lesion texture feature map; on the other hand, the input feature information is extracted into horizontal edge features and vertical edge features through horizontal edge detection and vertical edge detection respectively, and the two are added to obtain enhanced feature information, which is further extracted through the second convolution module, the batch normalization layer and the activation layer;

[0085] The formula for obtaining the horizontal direction and the vertical direction by the Sobel operator is:

[0086]

[0087] wherein, I in is the input feature information, f sobelX represents the extraction of horizontal edge information, f sobelY represents the extraction of vertical edge information, and I sobelXY represents the extracted edge contour information.

[0088] The formula of the edge detail information extraction module is:

[0089] I din =f dc3 (I in );

[0090] I en =I din +I sobelXY ;

[0091] I out =f r (f bn (f dc3 (I en )));

[0092] wherein, f dc3 represents 3x3 deep convolution, f bn represents Batch Normalization regularization, f r represents ReLU activation function, I in is the input feature information, I out is the output feature information, and I din and I en are intermediate processes.

[0093] As Figure 5 shown, the cross-channel spatial attention module is used to fuse the mutual relationship between the encoder and decoder feature maps in the channel and spatial dimensions in the step S2, and the specific steps are as follows:

[0094] First, the information from the skip connection and upsampling is subjected to a pooling operation, and then added after passing through the first convolution module. Then, the information is multiplied by the feature information from the skip connection after passing through the sigmoid activation function, thereby fusing the information from the skip connection and upsampling and strengthening the channel feature information from the skip connection.

[0095] Then, the information from the skip connection and upsampling is added after passing through the second convolution module, the batch normalization layer and the activation layer, and then passing through the second convolution module. The feature mapping is multiplied by the information from the upsampling after passing through the sigmoid activation function, thereby fusing the information from the skip connection and upsampling and strengthening the spatial feature information from the upsampling.

[0096] The formula of the cross-channel spatial attention module is as follows:

[0097] f CAM =f sk ×σ(f1(avgpool(f up ))+f1(avgpool(f sk )));

[0098] f SAM =f up ×σ(f2(f2(f up )+f2(f sk )));

[0099] wherein f CAM and f SAM represent the channel attention output feature map and the spatial attention output feature map, respectively, f sk and f up are the input feature map from the skip connection and the input feature map from the upsampling, respectively, f1 represents the first convolution module, f2 represents the second convolution module, avgpool represents the average pooling, and σ is the sigmoid activation function.

[0100] The formula of the sigmoid activation function S(x) is as follows:

[0101]

[0102] wherein x represents an input value, which is a scalar, a vector or a matrix, and the sigmoid activation function maps the input value to the interval (0, 1).

[0103] The specific step of training the model in step S2 is:

[0104] S21: adjust the image size, and perform data enhancement by random flipping and random rotation;

[0105] S22: use the AdamW optimizer with an initial learning rate of 0.0001; set the training parameters and the number of training iterations, input the images of the training set into the constructed model for training, and obtain the model with the minimum loss loss obtained by the loss function as the training model result of the network and save it;

[0106] S23: input the images of the verification set into the trained model for verification, thereby obtaining the skin lesion segmentation model based on the edge information and attention mechanism of Mamba;

[0107] Data augmentation on the training set not only improves the robustness and generalization ability of the model, but also increases the attention to target details, making the model more adaptable to slight changes in data distribution.

[0108] Preferably, the loss function used in step S22 is the BceDice Loss function, and the formula is:

[0109] BceDiceLoss=BceLoss+DiceLoss;

[0110] BceLoss=-[ylog(p)+(1-y)log(1-p)];

[0111]

[0112] Wherein, y represents the true label, which takes the value of 0 or 1; p represents the prediction probability, which is between 0 and 1; |X| represents the actual label of the segmentation map; |Y| represents the predicted segmentation map; |X∩Y| represents the intersection of the predicted value and the label value;

[0113] S3: input the data to be detected into the skin lesion segmentation model based on the edge information and attention mechanism of Mamba, realize segmentation, and finally output the segmentation result.

[0114] Specific embodiments: a public ISIC2018 dermoscope dataset is selected for extensive experiments; NVIDIARTX 4090 GPU is used for model training; ISIC2018 dataset contains 2594 dermoscope images, which has become the main benchmark for evaluating skin imaging algorithms; randomly divided into training set and verification set according to the predetermined proportion; 2076 pictures as training set, 518 pictures as verification set.

[0115] The specific steps of the training are as follows: first, the creation of the SMEANet model is realized in PyTorch using PyCharm software; training is performed using an NVIDIA RTX 4090 GPU; the size of the input image is adjusted to 256*256. The image data of the training set is enhanced using data enhancement techniques such as random flipping and random rotation, thereby preventing overfitting; an AdamW optimizer is used, and the initial learning rate is 0.0001; 100 rounds of training are performed when the model is trained, and the batch size is set to 16; the training parameters are set, the image input of the training set is used to construct the SMEANet network for skin lesion segmentation, the model is trained, the model with the minimum validation loss loss is saved as the training model result of the network, and then the image of the validation set is input into the SMEANet model for skin lesion segmentation obtained by training to perform verification, thereby obtaining the SMEANet model for skin lesion segmentation.

[0116] To verify the effectiveness of the integrated method, the method of the present technical solution is compared with other models, and the results are shown in Table 1. The U-Net, EGE-Unet, MiSSFormer, TransAttUnet and VM-Unet network architectures are selected as comparative methods to compare with the SMFANet network proposed in the present application.

[0117] Table 1 Comparison of segmentation results of various models and SMFANet network on ISIC2018 dataset

[0118] Network mIoU DSC ACC Spe Sen U-Net 0.7688 0.8693 0.9434 0.9715 0.8451 MiSSFormer 0.8139 0.8974 0.9555 0.9790 0.8736 EGE-UNet 0.8299 0.9070 0.9590 0.9768 0.8971 TransAttUnet 0.8071 0.8933 0.9546 0.9835 0.8536 VM-UNet 0.8136 0.8972 0.9554 0.9790 0.8732 SMFANet (the application) 0.8452 0.9161 0.9632 0.9810 0.9012

[0119] As shown in Table 1, the image segmentation accuracy (dice similarity score DSC) of the SMEANet model for skin lesion segmentation proposed in the application is 91.61%. The SMEANet model for skin lesion segmentation proposed in the application is better than the best EGE-UNet in other models, and the average intersection over union mIoU, dice similarity score DSC, accuracy ACC, specificity Spe and recall Sen are improved by 1.53%, 0.91%, 0.42%, 0.42% and 0.41% respectively. DSC and mIoU are commonly used in medical segmentation to consider the foreground and overall segmentation quality. In the classification task, ACC, Spe and Sen are often combined to balance false positives and false negatives according to the scene. Medical detection emphasizes recall Sen and controls false negatives. As can be seen, the application uses a parallel two-way architecture to extract overall structure information and edge detail information, and then further extracts feature information through the Mamba model, which greatly improves the lesion segmentation accuracy. At the same time, the cross-channel spatial attention module is used to improve the sensitivity of the model to channel dimension and spatial dimension, and to reduce the semantic gap between the encoder and the decoder feature map. Therefore, the SMEANet model for skin lesion segmentation in the method achieves superior image segmentation performance.

[0120] As shown in Figure 6 , the segmentation results of each model and the network architecture constructed by the application on the ISIC2018 dataset are compared. It can be seen that the segmentation results of the technical solution of the application have obviously improved in integrity and accuracy compared with other networks, and good results have been achieved in the accuracy of shape segmentation. The SMEANet model for skin lesion segmentation in the application can accurately identify the main shape of the lesion with very light color and difficult to distinguish from normal skin, accurately capture some edge detail features of the lesion, and accurately segment, and the segmentation result is closer to the real GroundTruth.

[0121] For those skilled in the art, the specific embodiments are only exemplary descriptions of the application, and it is obvious that the specific implementation of the application is not limited by the above method. Various non-essential improvements or direct application of the concept and technical solution of the application to other occasions without improvement are within the protection scope of the application.

Claims

1. A skin lesion segmentation method based on Mamba's edge information and attention mechanism, characterized in that: The specific steps include: S1: Obtain the data set and divide it into parts; S2: Use skip connections to connect the encoder and decoder, introduce edge information and attention mechanism to build the SMEANet network for skin lesion segmentation; set training parameters, perform model training, and obtain the optimal model, that is, the skin lesion segmentation model based on Mamba's edge information and attention mechanism; S3: Input the data to be detected into the skin lesion segmentation model based on Mamba's edge information and attention mechanism to achieve segmentation, and finally output the segmentation result.

2. The skin lesion segmentation method based on Mamba's edge information and attention mechanism according to claim 1 is characterized in that: In step S1, the data set is divided into a training set and a validation set in proportion.

3. The skin lesion segmentation method based on Mamba's edge information and attention mechanism according to claim 2, characterized in that: The encoder constructed in the SMEANet network for skin lesion segmentation in step S2 consists of four stages, including a first editing unit, a second editing unit, a third editing unit and a fourth editing unit, wherein the first editing unit, the second editing unit, the third editing unit and the fourth editing unit all include an overall structure information extraction module and an edge detail information extraction module, as well as a Mamba scanning module and a downsampling layer; the encoder is used to receive the image to be segmented and perform feature extraction on the image to be segmented; in the encoder, the feature information of the image is respectively extracted from the overall shape information and the edge detail information through the overall structure information extraction module and the edge detail information extraction module in parallel, and then further performs feature extraction by using the Mamba state space selective scanning mechanism through the Mamba scanning module, so that the encoder can more fully extract feature information in each layer of editing unit.

4. The skin lesion segmentation method based on Mamba's edge information and attention mechanism according to claim 3, characterized in that: In the step S2, the decoder in the SMEANet network for skin lesion segmentation is constructed to restore image details and finally output the image segmentation result; the decoder consists of three stages, including a first decoding unit, a second decoding unit and a third decoding unit, and the first decoding unit, the second decoding unit and the third decoding unit each include a residual module and an upsampling layer, and a cross-channel spatial attention module at the jump connection and the upsampling connection of each layer of the decoding unit of the decoder.

5. The skin lesion segmentation method based on Mamba's edge information and attention mechanism according to claim 4, characterized in that: The first editing unit is jump-connected to the third decoding unit; the second editing unit is jump-connected to the second decoding unit; and the third editing unit and the fourth editing unit are both jump-connected to the first decoding unit.

6. The skin lesion segmentation method based on Mamba's edge information and attention mechanism according to claim 3, characterized in that: The overall structure information extraction module in the encoder of step S2 converts the input feature map from the spatial domain to the frequency domain using discrete Fourier transform (DFT). After the DFT operation, the low-frequency components are initially located at the four corners of the frequency domain graph, and then moved to the center of the frequency domain graph to obtain a frequency center spectrum graph. Then, an adaptive radius adjustment method is established in the frequency center spectrum graph to set the cutoff frequency to allow low-frequency components to pass through and block the passage of high-frequency components. After that, the frequency domain feature map is converted back to the spatial domain using the inverse discrete Fourier transform (IDFT) to obtain the overall shape information. The formula for adaptive radius adjustment is expressed as: Among them, X mn represents the central spectrum, Indicates filtering out the low-frequency components, (m, n) is the coordinate of any point, and r is defined as the distance from (m, n) to (0, 0); The formulas for discrete Fourier transform DFT and inverse discrete Fourier transform IDFT are: Where m and n correspond to the horizontal and vertical frequency component coordinates, respectively; H and W are the height and width of the input signal, respectively; h and w represent the vertical and horizontal frequency component indices, respectively; i is an imaginary unit; f(m, n) represents the result of converting from the spatial domain to the frequency domain using discrete Fourier transform, and F(h, w) represents the result of converting from the frequency domain to the spatial domain using inverse discrete Fourier transform.

7. The skin lesion segmentation method based on Mamba's edge information and attention mechanism according to claim 3, characterized in that: The edge detail information extraction module in the encoder of step S2 adopts the Sobel operator to extract edge information, extracts edge contour information in the horizontal direction and the vertical direction of the feature map respectively, and finally performs a summation operation to output the value in the feature map; the edge detail extraction module integrates the extracted gradient information into the convolution operation, and the input feature map is processed by the first convolution module to obtain a lesion texture feature map; on the other hand, the input feature information is respectively extracted by horizontal edge detection and vertical edge detection to extract horizontal edge features and vertical edge features, and the two are added to obtain enhanced feature information, and then the feature information is further extracted by the second convolution module, the batch normalization layer and the activation layer; The formula for obtaining the horizontal and vertical directions through the Sobel operator is: Among them, I in is the input feature information, f sobelX Indicates the extraction of horizontal edge information, f sobelY Indicates the extraction of vertical edge information, I sobelXY Represents the extracted edge contour information; The formula of the edge detail information extraction module is: I din =f dc3 (I in ); I en =I din +I sobelXY ; I out =f r (f bn (f dc3 (I en ))); Among them, f dc3 represents a 3×3 depth convolution, f bn represents BatchNormalization regularization, f r ReLU activation function, I in is the input feature information, I out is the output feature information, I din and I en It’s an intermediate process.

8. The skin lesion segmentation method based on Mamba's edge information and attention mechanism according to claim 3, characterized in that: In step S2, the relationship between the encoder and decoder feature maps in the channel and spatial dimensions is fused through a cross-channel spatial attention module, and the specific steps are as follows: First, the information from the skip connection and upsampling is pooled and added through the first convolution module. Then, it is multiplied with the feature information from the skip connection through the sigmoid activation function. This integrates the information from the skip connection and upsampling while also strengthening the channel feature information from the skip connection. The information from the skip connection and upsampling is then passed through the second convolution module, batch normalization layer, and activation layer respectively, and then added together and passed through the second convolution module. The feature map is multiplied by the upsampling information through the sigmoid activation function. The formula of the cross-channel spatial attention module is: f CAM =f sk ×σ(f1(avgpool(f up ))+f1(avgpool(f sk ))); f SAM =f up ×σ(f2(f2(f up )+f2(f sk ))); Among them, f CAM and f SAM They represent the channel attention output feature map and the spatial attention output feature map, respectively, and f sk and f up are the input feature maps from the jump connection and the upsampled input feature maps, respectively. f1 represents the first convolution module, f2 represents the second convolution module, avgpool represents average pooling, σ represents the sigmoid activation function, and the formula of the sigmoid activation function S(x) is: Where x represents the input value, which is a scalar, vector, or matrix. The sigmoid activation function maps the input value to the (0, 1) interval.

9. The skin lesion segmentation method based on Mamba's edge information and attention mechanism according to claim 3, characterized in that: The specific steps of using the training set to train the model in step S2 are: S21: resize the image and perform data augmentation using random flipping and random rotation; S22: Use the AdamW optimizer and the initial learning rate; set the training parameters and the number of training iterations, input the images of the training set into the constructed model for training, and save the model with the minimum loss obtained by the loss function as the training model result of the network; S23: The images of the validation set are then input into the trained model for verification, thereby obtaining a skin lesion segmentation model based on Mamba's edge information and attention mechanism.

10. The skin lesion segmentation method based on Mamba edge information and attention mechanism according to claim 9, characterized in that: The loss function used in step S22 is the BceDiceLoss function, and the formula is: BceDiceLoss=BceLoss+DiceLoss; BceLoss=-[y log(p)+(1-y)log(1-p)]; Among them, y represents the true label, which takes a value of 0 or 1; p represents the predicted probability, which is between 0 and 1; |X| represents the actual label of the segmentation map; |Y| represents the predicted segmentation map; |X∩Y| represents the intersection of the predicted value and the label value.

Images