Skin lesion image segmentation method based on edge guidance and attention mechanism

Abstract

The invention relates to the technical field of deep learning and skin lesion recognition, in particular to a skin lesion image segmentation method based on edge guidance and an attention mechanism, and the method comprises the steps: inputting a to-be-segmented dermatoscope image into an image segmentation model for segmentation, and outputting a skin lesion segmented image; wherein the image segmentation model performs edge feature extraction on a dermatoscope image through an EEM module; performing multi-scale hole convolution processing on the semantic feature map through a DASPP module; adaptive fusion of channel and space attention is carried out on the segmented feature map through a CSAM module, a channel dimension and a space dimension are processed in sequence by adopting a serial design of channel priority and space postposition, and edge information of an edge prior feature map is introduced to carry out feature enhancement. According to the invention, the marginal region of the skin lesion is subjected to refined learning, and the marginal information of the lesion region is fully utilized to realize accurate positioning of the skin lesion, so that the accuracy and effect of skin lesion image segmentation are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning and skin lesion recognition, and particularly relates to a skin lesion image segmentation method based on edge guidance and attention mechanism. BACKGROUND

[0002] In recent years, the incidence of skin diseases has shown an increasing trend year by year. Among skin diseases, skin cancer is one of the highest incidence of malignant tumors worldwide, and melanoma is of great concern due to its high metastasis and mortality. According to statistics, melanoma is the most deadly skin cancer, with about 325,000 new cases and about 57,000 deaths worldwide each year. Melanoma skin cancer (such as basal cell carcinoma, squamous cell carcinoma) has a higher incidence, with more than 3 million new cases each year. Early and accurate diagnosis is the key to improving patient survival, and dermoscopy image analysis as a non-invasive diagnostic method has become an important direction of clinical research. Since benign and malignant skin lesions are visually similar, it is a difficult process to detect skin lesions through dermoscopy images with the human eye.

[0003] Skin lesion image segmentation is an important part of dermoscopy image analysis. Through skin lesion image segmentation, the lesion area in the dermoscopy image can be accurately extracted, providing accurate lesion area information for subsequent feature extraction, classification and diagnosis.

[0004] Traditional skin lesion image segmentation methods such as threshold-based segmentation, edge detection, region growing, etc. can meet the needs of simple segmentation to some extent, but have many limitations. First, skin lesion images may have varying degrees of noise and uneven illumination during actual acquisition, which can significantly reduce the segmentation accuracy of traditional methods. Traditional segmentation methods require high image quality and are easily affected by noise, illumination changes and image distortion. Second, for lesions with unclear boundaries or irregular shapes, traditional methods often cannot accurately segment the lesion area. In addition, traditional methods have low computational efficiency and are difficult to meet the needs of real-time diagnosis.

[0005] With the increasing amount of medical data, there is an increasing demand for efficient and accurate image segmentation algorithms. Therefore, it is of great theoretical significance and practical value to research and develop more advanced segmentation methods, such as deep learning-based segmentation algorithms. Deep learning methods can automatically learn features in images, adapt to complex image changes, improve segmentation accuracy and efficiency, and provide stronger support for skin lesion diagnosis and treatment. In recent years, medical image segmentation technology based on deep learning has made significant progress, and UNet structure has attracted attention in medical image segmentation due to its encoder-decoder architecture and skip connection design.

[0006] However, the standard UNet is often difficult to fully capture subtle edges and multi-scale information, resulting in limited segmentation effect. Therefore, many improved models have been proposed, such as UNet++, AttentionUNet, UNet3+, etc. These models optimize the segmentation performance to some extent through multi-scale feature fusion and attention mechanism. The applicant found in actual research that for dermoscopy images, the edge information of the lesion area is particularly important, as it not only directly affects the accurate positioning of the lesion, but also relates to the fine analysis of subsequent diagnosis. However, existing methods often ignore the full use of edge information. Therefore, how to design a skin lesion image segmentation method that can fully utilize the edge information of the lesion area is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0007] In view of the above problems of the prior art, the technical problem to be solved by the present application is to provide a skin lesion image segmentation method based on edge guidance and attention mechanism, which can fully utilize the edge information of the lesion area to realize accurate positioning of the skin lesion, thereby improving the accuracy and effect of skin lesion image segmentation.

[0008] To solve the above technical problems, the present application adopts the following technical solutions:

[0009] A skin lesion image segmentation method based on edge guidance and attention mechanism, comprising:

[0010] S1: obtaining a dermoscopy image to be segmented;

[0011] S2: inputting the dermoscopy image to be segmented into an image segmentation model for segmentation, and outputting a skin lesion segmentation image;

[0012] The processing steps of the image segmentation model include:

[0013] S202: using a pre-trained ResNet34 as an encoder backbone to extract features from the input dermoscopy image using a plurality of cascaded down-sampling modules, each down-sampling module outputting a corresponding semantic feature map of a certain scale;

[0014] S203: extracting edge features from the input dermoscopy image through an EEM module to obtain an edge prior feature map; performing level-by-level down-sampling on the edge prior feature map to obtain an edge prior feature map corresponding to each scale of the semantic feature map;

[0015] S204: The semantic feature map corresponding to the scale and the edge prior feature map are weighted and fused to obtain an edge semantic fusion feature map corresponding to the scale; wherein the edge information of the edge prior feature map is highlighted through the edge gate, and then weighted and fused with the semantic feature map, and then the semantic feature expression is enhanced through the attention gate to obtain the edge semantic fusion feature map;

[0016] S205: The semantic feature map output by the last down-sampling module of the encoder is processed by a multi-scale hollow convolution in a DASPP module to obtain an enhanced feature map.

[0017] S206: The enhanced feature map is layer by layer up-sampled and feature restored by the decoder using a plurality of cascaded up-sampling modules, and each up-sampling module outputs a segmentation feature map corresponding to the scale; during the up-sampling process, each up-sampling module of the decoder is jump-connected with the edge semantic fusion feature map corresponding to the scale.

[0018] S207: The segmentation feature map output by the last up-sampling module of the decoder is processed by a CSAM module to adaptively fuse channel and spatial attention; the CSAM module adopts a serialized design of channel first and spatial last, processes the channel dimension and the spatial dimension in turn, and introduces the edge information of the edge prior feature map through an edge guided feature fusion module for feature enhancement, and finally generates an optimized fusion feature map through residual connection.

[0019] S208: The fusion feature map is up-sampled and feature restored to obtain a skin lesion segmentation image.

[0020] Preferably, the encoder includes an initial convolutional layer, a first stage down-sampling module, a second stage down-sampling module, a third stage down-sampling module, and a fourth stage down-sampling module in cascade; wherein the semantic feature map output by the fourth stage down-sampling module is input to the DASPP module.

[0021] The decoder includes a first stage up-sampling module, a second stage up-sampling module, a third stage up-sampling module, and a fourth stage up-sampling module in cascade; wherein the semantic feature map output by the third stage down-sampling module, the second stage down-sampling module, the first stage down-sampling module, and the initial convolutional layer is jump-connected to the first stage up-sampling module, the second stage up-sampling module, the third stage up-sampling module, and the fourth stage up-sampling module, respectively, after being fused with the semantic feature map corresponding to the scale to obtain the edge semantic fusion feature map corresponding to the scale.

[0022] Preferably, in step S203, the processing steps of the EEM module include:

[0023] S2031: The skin scope image is processed by a learnable edge path to extract edge features and obtain a first edge feature map E learned ;

[0024] Among them, the learnable edge path includes a cascaded first convolution combination module, a second convolution combination module, a third convolution combination module, a 3×3 convolution layer and a 1×1 convolution layer; the first convolution combination module includes a cascaded 3×3 convolution layer, a batch normalization layer and a ReLU function, the second convolution combination module includes a cascaded 3×3 dilated convolution layer with a dilation rate of 2, a batch normalization layer and a ReLU function, and the third convolution combination module includes a cascaded 3×3 dilated convolution layer with a dilation rate of 4, a batch normalization layer and a ReLU function;

[0025] The specific formula is as follows:

[0026] f c =Conv 3×3 (Conv 3×3,dil＝ (Conv 3×3,dil＝2 (Conv 3×3 (x))));

[0027] Where: x represents the input feature map, x∈R C×H×W ; In order to optimize the efficiency of gradient propagation and retain the original feature information, the residual connection mechanism can be introduced into the learning edge path to convert f c The enhanced features are added to the residual of the input feature map x, and then a 1×1 convolution layer and Sigmoid activation are used to project the enhanced features into a single-channel learnable edge response to obtain the edge features learned by the deep convolutional network. The formula is expressed as:

[0028] E learned =σ(Conv 1×1 (f c +x));

[0029] Where: σ is the activation function;

[0030] S2032: Perform edge detection on the dermoscopic image using the Sobel operator to obtain a second edge feature map E sobel ;

[0031] S2033: Perform edge detection on the dermatoscope image using the Laplacian operator to obtain the third edge feature map E laplacian ;

[0032] S2034: Perform edge detection on the dermoscopic image using the Canny operator to obtain a fourth edge feature map E canny ;

[0033] S2035: First edge feature map E learned , the second edge feature map E sobel , the third edge feature map E laplacian And the fourth edge feature map Ecanny The weighted calculation is performed to obtain an edge prior feature map E out ;

[0034] The formula is expressed as:

[0035]

[0036] In the formula, a is an adaptive weight parameter, a [0, 1]; E learned is a first edge feature map learned through a deep convolutional network; E sobel , E laplacian , E canny are a second edge feature map, a third edge feature map and a fourth edge feature map detected through Sobel operators, Laplacian operators and Canny operators respectively.

[0037] Preferably, in step S2035, the input feature map is input into a 1x1 convolutional layer for processing by using a spatial adaptive weight map generation mechanism to obtain a weight map a, so as to balance the contribution degrees of the learning edge and the classical operator at a pixel level.

[0038] Preferably, the edge prior feature map output by the EEM module is down-sampled layer by layer to obtain edge prior feature maps corresponding to the semantic feature maps output by the initial convolutional layer, the first stage down-sampling module, the second stage down-sampling module and the third stage down-sampling module of the encoder.

[0039] Preferably, in step S204, the following steps are included:

[0040] S2041: Obtain the input semantic feature map and the edge prior feature map of the corresponding scale;

[0041] S2042: Input the edge prior feature map into an edge gate, extract weight information through convolution operation, and map the weight value to the interval of 0-1 by using an activation function to generate a weight mask; the edge prior feature map is weighted through the weight mask to highlight important edge information;

[0042] S2043: The weighted edge prior feature map is weighted and fused with the semantic feature map to generate a preliminary fusion feature map;

[0043] S2044: The preliminary fusion feature map is input into an attention gate, the semantic features are mapped through convolution operation, important feature regions are selected by an activation function, the semantic feature expression is enhanced, and an edge semantic fusion feature map focusing on the key regions of the target object is generated.

[0044] Preferably, in step S205, the processing steps of the DASPP module are as follows:

[0045] S2051: input the input feature map X into the first branch to perform processing by cascading a 1x1 convolutional layer, a BN layer and a ReLU function, to obtain a first feature map b1;

[0046] S2052: input the input feature map X into the second branch to perform processing by cascading a 3x3 dilated convolutional layer with a dilated rate of 2, a BN layer and a ReLU function, to obtain a second feature map b2;

[0047] S2053: input the input feature map X into the third branch to perform processing by cascading a 3x3 dilated convolutional layer with a dilated rate of 4, a BN layer and a ReLU function, to obtain a third feature map b3;

[0048] S2054: input the input feature map X into the fourth branch to perform processing by cascading a 3x3 dilated convolutional layer with a dilated rate of 6, a BN layer and a ReLU function, to obtain a fourth feature map b4;

[0049] S2055: input the input feature map X into the fifth branch to perform processing by cascading a 1x1 global average pooling layer, a 1x1 convolutional layer, a BN layer, a ReLU function and a bilinear interpolation upsampling layer, to obtain a fifth feature map b5;

[0050] S2056: after splicing the first feature map b1, the second feature map b2, the third feature map b3, the fourth feature map b4 and the fifth feature map b5, input them into the first convolutional combination block and the second convolutional combination block in sequence for processing, and then perform residual addition with the input feature map X, to obtain an enhanced feature map; wherein the first convolutional combination block includes a cascaded 1x1 convolutional layer, a BN layer and a ReLU function; and the second convolutional combination block includes a cascaded 1x1 convolutional layer and a BN layer;

[0051] The formula is:

[0052] F out =Conv 1×1 (Conv 1×1 (Concat(b1,b2,b3,b4,b5)))+X;

[0053] In the formula, F out represents the enhanced feature map; and Concat represents a splicing operation.

[0054] Preferably, in step S207, the processing steps of the CSAM module include:

[0055] S2071: performing multi-scale channel attention feature extraction on the input feature map X by a multi-scale channel attention module, to obtain a channel feature map y c ;

[0056] S2072: Perform deformable spatial attention feature extraction on the channel feature map y c to obtain a channel spatial feature map y s ;

[0057] S2073: Perform feature fusion on the channel spatial feature map y s and the edge prior feature map to obtain an edge channel spatial fusion feature map y fuse ;

[0058] S2074: After performing 1x1 convolution processing on the input feature map X, perform element-wise addition with the edge channel spatial fusion feature map y fuse to obtain the final fusion feature map.

[0059] The formula is:

[0060] y = ReLU(Conv 1×1 (X) + y fuse );

[0061] wherein y represents the final fusion feature map; y fuse represents the edge channel spatial fusion feature map.

[0062] Preferably, in step S2071, the processing steps of the multi-scale channel attention module include:

[0063] S20711: Perform processing on the input feature map X through a 1x1 adaptive average pooling layer to obtain a feature map F 1×1 ;

[0064] The formula is:

[0065] F 1×1 = Adaptive AvgPool 1×1 (X);

[0066] wherein Adaptive AvgPool 1×1 represents a 1x1 adaptive average pooling layer;

[0067] S20712: After performing processing on the input feature map X through a 2x2 adaptive average pooling layer, perform nearest neighbor interpolation up-sampling to obtain a feature map F 2×2 ;

[0068] The formula is:

[0069] F 2×2 = UP(Adaptive AvgPool 2×2 (X));

[0070] Adaptive AvgPool 2×2 denotes a 2x2 adaptive average pooling layer; UP denotes nearest neighbor interpolation up-sampling;

[0071] S20713: The input feature map X is processed through a 4x4 adaptive average pooling layer, and then nearest neighbor interpolation up-sampling is performed to obtain a feature map F 4×4 ;

[0072] The formula is:

[0073] F 4×4 = UP(Adaptive AvgPool 4×4 (X));

[0074] Adaptive AvgPool 4×4 denotes a 4x4 adaptive average pooling layer;

[0075] S20714: The input feature map X is processed through a cascaded 3x1 convolution layer, 1x3 convolution layer, BN layer and ReLU function to obtain a feature map F dir ;

[0076] The formula is:

[0077] F dir = ReLU(BN(Conv 1×3 (Conv 3×1 (X))));

[0078] S20715: The feature map F 1×1 , the feature map F 2×2 , the feature map F 4×4 , and the feature map F dir are input into an MLP to generate a channel attention weight map A c ;

[0079] The formula is:

[0080] X c = σ(∑ f∈F MLP(f)), F = {F 1×1 , F 2×2 , F 4×4 , F dir};

[0081] S20716: The channel attention weight map A c is multiplied element by element with the input feature map X to obtain a channel feature map y c ;

[0082] The formula is:

[0083] y c = X © A c ;

[0084] where © represents an element-wise multiplication operation.

[0085] Preferably, in step S2072, the processing step of the deformable spatial attention module comprises:

[0086] S20721: processing the input channel feature map y c through an offset convolution module to obtain a deformation field Offset;

[0087] The formula is:

[0088] Offset = tanh(Conv 3×3 (y c ));

[0089] where the tanh() function effectively prevents feature distortion caused by excessive offset of sampling points by constraining the offset to the interval [0, 1], while maintaining gradient stability to improve training efficiency;

[0090] S20722: processing the input channel feature map y c and the deformation field Offset through a deformable convolution module to obtain a spatial feature map X deform ;

[0091] The formula is:

[0092] X deform = DeformConv(y c , Offset, W conv );

[0093] where DeformConv represents a deformable convolution module; W conv is the convolution kernel weight used for deformable convolution operation;

[0094] S20723: performing convolution processing on the input channel feature map y c through a 3x3 convolution layer, a 5x5 convolution layer, and a 7x7 convolution layer, respectively, to obtain corresponding spatial feature maps X k ;

[0095] The formula is:

[0096] X k = {Conv k×k (X c )|k∈{3,5,7}};

[0097] S20724: processing the spatial feature map X deformand each spatial feature map X k Concatenate to obtain the feature map X all ;

[0098] The formula is:

[0099] X all =Concat(X deform ,{X k} k＝3,5,7 );

[0100] In the formula, Concat represents a concatenation operation;

[0101] S20725: After respectively performing a maximum pooling operation and an average pooling operation on the feature map X all , the maximum pooling operation and the average pooling operation are concatenated and input to an attention convolution layer for processing to obtain a spatial attention weight map A s ;

[0102] The formula is:

[0103] A s =σ(Conv 3×3 (Concat(MaxPool(X all ),AvgPool(X all ))));

[0104] In the formula, MaxPool and AvgPool respectively represent a maximum pooling operation and an average pooling operation; and sigma represents a sigmoid activation function;

[0105] S20726: The spatial attention weight map A s is element-wise multiplied with the channel feature map y c to obtain a channel-spatial feature map y s ;

[0106] The formula is:

[0107] y s =y c ⊙A s ;

[0108] In the formula, represents an element-wise multiplication operation.

[0109] Compared with the prior art, the skin lesion image segmentation method based on edge guidance and attention mechanism has the following beneficial effects:

[0110] The present application extracts edge features of an input dermoscopy image through an edge extraction (EEM) module to obtain an edge prior feature map, and gradually down-samples to obtain an edge prior feature map corresponding to a scale. In order to enhance the sensitivity of the image segmentation model to the lesion boundary, the present application designs an EEM module, which integrates the advantages of classic operators (including Sobel operator, Laplacian operator and Canny operator) and learnable convolution to construct a hybrid edge detector, so that the image segmentation model can learn the edge region of the skin lesion in detail while maintaining the overall semantic consistency, fully utilize the edge information of the lesion region to realize accurate positioning of the skin lesion, and thus improve the accuracy of skin lesion image segmentation.

[0111] The present application weights and fuses the semantic feature map corresponding to the scale and the edge prior feature map to obtain an edge semantic fusion feature map. The edge prior feature map is first subjected to an edge gate (Edge-gate), extracts weight information through convolution operation, and maps the weight value to the 0-1 interval using an activation function to generate a weight mask, which is used to weight the edge feature map to highlight important edge information; then, the weighted edge feature map and the semantic feature map are weighted and fused to generate a preliminary fusion feature map; finally, the fused feature map is subjected to an attention gate (Attention-gate), the semantic features are mapped through convolution operation, and the important feature regions are selected by the activation function to enhance the semantic feature expression, so that the final edge semantic fusion feature map focuses more on the key regions of the target object, thereby further improving the accuracy of skin lesion image segmentation.

[0112] The present application processes the semantic feature map output by the last down-sampling module of the encoder through a multi-scale dilated spatial pyramid (DASPP) module to obtain an enhanced feature map. The DASPP module captures multi-scale context information through convolution kernels with different dilation rates, solving the problem of variable morphology of skin lesions; at the same time, through the cooperation of deformable convolution and multi-scale dilated convolution, fine context modeling of the lesion area is realized, the adaptability of the image segmentation model to scale changes is enhanced, and multi-scale semantic information is fully utilized to realize accurate positioning of the skin lesion, thereby improving the effect and adaptability of skin lesion image segmentation.

[0113] The application fuses the channel and spatial attention of the segmentation feature map output by the last upsampling module of the decoder through a channel-space attention (CSAM) module, adopts a serialized design of channel priority and spatial postposition, processes the channel dimension and the spatial dimension in turn, and introduces edge information of an edge prior feature map through an edge guided feature fusion (EGF) module to enhance the features, and finally generates an optimized fusion feature map through residual connection. The CSAM module fuses multi-scale channel attention and deformable spatial attention, enhances the feature direction sensitivity through direction perception convolution, realizes adaptive modeling and enhancement of lesion deformation by combining dynamic deformation convolution, and significantly improves the feature expression capability; meanwhile, the edge information of the edge prior feature map is introduced through the edge guided feature fusion module to enhance the features, which can fully utilize the edge information of the lesion area to realize accurate positioning of the skin lesion, thereby further improving the accuracy of skin lesion image segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0114] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below with reference to the drawings, in which:

[0115] Figure 1 The network structure diagram of the skin lesion image segmentation method based on edge guidance and attention mechanism.

[0116] Figure 2 The network structure diagram of the EEM module.

[0117] Figure 3 The network structure diagram of the DASPP module.

[0118] Figure 4 The network structure diagram of the CSAM module.

[0119] Figure 5 The network structure diagram of the multi-scale channel attention module (CAM).

[0120] Figure 6 The network structure diagram of the deformable spatial attention module (SAM).

[0121] Figure 7 The visual segmentation results of different methods on ISIC 2016: the ground truth is represented by red isograms, and the segmentation results are represented by blue isograms.

[0122] Figure 8 The visual segmentation results of different methods on ISIC 2018: the ground truth is represented by red isograms, and the segmentation results are represented by blue isograms. DETAILED DESCRIPTION

[0123] The specific embodiments are further described in detail below:

[0124] Embodiments:

[0125] A skin lesion image segmentation method based on edge guidance and attention mechanism is disclosed in this embodiment,

[0126] As Figure 1 shown, the skin lesion image segmentation method based on edge guidance and attention mechanism comprises:

[0127] S1: obtaining a dermoscopy image to be segmented;

[0128] S2: inputting the dermoscopy image to be segmented into an image segmentation model (hereinafter also referred to as EGuide-UNet) for segmentation, and outputting a skin lesion segmentation image;

[0129] The processing steps of the image segmentation model comprise:

[0130] S202: using a pre-trained ResNet34 as an encoder backbone to extract features of the input dermoscopy image by using a plurality of cascaded down-sampling modules, each down-sampling module outputting a semantic feature map of a corresponding scale; wherein the ResNet34 has pre-trained weights from Image-Net to extract features, which can effectively extract rich hierarchical features.

[0131] S203: performing edge feature extraction on the input dermoscopy image by an edge extraction (EEM) module to obtain an edge prior feature map; performing level-by-level down-sampling on the edge prior feature map to obtain an edge prior feature map corresponding to each scale of semantic feature map;

[0132] S204: weighting and fusing the semantic feature map and the edge prior feature map of the corresponding scale to obtain an edge semantic fusion feature map of the corresponding scale; wherein the edge information of the edge prior feature map is highlighted through an edge gate, and then weighted and fused with the semantic feature map, and then the semantic feature expression is enhanced through an attention gate to obtain the edge semantic fusion feature map;

[0133] S205: performing multi-scale and hollow convolution processing on the semantic feature map output by the last down-sampling module of the encoder by a multi-scale and hollow spatial pyramid (DASPP) module to obtain an enhanced feature map;

[0134] S206: performing layer-by-layer up-sampling and feature restoration on the enhanced feature map by a decoder using a plurality of cascaded up-sampling modules, each up-sampling module outputting a segmentation feature map of a corresponding scale; during the up-sampling process, each up-sampling module of the decoder is connected to the edge semantic fusion feature map of the corresponding scale by a skip connection;

[0135] S207: The segmentation feature map output by the last upsampling module of the decoder is subjected to adaptive fusion of channel and spatial attention through a channel-space attention (CSAM) module, the CSAM module adopts a serialized design of channel priority and spatial postposition, processes the channel dimension and the spatial dimension in turn, and introduces edge information of an edge prior feature map through an edge-guided fusion (EGF) module to enhance the features, and finally generates an optimized fused feature map through residual connection;

[0136] S208: The fused feature map is subjected to upsampling and feature restoration to obtain a skin lesion segmentation image.

[0137] The EEM module extracts edge features from the input dermoscopy image to obtain an edge prior feature map, and gradually down-samples to obtain an edge prior feature map of a corresponding scale. In order to enhance the sensitivity of the image segmentation model to the lesion boundary, the EEM module is designed, which integrates the advantages of classical operators (including Sobel operator, Laplacian operator and Canny operator) and learnable convolution to construct a hybrid edge detector, so that the image segmentation model can finely learn the edge region of the skin lesion while maintaining the overall semantic consistency, fully utilize the edge information of the lesion region to realize accurate positioning of the skin lesion, thereby improving the accuracy of skin lesion image segmentation.

[0138] The EEM module extracts edge features from the input dermoscopy image to obtain an edge prior feature map, and gradually down-samples to obtain an edge prior feature map of a corresponding scale. In order to enhance the sensitivity of the image segmentation model to the lesion boundary, the EEM module is designed, which integrates the advantages of classical operators (including Sobel operator, Laplacian operator and Canny operator) and learnable convolution to construct a hybrid edge detector, so that the image segmentation model can finely learn the edge region of the skin lesion while maintaining the overall semantic consistency, fully utilize the edge information of the lesion region to realize accurate positioning of the skin lesion, thereby improving the accuracy of skin lesion image segmentation.

[0139] The application obtains an enhanced feature map by performing multi-scale dilated convolution on a semantic feature map output by a last down-sampling module of an encoder through a multi-scale dilated space pyramid (DASPP) module. The DASPP module captures multi-scale context information through convolution kernels with different dilation rates, thereby solving the problem of variable morphology of skin lesions. Meanwhile, through the cooperation of deformable convolution and multi-scale dilated convolution, fine context modeling of the lesion area is realized, the adaptability of the image segmentation model to scale changes is enhanced, and multi-scale semantic information is fully utilized to realize accurate positioning of the skin lesions, thereby improving the effect and adaptability of skin lesion image segmentation.

[0140] The application performs adaptive fusion of channel and spatial attention on a segmentation feature map output by a last up-sampling module of the decoder through a channel-space attention (CSAM) module, adopts a serialization design with channel priority and spatial postposition, processes the channel dimension and the spatial dimension in turn, and introduces edge information of an edge prior feature map through an edge-guided feature fusion (EGF) module to enhance the features, and finally generates an optimized fused feature map through residual connection. The CSAM module fuses multi-scale channel attention and deformable spatial attention, enhances the feature direction sensitivity through direction perception convolution, realizes adaptive modeling and enhancement of lesion deformation in combination with dynamic deformation convolution, and significantly improves the feature expression capability. Meanwhile, the edge information of the edge prior feature map is introduced through the edge-guided feature fusion module to enhance the features, which can fully utilize the edge information of the lesion area to realize accurate positioning of the skin lesions, thereby further improving the accuracy of skin lesion image segmentation.

[0141] In order to better introduce the technical scheme of the application, the embodiments are described through the following several parts.

[0142] I. Encoder and decoder

[0143] In combination Figure 1 As shown in the figure, the encoder includes cascaded initial convolution layers, a first stage down-sampling module, a second stage down-sampling module, a third stage down-sampling module and a fourth stage down-sampling module; wherein the semantic feature map output by the fourth stage down-sampling module is input to the DASPP module;

[0144] The decoder includes cascaded first stage up-sampling modules, second stage up-sampling modules, third stage up-sampling modules and fourth stage up-sampling modules; wherein the semantic feature maps output by the third stage down-sampling module, the second stage down-sampling module, the first stage down-sampling module and the initial convolution layers are respectively connected to the first stage up-sampling module, the second stage up-sampling module, the third stage up-sampling module and the fourth stage up-sampling module through the edge semantic fusion feature maps obtained by performing feature fusion on the corresponding scale semantic feature maps.

[0145] 2. EEM module

[0146] The EEM module proposed in this invention is a hybrid edge extraction module that integrates learning-based edge features and traditional edge detection operators. Figure 2 As shown in Figure 2, this module adopts a dual-path architecture: 1) a learnable edge path. By introducing dilated convolution, it captures fine edge features while also perceiving semantic information; 2) a classical operator path. By integrating the Sobel gradient operator, the second-order derivative of the Laplacian, and the OpenCV Canny operator, it constructs multi-scale geometric edge features, ensuring accurate contour information is obtained even in areas with rich texture details.

[0147] Combine Figure 2 As shown, the processing steps of the EEM module include:

[0148] S2031: Extract edge features from the dermoscopic image using a learnable edge path to obtain a first edge feature map E learned ; Wherein, the learnable edge path includes a cascaded first convolution combination module, a second convolution combination module, a third convolution combination module, a 3×3 convolution layer and a 1×1 convolution layer; the first convolution combination module includes a cascaded 3×3 convolution layer, a batch normalization layer and a ReLU function, the second convolution combination module includes a cascaded 3×3 dilated convolution layer with a dilation rate of 2, a batch normalization layer and a ReLU function, and the third convolution combination module includes a cascaded 3×3 dilated convolution layer with a dilation rate of 4, a batch normalization layer and a ReLU function;

[0149] The specific formula is as follows:

[0150] f c =Conv 3×3 (Conv 3×3,dil＝4 (Conv 3×3,dil＝2 (Conv 3×3 (x))));

[0151] Where: x represents the input feature map, x∈R C×H×W ; In order to optimize the efficiency of gradient propagation and retain the original feature information, the residual connection mechanism can be introduced into the learning edge path to convert f c Add the residual to the input feature map x to obtain the enhanced feature, and then use a 1×1 convolution layer and Sigmoid activation to project this enhanced feature into a single-channel learnable edge response to obtain the edge feature learned by the deep convolutional network. The formula is as follows:

[0152] E learned =σ(Conv 1×1 (f c +x));

[0153] Where: σ is the activation function;

[0154] S2032: edge detection is performed on the dermoscope image by a Sobel operator to obtain a second edge feature map E sobel ;

[0155] S2033: edge detection is performed on the dermoscope image by a Laplacian operator to obtain a third edge feature map E laplacian ;

[0156] S2034: edge detection is performed on the dermoscope image by a Canny operator to obtain a fourth edge feature map E canny ;

[0157] S2035: weighted calculation is performed on the first edge feature map E learned , the second edge feature map E sobel , the third edge feature map E laplacian and the fourth edge feature map E canny to obtain an edge prior feature map E out ;

[0158] The formula is as follows:

[0159]

[0160] In the formula, a is an adaptive weight parameter, a [0, 1]; E learned is the first edge feature map learned by a deep convolutional network; E sobel , E laplacian , E canny are the second edge feature map, the third edge feature map and the fourth edge feature map detected by a Sobel operator, a Laplacian operator and a Canny operator respectively.

[0161] To effectively fuse heterogeneous edge information, the application innovatively introduces a spatial adaptive weight map generation mechanism in the EEM module, inputs the input feature map into a 1x1 convolutional layer (self.alpha_conv) for processing to obtain a weight map a (a [0, 1]), and balances the contribution degree of the learning edge and the classical operator with pixel-level precision. Each value in the weight feature map represents the importance weight of the corresponding position, which can be directly used as the weight. This design makes the EEM have both the semantic perception ability of deep features and the geometric fidelity characteristics of traditional operators, and still maintains the robustness of edge detection under complex light and noise interference.

[0162] Specifically, the input feature map x is generated into a single-channel feature response through a 1x1 convolution layer (alpha_conv), and then quantified into a weight map a in the range of [0, 1] through a Sigmoid activation function. Each pixel value in the weight map a represents the confidence of the position to the learned edge. Finally, the edge map is fused according to the formula weight map a x learned + (1-weight map a) x (mean of classical operators). The high weight map a value area (such as complex texture) preferentially adopts the learned edge detection result, and the low weight map a value area (such as clear outline) focuses on the traditional operator output, realizing adaptive edge fusion based on the local characteristics of the image.

[0163] In this embodiment, the edge prior feature map output by the EEM module also needs to be down-sampled layer by layer to obtain an edge prior feature map corresponding to the semantic feature map output by the initial convolution layer, the first-stage down-sampling module, the second-stage down-sampling module and the third-stage down-sampling module of the encoder, so as to realize the fusion of the edge prior feature map and the semantic feature map of the corresponding scale.

[0164] III. Feature fusion

[0165] In this embodiment, in step S204, the following steps are specifically included:

[0166] S2041: Obtain the input semantic feature map and edge prior feature map of the corresponding scale;

[0167] S2042: Input the edge prior feature map into the edge gate (Edge-gate), extract the weight information through convolution operation, and map the weight value to the interval of 0-1 by using the activation function to generate a weight mask. The edge prior feature map is weighted through the weight mask to highlight important edge information.

[0168] S2043: Weighted fusion of the weighted edge prior feature map and the semantic feature map to generate a preliminary fusion feature map;

[0169] S2044: Input the preliminary fusion feature map into the attention gate (Attention-gate), map the semantic features through convolution operation, and select important feature regions through the activation function to enhance the expression of the semantic features, to generate an edge semantic fusion feature map. The final edge semantic fusion feature map focuses more on the key regions of the target object.

[0170] IV. DASPP module

[0171] In the skin lesion segmentation task, in order to enhance the model's ability to capture different scale features, the application introduces DASPP in the bottleneck part of the EGuide-UNet network. DASPP realizes the comprehensive capture of multi-scale features by combining convolution operations with different dilation rates and global average pooling. The structural diagram of DASPP is shown in Figure 3 .

[0172] The DASPP module contains five branches, each responsible for capturing features of different scales: branch 1 reduces the number of channels through 1x1 convolution to reduce computational complexity; branches 2, 3, and 4 use 3x3 dilated convolution with dilation rates of 2, 4, and 6, respectively, to expand the receptive field while maintaining the resolution of the feature map, thereby obtaining feature information of different scales; branch 5 uses global average pooling to compress the input features to 1x1, then processes them through 1x1 convolution, and then uses bilinear interpolation upsampling to the original feature map size to capture global context information. Then, through 1x1 convolution, the channel is compressed, and then another 1x1 convolution is used to adjust to the target channel number. Finally, the input features are added to the fused features using residual connection to enhance gradient transmission and feature expression capability.

[0173] In combination with Figure 2 , the processing steps of the DASPP module are as follows:

[0174] S2051: After inputting the input feature map X into the first branch and processing it using a cascaded 1x1 convolution layer, BN layer, and ReLU function, the first feature map b1 is obtained.

[0175] S2052: After inputting the input feature map X into the second branch and processing it using a cascaded 3x3 dilated convolution layer with a dilation rate of 2, BN layer, and ReLU function, the second feature map b2 is obtained.

[0176] S2053: After inputting the input feature map X into the third branch and processing it using a cascaded 3x3 dilated convolution layer with a dilation rate of 4, BN layer, and ReLU function, the third feature map b3 is obtained.

[0177] S2054: After inputting the input feature map X into the fourth branch and processing it using a cascaded 3x3 dilated convolution layer with a dilation rate of 6, BN layer, and ReLU function, the fourth feature map b4 is obtained.

[0178] S2055: After inputting the input feature map X into the fifth branch and processing it using a cascaded 1x1 global average pooling layer, 1x1 convolution layer, BN layer, ReLU function, and bilinear interpolation upsampling layer, the fifth feature map b5 is obtained.

[0179] S2056: After splicing the first feature map b1, the second feature map b2, the third feature map b3, the fourth feature map b4 and the fifth feature map b5, the first convolution combination block and the second convolution combination block are sequentially inputted for processing, and then residual addition is performed with the input feature map X to obtain an enhanced feature map; wherein the first convolution combination block comprises a 1x1 convolution layer, a BN layer and a ReLU function connected in cascade; and the second convolution combination block comprises a 1x1 convolution layer and a BN layer connected in cascade;

[0180] The formula is expressed as:

[0181] F out = Conv 1×1 (Conv 1×1 (Concat(b1,b2,b3,b4,b5)))+X;

[0182] In the formula, F out represents the enhanced feature map; and Concat represents a splicing operation.

[0183] Five, CSAM module

[0184] The CSAM module is a core attention mechanism proposed by us, and the structure thereof is shown in Figure 4 The CSAM module significantly enhances the feature extraction capability of the network for the skin lesion area by cooperatively optimizing the attention weights of the channel and spatial dimensions. The CSAM adopts a serialized design of channel priority and spatial postposition, sequentially processes the channel and spatial dimensions, avoids the problem of mutual interference between the channel attention and the spatial attention in the parallel architecture, finally introduces edge information through an edge-guided fusion (EGF) module for feature enhancement, and finally outputs the optimized features through residual connection.

[0185] In combination with Figure 4 , the processing steps of the CSAM module include:

[0186] S2071: Multi-scale channel attention feature extraction is performed on the input feature map X through a multi-scale channel attention module to obtain a channel feature map y c ;

[0187] S2072: Deformable spatial attention feature extraction is performed on the channel feature map y c through a deformable spatial attention module to obtain a channel spatial feature map y s ;

[0188] S2073: The channel spatial feature map y s is fused with an edge prior feature map through an EGF module to obtain an edge channel spatial fusion feature map y fuse ;

[0189] S2074: After 1x1 convolution processing on the input feature map X, the edge channel spatial fusion feature map y is obtained fuse Element-wise addition is performed to obtain the final fusion feature map;

[0190] The formula is:

[0191] y = ReLU (Conv 1×1 (X) + y fuse );

[0192] In the formula: y represents the final fusion feature map; X represents the input feature map; y fuse represents the edge channel spatial fusion feature map.

[0193] 1. Multi-scale channel attention module

[0194] The traditional channel attention mechanism only considers global average pooling, and it is difficult to capture multi-scale features of skin lesion regions. The multi-scale channel attention module (CAM) is proposed in the present application, and its structure is shown in Figure 5 .

[0195] In combination with Figure 5 , the processing steps of the multi-scale channel attention module include:

[0196] S20711: The input feature map X is processed through a 1x1 adaptive average pooling layer to obtain a feature map F 1×1 ;

[0197] The formula is:

[0198] F 1×1 = Adaptive AvgPool 1×1 (X);

[0199] In the formula: Adaptive AvgPool 1×1 represents a 1x1 adaptive average pooling layer;

[0200] S20712: After processing the input feature map X through a 2x2 adaptive average pooling layer, nearest neighbor interpolation up-sampling is performed to obtain a feature map F 2×2 ;

[0201] The formula is:

[0202] F 2×2 = UP (Adaptive AvgPool 2×2 (X));

[0203] In the formula: Adaptive AvgPool 2×2denotes a 2x2 adaptive average pooling layer; UP denotes nearest neighbor interpolation up-sampling;

[0204] S20713: The input feature map X is processed through a 4x4 adaptive average pooling layer, and then nearest neighbor interpolation up-sampling is performed to obtain a feature map F 4×4 ;

[0205] The formula is:

[0206] F 4×4 = UP(Adaptive AvgPool 4×4 (X));

[0207] In the formula: Adaptive AvgPool 4×4 denotes a 4x4 adaptive average pooling layer;

[0208] S20714: The input feature map X is processed through a cascaded 3x1 convolution layer, 1x3 convolution layer, BN layer and ReLU function to obtain a feature map F dir ;

[0209] The formula is:

[0210] F dir = ReLU(BN(Conv 1×3 (Conv 3×1 (X)));

[0211] S20715: The feature map F 1×l , the feature map F 2×2 , the feature map F 4×4 , and the feature map F dir are input into an MLP to generate a channel attention weight map A c ;

[0212] The formula is:

[0213] A c = σ(∑ f∈F MLP(f)), F = {F 1×1 , F 2×2 , F 4×4 , F dir};

[0214] S20716: The channel attention weight map A c is element-wise multiplied with the input feature map X to obtain a channel feature map y c ;

[0215] The formula is:

[0216] y c = X ⊙ A c;

[0217] In the formula: ⊙ represents element-by-element multiplication operation.

[0218] It should be noted that 1x1dentity() in the figure represents an identity operation, that is, it does not change the input data at all, and directly outputs the input data unchanged.

[0219] 2. Deformable spatial attention module

[0220] The spatial attention mechanism improves the segmentation accuracy by enhancing the feature response of important regions. The present application proposes a deformable spatial attention module (SAM), the structure of which is shown in Figure 6 .

[0221] In combination with Figure 6 , the processing steps of the deformable spatial attention module include:

[0222] S20721: Process the input channel feature map y c through the offset convolution module to obtain the deformation field Offset;

[0223] The formula is:

[0224] Offset=tanh(Conv 3×3 (y c ));

[0225] In the formula: the tanh() function constrains the offset to the interval [0, 1], effectively preventing feature distortion caused by excessive sampling point offset, while maintaining gradient stability to improve training efficiency.

[0226] S20722: Process the input channel feature map y c and the deformation field Offset through the deformable convolution module to obtain the spatial feature map X deform .

[0227] The formula is:

[0228] X deform =DeformConv(y c ,Offset,W conv );

[0229] In the formula: DeformConv represents the deformable convolution module; W conv is the convolution kernel weight used for deformable convolution operation;

[0230] S20723: Process the input channel feature map y cConvolution is performed to obtain the corresponding spatial feature map X k ;

[0231] The formula is as follows:

[0232] X k = {Conv k×k (X c )|k∈{3,5,7}};

[0233] S20724: The spatial feature maps X deform and the respective spatial feature maps X k are spliced to obtain the feature map X all .

[0234] The formula is as follows:

[0235] X all = Concat(X deform , {X k} k＝3,5,7 );

[0236] In the formula, Concat represents a splicing operation.

[0237] S20725: After the feature map X all is respectively subjected to a maximum pooling operation and an average pooling operation, the results of the maximum pooling operation and the average pooling operation are spliced and input to an attention convolution layer for processing to obtain a spatial attention weight map A s .

[0238] In this embodiment, the attention convolution layer is a 3x3 convolution layer, which is used to fuse and process the feature maps after the maximum pooling and the average pooling to generate the spatial attention weight map A s .

[0239] The formula is as follows:

[0240] A s = σ(Conv 3×3 (Concat(MaxPool(X all ),AvgPool(X all ))));

[0241] In the formula, MaxPool and AvgPool respectively represent a maximum pooling operation and an average pooling operation; and σ represents a sigmoid activation function.

[0242] S20726: The spatial attention weight map A s is element-wise multiplied with the channel feature map y c to obtain a channel-spatial feature map y s .

[0243] The formula is expressed as:

[0244] y s = y c ⊙ A s ;

[0245] In the formula: ⊙ represents an element-by-element multiplication operation.

[0246] Six, experimental explanation

[0247] 1, data set

[0248] In this experiment, we conducted extensive experiments on two public data sets of skin lesions segmentation, namely the International Skin Imaging Collaboration Challenge data set (ISIC 2016 and ISIC 2018), to train and evaluate the proposed model. ISIC2016 and ISIC2018 have 1279 and 2594 skin microscopy images with segmentation mask labels, respectively. To ensure the universality and robustness of the experimental results, we use a random division strategy to divide the data set into a training set and a test set in a 7:3 ratio. For ISIC 2016, there are 895 images in the training set and 384 images in the test set. For ISIC 2018, there are 1816 images in the training set and 778 images in the test set. We conducted comparative experiments on ISIC 2016 and ISIC 2018, and conducted ablation experiments on ISIC 2018.

[0249] 2, data set preprocessing

[0250] During preprocessing, each sample was rescaled to 256x256 pixels. To improve the model's adaptability to spatial position and orientation changes of lesions, random horizontal flipping (probability 0.5) and random vertical flipping (probability 0.3) were implemented in the spatial dimension; in the color dimension, color jitter with limited parameters (brightness ± 0.15, contrast ± 0.1, saturation ± 0.1, hue ± 0.02) was used to simulate the color variations that may occur in clinical imaging devices. After converting the image data to tensors (ToTensor), further normalization processing was performed according to the ImageNet standard parameters, with the mean set to μ = [0.485, 0.456, 0.406] and the standard deviation set to σ = [0.229, 0.224, 0.225], so that the input feature distribution is consistent with the requirements of the pre-trained backbone network. To ensure the consistency of the image and the corresponding segmentation mask in geometric transformation, we set the same random seed to force the same flipping operation on both, avoiding anatomical structure misplacement caused by independent random transformation; at the same time, the binary characteristics were strictly maintained during the mask processing, and the nearest neighbor interpolation method was used to prevent label smoothing. For the test set, all random enhancement operations were removed in the processing flow, and only the normalization step was retained to ensure the objectivity of the diagnostic evaluation and the repeatability of the results.

[0251] 3. Experimental setup and parameters

[0252] To verify the effectiveness of the proposed method, all experiments were conducted under the Windows 11 operating system, with hardware platform configurations including 13th Gen Core TM i7-13620H CPU and NVIDIA GeForce RTX 4060 Laptop GPU (8GB video memory), which can meet the computing throughput and video memory capacity required for deep network training. The software environment uses the PyTorch 2.0.1+cu118 deep learning framework combined with CUDA 12.6 driver to fully utilize the GPU acceleration capability. In the training phase, the AdamW optimizer is used with an initial learning rate of 1e-4, a weight decay coefficient of 1e-4, a gradient momentum coefficient β1 = 0.9, β2 = 0.999, a numerical stability term ε = 1e-8, and a OneCycleLR scheduling strategy to perform cosine annealing after linear warm-up; the gradient accumulation step is 2 to maintain a larger effective batch size when the video memory is limited. The batch size is 8, and the maximum training rounds (epochs) is 100. To balance training efficiency and numerical accuracy, mixed precision training (Automatic Mixed Precision, AMP) is enabled throughout the experiment, which significantly reduces memory usage and speeds up the forward / backward propagation process while maintaining model performance.

[0253] 4、Loss function

[0254] In the task of skin lesion segmentation, in order to improve the performance of the model in dealing with unbalanced data, the present application proposes a hybrid loss function (BceDiceLoss) combining binary cross-entropy (BCE) and Dice loss. The loss function combines the two losses in a weighted manner to balance the class imbalance problem and improve the segmentation accuracy.

[0255] Given the predicted logits P∈R H×W and the real label Y∈{0,1} H×W , the total loss of BceDiceLoss is defined as:

[0256]

[0257] In the formula, the BCE loss L BCE is the standard binary cross-entropy loss, defined as:

[0258]

[0259] where σ represents the sigmoid function, and N=H×W is the total number of pixels. The Dice loss L Dice is optimized to segment the boundary by measuring the spatial overlap between the prediction and the ground truth, as follows:

[0260]

[0261] In the formula, ε=10 -6 is the smoothing coefficient to avoid numerical instability when the denominator is zero.

[0262] 5、Evaluation index

[0263] In this experiment, in order to comprehensively evaluate the performance of the skin lesion segmentation model, we introduced a variety of evaluation indexes, including Dice coefficient, IoU index, accuracy (Acc), sensitivity (Sensitivity) and specificity (Specificity). Among them, the Dice coefficient is used to measure the overlap between the predicted segmentation result and the true label, while the IoU index reflects the ratio of the intersection to the union of the predicted region and the true region. The accuracy measures the overall classification accuracy of the model on all pixels. Sensitivity mainly examines the recognition ability of the model to lesion areas (positive examples), while specificity reflects the ability of the model to correctly identify non-lesion areas (negative examples). The specific mathematical expressions of the above indexes are as follows:

[0264]

[0265] Where TP, FP, FN, TN represent the number of true positive, false positive, false negative and true negative pixels, respectively. These evaluation indicators provide quantitative evaluation from global to local, positive and negative sample recognition, etc. for skin lesion segmentation task, thus providing a solid theoretical support for model optimization and performance comparison.

[0266] 6、Experimental results

[0267] To comprehensively evaluate the performance of the proposed EGuide-UNet network model, we compared it with six state-of-the-art methods, including UNet, UNeXt, MALUNet, EGE-UNet, LB-UNet, and MHorUNet. All the compared methods were trained for 100 epochs on the ISIC 2016 and ISIC 2018 datasets.

[0268] Table 1. Comparison results on the ISIC 2016 dataset. (BOLD indicates the best, and underlined indicates the second best.)

[0269]

[0270] Note:

[0271] Unet from: Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation [C] / / Medical Image Computing and Computer-Assisted Intervention--MICCAI 2015. Cham: Springer International Publishing, 2015: 234-241.

[0272] UneXt from: Valanarasu J M J, Patel V M. Unext: Mlp-based rapid medical image segmentation network [C] / / Medical Image Computing and Computer Assisted Intervention--MICCAI 2022. Cham: Springer Nature Switzerland, 2022: 23-33.

[0273] MALUNet from: Ruan J, Xiang S, Xie M, et al. Malunet: A multi-attention and light-weight unet for skin lesion segmentation [C] / / 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Las Vegas, NV, USA: IEEE, 2022: 1150-1156.

[0274] EGE-Unet from: Ruan J, Xie M, Gao J, et al. Ege-unet: an efficient group enhanced unet for skin lesion segmentation [C] / / Medical Image Computing and Computer Assisted Intervention--MICCAI 2023. Cham: Springer Nature Switzerland, 2023: 481-490.

[0275] LB-Unet from: Xu J, Tong L. LB-UNet: A Lightweight Boundary-Assisted UNet for Skin Lesion Segmentation [C] / / Medical Image Computing and Computer Assisted Intervention--MICCAI 2024. Cham: Springer Nature Switzerland, 2024: 361-371.

[0276] MhorUNet from: Wu R, Liang P, Huang X, et al. Mhorunet: High-order spatial interaction unet for skin lesion segmentation [J]. Biomedical Signal Processing and Control, 2024, 88: 105517.

[0277] Table 1 shows the comparison results on the ISIC 2016 dataset. Experimental results demonstrate that the proposed EGuide-UNet model outperforms other comparison models on the ISIC 2016 dataset. Specifically, EGuide-UNet achieves 92.84% and 86.77% in Dice coefficient and Intersection over Union (IoU), respectively, representing improvements of 1.59 and 2.20 percentage points over the next-best method, MHOrUNet. It also achieves the highest levels of accuracy (96.18%), sensitivity (93.01%), and specificity (97.17%). The significant improvement in sensitivity, in particular, demonstrates the network's enhanced robustness in detecting small lesions. Notably, despite having a higher number of parameters (56.08M) than many lightweight networks, EGuide-UNet maintains only 29.07 GFLOPs, a nearly 47% reduction compared to the 54.64 GFLOPs of the traditional UNet. This significantly improves computational efficiency while maintaining or even exceeding the original accuracy.

[0278] Figure 7 The visualization of segmentation results of different methods on ISIC 2016 is shown. In fuzzy boundary scenarios, EGuide-UNet effectively maintains edge continuity, adaptively suppresses noise interference, avoids common segmentation boundary breaks or over-expansion, and achieves accurate segmentation of target areas. In tiny spot samples, UNet and MHorUNet suffer from serious missed detections, while MALUNet and LB-UNet misdetect adjacent pigments. EGuide-UNet, however, fully captures tiny lesion areas, reducing missed detections and segmentation breaks. In hair-occluded samples, UNeXt, MALUNet, LB-UNet, and MHorUNet exhibit jagged segmentation boundaries, while EGuide-UNet accurately penetrates hair coverage, achieving continuous and seamless segmentation of lesion areas.

[0279] Table 2 Comparative experimental results on the ISIC 2018 dataset. (BOLD indicates the best, underlined indicates the second best.)

[0280]

[0281] Table 2 shows the segmentation results of different methods on the ISIC 2018 dataset. The EGuide-UNet model proposed by the present application performs outstandingly in the ISIC 2018 skin lesion segmentation task. On key evaluation indicators, the model achieves a Dice coefficient of 90.05% and an IoU value of 82.09%. It is particularly noteworthy that the model simultaneously achieves the optimal value on both sensitivity (91.46%) and accuracy (95.11%), which indicates that the edge guiding mechanism not only improves the overall segmentation accuracy, but also significantly enhances the model's recognition ability for small lesions, effectively reducing the missed detection rate. Although the model parameter quantity (56.08M) and computational complexity (29.07 GFLOPs) are higher than those of the lightweight architecture, the significant improvement in key lesion recognition indicators proves the rational investment of computing resources.

[0282] Figure 8 The visualization segmentation results of different methods on ISIC 2018 are shown. In the fuzzy boundary area, traditional methods such as UNet produce severely inaccurate segmentation contours, lightweight methods LBUNet produce jagged segmentation contours, while EGuide-UNet generates smooth and continuous boundaries that closely fit the true annotation, proving that the edge guiding module can effectively enhance the boundary perception ability; in the hair occlusion area, EGuide-UNet generates the clearest segmentation result in the hair area, without over-segmentation, which shows that the edge guiding fusion module effectively suppresses non-edge noise interference, enabling the model to maintain the continuity of the segmentation structure under complex interference; for complex background lesion areas, compared with other models, our model maintains complete lesion recognition ability, generates the most accurate segmentation result closest to the ground truth annotation, and can successfully suppress missegmentation in non-lesion areas. In summary, EGuide-UNet, through the collaborative design of the edge guiding mechanism and the adaptive attention module, exhibits comprehensive, balanced and significantly superior performance to existing advanced models in the skin lesion segmentation task, solving the three major challenges of fuzzy boundaries, variable morphology and interference in skin lesion segmentation, and providing a more reliable solution for accurate skin lesion analysis.

[0283] 7. Ablation study

[0284] To systematically verify the effectiveness of the proposed module, the present experiment conducted a rigorous ablation experiment on the ISIC 2018 dataset.

[0285] Table 3 Ablation experiment on the ISIC 2018 dataset. (BOLD indicates the best)

[0286]

[0287]

[0288] As shown in Table 3, we adopted a progressive module integration strategy to quantitatively evaluate the contribution of each innovative component by gradually introducing them into the original UNet (Dice: 87.85%) as the baseline. First, we replaced the encoder with a pre-trained ResNet34 to build the Baseline model, which improved the Dice by 1.10% to 88.95%, verifying the effectiveness of the pre-trained feature extractor. Subsequently, we focused on analyzing the three implementation modes of the edge extraction module (EEM): the pure learning path mode (EEM-learned) improved the Dice to 89.20%, the pure classical operator mode (EEM-classical) performed outstandingly in the sensitivity index (90.05%), and the fusion mode of classical operators and learnable paths (EEM-fusion) achieved the best balance (Dice: 89.59%, Sen: 90.07%), which confirmed the significant complementarity between traditional edge detection operators and deep learning features. After further introducing the multi-scale context module (DASPP), the model achieved significant improvements in Dice (89.86%), IoU (81.85%), and other indicators, indicating that multi-scale feature fusion can effectively deal with the morphological diversity of skin lesions. Finally, the complete model integrating the channel-spatial attention mechanism (CSAM) achieved the optimal performance (Dice: 90.05%, Sen: 91.46%), which explicitly modeled the channel dependency and spatial deformation features through dual attention mechanisms, allowing the model to maintain computational efficiency (29.07 GFLOPs) while achieving a 1.10% improvement in Dice compared to the Baseline. It is particularly noteworthy that the integration of each module presents a clear synergistic effect: EEM ensures boundary positioning accuracy, DASPP enhances multi-scale feature representation capability, and CSAM optimizes feature discriminability. This hierarchical design approach provides a new technical paradigm for medical image segmentation tasks. Experimental results show that the proposed EGuide-UNet not only significantly outperforms the traditional UNet (Dice improvement of 2.20%), but also achieves a good balance between model complexity and segmentation accuracy.

[0289] 8、Experimental discussion

[0290] The core goal of skin lesion segmentation is to provide accurate lesion area definition for clinical diagnosis, and various evaluation indicators are closely related to actual clinical needs. Sensitivity, as a key indicator of the model's ability to identify lesion areas, has clinical value in effectively suppressing early-stage skin cancer misdiagnosis. In the ISIC 2016 dataset, the sensitivity of EGuide-UNet is 0.36% higher than that of the suboptimal model MHorUNet, which means that the model can capture more small or fuzzy boundary malignant lesions. If these lesions are missed, it may lead to the missed best treatment window for malignant lesions such as melanoma. According to the World Health Organization data, the 5-year survival rate of early-stage melanoma patients is over 90%, while that of late-stage patients is only 15%-20%. Therefore, high sensitivity segmentation is directly related to the improvement of patient prognosis.

[0291] Specificity and sensitivity need to maintain a dynamic balance. For example, EGE-UNet has a specificity of 97.73% but a sensitivity of only 86.37%. This "high specificity-low sensitivity" pattern can reduce false positives in normal skin, but it significantly increases the risk of missing malignant lesions. EGuide-UNet uses the channel-space attention coordination mechanism of the CSAM module to maintain the advantage of specificity while significantly optimizing sensitivity. This balance is crucial for clinical decision-making, as it can both avoid the waste of medical resources caused by over-diagnosis and prevent treatment delays caused by missed diagnoses.

[0292] Compared with existing advanced methods, EGuide-UNet has significant advantages in multiple key indicators. For example, compared with MHorUNet, EGuide-UNet has a 1.59% higher Dice coefficient on the ISIC 2016 dataset and also has improved sensitivity. This performance difference is mainly due to the innovative modules in EGuide-UNet. Compared with the deformable convolution in MHorUNet, the deformable spatial attention module (SAM) in EGuide-UNet shows greater adaptability in handling irregular boundaries. SAM generates a deformation field through offset convolution, which can more flexibly capture the complex shape of the lesion area and extract spatial features with multi-scale convolution kernels, thus achieving more accurate feature representation. In addition, EGuide-UNet's hybrid edge extraction module (EEM) combines learnable convolution with classic operators (Sobel / Laplacian / Canny) and has a significant advantage in enhancing boundary sensitivity, allowing the model to better capture the edge details of the lesion area.

[0293] Finally, it needs to be explained that the above examples are only used to illustrate the technical solutions of the present application but not to limit the technical solutions, and those of ordinary skill in the art should understand that the technical solutions of the present application are modified or equivalently replaced without departing from the purpose and scope of the technical solutions, which should be covered in the scope of claims of the present application.

Claims

1. A skin lesion image segmentation method based on edge guidance and attention mechanism, characterized in that: include: S1: Acquire the dermatoscopic image to be segmented; S2: Input the dermatoscopic image to be segmented into the image segmentation model for segmentation, and output the skin lesion segmentation image; The processing steps of the image segmentation model include: S202: Using a pre-trained ResNet34 as the encoder backbone, a cascade of multiple downsampling modules is used to extract features from the input dermoscopic image, where each downsampling module outputs a semantic feature map of a corresponding scale; S203: extracting edge features from the input dermoscopic image through the EEM module to obtain an edge prior feature map; downsampling the edge prior feature map step by step to obtain edge prior feature maps corresponding to semantic feature maps at each scale; S204: performing weighted fusion on the semantic feature map and the edge prior feature map of the corresponding scale to obtain an edge semantic fusion feature map of the corresponding scale; wherein the edge information of the edge prior feature map is highlighted by an edge gate and then weighted fused with the semantic feature map, and the semantic feature expression is further enhanced by an attention gate to obtain an edge semantic fusion feature map; S205: Perform multi-scale dilated convolution processing on the semantic feature map output by the last downsampling module of the encoder through the DASPP module to obtain an enhanced feature map; S206: The decoder uses a plurality of cascaded upsampling modules to perform layer-by-layer upsampling and feature recovery on the enhanced feature map, and each upsampling module outputs a segmentation feature map of a corresponding scale; during the upsampling process, each upsampling module of the decoder is jump-connected to the edge semantic fusion feature map of the corresponding scale; S207: Adaptively fuse the segmentation feature map output by the last upsampling module of the decoder with channel and spatial attention through the CSAM module; the CSAM module adopts a serial design with channel priority and spatial postposition, processes the channel dimension and spatial dimension in sequence, and introduces edge information of the edge prior feature map through the edge-guided feature fusion module for feature enhancement, and finally generates an optimized fused feature map through residual connection; S208: Upsampling and feature recovery are performed on the fused feature map to obtain a skin lesion segmentation image.

2. The skin lesion image segmentation method based on edge guidance and attention mechanism according to claim 1, characterized in that: The encoder includes a cascade of initial convolutional layers, a first-stage downsampling module, a second-stage downsampling module, a third-stage downsampling module, and a fourth-stage downsampling module; the semantic feature map output by the fourth-stage downsampling module serves as the input of the DASPP module; The decoder includes a cascaded first-stage upsampling module, a second-stage upsampling module, a third-stage upsampling module, and a fourth-stage upsampling module; The semantic feature maps output by the third-stage downsampling module, the second-stage downsampling module, the first-stage downsampling module and the initial convolutional layer are fused with the semantic feature maps of the corresponding scale to obtain the edge semantic fusion feature maps of the corresponding scale, which are jump-connected to the first-stage upsampling module, the second-stage upsampling module, the third-stage upsampling module and the fourth-stage upsampling module respectively.

3. The skin lesion image segmentation method based on edge guidance and attention mechanism according to claim 2, characterized in that: In step S203, the processing steps of the EEM module include: S2031: Extract edge features from the dermoscopic image using a learnable edge path to obtain a first edge feature map E learned ; Among them, the learnable edge path includes a cascaded first convolution combination module, a second convolution combination module, a third convolution combination module, a 3×3 convolution layer and a 1×1 convolution layer; the first convolution combination module includes a cascaded 3×3 convolution layer, a batch normalization layer and a ReLU function, the second convolution combination module includes a cascaded 3×3 dilated convolution layer with a dilation rate of 2, a batch normalization layer and a ReLU function, and the third convolution combination module includes a cascaded 3×3 dilated convolution layer with a dilation rate of 4, a batch normalization layer and a ReLU function; The specific formula is as follows: f c =Conv 3×3 (Conv 3×3，dil＝4 (Conv 3×3，dil＝2 (Conv 3×3 (x)))); Where: x represents the input feature map, x∈R C×H×W ; In order to optimize the efficiency of gradient propagation and retain the original feature information, the residual connection mechanism can be introduced into the learning edge path to convert f c The enhanced features are added to the residual of the input feature map x, and then a 1×1 convolution layer and Sigmoid activation are used to project the enhanced features into a single-channel learnable edge response to obtain the edge features learned by the deep convolutional network. The formula is expressed as: E learned =σ(Conv1×1(f c +x)); Where: σ is the activation function; S2032: Perform edge detection on the dermoscopic image using the Sobel operator to obtain a second edge feature map E sobel ; S2033: Perform edge detection on the dermatoscope image using the Laplacian operator to obtain the third edge feature map E laplacian ; S2034: Perform edge detection on the dermoscopic image using the Canny operator to obtain a fourth edge feature map E canny ; S2035: First edge feature map E learned , the second edge feature map E sobel , the third edge feature map E laplacian And the fourth edge feature map E canny Perform weighted calculation to obtain the edge prior feature map E out ; The formula is: Where: α is the adaptive weight parameter, α∈[0,1]; E learned is the first edge feature map obtained by deep convolutional network learning; E sobel 、E laplacian 、E canny They are the second edge feature map, the third edge feature map and the fourth edge feature map detected by the Sobel operator, the Laplacian operator and the Canny operator respectively.

4. The skin lesion image segmentation method based on edge guidance and attention mechanism according to claim 3, characterized in that: In step S2035, the input feature map is input into a 1×1 convolutional layer for processing using a spatially adaptive weight map generation mechanism to obtain a weight map α that balances the contribution of the learned edge and the classical operator with pixel-level accuracy.

5. The skin lesion image segmentation method based on edge guidance and attention mechanism according to claim 2, characterized in that: The edge prior feature map output by the EEM module is downsampled layer by layer to obtain an edge prior feature map corresponding to the scale of the semantic feature map output by the encoder's initial convolutional layer, the first-stage downsampling module, the second-stage downsampling module, and the third-stage downsampling module.

6. The skin lesion image segmentation method based on edge guidance and attention mechanism according to claim 1, characterized in that: Step S204 specifically includes the following steps: S2041: Obtaining a semantic feature map and an edge prior feature map of the corresponding scale of the input; S2042: Input the edge prior feature map into the edge gate, extract weight information through convolution operation, and use activation function to map the weight value to the range of 0-1 to generate a weight mask; weight the edge prior feature map through the weight mask to highlight important edge information; S2043: performing weighted fusion on the weighted edge prior feature map and the semantic feature map to generate a preliminary fused feature map; S2044: Input the preliminary fusion feature map into the attention gate, map the semantic features through convolution operation, and use the activation function to filter out important feature areas, enhance the expression of semantic features, and generate an edge semantic fusion feature map that is more focused on the key areas of the target object.

7. The skin lesion image segmentation method based on edge guidance and attention mechanism according to claim 1, characterized in that: In step S205, the processing steps of the DASPP module are as follows: S2051: Input the input feature map X into the first branch and process it using the cascaded 1×1 convolution layer, BN layer and ReLU function to obtain the first feature map b1; S2052: Input the input feature map X into the second branch and process it using a cascaded 3×3 dilated convolution layer with a dilation rate of 2, a BN layer, and a ReLU function to obtain a second feature map b2; S2053: Input the input feature map X into the third branch and process it using a cascaded 3×3 dilated convolution layer with a dilation rate of 4, a BN layer, and a ReLU function to obtain a third feature map b3; S2054: Input the input feature map X into the fourth branch and process it using a cascaded 3×3 dilated convolution layer with a dilation rate of 6, a BN layer, and a ReLU function to obtain a fourth feature map b4; S2055: Input the input feature map X into the fifth branch and process it using a cascaded 1×1 global average pooling layer, a 1×1 convolutional layer, a BN layer, a ReLU function, and a bilinear interpolation upsampling layer to obtain a fifth feature map b5; S2056: After concatenating the first feature map b1, the second feature map b2, the third feature map b3, the fourth feature map b4, and the fifth feature map b5, the concatenation is sequentially input into the first convolutional combination block and the second convolutional combination block for processing, and then residually added to the input feature map X to obtain an enhanced feature map; wherein the first convolutional combination block includes a cascaded 1×1 convolutional layer, a BN layer, and a ReLU function; the second convolutional combination block includes a cascaded 1×1 convolutional layer and a BN layer; The formula is: F out =Conv 1×1 (Conv 1×1 (Concat(b1,b2,b3,b4,b5)))+X; Where: F out Represents an enhanced feature map; Concat represents a concatenation operation.

8. The skin lesion image segmentation method based on edge guidance and attention mechanism according to claim 1, characterized in that: In step S207, the processing steps of the CSAM module include: S2071: Perform multi-scale channel attention feature extraction on the input feature map X through the multi-scale channel attention module to obtain the channel feature map y c ; S2072: Channel feature map y through deformable spatial attention module c Perform deformable spatial attention feature extraction to obtain the channel space feature map y s ; S2073: channel space feature map y s Perform feature fusion with the edge prior feature map to obtain the edge channel space fusion feature map y fuse ; S2074: After performing 1×1 convolution on the input feature map X, it is spatially fused with the edge channel feature map y fuse Add element by element to get the final fusion feature map; The formula is: y=Re LU(Conv 1×1 (X)+y fuse ); Where: y represents the final fusion feature map; y fuse Represents the edge channel spatial fusion feature map.

9. The skin lesion image segmentation method based on edge guidance and attention mechanism according to claim 8, characterized in that: In step S2071, the processing steps of the multi-scale channel attention module include: S20711: Process the input feature map X through a 1×1 adaptive average pooling layer to obtain the feature map F 1×1 ; The formula is: F 1×1 =Adaptive AvgPool 1×1 (X); Where: Adaptive AvgPool 1×1 represents a 1×1 adaptive average pooling layer; S20712: After processing the input feature map X through a 2×2 adaptive average pooling layer, the nearest neighbor interpolation upsampling is performed to obtain the feature map F 2×2 ; The formula is: F 2×2 =UP(Adaptive AvgPool 2×2 (X)); Where: Adaptive AvgPool 2×2 Represents a 2×2 adaptive average pooling layer; UP represents nearest neighbor interpolation upsampling; S20713: After processing the input feature map X through a 4×4 adaptive average pooling layer, the nearest neighbor interpolation upsampling is performed to obtain the feature map F 4×4 ; The formula is: F 4×4 =UP(Adaptive AvgPool 4×4 (X)); Where: Adaptive AvgPool 4×4 Represents a 4×4 adaptive average pooling layer; S20714: The input feature map X is processed by a cascade of 3×1 convolutional layers, 1×3 convolutional layers, BN layers, and ReLU functions to obtain the feature map F dir ; The formula is: F dir =Re LU(BN(Conv 1×3 (Conv 3×1 (X)))); S20715: The feature map F 1×1 , feature map F 2×2 , feature map F 4×4 and feature map F dir Input MLP to generate channel attention weight map A c ; The formula is: A c =σ(∑ f∈F MLP(f)),F={F 1×1 ,F 2×2 ,F 4×4 ,F dir }; S20716: Channel attention weight map A c Multiply element-by-element with the input feature map X to obtain the channel feature map y c ; The formula is: and c =X⊙A c ; Where: ⊙ represents the element-by-element multiplication operation.

10. The skin lesion image segmentation method based on edge guidance and attention mechanism according to claim 9, characterized in that: In step S2072, the processing steps of the deformable spatial attention module include: S20721: The channel feature map y of the input is processed by the offset convolution module c Processing is performed to obtain the deformation field Offset; The formula is: offset=tanh(Conv 3×3 (y c )); Where: The tanh() function effectively prevents feature distortion caused by excessive offset of sampling points by constraining the offset to the interval [0, 1], while maintaining gradient stability to improve training efficiency; S20722: Input channel feature map y through deformable convolution module c And the deformation field Offset is processed to obtain the spatial feature map X deform ; The formula is: X deform =DeformConv(y c ,Offset,W convv ); Where: DeformConv represents the deformable convolution module; W conv is the convolution kernel weight used for the deformable convolution operation; S20723: The channel feature map y of the input is respectively processed through a 3×3 convolution layer, a 5×5 convolution layer, and a 7×7 convolution layer. c Perform convolution processing to obtain the corresponding spatial feature map X k ; The formula is: X k ={Conv k×k (X c )|k∈{3,5,7}}; S20724: Spatial feature map X deform And each spatial feature map X k Splice and get the feature map X all ; The formula is: X all =Concat(X deform ,{X k } k＝3，5，7 ); Where: Concat represents the concatenation operation; S20725: Feature map X all After performing the maximum pooling operation and the average pooling operation respectively, the results of the maximum pooling operation and the average pooling operation are spliced ​​and input into the attention convolution layer for processing to obtain the spatial attention weight map A s ; The formula is: A s =σ(Conv 3×3 (Concat(MaxPool(X all ),AvgPool(X all )))); Where: MaxPool and AvgPool represent the maximum pooling operation and the average pooling operation respectively; σ represents the sigmoid activation function; S20726: Spatial attention weight map A s With channel feature map y c Perform element-by-element multiplication to obtain the channel space feature map y s ; The formula is: and s =and c ⊙A s ; Where: ⊙ represents the element-by-element multiplication operation.

Images