Method and system for segmentation of skin lesions

Abstract

This invention discloses a method for segmenting skin lesions, comprising: extracting an initial feature map from a target skin lesion image; then performing feature extraction using an encoder incorporating a wavelet convolutional layer to obtain a wavelet feature map; multiplying the wavelet feature map with a tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain a processed wavelet feature map, dividing it into four groups of feature maps for processing to obtain a preliminary output feature map; performing multi-scale feature fusion on the wavelet feature map and the preliminary output feature map respectively; inputting the preliminary output feature map into a decoder; and performing an add operation on the intermediate output of the decoder with multiple multi-scale fused feature maps. The final output of the decoder is the skin lesion segmentation result. This application, through grouped feature processing, the introduction of partial convolution, and multi-scale feature fusion, not only improves the accuracy of feature extraction but also reduces the number of model parameters, improves computational efficiency, and enhances the module's adaptability to data diversity.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and system for segmenting skin lesions. Background Technology

[0002] Malignant melanoma, a deadly form of skin cancer, has a rising global incidence rate, resulting in numerous deaths annually. Automated skin lesion segmentation systems are widely used in medical diagnostics due to their ability to quickly and accurately identify lesion areas.

[0003] Among existing technical solutions, Dense-UNet improves its ability to capture multi-scale information by effectively reusing features through dense blocks. U-Netv2 employs an innovative skip connection mechanism to combine high-level and low-level features, achieving finer feature fusion and enhancing the integration of features at different scales. The FCBFormer model uses a dual-path structure, combining a fully convolutional network (FCN) and a Vision Transformer (ViT) to simultaneously extract local and global features, thereby improving segmentation accuracy. The UNeXt model creates a lightweight network structure by combining UNet with a multilayer perceptron (MLP), maintaining high performance while reducing parameters and computational cost. The MALUNet model further reduces the model size by decreasing the number of channels and introducing an attention module, and demonstrates superior performance to UNeXt in skin lesion segmentation tasks.

[0004] While existing technologies have made some progress in automated skin lesion segmentation, they still have the following shortcomings: Models such as Dense-UNet and U-Net v2 perform poorly when dealing with lesion regions that exhibit significant size differences and complex texture features. These models fail to effectively address the challenges of size variations and texture complexity, thus affecting segmentation accuracy. High-precision segmentation models such as FCBFormer, while providing accurate segmentation results, require a large number of parameters and computational resources. This high resource requirement limits their widespread adoption and application on medical devices and mobile devices with limited computing power. Although lightweight models such as MALUNet improve accessibility by reducing computational costs, their segmentation accuracy often cannot match that of models with a large number of parameters. This compromise in accuracy can affect the reliability of diagnosis, especially in complex cases requiring high-precision segmentation. Therefore, existing technologies present a difficult trade-off between pursuing lightweight design and high accuracy. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a method and system for segmenting skin lesions, resolving the technical problem of the difficulty in balancing lightweight design and high precision in related technologies.

[0006] In a first aspect, this application provides a method for segmenting skin lesions, the method comprising:

[0007] The target skin lesion image is input into a convolutional neural network to extract image features and obtain an initial feature map;

[0008] An encoder incorporating wavelet convolutional layers is used to extract features from the initial feature map to obtain a wavelet feature map;

[0009] The wavelet feature map is multiplied by the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain the processed wavelet feature map. The processed wavelet feature map is divided into four groups of feature maps. The first three groups of feature maps are subjected to HPA operations along the height-width axis, channel-height axis, and channel-width axis, respectively. The fourth group of feature maps is subjected to a partial convolution operation. The processing results of the four groups of feature maps are concatenated along the channel dimension and then multiplied again by the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain the preliminary output feature map. The convolution kernel size is 3.

[0010] The wavelet feature map and the preliminary output feature map are respectively subjected to multi-scale feature fusion to obtain multiple multi-scale fused feature maps;

[0011] The initial output feature map is input into the decoder, and the intermediate output result of the decoder is added to the multiple multi-scale fused feature maps. The final output result of the decoder is the skin lesion segmentation result.

[0012] In some embodiments, the step of inputting the target skin lesion image into a convolutional neural network to extract image features and obtain an initial feature map includes:

[0013] The target skin lesion image is input into a convolutional neural network, and image features are extracted from the target skin lesion image through a convolutional layer with a kernel size of 3x3 to obtain an initial feature map.

[0014] In some embodiments, the step of multiplying the wavelet feature map with a tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain the processed wavelet feature map includes:

[0015] Using the tensor p formula:

[0016] p=DepthwiseConv[BilinearInterpolation(p,size=Y.size())]

[0017] X = p * Y

[0018] Where Y represents the input wavelet feature map, X represents the processed wavelet feature map, DepthwiseConv represents the depthwise separable convolution operation, and BilinearInterpolation represents the bilinear interpolation operation.

[0019] In some embodiments, the method further includes:

[0020] Perform a maximum downsampling operation on the wavelet feature map to obtain a first sampled wavelet feature map;

[0021] The first sampled wavelet feature map is used as the initial feature map, and the encoder with the wavelet convolution layer is used repeatedly to extract features from the initial feature map to obtain the wavelet feature map.

[0022] The wavelet feature map obtained by repeating the steps is subjected to maximum downsampling again to obtain the second sampled wavelet feature map, which is then used as the latest wavelet feature map.

[0023] In some embodiments, the method further includes:

[0024] Perform a maximum downsampling operation on the preliminary output feature map to obtain a first sampled preliminary output feature map;

[0025] The first sampled preliminary output feature map is used as a wavelet feature map. The process of multiplying the wavelet feature map with the tensor p that has undergone depth-separable convolution and bilinear interpolation is repeated to obtain the preliminary output feature map.

[0026] The multi-scale feature maps obtained from the repeated steps are subjected to maximum downsampling again to obtain the second sampled preliminary output feature map.

[0027] The second sampled preliminary output feature map is used as a wavelet feature map. The process of multiplying the wavelet feature map with the tensor p that has undergone depth-separable convolution and bilinear interpolation is repeated to obtain a preliminary output feature map. The preliminary output feature map obtained by repeating the process is used as the latest preliminary output feature map.

[0028] In some embodiments, the step of fusing the wavelet feature map and the preliminary output feature map at multiple scales to obtain multiple multi-scale fused feature maps includes:

[0029] The second sampled preliminary output feature map is adjusted to the same spatial resolution as the latest preliminary output feature map by employing depthwise separable convolution and bilinear interpolation operations.

[0030] The adjusted second sampled preliminary output feature map and the latest preliminary output feature map are respectively divided into a first feature group, a second feature group and a third feature group along the channel dimension;

[0031] The first feature group of the adjusted second sampled preliminary output feature map and the first feature group of the latest preliminary output feature map are fused together, and the fused features are connected with the mask information to obtain the first fused feature.

[0032] The second feature group of the adjusted second sampled preliminary output feature map and the second feature group of the latest preliminary output feature map are fused together, and the fused features are concatenated with the mask information to obtain the second fused feature.

[0033] The third feature group of the adjusted second sampled preliminary output feature map and the third feature group of the latest preliminary output feature map are fused together, and the fused features are connected with the mask information to obtain the third fused feature.

[0034] The first fused feature is subjected to dilated convolution and pooling operations according to the first dilation rate, and then Triplet Attention is applied to obtain the processed first fused feature.

[0035] The second fused feature is subjected to dilated convolution and pooling operations according to the second dilation rate, and then processed by Triplet Attention to obtain the processed second fused feature.

[0036] The third fusion feature is subjected to dilated convolution and pooling operations according to the third dilation rate, and then processed by Triplet Attention to obtain the processed third fusion feature.

[0037] The processed first fusion feature, the processed second fusion feature, and the processed third fusion feature are concatenated and processed by ordinary convolution with a kernel size of 1 to obtain the fifth multi-scale fusion feature map.

[0038] By employing depthwise separable convolution and bilinear interpolation, the initial output feature map of the first sampling is adjusted to have the same spatial resolution as the fifth multi-scale fused feature map.

[0039] The adjusted first sampled preliminary output feature map and the fifth multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, to obtain the fourth multi-scale fusion feature map;

[0040] The second sampled wavelet feature map is adjusted to the same spatial resolution as the fourth multi-scale fused feature map by employing depthwise separable convolution and bilinear interpolation operations.

[0041] The adjusted second-sampled wavelet feature map and the fourth multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second-sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, to obtain the third multi-scale fusion feature map;

[0042] The first sampled wavelet feature map is adjusted to the same spatial resolution as the third multi-scale fused feature map by employing depthwise separable convolution and bilinear interpolation operations.

[0043] The adjusted first sampled wavelet feature map and the third multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension to obtain the second multi-scale fusion feature map;

[0044] The initial feature map is adjusted to the same spatial resolution as the second multi-scale fused feature map by employing depthwise separable convolution and bilinear interpolation operations.

[0045] The adjusted initial feature map and the second multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, to obtain the first multi-scale fusion feature map.

[0046] In some embodiments, the step of inputting the preliminary output feature map into the decoder and performing an add operation on the intermediate output result of the decoder and the plurality of multi-scale fused feature maps, wherein the final output result of the decoder is the skin lesion segmentation result, includes:

[0047] The latest preliminary output feature map is input into the decoder, and the first decoding preliminary output feature map is obtained by multiplying the latest preliminary output feature map with the tensor p that has completed the depthwise separable convolution operation and bilinear interpolation operation.

[0048] The fifth multi-scale fused feature map, obtained by multi-scale feature fusion based on the second sampled preliminary output feature map, is added to the first decoding preliminary output feature map to obtain the first decoding output result;

[0049] The first decoding output result is multiplied by the tensor p that has completed the depth-separable convolution operation and bilinear interpolation operation to obtain the second decoding preliminary output feature map;

[0050] The second initial decoded output feature map is subjected to bilinear interpolation to obtain the second initial decoded output feature map after bilinear interpolation.

[0051] The fourth multi-scale fused feature map obtained by multi-scale feature fusion based on the first sampled preliminary output feature map is added to the second decoding preliminary output feature map after bilinear interpolation to obtain the second decoding output result.

[0052] The third decoding preliminary output feature map is obtained by multiplying the second decoding output result with the tensor p that has completed the depth-separable convolution operation and bilinear interpolation operation.

[0053] The third decoding preliminary output feature map is subjected to bilinear interpolation to obtain the third decoding preliminary output feature map after bilinear interpolation.

[0054] The third multi-scale fused feature map obtained by multi-scale feature fusion based on the second sampled wavelet feature map is added to the third decoding preliminary output feature map after the bilinear interpolation operation to obtain the third decoding output result.

[0055] Wavelet convolution is used to extract features from the third decoding output to obtain the preliminary output feature map of the fourth decoding.

[0056] The fourth decoding preliminary output feature map is subjected to bilinear interpolation to obtain the fourth decoding preliminary output feature map after bilinear interpolation.

[0057] The second multi-scale fused feature map obtained by multi-scale feature fusion based on the first sampled wavelet feature map is added to the fourth decoding preliminary output feature map after the bilinear interpolation operation to obtain the fourth decoding output result.

[0058] Wavelet convolution is used to extract features from the fourth decoding output to obtain the preliminary output feature map of the fifth decoding.

[0059] The fifth decoded preliminary output feature map is subjected to bilinear interpolation to obtain the fifth decoded preliminary output feature map after bilinear interpolation.

[0060] The first multi-scale fused feature map obtained by multi-scale feature fusion based on the initial feature map is added to the fifth decoding preliminary output feature map after bilinear interpolation to obtain the fifth decoding output result.

[0061] Two-dimensional convolution is used to extract features from the fifth decoding output to obtain the preliminary output feature map of the sixth decoding.

[0062] The sixth decoded preliminary output feature map is subjected to bilinear interpolation to obtain the sixth decoded preliminary output feature map after bilinear interpolation. The sixth decoded preliminary output feature map after bilinear interpolation is used as the skin lesion segmentation result.

[0063] In some embodiments, the method further includes:

[0064] The initial feature map is subjected to a maximum downsampling operation, and the sampling result is used as the latest initial feature map.

[0065] Secondly, a skin lesion segmentation system, the system comprising:

[0066] The first extraction module is used to input the target skin lesion image into a convolutional neural network to extract image features and obtain an initial feature map;

[0067] The second extraction module is used to extract features from the initial feature map using an encoder that incorporates a wavelet convolutional layer to obtain a wavelet feature map.

[0068] The processing module is used to multiply the wavelet feature map with the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain a processed wavelet feature map. The processed wavelet feature map is divided into four groups of feature maps. The first three groups of feature maps are subjected to HPA operations along the height-width axis, channel-height axis, and channel-width axis, respectively. The fourth group of feature maps is subjected to a partial convolution operation. The processing results of the four groups of feature maps are concatenated along the channel dimension and then multiplied again with the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain a preliminary output feature map. The convolution kernel size is 3.

[0069] The fusion module is used to perform multi-scale feature fusion on the wavelet feature map and the preliminary output feature map respectively to obtain multiple multi-scale fused feature maps;

[0070] The output module is used to input the preliminary output feature map into the decoder, and to add the intermediate output result of the decoder to the multiple multi-scale fused feature maps. The final output result of the decoder is the skin lesion segmentation result.

[0071] In some embodiments, the first extraction module is used to input the target skin lesion image into a convolutional neural network, and extract image features from the target skin lesion image through a convolutional layer with a kernel size of 3x3 to obtain an initial feature map.

[0072] This application provides a method and system for segmenting skin lesions, comprising: inputting a target skin lesion image into a convolutional neural network to extract image features, obtaining an initial feature map; using an encoder incorporating a wavelet convolutional layer to extract features from the initial feature map, obtaining a wavelet feature map; multiplying the wavelet feature map with a tensor p that has undergone depthwise separable convolution and bilinear interpolation operations, obtaining a processed wavelet feature map; dividing the processed wavelet feature map into four groups of feature maps; performing HPA operations on the first three groups of feature maps along the height-width axis, channel-height axis, and channel-width axis, respectively; and multiplying the fourth group of feature maps... The group feature maps undergo partial convolution operations, and the processing results of the four groups of feature maps are concatenated along the channel dimension and then multiplied again with the tensor p that has undergone depthwise separable convolution and bilinear interpolation operations to obtain a preliminary output feature map; wherein the convolution kernel size is 3; the wavelet feature map and the preliminary output feature map are respectively subjected to multi-scale feature fusion to obtain multiple multi-scale fused feature maps; the preliminary output feature map is input into the decoder, and the intermediate output result of the decoder is added to the multiple multi-scale fused feature maps, and the final output result of the decoder is the skin lesion segmentation result. This application, through group feature processing, the introduction of partial convolution, and multi-scale feature fusion, not only improves the accuracy of feature extraction, but also reduces the number of model parameters, improves computational efficiency, and enhances the module's adaptability to data diversity. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0074] Figure 1 A schematic flowchart illustrating a skin lesion segmentation method provided in an embodiment of this application;

[0075] Figure 2 This is a schematic diagram of the wavelet convolution feature extraction process provided in an embodiment of this application;

[0076] Figure 3 A schematic diagram of a linear attention module provided in an embodiment of this application;

[0077] Figure 4 A schematic diagram of the multi-scale feature fusion module provided in an embodiment of this application;

[0078] Figure 5 A complete schematic diagram of a convolutional neural network is provided for the embodiments of this application;

[0079] Figure 6 The effect diagram obtained by validating the skin lesion segmentation method provided in the embodiments of this application in the dataset;

[0080] Figure 7 This is a schematic diagram of a skin lesion segmentation system provided in an embodiment of this application. Detailed Implementation

[0081] The following detailed description of the embodiments of this application, in conjunction with the accompanying drawings, will provide a thorough understanding of how this application uses technical means to solve technical problems and achieve corresponding technical effects, enabling its implementation. The embodiments of this application and the various features within them can be combined with each other without conflict, and all resulting technical solutions are within the protection scope of this application.

[0082] Example 1

[0083] Figure 1 This is a flowchart illustrating a method for segmenting skin lesions provided in an embodiment of this application, as shown below. Figure 1 As shown, this method includes:

[0084] S101. Input the target skin lesion image into a convolutional neural network to extract image features and obtain an initial feature map;

[0085] S102. An encoder with a wavelet convolutional layer is used to extract features from the initial feature map to obtain a wavelet feature map;

[0086] S103. Multiply the wavelet feature map with the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain the processed wavelet feature map. Divide the processed wavelet feature map into four groups of feature maps. Perform HPA operations on the first three groups of feature maps along the height-width axis, channel-height axis, and channel-width axis, respectively. Perform partial convolution operations on the fourth group of feature maps. Concatenate the processing results of the four groups of feature maps along the channel dimension, and multiply them again with the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain the preliminary output feature map. The convolution kernel size is 3.

[0087] S104. Perform multi-scale feature fusion on the wavelet feature map and the preliminary output feature map respectively to obtain multiple multi-scale fused feature maps;

[0088] S105. Input the preliminary output feature map into the decoder, and add the intermediate output result of the decoder with the multiple multi-scale fused feature maps. The final output result of the decoder is the skin lesion segmentation result.

[0089] It should be noted that, addressing the limitations of existing models, particularly their inadequacy in handling lesions with significant size variations and complex textures, this invention discloses an efficient and lightweight method for skin lesion segmentation. The main objective of this method is to significantly reduce the number of model parameters and computational costs while maintaining high accuracy, thereby improving the model's computational efficiency. This method is easier to deploy and apply on resource-constrained medical and mobile devices, providing medical professionals with a fast, accurate, and easily accessible tool for skin lesion segmentation. Through this invention, we hope to overcome the shortcomings of existing technologies in handling size diversity and texture complexity, as well as the excessive consumption of computational resources, enabling wider and more convenient clinical applications.

[0090] In some embodiments, the step of inputting the target skin lesion image into a convolutional neural network to extract image features and obtain an initial feature map includes:

[0091] The target skin lesion image is input into a convolutional neural network, and image features are extracted from the target skin lesion image through a convolutional layer with a kernel size of 3x3 to obtain an initial feature map.

[0092] In some embodiments, the method further includes:

[0093] The initial feature map is subjected to a maximum downsampling operation, and the sampling result is used as the latest initial feature map.

[0094] In some embodiments, the step of multiplying the wavelet feature map with a tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain the processed wavelet feature map includes:

[0095] Using the tensor p formula:

[0096] p=DepthwiseConv[BilinearInterpolation(p,size=Y.size())]

[0097] X = p * Y

[0098] Where Y represents the input wavelet feature map, X represents the processed wavelet feature map, DepthwiseConv represents the depthwise separable convolution operation, and BilinearInterpolation represents the bilinear interpolation operation.

[0099] In some embodiments, the method further includes:

[0100] Perform a maximum downsampling operation on the wavelet feature map to obtain a first sampled wavelet feature map;

[0101] The first sampled wavelet feature map is used as the initial feature map, and the encoder with the wavelet convolution layer is used repeatedly to extract features from the initial feature map to obtain the wavelet feature map.

[0102] The wavelet feature map obtained by repeating the steps is subjected to maximum downsampling again to obtain the second sampled wavelet feature map, which is then used as the latest wavelet feature map.

[0103] It should be noted that a wavelet convolutional layer is introduced into the encoder part of the convolutional model to extract features from the initial skin lesion feature map. This reduces the number of network parameters and expands the receptive field, enabling the network to capture key features of the image more effectively.

[0104] like Figure 2 The diagram illustrates the wavelet convolution feature extraction process. First, the wavelet object is initialized to obtain the coefficients of the decomposition and reconstruction filters. These coefficients are then converted into tensors and flipped to accommodate the convolution operation. Next, a decomposition filter bank is constructed, combining low-pass and high-pass filters into four different filters (LL, LH, HL, HH), and these filters are replicated according to the number of input channels. Simultaneously, a reconstruction filter bank is also constructed, combining filters in the same manner and replicating them according to the number of output channels.

[0105] For the input skin lesion feature map, we perform a multi-level wavelet transform. In each level of the transform, a decomposition filter bank is used to convolve the input feature map. For the i-th level wavelet transform, the specific steps are as follows:

[0106]

[0107] Where h and g represent the low-pass filter and high-pass filter in wavelet transform, respectively. This indicates that the low-pass filter h is convolved with itself, and then this result is used to modify the feature map F of the previous stage. i-1 Perform convolution to obtain the approximate part (low-frequency part) of the i-th level wavelet transform; the other three formulas are similar. LL i This indicates the low-frequency sub-band (approximate sub-band), representing the low-frequency portion of the image, LH. i Representing the low-frequency to high-frequency subband, which contains horizontal edge information of the image, HL i This represents the high-frequency to low-frequency subband, containing vertical edge information of the image. HH iThe high-frequency subband (detail subband) contains detailed image information such as texture and noise. The LL subband is downsampled and preserved for the next wavelet transform, while the LH, HL, and HH subbands undergo further processing. For these high-frequency subbands, we flatten them and convolve them with the corresponding kernels, then apply a learnable scaling factor for scaling. Finally, we combine the scaled subbands with the low-frequency LL subbands and reassemble them into a four-channel form, resulting in the skin lesion feature map F_combined after one wavelet transform. i ,Right now:

[0108] F_combined i =Concat(downsample(LL) i ),α*LH i ,α*HL i ,α*HH i )

[0109] Where downsample represents the downsampling operation, and α represents the learnable scaling factor.

[0110] Then, using the LL subband of the previous level as the input of the next level, repeat the above steps.

[0111] After completing the multi-level wavelet transform, an inverse wavelet transform is performed: In each level, the combined skin lesion feature map is convolved through a reconstruction filter bank, and upsampling is achieved through a transposed convolution with a stride of 2. Then, the upsampled skin lesion feature map is added to the high-frequency feature map from the previous level to gradually restore it to its original size. The specific formula is as follows:

[0112] F_merged i =ConTranspose(F_combined i ,G)+Concat(LH i-1 ,HH i-1 HL i-1 )

[0113] After all wavelet transform levels are completed, a basic convolution operation is performed on the original input skin lesion feature map F0, and scaled using a learnable scaling factor β. The result of the basic convolution is added to the result of the inverse wavelet transform to obtain the final output skin lesion feature map F_final, as follows:

[0114] F_final=β*Conv(F0)+F_merged0

[0115] If the model is set with a step size greater than 1, additional downsampling is performed on the resulting wavelet feature map to ensure that the size of the output feature map meets the requirements.

[0116] Finally, a maximum downsampling operation is performed on the feature map obtained by wavelet convolution.

[0117] To improve accuracy, the above steps can be repeated, usually twice.

[0118] In some embodiments, the method further includes:

[0119] Perform a maximum downsampling operation on the preliminary output feature map to obtain a first sampled preliminary output feature map;

[0120] The first sampled preliminary output feature map is used as a wavelet feature map. The process of multiplying the wavelet feature map with the tensor p that has undergone depth-separable convolution and bilinear interpolation is repeated to obtain the preliminary output feature map.

[0121] The multi-scale feature maps obtained from the repeated steps are subjected to maximum downsampling again to obtain the second sampled preliminary output feature map.

[0122] The second sampled preliminary output feature map is used as a wavelet feature map. The process of multiplying the wavelet feature map with the tensor p that has undergone depth-separable convolution and bilinear interpolation is repeated to obtain a preliminary output feature map. The preliminary output feature map obtained by repeating the process is used as the latest preliminary output feature map.

[0123] It should be noted that, as Figure 3 The diagram shown is a schematic of the linear attention module provided in an embodiment of this application. Specifically, the above content, combined with step S103, involves multiplying the wavelet feature map with the tensor p that has undergone depthwise separable convolution and bilinear interpolation operations to obtain the processed wavelet feature map; specifically, this is achieved through the tensor p formula:

[0124] p=DepthwiseConv[BilinearInterpolation(p,size=Y.size())]

[0125] X = p * Y

[0126] Where Y represents the input wavelet feature map, X represents the processed wavelet feature map, DepthwiseConv represents the depthwise separable convolution operation, and BilinearInterpolation represents the bilinear interpolation operation.

[0127] It's worth noting that in the latter part of the convolutional network, we designed a Grouplinear Attention Module (GAM), which is applied to the last three layers of the encoder to delve deeper into the image's information from multiple perspectives. The core of this module is a learnable tensor p, which is resized using bilinear interpolation to match the size of the input skin lesion feature map. Based on this, we process the tensor p using depthwise separable convolution, and then generate the initial output by performing a standard product operation with the input feature map.

[0128] The processed wavelet feature map X is then divided into four chunks along the channel dimension: x1, x2, x3, and x4. The first three chunks, x1, x2, and x3, undergo Hadamard Product Attention (HPA) operations (height-width axis, channel-height axis, and channel-width axis processing) along the height-width axis, channel-height axis, and channel-width axis, respectively, to fully extract information from different dimensions. We also introduce partial convolution to process feature map x4.

[0129] x1,x2,x3,x4=SplitChannels(X,num_groups=4)

[0130] x1 = HPA(x1, axis = (0, 1))

[0131] x2 = HPA(x2, axis = (1, 2))

[0132] x3 = HPA(x3, axis = (1, 3))

[0133] x4 = PartialConv(x4)

[0134] PartialConv represents partial convolution. Partial convolution dynamically adjusts the effective region of the convolution kernel based on the validity of the data, performing convolution operations only on valid data points. This method not only improves the accuracy of feature extraction but also reduces the number of model parameters, increases computational efficiency, and enhances the module's adaptability to data diversity.

[0135] After processing the four sets of features independently, they are concatenated along the channel dimension and normalized. Then, depthwise convolution (DVC) is applied again to integrate the information from different angles of the skin lesion feature map, yielding the final feature map output (i.e., the initial output feature map). Throughout the process, a kernel size of 3 is used, which helps maintain the locality of features while reducing the number of model parameters. The process is illustrated in the following formula:

[0136] output=DepthwiseConv(Concat(x1,x2,x3,x4,dim=1))

[0137] Next, the initial output feature map obtained through the GAM module is subjected to maximum downsampling.

[0138] To improve accuracy, the above steps can be repeated, usually three times. The last step does not require the maximum downsampling operation.

[0139] In some embodiments, the step of fusing the wavelet feature map and the preliminary output feature map at multiple scales to obtain multiple multi-scale fused feature maps includes:

[0140] The second sampled preliminary output feature map is adjusted to the same spatial resolution as the latest preliminary output feature map by employing depthwise separable convolution and bilinear interpolation operations.

[0141] The adjusted second sampled preliminary output feature map and the latest preliminary output feature map are respectively divided into a first feature group, a second feature group and a third feature group along the channel dimension;

[0142] The first feature group of the adjusted second sampled preliminary output feature map and the first feature group of the latest preliminary output feature map are fused together, and the fused features are connected with the mask information to obtain the first fused feature.

[0143] The second feature group of the adjusted second sampled preliminary output feature map and the second feature group of the latest preliminary output feature map are fused together, and the fused features are concatenated with the mask information to obtain the second fused feature.

[0144] The third feature group of the adjusted second sampled preliminary output feature map and the third feature group of the latest preliminary output feature map are fused together, and the fused features are connected with the mask information to obtain the third fused feature.

[0145] The first fused feature is subjected to dilated convolution and pooling operations according to the first dilation rate, and then Triplet Attention is applied to obtain the processed first fused feature.

[0146] The second fused feature is subjected to dilated convolution and pooling operations according to the second dilation rate, and then processed by Triplet Attention to obtain the processed second fused feature.

[0147] The third fusion feature is subjected to dilated convolution and pooling operations according to the third dilation rate, and then processed by Triplet Attention to obtain the processed third fusion feature.

[0148] The processed first fusion feature, the processed second fusion feature, and the processed third fusion feature are concatenated and processed by ordinary convolution with a kernel size of 1 to obtain the fifth multi-scale fusion feature map.

[0149] By employing depthwise separable convolution and bilinear interpolation, the initial output feature map of the first sampling is adjusted to have the same spatial resolution as the fifth multi-scale fused feature map.

[0150] The adjusted first sampled preliminary output feature map and the fifth multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, to obtain the fourth multi-scale fusion feature map;

[0151] The second sampled wavelet feature map is adjusted to the same spatial resolution as the fourth multi-scale fused feature map by employing depthwise separable convolution and bilinear interpolation operations.

[0152] The adjusted second-sampled wavelet feature map and the fourth multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second-sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, to obtain the third multi-scale fusion feature map;

[0153] The first sampled wavelet feature map is adjusted to the same spatial resolution as the third multi-scale fused feature map by employing depthwise separable convolution and bilinear interpolation operations.

[0154] The adjusted first sampled wavelet feature map and the third multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension to obtain the second multi-scale fusion feature map;

[0155] The initial feature map is adjusted to the same spatial resolution as the second multi-scale fused feature map by employing depthwise separable convolution and bilinear interpolation operations.

[0156] The adjusted initial feature map and the second multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, to obtain the first multi-scale fusion feature map.

[0157] It should be noted that in this step, we designed a multi-scale feature fusion module (MTM) to replace the traditional skip connection operation.

[0158] Specifically, such as Figure 4 The diagram shows a multi-scale feature fusion module provided in this embodiment of the application. The input of the current MTM is fused with the multi-scale fused feature map output by the next-order MTM. The MTM module plays a key role in fusing feature information at different scales. It not only enhances the network's ability to segment skin disease images, but also enables the network to extract richer features at different scales.

[0159] The specific process is as follows:

[0160] First, we use depthwise separable convolution and bilinear interpolation to adjust the size of the high-level features to match the spatial resolution of the low-level features. Then, we divide the adjusted high-level features (the input to the current MTM) and the low-level features (the output of the next-order MTM) into three groups along the channel dimension: group1, group2, and group3. Each group of low-level features is then concatenated with its corresponding high-level feature to form a fused feature. Simultaneously, these fused features are concatenated with masking information.

[0161] group1,group2,group3=SplitChannels(low_level_features,num_groups

[0162] =3)

[0163] concated_group1

[0164] =Concat(Mask,Concat(group1,high_level_features_resized[0],dim=1)))

[0165] concated_group2

[0166] =Concat(Mask,Concat(group2,high_level_features_resized[1],dim=1)))

[0167] concated_group2

[0168] =Concat(Mask,Concat(group3,high_level_features_resized[2],dim=1)))

[0169] SplitChannels represents grouping operations, Concat represents concatenation operations along the channel dimension, and high_level_features_resized represents the adjusted high-level features.

[0170] Furthermore, to extract multi-scale information, we apply dilated convolutions with different dilation rates (3, 5, and 7) to the fused features, thereby capturing contextual information at different scales. Building upon this, we introduce the TripletAttention channel attention module, which delves into the intrinsic relationships between features through a nearly parameter-free attention mechanism. The TripletAttention module comprises three branches: the first branch calculates channel attention, where the input features are pooled, processed by a 7x7 convolution, and spatial attention weights are generated using the Sigmoid activation function. The second branch captures the interaction between channel C and spatial W dimensions, where the input features are transformed in dimension, pooled in dimension H, and then processed to recover CXHXW dimension features for element-wise addition. The third branch captures the interaction between channel C and spatial H dimensions, with a similar process to the second branch, except that pooling is performed in dimension W. The output features of the three branches are summed and averaged, which not only avoids the information loss problem in traditional methods but also effectively captures the interactions between different dimensions in the feature map. The above operations can be summarized by the following formula:

[0171] channel_attention=ChannelAttention(dilated_features)

[0172] spatial_HW_attention=SpatialHWAttention(dilated_features)

[0173] spatial_CW_attention=SpatialCWAttention(dilated_features)

[0174] output=(channel_attention+spatial_HW_attention

[0175] +spatial_CW_attention) / 3

[0176] Where channel_attention, spatial_HW_attention, and spatial_CW_attention represent the three branches of the Triplet Attention module, and dilated_features represent the fused feature map obtained after pooling and dilated convolution operations.

[0177] Finally, the three sets of features processed by Triplet Attention are concatenated, and the interaction between features is achieved through ordinary convolution with a kernel size of 1, thereby further enhancing the expressive power of the features at different scales. The processed feature map is then output. In this way, the MTM module effectively achieves multi-scale feature fusion, providing strong feature support for skin disease segmentation.

[0178] In some embodiments, the step of inputting the preliminary output feature map into the decoder and performing an add operation on the intermediate output result of the decoder and the plurality of multi-scale fused feature maps, wherein the final output result of the decoder is the skin lesion segmentation result, includes:

[0179] The latest preliminary output feature map is input into the decoder, and the first decoding preliminary output feature map is obtained by multiplying the latest preliminary output feature map with the tensor p that has completed the depthwise separable convolution operation and bilinear interpolation operation.

[0180] The fifth multi-scale fused feature map, obtained by multi-scale feature fusion based on the second sampled preliminary output feature map, is added to the first decoding preliminary output feature map to obtain the first decoding output result;

[0181] The first decoding output result is multiplied by the tensor p that has completed the depth-separable convolution operation and bilinear interpolation operation to obtain the second decoding preliminary output feature map;

[0182] The second initial decoded output feature map is subjected to bilinear interpolation to obtain the second initial decoded output feature map after bilinear interpolation.

[0183] The fourth multi-scale fused feature map obtained by multi-scale feature fusion based on the first sampled preliminary output feature map is added to the second decoding preliminary output feature map after bilinear interpolation to obtain the second decoding output result.

[0184] The third decoding preliminary output feature map is obtained by multiplying the second decoding output result with the tensor p that has completed the depth-separable convolution operation and bilinear interpolation operation.

[0185] The third decoding preliminary output feature map is subjected to bilinear interpolation to obtain the third decoding preliminary output feature map after bilinear interpolation.

[0186] The third multi-scale fused feature map obtained by multi-scale feature fusion based on the second sampled wavelet feature map is added to the third decoding preliminary output feature map after the bilinear interpolation operation to obtain the third decoding output result.

[0187] Wavelet convolution is used to extract features from the third decoding output to obtain the preliminary output feature map of the fourth decoding.

[0188] The fourth decoding preliminary output feature map is subjected to bilinear interpolation to obtain the fourth decoding preliminary output feature map after bilinear interpolation.

[0189] The second multi-scale fused feature map obtained by multi-scale feature fusion based on the first sampled wavelet feature map is added to the fourth decoding preliminary output feature map after the bilinear interpolation operation to obtain the fourth decoding output result.

[0190] Wavelet convolution is used to extract features from the fourth decoding output to obtain the preliminary output feature map of the fifth decoding.

[0191] The fifth decoded preliminary output feature map is subjected to bilinear interpolation to obtain the fifth decoded preliminary output feature map after bilinear interpolation.

[0192] The first multi-scale fused feature map obtained by multi-scale feature fusion based on the initial feature map is added to the fifth decoding preliminary output feature map after bilinear interpolation to obtain the fifth decoding output result.

[0193] Two-dimensional convolution is used to extract features from the fifth decoding output to obtain the preliminary output feature map of the sixth decoding.

[0194] The sixth decoded preliminary output feature map is subjected to bilinear interpolation to obtain the sixth decoded preliminary output feature map after bilinear interpolation. The sixth decoded preliminary output feature map after bilinear interpolation is used as the skin lesion segmentation result.

[0195] It should be noted that in the decoder stage, the network structure exhibits a symmetrical design to the encoder, creating a harmonious correspondence. The main task of the decoder is to recover the details and spatial information of the image, which is achieved through bilinear interpolation. In this process, we perform an add operation on the feature map output of each order of the MTM and the output of the decoder after bilinear interpolation, and then obtain a mask through a 1×1 convolution. This mask is used to supervise the training of the network.

[0196] Mask=Conv1×1(BilinearInterpolation(high_level_features,target_size

[0197] =MTM_output.size())+MTM_output)

[0198] BilinearInterpolation represents the bilinear interpolation operation, a common upsampling method used to enlarge the size of feature maps to the target size. Here, it adjusts the size of the high-level feature maps (high_level_features) to the same size as the MTM module output (MTM_output).

[0199] In summary, such as Figure 5 The diagram shown is a complete convolutional neural network disclosed in one embodiment of this application, which includes a skin lesion segmentation method and a supervised network training part.

[0200] It should be noted that during the training process of the convolutional neural network in this invention:

[0201] (1) The ISIC2018 public dataset was used. All experiments were performed on a single NVIDIA RTX 3090 GPU. Before the experiments, the image data was standardized and uniformly adjusted to a size of 256×256 pixels. To enhance the generalization ability of the model, the following data augmentation strategies were adopted: horizontal flipping, vertical flipping, and random rotation. During network model training, AdamW was selected as the optimizer, and the initial learning rate was set to 0.001. The learning rate scheduling adopted the CosineAnnealingLR strategy, using CosineAnnealingLR as the scheduler, with a maximum of 50 iterations and a minimum learning rate of 1e-5. A total of 300 epochs were trained, and the batch size was set to 8.

[0202] (2) Since the MTM module in this invention requires mask information at different scales, we use deep supervision to calculate the loss function at different stages in order to generate more accurate mask information. Our loss function can be expressed as:

[0203] α i =Bce(y,y^)+Dice(y,y^)

[0204]

[0205] Where Bce(y,y^) and Dice(y,y^) represent the binary cross-entropy and dice loss, respectively. α i For the loss function at different stages, β i The weights are assigned to different stages. In this invention, we set β by default between i=0 and i=5. i The values ​​are 1, 0.5, 0.4, 0.3, 0.2, and 0.1.

[0206] (3) After completing model training, save the trained model, load the trained model parameters, input test data, and you can obtain the data segmentation results end-to-end. For example... Figure 6 The image shown is a diagram illustrating the results of validating a skin lesion segmentation method according to an embodiment of the present invention on a dataset.

[0207] To further demonstrate the effectiveness of the skin lesion segmentation method of this application, a test dataset was collected for performance testing. Specifically, the evaluation metrics of this invention are the number of parameters, the mean intersection-over-union ratio (mIOU), and the Dice similarity coefficient (DSC). On the test dataset, the performance of the entire algorithm was evaluated. The method of this invention achieved competitive results. It is particularly noteworthy that, compared to mainstream network environments, this method also exhibits high accuracy and performance even with a lower number of parameters.

[0208] The experimental results of this invention are compared with those of mainstream algorithms, as shown in Table 1:

[0209]

[0210] Table 1

[0211] As can be seen from Table 1, the skin lesion segmentation method of this application (ours column in the table) has a lower number of parameters, a higher mean intersection-over-union ratio (mIOU), and a higher Dice similarity coefficient (DSC) compared to other methods.

[0212] Therefore, this invention is designed based on the EGE-UNet network. Addressing the problems of existing high-precision networks having large parameter counts, lightweight networks having low accuracy, and lesion regions exhibiting significant size differences and complex texture features in skin disease segmentation tasks, the designed network offers better performance. This invention incorporates a wavelet convolution module, a linear attention module based on partial convolution and depthwise separable convolution, and a multi-scale feature fusion module (the third innovation). Experimental results demonstrate that this network achieves high accuracy and performance while maintaining a low parameter count.

[0213] This invention addresses the need for image segmentation in dermatological diseases by designing a novel convolutional neural network (CNN) structure. This structure first employs two wavelet convolutions to capture multi-scale features of the image, followed by two attention mechanisms combining partial convolution and depthwise separable convolution to further refine the feature representation. In our network structure, the two wavelet convolution operations effectively extract multi-scale texture information from dermatological disease images, which is crucial for identifying lesion regions of different sizes and shapes. Wavelet convolution, through its multi-resolution analysis capabilities, provides rich feature information for subsequent processing. Next, the network introduces two attention mechanism modules based on partial convolution and depthwise separable convolution. This module utilizes partial convolution to maintain feature sparsity while combining it with depthwise separable convolution for efficient feature extraction, allowing the network to focus more on salient features of lesion regions while ignoring unimportant background information. This invention not only enhances the network's ability to perceive multi-scale texture features and maintains efficient extraction of detailed features, but also significantly reduces the number of network parameters. This strategy enables the network module to have better feature processing capabilities when processing skin disease images, thereby improving the segmentation accuracy of the lesion area and the overall performance.

[0214] This invention addresses the problem of large variations in texture size and complex feature relationships in lesion regions of skin disease images by proposing a novel linear attention module. This module combines partial convolution and depthwise separable convolution. The design of this attention module aims to simultaneously extract local features of the lesion region and enhance focus on key features. In the attention module, partial convolution refines the edge information of the lesion region, focusing on important boundary features through dynamic mask adjustments. Simultaneously, depthwise separable convolution efficiently extracts texture and vascular details from the image while reducing computational cost. The fusion of these two convolutional operations not only improves the module's ability to process lesion features but also forms an attention mechanism through their combined action, adaptively adjusting feature weights so that the module can focus more intently on the features of the lesion region.

[0215] This invention addresses the challenges of multi-scale texture features and complex feature relationships in lesion regions during skin disease image segmentation by proposing an innovative Multi-Scale Feature Fusion (MTM) module. This module effectively solves the problems of large differences in texture size and complex feature relationships in lesion regions by integrating depthwise separable convolution, bilinear interpolation, and linear attention mechanisms. The design philosophy of the MTM module is to not only extract local details of the lesion region but also enhance attention to key features. In the MTM module of this invention, depthwise separable convolution is used to refine the edge information of the lesion region, and bilinear interpolation is used to adjust the feature size to capture textures at different scales. The linear attention module focuses on the edge information of the lesion region through partial convolution operations and utilizes dynamic masking technology to specifically enhance key boundary features. Simultaneously, depthwise separable convolution is used to efficiently extract texture details from the image while reducing computational complexity, ensuring the module's high efficiency. The fusion of these two convolution techniques enables the MTM module to extract rich feature information from skin disease images at different scales, and the linear attention mechanism adaptively adjusts feature weights, making the network more focused on the features of the lesion region. This design not only enhances the network's ability to process lesion features, but also forms an effective attention mechanism through their combined action, improving the accuracy and efficiency of skin disease image segmentation.

[0216] In summary, this application provides a method for skin lesion segmentation, comprising: inputting a target skin lesion image into a convolutional neural network to extract image features to obtain an initial feature map; using an encoder with a wavelet convolutional layer to extract features from the initial feature map to obtain a wavelet feature map; multiplying the wavelet feature map with a tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain a processed wavelet feature map; dividing the processed wavelet feature map into four groups of feature maps; and performing HPA operations on the first three groups of feature maps along the height-width axis, the channel-height axis, and the channel-width axis, respectively. The fourth set of feature maps undergoes partial convolution, and the processing results of the four sets of feature maps are concatenated along the channel dimension. This concatenation is then multiplied again by the tensor p, which has undergone depthwise separable convolution and bilinear interpolation, to obtain a preliminary output feature map. The convolution kernel size is 3. The wavelet feature map and the preliminary output feature map are then fused at multiple scales to obtain multiple multi-scale fused feature maps. The preliminary output feature map is input into the decoder, and the intermediate output of the decoder is added to the multiple multi-scale fused feature maps. The final output of the decoder is the skin lesion segmentation result. This application, through grouped feature processing, the introduction of partial convolution, and multi-scale feature fusion, not only improves the accuracy of feature extraction but also reduces the number of model parameters, increases computational efficiency, and enhances the module's adaptability to data diversity.

[0217] Example 2

[0218] Based on the skin lesion segmentation method disclosed in the above embodiments of the present invention, Figure 7 Specifically, a skin lesion segmentation system applying this skin lesion segmentation method is disclosed.

[0219] like Figure 7 As shown in the figure, an embodiment of the present invention discloses a skin lesion segmentation system, the system comprising:

[0220] The first extraction module 701 is used to input the target skin lesion image into a convolutional neural network to extract image features and obtain an initial feature map;

[0221] The second extraction module 702 is used to extract features from the initial feature map using an encoder that incorporates a wavelet convolutional layer to obtain a wavelet feature map;

[0222] Processing module 703 is used to multiply the wavelet feature map with the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain a processed wavelet feature map. The processed wavelet feature map is divided into four groups of feature maps. The first three groups of feature maps are subjected to HPA operations along the height-width axis, channel-height axis, and channel-width axis, respectively. The fourth group of feature maps is subjected to a partial convolution operation. The processing results of the four groups of feature maps are concatenated along the channel dimension and then multiplied again with the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain a preliminary output feature map. The convolution kernel size is 3.

[0223] The fusion module 704 is used to perform multi-scale feature fusion on the wavelet feature map and the preliminary output feature map respectively to obtain multiple multi-scale fused feature maps.

[0224] The output module 705 is used to input the preliminary output feature map into the decoder, and to add the intermediate output result of the decoder to the plurality of multi-scale fused feature maps. The final output result of the decoder is the skin lesion segmentation result.

[0225] In some embodiments, the first extraction module 701 is used to input the target skin lesion image into a convolutional neural network, and extract image features from the target skin lesion image through a convolutional layer with a kernel size of 3x3 to obtain an initial feature map.

[0226] The specific working process of the first extraction module 701, the second extraction module 702, the processing module 703, the fusion module 704, and the output module 705 in the skin lesion segmentation system disclosed in the above embodiments of the present invention can be found in the corresponding content of the skin lesion segmentation method disclosed in the above embodiments of the present invention, and will not be repeated here.

[0227] In summary, this application provides a skin lesion segmentation system, comprising: inputting a target skin lesion image into a convolutional neural network to extract image features to obtain an initial feature map; using an encoder with a wavelet convolutional layer to extract features from the initial feature map to obtain a wavelet feature map; multiplying the wavelet feature map with a tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain a processed wavelet feature map; dividing the processed wavelet feature map into four groups of feature maps; and performing HPA operations on the first three groups of feature maps along the height-width axis, the channel-height axis, and the channel-width axis, respectively. The fourth set of feature maps undergoes partial convolution, and the processing results of the four sets of feature maps are concatenated along the channel dimension. This concatenation is then multiplied again by the tensor p, which has undergone depthwise separable convolution and bilinear interpolation, to obtain a preliminary output feature map. The convolution kernel size is 3. The wavelet feature map and the preliminary output feature map are then fused at multiple scales to obtain multiple multi-scale fused feature maps. The preliminary output feature map is input into the decoder, and the intermediate output of the decoder is added to the multiple multi-scale fused feature maps. The final output of the decoder is the skin lesion segmentation result. This application, through grouped feature processing, the introduction of partial convolution, and multi-scale feature fusion, not only improves the accuracy of feature extraction but also reduces the number of model parameters, increases computational efficiency, and enhances the module's adaptability to data diversity.

[0228] In the several embodiments provided in this application, it should be understood that the disclosed methods can also be implemented in other ways. The method embodiments described above are merely illustrative.

[0229] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0230] Although the embodiments disclosed in this application are as described above, the above content is merely for the purpose of facilitating understanding of this application and is not intended to limit this application. Any person skilled in the art to which this application pertains may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in this application; however, the scope of patent protection of this application shall still be determined by the scope defined in the appended claims.

Claims

1. A method for segmenting skin lesions, characterized in that, The method includes: The target skin lesion image is input into a convolutional neural network to extract image features and obtain an initial feature map; An encoder incorporating wavelet convolutional layers is used to extract features from the initial feature map to obtain a wavelet feature map; The wavelet feature map is multiplied by the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain the processed wavelet feature map. This processed wavelet feature map is divided into four groups of feature maps. The first three groups of feature maps are subjected to Hadamard attention operations along the height-width axis, channel-height axis, and channel-width axis, respectively. The fourth group of feature maps undergoes a partial convolution operation. The processing results of the four groups of feature maps are concatenated along the channel dimension and then multiplied again by the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain the preliminary output feature map. The convolution kernel size is 3, and the partial convolution operation dynamically adjusts the kernel's effective area based on data validity, performing convolution operations only on valid data points. The formula for calculating the tensor p is: ； ； In the formula, Y represents the input wavelet feature map, and X represents the processed wavelet feature map. This indicates a depthwise separable convolution operation. This represents a bilinear interpolation operation, where p0 is the initial tensor. The wavelet feature map and the preliminary output feature map are respectively subjected to multi-scale feature fusion to obtain multiple multi-scale fused feature maps; The initial output feature map is input into the decoder, and the intermediate output result of the decoder is added to the multiple multi-scale fused feature maps. The final output result of the decoder is the skin lesion segmentation result.

2. The method according to claim 1, characterized in that, The step of inputting the target skin lesion image into a convolutional neural network to extract image features and obtain an initial feature map includes: The target skin lesion image is input into a convolutional neural network, and image features are extracted from the target skin lesion image through a convolutional layer with a kernel size of 3x3 to obtain an initial feature map.

3. The method according to claim 1, characterized in that, Also includes: Perform a maximum downsampling operation on the wavelet feature map to obtain a first sampled wavelet feature map; The first sampled wavelet feature map is used as the initial feature map, and the encoder with the wavelet convolution layer is used repeatedly to extract features from the initial feature map to obtain the wavelet feature map. The wavelet feature map obtained by repeating the steps is subjected to maximum downsampling again to obtain the second sampled wavelet feature map, which is then used as the latest wavelet feature map.

4. The method according to claim 3, characterized in that, Also includes: Perform a maximum downsampling operation on the preliminary output feature map to obtain a first sampled preliminary output feature map; The first sampled preliminary output feature map is used as a wavelet feature map. The process of multiplying the wavelet feature map with the tensor p that has undergone depth-separable convolution and bilinear interpolation is repeated to obtain the preliminary output feature map. The multi-scale feature maps obtained from the repeated steps are subjected to maximum downsampling again to obtain the second sampled preliminary output feature map. The second sampled preliminary output feature map is used as a wavelet feature map. The process of multiplying the wavelet feature map with the tensor p that has undergone depth-separable convolution and bilinear interpolation is repeated to obtain a preliminary output feature map. The preliminary output feature map obtained by repeating the process is used as the latest preliminary output feature map.

5. The method according to claim 4, characterized in that, The step involves fusing the wavelet feature map and the initial output feature map at multiple scales to obtain multiple multi-scale fused feature maps, including: The second sampled preliminary output feature map is adjusted to the same spatial resolution as the latest preliminary output feature map by employing depthwise separable convolution and bilinear interpolation operations. The adjusted second sampled preliminary output feature map and the latest preliminary output feature map are respectively divided into a first feature group, a second feature group and a third feature group along the channel dimension; The first feature group of the adjusted second sampled preliminary output feature map and the first feature group of the latest preliminary output feature map are fused together, and the fused features are connected with the mask information to obtain the first fused feature. The second feature group of the adjusted second sampled preliminary output feature map and the second feature group of the latest preliminary output feature map are fused together, and the fused features are concatenated with the mask information to obtain the second fused feature. The third feature group of the adjusted second sampled preliminary output feature map and the third feature group of the latest preliminary output feature map are fused together, and the fused features are connected with the mask information to obtain the third fused feature. The first fused feature is subjected to dilated convolution and pooling operations according to the first dilation rate, and then processed by TripletAttention to obtain the processed first fused feature. The second fused feature is subjected to dilated convolution and pooling operations according to the second dilation rate, and then processed by TripletAttention to obtain the processed second fused feature. The third fusion feature is subjected to dilated convolution and pooling operations according to the third dilation rate, and then processed by TripletAttention to obtain the processed third fusion feature. The processed first fusion feature, the processed second fusion feature, and the processed third fusion feature are concatenated and processed by ordinary convolution with a kernel size of 1 to obtain the fifth multi-scale fusion feature map. By employing depthwise separable convolution and bilinear interpolation, the initial output feature map of the first sampling is adjusted to have the same spatial resolution as the fifth multi-scale fused feature map. The adjusted first sampled preliminary output feature map and the fifth multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, to obtain the fourth multi-scale fusion feature map; The second sampled wavelet feature map is adjusted to the same spatial resolution as the fourth multi-scale fused feature map by employing depthwise separable convolution and bilinear interpolation operations. The adjusted second-sampled wavelet feature map and the fourth multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second-sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, to obtain the third multi-scale fusion feature map; The first sampled wavelet feature map is adjusted to the same spatial resolution as the third multi-scale fused feature map by employing depthwise separable convolution and bilinear interpolation operations. The adjusted first sampled wavelet feature map and the third multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension to obtain the second multi-scale fusion feature map; The initial feature map is adjusted to the same spatial resolution as the second multi-scale fused feature map by employing depthwise separable convolution and bilinear interpolation operations. The adjusted initial feature map and the second multi-scale fusion feature map are repeatedly processed from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, to obtain the first multi-scale fusion feature map.

6. The method according to claim 5, characterized in that, The process involves inputting the initial output feature map into the decoder, and then performing an add operation on the intermediate output of the decoder and the multiple multi-scale fused feature maps. The final output of the decoder is the skin lesion segmentation result, which includes: The latest preliminary output feature map is input into the decoder, and the first decoding preliminary output feature map is obtained by multiplying the latest preliminary output feature map with the tensor p that has completed the depthwise separable convolution operation and bilinear interpolation operation. The fifth multi-scale fused feature map, obtained by multi-scale feature fusion based on the second sampled preliminary output feature map, is added to the first decoding preliminary output feature map to obtain the first decoding output result; The first decoding output result is multiplied by the tensor p that has completed the depth-separable convolution operation and bilinear interpolation operation to obtain the second decoding preliminary output feature map; The second initial decoded output feature map is subjected to bilinear interpolation to obtain the second initial decoded output feature map after bilinear interpolation. The fourth multi-scale fused feature map obtained by multi-scale feature fusion based on the first sampled preliminary output feature map is added to the second decoding preliminary output feature map after bilinear interpolation to obtain the second decoding output result. The third decoding preliminary output feature map is obtained by multiplying the second decoding output result with the tensor p that has completed the depth-separable convolution operation and bilinear interpolation operation. The third decoding preliminary output feature map is subjected to bilinear interpolation to obtain the third decoding preliminary output feature map after bilinear interpolation. The third multi-scale fused feature map obtained by multi-scale feature fusion based on the second sampled wavelet feature map is added to the third decoding preliminary output feature map after the bilinear interpolation operation to obtain the third decoding output result. Wavelet convolution is used to extract features from the third decoding output to obtain the preliminary output feature map of the fourth decoding. The fourth decoding preliminary output feature map is subjected to bilinear interpolation to obtain the fourth decoding preliminary output feature map after bilinear interpolation. The second multi-scale fused feature map obtained by multi-scale feature fusion based on the first sampled wavelet feature map is added to the fourth decoding preliminary output feature map after the bilinear interpolation operation to obtain the fourth decoding output result. Wavelet convolution is used to extract features from the fourth decoding output to obtain the preliminary output feature map of the fifth decoding. The fifth decoded preliminary output feature map is subjected to bilinear interpolation to obtain the fifth decoded preliminary output feature map after bilinear interpolation. The first multi-scale fused feature map obtained by multi-scale feature fusion based on the initial feature map is added to the fifth decoding preliminary output feature map after bilinear interpolation to obtain the fifth decoding output result. Two-dimensional convolution is used to extract features from the fifth decoding output to obtain the preliminary output feature map of the sixth decoding. The sixth decoded preliminary output feature map is subjected to bilinear interpolation to obtain the sixth decoded preliminary output feature map after bilinear interpolation. The sixth decoded preliminary output feature map after bilinear interpolation is used as the skin lesion segmentation result.

7. The method according to claim 1, characterized in that, Also includes: The initial feature map is subjected to a maximum downsampling operation, and the sampling result is used as the latest initial feature map.

8. A skin lesion segmentation system, characterized in that, The system includes: The first extraction module is used to input the target skin lesion image into a convolutional neural network to extract image features and obtain an initial feature map; The second extraction module is used to extract features from the initial feature map using an encoder that incorporates a wavelet convolutional layer to obtain a wavelet feature map. The processing module is used to multiply the wavelet feature map with the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain the processed wavelet feature map. The processed wavelet feature map is divided into four groups of feature maps. The first three groups of feature maps are subjected to Hadamard attention operations along the height-width axis, channel-height axis, and channel-width axis, respectively. The fourth group of feature maps is subjected to partial convolution. The processing results of the four groups of feature maps are concatenated along the channel dimension and multiplied again with the tensor p that has undergone depthwise separable convolution and bilinear interpolation to obtain the preliminary output feature map. The convolution kernel size is 3, and the partial convolution operation dynamically adjusts the kernel's effective area based on data validity, performing convolution only on valid data points. The formula for calculating the tensor p is: ； ； In the formula, Y represents the input wavelet feature map, and X represents the processed wavelet feature map. This indicates a depthwise separable convolution operation. This represents a bilinear interpolation operation, where p0 is the initial tensor. The fusion module is used to perform multi-scale feature fusion on the wavelet feature map and the preliminary output feature map respectively to obtain multiple multi-scale fused feature maps; The output module is used to input the preliminary output feature map into the decoder, and to add the intermediate output result of the decoder to the multiple multi-scale fused feature maps. The final output result of the decoder is the skin lesion segmentation result.

9. The system according to claim 8, characterized in that, The first extraction module is used to input the target skin lesion image into a convolutional neural network, and extract image features from the target skin lesion image through a convolutional layer with a kernel size of 3x3 to obtain an initial feature map.

Images