Skin lesion segmentation method and system

Abstract

The invention discloses a skin lesion segmentation method. The method comprises the following steps: extracting an initial feature map from a target skin lesion image; carrying out feature extraction by adopting an encoder introduced with a wavelet convolution layer to obtain a wavelet feature map; multiplying the wavelet feature map with the tensor p after the depth separable convolution operation and the bilinear interpolation operation to obtain a processed wavelet feature map, and dividing the wavelet feature map into four groups of feature mappings for processing to obtain a preliminary output feature map; and performing multi-scale feature fusion on the wavelet feature map and the preliminary output feature map, inputting the preliminary output feature map into a decoder, performing add operation on an intermediate output result of the decoder and a plurality of multi-scale fusion feature maps, and taking a final output result of the decoder as a skin lesion segmentation result. Through grouping feature processing, partial convolution introduction and multi-scale feature fusion, the accuracy of feature extraction is improved, the parameter quantity of the model is reduced, the calculation efficiency is improved, and the adaptability of the module to data diversity is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a skin lesion segmentation method and system. Background Art

[0002] Malignant melanoma is a deadly skin cancer with an increasing incidence worldwide, causing a large number of deaths each year. Automatic skin lesion segmentation systems have been widely used in the field of medical diagnosis because they can quickly and accurately identify lesion areas.

[0003] Among the existing technical solutions, Dense-UNet improves the ability to capture multi-scale information by using Dense-blocks to achieve effective reuse of features. U-Netv2 adopts an innovative jump connection mechanism to combine high-level and low-level features, achieve more refined feature fusion, and enhance the integration of features at different scales. The FCBFormer model adopts a dual-path structure, combining a fully convolutional network (FCN) and a Vision Transformer (ViT) to simultaneously extract local and global features to improve segmentation accuracy. The UNeXt model combines UNet with a multi-layer perceptron (MLP) to create a lightweight network structure that reduces parameters and computation while maintaining high performance. The MALUNet model further reduces the model size by reducing the number of channels and introducing an attention module, and has demonstrated superior performance to UNeXt in skin lesion segmentation tasks.

[0004] Although existing technical solutions have made some progress in the field of automatic skin lesion segmentation, they still have the following shortcomings: Models such as Dense-UNet and U-Net v2 perform poorly when dealing with problems where the lesion area has significant size differences and complex texture features. These models fail to effectively address the challenges of size differences and texture complexity, resulting in segmentation accuracy being affected. Although high-precision segmentation models such as FCBFormer can provide accurate segmentation results, they require a large number of parameters and computing resources. This high resource requirement limits their popularity and application in medical devices and mobile devices with limited computing power. Although lightweight models such as MALUNet increase popularity by reducing computing costs, their segmentation accuracy is usually not comparable to models with large parameters. This compromise in accuracy may affect the reliability of diagnosis, especially in complex cases that require high-precision segmentation. Therefore, existing technical solutions have a difficult-to-balance contradiction between pursuing lightweight and high accuracy. Summary of the invention

[0005] In response to the above problems, the present application provides a skin lesion segmentation method and system, which solves the technical problem in the related art that there is a difficult-to-balance contradiction between the pursuit of lightweight and high precision.

[0006] In a first aspect, the present application provides a skin lesion segmentation method, the method comprising:

[0007] Input the target skin lesion image into the convolutional neural network to extract image features and obtain an initial feature map;

[0008] Using an encoder that introduces a wavelet convolution layer to extract features from the initial feature map to obtain a wavelet feature map;

[0009] According to the multiplication of the wavelet feature map and the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation, a processed wavelet feature map is obtained, the processed wavelet feature map is divided into four groups of feature maps, the first three groups of feature maps are respectively subjected to HPA operations along the height-width axis, the channel-height axis and the channel-width axis, the fourth group of feature maps is subjected to partial convolution operations, and the processing results of the four groups of feature maps are spliced ​​along the channel dimension, and then multiplied again with the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map; wherein the convolution kernel size is 3;

[0010] Performing multi-scale feature fusion on the wavelet feature map and the preliminary output feature map respectively to obtain a plurality of multi-scale fused feature maps;

[0011] The preliminary output feature map is input into a decoder, and an add operation is performed on the intermediate output result of the decoder and the multiple multi-scale fusion feature maps, and the final output result of the decoder is a skin lesion segmentation result.

[0012] In some embodiments, the step of inputting the target skin lesion image into a convolutional neural network to extract image features and obtain an initial feature map includes:

[0013] The target skin lesion image is input into the convolutional neural network, and the image features of the target skin lesion image are extracted through a convolution layer with a convolution kernel size of 3x3 to obtain an initial feature map.

[0014] In some embodiments, the multiplication of the wavelet feature map with the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation to obtain the processed wavelet feature map includes:

[0015] Through the tensor p formula:

[0016] p=DepthwiseConv[BilinearInterpolation(p,size=Y.size())]

[0017] X=p*Y

[0018] Among them, Y represents the input wavelet feature map, X represents the processed wavelet feature map, DepthwiseConv represents the depthwise separable convolution operation, and BilinearInterpolation represents the bilinear interpolation operation.

[0019] In some embodiments, the method further comprises:

[0020] Performing a maximum downsampling operation on the wavelet feature map to obtain a first sampled wavelet feature map;

[0021] The first sampled wavelet feature map is used as an initial feature map, and an encoder introduced with a wavelet convolution layer is repeatedly used to extract features from the initial feature map to obtain a wavelet feature map;

[0022] The wavelet feature map obtained by repeating the steps is subjected to a maximum downsampling operation again to obtain a second sampled wavelet feature map, and the second sampled wavelet feature map is used as the latest wavelet feature map.

[0023] In some embodiments, the method further comprises:

[0024] Performing a maximum downsampling operation on the preliminary output feature map to obtain a first sampled preliminary output feature map;

[0025] The first sampled preliminary output feature map is used as a wavelet feature map, and the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation is multiplied by the wavelet feature map to obtain a preliminary output feature map.

[0026] The multi-scale feature map obtained by the repeated steps is subjected to the maximum downsampling operation again to obtain the second sampling preliminary output feature map;

[0027] The second sampled preliminary output feature map is used as a wavelet feature map, and the step of multiplying the wavelet feature map by the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map is repeated again, and the preliminary output feature map obtained by repeating the steps again is used as the latest preliminary output feature map.

[0028] In some embodiments, the wavelet feature map and the preliminary output feature map are respectively subjected to multi-scale feature fusion to obtain a plurality of multi-scale fused feature maps, including:

[0029] Using a depthwise separable convolution operation and a bilinear interpolation operation, adjusting the second sampled preliminary output feature map to the same spatial resolution as the latest preliminary output feature map;

[0030] Dividing the adjusted second sample preliminary output feature map and the latest preliminary output feature map into a first feature group, a second feature group and a third feature group along the channel dimension respectively;

[0031] Fusing the adjusted first feature group of the second sampled preliminary output feature map with the first feature group of the latest preliminary output feature map, and connecting the fused features with the mask information to obtain a first fused feature;

[0032] Performing feature fusion on the second feature group of the adjusted second sampled preliminary output feature map and the second feature group of the latest preliminary output feature map, and connecting the fused features with the mask information to obtain a second fused feature;

[0033] Performing feature fusion on the third feature group of the adjusted second sample preliminary output feature map and the third feature group of the latest preliminary output feature map, and connecting the fused features with the mask information to obtain a third fused feature;

[0034] Performing a dilated convolution operation and a pooling operation on the first fused feature according to a first dilation rate, and then performing a Triplet Attention process to obtain a processed first fused feature;

[0035] Performing a dilated convolution operation and a pooling operation on the second fused feature according to a second dilation rate, and then performing a Triplet Attention process to obtain a processed second fused feature;

[0036] Performing a dilated convolution operation and a pooling operation on the third fused feature according to a third dilation rate, and then performing a Triplet Attention process to obtain a processed third fused feature;

[0037] splicing the processed first fusion feature, the processed second fusion feature, and the processed third fusion feature, and processing them through ordinary convolution with a convolution kernel size of 1 to obtain a fifth multi-scale fusion feature map;

[0038] Using a depthwise separable convolution operation and a bilinear interpolation operation, the first sampling preliminary output feature map is adjusted to the same spatial resolution as the fifth multi-scale fusion feature map;

[0039] Repeat the steps starting from dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, on the adjusted first sampled preliminary output feature map and the fifth multi-scale fused feature map to obtain a fourth multi-scale fused feature map;

[0040] Using a depthwise separable convolution operation and a bilinear interpolation operation, adjusting the second sampled wavelet feature map to the same spatial resolution as the fourth multi-scale fused feature map;

[0041] Repeat the steps starting from dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into a first feature group, a second feature group and a third feature group along the channel dimension, with the adjusted second sampled wavelet feature map and the fourth multi-scale fusion feature map, to obtain a third multi-scale fusion feature map;

[0042] Using a depthwise separable convolution operation and a bilinear interpolation operation, adjusting the first sampled wavelet feature map to the same spatial resolution as the third multi-scale fused feature map;

[0043] Repeat the steps starting from dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, to obtain a second multi-scale fusion feature map;

[0044] Using a depthwise separable convolution operation and a bilinear interpolation operation, the initial feature map is adjusted to the same spatial resolution as the second multi-scale fused feature map;

[0045] The adjusted initial feature map and the second multi-scale fused feature map are repeatedly executed starting from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into a first feature group, a second feature group and a third feature group along the channel dimension, respectively, to obtain a first multi-scale fused feature map.

[0046] In some embodiments, the preliminary output feature map is input into a decoder, and an add operation is performed on the intermediate output result of the decoder and the multiple multi-scale fusion feature maps, and the final output result of the decoder is a skin lesion segmentation result, including:

[0047] Inputting the latest preliminary output feature map into a decoder, and multiplying the latest preliminary output feature map by a tensor p that completes a depthwise separable convolution operation and a bilinear interpolation operation, to obtain a first decoded preliminary output feature map;

[0048] Performing an add operation on a fifth multi-scale fused feature map obtained by performing multi-scale feature fusion according to the second sampling preliminary output feature map and the first decoding preliminary output feature map to obtain a first decoding output result;

[0049] Multiplying the first decoding output result by the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation to obtain a second decoding preliminary output feature map;

[0050] Performing a bilinear interpolation operation on the second decoded preliminary output feature map to obtain a second decoded preliminary output feature map after the bilinear interpolation operation;

[0051] Performing an add operation on a fourth multi-scale fused feature map obtained by performing multi-scale feature fusion according to the first sampling preliminary output feature map and the second decoded preliminary output feature map after the bilinear interpolation operation to obtain a second decoding output result;

[0052] Multiplying the second decoding output result by the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation to obtain a third decoding preliminary output feature map;

[0053] Performing a bilinear interpolation operation on the third decoded preliminary output feature map to obtain a third decoded preliminary output feature map after the bilinear interpolation operation;

[0054] Performing an add operation on a third multiscale fused feature map obtained by performing multiscale feature fusion according to the second sampled wavelet feature map and the third decoded preliminary output feature map after the bilinear interpolation operation to obtain a third decoding output result;

[0055] Using wavelet convolution to perform feature extraction on the third decoding output result to obtain a fourth decoding preliminary output feature map;

[0056] Performing a bilinear interpolation operation on the fourth decoded preliminary output feature map to obtain a fourth decoded preliminary output feature map after the bilinear interpolation operation;

[0057] Performing an add operation on a second multi-scale fused feature map obtained by performing multi-scale feature fusion according to the first sampled wavelet feature map and the fourth decoded preliminary output feature map after the bilinear interpolation operation to obtain a fourth decoding output result;

[0058] Performing feature extraction on the fourth decoding output result by using wavelet convolution to obtain a fifth decoding preliminary output feature map;

[0059] Performing a bilinear interpolation operation on the fifth decoding preliminary output feature map to obtain a fifth decoding preliminary output feature map after the bilinear interpolation operation;

[0060] Performing an add operation on a first multi-scale fused feature map obtained by performing multi-scale feature fusion according to the initial feature map and a fifth decoding preliminary output feature map after the bilinear interpolation operation to obtain a fifth decoding output result;

[0061] Performing feature extraction on the fifth decoding output result by using two-dimensional convolution to obtain a sixth decoding preliminary output feature map;

[0062] A bilinear interpolation operation is performed on the sixth decoded preliminary output feature map to obtain a sixth decoded preliminary output feature map after the bilinear interpolation operation, and the sixth decoded preliminary output feature map after the bilinear interpolation operation is used as a skin lesion segmentation result.

[0063] In some embodiments, the method further comprises:

[0064] A maximum downsampling operation is performed on the initial feature map, and the sampling result is used as the latest initial feature map.

[0065] In a second aspect, a skin lesion segmentation system is provided, the system comprising:

[0066] The first extraction module is used to input the target skin lesion image into the convolutional neural network to extract image features and obtain an initial feature map;

[0067] A second extraction module is used to extract features from the initial feature map using an encoder that introduces a wavelet convolution layer to obtain a wavelet feature map;

[0068] A processing module, used for multiplying the wavelet feature map with the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain a processed wavelet feature map, dividing the processed wavelet feature map into four groups of feature maps, performing HPA operations on the first three groups of feature maps along the height-width axis, the channel-height axis and the channel-width axis respectively, performing a partial convolution operation on the fourth group of feature maps, and splicing the processing results of the four groups of feature maps along the channel dimension, and multiplying them again with the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map; wherein the convolution kernel size is 3;

[0069] A fusion module, used for performing multi-scale feature fusion on the wavelet feature map and the preliminary output feature map to obtain multiple multi-scale fused feature maps;

[0070] The output module is used to input the preliminary output feature map into a decoder, and perform an add operation on the intermediate output result of the decoder and the multiple multi-scale fusion feature maps, and the final output result of the decoder is the skin lesion segmentation result.

[0071] In some embodiments, the first extraction module is used to input the target skin lesion image into a convolutional neural network, extract image features of the target skin lesion image through a convolution layer with a convolution kernel size of 3x3, and obtain an initial feature map.

[0072] The present application provides a skin lesion segmentation method and system, comprising: inputting a target skin lesion image into a convolutional neural network to extract image features to obtain an initial feature map; using an encoder that introduces a wavelet convolution layer to extract features from the initial feature map to obtain a wavelet feature map; multiplying the wavelet feature map with a tensor p that completes a depth-separable convolution operation and a bilinear interpolation operation to obtain a processed wavelet feature map, dividing the processed wavelet feature map into four groups of feature maps, performing HPA operations on the first three groups of feature maps along a height-width axis, a channel-height axis, and a channel-width axis, respectively, and performing HPA operations on the fourth group of feature maps along a height-width axis, a channel-height axis, and a channel-width axis, respectively. The group feature map performs a partial convolution operation, and after splicing the processing results of the four groups of feature maps along the channel dimension, they are multiplied again with the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map; wherein the convolution kernel size is 3; the wavelet feature map and the preliminary output feature map are respectively subjected to multi-scale feature fusion to obtain multiple multi-scale fusion feature maps; the preliminary output feature map is input into the decoder, and the intermediate output result of the decoder is added to the multiple multi-scale fusion feature maps, and the final output result of the decoder is the skin lesion segmentation result. This application not only improves the accuracy of feature extraction, but also reduces the number of model parameters, improves computational efficiency, and enhances the module's adaptability to data diversity by grouping feature processing, introducing partial convolution and multi-scale feature fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0074] Figure 1 A schematic diagram of a process for segmenting skin lesions provided in an embodiment of the present application;

[0075] Figure 2 A schematic diagram of the wavelet convolution feature extraction process provided in the embodiment of the present application;

[0076] Figure 3 A schematic diagram of a linear attention module provided in an embodiment of the present application;

[0077] Figure 4 A schematic diagram of a multi-scale feature fusion module provided in an embodiment of the present application;

[0078] Figure 5 A complete convolutional neural network schematic diagram provided for an embodiment of the present application;

[0079] Figure 6 This is a rendering of the skin lesion segmentation method provided in the embodiment of the present application verified in a data set;

[0080] Figure 7 A schematic diagram of the structure of a skin lesion segmentation system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The following will describe the implementation methods of the present application in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present application applies technical means to solve technical problems and achieve corresponding technical effects can be fully understood and implemented accordingly. The embodiments of the present application and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present application.

[0082] Embodiment 1

[0083] Figure 1 A schematic diagram of a process for segmenting skin lesions provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0084] S101, inputting the target skin lesion image into a convolutional neural network to extract image features and obtain an initial feature map;

[0085] S102, using an encoder that introduces a wavelet convolution layer to perform feature extraction on the initial feature map to obtain a wavelet feature map;

[0086] S103, multiplying the wavelet feature map by the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation to obtain a processed wavelet feature map, dividing the processed wavelet feature map into four groups of feature maps, performing HPA operations on the first three groups of feature maps along the height-width axis, the channel-height axis, and the channel-width axis, respectively, performing a partial convolution operation on the fourth group of feature maps, and splicing the processing results of the four groups of feature maps along the channel dimension, and multiplying them again with the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map; wherein the convolution kernel size is 3;

[0087] S104, performing multi-scale feature fusion on the wavelet feature map and the preliminary output feature map respectively to obtain multiple multi-scale fusion feature maps;

[0088] S105. Input the preliminary output feature map into a decoder, and perform an add operation on the intermediate output result of the decoder and the multiple multi-scale fusion feature maps. The final output result of the decoder is a skin lesion segmentation result.

[0089] It should be noted that in view of the limitations of existing models, especially their deficiencies in dealing with problems where the lesion area has significant size differences and complex texture features, the present invention discloses an efficient and lightweight skin disease segmentation method. The main purpose of this method is to significantly reduce the number of model parameters and computational costs while maintaining high accuracy, thereby improving the operational efficiency of the model. This method is easier to deploy and apply on resource-constrained medical devices and mobile devices, providing medical professionals with a fast, accurate and easily accessible skin lesion segmentation tool. Through the present invention, we hope to overcome the shortcomings of the prior art in terms of the ability to handle size diversity and texture complexity, as well as the problem of excessive consumption of computing resources, and achieve more extensive and convenient clinical applications.

[0090] In some embodiments, the step of inputting the target skin lesion image into a convolutional neural network to extract image features and obtain an initial feature map includes:

[0091] The target skin lesion image is input into the convolutional neural network, and the image features of the target skin lesion image are extracted through a convolution layer with a convolution kernel size of 3x3 to obtain an initial feature map.

[0092] In some embodiments, the method further comprises:

[0093] A maximum downsampling operation is performed on the initial feature map, and the sampling result is used as the latest initial feature map.

[0094] In some embodiments, the multiplication of the wavelet feature map with the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation to obtain the processed wavelet feature map includes:

[0095] Through the tensor p formula:

[0096] p=DepthwiseConv[BilinearInterpolation(p,size=Y.size())]

[0097] X=p*Y

[0098] Among them, Y represents the input wavelet feature map, X represents the processed wavelet feature map, DepthwiseConv represents the depthwise separable convolution operation, and BilinearInterpolation represents the bilinear interpolation operation.

[0099] In some embodiments, the method further comprises:

[0100] Performing a maximum downsampling operation on the wavelet feature map to obtain a first sampled wavelet feature map;

[0101] The first sampled wavelet feature map is used as an initial feature map, and an encoder introduced with a wavelet convolution layer is repeatedly used to extract features from the initial feature map to obtain a wavelet feature map;

[0102] The wavelet feature map obtained by repeating the steps is subjected to a maximum downsampling operation again to obtain a second sampled wavelet feature map, and the second sampled wavelet feature map is used as the latest wavelet feature map.

[0103] It should be noted that the introduction of a wavelet convolution layer in the encoder part of the convolutional model to extract features from the initial skin lesion feature map can reduce the number of network parameters and expand the receptive field, allowing the network to more effectively capture the key features of the image.

[0104] like Figure 2 As shown in the figure, it is a schematic diagram of the wavelet convolution feature extraction process. First, the wavelet object is initialized to obtain the decomposition and reconstruction filter coefficients, and these coefficients are converted into tensors and flipped to meet the needs of the convolution operation. Then, we construct the decomposition filter bank, combining the low-pass filter and the high-pass filter into four different filters (LL, LH, HL, HH), and copying these filters according to the number of input channels. At the same time, we also construct the reconstruction filter bank, combining filters in the same way and copying them according to the number of output channels.

[0105] For the input skin lesion feature map, we perform a multi-level wavelet transform. In each level of the transform, the input feature map is convolved using a decomposition filter bank. For the i-th level wavelet transform, the details are as follows:

[0106]

[0107] Where h and g represent the low-pass filter and high-pass filter in wavelet transform, respectively. It means that the low-pass filter h performs a convolution operation with itself, and then uses this result to convolve the feature map F of the previous level. i-1 Convolution is performed to obtain the approximate part (low-frequency part) of the i-th level wavelet transform. The other three formulas are the same. i Indicates low-frequency-low-frequency subband (approximate subband), representing the low-frequency part of the image, LH i Represents the low-frequency-high-frequency subband, which contains the horizontal edge information of the image. i Represents the high-low frequency subband, which contains the vertical edge information of the image. iRepresents the high-frequency-high-frequency subband (detail subband), which contains the image's detail information, such as texture and noise. The LL subband is downsampled and retained for the next level of wavelet transform, while the LH, HL, and HH subbands are further processed. For these high-frequency subbands, we flatten them and convolve them with the corresponding convolution kernels, then apply a learnable scale factor to scale them, and finally combine the scaled subbands with the low-frequency LL subbands and re-stitch them into a four-channel form to obtain the skin lesion feature map F_combined after a layer of wavelet transform. i ,Right now:

[0108] F_combined i =Concat(downsample(LL i ),α*LH i ,α*HL i ,α*HH i )

[0109] Where downsample represents the downsampling operation and α represents the learnable scale factor

[0110] Then use the LL subband of the previous stage as the input of the next stage and repeat the above steps.

[0111] After completing the multi-level wavelet transform, the inverse wavelet transform is performed: in each level, the combined skin lesion feature map is convolved through the reconstruction filter bank, and up-sampled by transposed convolution with a step size of 2. Then the skin lesion feature map obtained after upsampling is added to the high-frequency feature map of the previous level, and gradually restored to the original size. The specific formula is as follows:

[0112] F_merged i =ConTranspose(F_combined i ,G)+Concat(LH i-1 ,HH i-1 ,HL i-1 )

[0113] After all wavelet transform series are completed, a basic convolution operation is performed on the original input skin lesion feature map F0 and scaled using a learnable scale factor β. The result of the basic convolution is added to the result of the inverse wavelet transform to obtain the final output skin lesion feature map F_final, as follows:

[0114] F_final=β*Conv(F0)+F_merged0

[0115] If the model sets a step size greater than 1, an additional downsampling operation is performed on the resulting wavelet feature map to ensure that the size of the output feature map meets the requirements.

[0116] Finally, the feature map obtained by wavelet convolution is subjected to maximum downsampling operation.

[0117] To improve accuracy, the above steps can be repeated, usually twice.

[0118] In some embodiments, the method further comprises:

[0119] Performing a maximum downsampling operation on the preliminary output feature map to obtain a first sampled preliminary output feature map;

[0120] The first sampled preliminary output feature map is used as a wavelet feature map, and the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation is multiplied by the wavelet feature map to obtain a preliminary output feature map.

[0121] The multi-scale feature map obtained by the repeated steps is subjected to the maximum downsampling operation again to obtain the second sampling preliminary output feature map;

[0122] The second sampled preliminary output feature map is used as a wavelet feature map, and the step of multiplying the wavelet feature map by the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map is repeated again, and the preliminary output feature map obtained by repeating the steps again is used as the latest preliminary output feature map.

[0123] It should be noted that if Figure 3 As shown, it is a schematic diagram of the linear attention module provided by the embodiment of the present application. Specifically, the above content is combined with step S103, and the wavelet feature map is multiplied by the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain the processed wavelet feature map; specifically, through the tensor p formula:

[0124] p=DepthwiseConv[BilinearInterpolation(p,size=Y.size())]

[0125] X=p*Y

[0126] Among them, Y represents the input wavelet feature map, X represents the processed wavelet feature map, DepthwiseConv represents the depthwise separable convolution operation, and BilinearInterpolation represents the bilinear interpolation operation.

[0127] It should be noted that in the second half of the convolutional network, we designed a linear attention module (Grouplinear Attention Module, GAM), which is applied to the last three levels of the encoder to deeply explore the deep information of the image from multiple angles. The core of the module is a learnable tensor p, which is resized by bilinear interpolation to match the size of the input skin lesion feature map. On this basis, we use depthwise separable convolution to process the tensor p, and then generate preliminary output through the standard product operation with the input feature map.

[0128] Then the processed wavelet feature map X is evenly divided into four groups of feature maps (chunk operation) along the channel dimension, namely x1, x2, x3, and x4. The first three groups of features x1, x2, and x3 perform Hadamard Product Attention (HPA) operations along the height-width axis, channel-height axis, and channel-width axis (i.e., height-width axis, channel-height axis, and channel-width axis processing) to fully mine information of different dimensions. At the same time, we introduce partial convolution to process feature map x4:

[0129] x1,x2,x3,x4=SplitChannels(X,num_groups=4)

[0130] x1=HPA(x1,axis=(0,1))

[0131] x2=HPA(x2,axis=(1,2))

[0132] x3=HPA(x3,axis=(1,3))

[0133] x4=PartialConv(x4)

[0134] PartialConv represents a partial convolution operation. Partial convolution can dynamically adjust the area of ​​action of the convolution kernel according to the validity of the data, and only perform convolution operations on valid data points. This method not only improves the accuracy of feature extraction, but also reduces the number of model parameters, improves computational efficiency, and enhances the module's adaptability to data diversity.

[0135] After completing the independent processing of the four sets of features, the four sets of features are concatenated along the channel dimension and normalized, and then the depthwise separable convolution (DepthwiseConv) is applied again to integrate the information from different angles of the skin lesion feature map to obtain the final output feature map output (i.e., the preliminary output feature map). The convolution kernel size used in the whole process is 3, which helps to maintain the locality of the features and reduce the number of parameters of the model. The process is as follows:

[0136] output=DepthwiseConv(Concat(x1,x2,x3,x4,dim=1))

[0137] Then, the preliminary output feature map obtained by the GAM module is subjected to the maximum downsampling operation.

[0138] In order to improve the accuracy, the above steps can be repeated, generally three times, and the last operation does not require the maximum downsampling operation.

[0139] In some embodiments, the wavelet feature map and the preliminary output feature map are respectively subjected to multi-scale feature fusion to obtain a plurality of multi-scale fused feature maps, including:

[0140] Using a depthwise separable convolution operation and a bilinear interpolation operation, adjusting the second sampled preliminary output feature map to the same spatial resolution as the latest preliminary output feature map;

[0141] Dividing the adjusted second sample preliminary output feature map and the latest preliminary output feature map into a first feature group, a second feature group and a third feature group along the channel dimension respectively;

[0142] Fusing the adjusted first feature group of the second sampled preliminary output feature map with the first feature group of the latest preliminary output feature map, and connecting the fused features with the mask information to obtain a first fused feature;

[0143] Performing feature fusion on the second feature group of the adjusted second sampled preliminary output feature map and the second feature group of the latest preliminary output feature map, and connecting the fused features with the mask information to obtain a second fused feature;

[0144] Performing feature fusion on the third feature group of the adjusted second sample preliminary output feature map and the third feature group of the latest preliminary output feature map, and connecting the fused features with the mask information to obtain a third fused feature;

[0145] Performing a dilated convolution operation and a pooling operation on the first fused feature according to a first dilation rate, and then performing a Triplet Attention process to obtain a processed first fused feature;

[0146] Performing a dilated convolution operation and a pooling operation on the second fused feature according to a second dilation rate, and then performing a Triplet Attention process to obtain a processed second fused feature;

[0147] Performing a dilated convolution operation and a pooling operation on the third fused feature according to a third dilation rate, and then performing a Triplet Attention process to obtain a processed third fused feature;

[0148] splicing the processed first fusion feature, the processed second fusion feature, and the processed third fusion feature, and processing them through ordinary convolution with a convolution kernel size of 1 to obtain a fifth multi-scale fusion feature map;

[0149] Using a depthwise separable convolution operation and a bilinear interpolation operation, the first sampling preliminary output feature map is adjusted to the same spatial resolution as the fifth multi-scale fusion feature map;

[0150] Repeat the steps starting from dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, on the adjusted first sampled preliminary output feature map and the fifth multi-scale fused feature map to obtain a fourth multi-scale fused feature map;

[0151] Using a depthwise separable convolution operation and a bilinear interpolation operation, adjusting the second sampled wavelet feature map to the same spatial resolution as the fourth multi-scale fused feature map;

[0152] Repeat the steps starting from dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into a first feature group, a second feature group and a third feature group along the channel dimension, with the adjusted second sampled wavelet feature map and the fourth multi-scale fusion feature map, to obtain a third multi-scale fusion feature map;

[0153] Using a depthwise separable convolution operation and a bilinear interpolation operation, adjusting the first sampled wavelet feature map to the same spatial resolution as the third multi-scale fused feature map;

[0154] Repeat the steps starting from dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, to obtain a second multi-scale fusion feature map;

[0155] Using a depthwise separable convolution operation and a bilinear interpolation operation, the initial feature map is adjusted to the same spatial resolution as the second multi-scale fused feature map;

[0156] The adjusted initial feature map and the second multi-scale fused feature map are repeatedly executed starting from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into a first feature group, a second feature group and a third feature group along the channel dimension, respectively, to obtain a first multi-scale fused feature map.

[0157] It should be noted that in this step we designed a multi-scale feature fusion module (Multi-scaleTE Module, MTM) to replace the traditional skip connection operation.

[0158] Specifically, Figure 4 As shown, it is a schematic diagram of the multi-scale feature fusion module provided in the embodiment of the present application. The input of the current MTM will be fused with the multi-scale fusion feature map output by the next-order MTM. The MTM module plays a key role in fusing feature information of different scales. It not only enhances the network's ability to segment skin disease images, but also enables the network to extract richer features at different scales.

[0159] The specific process is as follows:

[0160] First, we use depthwise separable convolution and bilinear interpolation to adjust the size of high-level features to make them consistent with the spatial resolution of low-level features. Then, the adjusted high-level features (input of the current MTM) and low-level features (output of the next-order MTM) are divided into three groups along the channel dimension, namely group1, group2, and group3, and each group of low-level features is concatenated with the corresponding high-level features to form fused features. At the same time, these fused features are connected with the mask information:

[0161] group1,group2,group3=SplitChannels(low_level_features,num_groups

[0162] =3)

[0163] concated_group1

[0164] =Concat(Mask,Concat(group1,high_level_features_resized[0],dim=1)))

[0165] concated_group2

[0166] =Concat(Mask,Concat(group2,high_level_features_resized[1],dim=1)))

[0167] concated_group2

[0168] =Concat(Mask,Concat(group3,high_level_features_resized[2],dim=1)))

[0169] Among them, SplitChannels represents the grouping operation, Concat represents the concatenation operation on the channel dimension, and high_level_features_resized represents the adjusted high-level features.

[0170] Furthermore, in order to extract multi-scale information, we apply dilated convolution with different dilation rates (dilation rates are 3, 5, and 7 respectively) to the fused features to capture contextual information of different scales. On this basis, we introduce the TripletAttention channel attention module, which uses an almost parameter-free attention mechanism to deeply explore the intrinsic connection between features. The Triplet Attention module contains three branches: the first branch is the channel attention calculation branch. After the input features are pooled, they are processed by 7x7 convolution and the spatial attention weights are generated using the Sigmoid activation function. The second branch is the channel C and spatial W dimension interaction capture branch. After the input features are transformed in dimension, they are pooled in the H dimension and then processed in a series of ways to restore them to CXHXW dimensional features for element-wise addition. The third branch is the channel C and spatial H dimension interaction capture branch. The operation process is similar to the second branch, except that the pooling operation is performed in the W dimension. The output features of the three branches are added and averaged, which not only avoids the information loss problem in traditional methods, but also effectively captures the interaction between different dimensions in the feature map. The above operations can be summarized as the following formula:

[0171] channel_attention=ChannelAttention(dilated_features)

[0172] spatial_HW_attention=SpatialHWAttention(dilated_features)

[0173] spatial_CW_attention=SpatialCWAttention(dilated_features)

[0174] output=(channel_attention+spatial_HW_attention

[0175] +spatial_CW_attention) / 3

[0176] Among them, channel_attention, spatial_HW_attention, and spatial_CW_attention represent the three branches of the Triplet Attention module respectively, and dilated_features represents the fused feature map obtained after pooling and dilated convolution operations.

[0177] Finally, the three sets of features processed by Triplet Attention are concatenated together, and the interaction between features is achieved through ordinary convolution with a kernel size of 1, so as to further improve the expressiveness of features at different scales, and finally the processed feature map is output. In this way, the MTM module effectively realizes multi-scale feature fusion and provides strong feature support for skin disease segmentation.

[0178] In some embodiments, the preliminary output feature map is input into a decoder, and an add operation is performed on the intermediate output result of the decoder and the multiple multi-scale fusion feature maps, and the final output result of the decoder is a skin lesion segmentation result, including:

[0179] Inputting the latest preliminary output feature map into a decoder, and multiplying the latest preliminary output feature map by a tensor p that completes a depthwise separable convolution operation and a bilinear interpolation operation, to obtain a first decoded preliminary output feature map;

[0180] Performing an add operation on a fifth multi-scale fused feature map obtained by performing multi-scale feature fusion according to the second sampling preliminary output feature map and the first decoding preliminary output feature map to obtain a first decoding output result;

[0181] Multiplying the first decoding output result by the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation to obtain a second decoding preliminary output feature map;

[0182] Performing a bilinear interpolation operation on the second decoded preliminary output feature map to obtain a second decoded preliminary output feature map after the bilinear interpolation operation;

[0183] Performing an add operation on a fourth multi-scale fused feature map obtained by performing multi-scale feature fusion according to the first sampling preliminary output feature map and the second decoded preliminary output feature map after the bilinear interpolation operation to obtain a second decoding output result;

[0184] Multiplying the second decoding output result by the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation to obtain a third decoding preliminary output feature map;

[0185] Performing a bilinear interpolation operation on the third decoded preliminary output feature map to obtain a third decoded preliminary output feature map after the bilinear interpolation operation;

[0186] Performing an add operation on a third multiscale fused feature map obtained by performing multiscale feature fusion according to the second sampling wavelet feature map and the third decoded preliminary output feature map after the bilinear interpolation operation to obtain a third decoding output result;

[0187] Using wavelet convolution to perform feature extraction on the third decoding output result to obtain a fourth decoding preliminary output feature map;

[0188] Performing a bilinear interpolation operation on the fourth decoded preliminary output feature map to obtain a fourth decoded preliminary output feature map after the bilinear interpolation operation;

[0189] Performing an add operation on a second multi-scale fused feature map obtained by performing multi-scale feature fusion according to the first sampled wavelet feature map and the fourth decoded preliminary output feature map after the bilinear interpolation operation to obtain a fourth decoding output result;

[0190] Performing feature extraction on the fourth decoding output result by using wavelet convolution to obtain a fifth decoding preliminary output feature map;

[0191] Performing a bilinear interpolation operation on the fifth decoding preliminary output feature map to obtain a fifth decoding preliminary output feature map after the bilinear interpolation operation;

[0192] Performing an add operation on a first multi-scale fused feature map obtained by performing multi-scale feature fusion according to the initial feature map and the fifth decoded preliminary output feature map after the bilinear interpolation operation to obtain a fifth decoding output result;

[0193] Performing feature extraction on the fifth decoding output result by using two-dimensional convolution to obtain a sixth decoding preliminary output feature map;

[0194] A bilinear interpolation operation is performed on the sixth decoded preliminary output feature map to obtain a sixth decoded preliminary output feature map after the bilinear interpolation operation, and the sixth decoded preliminary output feature map after the bilinear interpolation operation is used as a skin lesion segmentation result.

[0195] It should be noted that in the decoder stage, the network structure presents a symmetrical design with the encoder, forming a harmonious echo. The main task of the decoder is to restore the details and spatial information of the image, which is achieved through bilinear interpolation. In this process, we use the feature map output by each order MTM and the output after the decoder bilinear interpolation operation to perform an add operation, and then obtain the mask through 1×1 convolution for supervising the training of the network, that is:

[0196] Mask=Conv1×1(BilinearInterpolation(high_level_features,target_size

[0197] =MTM_output.size())+MTM_output)

[0198] Among them, BilinearInterpolation represents the bilinear interpolation operation, which is a common upsampling method used to enlarge the size of the feature map to the target size. Here, it adjusts the size of the high-level feature map high_level_features to the same size as the MTM module output MTM_output.

[0199] Combining the above content, such as Figure 5 As shown, it is a complete convolutional neural network schematic diagram disclosed in one embodiment of the present application, which includes a skin lesion segmentation method part and a supervised network training part.

[0200] It should be noted that, in the convolutional neural network training process of the present invention:

[0201] (1) The public dataset ISIC2018 was used. All experiments were performed on a single NVIDIA RTX 3090 GPU. Before the experiment, the present invention standardized the image data and uniformly adjusted it to a size of 256×256 pixels. In order to enhance the generalization ability of the model, we adopted the following data augmentation strategies: horizontal flipping, vertical flipping, and random rotation. During the network model training process, we selected AdamW as the optimizer and set the initial learning rate to 0.001. The learning rate scheduling adopted the CosineAnnealingLR strategy, using CosineAnnealingLR as the scheduler, with a maximum number of iterations of 50 and a minimum learning rate of 1e-5. A total of 300 epochs were trained, and the batch-size was set to 8.

[0202] (2) Since the MTM module in the present invention requires mask information of different scales, in order to generate more accurate mask information, we use deep supervision to calculate the loss functions of different stages. Our loss function can be expressed as:

[0203] α i =Bce(y,y^)+Dice(y,y^)

[0204]

[0205] Where Bce(y,y^) and Dice(y,y^) represent binary cross entropy and dice loss. i is the loss function at different stages, β i is the weight of different stages. In the present invention, we set β by default between i=0 and i=5. i It is 1, 0.5, 0.4, 0.3, 0.2, 0.1.

[0206] (3) After completing the model training, save the trained model, load the trained model parameters, input the test data, and obtain the data segmentation results end-to-end. Figure 6 , which is a diagram showing the effect of verifying the skin lesion segmentation method in a data set according to an embodiment of the present invention.

[0207] In order to further demonstrate the effect of the skin lesion segmentation method of the present application, the present application collected a test data set for effect testing. Specifically, the evaluation indicators of the present invention are parameter quantity (Params), mean intersection over union (mIOU), and Dice similarity coefficient (DSC). On the test data set, the performance of the entire algorithm was evaluated, and the method of the present invention obtained competitive results, and it should be noted that compared with the mainstream network environment, the method of the present application also has higher accuracy and performance in an environment with lower parameter quantity.

[0208] The comparison between the experimental results of the present invention and those of the mainstream algorithms is shown in Table 1:

[0209]

[0210] Table 1

[0211] It can be seen from Table 1 that the skin lesion segmentation method of the present application (ours column in the table) has a lower parameter amount (Params), a higher mean intersection-over-union ratio (mIOU), and a higher Dice similarity coefficient (DSC) compared to other methods.

[0212] Therefore, it can be explained that the present invention is designed based on the EGE-UNet network, and the network performance designed is better for the existing high-precision network parameters in the skin disease segmentation task, such as large number of parameters, low precision of lightweight network, significant size difference of lesion area and complex texture characteristics. The present invention designs a wavelet convolution module, a linear attention module based on partial convolution and depth-separable convolution, and a multi-scale feature fusion module (the third innovation). After experimental verification, the network has high accuracy and performance while having a lower number of parameters.

[0213] Aiming at the needs of skin disease image segmentation, the present invention designs a new convolutional neural network structure, which first uses two wavelet convolutions to capture the multi-scale features of the image, and then further refines the feature representation through two attention mechanisms combining partial convolution and deep separable convolution. In our network structure, first through two wavelet convolution operations, the network can effectively extract multi-scale texture information in skin disease images, which is crucial for identifying lesion areas of different sizes and shapes. Wavelet convolution provides rich feature information for subsequent processing through its multi-resolution analysis capability. Then, the network introduces two attention mechanism modules based on partial convolution and deep separable convolution. This module uses partial convolution to maintain the sparsity of features, and combines deep separable convolution to efficiently extract features, so that the network can focus more on the significant features of the lesion area, while ignoring unimportant background information. The present invention not only enhances the network's perception of multi-scale texture features, maintains the efficient extraction of detail features, but also significantly reduces the number of network parameters. This strategy enables the network module to have better feature processing capabilities when processing skin disease images, thereby improving the segmentation accuracy and overall performance of the lesion area.

[0214] In view of the large differences in texture size and complex feature relationships in lesion areas in skin disease images, the present invention proposes a new linear attention module, which is implemented by combining partial convolution and depthwise separable convolution. The design of this attention module aims to simultaneously extract local features of the lesion area and enhance attention to key features. In the attention module, partial convolution is responsible for refining the edge information of the lesion area and focusing on important boundary features by adjusting the dynamic mask. At the same time, the depthwise separable convolution is used to efficiently extract texture and vascular details in the image while reducing the amount of calculation. The fusion of these two convolution operations not only improves the module's ability to process lesion features, but also forms an attention mechanism through their joint action, which can adaptively adjust feature weights so that the module can focus more on the features of the lesion area.

[0215] The present invention proposes an innovative multi-scale feature fusion module (MTM) for the multi-scale texture features and complex feature relationships of the lesion area in the segmentation of skin disease images. The module effectively solves the problems of large texture size differences and complex feature relationships in the lesion area by integrating deep separable convolution, bilinear interpolation and linear attention mechanism. The design concept of the MTM module is not only to extract local details of the lesion area, but also to strengthen the focus on key features. In the MTM module of the present invention, we use deep separable convolution to refine the edge information of the lesion area, and adjust the feature size by bilinear interpolation to achieve the capture of textures of different scales. The linear attention module focuses on the edge information of the lesion area through partial convolution operations, and uses dynamic masking technology to strengthen key boundary features in a targeted manner. At the same time, deep separable convolution is used to efficiently extract texture details in the image, while reducing computational complexity, ensuring the efficiency of the module. The fusion of these two convolution techniques enables the MTM module to extract rich feature information of dermatological images at different scales, and adaptively adjusts feature weights through the linear attention mechanism, so that the network is more focused on the features of the lesion area. This design not only enhances the network's ability to process lesion features, but also forms an effective attention mechanism through joint action, thereby improving the accuracy and efficiency of skin disease image segmentation.

[0216] In summary, the embodiment of the present application provides a skin lesion segmentation method, including: inputting a target skin lesion image into a convolutional neural network to extract image features to obtain an initial feature map; using an encoder that introduces a wavelet convolution layer to extract features from the initial feature map to obtain a wavelet feature map; multiplying the wavelet feature map with a tensor p that completes a depth-separable convolution operation and a bilinear interpolation operation to obtain a processed wavelet feature map, dividing the processed wavelet feature map into four groups of feature maps, performing HPA operations on the first three groups of feature maps along the height-width axis, the channel-height axis, and the channel-width axis, respectively, The fourth group of feature maps is subjected to partial convolution operation, and the processing results of the four groups of feature maps are spliced ​​along the channel dimension, and then multiplied again with the tensor p that has completed the depth-separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map; wherein the convolution kernel size is 3; the wavelet feature map and the preliminary output feature map are subjected to multi-scale feature fusion respectively to obtain multiple multi-scale fusion feature maps; the preliminary output feature map is input into the decoder, and the intermediate output result of the decoder is added to the multiple multi-scale fusion feature maps, and the final output result of the decoder is the skin lesion segmentation result. This application not only improves the accuracy of feature extraction, but also reduces the number of model parameters, improves computational efficiency, and enhances the module's adaptability to data diversity by grouping feature processing, introducing partial convolution and multi-scale feature fusion.

[0217] Embodiment 2

[0218] Based on the skin lesion segmentation method disclosed in the above embodiment of the present invention, Figure 7 Specifically disclosed is a skin lesion segmentation system using the skin lesion segmentation method.

[0219] like Figure 7 As shown, an embodiment of the present invention discloses a skin lesion segmentation system, the system comprising:

[0220] The first extraction module 701 is used to input the target skin lesion image into the convolutional neural network to extract image features and obtain an initial feature map;

[0221] A second extraction module 702 is used to extract features from the initial feature map using an encoder that introduces a wavelet convolution layer to obtain a wavelet feature map;

[0222] The processing module 703 is used to obtain a processed wavelet feature map by multiplying the wavelet feature map with the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation, and divide the processed wavelet feature map into four groups of feature maps, perform HPA operations on the first three groups of feature maps along the height-width axis, the channel-height axis, and the channel-width axis, respectively, perform a partial convolution operation on the fourth group of feature maps, and splice the processing results of the four groups of feature maps along the channel dimension, and multiply them again with the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map; wherein the convolution kernel size is 3;

[0223] A fusion module 704 is used to perform multi-scale feature fusion on the wavelet feature map and the preliminary output feature map to obtain multiple multi-scale fused feature maps;

[0224] The output module 705 is used to input the preliminary output feature map into a decoder, and perform an add operation on the intermediate output result of the decoder and the multiple multi-scale fusion feature maps. The final output result of the decoder is a skin lesion segmentation result.

[0225] In some embodiments, the first extraction module 701 is used to input the target skin lesion image into a convolutional neural network, extract image features of the target skin lesion image through a convolution layer with a convolution kernel size of 3x3, and obtain an initial feature map.

[0226] The specific working processes of the first extraction module 701, the second extraction module 702, the processing module 703, the fusion module 704 and the output module 705 in the skin lesion segmentation system disclosed in the above embodiment of the present invention can be referred to the corresponding content in the skin lesion segmentation method disclosed in the above embodiment of the present invention, and will not be repeated here.

[0227] In summary, the embodiment of the present application provides a skin lesion segmentation system, including: inputting a target skin lesion image into a convolutional neural network to extract image features to obtain an initial feature map; using an encoder that introduces a wavelet convolution layer to extract features from the initial feature map to obtain a wavelet feature map; multiplying the wavelet feature map with a tensor p that completes a depth-separable convolution operation and a bilinear interpolation operation to obtain a processed wavelet feature map, dividing the processed wavelet feature map into four groups of feature maps, performing HPA operations on the first three groups of feature maps along the height-width axis, the channel-height axis, and the channel-width axis, respectively, The fourth group of feature maps is subjected to partial convolution operation, and the processing results of the four groups of feature maps are spliced ​​along the channel dimension, and then multiplied again with the tensor p that has completed the depth-separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map; wherein the convolution kernel size is 3; the wavelet feature map and the preliminary output feature map are subjected to multi-scale feature fusion respectively to obtain multiple multi-scale fusion feature maps; the preliminary output feature map is input into the decoder, and the intermediate output result of the decoder is added to the multiple multi-scale fusion feature maps, and the final output result of the decoder is the skin lesion segmentation result. This application not only improves the accuracy of feature extraction, but also reduces the number of model parameters, improves computational efficiency, and enhances the module's adaptability to data diversity by grouping feature processing, introducing partial convolution and multi-scale feature fusion.

[0228] In the several embodiments provided in the present application, it should be understood that the disclosed method can also be implemented in other ways. The method embodiments described above are merely illustrative.

[0229] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0230] Although the implementation methods disclosed in this application are as above, the above contents are only the implementation methods adopted for facilitating the understanding of this application, and are not intended to limit this application. Any technician in the technical field to which this application belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in this application, but the scope of patent protection of this application shall still be based on the scope defined in the attached claims.

Claims

1. A skin lesion segmentation method, characterized in that: The method comprises: Input the target skin lesion image into the convolutional neural network to extract image features and obtain an initial feature map; Using an encoder that introduces a wavelet convolution layer to extract features from the initial feature map to obtain a wavelet feature map; According to the multiplication of the wavelet feature map and the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation, a processed wavelet feature map is obtained, the processed wavelet feature map is divided into four groups of feature maps, the first three groups of feature maps are respectively subjected to HPA operations along the height-width axis, the channel-height axis and the channel-width axis, the fourth group of feature maps is subjected to partial convolution operations, and the processing results of the four groups of feature maps are spliced ​​along the channel dimension, and then multiplied again with the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map; wherein the convolution kernel size is 3; Performing multi-scale feature fusion on the wavelet feature map and the preliminary output feature map respectively to obtain a plurality of multi-scale fused feature maps; The preliminary output feature map is input into a decoder, and an add operation is performed on the intermediate output result of the decoder and the multiple multi-scale fusion feature maps, and the final output result of the decoder is a skin lesion segmentation result.

2. The method according to claim 1, characterized in that The target skin lesion image is input into a convolutional neural network to extract image features to obtain an initial feature map, including: The target skin lesion image is input into the convolutional neural network, and the image features of the target skin lesion image are extracted through a convolution layer with a convolution kernel size of 3x3 to obtain an initial feature map.

3. The method according to claim 1, characterized in that The multiplication of the wavelet feature map and the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain the processed wavelet feature map includes: Through the tensor p formula: p=DepthwiseConv[BilinearInterpolation(p,size=Y.size())] X=p*Y Among them, Y represents the input wavelet feature map, X represents the processed wavelet feature map, DepthwiseConv represents the depthwise separable convolution operation, and BilinearInterpolation represents the bilinear interpolation operation.

4. The method according to claim 1, characterized in that: Also includes: Performing a maximum downsampling operation on the wavelet feature map to obtain a first sampled wavelet feature map; The first sampled wavelet feature map is used as an initial feature map, and an encoder introduced with a wavelet convolution layer is repeatedly used to extract features from the initial feature map to obtain a wavelet feature map; The wavelet feature map obtained by repeating the steps is subjected to a maximum downsampling operation again to obtain a second sampled wavelet feature map, and the second sampled wavelet feature map is used as the latest wavelet feature map.

5. The method according to claim 4, characterized in that Also includes: Performing a maximum downsampling operation on the preliminary output feature map to obtain a first sampled preliminary output feature map; The first sampled preliminary output feature map is used as a wavelet feature map, and the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation is multiplied by the wavelet feature map to obtain a preliminary output feature map. The multi-scale feature map obtained by the repeated steps is subjected to the maximum downsampling operation again to obtain the second sampling preliminary output feature map; The second sampled preliminary output feature map is used as a wavelet feature map, and the step of multiplying the wavelet feature map by the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map is repeated again, and the preliminary output feature map obtained by repeating the steps again is used as the latest preliminary output feature map.

6. The method according to claim 5, characterized in that The step of performing multi-scale feature fusion on the wavelet feature map and the preliminary output feature map to obtain a plurality of multi-scale fused feature maps comprises: Using a depthwise separable convolution operation and a bilinear interpolation operation, adjusting the second sampled preliminary output feature map to the same spatial resolution as the latest preliminary output feature map; Dividing the adjusted second sample preliminary output feature map and the latest preliminary output feature map into a first feature group, a second feature group and a third feature group along the channel dimension respectively; Fusing the adjusted first feature group of the second sampled preliminary output feature map with the first feature group of the latest preliminary output feature map, and connecting the fused features with the mask information to obtain a first fused feature; Performing feature fusion on the second feature group of the adjusted second sampled preliminary output feature map and the second feature group of the latest preliminary output feature map, and connecting the fused features with the mask information to obtain a second fused feature; Performing feature fusion on the third feature group of the adjusted second sample preliminary output feature map and the third feature group of the latest preliminary output feature map, and connecting the fused features with the mask information to obtain a third fused feature; Performing a dilated convolution operation and a pooling operation on the first fused feature according to a first dilation rate, and then performing a TripletAttention process to obtain a processed first fused feature; Performing a dilated convolution operation and a pooling operation on the second fused feature according to a second dilation rate, and then performing a TripletAttention process to obtain a processed second fused feature; Performing a dilated convolution operation and a pooling operation on the third fused feature according to a third dilation rate, and then performing a TripletAttention process to obtain a processed third fused feature; splicing the processed first fusion feature, the processed second fusion feature, and the processed third fusion feature, and processing them through ordinary convolution with a convolution kernel size of 1 to obtain a fifth multi-scale fusion feature map; Using a depthwise separable convolution operation and a bilinear interpolation operation, the first sampling preliminary output feature map is adjusted to the same spatial resolution as the fifth multi-scale fusion feature map; Repeat the steps starting from dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, respectively, on the adjusted first sampled preliminary output feature map and the fifth multi-scale fused feature map to obtain a fourth multi-scale fused feature map; Using a depthwise separable convolution operation and a bilinear interpolation operation, adjusting the second sampled wavelet feature map to the same spatial resolution as the fourth multi-scale fused feature map; Repeat the steps starting from dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into a first feature group, a second feature group and a third feature group along the channel dimension, with the adjusted second sampled wavelet feature map and the fourth multi-scale fusion feature map, to obtain a third multi-scale fusion feature map; Using a depthwise separable convolution operation and a bilinear interpolation operation, adjusting the first sampled wavelet feature map to the same spatial resolution as the third multi-scale fused feature map; Repeat the steps starting from dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into the first feature group, the second feature group and the third feature group along the channel dimension, to obtain a second multi-scale fusion feature map; Using a depthwise separable convolution operation and a bilinear interpolation operation, the initial feature map is adjusted to the same spatial resolution as the second multi-scale fused feature map; The adjusted initial feature map and the second multi-scale fused feature map are repeatedly executed starting from the step of dividing the adjusted second sampled preliminary output feature map and the latest preliminary output feature map into a first feature group, a second feature group and a third feature group along the channel dimension, respectively, to obtain a first multi-scale fused feature map.

7. The method according to claim 6, characterized in that The preliminary output feature map is input into a decoder, and an add operation is performed on the intermediate output result of the decoder and the multiple multi-scale fusion feature maps, wherein the final output result of the decoder is a skin lesion segmentation result, including: Inputting the latest preliminary output feature map into a decoder, and multiplying the latest preliminary output feature map by a tensor p that completes a depthwise separable convolution operation and a bilinear interpolation operation, to obtain a first decoded preliminary output feature map; Performing an add operation on a fifth multi-scale fused feature map obtained by performing multi-scale feature fusion according to the second sampling preliminary output feature map and the first decoding preliminary output feature map to obtain a first decoding output result; Multiplying the first decoding output result by the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation to obtain a second decoding preliminary output feature map; Performing a bilinear interpolation operation on the second decoded preliminary output feature map to obtain a second decoded preliminary output feature map after the bilinear interpolation operation; Performing an add operation on a fourth multi-scale fused feature map obtained by performing multi-scale feature fusion according to the first sampling preliminary output feature map and the second decoded preliminary output feature map after the bilinear interpolation operation to obtain a second decoding output result; Multiplying the second decoding output result by the tensor p that completes the depthwise separable convolution operation and the bilinear interpolation operation to obtain a third decoding preliminary output feature map; Performing a bilinear interpolation operation on the third decoded preliminary output feature map to obtain a third decoded preliminary output feature map after the bilinear interpolation operation; Performing an add operation on a third multiscale fused feature map obtained by performing multiscale feature fusion according to the second sampling wavelet feature map and the third decoded preliminary output feature map after the bilinear interpolation operation to obtain a third decoding output result; Using wavelet convolution to perform feature extraction on the third decoding output result to obtain a fourth decoding preliminary output feature map; Performing a bilinear interpolation operation on the fourth decoded preliminary output feature map to obtain a fourth decoded preliminary output feature map after the bilinear interpolation operation; Performing an add operation on a second multi-scale fused feature map obtained by performing multi-scale feature fusion according to the first sampled wavelet feature map and the fourth decoded preliminary output feature map after the bilinear interpolation operation to obtain a fourth decoding output result; Performing feature extraction on the fourth decoding output result by using wavelet convolution to obtain a fifth decoding preliminary output feature map; Performing a bilinear interpolation operation on the fifth decoding preliminary output feature map to obtain a fifth decoding preliminary output feature map after the bilinear interpolation operation; Performing an add operation on a first multi-scale fused feature map obtained by performing multi-scale feature fusion according to the initial feature map and the fifth decoded preliminary output feature map after the bilinear interpolation operation to obtain a fifth decoding output result; Performing feature extraction on the fifth decoding output result by using two-dimensional convolution to obtain a sixth decoding preliminary output feature map; A bilinear interpolation operation is performed on the sixth decoded preliminary output feature map to obtain a sixth decoded preliminary output feature map after the bilinear interpolation operation, and the sixth decoded preliminary output feature map after the bilinear interpolation operation is used as a skin lesion segmentation result.

8. The method according to claim 1, characterized in that Also includes: A maximum downsampling operation is performed on the initial feature map, and the sampling result is used as the latest initial feature map.

9. A skin lesion segmentation system, characterized in that: The system comprises: The first extraction module is used to input the target skin lesion image into the convolutional neural network to extract image features and obtain an initial feature map; A second extraction module is used to extract features from the initial feature map using an encoder that introduces a wavelet convolution layer to obtain a wavelet feature map; A processing module, used for multiplying the wavelet feature map with the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain a processed wavelet feature map, dividing the processed wavelet feature map into four groups of feature maps, performing HPA operations on the first three groups of feature maps along the height-width axis, the channel-height axis and the channel-width axis respectively, performing a partial convolution operation on the fourth group of feature maps, and splicing the processing results of the four groups of feature maps along the channel dimension, and multiplying them again with the tensor p that completes the depth-separable convolution operation and the bilinear interpolation operation to obtain a preliminary output feature map; wherein the convolution kernel size is 3; A fusion module, used for performing multi-scale feature fusion on the wavelet feature map and the preliminary output feature map to obtain multiple multi-scale fused feature maps; The output module is used to input the preliminary output feature map into a decoder, and perform an add operation on the intermediate output result of the decoder and the multiple multi-scale fusion feature maps, and the final output result of the decoder is the skin lesion segmentation result.

10. The system according to claim 9, characterized in that The first extraction module is used to input the target skin lesion image into a convolutional neural network, extract image features of the target skin lesion image through a convolution layer with a convolution kernel size of 3x3, and obtain an initial feature map.

Images