Lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization

Abstract

The invention provides a lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization. The lightweight skin lesion segmentation method comprises the steps of obtaining a skin lesion image, performing enhancement preprocessing, and dividing the skin lesion image into a training set and a test set; constructing a lightweight skin segmentation network model fusing asymmetric convolution and a dynamic boundary optimization mechanism; and performing feature extraction, fusion and prediction processing on the skin lesion image by using the trained lightweight skin segmentation network model, and finally obtaining a global segmentation prediction result of the skin lesion image. According to the method, the complexity of the model is reduced through asymmetric convolution, the boundary truth value and the multi-level attention mechanism are dynamically generated in combination with the genetic algorithm, the perception ability of the model to the fuzzy boundary is effectively improved, and boundary prediction is effectively optimized; efficient and accurate segmentation on mobile and embedded equipment is realized, and meanwhile, the robustness of the model to irregular boundaries and noise interference is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization. Background Art

[0002] Skin cancer is one of the most common cancers worldwide. Although melanoma accounts for only 1% of skin cancer cases, it is the main cause of most skin cancer-related deaths. Early diagnosis is crucial to improving patient survival. Dermoscopy is a commonly used non-invasive diagnostic technique, but it relies on the doctor's experience and is highly subjective, making it less efficient.

[0003] In recent years, deep learning-based computer-aided diagnosis (CAD) systems have made significant progress in skin lesion segmentation, but they still face the following challenges:

[0004] 1) Lesion diversity: Skin lesions vary significantly in color, shape, and size;

[0005] 2) Interference factors: Interference factors such as blood vessels, hair, skin texture, light and water stains increase the difficulty of segmentation;

[0006] 3) Boundary fuzziness: The contrast between the lesion area and healthy skin is low and the boundary is blurred;

[0007] 4) Model complexity: Existing deep learning models have many parameters and large computational complexity, making them difficult to deploy on resource-constrained devices.

[0008] In recent years, deep learning methods have received widespread attention, and many convolutional neural networks (CNNs) based on encoder-decoder architectures have been used for skin lesion segmentation. However, due to the lack of lightweight design of the models, these models have many parameters, which poses a challenge to the computing power and memory of mobile and embedded devices. It is challenging to apply these methods to clinical devices, especially dermoscopic devices with limited memory and low computing resources. The existing classic lightweight image segmentation model ENet has also been applied to the skin lesion segmentation task. However, ENet lacks attention to important features, has a relatively single information flow, and has difficulty coping with the complexity and various interferences of skin lesion images. Its segmentation performance lags far behind the current state-of-the-art skin lesion segmentation models. Summary of the invention

[0009] In view of the shortcomings of the prior art, the present invention provides a lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization. The present invention reduces the complexity of the model through asymmetric convolution, combines the genetic algorithm to dynamically generate boundary truth values ​​and a multi-level attention mechanism, to achieve efficient and accurate segmentation on mobile and embedded devices, and at the same time enhances the robustness of the model to irregular boundaries and noise interference.

[0010] The technical solution of the present invention is: a lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization, comprising the following steps:

[0011] S1), obtaining skin lesion images and performing enhancement preprocessing before dividing them into training set and test set;

[0012] S2), constructing a lightweight skin segmentation network model integrating asymmetric convolution and dynamic boundary optimization mechanism; training the lightweight skin segmentation network model using the training set, and testing it using the test set after the training is completed;

[0013] S3) Using the trained lightweight skin segmentation network model, the skin lesion image is subjected to feature extraction, fusion and prediction processing, and finally a global segmentation prediction result of the skin lesion image is obtained.

[0014] Preferably, in step S2), the lightweight skin segmentation network model includes an encoder and a decoder, and the encoder and decoder include multiple hierarchical cascade processing stages; the features of the skin lesion image are extracted by the encoder, and the encoded features and the decoded features are fused by the decoder, and the image segmentation is achieved by combining regional prediction and boundary prediction.

[0015] Preferably, in step S2), the first to third stages of the encoder adopt a parallel asymmetric convolution module based on ULFAC-Net; the low-level features of the skin lesion image in different directions are extracted through the parallel asymmetric convolution module, and the feature resolution is reduced by downsampling; the fourth to sixth stages of the encoder adopt a group mixed attention module based on LB-UNet, and the group mixed attention module is used to extract higher-level semantic information features, enhance the long-distance dependency modeling capability of the features, improve the key area recognition capability in the skin lesion image, and further reduce the resolution through the maximum pooling layer to improve the feature extraction efficiency.

[0016] Preferably, in step S2), the parallel asymmetric convolution module includes multiple convolution layers of sizes 1×3 and 3×1, and multiple batch normalization layers; the skin lesion image is input into the parallel asymmetric convolution module, wherein the 3×1 convolution layer extracts the horizontal edges, followed by the 1×3 convolution longitudinal aggregation, and then dynamically fused through the channel attention based on global average pooling and activation function to optimize the multi-scale lesion boundary representation.

[0017] Preferably, in step S2), the group mixed attention module divides the input image features into different groups, performs Hadamard product attention calculation in each group, and uses the group shuffling operation to capture effective information.

[0018] Preferably, in step S2), the decoder integrates multiple LB-UNet prediction information fusion modules from bottom to top, and the prediction information fusion module performs weighted fusion of the boundary prediction map and the regional features to ensure full utilization of the boundary and regional information in the decoding stage.

[0019] Preferably, in step S2), the prediction information fusion module based on LB-UNet focuses on the features related to the segmented area and the boundary, and the encoder features related to the segmented area and the boundary are superimposed on the retained decoder features with specific weights, thereby providing additional information contribution; the calculation formula of the prediction information fusion module is:

[0020]

[0021] Among them, E i represents the features of the i-th layer of the encoder, Indicates that each pixel in the image belongs to the region prediction feature of the target region, Indicates the boundary prediction feature that each pixel in the image belongs to the boundary of the target area, V i-1 The result of the decoder layer i-1 output; and is the weight parameter of the encoder.

[0022] Preferably, in step S2), the decoder performs skin lesion image region segmentation and boundary detection by adopting region prediction and boundary prediction. In order to enhance the accuracy of region and boundary prediction, a residual attention mechanism is introduced to take region prediction features as and boundary prediction features Weighted fusion is performed via attention weights, which are generated through additional convolutional layers and activation functions.

[0023] Preferably, in step S2), the decoder uses an asymmetric convolution module or a hybrid attention module to perform the encoder feature E using the region prediction module. i Processing to generate regional prediction features

[0024] The decoder uses an asymmetric convolution module or a hybrid attention module to the encoder feature E using a boundary prediction module i Processing to generate boundary prediction features Its calculation formula is expressed as:

[0025]

[0026] Among them, F i represents the features of the i-th layer of the decoder, represents the regional prediction feature, represents the regional prediction feature, D i represents the retained features, δ is the Sigmoid function, C 1×1 Represents a convolution function with a kernel size of 1.

[0027] Preferably, in step S2), the decoder gradually upsamples the segmentation prediction result by bilinear interpolation to restore it to the same size as the input image.

[0028] Preferably, in step S2), the decoder generates a boundary image of skin lesions using regional prediction features and boundary prediction features through a dynamic boundary optimization module to gradually optimize the parameters of the model; the real boundary image consists of two parts: a boundary line and key points, wherein the boundary line part is obtained by an edge detection algorithm; and the key points are obtained by a genetic algorithm.

[0029] Preferably, in step S2), the segmentation loss L is used in the training process of the lightweight skin segmentation network model using the training set. 分割 , Boundary loss and regional losses The details are as follows:

[0030]

[0031] Where, L BCE and L DICE denote binary cross entropy loss and Dice loss respectively, BI denotes bilinear interpolation, BG denotes a boundary generator based on a genetic algorithm, and λ i Representing regional characteristics at different stages The weight, μ i Represents boundary features at different stages The weight of ; y is the sample label, its value is 1 or 0; is the predicted probability of sample label y.

[0032] Preferably, in step S3), segmenting the skin lesion image based on the trained lightweight skin segmentation network model specifically includes the following steps:

[0033] S31), adjusting the dermatoscopic image to a fixed resolution and normalizing the pixel values;

[0034] S32), encoder feature extraction: extract features through asymmetric convolution modules, and downsample through maximum pooling at each feature extraction stage;

[0035] S33) Decoder prediction: restore the predicted image to the same size as the original image through bilinear interpolation and fuse the feature maps of each stage of the encoder; use 1×1 convolution to generate the final segmentation mask and output the lesion probability map through the Sigmoid function.

[0036] The beneficial effects of the present invention are:

[0037] 1. This invention reduces the complexity of the model through asymmetric convolution, combines the genetic algorithm to dynamically generate boundary truth values ​​and multi-level attention mechanism, realizes efficient and accurate segmentation on mobile and embedded devices, and enhances the robustness of the model to irregular boundaries and noise interference;

[0038] 2. The present invention optimizes the feature extraction efficiency and improves the ability to capture long-distance features, and uses genetic algorithm adaptive adjustment and Flip-Test to enhance boundary robustness, effectively improving the model's perception of fuzzy boundaries and effectively optimizing boundary prediction;

[0039] 3. The number of model parameters of the present invention is reduced to 2.32M, and the amount of calculation is reduced to 1.53GFLOPs, achieving a lightweight design; it is suitable for resource-constrained mobile devices and embedded devices, and can be integrated into clinical equipment such as dermatoscopes. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a structural framework diagram of the lightweight skin segmentation network model of the present invention;

[0041] Figure 2 It is a structural framework diagram of the parallel asymmetric convolution module of the present invention;

[0042] Figure 3 This is a structural framework diagram of the hybrid attention module of the present invention;

[0043] Figure 4 This is a structural framework diagram of the prediction information fusion module of the present invention;

[0044] Figure 5 Schematic diagram of segmentation results of different methods on the ISIC 2017 dataset in embodiments of the present invention. DETAILED DESCRIPTION

[0045] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings:

[0046] like Figure 1 As shown, this embodiment provides a lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization, comprising the following steps:

[0047] S1), obtaining skin lesion images and performing preprocessing;

[0048] In this embodiment, skin lesion images are obtained by using a dermatoscope, and the obtained skin lesion images are normalized to a fixed size, and the pixel values ​​are scaled to the interval [0,1]. At the same time, random flipping, random cropping, elastic change, etc. are used to enhance the skin lesion images. The enhanced skin lesion image dataset is divided into a training set and a test set in a ratio of 7:3.

[0049] S2), constructing a lightweight skin segmentation network model integrating asymmetric convolution and dynamic boundary optimization mechanism; using the training set to train the lightweight skin segmentation network model, and after the training is completed, using the test to test it; the structure diagram of the lightweight skin segmentation network model integrating asymmetric convolution and dynamic boundary optimization mechanism constructed in this embodiment can be seen in Figure 1 shown.

[0050] In this embodiment, the lightweight skin segmentation network model includes an encoder and a decoder, and the encoder and decoder include multiple hierarchical cascade processing stages; the features of the skin lesion image are extracted by the encoder, and the encoding features and the decoding features are fused by the decoder, and the image segmentation is achieved by combining regional prediction and boundary prediction.

[0051] Among them, the structure of the encoder is divided into six stages from top to bottom. The first to third stages adopt parallel asymmetric convolution modules based on ULFAC-Net. Three parallel asymmetric convolution modules are connected in series to expand the perception domain; low-level features of skin lesion images in different directions are extracted through the parallel asymmetric convolution modules, and the feature resolution is reduced by downsampling; the fourth to sixth stages of the encoder adopt a group mixed attention module based on LB-UNet, and higher-level semantic information features are extracted through the group mixed attention module, the long-distance dependency modeling capability of the features is enhanced, the key area recognition capability in the skin lesion images is improved, and the resolution is further reduced through the maximum pooling layer to improve the feature extraction efficiency.

[0052] like Figure 2 As shown, the parallel asymmetric convolution module includes multiple convolution layers of size 1×3 and 3×1, and multiple batch normalization layers; the skin lesion image is input into the parallel asymmetric convolution module, wherein the 3×1 convolution layer extracts the horizontal edge, followed by the 1×3 convolution for vertical aggregation, and then dynamically fused through the channel attention based on global average pooling and activation function to optimize the multi-scale lesion boundary representation.

[0053] like Figure 3 As shown, the group mixed attention module divides the input image features into different groups, performs Hadamard product attention calculation in each group, and uses the group shuffle operation to capture effective information. The specific steps are as follows:

[0054] First, the input features are divided into G groups, each of which has a size of d / G, where d is the feature dimension;

[0055] Then, the Hadamard product attention is calculated within each group, namely:

[0056]

[0057] Among them, (Q i ,K i ,V i ) are the query, key and value of the i-th group respectively; ° represents the Hadamard product;

[0058] Finally, the features are rearranged through the group shuffling operation to fully integrate the information between different groups.

[0059] In this embodiment, the decoder integrates multiple LB-UNet prediction information fusion modules from bottom to top. The prediction information fusion module performs weighted fusion of the boundary prediction map and the regional features to ensure full utilization of the boundary and regional information in the decoding stage.

[0060] like Figure 4 As shown in FIG. 1 , the prediction information fusion module based on LB-UNet focuses on the features related to the segmented area and the boundary. The encoder features related to the segmented area and the boundary are superimposed on the retained decoder features with specific weights, thereby providing additional information contribution. The calculation formula of the prediction information fusion module is:

[0061]

[0062] Among them, E i represents the features of the i-th layer of the encoder, Indicates that each pixel in the image belongs to the region prediction feature of the target region, Indicates the boundary prediction feature that each pixel in the image belongs to the boundary of the target area, V i-1 The result of the decoder layer i-1 output; and is the weight parameter of the encoder. In this embodiment, the weight is set is 0.5, 0.4, 0.3, 0.2, and 0.1, corresponding to i from 2 to 6 layers respectively; at the same time, set the weight are 0.3, 0.2, and 0.1, corresponding to i from 2 to 4 layers respectively.

[0063] like Figure 1 As shown in the figure, the decoder performs skin lesion image region segmentation and boundary detection by adopting region prediction and boundary prediction. In order to enhance the accuracy of region and boundary prediction, the residual attention mechanism is introduced to take the region prediction features into account. and boundary prediction features The weighted fusion is performed through attention weights, which are generated through additional convolutional layers and activation functions. The region prediction module of the decoder uses an asymmetric convolution module or a hybrid attention module to the encoder feature E i Processing to generate regional prediction features The boundary prediction module of the decoder uses an asymmetric convolution module or a hybrid attention module to the encoder feature E i Processing to generate boundary prediction features Its calculation formula is expressed as:

[0064]

[0065] Among them, F i represents the features of the i-th layer of the decoder, represents the regional prediction feature, represents the regional prediction feature, D i represents the retained features, δ is the Sigmoid function, C 1×1 Represents a convolution function with a kernel size of 1.

[0066] like Figure 1 As shown, the decoder gradually upsamples the segmentation prediction result by bilinear interpolation and restores it to the same size as the input image.

[0067] In addition, in this embodiment, the dynamic boundary optimization module of the decoder generates a real boundary image through the regional prediction features and the boundary prediction features; the real boundary image consists of two parts: the boundary line and the key points, wherein the boundary line part is obtained by the edge detection algorithm; the key points are obtained by the genetic algorithm, as follows:

[0068] S21) Evaluation key points

[0069] Let S = {(x 1 ,y 1 ),(x 2 ,y 2 ),…,(x n ,y n )} represents a set of m key points; the function f(·) is defined as connecting the points in the set S in sequence to form a boundary region;

[0070] The evaluation criteria and fitness function are based on the region f(S) formed by the key points and the true region R GT The intersection-over-union ratio (IOU) between them;

[0071] S22), using a genetic algorithm to search for a key point set with a maximum intersection-over-union ratio (IOU);

[0072] S221), randomly generate T 1 Group key points to form a size N P The initial population P 0 ;

[0073] S222), perform T 2 Genetic evolution iteration, updating the population;

[0074] S223), among all populations, select the key point set with the largest intersection-over-union ratio (IOU) as the final result.

[0075] In this embodiment, the segmentation loss L is used in the process of training the lightweight skin segmentation network model using the training set. 分割 , Boundary loss and regional losses The details are as follows:

[0076]

[0077] Where, L BCE and L DICE denote binary cross entropy loss and Dice loss respectively, BI denotes bilinear interpolation, BG denotes a boundary generator based on a genetic algorithm, and λ i Representing regional characteristics at different stages The weight, μ i Represents boundary features at different stages The weight of ; y is the sample label, its value is 1 or 0; is the predicted probability of sample label y. In this embodiment, set λ i is 0.5, 0.4, 0.3, 0.2, and 0.1, corresponding to layers i from 2 to 6 respectively; μ i are 0.3, 0.2, and 0.1, corresponding to layers i from 2 to 4, respectively.

[0078] After training, the trained lightweight skin segmentation network model is tested using the test set. During the test, the images of the test set are flipped in multiple directions and combined with multi-scale prediction results to enhance the robustness of the model to boundaries.

[0079] S3), using the trained lightweight skin segmentation network model to perform feature extraction, fusion and prediction processing on the skin lesion image, and finally obtain the global segmentation prediction result of the skin lesion image, which specifically includes the following steps:

[0080] S31), adjusting the dermatoscopic image to a fixed resolution, and normalizing the pixel values ​​to 256×256 pixels;

[0081] S32), encoder feature extraction: extract features through asymmetric convolution modules, and downsample each feature extraction stage through maximum pooling, with output resolutions of 128×128, 64×64, 32×32, and 16×16 respectively;

[0082] S33), decoder prediction: gradually upsample from 16×16 to 256×256 resolution through bilinear interpolation, and fuse the feature maps of each stage of the encoder; use 1×1 convolution to generate the final segmentation mask, and output the lesion probability map through the Sigmoid function;

[0083] S34) Relationship between prediction results and labels: Compare the final segmentation results with the labels, calculate indicators such as Dice coefficient and accuracy, and evaluate model performance.

[0084] In order to verify the segmentation performance of the method of this embodiment in skin lesion images, this embodiment uses the existing commonly used methods to compare with the method of this embodiment on the ISIC 2017 and ISIC 2018 datasets. The results are shown in Tables 1 and Figure 5 shown.

[0085] Table 1 Segmentation results of different methods on ISIC 2017 and ISIC 2018 datasets

[0086]

[0087]

[0088] From Table 1 and Figure 5 It can be seen that the segmentation result of the method in this embodiment is better than that of the existing common methods.

[0089] The above embodiments and descriptions are only for illustrating the principles and best embodiments of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, all of which fall within the scope of the present invention to be protected.

Claims

1. A lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization, characterized in that: The following steps are involved: S1), obtaining skin lesion images and performing enhancement preprocessing before dividing them into training set and test set; S2), construct a lightweight skin segmentation network model that integrates asymmetric convolution and dynamic boundary optimization mechanism; The lightweight skin segmentation network model is trained using the training set, and after the training is completed, it is tested using the test set; S3) Using the trained lightweight skin segmentation network model, the skin lesion image is subjected to feature extraction, fusion and prediction processing, and finally a global segmentation prediction result of the skin lesion image is obtained.

2. The lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization according to claim 1, characterized in that: The lightweight skin segmentation network model includes an encoder and a decoder, and the encoder and decoder include multiple hierarchical cascade processing stages; the features of the skin lesion image are extracted by the encoder, and the encoding features and the decoding features are fused by the decoder, and the image segmentation is achieved by combining regional prediction and boundary prediction.

3. The lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization according to claim 2, characterized in that: In step S2), the first to third stages of the encoder adopt a parallel asymmetric convolution module based on ULFAC-Net; the low-level features of the skin lesion image in different directions are extracted through the parallel asymmetric convolution module, and the feature resolution is reduced by downsampling; the fourth to sixth stages of the encoder adopt a group mixed attention module based on LB-UNet, extract higher-level semantic information features through the group mixed attention module, and further reduce the resolution through the maximum pooling layer to improve the feature extraction efficiency.

4. The lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization according to claim 3, characterized in that: In step S2), the parallel asymmetric convolution module includes multiple convolution layers of sizes 1×3 and 3×1, and multiple batch normalization layers; the skin lesion image is input into the parallel asymmetric convolution module, wherein the 3×1 convolution layer extracts the horizontal edges, followed by the 1×3 convolution for longitudinal aggregation, and then dynamically fused through the channel attention based on global average pooling and activation function.

5. The lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization according to claim 3, characterized in that: In step S2), the group mixed attention module divides the input image features into different groups, and performs Hadamard product attention calculation in each group, using the group shuffle operation to capture effective information.

6. The lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization according to claim 2, characterized in that: In step S2), the decoder integrates multiple LB-UNet prediction information fusion modules from bottom to top, and the prediction information fusion module performs weighted fusion of the boundary prediction map and the regional features.

7. The lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization according to claim 6, characterized in that: In step S2), the prediction information fusion module based on LB-UNet focuses on the features related to the segmented area and the boundary, and the encoder features related to the segmented area and the boundary are superimposed on the retained decoder features with specific weights; the calculation formula of the prediction information fusion module is: Among them, E i represents the features of the i-th layer of the encoder, Indicates that each pixel in the image belongs to the region prediction feature of the target region, Indicates the boundary prediction feature that each pixel in the image belongs to the boundary of the target area, V i-1 The result of the decoder layer i-1 output; and is the weight parameter of the encoder.

8. The lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization according to claim 7, characterized in that: In step S2), the decoder performs skin lesion image region segmentation and boundary detection by adopting region prediction and boundary prediction, and introduces the residual attention mechanism to integrate the region prediction features. and boundary prediction features Weighted fusion via attention weights generated via additional convolutional layers and activation functions; The decoder uses an asymmetric convolution module or a hybrid attention module to the encoder feature E using the region prediction module i Processing to generate regional prediction features The decoder uses an asymmetric convolution module or a hybrid attention module to the encoder feature E using a boundary prediction module i Processing to generate boundary prediction features Its calculation formula is expressed as: Among them, F i represents the features of the i-th layer of the decoder, represents the regional prediction feature, represents the regional prediction feature, D i represents the retained features, δ is the Sigmoid function, C 1×1 Represents a convolution function with a kernel size of 1.

9. The lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization according to claim 8, characterized in that: In step S2), the decoder gradually upsamples the segmentation prediction results through bilinear interpolation and restores them to the same size as the input image; and the decoder generates a boundary image of the skin lesion using regional prediction features and boundary prediction features through a dynamic boundary optimization module to gradually optimize the parameters of the model; the real boundary image consists of two parts: a boundary line and key points, wherein the boundary line part is obtained by an edge detection algorithm; and the key points are obtained by a genetic algorithm.

10. The lightweight skin lesion segmentation method based on asymmetric convolution and dynamic boundary optimization according to claim 9, characterized in that: In step S2), the segmentation loss L is used in the training process of the lightweight skin segmentation network model using the training set. 分割 , Boundary loss and regional losses The details are as follows: Where, L BCE and L DICE denote binary cross entropy loss and Dice loss respectively, BI denotes bilinear interpolation, BG denotes a boundary generator based on a genetic algorithm, and λ i Representing regional characteristics at different stages The weight, μ i Represents boundary features at different stages The weight of ; y is the sample label, its value is 1 or 0; is the predicted probability of sample label y.

Images