Skin lesion image segmentation method based on lightweight multi-scale UNet

Abstract

The present invention provides a method for skin lesion image segmentation based on a lightweight multi-scale Unified Network (UNet). The method comprises: obtaining and preprocessing a skin lesion image; establishing a lightweight multi-scale Unified Network (LMUNet) network structure: based on the original UNet model, replacing the original convolutional module in the UNet encoding path with a multi-scale inverted residual module, adding an asymmetric dilated spatial pyramid pooling module between the encoding and decoding paths, reducing the number of channels per layer, and modifying the original UNet skip connections to channel addition; training the LMUNet network using the preprocessed skin lesion image; and inputting the trained LMUNet network into the skin lesion image to be segmented to obtain a segmentation result. Experimental results demonstrate that the present invention addresses the high complexity, computational complexity, and parameter-intensive nature of the UNet-based skin lesion image model, improves the accuracy and speed of skin lesion image segmentation, and achieves rapid segmentation of skin lesion images.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and specifically provides a skin lesion image segmentation method based on lightweight multi-scale UNet. Background Art

[0002] Skin lesion image segmentation is an important part of skin diagnosis research. The traditional method is to produce high-resolution images of damaged skin under a dermatoscope, which are then diagnosed by professional doctors. However, due to the varying sizes and shapes of skin lesions, manual judgment is time-consuming and labor-intensive, and contains subjective elements, which increases the difficulty of diagnosis. Early computer-assisted medical image segmentation methods generally rely on edge detection, template matching technology, and traditional machine learning techniques. These methods have achieved good results to a certain extent, but they often require complex preprocessing of the original image, which requires experienced engineers to design feature extractors and select appropriate classifiers for classification. Therefore, this method has weak generalization ability and is difficult to implement complex multi-classification tasks.

[0003] With the advent of the big data era and tremendous advances in computer hardware, deep learning techniques, particularly convolutional neural networks, have achieved superior results compared to traditional methods in many tasks such as image classification and detection. In recent years, many researchers have focused on developing high-precision segmentation methods. Among them, UNet is a semantic segmentation network based on a fully convolutional network, suitable for medical image segmentation. This network modifies and expands the fully convolutional neural network structure, achieving accurate segmentation results when trained with small amounts of data, and has therefore attracted widespread attention. Since then, many neural networks have been developed based on UNet, such as UNet++, LinkNet, 3DUNet, and ResUNet. Although these improved models have improved segmentation results, they often have too many parameters and a high computational load, which is not conducive to achieving rapid image segmentation. Summary of the Invention

[0004] In order to solve the above technical defects in the prior art, the present invention proposes a skin lesion image segmentation method based on lightweight multi-scale UNet.

[0005] The technical solution for achieving the purpose of the present invention is: a skin lesion image segmentation method based on lightweight multi-scale UNet, comprising the following steps:

[0006] Step 1: Acquire skin lesion images and perform preprocessing;

[0007] Step 2: Establish a lightweight multi-scale UNet network structure, namely LMUNet: Based on the original UNet model, use a multi-scale inverted residual module to replace the original convolution module in the UNet encoding path, and add an asymmetric dilated spatial pyramid pooling module between the encoding path and the decoding path. At the same time, reduce the number of channels per layer and modify the original skip connection of UNet to channel addition;

[0008] Step 3: Use the pre-processed skin lesion images to train the LMUNet network;

[0009] Step 4: Input the skin lesion image to be segmented into the trained LMUNet network to obtain the segmentation result.

[0010] Preferably, step 1 of preprocessing the skin lesion image specifically includes:

[0011] Unified image size, data augmentation and dataset partitioning, among which, unified image size means unifying the size of the acquired skin lesion images by scaling or cropping; data augmentation adopts one or more of the following methods: rotation and flipping image geometric transformation methods and histogram equalization; dataset partitioning means dividing the images after data augmentation operation into training set, validation set and test set.

[0012] Preferably, the lightweight multi-scale UNet network structure, i.e., LMUNet, described in step 2 is established as follows:

[0013] The LMUNet network structure includes an encoding path, a decoding path, and a multi-scale information fusion module;

[0014] The encoding path consists of a multi-scale inverted residual module and a 2×2 maximum pooling module, which is responsible for extracting features from the input image and reducing the image size, thereby reducing the amount of redundant parameters. The decoding path consists of a deconvolution layer and a standard convolution layer, which is responsible for recovering feature map information. The multi-scale information fusion module is located between the encoding and decoding paths and is used to fuse features at different scales to enrich contextual information.

[0015] Preferably, the encoding path includes a multi-scale inverted residual module and a 2×2 maximum pooling module; wherein the multi-scale inverted residual module is responsible for extracting feature information of the image at multiple scales, and the 2×2 maximum pooling module can compress the image, extract the main features of the image and reduce the amount of redundant parameters of the network.

[0016] The multi-scale inverted residual module includes a first 3×3 depth convolution module, a second 3×3 depth convolution module, a 3×3 depth void convolution module and a 1×1 point convolution module. The first 3×3 depth convolution module includes a 3×3 depth convolution, a batch normalization layer and a ReLU6 activation function. The input information passes through the 3×3 depth convolution, the batch normalization layer and the ReLU6 activation function once, and the obtained result is channel-spliced ​​with the input information. The spliced ​​information is respectively input into the second 3×3 depth convolution module and the 3×3 depth void convolution module. The second 3×3 depth convolution module includes a 3×3 depth convolution, a batch normalization layer and a ReLU6 activation function; the 3×3 depth void convolution module includes a 3×3 depth void convolution with a void rate of 2, a batch normalization layer and a ReLU6 activation function.

[0017] The results obtained by the second 3×3 depth convolution module and the 3×3 depth hole convolution module are added and input into the 1×1 point convolution module. The 1×1 point convolution module includes a 3×3 depth convolution, a batch normalization layer and a ReLU6 activation function. After the 3×3 depth convolution, the batch normalization layer and the ReLU6 activation function, the obtained results are channel-added with the input information at the beginning to obtain feature information at different scales.

[0018] Preferably, the multi-scale information fusion module is composed of an asymmetric dilated spatial pyramid pooling module, including a 1×1 point convolution and three parallel branches, and the three parallel branches are:

[0019] Branch 1: consists of 3×1 depthwise asymmetric convolution and 1×3 depthwise asymmetric convolution;

[0020] Branch 2: consists of a 3×1 depthwise asymmetric dilated convolution with a dilation rate of 2 and a 1×3 depthwise asymmetric dilated convolution;

[0021] Branch 3: consists of a 3×1 depthwise asymmetric dilated convolution with a dilation rate of 3 and a 1×3 depthwise asymmetric dilated convolution;

[0022] The input information undergoes a 1×1 point convolution, a batch normalization layer, and a ReLU6 activation function to reduce the number of channels to half of the original number. After passing through three parallel branches, the obtained result is concatenated with the information of the input branch, and finally, the feature information is fused through a 1×1 point convolution to obtain the fused multi-scale feature information.

[0023] Preferably, in step 3, the LMUNet network is trained using the pre-processed skin lesion images, and the loss function used is the cross entropy function, which has the form:

[0024]

[0025] Where H(P,Q represents cross entropy, P(xi ) represents the true probability distribution, Q(x i ) represents the predicted probability distribution.

[0026] Compared with existing technologies, this invention offers significant advantages: Based on the UNet model, it improves upon it by introducing a multi-scale inverted residual module, an asymmetric atrous spatial pyramid pooling module, and reducing the number of channels. Experimental results demonstrate that the proposed LMUNet model, compared to other segmentation models, is superior in performance and lightweight, achieving superior segmentation results with minimal computational effort.

[0027] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description or be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like parts throughout the drawings.

[0029] Figure 1 It is the overall structure diagram of the LMUNet network in this invention.

[0030] Figure 2 2 is a structural diagram of the multi-scale inverted residual module in the present invention.

[0031] Figure 3 This is a module structure diagram of the multi-scale information fusion module in the present invention.

[0032] Figure 4 Schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0033] It is easy to understand that, based on the technical solution of the present invention, a person of ordinary skill in the art can imagine various embodiments of the present invention without changing the essential spirit of the present invention. Therefore, the following specific embodiments and drawings are merely illustrative of the technical solution of the present invention and should not be regarded as the whole of the present invention or as a limitation or limitation of the technical solution of the present invention. On the contrary, the purpose of providing these embodiments is to enable those skilled in the art to understand the present invention more thoroughly. The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the innovative concept of the present invention.

[0034] The present invention is conceived as follows: Figures 1 to 4As shown in FIG, a skin lesion image segmentation method based on a lightweight multi-scale UNet is described. The specific steps of the method are as follows:

[0035] Step 1: Acquire skin lesion images and perform preprocessing;

[0036] Step 2: Establish a lightweight multi-scale UNet network structure, namely LMUNet: Based on the original UNet model, use a multi-scale inverted residual module to replace the original convolution module in the UNet encoding path, and add an asymmetric dilated spatial pyramid pooling module between the encoding path and the decoding path. At the same time, reduce the number of channels per layer and modify the original skip connection of UNet to channel addition;

[0037] Step 3: Use the pre-processed skin lesion images to train the LMUNet network;

[0038] Step 4: Input the skin lesion image to be segmented into the trained LMUNet network to obtain the segmentation result.

[0039] In a further embodiment, the specific method for obtaining skin lesion images and performing preprocessing is: unifying image size, data enhancement and data set division, wherein unifying image size is to unify the size of the obtained skin lesion images by scaling or cropping; data enhancement adopts one or more geometric transformation methods such as rotating and flipping images and image enhancement methods such as histogram equalization; data set division divides the images after data enhancement operation into training set, validation set and test set.

[0040] In a further embodiment, Figure 1 As shown in Figure 3, the lightweight multi-scale UNet network structure, or LMUNet, consists of an encoding path, a decoding path, and a multi-scale information fusion module. The encoding path consists of a multi-scale inverted residual module and a 2×2 maximum pooling module, where the maximum pooling module performs four downsampling operations. The decoding path consists of a 3×3 standard convolution block and a 2×2 deconvolution layer, where the deconvolution layer performs four upsampling operations. The multi-scale information fusion module, located between the encoding and decoding paths, is used to fuse features at different scales and enrich contextual information.

[0041] Specifically, the multi-scale inverted residual module structure is as follows: Figure 2 As shown in the figure, the structure of the multi-scale inverted residual module consists of 3×3 depth convolution, 3×3 depth void convolution, 1×1 point convolution, batch normalization layer and ReLU6 activation function. The input information of this module first passes through 3×3 depth convolution, batch normalization layer and ReLU6 activation function, and the result is channel-concatenated with the input information, and then passes through two parallel branches:

[0042] Branch 1: consists of 3×3 depth convolution;

[0043] Branch 2: consists of a 3×3 depth-wise dilated convolution with a dilation rate of 2;

[0044] The results from the two branches are added together and then passed through a 3×3 depthwise convolution, a batch normalization layer, and a ReLU6 activation function. Finally, the result is channel-wise added to the initial input information. This module can be used to obtain feature information at different scales.

[0045] Furthermore, the multi-scale information fusion module structure is as follows Figure 3 As shown in the figure, the multi-scale information fusion module is composed of an asymmetric dilated spatial pyramid pooling module, which includes three parallel branches:

[0046] Branch 1: consists of 3×1 depthwise asymmetric convolution and 1×3 depthwise asymmetric convolution;

[0047] Branch 2: consists of a 3×1 depthwise asymmetric dilated convolution with a dilation rate of 2 and a 1×3 depthwise asymmetric dilated convolution;

[0048] Branch 3: consists of a 3×1 depthwise asymmetric dilated convolution with a dilation rate of 3 and a 1×3 depthwise asymmetric dilated convolution;

[0049] The input information first undergoes a 1×1 point convolution, a batch normalization layer, and a ReLU6 activation function to reduce the number of channels to half. It then passes through three parallel branches, where the results are concatenated with the input branch's information. Finally, a 1×1 point convolution is performed for feature fusion. This module can fuse feature information at multiple scales, enhancing model capabilities.

[0050] During training, the network model was implemented on the PyTorch platform, version 1.11.0. The hyperparameters used during training were as follows: the learning rate was set to 0.01, the number of training iterations was set to 100, and the number of samples per training was 4. The optimizer used stochastic gradient descent to update the network parameters to accelerate convergence. The loss function used was the cross entropy function, which has the form:

[0051]

[0052] Where H(P,Q represents cross entropy, P(x i ) represents the true probability distribution, Q(x i ) represents the predicted probability distribution.

[0053] Table 1 Comparison of segmentation results between the method of the present invention and some convolutional neural network methods

[0054]

[0055] It can be seen from the segmentation results in Table 1 that compared with other convolutional neural networks in Table 1, the present invention has superior performance and is very lightweight, reduces the number of parameters and calculations, and improves the accuracy and speed of skin lesion image segmentation.

[0056] The above description is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto.

[0057] Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention.

[0058] It should be understood that in order to simplify the present invention and help those skilled in the art understand the various aspects of the present invention, in the above description of the exemplary embodiments of the present invention, various features of the present invention are sometimes described in a single embodiment or described with reference to a single figure. However, the present invention should not be interpreted as if all the features included in the exemplary embodiments are essential technical features of the claims of this patent.

[0059] It should be understood that the modules, units, components, etc. included in the device of one embodiment of the present invention can be adaptively changed to be installed in a device different from the embodiment. The different modules, units, or components included in the device of the embodiment can be combined into a single module, unit, or component, or they can be divided into multiple sub-modules, sub-units, or sub-components.

Claims

1. A skin lesion image segmentation method based on lightweight multi-scale UNet, characterized by: The following steps are involved: Step 1: Acquire skin lesion images and perform preprocessing; Step 2: Establish a lightweight multi-scale UNet network structure, namely LMUNet: Based on the original UNet model, use a multi-scale inverted residual module to replace the original convolution module in the UNet encoding path, and add an asymmetric void spatial pyramid pooling module between the encoding path and the decoding path. At the same time, reduce the number of channels per layer and modify the original skip connection of UNet to channel addition. The specific structure is as follows: The LMUNet network structure includes an encoding path, a decoding path, and a multi-scale information fusion module; The encoding path consists of a multi-scale inverted residual module and a 2×2 maximum pooling module, which is responsible for extracting features from the input image and reducing the image size to reduce redundant parameters. The decoding path consists of a deconvolution layer and a standard convolution layer, which is responsible for recovering feature map information. The multi-scale information fusion module is located between the encoding and decoding paths and is used to fuse features at different scales to enrich contextual information. Step 3: Use the pre-processed skin lesion images to train the LMUNet network; Step 4: Input the skin lesion image to be segmented into the trained LMUNet network to obtain the segmentation result.

2. The skin lesion image segmentation method based on lightweight multi-scale UNet according to claim 1 is characterized in that: Step 1: Preprocessing the skin lesion image specifically includes: Unified image size, data augmentation and dataset partitioning, among which, unified image size means unifying the size of the acquired skin lesion images by scaling or cropping; data augmentation adopts one or more of the following methods: rotation and flipping image geometric transformation methods and histogram equalization; dataset partitioning means dividing the images after data augmentation operation into training set, validation set and test set.

3. The skin lesion image segmentation method based on lightweight multi-scale UNet according to claim 1, characterized in that: The encoding path includes a multi-scale inverted residual module and a 2×2 maximum pooling module; wherein the multi-scale inverted residual module is responsible for extracting feature information of images at multiple scales, while the 2×2 maximum pooling module can compress the image, extract the main features of the image and reduce the number of redundant network parameters; The multi-scale inverted residual module includes a first 3×3 depth convolution module, a second 3×3 depth convolution module, a 3×3 depth void convolution module and a 1×1 point convolution module. The first 3×3 depth convolution module includes a 3×3 depth convolution, a batch normalization layer and a ReLU6 activation function. The input information passes through the 3×3 depth convolution, the batch normalization layer and the ReLU6 activation function once, and the obtained result is channel-spliced ​​with the input information. The spliced ​​information is respectively input into the second 3×3 depth convolution module and the 3×3 depth void convolution module. The second 3×3 depth convolution module includes a 3×3 depth convolution, a batch normalization layer and a ReLU6 activation function; the 3×3 depth void convolution module includes a 3×3 depth void convolution with a void rate of 2, a batch normalization layer and a ReLU6 activation function. The results obtained by the second 3×3 depth convolution module and the 3×3 depth hole convolution module are added and input into the 1×1 point convolution module. The 1×1 point convolution module includes a 3×3 depth convolution, a batch normalization layer and a ReLU6 activation function. After the 3×3 depth convolution, the batch normalization layer and the ReLU6 activation function, the obtained results are channel-added with the input information at the beginning to obtain feature information at different scales.

4. The skin lesion image segmentation method based on lightweight multi-scale UNet according to claim 1, characterized in that: The multi-scale information fusion module consists of an asymmetric dilated spatial pyramid pooling module, including a 1×1 point convolution and three parallel branches. The three parallel branches are: Branch 1: consists of 3×1 depthwise asymmetric convolution and 1×3 depthwise asymmetric convolution; Branch 2: consists of a 3×1 depthwise asymmetric dilated convolution with a dilation rate of 2 and a 1×3 depthwise asymmetric dilated convolution; Branch 3: consists of a 3×1 depthwise asymmetric dilated convolution with a dilation rate of 3 and a 1×3 depthwise asymmetric dilated convolution; The input information undergoes 1×1 point convolution, batch normalization layer and ReLU6 activation function to reduce the number of channels to half of the original. After passing through three parallel branches, the obtained result is channel-concatenated with the information of the input branch, and finally, the feature information is fused through 1×1 point convolution to obtain the fused multi-scale feature information.

5. The skin lesion image segmentation method based on lightweight multi-scale UNet according to claim 1, characterized in that: Step 3 uses the preprocessed skin lesion images to train the LMUNet network. The loss function used is the cross entropy function, which is in the form of: Where H(P,Q) represents cross entropy, P(x i ) represents the true probability distribution, Q(x i ) represents the predicted probability distribution, x i is the variable, and n is the number of variables.

Images