Improved melanoma segmentation method

Abstract

The invention relates to an improved melanoma segmentation method, and belongs to the technical field of image segmentation, and the method comprises the following steps: S1, constructing a data set, carrying out the preprocessing of the data set, and dividing the data set into a training set and a test set; s2, constructing a melanoma segmentation model based on an improved DeepLabV3 + network, and replacing a backbone network with ResNet50; the void space pyramid pooling ASPP is improved into a dense void space pyramid pooling DenseASPP; convolution attention CBAM is added to the decoder part; a conventional convolution is replaced by a depth separable convolution in a decoder; s3, training and testing the melanoma segmentation model; and S4, segmenting the melanoma by using the trained melanoma segmentation model. According to the method, the calculation amount of the model is reduced, the convergence speed is improved, the training time is saved, and the segmentation accuracy is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of image segmentation and relates to an improved melanoma segmentation method. Background Art

[0002] Melanoma is a highly metastatic skin disease and one of the most common malignant tumors in the world. It is highly malignant, grows rapidly, and metastasizes early. Although it only accounts for 1% of skin cancer, most skin cancer deaths are caused by melanoma. Therefore, early diagnosis and treatment of skin diseases can effectively prevent and avoid the risk of skin diseases, greatly improving the survival rate of patients.

[0003] In the clinical diagnosis of melanoma, dermoscopic imaging technology is currently widely used. However, manual segmentation is time-consuming and labor-intensive, with poor repeatability. Even experienced doctors may miss or misclassify due to various factors. Therefore, it is of great significance to introduce a computer-aided diagnosis (CAD) system to achieve automatic segmentation of skin lesions to help doctors perform clinical evaluation and diagnosis.

[0004] So far, researchers have proposed many methods to segment skin lesions. Traditional methods can be divided into three categories: histogram thresholding, clustering and region merging, and active contour models.

[0005] With the emergence of convolutional neural networks, new progress has been made in the field of image segmentation. End-to-end pixel-level segmentation of skin lesions is achieved based on FCN, eliminating the preprocessing process in traditional algorithms. However, it does not make sufficient use of global context information, has poor feature extraction capabilities, and cannot fuse shallow and deep features.

[0006] In the classic symmetric network U-Net, the encoder captures contextual semantic information through a contraction path, and the decoder upsamples the coarse-grained deep features and fine-grained shallow feature maps to generate accurate segmentation maps. Following this technical route, many U-Net variants have emerged, such as U-Net++, Res-UNet, AttentionUNet, and MultiResUNet to improve segmentation performance. However, the above U-Net architecture is limited by the difference in the size of the receptive field, which makes the deep model unable to capture enough contextual information, resulting in unsatisfactory segmentation in complex areas such as boundaries. For complex scenes and images, a more complex network structure is required to achieve better segmentation results.

[0007] Since melanoma samples have (1) irregular shapes and unclear edges; (2) many bubbles, marks and other interferences around them; (3) the lesion areas vary in size, and the diversity of samples makes them prone to mis-segmentation. Therefore, a new method is urgently needed to achieve efficient and accurate segmentation of melanoma. Summary of the invention

[0008] In view of this, an object of the present invention is to provide an improved melanoma segmentation method.

[0009] In order to achieve the above object, the present invention provides the following technical solutions:

[0010] An improved melanoma segmentation method comprises the following steps:

[0011] S1: Construct a data set, preprocess the data set, and divide it into a training set and a test set;

[0012] S2: Build a melanoma segmentation model based on the improved DeepLabV3+ network;

[0013] S3: training and testing the melanoma segmentation model;

[0014] S4: Use the trained melanoma segmentation model to segment melanoma.

[0015] Furthermore, the preprocessing of the data set includes clipping and normalizing the data set.

[0016] Furthermore, the specific improvements of the improved DeepLabV3+ network include: based on the traditional DeepLabV3+ network, the backbone network is replaced with ResNet50, and the atrous spatial pyramid pooling ASPP is improved to dense atrous spatial pyramid pooling DenseASPP; convolutional attention CBAM is added to the decoder part; and depthwise separable convolution is used in the decoder instead of conventional convolution.

[0017] Furthermore, the construction of a melanoma segmentation model based on the improved DeepLabV3+ network includes: using an Adam optimizer and a DICE loss function, setting an initial learning rate and a batch size, and initializing the melanoma segmentation model using a weighted ResNet50 model already available on ImageNet.

[0018] The beneficial effects of the present invention are as follows: the semantic segmentation model of DeepLabV3+ improved by the present invention uses a more lightweight backbone network and a deep separable convolution to reduce the computational load of the model and reduce the computational burden compared to the baseline model DeepLabV3+. The parameter amount is reduced from the original 41.1M to 9.1M, a decrease of 78%. The loss function of the improved model of the present invention decreases the fastest compared to other segmentation models, and can converge better. The transfer learning adopted by the present invention can quickly initialize the network, which can greatly improve the training speed and save training time compared to the benchmark model. By optimizing the ASPP module and adding technologies such as CBAM, mIoU, DICE, Acc, Sen, and Spe are 0.8249, 0.9041, 0.9424, 0.8955, and 0.9383, respectively, which increase by 3.34, 3.64, 1.82, 2.84, and 0.58 percentage points, respectively. The model of the present invention effectively improves the segmentation accuracy. Compared with other segmentation models, the model of the present invention is less affected by hair, rulers and other markers. For samples with blurred boundary areas, the segmentation mask and label mask are also the most similar, and can produce intuitive and accurate segmentation results in different scenarios.

[0019] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0021] Figure 1 To improve the DeepLabV3+ network structure diagram;

[0022] Figure 2 This is the ResNet50 network structure diagram;

[0023] Figure 3 This is the BottleNeck structure diagram;

[0024] Figure 4 This is a schematic diagram of the improvement of the hollow space pyramid pooling ASPP;

[0025] Figure 5 This is the CBAM structure diagram;

[0026] Figure 6 (a) is a schematic diagram of depth-wise convolution, and (b) is a schematic diagram of point-wise convolution;

[0027] Figure 7 It is a line graph of the loss function of the segmentation model;

[0028] Figure 8 The segmentation result image. DETAILED DESCRIPTION

[0029] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0030] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0031] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0032] The present invention provides an improved melanoma segmentation method, such as Figure 1 As shown in the figure, an encoder-decoder structure is adopted. The encoder is responsible for feature extraction, and the decoder is responsible for restoring the spatial information of the image and classifying each pixel. The improved algorithm is mainly divided into the following parts:

[0033] ① Replace the backbone network: The Xception network has many branches and a deeper network structure. These characteristics require a large number of parameters during training and reasoning. Using it on a small-scale data set may cause overfitting problems and slow reasoning speed. This paper replaces the backbone network with ResNet50, such as Figure 2 As shown in Figure 2, ResNet50 introduces the concept of residual learning, which solves the gradient vanishing and gradient exploding problems in the deep neural network training process through direct connections across layers (i.e., skip connections). This direct connection allows information to propagate more easily, which helps train deeper neural network models.

[0034] The architecture of ResNet50 includes multiple BottleNecks. Each BottleNeck contains multiple convolutional layers. The BottleNeck structure is as follows: Figure 3 shown.

[0035] ② Dense Atrous Spatial Pyramid Pooling (DenseASPP): Atrous spatial pyramid pooling connects features from multiple parallel atrous convolutional layers with different dilation rates, while DenseASPP combines the advantages of parallelism and cascading to generate features of more scales in a larger range, such as Figure 4 This not only means that the feature pyramid has better scale diversity, but also means that more pixels are involved in the convolution than in the atrous spatial pyramid pooling ASPP.

[0036] ③Introduce attention mechanism: Add convolutional attention (Convolutional Block Attention Module, CBAM) to the decoder part, such as Figure 5 CBAM is an attention mechanism module used to enhance the performance of convolutional neural networks, helping the network focus on important features, thereby improving the network's perception of key features and further improving the network's performance.

[0037] ④ Depthwise separable convolution: To further improve the efficiency of the model, depthwise separable convolution is used in the decoder to replace conventional convolution. Depthwise separable convolution is a combination of depthwise convolution and pointwise convolution to extract features. Figure 6 As can be seen in (a), a convolution kernel in depth-wise convolution is responsible for one channel, and a channel is convolved by only one convolution kernel. Figure 6 As shown in (b), the feature maps of the deep convolution are connected together through 1*1 convolution kernels, and there are as many output feature maps as there are convolution kernels.

[0038] Experimental example:

[0039] 1. Data preprocessing:

[0040] The ISIC2018 dataset is used. This dataset contains 2594 training images, 100 validation images, and 1000 test images, all of which have corresponding ground truths. The ISIC2018 dataset is cropped so that the size of each image is 256×256 to speed up the convolution and solve the excessive memory limitations, and then normalized.

[0041] 2. Build a semantic segmentation model:

[0042] The Adam optimizer and DICE loss function are selected, the initial learning rate is set to 1e-4, and the batch size is set to 10. The present invention uses the existing ResNet50 model weights on ImageNet to initialize the network. Transfer learning avoids excessive adjustment of weights, and only needs to fine-tune the weights, so that the pre-trained weights can effectively initialize the network.

[0043] The preprocessed training set and validation set are input into the model. After 50 rounds of training, the parameters are adjusted continuously, the loss function is continuously optimized, and finally it tends to be flat, and an optimal segmentation model is obtained. The model file is loaded into the prediction code, and the black and white mask image results are predicted.

[0044] 3. Verify the model:

[0045] The predicted black and white mask image results are compared with the test set labels to obtain the evaluation results. This experiment uses mean intersection over Union (mIoU), Dice coefficient (DICE), accuracy (Acc), sensitivity (Sen), and specificity (Spe) as indicators to evaluate the segmentation effect of the algorithm.

[0046]

[0047] Among them, mIoU measures the average of the intersection over union (IoU) ratios of each class between the predicted results and the true labels; the DICE coefficient is used to evaluate the similarity between the model segmentation effect and the actual labels; Acc is used to measure the proportion of samples correctly classified by the model during the prediction process; Sen represents the recall rate of the skin lesion area; and Spe represents the accuracy of the segmentation of the non-lesion area.

[0048] Table 1

[0049]

[0050] Table 1 shows the results of the ablation experiment. It can be seen that the present invention designs 6 groups of ablation experiments to study the impact of different improvement schemes on the experimental results. Scheme 1 represents the original DeeplabV3+ network. The experimental results show that based on the original DeeplabV3+ model, after introducing the improved ResNet50 as the backbone network, the mIoU, DICE coefficient, and accuracy are all improved, and the number of parameters is greatly reduced, which verifies the feasibility of replacing the backbone network. On this basis, Scheme 3 introduces CBAM. With the parameter amount basically unchanged, the three indicators are slightly improved. Therefore, the CBAM module helps to improve the segmentation accuracy of the model. Scheme 4 replaces ASPP with DenseASPP. Compared with Scheme 2, the mIoU, DICE coefficient, and accuracy are increased by 1.51, 1.19, and 0.58 percentage points respectively. Scheme 5 replaces the ordinary convolution of the decoder with a depth-separable convolution, the number of parameters is reduced, and the three indicators are improved. Finally, with the addition of CBAM, the three evaluation indicators all reach the highest point in the ablation experiment, verifying the feasibility of the improvement. In summary, each improved step of the present invention has made a certain contribution to the experiment.

[0051] Table 2

[0052]

[0053] Table 2 shows the segmentation results of different network models. It can be seen that the improved DeepLabV3+ model is trained with several semantic segmentation models such as DeepLabV3+, UNet, DANet, TransUNet, etc. under the same environment and training parameters. The mIoU, DICE, Acc, Sen, and Spe of the improved network of the present invention are 0.8249, 0.9041, 0.9424, 0.8955, and 0.9383, respectively, which are almost the highest, and the number of parameters is the lowest.

[0054] from Figure 7 It can be seen that the loss function of the improved model of the present invention decreases fastest compared to other segmentation models and can converge better. At the same time, it is verified that the transfer learning adopted by the present invention can quickly initialize the network, greatly improve the training speed and save training time compared to the benchmark model.

[0055] Figure 8The segmentation results of the method of the present invention and other models are given. Among them, it can be seen from the first column that for large samples with blurred boundaries and irregular shapes, there are over-segmentation on the left side of Att-UNet and the bottom of DANet, and there is also over-segmentation on the top of DeepLabV3+. The segmentation edge of PSPNet is not smooth, resulting in jagged segmentation results. There is obvious under-segmentation in the TransUNet sample. The segmentation boundary of HRNet is not smooth, and many errors occur. It can be seen from the second column that lesions with smooth boundaries can basically be accurately segmented. DeepLabV3+ pays too much attention to the boundary, resulting in uneven segmentation boundaries. The improved model of the present invention effectively alleviates this problem. It can be seen from the third column that for samples marked with marks, there is basically no interference. It can be seen from the fourth column that for samples occluded by a ruler, Att-UNet over-segmented. This is because Att-UNet introduced an attention mechanism and paid too much attention to local details during training. MultiResUNet is also disturbed by the ruler to a certain extent and produces over-segmentation; DeepLabV3+ and TransUNet also detect the normal skin in the lower left corner as a lesion area. In summary, the segmentation result of the model of the present invention is closest to the label, which proves the effectiveness of the model of the present invention.

[0056] Therefore, the present invention designs a semantic segmentation model that improves DeepLabV3+. Compared with the baseline model DeepLabV3+, the use of a more lightweight backbone network and deep separable convolution reduces the computational complexity of the model and reduces the computational burden. As can be seen from Table 1, the number of parameters is reduced from the original 41.1M to 9.1M, a decrease of 78%. Figure 7 It can be seen that the loss function of the improved model of the present invention decreases fastest compared to other segmentation models and can converge better. At the same time, it is verified that the transfer learning adopted by the present invention can quickly initialize the network, greatly improve the training speed and save training time compared to the benchmark model.

[0057] By optimizing the ASPP module and adding CBAM and other technologies, mIoU, DICE, Acc, Sen, and Spe are 0.8249, 0.9041, 0.9424, 0.8955, and 0.9383, respectively, which increase by 3.34, 3.64, 1.82, 2.84, and 0.58 percentage points, respectively. The model of the present invention effectively improves the segmentation accuracy.

[0058] Figure 8 It can be seen that compared with other segmentation models, the model of the present invention is less affected by hair, rulers and other markers to the greatest extent. For samples with blurred boundary areas, the segmentation mask and the label mask are also the most similar. Intuitive and accurate segmentation results can be obtained in different scenarios.

[0059] In the above embodiments, the description's reference to "this embodiment" indicates that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily all refer to the same embodiment.

[0060] In the above-described embodiments, although the invention has been described in conjunction with specific embodiments of the invention, many substitutions, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other storage structures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed. Embodiments of the invention are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims.

[0061] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0062] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0063] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0064] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0065] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0066] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0067] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0068] The present invention can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0069] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. An improved melanoma segmentation method, characterized in that: The following steps are involved: S1: Construct a data set, preprocess the data set, and divide it into a training set and a test set; S2: Build a melanoma segmentation model based on the improved DeepLabV3+ network; S3: training and testing the melanoma segmentation model; S4: Use the trained melanoma segmentation model to segment melanoma.

2. The improved melanoma segmentation method according to claim 1, characterized in that: The preprocessing of the data set includes clipping and normalizing the data set.

3. The improved melanoma segmentation method according to claim 1, characterized in that: The specific improvements of the improved DeepLabV3+ network include: based on the traditional DeepLabV3+ network, the backbone network is replaced with ResNet50, and the atrous spatial pyramid pooling ASPP is improved to dense atrous spatial pyramid pooling DenseASPP; convolutional attention CBAM is added to the decoder part; and depthwise separable convolution is used in the decoder instead of conventional convolution.

4. The improved melanoma segmentation method according to claim 3, characterized in that: The construction of a melanoma segmentation model based on an improved DeepLabV3+ network includes: using an Adam optimizer and a DICE loss function, setting an initial learning rate and a batch size, and initializing the melanoma segmentation model using a weighted ResNet50 model already available on ImageNet.

Images