Melanoma image segmentation method, system and device based on improved DeepLabv3 + model and medium

Abstract

The invention discloses a melanoma image segmentation method, system and device based on an improved DeepLabv3 + model and a medium, and the method comprises the steps: obtaining a melanoma data set, carrying out the preprocessing of the melanoma data set, and dividing the preprocessed image set into a training set, a verification set and a test set; an improved DeepLabv3 + model is built, the improved DeepLabv3 + model comprises an encoder and a decoder, the encoder comprises a feature extraction module and an ASPP module, and the decoder comprises an FCAM module and a sampling and output module; inputting the training set and the verification set into an improved DeepLabv3 + model for training and verification, calculating a loss function and performing back propagation, and updating network parameters to obtain a trained optimal parameter network; and inputting the test set into the model after parameter optimization, and outputting a melanoma image segmentation effect picture. By introducing a cross attention mechanism, changing a backbone network and adding branch fusion features between convolutional layers, the precision and efficiency of the DeepLabv3 + model in skin disease segmentation are improved, and the method is suitable for practical clinical application.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a melanoma image segmentation method, system, device and medium based on an improved DeepLabv3+ model. Background Art

[0002] Skin cancer is the most common type of cancer, of which melanoma accounts for 75% of all skin cancer deaths and is the most lethal type of skin cancer. The early symptoms of melanoma are usually a small mole or spot, which may change, such as darkening or enlarging. However, in the early stages of melanoma, it can be cured with a minor surgical procedure. Studies on skin lesions in recent years have shown that the risk of melanoma is increasing in people under the age of 40. Understanding the early warning signs of skin cancer helps ensure that cancer is detected and treated before it spreads. Since it is extremely difficult for ordinary people to detect abnormal skin lesions in time, segmenting dermoscopic images to assist dermatologists in timely diagnosing worsening melanoma lesions and taking treatment measures as soon as possible has important clinical value.

[0003] At present, a series of neural network algorithms such as Convolutional Neural Networks (CNN) have been applied to melanoma image segmentation, which has improved the accuracy of previous melanoma image segmentation models and achieved good prediction results. Deep learning models can perform multi-level deep abstraction of data, discover complex dependencies in data, and extract data features. In addition, due to the large differences in the morphology and size of melanoma, multi-scale training is required during the training of the convolutional neural network model to obtain better segmentation results. In summary, the study of melanoma image segmentation for skin images is of great significance in helping dermatologists to perform automated auxiliary diagnosis. It is mainly manifested in the following aspects: (1) Early medical auxiliary diagnosis of melanoma Melanoma is highly lethal, has a rapid onset, a large number of cases, and a large number of deaths. It is one of the fastest-causing and most deadly cancers in the world. Due to the resolution and other reasons, the dermoscopic images of melanoma skin lesions are not accurate even if they are visually diagnosed by professional institutions and physicians. The early surgical cure rate of melanoma skin lesions is high. Therefore, the diagnosis and treatment of melanoma in the early stage is crucial. (2) Automating analysis The purpose of image segmentation is to extract and further analyze the contours of the target object. It is an important step in automating medical image analysis and is also the key to various medical image applications, such as computer-aided diagnosis, treatment planning and delivery, and computer-assisted intervention.

[0004] The "Deeplabv3+" in the convolutional neural network model realizes the classification of each pixel in the image by combining the encoder-decoder structure and atrous convolution, thereby realizing the segmentation of different objects in the image. It is widely used in various semantic segmentation tasks, especially in the fields of autonomous driving and medical image analysis. However, the DeepLabv3+ model has the following shortcomings:

[0005] 1. Weak feature correlation: Although the DeepLabv3+ model can capture multi-scale features during feature extraction, it lacks sufficient interaction and correlation between different features. This deficiency may cause the model to perform less than ideal in complex scenes, especially in images with rich details and complex backgrounds. When processing melanoma lesions with rich details and complex structures, the original model cannot fully utilize the features of each level, resulting in inaccurate segmentation results.

[0006] 2. The model has a large amount of computation: The DeepLabv3+ model uses a relatively complex backbone network (Xception). Although these networks can extract rich features, they also bring large computational overhead and memory usage. The large amount of computation leads to slow model inference speed and is not suitable for real-time applications, especially when running on resource-constrained devices.

[0007] 3. Insufficient fusion of multi-scale features: Although the ASPP module can extract multi-scale features, the degree of feature fusion between different atrous convolutional layers is limited, and the information of each scale cannot be fully utilized. As a result, the segmentation results are unstable and the generalization ability is poor when facing lesions of different sizes. Summary of the invention

[0008] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to propose a melanoma image segmentation method, system, device and medium based on the improved DeepLabv3+ model, which improves the accuracy and efficiency of the DeepLabv3+ model in skin disease segmentation by introducing a cross-attention mechanism (FCAM), changing the backbone network and adding branch fusion features between convolutional layers.

[0009] The technical solution adopted by the present invention is:

[0010] The melanoma image segmentation method based on the improved DeepLabv3+ model includes the following steps:

[0011] Obtain a melanoma dataset and preprocess the image set, and divide the preprocessed image set into a training set, a validation set, and a test set;

[0012] Building an improved DeepLabv3+ model, wherein the improved DeepLabv3+ model includes an encoder and a decoder, wherein the encoder includes a feature extraction module and an ASPP module, and the decoder includes an FCAM module and a sampling and output module;

[0013] The input image enters the feature extraction module, and resnet18 is used as the backbone network to extract the low-level features and high-level features of the input image; the high-level features enter the ASPP module, which consists of 4 convolutional layers and a global average pooling layer. Branches are added between the convolutional layers to fuse features of different scales, and the outputs of the convolutional layers and the global average pooling layers with different expansion rates are spliced. The features are integrated through 1*1 convolution, and finally the output features of the ASPP module are formed;

[0014] The output features of the ASPP module are spliced ​​with the low-level features to obtain spliced ​​features, and the spliced ​​features and the output features of the ASPP module are used as input features and input into the FCAM module to obtain spatial attention weights and channel attention weights. The input features are spliced ​​and 3*3 convolved to extract higher-level features, and the higher-level features are multiplied by the spatial attention weights and the channel attention weights respectively to obtain spatial attention output features and channel attention output features. The higher-level features are added to the spatial attention output features and the channel attention output features and then convolved to output an integrated feature map. The sampling and output module upsamples the integrated feature map to the size of the input image to obtain the output image of the encoder;

[0015] The training set and validation set are input into the improved DeepLabv3+ model for training and validation, the loss function is calculated and back-propagation is performed, and the network parameters are updated in combination with the optimization algorithm to obtain the trained optimal parameter network;

[0016] The test set is input into the model with optimized parameters, and the melanoma image segmentation effect diagram is output.

[0017] Furthermore, the image preprocessing is specifically to convert the color space of the image into the RGB space and normalize the image so that the pixel value is in the range of [0,1].

[0018] Furthermore, low-level features are extracted through conv1, bn1, relu, and maxpool layers, and high-level features are extracted through conv1, bn1, relu, maxpool, and layer1 to layer4 layers to extract high-level features layer by layer.

[0019] Furthermore, the convolution kernel sizes of the four convolutional layers are set to 1 1×1, 3 3×3, and the dilation rates of the three 3×3 convolutional layers are set to 6, 12, and 18, respectively, wherein branches are added between the convolutional layers with dilation rates of 6 and 12, and between the convolutional layers with dilation rates of 12 and 18 to fuse features of different scales.

[0020] Furthermore, the FCAM module includes a spatial attention block SA and a channel attention block CA. The spatial attention block SA takes the concatenated features and the output features of the ASPP module as input to obtain the spatial attention weight; the channel attention block CA takes the output features of the ASPP module as input to obtain the channel attention weight.

[0021] Furthermore, the concatenated features and the output features of the ASPP module are subjected to batch normalization and Sigmoid nonlinear 3×3 convolution to obtain spatial attention weights.

[0022] Furthermore, the output features of the ASPP module are compressed along the spatial dimension using global pooling and maximum pooling methods to obtain two vectors, and the two vectors are applied to a shared fully connected layer and a Sigmoid operator to obtain channel attention weights.

[0023] Melanoma image segmentation system based on improved DeepLabv3+ model, including

[0024] Dataset acquisition module: obtains melanoma dataset and preprocesses the image set, and divides the preprocessed image set into training set, validation set and test set;

[0025] DeepLabv3+ model building module: building an improved DeepLabv3+ model, wherein the improved DeepLabv3+ model includes an encoder and a decoder, wherein the encoder includes a feature extraction module and an ASPP module, and the decoder includes an FCAM module and a sampling and output module;

[0026] The input image enters the feature extraction module, and resnet18 is used as the backbone network to extract the low-level features and high-level features of the input image; the high-level features enter the ASPP module, which consists of 4 convolutional layers and a global average pooling layer. Branches are added between the convolutional layers to fuse features of different scales, and the outputs of the convolutional layers and the global average pooling layers with different expansion rates are spliced. The features are integrated through 1*1 convolution, and finally the output features of the ASPP module are formed;

[0027] The output features of the ASPP module are spliced ​​with the low-level features to obtain spliced ​​features, and the spliced ​​features and the output features of the ASPP module are used as input features and input into the FCAM module to obtain spatial attention weights and channel attention weights. The input features are spliced ​​and 3*3 convolved to extract higher-level features, and the higher-level features are multiplied by the spatial attention weights and the channel attention weights respectively to obtain spatial attention output features and channel attention output features. The higher-level features are added to the spatial attention output features and the channel attention output features and then convolved to output an integrated feature map. The sampling and output module upsamples the integrated feature map to the size of the input image to obtain the output image of the encoder;

[0028] Model training module: input the training set and validation set into the improved DeepLabv3+ model for training and validation, calculate the loss function and perform back propagation, and update the network parameters in combination with the optimization algorithm to obtain the trained optimal parameter network;

[0029] Model testing module: Input the test set into the model with optimized parameters and output the melanoma image segmentation effect diagram.

[0030] Melanoma image segmentation device based on the improved DeepLabv3+ model, including:

[0031] Memory: a computer program storing a melanoma image segmentation method based on an improved DeepLabv3+ model, which is a computer-readable device;

[0032] Processor: used to implement a melanoma image segmentation method based on an improved DeepLabv3+ model when executing the computer program.

[0033] A computer-readable storage medium stores a computer program, which, when executed by a processor, can implement a melanoma image segmentation method based on an improved DeepLabv3+ model.

[0034] The beneficial effects of the present invention are:

[0035] 1. By introducing the cross-attention mechanism (FCAM), the interaction and correlation between different features are enhanced, which can help the model more accurately identify and segment lesion areas when dealing with melanoma lesion areas with rich details; the cross-attention mechanism can comprehensively consider global and local information, making the model's performance more stable and accurate in complex scenarios.

[0036] 2. Change the backbone network to ResNet18 to reduce the computational complexity and memory usage of the model, so as to improve the model's reasoning speed and resource utilization efficiency. Compared with more complex networks, ResNet18 has fewer parameters and computational complexity, making it suitable for running in resource-constrained environments while maintaining high feature extraction capabilities.

[0037] 3. Add branches between different atrous convolutional layers in the ASPP module for fusion to improve the fusion effect of multi-scale features, thereby improving the model's ability to identify and segment lesions of different sizes. Multi-scale feature fusion can reduce the deviation that may be caused by single-scale features and improve the model's adaptability and robustness to lesions of various shapes and sizes.

[0038] In summary, through the above improvements, the present invention not only overcomes the shortcomings of the original DeepLabv3+ model in feature correlation, computational complexity and multi-scale feature fusion, but also significantly improves the accuracy and efficiency of the model in skin disease segmentation, making it suitable for application in actual clinical environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is the structure diagram of the improved deeplabv3+ model of the present invention;

[0040] Figure 2 This is a structural diagram of the FCAM cross attention module of the present invention;

[0041] Figure 3 This is the structural diagram of the spatial attention block SA of the present invention;

[0042] Figure 4 This is the structure diagram of the channel attention block CA of the present invention;

[0043] Figure 5 This is a comparison chart of the iou values ​​under different model training in an embodiment of the present invention;

[0044] Figure 6 This is a comparison chart of the MIOU values ​​of different backbone networks in different deeplabv3+ model structures according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The specific contents of the present invention are further explained in detail below in conjunction with embodiments.

[0046] The melanoma image segmentation method based on the improved DeepLabv3+ model includes the following steps:

[0047] Obtain a melanoma dataset and preprocess the image set, and divide the preprocessed image set into a training set, a validation set, and a test set;

[0048] Building an improved DeepLabv3+ model, wherein the improved DeepLabv3+ model includes an encoder and a decoder, wherein the encoder includes a feature extraction module and an ASPP module, and the decoder includes an FCAM module and a sampling and output module;

[0049] The input image enters the feature extraction module, and resnet18 is used as the backbone network to extract the low-level features and high-level features of the input image; the high-level features enter the ASPP module, which consists of 4 convolutional layers and a global average pooling layer. Branches are added between the convolutional layers to fuse features of different scales, and the outputs of the convolutional layers and the global average pooling layers with different expansion rates are spliced. The features are integrated through 1*1 convolution, and finally the output features of the ASPP module are formed;

[0050] The output features of the ASPP module are spliced ​​with the low-level features to obtain spliced ​​features, and the spliced ​​features and the output features of the ASPP module are used as input features and input into the FCAM module to obtain spatial attention weights and channel attention weights. The input features are spliced ​​and 3*3 convolved to extract higher-level features, and the higher-level features are multiplied by the spatial attention weights and the channel attention weights respectively to obtain spatial attention output features and channel attention output features. The higher-level features are added to the spatial attention output features and the channel attention output features and then convolved to output an integrated feature map. The sampling and output module upsamples the integrated feature map to the size of the input image to obtain the output image of the encoder;

[0051] The training set and validation set are input into the improved DeepLabv3+ model for training and validation, the loss function is calculated and back-propagation is performed, and the network parameters are updated in combination with the optimization algorithm to obtain the trained optimal parameter network;

[0052]

[0053] in

[0054] ·y i ——Represents the label of sample i, the positive class is 1 and the negative class is 0

[0055] ·p i ——Indicates the probability that sample i is predicted to be a positive class

[0056] The loss function is the cross entropy loss function. In the case of binary classification, the model only needs to predict two results. The predicted probabilities for each category are p and 1-p. The expression is (the base of log is e). N represents the total number of pixels used to calculate the average loss of all samples or pixels, thereby avoiding the absolute value of the loss from changing due to different sample numbers, making the optimization more stable.

[0057] In the training and validation process of DeepLabv3+, the cross-entropy loss function is used to measure the difference between the pixel-level category probability distribution predicted by the model and the true label distribution. Specifically, the cross-entropy loss calculates the prediction of each pixel, generates the gradient of the loss to the network parameters through back-propagation, and gradually adjusts the parameters in combination with the optimization algorithm (such as Adam or SGD) to make the model output closer to the true label distribution. During the training process, the cross-entropy loss guides the network to optimize the parameters of the feature extraction module and the decoder to enhance the semantic segmentation performance of the model; during the validation process, the cross-entropy loss is calculated to evaluate the generalization ability of the model for unseen data, and the reduction of the loss is used as a measure of whether the optimization is effective.

[0058] The test set is input into the model with optimized parameters, and the melanoma image segmentation effect diagram is output.

[0059] Furthermore, the image preprocessing is specifically to convert the color space of the image into the RGB space and normalize the image so that the pixel value is in the range of [0,1].

[0060] Furthermore, low-level features are extracted through conv1, bn1, relu, and maxpool layers, and high-level features are extracted through conv1, bn1, relu, maxpool, and layer1 to layer4 layers to extract high-level features layer by layer.

[0061] Furthermore, the convolution kernel sizes of the four convolutional layers are set to 1 1×1, 3 3×3, and the dilation rates of the three 3×3 convolutional layers are set to 6, 12, and 18, respectively, wherein branches are added between the convolutional layers with dilation rates of 6 and 12, and between the convolutional layers with dilation rates of 12 and 18 to fuse features of different scales.

[0062] Furthermore, the FCAM module includes a spatial attention block SA and a channel attention block CA. The spatial attention block SA takes the spliced ​​features and the output features of the ASPP module as input, which helps to refine the pixel positioning and target boundaries and obtain the spatial attention weight; the channel attention block CA takes the output features of the ASPP module as input to obtain the channel attention weight.

[0063] Furthermore, the concatenated features and the output features of the ASPP module are subjected to batch normalization and Sigmoid nonlinear 3×3 convolution to obtain spatial attention weights.

[0064] Furthermore, the output features of the ASPP module are compressed along the spatial dimension using global pooling and maximum pooling methods to obtain two vectors, and the two vectors are applied to a shared fully connected layer and a Sigmoid operator to obtain channel attention weights.

[0065] Melanoma image segmentation system based on improved DeepLabv3+ model, including

[0066] Dataset acquisition module: obtains melanoma dataset and preprocesses the image set, and divides the preprocessed image set into training set, validation set and test set;

[0067] DeepLabv3+ model building module: building an improved DeepLabv3+ model, wherein the improved DeepLabv3+ model includes an encoder and a decoder, wherein the encoder includes a feature extraction module and an ASPP module, and the decoder includes an FCAM module and a sampling and output module;

[0068] The input image enters the feature extraction module, and resnet18 is used as the backbone network to extract the low-level features and high-level features of the input image; the high-level features enter the ASPP module, which consists of 4 convolutional layers and a global average pooling layer. Branches are added between the convolutional layers to fuse features of different scales, and the outputs of the convolutional layers and the global average pooling layers with different expansion rates are spliced. The features are integrated through 1*1 convolution, and finally the output features of the ASPP module are formed;

[0069] The output features of the ASPP module are spliced ​​with the low-level features to obtain spliced ​​features, and the spliced ​​features and the output features of the ASPP module are used as input features and input into the FCAM module to obtain spatial attention weights and channel attention weights. The input features are spliced ​​and 3*3 convolved to extract higher-level features, and the higher-level features are multiplied by the spatial attention weights and the channel attention weights respectively to obtain spatial attention output features and channel attention output features. The higher-level features are added to the spatial attention output features and the channel attention output features and then convolved to output an integrated feature map. The sampling and output module upsamples the integrated feature map to the size of the input image to obtain the output image of the encoder;

[0070] Model training module: input the training set and validation set into the improved DeepLabv3+ model for training and validation, calculate the loss function and perform back propagation, and update the network parameters in combination with the optimization algorithm to obtain the trained optimal parameter network;

[0071] Model testing module: Input the test set into the model with optimized parameters and output the melanoma image segmentation effect diagram.

[0072] Melanoma image segmentation device based on the improved DeepLabv3+ model, including:

[0073] Memory: a computer program storing a melanoma image segmentation method based on an improved DeepLabv3+ model, which is a computer-readable device;

[0074] Processor: used to implement a melanoma image segmentation method based on an improved DeepLabv3+ model when executing the computer program.

[0075] A computer-readable storage medium stores a computer program, which, when executed by a processor, can implement a melanoma image segmentation method based on an improved DeepLabv3+ model.

[0076] Example

[0077] 1. Collecting Melanoma Dataset

[0078] The melanoma dataset was obtained from the official website of the International Skin Imaging Collaboration (ISIC), and then the melanoma disease image dataset with sample labels was divided into training set, validation set, and test set in a ratio of 7:2:1.

[0079] 2. Train in the improved deeplabv3+ network

[0080] The labeled data set in step 1 is put into the improved DeepLabv3+ network for training, so that it has the ability to segment the lesion area and form an image semantic segmentation model of DeepLabv3+ based on the cross attention mechanism. Figure 1 As shown in the figure, the network mainly includes an image preprocessing module, a feature extraction module, an ASPP (atrous spatial pyramid pooling) module, a FCAM (feature cross attention module) module, and a final upsampling and output module.

[0081] Image preprocessing module: The image preprocessing module can convert the input 512*512 image into a format that the network can process. First, the color space of the image is converted into RGB space, and then the image is normalized so that the pixel value is in the range of [0,1].

[0082] Feature extraction module: The present invention uses the pre-trained backbone network resnet18 as the backbone network to extract low-level and high-level features of the input image. Different from the backbone network Xception of the original model, the original Xception network has about 22.9M parameters, while the resnet18 network has only about 11.7M parameters, which significantly reduces the computational complexity and memory usage, improves the model's reasoning speed and resource utilization efficiency, and makes it more suitable for real-time medical applications and resource-constrained devices. The low-level feature extraction is obtained through the conv1, bn1, relu, and maxpool layers, and the high-level features are then extracted through the layer1 to layer4 layers to extract high-level features layer by layer.

[0083] Improved Atrous Spatial Pyramid Pooling (ASPP) module: The ASPP module uses atrous convolutions with different dilation rates to extract features and capture multi-scale information. It consists of 4 convolution modules and a global average pooling module. Global average pooling can average each channel of the entire feature map to generate a global average vector, which contains the global context information of the entire image and can improve the robustness of the model. The convolution kernel sizes of the 4 convolution modules are set to 1*1, 3*3 (atrous rate 6), 3*3 (atrous rate 12), and 3*3 (atrous rate 18), respectively. Adding branch links between 3*3 (atrous rate 6) and 3*3 (atrous rate 12) and between 3*3 (atrous rate 12) and 3*3 (atrous rate 18) can better combine features of different scales and improve the model's recognition ability and segmentation stability of lesion areas at various scales. The outputs of these convolutions with different dilation rates and the global average pooling layer are concatenated, and then feature integration is performed through 1*1 convolution to finally form the output of the ASPP module.

[0084] The FCAM module, i.e., the feature cross attention module, includes a spatial attention block SA and a channel attention block CA. Through the spatial attention mechanism and the channel attention mechanism, the spatial salient area and important channel features of the feature map can be effectively enhanced, thereby improving the expressiveness of the features. Figure 2As shown in the figure, first, the output features of the ASPP module and the low-level features are concatenated, and then 3×3 convolution, batch normalization and ReLU units are applied to the connected features; then the concatenated features and the output features of the ASPP are input into the FCAM as two branches, and the spatial attention block takes the fused features and the output features of the ASPP module as input to obtain the spatial attention weights; the channel attention block takes the output features of the ASPP module as input, and generates the channel attention weights through the CA module; in the backbone, the input features (i.e., the fused features and the ASPP output features) are concatenated and a 3*3 convolutional layer (Conv) is used to extract higher-level features, and are multiplied with the attention weights generated by the spatial and channel attention respectively, and the new features after the original features are multiplied by the attention weights are added, and finally the convolution outputs the required feature map.

[0085] like Figure 3 As shown in Figure 1, the spatial attention block takes the fused features and the output of the spatial branch as input, which helps to refine pixel localization and object boundaries. The features from the spatial branch are batch normalized and convolved with a sigmoid nonlinear 3×3 convolution, and then multiplied by the fused features. The two-dimensional attention map generated by the spatial features corresponds to the importance of each pixel. It focuses on the localization of objects and refines boundaries using spatial information. Figure 4 As shown in Figure 2, the context features of the spatial attention block and the context branch are output to the channel attention block. The global pooling and maximum pooling methods are used to compress the context features along the spatial dimension to obtain two vectors. These two vectors are then applied to a shared fully connected layer and a Sigmoid operator to generate an attention map. Next, the attention map is multiplied by the output features of the spatial attention block and added to the fused features.

[0086] The FCAM module enhances the expressiveness of feature maps and the saliency of important regions by combining spatial attention and channel attention mechanisms. Through the effective fusion of low-level and high-level features, the FCAM module improves the performance of the model in semantic segmentation tasks, especially when dealing with complex and detailed scenes, providing higher segmentation accuracy and robustness.

[0087] The final upsampling and output module upsamples the integrated feature map to the size of the original input image and generates the final segmentation result.

[0088]

[0089] Table 1

[0090] like Figure 5 As shown, the iou value comparison chart of the present invention and the other 6 models, among which the iou value of the present invention is the highest, indicating that the evaluation of the present invention is more accurate and effective.

[0091] like Figure 6 As shown in Table 1, the original deeplabv3+ is compared with the original deeplabv3+ with branches added, the original deeplabv3+ with only the cross-attention mechanism added, and the added branches and cross-attention mechanism of the present invention. At the same time, 7 different backbone networks are used for feature extraction in each model. It can be concluded that the model improvement of the present invention combines the resnet18 backbone network to extract low-level and high-level features of the input image, and the obtained MIOU value is the highest, indicating that its evaluation is more accurate and effective.

[0092] In summary, the improved DeepLabv3+ model in the present invention shows higher accuracy, real-time performance and robustness when processing the melanoma lesion area segmentation task, and provides more accurate lesion localization.

Claims

1. A melanoma image segmentation method based on an improved DeepLabv3+ model, characterized in that: The following steps are involved: Obtain a melanoma dataset and preprocess the image set, and divide the preprocessed image set into a training set, a validation set, and a test set; Building an improved DeepLabv3+ model, wherein the improved DeepLabv3+ model includes an encoder and a decoder, wherein the encoder includes a feature extraction module and an ASPP module, and the decoder includes an FCAM module and a sampling and output module; The input image enters the feature extraction module, and resnet18 is used as the backbone network to extract the low-level features and high-level features of the input image; the high-level features enter the ASPP module, which consists of 4 convolutional layers and a global average pooling layer. Branches are added between the convolutional layers to fuse features of different scales, and the outputs of the convolutional layers and the global average pooling layers with different expansion rates are spliced. The features are integrated through 1*1 convolution, and finally the output features of the ASPP module are formed; The output features of the ASPP module are spliced ​​with the low-level features to obtain spliced ​​features, and the spliced ​​features and the output features of the ASPP module are used as input features and input into the FCAM module to obtain spatial attention weights and channel attention weights. The input features are spliced ​​and 3*3 convolved to extract higher-level features, and the higher-level features are multiplied by the spatial attention weights and the channel attention weights respectively to obtain spatial attention output features and channel attention output features. The higher-level features are added to the spatial attention output features and the channel attention output features and then convolved to output an integrated feature map. The sampling and output module upsamples the integrated feature map to the size of the input image to obtain the output image of the encoder; The training set and validation set are input into the improved DeepLabv3+ model for training and validation, the loss function is calculated and back-propagation is performed, and the network parameters are updated in combination with the optimization algorithm to obtain the trained optimal parameter network; The test set is input into the model with optimized parameters, and the melanoma image segmentation effect diagram is output.

2. The melanoma image segmentation method based on the improved DeepLabv3+ model according to claim 1, characterized in that: Image preprocessing specifically involves converting the image's color space into RGB space and normalizing the image so that the pixel values ​​are in the range of [0,1].

3. The melanoma image segmentation method based on the improved DeepLabv3+ model according to claim 1, characterized in that: The low-level features are extracted through the conv1, bn1, relu, and maxpool layers, and the high-level features are extracted through the conv1, bn1, relu, maxpool, and layer1 to layer4 layers to extract high-level features layer by layer.

4. The melanoma image segmentation method based on the improved DeepLabv3+ model according to claim 1, characterized in that: The convolution kernel sizes of the four convolutional layers are set to 1 1×1, 3 3×3, and the dilation rates of the three 3×3 convolutional layers are set to 6, 12, and 18, respectively. Branches are added between the convolutional layers with dilation rates of 6 and 12, and between the convolutional layers with dilation rates of 12 and 18 to fuse features of different scales.

5. The melanoma image segmentation method based on the improved DeepLabv3+ model according to claim 1, characterized in that: The FCAM module includes a spatial attention block SA and a channel attention block CA. The spatial attention block SA takes the concatenated features and the output features of the ASPP module as input to obtain the spatial attention weight; the channel attention block CA takes the output features of the ASPP module as input to obtain the channel attention weight.

6. The melanoma image segmentation method based on the improved DeepLabv3+ model according to claim 5, characterized in that: The concatenated features and the output features of the ASPP module are subjected to batch normalization and Sigmoid nonlinear 3×3 convolution to obtain spatial attention weights.

7. The melanoma image segmentation method based on the improved DeepLabv3+ model according to claim 5, characterized in that: The output features of the ASPP module are compressed along the spatial dimension using global pooling and maximum pooling methods to obtain two vectors, and the two vectors are applied to a shared fully connected layer and a Sigmoid operator to obtain channel attention weights.

8. Melanoma image segmentation system based on improved DeepLabv3+ model, characterized in that: include Dataset acquisition module: obtains melanoma dataset and preprocesses the image set, and divides the preprocessed image set into training set, validation set and test set; DeepLabv3+ model building module: building an improved DeepLabv3+ model, wherein the improved DeepLabv3+ model includes an encoder and a decoder, wherein the encoder includes a feature extraction module and an ASPP module, and the decoder includes an FCAM module and a sampling and output module; The input image enters the feature extraction module, and resnet18 is used as the backbone network to extract the low-level features and high-level features of the input image; the high-level features enter the ASPP module, which consists of 4 convolutional layers and a global average pooling layer. Branches are added between the convolutional layers to fuse features of different scales, and the outputs of the convolutional layers and the global average pooling layers with different expansion rates are spliced. The features are integrated through 1*1 convolution, and finally the output features of the ASPP module are formed; The output features of the ASPP module are spliced ​​with the low-level features to obtain spliced ​​features, and the spliced ​​features and the output features of the ASPP module are used as input features and input into the FCAM module to obtain spatial attention weights and channel attention weights. The input features are spliced ​​and 3*3 convolved to extract higher-level features, and the higher-level features are multiplied by the spatial attention weights and the channel attention weights respectively to obtain spatial attention output features and channel attention output features. The higher-level features are added to the spatial attention output features and the channel attention output features and then convolved to output an integrated feature map. The sampling and output module upsamples the integrated feature map to the size of the input image to obtain the output image of the encoder; Model training module: input the training set and validation set into the improved DeepLabv3+ model for training and validation, calculate the loss function and perform back propagation, and update the network parameters in combination with the optimization algorithm to obtain the trained optimal parameter network; Model testing module: Input the test set into the model with optimized parameters and output the melanoma image segmentation effect diagram.

9. A melanoma image segmentation device based on an improved DeepLabv3+ model, characterized in that: include: Memory: a computer program storing a melanoma image segmentation method based on an improved DeepLabv3+ model as described in any one of claims 1 to 7, which is a computer-readable device; Processor: used to implement the melanoma image segmentation method based on the improved DeepLabv3+ model described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement the melanoma image segmentation method based on the improved DeepLabv3+ model described in any one of claims 1 to 7.

Images