Skin lesion segmentation method based on multi-scale features and boundary cooperation

Abstract

The invention relates to a skin lesion segmentation method based on multi-scale features and boundary cooperation, and belongs to the technical field of medical image segmentation. The method focuses on multi-scale feature combination and accurate boundary modeling, skin lesion segmentation is decomposed into multi-scale feature extraction and boundary modeling tasks, and challenges caused by large size difference of a lesion area and fuzzy lesion boundary are solved by enhancing the representation capability of multi-scale features and optimizing boundary representation of the lesion area. According to the method, the multi-scale features and boundary information are effectively aggregated, so that the accuracy of a segmentation result is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to a skin lesion segmentation method based on multi-scale feature and boundary collaboration. Background Art

[0002] Skin lesion segmentation plays a vital role in medical diagnosis. Its core lies in automatically identifying and accurately locating the lesion area from the dermoscopic image. This technology not only helps doctors quickly and accurately obtain information about the lesion area, but also plays a key role in the early detection and treatment of high-risk skin diseases such as skin cancer. Through effective image segmentation methods, the sensitivity and specificity of lesion detection can be significantly improved, reducing the workload of doctors in processing massive image data, reducing the misdiagnosis rate, and optimizing the overall treatment process.

[0003] Deep learning technology has developed rapidly in the field of medical image segmentation with its powerful feature extraction capabilities and efficient data processing capabilities. Among them, convolutional neural networks (CNNs) have become the preferred method for medical image segmentation with their excellent performance and accuracy. However, compared with other medical image segmentation tasks, the size of lesion areas in skin lesion segmentation tasks varies greatly, and the contrast between normal skin and lesion areas is low, which affects the accuracy of segmentation. Therefore, it is necessary to propose a new method to cope with the challenges brought by the large size differences of skin lesion images and the blurred lesion boundaries, so as to achieve accurate segmentation of skin lesions. Summary of the invention

[0004] The present invention designs and develops a skin lesion segmentation method based on multi-scale features and boundary collaboration, which extracts and strengthens multi-scale features, models boundary features, and solves the impact of boundary fuzziness on segmentation, thereby improving the model segmentation performance. The technical solution provided by the present invention is:

[0005] One aspect of the present invention provides a skin lesion segmentation method based on multi-scale features and boundary collaboration, comprising the following steps:

[0006] Step 1: Select a skin lesion segmentation dataset (such as ISIC2018), divide the skin lesion segmentation dataset into a training set, a validation set, and a test set, preprocess the dermoscopic image dataset, and the preprocessed image to be segmented and its corresponding segmentation label are represented as) and Y respectively;

[0007] Step 2: Sample the image to be segmented X and the corresponding segmentation label Y from the training set to form a batch, that is, batch = Cat(X, Y), where Cat is a tensor concatenation operation;

[0008] Step 3: Input the image X to be segmented in the batch data into the shallow semantic module to extract low-level features to retain the location and boundary information of the lesion area in the shallow semantic information for subsequent segmentation prediction and boundary prediction. The formula is:

[0009]

[0010] Among them, F i Represents the combination of convolution, ReLU activation function and BatchNorm, W represents the weight corresponding to each channel, represents the addition of elements, It means multiplication by channel, X1 is the shallow semantic feature map;

[0011] Step 4: Input the shallow semantic feature map X1 into the encoder to extract image features, and further input the encoded feature map X2 into the decoder to obtain the segmentation prediction result X seg , the specific formula is as follows:

[0012] X2=EM(X1)

[0013] X seg =DM(X2)

[0014] Among them, EM(·) represents the encoder, DM(·) represents the decoder, and X seg Represents the segmentation prediction result;

[0015] Step 5: Input the shallow semantic feature map X1 and the encoding feature map X2 into the boundary information modeling module at the same time, model the boundary features, and output the boundary prediction result X boundary , the specific formula is as follows:

[0016] X boundary =BM(X1,X2)

[0017] Among them, BM(·,·) represents the boundary modeling module, X boundary Indicates the boundary prediction result;

[0018] Step 6: Based on the output segmentation prediction result X seg And the boundary prediction result X boundary , respectively calculate the segmentation loss L seg and boundary loss L boundary , and construct the total loss function L total First, the cross entropy loss is used to optimize the skin lesion segmentation and calculate the segmentation loss. Secondly, given the imbalance of the foreground and background in the boundary label, the Dice loss function is used to calculate the boundary loss. Finally, the segmentation loss and boundary loss are integrated to obtain the overall loss. The formula for this process is as follows:

[0019]

[0020] L total =L seg +L boundary

[0021] Among them, Y i and Represent the true label and predicted label of the i-th pixel, N represents the number of pixels, L ce represents the cross entropy loss, L Dice represents the Dice loss function, L seg and L boundary Represent the segmentation loss and boundary loss respectively, L total Represents the total loss.

[0022] Step 7: Repeat steps 3 to 6 to complete model training;

[0023] Step 8: Input the dermoscopic image to be segmented into the trained model to obtain the final segmentation result.

[0024] The encoder and decoder described in step 4 are mainly composed of a multi-scale grouping attention module, which can be divided into two parts: feature grouping and attention allocation. The input features are grouped by feature grouping. On this basis, grouping attention is used to supplement the information between feature map channels (i.e., attention allocation). The specific calculation formula is as follows:

[0025]

[0026] Among them, X concat Represents the feature map generated by feature grouping. H, W, and C represent the height, width, and channel of the image, respectively. i represents the transformation of the i-th group of features, c represents the c channel of the feature map, S c Represents the information statistics of the cth channel generated by the concatenated features after global pooling. represents the convolution and activation function of size 1, a i (c) is the grouped channel attention, represents the weight of the cth channel in the global spatial information, V c Represents the information of the cth channel of the final feature map.

[0027] The calculation formula of the boundary information modeling module described in step 5 is as follows:

[0028]

[0029] Among them, X1 is the shallow semantic feature map, X2 is the feature map obtained by the encoder, up(·) represents upsampling, Indicates channel-wise multiplication, represents the addition of corresponding elements, F i Represents different convolution operations, F sobel represents sobel convolution, X boundary represents the boundary prediction result, and ψ(·) represents the multi-layer perceptron (MLP).

[0030] Another aspect of the present invention provides an electronic device, comprising: one or more processors and a memory, wherein the memory stores instructions for executing the above-mentioned skin lesion segmentation method based on multi-scale feature and boundary collaboration.

[0031] In addition, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the skin lesion segmentation method based on multi-scale feature and boundary collaboration as described above.

[0032] The beneficial effect of the present invention is that the present invention proposes a skin lesion segmentation method based on multi-scale features and boundary collaboration. The method first extracts more location information and boundary information of the lesion area through a shallow semantic module; secondly, a multi-scale grouping attention module is used to enhance the multi-scale feature representation; finally, a boundary information modeling module is designed to model the boundary of the lesion area using boundary information. Through the above operations, the present invention effectively solves the problem of large lesion size differences and blurred lesion boundaries in skin lesion images, and achieves an improvement in the segmentation effect of skin lesion images. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings described herein are used to provide further understanding of the present invention and constitute a part of this application. The illustrative examples of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0034] Figure 1 A flowchart of the skin lesion segmentation method based on multi-scale features and boundary collaboration of the present invention;

[0035] Figure 2 A schematic diagram of a skin lesion segmentation model based on multi-scale features and boundary collaboration according to the present invention;

[0036] Figure 3 It is a schematic diagram of the shallow semantic module of the present invention;

[0037] Figure 4 Schematic diagram of the multi-scale grouping attention module of the present invention;

[0038] Figure 5 It is a schematic diagram of the boundary information modeling module of the present invention. DETAILED DESCRIPTION

[0039] The technical solution will be described in more detail below in conjunction with the schematic diagram in the embodiment of the present invention. It should be noted that the embodiments only represent a part of the specific examples of the present invention and do not cover all possible implementations. Based on these embodiments, other implementation methods that can be derived by those skilled in the art without performing creative work should all be regarded as the scope of protection of the present invention.

[0040] Embodiment 1:

[0041] In view of the problems existing in the above-mentioned prior art, this embodiment provides a skin lesion segmentation method based on multi-scale features and boundary collaboration, which improves the performance of skin lesion image segmentation by enhancing the representation capability of multi-scale features and optimizing the boundary representation of the lesion area. The specific steps are as follows:

[0042] Step 1: Obtain a medical image of skin lesions to be segmented;

[0043] Step 2: Divide the image dataset into a training set, a validation set, and a test set according to a preset ratio;

[0044] Step 3: Preprocess the dermoscopic image dataset;

[0045] Step 4: Generate a skin lesion segmentation model, input the training set into the skin lesion segmentation model, train the skin lesion segmentation model, and evaluate the model using the validation set;

[0046] Step 5: Input the test set into the trained image segmentation model to obtain the skin disease segmentation image.

[0047] Specifically, the structure of the skin lesion segmentation model is as follows: Figure 2 As shown in the figure, the model is based on two subtasks: segmentation prediction and boundary prediction. The structure is as follows:

[0048] ① Shallow semantic module. The structure of the shallow semantic module is as follows: Figure 3 As shown in Figure 1, this module extracts the location and boundary information of the lesion area from the shallow semantic information, and adds a channel attention branch to make up for the lost information between channels. The specific calculation formula is as follows:

[0049]

[0050] Among them, F i Represents the combination of convolution, ReLU activation function and BatchNorm, W represents the weight corresponding to each channel, represents the addition of elements, It means channel multiplication, and X1 is the shallow feature map obtained.

[0051] ② Construct the encoder and decoder. Both the encoder and the decoder are mainly composed of a multi-scale group attention module, in which the encoder reduces the resolution by introducing a pooling operation; the decoder combines the encoder information of the corresponding level and performs upsampling to restore the size of the feature map. The structure of the multi-scale group attention module can be found in Figure 4 As shown in the figure, the multi-scale grouping attention module can be divided into two parts: feature grouping and attention allocation. The input features are grouped by feature grouping. On this basis, the grouping attention is used to supplement the information between the feature map channels (i.e., attention allocation). The specific calculation formula is as follows:

[0052]

[0053] Among them, X concat Represents the feature map generated by feature grouping. H, W, and C represent the height, width, and channel of the image, respectively. i represents the transformation of the i-th group of features, c represents the c channel of the feature map, S c Represents the information statistics of the cth channel generated by the concatenated features after global pooling. represents the convolution and activation function of size 1, a i (c) is the grouped channel attention, represents the weight of the cth channel in the global spatial information, V c Represents the information of the cth channel of the final feature map.

[0054] ③Boundary modeling module, see Figure 5 This module combines the shallow semantic feature map X1 and the encoder feature map X2 to model the boundary. The specific calculation process is as follows:

[0055]

[0056] Among them, X1 is the shallow semantic feature map of the input, X2 is the encoded feature obtained by the encoder, up(·) represents upsampling, Indicates channel-wise multiplication, Indicates the addition of corresponding elements. i Represents different convolution operations, F sobel represents sobel convolution, X boundary represents the boundary prediction result, and ψ(·) represents the multi-layer perceptron MLP.

[0057] ④ Set the loss function as the learning objective of the model. According to the output segmentation prediction results and boundary prediction results, calculate the segmentation loss and boundary loss respectively, and construct the total loss function. First, use the cross entropy loss to optimize the skin lesion segmentation and calculate the segmentation loss; second, in view of the imbalance of foreground and background in the boundary label, use the Dice loss function to calculate the boundary loss; finally, integrate the segmentation loss and boundary loss to get the overall loss. The formula for this process is as follows:

[0058]

[0059] L total =L seg +L boundary

[0060] Among them, Y i and Represent the true label and predicted label of the i-th pixel, N represents the number of pixels, L ce represents the cross entropy loss, L Dice represents the Dice loss function, L seg and L boundary Represent the segmentation loss and boundary loss respectively, L total Represents the total loss.

[0061] The following is an example of actual application, which includes the following steps:

[0062] Step 1, obtain the skin lesion segmentation dataset ISIC2018;

[0063] Step 2: Divide the image dataset into training set, validation set, and test set according to the preset ratio, including 1815, 259, and 520 images and corresponding labels respectively;

[0064] Step 3, preprocessing the original data. To ensure that the images have the same size and resolution, this implementation case uniformly adjusts the resolution of the original dermoscopic image and the corresponding mask to 256×256;

[0065] Step 4, generate a skin lesion segmentation model, input the training set into the skin lesion segmentation model, train the skin lesion segmentation model, and use the validation set to evaluate the model; the network uses the Adam optimizer, the batch size is set to 16, the epoch is 300, and the initial learning rate is set to 1e-3;

[0066] Step 5: Input the test set into the trained image segmentation model to obtain the segmented image. The segmentation results are measured by IoU, DSC and Precision to measure the model performance.

[0067] The experimental results of skin lesion segmentation show that the present invention has achieved significant results in IoU coefficient, Dice coefficient and Precision. Compared with the baseline model ResUNet, the present invention has improved 1.44%, 1.8% and 2.26% in each indicator respectively. Compared with Deeplabv3, which also uses multi-scale features, the present invention has improved 2.81%, 3.23% and 5.9% in IoU coefficient, Dice coefficient and Precision respectively, proving that the algorithm proposed in the present invention can effectively use multi-scale features to model the lesion area, thereby improving the segmentation effect of skin lesions.

[0068] Table 1 Experimental results

[0069]

[0070] Example 2

[0071] This embodiment provides an electronic device, including: a processor and a memory. The memory and the processor communicate with each other, and the memory stores one or more programs executable by the processor, specifically including instructions for executing the skin lesion segmentation method based on multi-scale features and boundary collaboration as described in Example 1.

[0072] Example 3

[0073] This embodiment provides a computer-readable storage medium for storing one or more computer programs. When the program is executed by a processor, the skin lesion segmentation method based on multi-scale feature and boundary collaboration described in Embodiment 1 is implemented.

[0074] In summary, the present invention provides a new skin lesion segmentation method for the problem of large lesion size differences and blurred lesion boundaries in the skin lesion image segmentation task. The main advantage of this algorithm is that it can enhance the representation ability of multi-scale features and optimize the boundary representation of the lesion area. Through the collaboration of multi-scale features and boundaries, the segmentation effect is effectively improved.

[0075] The above contents are only preferred embodiments of the present invention and are not intended to limit the scope of the present invention. For those skilled in the art, the present invention can be adjusted and modified in various ways. Any modification, equivalent substitution or improvement of the present invention should be considered to be within the scope of protection of the present invention.

Claims

1. A skin lesion segmentation method based on multi-scale features and boundary collaboration, characterized in that: The steps include: Step 1: Select a skin lesion segmentation dataset, divide the skin lesion segmentation dataset into a training set, a validation set, and a test set, preprocess the dermoscopic image dataset, and represent the preprocessed image to be segmented and its corresponding segmentation label as X and Y respectively; Step 2: Sample the image to be segmented X and the corresponding segmentation label Y from the training set to form a batch, that is, batch = Cat(X, Y), where Cat is a tensor concatenation operation; Step 3: Input the image X to be segmented in the batch data into the shallow semantic module to extract low-level features to retain the lesion area location and boundary information in the shallow semantic information for subsequent segmentation prediction and boundary prediction; the formula is: Among them, F i Represents the combination of convolution, ReLU activation function and BatchNorm, W represents the weight corresponding to each channel, represents the addition of elements, It means multiplication by channel, X1 is the shallow semantic feature map; Step 4: Input the shallow semantic feature map X1 into the encoder to extract image features, and further input the encoded feature map X2 into the decoder to obtain the segmentation prediction result X seg , the specific formula is as follows: X2=EM(X1) X seg =DM(X2) Among them, EM(·) represents the encoder, DM(·) represents the decoder, and X seg Represents the segmentation prediction result; Step 5: Input the shallow semantic feature map X1 and the encoding feature map X2 into the boundary information modeling module at the same time, model the boundary features, and output the boundary prediction result X boundary , the specific formula is as follows: X boundary =BM(X1,X2) Among them, BM(·,·) represents the boundary modeling module, X boundary Indicates the boundary prediction result; Step 6: Based on the output segmentation prediction result X seg And the boundary prediction result X boundary , respectively calculate the segmentation loss L seg and boundary loss L boundary , and construct the total loss function L total ; First, the cross entropy loss is used to optimize the skin lesion segmentation and calculate the segmentation loss; second, in view of the imbalance of the foreground and background in the boundary label, the Dice loss function is used to calculate the boundary loss; finally, the segmentation loss and boundary loss are integrated to obtain the overall loss; the formula for this process is as follows: L total =L seg +L boundary Among them, Y i and Represent the true label and predicted label of the i-th pixel, N represents the number of pixels, L ce represents the cross entropy loss, L Dice represents the Dice loss function, L seg and L boundary Represent the segmentation loss and boundary loss respectively, L total represents the total loss; Step 7: Repeat steps 3 to 6 to complete model training; Step 8: Input the dermoscopic image to be segmented into the trained model to obtain the final segmentation result.

2. The skin lesion segmentation method based on multi-scale features and boundary collaboration according to claim 1 is characterized in that: The encoder and decoder described in step 4 are mainly composed of a multi-scale grouping attention module, which can be divided into two parts: feature grouping and attention allocation. The input features are grouped by feature grouping. On this basis, group attention is used to supplement the information between feature map channels. The specific calculation formula is as follows: Among them, X concat Represents the feature map generated by feature grouping. H, W, and C represent the height, width, and channel of the image, respectively. i represents the transformation of the i-th group of features, c represents the c channel of the feature map, S c Represents the information statistics of the cth channel generated by the concatenated features after global pooling. represents the convolution and activation function of size 1, a i (c) is the grouped channel attention, represents the weight of the cth channel in the global spatial information, V c Represents the information of the cth channel of the final feature map.

3. The skin lesion segmentation method based on multi-scale features and boundary collaboration according to claim 1, characterized in that: The calculation formula of the boundary information modeling module described in step 5 is as follows: Among them, X1 is the shallow semantic feature map, X2 is the feature map obtained by the encoder, up(·) represents upsampling, Indicates channel-wise multiplication, represents the addition of corresponding elements, F i Represents different convolution operations, F sobel represents sobel convolution, X boundary represents the boundary prediction result, and ψ(·) represents the multi-layer perceptron (MLP).

Images