Skin lesion segmentation method based on auxiliary task and mutual information loss

Abstract

The invention relates to a skin lesion segmentation method based on an auxiliary task and mutual information loss, and belongs to the technical field of medical image segmentation. According to the method, through an image denoising auxiliary task, the ability of the network to confront lesion boundary noise is enhanced, and through an image restoration auxiliary task, the adaptability of the network to an incomplete lesion form is improved, so that a more accurate segmentation result is obtained. In addition, mutual information loss is adopted to supervise the auxiliary task, and it is ensured that feature consistency and space alignment are kept between the repair result of the auxiliary task and the original image. According to the method, the accuracy of skin lesion segmentation is remarkably improved by effectively combining the auxiliary task and the mutual information loss.

Description

Technical Field

[0001] The present invention relates to a skin lesion segmentation method based on auxiliary tasks and mutual information loss, belonging to the technical field of medical image segmentation. Background Art

[0002] Accurate segmentation of skin lesions is crucial in medical diagnosis. This method can automatically identify and locate the target area (lesion area) of concern to doctors from dermoscopic images. In clinical practice, skin lesion segmentation plays a key role in the early detection and treatment of a variety of high-risk skin diseases. It can greatly reduce the workload of doctors, reduce the overall treatment cost, and provide important basis for doctors' disease diagnosis, treatment, and surgical planning, thereby significantly improving patients' survival rate and quality of life.

[0003] In recent years, deep learning has made significant progress in the field of skin lesion segmentation with its excellent feature extraction and pattern recognition capabilities. However, when the lesion boundary is disturbed by environmental noise, the existing methods only implicitly extract boundary clues from the noise boundary, without explicitly removing the noise to robustly combat the noise factor, and lack the ability to deal with interference from external factors. In addition, hair or other artifacts are common in dermoscopic images and usually occlude the lesion area. Existing methods have difficulty distinguishing between the lesion area and the interfering area that causes the occlusion, resulting in inaccurate skin lesion segmentation. Therefore, it is necessary to propose a new method to deal with the problems of environmental noise interference and lesion occlusion that occur in the actual skin lesion segmentation process, so as to make skin lesion segmentation more accurate. Summary of the invention

[0004] The present invention designs and develops a skin lesion segmentation method based on auxiliary tasks and mutual information loss, which can enhance the robustness of the network to environmental noise and make the network adapt to incomplete lesion morphology, so as to overcome the defects of the prior art.

[0005] The technical solution provided by the present invention is:

[0006] One aspect of the present invention provides a skin lesion segmentation method based on auxiliary tasks and mutual information loss, comprising the following steps:

[0007] Step 1: Select a skin lesion segmentation dataset (e.g. ISIC2018) and preprocess the dermoscopic image dataset. The preprocessed dermoscopic image and its corresponding segmentation label are represented as X org and Y org , in X org Gaussian noise and random mask are added to obtain the preliminary noise image X N_org and the mask image X M_org , and finally X org and its corresponding X N_org, X M_org and Y org Perform data enhancement (including horizontal flipping, vertical flipping, and random rotation, etc.). The specific formula is described as follows:

[0008] X N_org =clip(X org +N(μ,σ 2 ); 0,255)

[0009]

[0010] Where clip(·; 0,255) represents the pixel clipping operation to ensure that the pixel values ​​of the noise image remain in the range of [0, 255], and N represents the Gaussian noise distribution with a mean of μ and a standard deviation of σ. org ; i, j) represents adding a mask operation, (i, j) is the pixel index, is the corresponding pixel value. Mask represents the mask area. When (i, j) is in the mask area, the pixel value is set to 0, otherwise it remains unchanged. DA (·, ·, ·, ·) represents the data enhancement strategy. X is the original dermoscopic image after data enhancement. X N , X M and Y represent the noise image, mask image and segmentation label corresponding to X;

[0011] Step 2: Divide the skin lesion segmentation dataset into training set, validation set and test set, and sample the original dermoscopic image X and its corresponding noise image X after data enhancement from the training set. N , mask image X M and segmentation label Y, forming a batch, that is, batch = Cat (X, X N ,X M , Y), where Cat is a tensor concatenation operation;

[0012] Step 3: X, X of batch data N and X M Input the image modeling network G(·) respectively to extract the original image features for the segmentation task Noisy image features for denoising tasks and mask image features for restoration tasks The formula is described as follows:

[0013]

[0014] Step 4: Features of the original image Noise Image Characteristics and mask image features Input different output layers respectively to get the final segmentation task results Image denoising task results And the image restoration task results The formula is described as follows:

[0015]

[0016] Among them, O seg (·) is the output layer for the main segmentation task, O denoise (·) and O mask (·) represent the output layers of the image denoising task and image restoration task respectively;

[0017] Step 5. Calculate and integrate the auxiliary task loss and the main task loss based on the output results to construct the total loss function. First, calculate the auxiliary task loss function to ensure the feature consistency and spatial alignment between the auxiliary task results and the original image, and use the mutual information loss to supervise the auxiliary task and quantify the statistical dependence between the auxiliary task results and the original dermoscopic image. Secondly, use the binary cross entropy loss and sieve loss to supervise the main segmentation task to ensure the pixel-level and structure-level consistency between the segmentation results and the labels. Finally, the total loss is defined as follows:

[0018] L total =L seg +L denoise +L mask

[0019] Among them, L total represents the total loss, L seg represents the main task loss, L denoise and L mask They represent the denoising auxiliary task loss and the restoration auxiliary task loss respectively;

[0020] Step 6: Repeat steps 3 to 5 to complete the training of the network;

[0021] Step 7: Input the dermoscopic image to be segmented into the trained network to obtain the final segmentation result.

[0022] The calculation formulas for the denoising auxiliary task loss and restoration auxiliary task loss described in step 5 are as follows:

[0023]

[0024] Where u and v represent two random variables, I(u; v) is the quantification of the statistical dependence of u and v, p(u) and p(v) are the marginal probability distributions of u and v respectively, p(u, v) is the joint probability distribution of u and v, and E p(u，v) represents the expectation of p(u, v). max represents the maximization operation, and maxI(·,·) represents maximizing the amount of information between two variables. Ldenoise and L mask Represents the image denoising task loss and image restoration task loss.

[0025] The main segmentation task loss described in step 5 is calculated as follows:

[0026]

[0027] Among them, Bec(·,·) represents the binary cross entropy loss, Dice(·,·) represents the sieve loss, and L seg represents the main segmentation task loss.

[0028] On the other hand, the present invention provides an electronic device comprising one or more processors and a memory, wherein the memory stores one or more instructions for executing a skin lesion segmentation method based on auxiliary tasks and mutual information loss.

[0029] In addition, the present invention also provides a computer-readable storage medium for storing computer instructions. When a processor executes these instructions, the above-mentioned skin lesion segmentation method based on auxiliary tasks and mutual information loss can be implemented.

[0030] The beneficial effects of the present invention are: a novel skin lesion segmentation method is proposed, which is designed to deal with noise and occlusion in dermoscopic images. The method requires the image modeling network to learn additional auxiliary tasks (i.e., image denoising tasks and image restoration tasks) while performing the main segmentation task. Through the additional auxiliary tasks, the modeling network can capture more robust boundary features and adapt to incomplete lesion morphology. At the same time, mutual information loss is used to ensure the effectiveness of auxiliary tasks. The method proposed in the present invention can improve the segmentation effect of the target lesions, and can also improve the stability of the network performance in actual medical environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings described herein are used to provide further understanding of the present invention and constitute a part of this application. The illustrative examples of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0032] Figure 1 It is a flow chart of the skin lesion segmentation method based on auxiliary tasks and mutual information loss of the present invention;

[0033] Figure 2 It is a schematic diagram of the skin lesion segmentation framework based on auxiliary tasks and mutual information loss of the present invention;

[0034] Figure 3 A flowchart of the image denoising auxiliary task of the present invention;

[0035] Figure 4It is a flow chart of the image restoration auxiliary task of the present invention. DETAILED DESCRIPTION

[0036] Next, the technical scheme in the embodiment of the present invention will be further described in detail in conjunction with the schematic diagram in the embodiment of the present invention. Note that the embodiments are only some specific examples of the present invention and do not cover all possible implementation methods. Based on the embodiments provided by the present invention, all other embodiments that can be derived by those skilled in the art without creative work should be regarded as the scope of protection of the present invention.

[0037] Embodiment 1:

[0038] In view of the problems existing in the above-mentioned prior art, this embodiment provides a skin lesion segmentation method based on auxiliary tasks and mutual information loss, which uses auxiliary tasks to improve segmentation performance. The method includes the following steps:

[0039] Step 1, obtaining a dermatoscopic image to be segmented;

[0040] Step 2: Preprocess the original data, perform noise interference and add masks to the processed images, and perform data enhancement on the images (including horizontal flipping, vertical flipping, and random rotation, etc.);

[0041] Step 3: Divide the image data set into a training set, a validation set, and a test set according to a preset ratio;

[0042] Step 4: Generate a skin lesion segmentation network, input the training set of the data set into the skin lesion segmentation network, use the mutual information loss based on information theory to supervise the auxiliary tasks, and integrate the losses of all tasks to train the skin lesion segmentation network, and use the validation set to evaluate the network performance;

[0043] Step 5: Input the test set of the data set into the trained image segmentation network to obtain the segmented image.

[0044] Specifically, the structure of the skin lesion segmentation network framework is as follows: Figure 2 The framework is based on two auxiliary tasks and mutual information loss, as follows:

[0045] ① Image denoising task. The image denoising task adds Gaussian noise to the dermoscopic images and then denoises these images to their original form. The image denoising task forces the image modeling network to remove noise to amplify the critical lesion boundaries, thereby improving the network's robustness to environmental noise. The overall process of the image denoising task can be expressed as:

[0046] X N_org =clip(X org +N(μ,σ 2 ); 0,255)

[0047] X N =DA(X N_org )

[0048]

[0049] ② Image restoration task. The image restoration task adds a random mask to the dermoscopic image and then restores the occluded area. It forces the network to infer the contextual information of the occluded lesion to reconstruct the complete lesion structure to enhance the network's adaptability to incomplete lesion morphology. The image restoration task can be expressed as:

[0050]

[0051] ③ Based on the mutual information loss of information theory, the statistical dependence between the results of the auxiliary task and the original image is calculated to ensure the feature consistency and spatial alignment between the two. The formula is as follows:

[0052]

[0053]

[0054] The following is an example of actual application, which includes the following steps:

[0055] Step 1, obtain the skin lesion segmentation dataset ISIC2018;

[0056] Step 2, preprocessing the original data. To ensure that the images have the same size and resolution, this implementation case uniformly adjusts the resolution of the original dermoscopic image and the corresponding mask to 256×256, adds noise and mask to the preprocessed image, and performs data enhancement on the image;

[0057] Step 3, divide the image dataset into training set, validation set and test set according to the preset ratio, including 1815, 259 and 520 images and corresponding labels respectively;

[0058] Step 4, generate a skin lesion segmentation network, input the training set of the data set into the skin lesion segmentation network to train the skin lesion segmentation network, and use the validation set to evaluate the model; the network uses the AdamW optimizer, the batch size is set to 8, the epoch is 300, the initial learning rate is set to 1e-3, and the CosineAnnealingLR is used to adjust the learning parameters, the minimum learning rate is 1e-5, and the learning rate adjustment cycle is 50;

[0059] Step 5: Input the test set of the dataset into the trained image segmentation model to obtain the segmented image. The segmentation results are measured by mIoU and DSC to measure the model performance.

[0060] After multiple experiments, the results of the skin lesion segmentation experiment show that the mIoU and DSC of this example reached 81.69% and 89.79%, as shown in Table 1. Compared with UNet, UTNetV2, TransFuse, MobileViTv2 and UNeXt-S, the method proposed in the present invention has improved the mIoU index by 3.83%, 2.72%, 1.06%, 2.46% and 2.60% respectively. The present invention puts forward additional requirements on the model through auxiliary tasks, and ensures the effectiveness of auxiliary tasks through mutual information loss based on information theory, so that the network is more robust to environmental noise, more adaptable to incomplete lesion morphology, and achieves more accurate segmentation.

[0061] Table 1 Experimental results

[0062]

[0063] Example 2

[0064] This embodiment provides an electronic device, including: a processor and a memory. The memory and the processor communicate with each other, and the memory stores one or more programs executable by the processor, specifically including instructions for executing the skin lesion segmentation method based on auxiliary tasks and mutual information loss as described in Example 1.

[0065] Example 3

[0066] This embodiment provides a computer-readable storage medium for storing one or more computer programs. When the program is executed by a processor, the skin lesion segmentation method based on auxiliary tasks and mutual information loss described in Embodiment 1 is implemented.

[0067] In summary, the present invention provides a new skin lesion segmentation method. This method uses image denoising and image restoration as auxiliary tasks and performs them in the same framework. By adding noise and masks to dermoscopic images and forcing the network to repair these images, the network is promoted to improve its robustness to environmental noise and adaptability to incomplete lesion morphology, thereby achieving improved segmentation performance.

[0068] The above contents are only preferred embodiments of the present invention and are not intended to limit the scope of the present invention. For those skilled in the art, the present invention can be adjusted and modified in various ways. Any modification, equivalent substitution or improvement of the present invention should be considered to be within the scope of protection of the present invention.

Claims

1. A skin lesion segmentation method based on auxiliary tasks and mutual information loss, characterized by: The steps include: Step 1: Select a skin lesion segmentation dataset and preprocess the dermoscopic image dataset; the preprocessed dermoscopic image and its corresponding segmentation label are represented as X org and Y org , in X org Gaussian noise and random mask are added to obtain the preliminary noise image X N_org and the mask image X M_org , and finally X org and its corresponding X N_org , X M_org and Y org Data enhancement is performed, including horizontal flipping, vertical flipping and random rotation. The specific formula is described as follows: X N_org =clip(X org +N(μ,σ 2 );0.255) Where, clip(·; 0, 255) represents the pixel clipping operation to ensure that the pixel values ​​of the noise image remain in the range of [0, 255], N represents the Gaussian noise distribution, with a mean of μ and a standard deviation of σ; M(X org ; i, j) represents adding a mask operation, (i, j) is the pixel index, X org(i，j) is the corresponding pixel value; Mask represents the mask area. When (i, j) is in the mask area, the pixel value is set to 0, otherwise it remains unchanged. DA (·, ·, ·, ·) represents the data enhancement strategy. X is the original dermoscopic image after data enhancement. X N , X M and Y represent the noise image, mask image and segmentation label corresponding to X; Step 2: Divide the skin lesion segmentation dataset into training set, validation set and test set, and sample the original dermoscopic image X and its corresponding noise image X after data enhancement from the training set. N , mask image X M and segmentation label Y, forming a batch, that is, batch = Cat (X, X N ,X M , Y), where Cat is a tensor concatenation operation; Step 3: X, X of batch data N and X M Input the image modeling network G(·) respectively to extract the original image features for the segmentation task Noisy image features for denoising tasks and mask image features for restoration tasks The formula is described as follows: Step 4: Features of the original image Noise Image Characteristics and mask image features Input different output layers respectively to get the final segmentation task results Image denoising task results And the image restoration task results The formula is described as follows: Among them, O seg (·) is the output layer for the main segmentation task, O denoise (·) and O mask (·) represent the output layers of the image denoising task and image restoration task respectively; Step 5. Calculate and integrate the auxiliary task loss and the main task loss according to the output results to construct the total loss function. First, use the mutual information loss to supervise the auxiliary task to ensure the feature consistency and spatial alignment between the auxiliary task results and the original image, and quantify the statistical dependence between the auxiliary task results and the original dermoscopic image. Secondly, use the binary cross entropy loss and sieve loss to supervise the main segmentation task to ensure the pixel-level and structure-level consistency between the segmentation results and the labels. Finally, the total loss is defined as follows: L total =L seg +L denoise +L mask Among them, L total represents the total loss, L seg represents the main segmentation task loss, L denoise and L mask They represent the denoising auxiliary task loss and the restoration auxiliary task loss respectively; Step 6: Repeat steps 3 to 5 to complete the training of the network; Step 7: Input the dermoscopic image to be segmented into the trained network to obtain the final segmentation result.

2. The skin lesion segmentation method based on auxiliary tasks and mutual information loss according to claim 1, characterized in that: The calculation formulas for the denoising auxiliary task loss and restoration auxiliary task loss described in step 5 are as follows: Where u and v represent two random variables, I(u; v) is the quantification of the statistical dependence of u and v, p(u) and p(v) are the marginal probability distributions of u and v respectively, p(u, v) is the joint probability distribution of u and v, and E p(u，v) represents the expectation of p(u, v); maxI(·, ·) represents maximizing the amount of information between two variables; L denoise and L mask Represents the image denoising task loss and image restoration task loss.

3. The skin lesion segmentation method based on auxiliary tasks and mutual information loss according to claim 1, characterized in that: The main segmentation task loss described in step 5 is calculated as follows: Among them, Bec(·,·) represents the binary cross entropy loss, Dice(·,·) represents the sieve loss, and L seg represents the main segmentation task loss.

Images