Skin lesion segmentation method based on boundary direction connectivity

Abstract

The invention relates to a skin lesion segmentation method based on boundary direction connectivity, and belongs to the technical field of medical image segmentation. According to the method, boundary features are decoupled from dermatoscope image features, and a decoder is guided to learn more boundary information by using boundary enhancement features. By extracting the direction information of the boundary, the perceptual ability of the network to the boundary is enhanced, and the predictive ability of the network model to the boundary part is improved, so that skin lesion segmentation is more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to a skin lesion segmentation method based on boundary direction connectivity. Background Art

[0002] Skin lesion segmentation is of great significance in modern medicine. The key lies in automatically extracting and accurately identifying lesion areas from skin images. This technology can not only provide doctors with fast and accurate lesion information, but also plays a vital role in the early screening and intervention of serious skin diseases such as skin cancer.

[0003] With the rapid development of deep learning technology, convolutional neural networks (CNNs) have become the preferred method for medical image segmentation. CNN can automatically extract image features, and its excellent performance makes it the main choice in the field of medical image segmentation. In the task of skin lesion segmentation, boundary information plays a vital role in revealing the spatial relationship between different regions in skin images. However, skin lesion areas usually do not differ significantly from normal skin in texture or brightness, resulting in unclear boundaries. In addition, hair in dermoscopic images may cover part of the lesion area, destroying the integrity of the boundary, making the prediction of the boundary more difficult. Therefore, it is necessary to propose a new method to address the challenges brought by the blurred lesion boundary and the incomplete lesion area boundary, so as to achieve more accurate segmentation of skin lesions. Summary of the invention

[0004] The present invention designs and develops a skin lesion segmentation method based on boundary directional connectivity. By extracting and strengthening boundary features, the lesion boundary is modeled to solve the problems of blurred boundaries and incomplete boundaries of the lesion area, thereby improving the model segmentation performance.

[0005] The above-mentioned object of the present invention is achieved by the following technical solutions:

[0006] One aspect of the present invention provides a skin lesion segmentation method based on boundary directional connectivity, comprising the following steps:

[0007] Step 1: Select a skin lesion segmentation dataset (e.g. ISIC2018), preprocess the dermoscopic image dataset, and obtain the preprocessed dermoscopic image X and its corresponding segmentation label Y;

[0008] Step 2: Divide the skin lesion segmentation dataset into a training set, a validation set, and a test set. Sample the dermoscopic image X to be segmented and the corresponding segmentation label Y from the training set to form a batch, which can be formulated as follows:

[0009] batch=Cat(X,Y)

[0010] Among them, Cat is the tensor concatenation operation;

[0011] Step 3: Input the dermoscopic image X of the batch data into the model, and extract the shallow semantic features e0 of the dermoscopic image through pre-convolution for subsequent segmentation prediction, boundary connectivity prediction and boundary prediction. The specific formula is expressed as:

[0012] e0=pconv(X)

[0013] Among them, pconv(·) represents the pre-convolution operation;

[0014] Step 4: Input the shallow semantic feature e0 into the encoder to extract the high-level semantic feature e4 from the shallow semantic feature e0. The calculation formula is:

[0015] e4=EM(e0)

[0016] Among them, EM(·) represents the encoder, e4 is the high-level semantic feature;

[0017] Step 5: Boundary connectivity prediction and boundary prediction. The shallow semantic feature map e0 and the high-level semantic feature map e4 are simultaneously input into the boundary connectivity decoupling module, the boundary features are decoupled from the overall image features and the boundary features are modeled, and finally the boundary connectivity prediction results and boundary prediction results are output. This process can be described by the following formula:

[0018] e b ,X b ,X c =BCDM(e0,e4)

[0019] Among them, BCDM(·,·) represents the boundary connection decoupling module, e b represents the boundary enhancement feature used to guide the decoder to learn boundary information, X b represents the boundary prediction result, X c Represents the boundary connectivity prediction result;

[0020] Step 6: Segmentation prediction. The high-level semantic feature map e4 is input into the decoder, and the additional boundary information e b As auxiliary input, the segmentation prediction result is obtained. This process can be formulated as:

[0021] X seg =DM(e4,e b )

[0022] Among them, DM(·,·) represents the decoder, X seg is the segmentation prediction result;

[0023] Step 7: Calculate the segmentation loss, boundary connectivity loss and boundary loss based on the output segmentation prediction, boundary connectivity prediction and boundary prediction, and construct the total loss function. First, use the cross entropy loss to optimize the skin lesion segmentation; second, use the Dice loss function to calculate the boundary loss; finally, use the connectivity consistency loss proposed in BiconNet to calculate the connectivity loss. All loss functions are defined as follows:

[0024]

[0025] L con =L BCE (X c ,Y c )

[0026]

[0027] L total =L seg +λ1L boundary +λ2L con

[0028] Among them, Y i and represents the true label and predicted label of the i-th pixel, N represents the number of pixels, and Y c represents the corresponding boundary connectivity label, L ce represents the cross entropy loss, L Dice represents the Dice loss, L BCE represents the connectivity consistency loss proposed in BiconNet, L seg , L boundary , L con and L total They represent segmentation loss, boundary connectivity loss, boundary loss and total loss respectively, λ1 and λ2 are parameters for adjusting loss weights;

[0029] Step 8. Repeat steps 3 to 7 to complete the training of the model;

[0030] Step 9: Input the dermoscopic image to be segmented into the trained model to obtain the final segmentation result.

[0031] The boundary connectivity decoupling module described in step 5 is composed of a feature direction information decoupling module and a boundary feature recoding module. The feature direction information decoupling module calculates the loss with the boundary direction connectivity label to obtain the boundary direction information, and adjusts the high-level semantic feature e4 to be consistent with V through convolution. direct The same channel dimensions. Then e4 and V directThe channels are divided into eight groups of features, corresponding to feature combinations in different directions, as the input of the boundary feature recoding module. The calculation formula of the feature direction information decoupling module is as follows:

[0032] d prior =F conv (e4)

[0033] V direct =σ(F conv (GAP(d prior )))

[0034] Among them, F conv represents the combination of convolution-normalization-activation function, d prior represents the connectivity feature in the boundary direction, GAP represents the global average pooling operation, σ(·) represents the Sigmoid activation function, V direct Represents the obtained boundary direction attention.

[0035] The calculation formula for boundary feature recoding is as follows:

[0036]

[0037] e b =F conv (MLP(F conv (Edge direct )))

[0038] X b =F3(F1(e b ))

[0039] Among them, F i Represents the combination of convolution, ReLU activation function and BatchNorm, represents the characteristics of the input subpath, and Represent the boundary features generated by the convolution branch and the MLP branch, respectively. Represents the boundary features in one direction, Represents the boundary features obtained by the subpath in a specific direction, represents channel-wise multiplication, Indicates the addition of corresponding elements, Edge direct represents the boundary direction enhancement feature, X b represents the boundary prediction result, e b Represents the boundary enhancement feature.

[0040] The decoder described in step 6 is composed of a designed boundary feature enhancement decoding module, and uses a residual connection to fuse edge features. enhanceAfter that, the corresponding features in the encoder are combined to restore the feature map size by upsampling and channel-by-channel splicing. The calculation formula of the boundary feature enhancement decoding module is as follows:

[0041]

[0042] Among them, e b represents the boundary enhancement feature, d i-1 represents the output of the previous layer decoder, σ(·) represents the Sigmoid activation function, F1(·) represents 1×1 convolution, Att edge Represents the obtained boundary attention

[0043] The beneficial effect of the present invention is that the present invention proposes a skin lesion segmentation method based on boundary directional connectivity. The method first uses boundary directional connectivity labels to guide the model to learn boundary information on different directional sub-paths, enhances the representation of boundary features on each directional sub-path, and simultaneously fuses shallow semantic information to predict the boundary. In addition, the method uses boundary enhancement features to guide the network to learn boundary information at the decoder stage. Through the above improvements, the present invention can effectively cope with the challenges brought by blurred lesion boundaries and incomplete lesion area boundaries, and improve the segmentation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide further understanding of the present invention and constitute a part of this application. The illustrative examples of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0045] Figure 1 It is a flow chart of the skin lesion segmentation method based on boundary direction connectivity of the present invention;

[0046] Figure 2 Schematic diagram of the skin lesion segmentation model based on boundary direction connectivity of the present invention;

[0047] Figure 3 It is a schematic diagram of a boundary feature enhancement decoding module of the present invention;

[0048] Figure 4 It is a schematic diagram of the boundary communication decoupling module of the present invention. DETAILED DESCRIPTION

[0049] The technical solution will be described in more detail below in conjunction with the schematic diagram in the embodiment of the present invention. It should be noted that the embodiments only represent a part of the specific examples of the present invention and do not cover all possible implementations. Based on these embodiments, other implementation methods that can be derived by those skilled in the art without performing creative work should all be regarded as the scope of protection of the present invention.

[0050] Example:

[0051] In view of the problems existing in the above-mentioned prior art, this embodiment provides a skin lesion segmentation method based on boundary direction connectivity, which significantly improves the image segmentation performance by enhancing the model's learning ability of lesion boundary information and modeling ability of lesion boundaries. The method includes the following steps:

[0052] Step 1, obtaining a dermatoscopic image to be segmented;

[0053] Step 2: Divide the skin lesion segmentation dataset into a training set, a validation set, and a test set according to a preset ratio;

[0054] Step 3: Preprocessing the dermoscopic image data;

[0055] Step 4: generating a skin lesion segmentation model, inputting the training set into the skin lesion segmentation model to train the skin lesion segmentation model, and evaluating the model using the validation set;

[0056] Step 5: Input the test set into the trained image segmentation model to obtain the skin lesion segmentation image.

[0057] Specifically, the structure of the skin lesion segmentation model is as follows: Figure 2 As shown in the figure, the model is based on three subtasks: segmentation prediction, boundary connectivity prediction, and boundary prediction. The structure is as follows:

[0058] ①Use the pre-convolution module of the U-Net network and add a channel attention branch to make up for the lost information between channels to extract the location and boundary information of the lesion area in the shallow semantic information. The specific calculation formula is as follows:

[0059]

[0060] Among them, F i Represents the combination of convolution, ReLU activation function and BatchNorm, W represents the weight corresponding to each channel, represents the addition of elements, represents channel-wise multiplication, pconv(·) represents the pre-convolution operation, and e0 is the shallow semantic feature obtained.

[0061] ② Construct an encoder and a boundary enhancement decoder. The encoder is composed of a ResNeXt-101 network, which extracts features by loading ImageNet pre-trained weights; the boundary enhancement decoder is composed of a boundary feature enhancement decoding module, and fuses edge features by using residual connections. enhance After that, the corresponding features in the encoder are combined to restore the feature map size by upsampling and channel-by-channel concatenation. The structure of the boundary feature enhancement decoding module is as follows: Figure 3 As shown, the calculation formula of the boundary feature enhancement decoding module is as follows:

[0062]

[0063] Among them, e b represents the boundary enhancement feature, d i-1 represents the output of the previous layer decoder, σ(·) represents the Sigmoid activation function, F1(·) represents a 1×1 convolution, GAP represents the global average pooling operation, Att edge represents boundary attention, d enhance Main feature.

[0064] ③Boundary connection decoupling module, see Figure 4 This module combines the shallow semantic feature map e0 with the high-level semantic feature map e4, decouples the boundary features from the overall image features and models the boundary features, and finally outputs the boundary connectivity prediction results and boundary prediction results.

[0065] ④ Set the loss function as the learning objective of the model. First, use the cross entropy loss to optimize the skin lesion segmentation. Second, use the Dice loss function to optimize the boundary prediction. Finally, use the connectivity consistency proposed in BiconNet to calculate the connectivity loss. Based on the above process, construct the total loss function, and its formula is as follows:

[0066]

[0067] L con =L BCE (X c ,Y c )

[0068] L total =L seg +λ1L boundary +λ2L con

[0069] Among them, Y i and represents the true label and predicted label of the i-th pixel, N represents the number of pixels, and Y c represents the corresponding boundary connectivity label, L ce represents the cross entropy loss, L Dice represents the Dice loss, L BCE represents the connectivity consistency loss proposed in BiconNet, L seg , L boundary , L con and L total They represent segmentation loss, boundary connectivity loss, boundary loss and total loss respectively, and λ1 and λ2 are parameters for adjusting the loss weights.

[0070] The following is an example of actual application, which includes the following steps:

[0071] Step 1, obtain the skin lesion segmentation dataset ISIC2018;

[0072] Step 2: Divide the image dataset into training set, validation set, and test set according to the preset ratio, including 1815, 259, and 520 images and corresponding labels respectively;

[0073] Step 3, preprocessing the dermoscopic image data. To ensure that the images have the same size and resolution, this implementation case uniformly adjusts the resolution of the original dermoscopic image and the corresponding mask to 256×256;

[0074] Step 4, generate a skin lesion segmentation model, input the training set into the skin lesion segmentation model to train the skin lesion segmentation model, and use the validation set to evaluate the model; the network uses the Adam optimizer, the batch size is set to 16, the epoch is 300, and the initial learning rate is set to 1e-3;

[0075] Step 5: Input the test set into the trained image segmentation model, obtain the segmented image, and measure the model performance by IoU, DSC, and Recall.

[0076] The experimental results of skin lesion segmentation show that this example has achieved significant improvements in IoU coefficient, Dice coefficient and Precision, reaching 86.76%, 88.50% and 93.07% respectively, as shown in Table 1. Compared with ResUNet, ICL-Net, MS-Red, nn-UNet and TransFuse, the method proposed in the present invention has improved the IoU coefficient by 1.95%, 1.89%, 2.16%, 1.15% and 1.01% respectively. The above results show that for the boundary direction connectivity prediction task and the lesion boundary prediction task, the present invention effectively improves the image segmentation effect by enhancing the model's learning ability of lesion boundary information and modeling ability of lesion boundaries.

[0077] Table 1 Experimental results

[0078]

[0079] In summary, the present invention provides a new skin lesion segmentation method for the problem of blurred lesion boundaries and incomplete lesion area boundaries in the skin lesion image segmentation task. The main advantage of this algorithm is that it effectively improves the segmentation effect by predicting lesion boundary connectivity and lesion boundary, and by enhancing the model's ability to learn lesion boundary information and model lesion boundaries.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made to the present invention shall be included in the protection scope of the present invention.

Claims

1. A skin lesion segmentation method based on boundary directional connectivity, characterized in that: The steps include: Step 1: Select a skin lesion segmentation dataset, preprocess the dermoscopic image dataset, and obtain the preprocessed dermoscopic image X and its corresponding segmentation label Y; Step 2: Divide the skin lesion segmentation dataset into a training set, a validation set, and a test set. Sample the dermoscopic image X to be segmented and the corresponding segmentation label Y from the training set to form a batch, which can be formulated as follows: batch=Cat(X,Y) Among them, Cat is the tensor concatenation operation; Step 3: Input the dermoscopic image X of the batch data into the model, and extract the shallow semantic features e0 of the dermoscopic image through pre-convolution for subsequent segmentation prediction, boundary connectivity prediction and boundary prediction. The specific formula is expressed as: e0=pconv(X) Among them, pconv(·) represents the pre-convolution operation; Step 4: Input the shallow semantic feature e0 into the encoder to extract the high-level semantic feature e4 from the shallow semantic feature e0. The calculation formula is: e4=EM(e0) Among them, EM(·) represents the encoder, e4 is the high-level semantic feature; Step 5: Boundary connectivity prediction and boundary prediction: The shallow semantic feature map e0 and the high-level semantic feature map e4 are simultaneously input into the boundary connectivity decoupling module, the boundary features are decoupled from the overall image features and the boundary features are modeled, and finally the boundary connectivity prediction results and boundary prediction results are output. This process can be described by the following formula: e b ,X b ,X c =BCDM(e0,e4) Among them, BCDM(·,·) represents the boundary connection decoupling module, e b represents the boundary enhancement feature used to guide the decoder to learn boundary information, X b represents the boundary prediction result, X c Represents the boundary connectivity prediction result; Step 6: Segmentation prediction: Input the high-level semantic feature map e4 into the decoder and use the additional boundary information e b As auxiliary input, the segmentation prediction result is obtained. This process can be formulated as: X seg =DM(e4,e b ) Among them, DM(·,·) represents the decoder, X seg is the segmentation prediction result; Step 7: Calculate the segmentation loss, boundary connectivity loss and boundary loss according to the output segmentation prediction, boundary connectivity prediction and boundary prediction, and construct the total loss function; first, use the cross entropy loss to optimize the skin lesion segmentation; second, use the Dice loss function to calculate the boundary loss; finally, use the connectivity consistency loss proposed in BiconNet to calculate the connectivity loss; all loss functions are defined as follows: L con =L BCE (X c ,Y c ) L total =L seg +λ1L boundary +λ2L con Among them, Y i and represents the true label and predicted label of the i-th pixel, N represents the number of pixels, and Y c represents the corresponding boundary connectivity label, L ce represents the cross entropy loss, L Dice represents the Dice loss, L BCE represents the connectivity consistency loss proposed in BiconNet, L seg , L boundary , L con and L total They represent segmentation loss, boundary connectivity loss, boundary loss and total loss respectively, λ1 and λ2 are parameters for adjusting loss weights; Step 8. Repeat steps 3 to 7 to complete the training of the model; Step 9: Input the dermoscopic image to be segmented into the trained model to obtain the final segmentation result.

2. The skin lesion segmentation method based on boundary direction connectivity according to claim 1, characterized in that: The boundary connection decoupling module described in step 5 is composed of a feature direction information decoupling module and a boundary feature recoding module. The feature direction information decoupling module adjusts the high-level semantic feature e4 to be consistent with V by convolution. direct Same channel dimensions, then e4 and V direct The features are divided into eight groups according to the channel, which are used as the input of the boundary feature recoding module. The calculation formula of the feature direction information decoupling module is as follows: d prior =F conv (e4) V direct =σ(F conv (GAP(d prior ))) Among them, F conv represents the combination of convolution-normalization-activation function, d prior represents the connectivity feature in the boundary direction, GAP represents the global average pooling operation, σ(·) represents the Sigmoid activation function, V direct Represents the obtained boundary direction attention; The calculation formula for boundary feature recoding is as follows: e b =F conv (MLP(F conv (Edge direct ))) X b =F3(F1(and b )) Among them, F i Represents the combination of convolution, ReLU activation function and BatchNorm, represents the characteristics of the input subpath, and Represent the boundary features generated by the convolution branch and the MLP branch, respectively. Represents the boundary features in one direction, Represents the boundary features obtained by the subpath in a specific direction, represents channel-wise multiplication, Indicates the addition of corresponding elements, Edge direct represents the boundary direction enhancement feature, X b represents the boundary prediction result, e b Represents the boundary enhancement feature.

3. The skin lesion segmentation method based on boundary direction connectivity according to claim 1, characterized in that: The decoder described in step 6 is composed of a boundary feature enhancement decoding module, and uses a residual connection to fuse edge features; after obtaining the main feature d enhance After that, combined with the corresponding features in the encoder, the size of the feature map is restored by upsampling and channel-by-channel splicing; the calculation formula of the boundary feature enhancement decoding module is as follows: Among them, e b represents the boundary enhancement feature, d i-1 represents the output of the previous layer decoder, σ(·) represents the Sigmoid activation function, F1(·) represents 1×1 convolution, Att edge Represents the obtained boundary attention.

Images