Skin disease image segmentation method based on dense encoder and hybrid attention

Abstract

The present invention discloses a skin disease image segmentation method based on dense encoder and hybrid attention. The specific steps are: first, the width of the original picture of the ISIC2017 data set is used as the side length after cropping, and the picture is automatically cropped according to the center of the picture, and the picture is scaled after cropping. Then, in the encoding part, the dense encoding module is used to extract strong features, and the Gaussian pooling layer is proposed according to the weighted correlation degree of each feature point in the feature map, and an efficient hybrid attention mechanism is designed to extract features in two dimensions of space and channel. Finally, a weighted auxiliary loss function is constructed on both sides of the output layer to calculate the total loss of the network structure and generate the final segmentation map.

Description

Technical Field

[0001] The present invention relates to the fields of computer-aided diagnosis and medical image processing, and specifically to a skin disease image segmentation framework built by using a convolutional neural network, and to an invention of a skin disease image segmentation method based on a dense encoder and hybrid attention. Background Art

[0002] According to the World Health Organization, the number of new cases of malignant melanoma in the world reached 287,723 in 2018, with 60,712 deaths. In medical practice, early detection using dermoscopy technology plays an important role in the detection of melanoma. Doctors can use dermoscopy technology for early detection and make subjective judgments based on medical knowledge and clinical experience, but misdiagnosis may occur. Skin lesion segmentation technology can quantitatively analyze the morphology and structure of lesions, provide data support and basis for medical research, and promote the development of dermatology. While realizing the automated diagnosis of skin lesions, it improves the efficiency and accuracy of medical services and provides better medical security for the public. In practical applications, the robustness of early skin lesion area segmentation methods based on digital image processing needs to be improved, and it is difficult to adapt to dermoscopic images with highly variable samples. The traditional method based on feature engineering design and feature extraction is very cumbersome to design and select appropriate features for specific tasks. The quality of the classifier depends on the quality of the features, and satisfactory segmentation effects cannot be achieved. With the continuous development of neural networks and the increasing depth of networks, the potential of convolutional neural networks in feature extraction has been tapped and has been widely studied and applied in fields such as target recognition, detection, and segmentation. It is no longer necessary to manually design features. Neural networks use supervised learning, with loss functions and gradient descent algorithms driving the network to learn features that are helpful for understanding image tasks. The entire process is end-to-end and relatively concise.

[0003] Reference 1 (M. Jafari, E. Nasr-Esfahani, N. Karimi, et al. Extraction of Skin Lesions from Non-Dermoscopic Images Using Deep Learning [J]. arXiv preprint arXiv: 1609.02374, 2017.) uses a convolutional neural network to segment melanoma lesions, using the global window and local window to determine whether each pixel has a lesion. Reference 2 (Y. Yuan, M. Chao, Yeh-Chi L. Automatic Skin Lesion Segmentation Using Deep Fully Convolutional Networks with Jaccard Distance [J]. IEEE Transactions on Medical Imaging, 2017, 36 (9): 1876-1886.) uses a 19-layer fully convolutional neural network for melanoma segmentation and proposes a novel loss function based on Jaccard distance. Reference 3 (Y. Wang, Simon S, Jahow J, et al. Skin lesion segmentation using atrous convolution via DeepLab v3[J]. arXiv preprint arXiv:.08891, 2018.) uses DeeplabV3 to creatively apply deep separation convolution to the decoder and pyramid pooling modules, and achieves good results in the skin lesion area segmentation task.

[0004] Patent 1 (Jiang Yun, Cao Simin, Tao Shengxin, Wu Chao, Liu Wenhuan. A skin lesion segmentation method based on a convolutional attention model [P]. Chinese Patent: CN113744178A, 2021.12.03) discloses a skin lesion segmentation method based on a convolutional attention model. Patent 2 (Shuai Renjun, Lu Wei. Skin lesion segmentation method based on a multi-attention mechanism model MCDC_Net [P]. Chinese Patent: CN115530766A, 2022.12.30) involves the fields of deep learning and medical image segmentation, and provides a novel deep learning model for solving the problem of skin lesion segmentation. Patent 3 (Dai Duwei, Xu Songhua, Li Zongfang. An image segmentation method and system based on a multi-scale residual encoding and decoding network [P]. Chinese Patent: CN114004811A, 2022.02.01) preprocesses the skin lesion image, and performs data enhancement on the preprocessed image. However, the above methods do not effectively distinguish the lesion area from the background, and the skin lesion dataset has large intra-class distances, small inter-class distances, and discontinuous and blurred boundaries of the lesion area. This patent uses a multi-scale fusion method to construct a dense encoder and designs a hybrid attention module to achieve skin disease image segmentation. Summary of the invention

[0005] The present invention discloses a skin disease segmentation method based on dense encoder and hybrid attention. In the encoding part, the method uses a dense encoding module to perform multi-scale fusion to enhance feature extraction, proposes a Gaussian pooling layer according to the weighted correlation degree of each feature point in the feature map, and designs an efficient hybrid attention mechanism module for extracting features in two dimensions, namely, space and channel. Finally, the auxiliary loss function idea is used, and a weighted cross entropy loss function is used on both sides of the output layer to calculate the total loss of the model to train the network.

[0006] 1. A skin disease segmentation method based on dense encoder and hybrid attention, characterized in that it mainly includes the following steps:

[0007] S1. Data preprocessing: The original image width of the dataset is used as the side length after cropping, and the image is automatically cropped based on the center of the image. The cropped image is scaled to form an image with a size of 256×256 pixels.

[0008] S2, feature extraction: The data obtained in S1 is used as a feature extraction network using the constructed dense encoder and hybrid attention to perform multi-scale feature extraction on the data;

[0009] S3, feature decoding: Use the features extracted in S2 to restore the original image size, update the parameter training model through the auxiliary loss function, and finally get the segmentation result;

[0010] The step S2 comprises the following steps:

[0011] The feature extraction backbone network is constructed by a convolutional neural network. The specific network structure is as follows:

[0012] S21, the encoding part uses dense encoding modules to extract multi-scale features. Each densely connected module consists of a 3×3 convolution kernel, two 3×3 depth-separable convolution kernels, batch normalization, ReLU activation function and void space pyramid pooling;

[0013] S22, densely connected layer and subsequent layer feature fusion, each layer learns new features based on the output of the previous layer, concatenates the feature maps of the first three layers, and the multi-scale fused features continue to be used as the input of the next layer;

[0014] S23, construct a global Gaussian pooling layer, and perform statistical weighting according to the degree of correlation of each feature point in the feature map, where x i represents the i-th one-dimensional eigenvector, N represents the number of one-dimensional eigenvectors, and the significant eigenvalue is calculated as shown in the formula:

[0015]

[0016] The parameter c represents the curvature of the Gaussian function curve. The larger the value of c, the flatter the Gaussian curve, and the smaller the value of c, the steeper the Gaussian curve. The calculation of parameter c is shown in the formula:

[0017]

[0018] Gaussian weight represents the degree of association between each element in the pooling area and the significant feature value b, where parameter a=1, b represents the significant feature value, and c is the curvature of the element in the pooling area. In order to prevent the curvature c from being 0, which leads to the problem of training interruption, ξ is used for adjustment, and the value is 0.1. The calculation formula of Gaussian weight is:

[0019]

[0020] According to the weighted fusion of the Gaussian weight of each element, a one-dimensional feature vector with multiple important element features is obtained. The formula of global Gaussian pooling is:

[0021]

[0022] S24, feature input channel attention mechanism, that is, after global mean pooling, one-dimensional convolution is used to complete the cross-channel information interaction. The size of the convolution kernel is determined by the adaptive function. The layer with a larger number of channels can perform more cross-channel interactions. The adaptive function formula is as follows:

[0023]

[0024] C is the number of channels, where b = 1, γ = 2;

[0025] S25. The input features are subjected to maximum pooling, global mean pooling and global Gaussian pooling in space. The three two-dimensional vectors generated by the pooling layer are concatenated and weighted by sigmoid. The features extracted by cross-channel attention and global spatial attention are fused at multiple scales and then passed through a dense residual connection layer.

[0026] 2. The decoding step S3 comprises the following steps:

[0027] S31, the decoding path converts the low-resolution image into high-resolution through deconvolution with a convolution kernel of 2 and a stride of 2, and then completes the prediction of the skin lesion area through 1×1 convolution and sigmoid;

[0028] S32, construct an auxiliary loss function and use the intermediate output layer and the final output layer to back propagate the output loss to update the training parameters. For the sample (x, y), x = {x i , i=1,……,n} represents training data, y={y i ,i=1,……,n} is the corresponding label true value, where y i ={0,1}, the probability that the i-th sample is predicted to be 1 is recorded as For each output image, the loss is The definition of is as follows:

[0029]

[0030] n represents the total number of samples, m represents the number of output layers, and the corresponding loss weight between output layers is expressed as a m ={0.5, 0.5}, the final total loss is as shown in the following formula:

[0031] BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flowchart of the implementation process of the segmentation method of the present invention;

[0033] Figure 2 It is a structural diagram of a dense encoder module (DSA Block) in the segmentation method of the present invention;

[0034] Figure 3 It is a structural diagram of the efficient hybrid attention (ECDB) module in the segmentation method of the present invention; Specific implementation plan

[0035] In order to understand the technical features, purposes and beneficial effects of the present invention more clearly, an embodiment of the present invention is further described in conjunction with the accompanying drawings. The embodiment is only used to further illustrate the present invention and cannot be understood as limiting the scope of protection of the present invention. Some non-essential improvements and adjustments made by technicians in this field based on the content of the present invention also fall within the scope of protection of the present invention.

[0036] 1. A skin disease segmentation method based on dense encoder and hybrid attention, characterized in that it mainly includes the following steps:

[0037] S1. Data preprocessing: The original image width of the dataset is used as the side length after cropping, and the image is automatically cropped based on the center of the image. The cropped image is scaled to form an image with a size of 256×256 pixels.

[0038] S2, feature extraction: The data obtained in S1 is used as a feature extraction network using the constructed dense encoder and hybrid attention to perform multi-scale feature extraction on the data;

[0039] S3, feature decoding: Use the features extracted in S2 to restore the original image size, update the parameter training model through the auxiliary loss function, and finally get the segmentation result;

[0040] The step S2 comprises the following steps:

[0041] The feature extraction backbone network is constructed by a convolutional neural network. The specific network structure is as follows:

[0042] S21, the encoding part uses dense encoding modules to extract multi-scale features. Each densely connected module consists of a 3×3 convolution kernel, two 3×3 depth-separable convolution kernels, batch normalization, ReLU activation function and void space pyramid pooling;

[0043] S22, densely connected layer and subsequent layer feature fusion, each layer learns new features based on the output of the previous layer, concatenates the feature maps of the first three layers, and the multi-scale fused features continue to be used as the input of the next layer;

[0044] S23, construct a global Gaussian pooling layer, and perform statistical weighting according to the degree of correlation of each feature point in the feature map, where x i represents the i-th one-dimensional eigenvector, N represents the number of one-dimensional eigenvectors, and the significant eigenvalue is calculated as shown in the formula:

[0045]

[0046] The parameter c represents the curvature of the Gaussian function curve. The larger the value of c, the flatter the Gaussian curve, and the smaller the value of c, the steeper the Gaussian curve. The calculation of parameter c is shown in the formula:

[0047]

[0048] Gaussian weight represents the degree of association between each element in the pooling area and the significant feature value b, where parameter a=1, b represents the significant feature value, and c is the curvature of the element in the pooling area. In order to prevent the curvature c from being 0, which leads to the problem of training interruption, ξ is used for adjustment, and the value is 0.1. The calculation formula of Gaussian weight is:

[0049]

[0050] According to the weighted fusion of the Gaussian weight of each element, a one-dimensional feature vector with multiple important element features is obtained. The formula of global Gaussian pooling is:

[0051]

[0052] S24, feature input channel attention mechanism, that is, after global mean pooling, one-dimensional convolution is used to complete the cross-channel information interaction. The size of the convolution kernel is determined by the adaptive function. The layer with a larger number of channels can perform more cross-channel interactions. The adaptive function formula is as follows:

[0053]

[0054] C is the number of channels, where b = 1, γ = 2;

[0055] S25. The input features are subjected to maximum pooling, global mean pooling and global Gaussian pooling in space. The three two-dimensional vectors generated by the pooling layer are concatenated and weighted by sigmoid. The features extracted by cross-channel attention and global spatial attention are fused at multiple scales and then passed through a dense residual connection layer.

[0056] 2. The decoding step S3 comprises the following steps:

[0057] S31, the decoding path converts the low-resolution image into high-resolution through deconvolution with a convolution kernel of 2 and a stride of 2, and then completes the prediction of the skin lesion area through 1×1 convolution and sigmoid;

[0058] S32, construct an auxiliary loss function and use the intermediate output layer and the final output layer to back propagate the output loss to update the training parameters. For the sample (x, y), x = {x i , i=1,……,n} represents training data, y={y i ,i=1,……,n} is the corresponding label true value, where y i ={0,1}, the probability that the i-th sample is predicted to be 1 is recorded as For each output image, the loss is The definition of is as follows:

[0059]

[0060] n represents the total number of samples, m represents the number of output layers, and the corresponding loss weight between output layers is expressed as a m ={0.5, 0.5}, the final total loss is as shown in the following formula:

[0061]

[0062] Simulation experiment

[0063] In order to quantitatively evaluate the segmentation ability of the network constructed by the present invention, the following evaluation indicators are used: similarity coefficient (Dice), foreground intersection over union (IoU), accuracy (Accuracy) and specificity (Specificity). The segmentation network of the present invention achieved 96.45%, 86.61%, 93.16% and 97.82% in Accuracy, IoU, Dice and Specificity indicators respectively, and the segmentation effect is good.

[0064] The above is a description of the method of the present invention, and those skilled in the art can implement the method of the present invention based on the description of the content. Based on the above content of the present invention, other embodiments obtained by those skilled in the art without creative work should all fall within the scope of protection of the present invention.

Claims

1. A skin disease segmentation method based on dense encoder and hybrid attention, characterized in that The main steps include: S1. Data preprocessing: The original image width of the dataset is used as the side length after cropping, and the image is automatically cropped based on the center of the image. The cropped image is scaled to form an image with a size of 256×256 pixels. S2, feature extraction: The data obtained in S1 is used as a feature extraction network using the constructed dense encoder and hybrid attention to perform multi-scale feature extraction on the data; S3, feature decoding: Use the features extracted in S2 to restore the original image size, update the parameter training model through the auxiliary loss function, and finally get the segmentation result; The step S2 comprises the following steps: The feature extraction backbone network is constructed by a convolutional neural network. The specific network structure is as follows: S21, the encoding part uses dense encoding modules to extract multi-scale features. Each densely connected module consists of a 3×3 convolution kernel, two 3×3 depth-separable convolution kernels, batch normalization, ReLU activation function and void space pyramid pooling; S22, densely connected layer and subsequent layer feature fusion, each layer learns new features based on the output of the previous layer, concatenates the feature maps of the first three layers, and the multi-scale fused features continue to be used as the input of the next layer; S23, construct a global Gaussian pooling layer, and perform statistical weighting according to the degree of correlation of each feature point in the feature map, where x i represents the i-th one-dimensional eigenvector, N represents the number of one-dimensional eigenvectors, and the significant eigenvalue is calculated as shown in the formula: The parameter c represents the curvature of the Gaussian function curve. The larger the value of c, the flatter the Gaussian curve, and the smaller the value of c, the steeper the Gaussian curve. The calculation of parameter c is shown in the formula: Gaussian weight represents the degree of association between each element in the pooling area and the significant feature value b, where parameter a=1, b represents the significant feature value, and c is the curvature of the element in the pooling area. In order to prevent the curvature c from being 0, which leads to the problem of training interruption, ξ is used for adjustment, and the value is 0.

1. The calculation formula of Gaussian weight is: According to the weighted fusion of the Gaussian weight of each element, a one-dimensional feature vector with multiple important element features is obtained. The formula of global Gaussian pooling is: S24, feature input channel attention mechanism, that is, after global mean pooling, one-dimensional convolution is used to complete the cross-channel information interaction. The size of the convolution kernel is determined by the adaptive function. The layer with a larger number of channels can perform more cross-channel interactions. The adaptive function formula is as follows: C is the number of channels, where b = 1, γ = 2; S25. The input features are subjected to maximum pooling, global mean pooling and global Gaussian pooling in space. The three two-dimensional vectors generated by the pooling layer are concatenated and weighted by sigmoid. The features extracted by cross-channel attention and global spatial attention are fused at multiple scales and then passed through a dense residual connection layer.

2. The skin disease image segmentation method based on dense encoder and hybrid attention according to claim 1 is characterized in that: The decoding step S3 comprises the following steps: S31, the decoding path converts the low-resolution image into high-resolution through deconvolution with a convolution kernel of 2 and a stride of 2, and then completes the prediction of the skin lesion area through 1×1 convolution and sigmoid; S32, construct an auxiliary loss function and use the intermediate output layer and the final output layer to back propagate the output loss to update the training parameters. For the sample (x, y), x = {x i , i=1,……,n} represents training data, y={y i ,i=1,……,n} is the corresponding label true value, where y i ={0,1}, the probability that the i-th sample is predicted to be 1 is recorded as For each output image, the loss is The definition of is as follows: n represents the total number of samples, m represents the number of output layers, and the corresponding loss weight between output layers is expressed as a m ={0.5, 0.5}, the final total loss is as shown in the following formula:

Images