Skin disease classification method based on half-region and half-global attention mechanism CAD

Abstract

The invention discloses a skin disease classification method based on a half-region and half-global attention mechanism CAD. Swin Transform is utilized to realize half-global attention, more long-distance correlation between lesion image features can be mined, global feature capture and enhancement are performed on previous blocks by utilizing jumping connection, and through a sliding window and a residual block of the Swin Transform, the skin disease classification method based on the half-region and half-global attention mechanism CAD is realized. Local and global features can be fused; meanwhile, a residual block is used for realizing half-region attention, so that global and local features are considered at the same time, a set of end-to-end network model parameters are trained, the accuracy of clinical skin disease type classification is effectively improved, and the accuracy, recall rate, precision rate, F1-score and the like are all superior to those in the prior art.

Description

Technical Field

[0001] The method of the present invention relates to a method for identifying skin disease types using a computer, in particular to a skin disease classification method based on a semi-regional and semi-global attention mechanism CAD. Background Art

[0002] In the past, when conducting clinical examinations of skin diseases, most skin disease types were judged by doctors based on comprehensive medical standards (such as the ABC standard, Seven Point checklist, Menzies method, etc.), which not only required rich experience and a large amount of professional knowledge reserves, but the accuracy of the judgment results was often affected by subjective judgment and personal ability. For example, the accuracy of unassisted visual examinations of melanoma by expert dermatologists is about 60%, while for the most dangerous types of skin cancer, primary care clinicians can only achieve an accuracy of 23-46%.

[0003] Using computer-aided diagnosis (CAD) system to extract features and automatically classify skin diseases is a feasible way to improve recognition accuracy. However, due to the small number of samples of dermatological images and the large variety of lesion features of skin diseases and their small proportion in skin disease images, it is difficult to summarize and learn the required lesion features. Therefore, existing image processing and machine algorithms cannot handle the classification problem of skin diseases well. Summary of the invention

[0004] The present invention aims to solve the above-mentioned technical problems existing in the prior art and provides a skin disease classification method based on semi-regional and semi-global attention mechanism CAD.

[0005] The technical solution of the present invention is: a skin disease classification method based on a semi-regional and semi-global attention mechanism CAD, which inputs a clinical skin disease image to be classified into a trained network model to obtain a classification result, and the trained network model is constructed according to the following steps:

[0006] Step 1. Select samples and preprocess them

[0007] The images in the skin disease dataset are divided into a training set and a validation set. The training set images are recorded as Pretrain_Skinimages, and the labels corresponding to the training set images are recorded as Train_Skinlabels.

[0008] Pretrain_Skinimages are preprocessed, i.e., randomly rotated by +20 degrees or -20 degrees, and randomly flipped horizontally, and then resized to 224*224. The preprocessed Pretrain_Skinimages are recorded as Train_Skinimages;

[0009] Step 2. Train the network

[0010] Step 2.1 Input Train_Skinimages into the network and start training the corresponding network model Model. The variable epoch is agreed to be the number of network cycle training, which is initially 0;

[0011] The network model Model consists of three parts. The first part is the patchpartition layer; the second part consists of four stages, which are respectively recorded as Stage1, Stage2, Stage3, and Stage4. Stage1 consists of linear_embedding and two LAAST blocks, Stage2 consists of patch_merging and two LAAST blocks, Stage3 consists of patch_merging and six LAAST blocks, and Stage4 consists of patch_merging and two LAAST blocks. The LAAST block consists of a SwinTransformer block and a residual block. The Swin Transformer block consists of a window attention mechanism or a sliding window attention mechanism and a multi-layer perceptron. The residual block consists of two Conv layers and a Relu activation function. The third part is the MLP layer, which consists of multiple layers of fully connected layers.

[0012] First, Train_Skinimages is segmented and mapped with a patch size of 4 in the first part. The mapped part is project_embedding, and the channel dimension of Train_Skinimages is expanded to 96, while the length and width are reduced to a quarter of the original, that is, 56*56; secondly, feature extraction is performed through the second part, and finally the extracted features are classified through the third part, and finally the skin disease type label predicted by the network model Model is obtained, which is recorded as Output_Skinlabels;

[0013] Step 2.2: Use the cross entropy loss calculation method to obtain the corresponding loss function value of Output_Skinlabels and Train_Skinlabels, recorded as label_loss, and set epoch = epoch + 1; when the epoch reaches the set number of times, the network stops training and saves the training model last_model.pt file; otherwise, the label_loss value is back-propagated, and the model parameters are updated through the ADAMW optimization algorithm, and then go to step 2.1.

[0014] The present invention uses Swin Transformer to achieve semi-global attention, which can mine more long-range correlations between lesion image features, uses skip connections to capture and strengthen global features of previous blocks, and can fuse local and global features through the sliding window and residual block of Swin Transformer; at the same time, residual blocks are used to achieve semi-regional attention, so that the present invention takes into account both global and local features, trains a set of end-to-end network model parameters, and effectively improves the accuracy of clinical skin disease type classification. Its accuracy, recall rate, precision and F1-score are all better than the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a network structure diagram of the overall model of the embodiment of the present invention.

[0016] Figure 2 It is a schematic diagram of LAAST in which a residual block and a Swin Transformer block are integrated according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The present invention provides a method for skin disease type recognition based on a semi-regional and semi-global attention mechanism, such as Figure 1 As shown:

[0018] Step 1. Select samples and preprocess them

[0019] 5267 images in the SD-198 dataset were used as training sets, and 1317 images were used as validation sets; 12751 images in the Fitzpatrck 17k dataset were used as training sets, and 3826 images were used as validation sets. The clinical skin disease images to be trained are recorded as Pretrain_Skinimages; and the labels of the skin disease types of the corresponding clinical images are recorded as Train_Skinlabels;

[0020] Pretrain_Skinimages are preprocessed, i.e., randomly rotated by +20 degrees or -20 degrees, and randomly flipped horizontally, and then resized to 224*224. The preprocessed Pretrain_Skinimages are recorded as Train_Skinimages;

[0021] Step 2. Train the network

[0022] Step 2.1 Input Train_Skinimages into the network and start training the corresponding network model Model. The variable epoch is agreed to be the number of network cycle training, which is initially 0;

[0023] The network model Model consists of three parts. The first part is the patchpartition layer; the second part consists of four stages, which are respectively recorded as Stage1, Stage2, Stage3, and Stage4. Stage1 consists of linear_embedding and two LAAST blocks, Stage2 consists of patch_merging and two LAAST blocks, Stage3 consists of patch_merging and six LAAST blocks, and Stage4 consists of patch_merging and two LAAST blocks. The LAAST block consists of a SwinTransformer block and a residual block. The Swin Transformer block consists of a window attention mechanism or a sliding window attention mechanism and a multi-layer perceptron. The residual block consists of two Conv layers and a Relu activation function. The third part is the MLP layer, which consists of multiple layers of fully connected layers.

[0024] First, Train_Skinimages is segmented and mapped with a patch size of 4 in the first part. The mapped part is project_embedding, and the channel dimension of Train_Skinimages is expanded to 96, while the length and width are reduced to a quarter of the original, that is, 56*56; secondly, feature extraction is performed through the second part, and finally the extracted features are classified through the third part, and finally the skin disease type label predicted by the network model Model is obtained, which is recorded as Output_Skinlabels;

[0025] Step 2.2: Use the cross entropy loss calculation method to obtain the corresponding loss function values ​​of Output_Skinlabels and Train_Skinlabels, recorded as label_loss, and set epoch = epoch + 1; when epoch reaches 100 times, the network stops training and saves the training model last_model.pt file; otherwise, the label_loss value is back-propagated, and the model parameters are updated through the ADAMW optimization algorithm, and then enter step 2.1 again, and the training is repeated in a cycle;

[0026] Step 3. Import the training model last_model.pt file into the test code, input the 1317 test images in the SD-198 dataset or the 3826 test images in Fitzpatrick 17k into the trained network model to obtain the classification results.

[0027] Using the same SD-198 dataset and Fitzpatrick 17k dataset, three networks, Resnet12, Resnet50, and Swin Transformer-base, were trained respectively. The comparison of their accuracy, recall, precision, and F1-score with those of the embodiment of the present invention (LAAST) is shown in Tables 1 and 2.

[0028] Table 1

[0029]

[0030] Table 2

[0031]

[0032] The results show that the accuracy, recall, precision and F1-score of the embodiment of the present invention are better than those of the other three networks.

Claims

1. A skin disease classification method based on a semi-regional and semi-global attention mechanism CAD is to input a clinical skin disease image to be classified into a trained network model to obtain a classification result, wherein the trained network model is constructed according to the following steps: Step 1. Select samples and preprocess them The images in the skin disease dataset are divided into a training set and a validation set. The training set images are recorded as Pretrain_Skinimages, and the labels corresponding to the training set images are recorded as Train_Skinlabels. Pretrain_Skinimages are preprocessed, i.e., randomly rotated by +20 degrees or -20 degrees, and randomly flipped horizontally, and then resized to 224*224. The preprocessed Pretrain_Skinimages are recorded as Train_Skinimages; Step 2. Train the network Step 2.1 Input Train_Skinimages into the network and start training the corresponding network model Model. The variable epoch is agreed to be the number of network cycle training, which is initially 0; The network model Model consists of three parts. The first part is the patchpartition layer; the second part consists of four stages, which are respectively recorded as Stage1, Stage2, Stage3, and Stage4. Stage1 consists of linear_embedding and two LAAST blocks, Stage2 consists of patch_merging and two LAAST blocks, Stage3 consists of patch_merging and six LAAST blocks, and Stage4 consists of patch_merging and two LAAST blocks. The LAAST block consists of a SwinTransformer block and a residual block. The Swin Transformer block consists of a window attention mechanism or a sliding window attention mechanism and a multi-layer perceptron. The residual block consists of two Conv layers and a Relu activation function. The third part is the MLP layer, which consists of multiple layers of fully connected layers. First, Train_Skinimages is segmented and mapped with a patch size of 4 in the first part. The mapped part is project_embedding, and the channel dimension of Train_Skinimages is expanded to 96, while the length and width are reduced to a quarter of the original, that is, 56*56; secondly, feature extraction is performed through the second part, and finally the extracted features are classified through the third part, and finally the skin disease type label predicted by the network model Model is obtained, which is recorded as Output_Skinlabels; Step 2.2: Use the cross entropy loss calculation method to obtain the corresponding loss function value of Output_Skinlabels and Train_Skinlabels, recorded as label_loss, and set epoch = epoch + 1; when the epoch reaches the set number of times, the network stops training and saves the training model last_model.pt file; otherwise, the label_loss value is back-propagated, and the model parameters are updated through the ADAMW optimization algorithm, and then go to step 2.1.

Images