A skin disease image segmentation method based on multi-scale split attention mechanism

Abstract

A method for skin disease image segmentation using a multi-scale split attention mechanism comprises the following steps: Step 1: Reading a skin disease image dataset, segmenting it, and preprocessing it; Step 2: Constructing a multi-scale split attention mechanism network; Step 3: Feeding the training set and validation set of the skin disease images from Step 1 into the network constructed in Step 2 for training; Step 4: Feeding the test set segmented in Step 1 into the trained network from Step 3 for prediction; Step 5: Outputting the final predicted image. The present invention aims to address the technical issues of uneven image lesion size and insufficient contrast in existing skin disease image segmentation technologies by providing a method for skin disease image segmentation using a multi-scale split attention mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning image segmentation, and in particular to a network combining a multi-scale split attention mechanism module for skin disease image segmentation. Background Art

[0002] Skin cancer is the most common malignancy in humans, and melanoma is one of the most lethal skin lesions. Early-stage melanoma can be removed with simple surgical procedures, with a cure rate of up to 95%. However, the mortality rate for advanced melanoma exceeds 85%, highlighting the critical importance of early diagnosis and treatment of skin diseases. Currently, dermatologists rely primarily on visual inspection and dermoscopy to identify the area of ​​skin lesions. However, due to external interference factors such as hair, blood vessels, and scars, even experienced physicians struggle to accurately segment skin lesions.

[0003] Therefore, there is an urgent need for methods that can automatically identify and segment skin lesions to assist medical professionals in diagnosing skin diseases. Image segmentation methods based on fully convolutional networks (FCNs) in deep learning are gradually replacing traditional algorithms and becoming the mainstream. Unified Networks (UNETs) are commonly used for medical image segmentation. The UNET architecture consists of an encoder-decoder structure: the encoder downsamples to extract features, the decoder restores the image to its original size, and skip connections fuse shallow and deep features to produce the final segmentation result. This U-shaped network reduces computational complexity by continuously reducing resolution through pooling in the encoder, while integrating multi-scale information for more accurate segmentation. This makes the UNET perform well in tasks such as lung nodule and liver segmentation. Attention UNET builds on the UNET by introducing an attention mechanism, adding spatial attention to the skip connections to enhance the focus of lesions. Res UNet and Dense UNet both improve upon UNET submodules, replacing them with residual and densely connected modules, respectively, to achieve a larger receptive field and more accurate segmentation results. CEnet adds a multi-scale module to the bottom layer of U-net, using different pooling kernels and dilated convolutions with different coefficients to capture information at multiple scales. DoubleU-Net uses a dual U-net architecture and adds an ASPP module to the bottom layer of each U-net. This structure enables the network to capture more semantic information. Summary of the Invention

[0004] The purpose of the present invention is to solve the technical problems of existing skin disease image segmentation technology, such as the uneven size of image lesion areas and insufficient contrast, and to provide a skin disease image segmentation method with a multi-scale split attention mechanism.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A skin disease image segmentation method based on a multi-scale split attention mechanism includes the following steps:

[0007] Step 1: Read the skin disease image dataset and perform segmentation and preprocessing;

[0008] Step 2: Construct a multi-scale split attention mechanism network;

[0009] Step 3: Send the training set and validation set of skin disease images in step 1 to the network constructed in step 2 for training;

[0010] Step 4: Send the test set divided in step 1 to the network trained in step 3 for prediction;

[0011] Step 5: Output the final predicted image.

[0012] In step 1, the skin disease image is preprocessed, specifically including the following steps:

[0013] Step 1-1) Divide the skin disease dataset into a training set, a validation set, and a test set;

[0014] Step 1-2) Normalize the divided data set, that is, calculate the image by channel, first calculate its variance and mean, then subtract the mean from each data in each channel and divide it by the variance to obtain the normalized result.

[0015] The constructed multi-scale split attention mechanism network is:

[0016] Coding layer split attention mechanism module first layer → Coding layer split attention mechanism module second layer → Coding layer split attention mechanism module third layer → Coding layer split attention mechanism module fourth layer;

[0017] The third layer of the encoding layer split attention mechanism module → the third layer residual hybrid attention mechanism module;

[0018] The third layer of the encoding layer split attention mechanism module, the third layer residual hybrid attention mechanism module → first splicing → the first layer of the decoding layer split attention mechanism module;

[0019] The second layer of the encoding layer split attention mechanism module → the second layer residual hybrid attention mechanism module;

[0020] The second layer of the encoding layer split attention mechanism module, the second layer of the residual hybrid attention mechanism module → second splicing → the second layer of the decoding layer split attention mechanism module;

[0021] The first layer of the encoding layer split attention mechanism module → the first layer of the residual hybrid attention mechanism module;

[0022] The first layer of the encoding layer split attention mechanism module, the first layer of the residual mixed attention mechanism module → second splicing → the third layer of the decoding layer split attention mechanism module → 1x1 convolution classification.

[0023] The first layer of the encoding layer split attention mechanism module, the second layer of the encoding layer split attention mechanism module, the third layer of the encoding layer split attention mechanism module, the fourth layer of the encoding layer split attention mechanism module, and the first layer of the decoding layer split attention mechanism module, the second layer of the decoding layer split attention mechanism module, and the third layer of the decoding layer split attention mechanism module all contain two multi-scale combined split attention mechanism modules. Each multi-scale combined split attention mechanism module divides the input image into four branches according to the channel, that is, four different feature maps, and then extracts multi-scale information through convolution with different void coefficients, and then adds an attention mechanism to the feature map obtained from each branch. The structure of each multi-scale combined split attention mechanism module is:

[0024] Split by channel → Features after splitting Figure 1 , features after splitting Figure 2 , features after splitting Figure 3 , features after splitting Figure 4 ;

[0025] Features after splitting Figure 1 →The first multi-scale feature extraction; features after splitting Figure 2 →Second multi-scale feature extraction;

[0026] Features after splitting Figure 3 →The third multi-scale feature extraction; features after splitting Figure 4 →The fourth multi-scale feature extraction;

[0027] The first multi-scale feature extraction, the second multi-scale feature extraction → the first superposition;

[0028] First superposition, third multi-scale feature extraction → second superposition;

[0029] Second superposition, fourth multi-scale feature extraction → third superposition;

[0030] The third superposition → global average pooling → generate attention weights → r-softmax layer → first attention ratio feature map, second attention ratio feature map, third attention ratio feature map, fourth attention ratio feature map;

[0031] Features after splitting Figure 1 , the first attention weight feature map → first multiplication;

[0032] Features after splitting Figure 2 , the second attention weight feature map → second multiplication;

[0033] Features after splitting Figure 3 , the third attention weight feature map → the third multiplication;

[0034] Features after splitting Figure 4 , the fourth attention weight feature map → the fourth multiplication;

[0035] First multiplication, second multiplication, third multiplication, fourth multiplication → fourth superposition → 1x1 convolution → residual connection.

[0036] In the multi-scale combined split attention mechanism module, extracting multi-scale feature information and adding the attention mechanism specifically include the following steps:

[0037] Step 1: Split the input feature map into 4 different feature maps by channel;

[0038] Step 2: Perform multi-scale feature extraction on the results of step 1;

[0039] Step 3: Perform feature aggregation on the results of step 2;

[0040] Step 4: Apply attention mechanism to the result of step 3;

[0041] In step 3, after feature extraction, the output of each branch is aggregated, and the formula is as follows:

[0042]

[0043] Among them, M i is the dilated convolution operation of the i-th branch, and the output F∈R H×W×C ;

[0044] In step 4, the attention mechanism includes the following steps:

[0045] Step 4.1: Perform global average pooling on the result of step 3 so that the feature size is compressed to 1x1xC. The formula is as follows:

[0046]

[0047] F(i,j) represents the value of the i-th and j-th pixel points of the feature map, i∈H, j∈W, H is the length of the feature map, W is the width of the feature map, S∈R C , C is the number of channels;

[0048] Step 4.2: Pass the result of step 4.1 through two fully connected layers to obtain the attention weight. Use the r-softmax activation function to convert the weight into a 0-1 ratio, multiply it by the output of each branch and add the attention weight, and then add them together. The formula is as follows:

[0049]

[0050] The value of i is 1, 2, 3, 4, F1, F2, F3, F4 are the outputs of each branch, and a i The formula is as follows:

[0051]

[0052] in is the split weight of the Cth channel in the i-th branch determined by S, and the value of i is 1, 2, 3, 4, is the split weight of the Cth channel in the jth branch determined by S.

[0053] The residual mixed attention mechanism module is used to add a mixed attention mechanism to the feature map, which specifically includes the following steps:

[0054] Step 1: Assign channel attention weights;

[0055] Step 2: Assign spatial attention weights;

[0056] In step 1, the channel attention weight is assigned by performing global maximum pooling and global average pooling operations on the input features to obtain two feature maps with a shape of 1x1xC. Then, the attention weight is obtained by two 1x1 convolutions. The obtained weights are added together and activated by the sigmoid function to obtain the final channel attention feature. The formula is as follows:

[0057] F ′ =M C (F)×F (5)

[0058] Among them, F is the input feature map, F∈C×H×W,M C It is the channel attention mechanism, and its formula is as follows:

[0059]

[0060] Among them, W0 is the weight matrix of the first fully connected layer, W0∈R C / r×C , W1 is the weight matrix of the second fully connected layer, W1∈R C×C / r ,σ is the sigmoid function, are the global average pooling features and global maximum pooling features in the channel attention mechanism, and r is the reduction rate;

[0061] In step 2, the results of step 1 are subjected to global maximum pooling and global average pooling in the channel dimension to obtain two HxWx1 feature maps. After splicing these two feature maps, they are reduced in dimension through a 7x7 convolution kernel and then passed through the sigmoid function to obtain the spatial attention feature. The final result is added to the input F. The formula is as follows:

[0062] F″=M S (F′)×F′+F (7)

[0063] Where F′ is the result of adding channel attention weight, M S It is the spatial attention mechanism, and its formula is as follows:

[0064]

[0065] Where σ is the sigmoid function, f 7×7 It is a 7x7 convolution; They are the global average pooling features and global maximum pooling features in the spatial attention mechanism respectively;

[0066] The maximum pooling is used for downsampling in the encoding layer, and bilinear interpolation is used for upsampling in the decoding layer.

[0067] The multi-scale split attention mechanism network works by taking the following steps:

[0068] Step 1: Pass the input feature map through the encoder part of the network for feature extraction;

[0069] Step 2: Pass the output of each encoder layer in step 1 through the residual mixed attention mechanism;

[0070] Step 3: Feed the output of steps 1 and 2 into the decoder part of the network;

[0071] In step 1, the feature extraction of the encoder part specifically includes the following steps:

[0072] Step 1.1: Split the input through the encoding layer to the first layer of the attention mechanism module;

[0073] Step 1.2: Downsample the result of step 1.1 and split the second layer of the attention mechanism module through the encoding layer;

[0074] Step 1.3: Downsample the result of step 1.2 and split the third layer of the attention mechanism module through the encoding layer;

[0075] Step 1.4: Downsample the result of step 1.3 and split the fourth layer of the attention mechanism module through the encoding layer;

[0076] In step 2, the output of each encoder layer is passed through the residual mixed attention mechanism, which specifically includes the following steps:

[0077] Step 2.1: Pass the result of step 1.1 through the first layer of residual hybrid attention mechanism module;

[0078] Step 2.2: Pass the result of step 1.2 through the second layer of residual hybrid attention mechanism module;

[0079] Step 2.3: Pass the result of step 1.3 through the third layer of residual hybrid attention mechanism module;

[0080] In step 3, the decoder part specifically includes the following steps:

[0081] Step 3.1: Upsample the result of step 1.4 and concatenate it with the result of step 2.3, then split the first layer of the attention mechanism module through the decoding layer;

[0082] Step 3.2: Upsample the result of step 3.1 and concatenate it with the result of step 2.2, then split the second layer of the attention mechanism module through the decoding layer;

[0083] Step 3.2: Upsample the result of step 3.2 and concatenate it with the result of step 2.3, then split the third layer of the attention mechanism module through the decoding layer.

[0084] A multi-scale split attention mechanism network, where the first layer of the coding layer split attention mechanism module → the second layer of the coding layer split attention mechanism module → the third layer of the coding layer split attention mechanism module → the fourth layer of the coding layer split attention mechanism module;

[0085] The third layer of the encoding layer split attention mechanism module → the third layer residual hybrid attention mechanism module;

[0086] The third layer of the encoding layer split attention mechanism module, the third layer residual hybrid attention mechanism module → first splicing (6) → the first layer of the decoding layer split attention mechanism module;

[0087] The second layer of the encoding layer split attention mechanism module → the second layer residual hybrid attention mechanism module;

[0088] The second layer of the encoding layer split attention mechanism module, the second layer of the residual hybrid attention mechanism module → second splicing → the second layer of the decoding layer split attention mechanism module;

[0089] The first layer of the encoding layer split attention mechanism module → the first layer of the residual hybrid attention mechanism module;

[0090] The first layer of the encoding layer split attention mechanism module, the first layer of the residual mixed attention mechanism module → second splicing → the third layer of the decoding layer split attention mechanism module → 1x1 convolution classification.

[0091] The first layer of the encoding layer split attention mechanism module, the second layer of the encoding layer split attention mechanism module, the third layer of the encoding layer split attention mechanism module, the fourth layer of the encoding layer split attention mechanism module, and the first layer of the decoding layer split attention mechanism module, the second layer of the decoding layer split attention mechanism module, and the third layer of the decoding layer split attention mechanism module all contain two multi-scale combined split attention mechanism modules. Each multi-scale combined split attention mechanism module divides the input image into four branches according to the channel, that is, four different feature maps, and then extracts multi-scale information through convolution with different void coefficients, and then adds an attention mechanism to the feature map obtained from each branch. The structure of each multi-scale combined split attention mechanism module is:

[0092] Split by channel → Features after splitting Figure 1 , features after splitting Figure 2 , features after splitting Figure 3 , features after splitting Figure 4 ;

[0093] Features after splitting Figure 1 →The first multi-scale feature extraction; features after splitting Figure 2 →Second multi-scale feature extraction;

[0094] Features after splitting Figure 3 →The third multi-scale feature extraction; features after splitting Figure 4 →The fourth multi-scale feature extraction;

[0095] The first multi-scale feature extraction, the second multi-scale feature extraction → the first superposition;

[0096] First superposition, third multi-scale feature extraction → second superposition;

[0097] Second superposition, fourth multi-scale feature extraction → third superposition;

[0098] The third superposition → global average pooling → generate attention weights → r-softmax layer → first attention ratio feature map, second attention ratio feature map, third attention ratio feature map, fourth attention ratio feature map;

[0099] Features after splitting Figure 1 , the first attention weight feature map → first multiplication;

[0100] Features after splitting Figure 2 , the second attention weight feature map → second multiplication;

[0101] Features after splitting Figure 3 , the third attention weight feature map → the third multiplication;

[0102] Features after splitting Figure 4 , the fourth attention weight feature map → the fourth multiplication;

[0103] First multiplication, second multiplication, third multiplication, fourth multiplication → fourth superposition → 1x1 convolution → residual connection.

[0104] Compared with the prior art, the present invention has the following technical effects:

[0105] 1) The split attention mechanism in this invention uses the learned global information to selectively increase the attention of useful features and suppress useless features, which can effectively solve the problem of insufficient contrast between the target area and the background area in dermoscopic image segmentation;

[0106] 2) Before assigning split attention, multi-scale information is extracted through dilated convolutions with different coefficients, so that each branch obtains a receptive field of different sizes. This multi-scale structure can effectively solve the problem of different sizes of lesion areas in dermatoscope images;

[0107] 3) A residual mixed attention mechanism is used in the jump connection to make the information of the two feature maps being spliced ​​closer, reduce information redundancy, and make the segmentation result more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] The present invention will be further described below with reference to the accompanying drawings and examples:

[0109] Figure 1 Schematic diagram of the structure of the MSCSA module in the present invention;

[0110] Figure 2 This is a schematic diagram of the overall network structure of the present invention;

[0111] Figure 3 This is a workflow diagram of the multi-scale split attention mechanism network in the present invention;

[0112] Figure 4 is the segmentation result map;

[0113] Figure 5 This is the segmentation result of the image with insufficient contrast. DETAILED DESCRIPTION

[0114] A skin disease image segmentation method based on a multi-scale split attention mechanism includes the following steps:

[0115] Step 1: Read the skin disease image dataset and perform segmentation and preprocessing;

[0116] Step 2: Construct a multi-scale split attention mechanism network;

[0117] Step 3: Send the training set and validation set of skin disease images in step 1 to the network constructed in step 2 for training;

[0118] Step 4: Send the test set divided in step 1 to the network trained in step 3 for prediction;

[0119] Step 5: Output the final predicted image.

[0120] In step 1, the skin disease image is preprocessed, specifically including the following steps:

[0121] Step 1-1) Divide the skin disease dataset into a training set, a validation set, and a test set;

[0122] Step 1-2) Normalize the divided data set, that is, calculate the image by channel, first calculate its variance and mean, then subtract the mean from each data in each channel and divide it by the variance to obtain the normalized result.

[0123] In step 2, the constructed multi-scale split attention mechanism network is:

[0124] Coding layer split attention mechanism module first layer 1 → coding layer split attention mechanism module second layer 2 → coding layer split attention mechanism module third layer 3 → coding layer split attention mechanism module fourth layer 4;

[0125] The third layer of the encoding layer split attention mechanism module 3 → the third layer residual hybrid attention mechanism module 5;

[0126] The third layer of the encoding layer split attention mechanism module 3, the third layer residual hybrid attention mechanism module 5 → first splicing 6 → the first layer of the decoding layer split attention mechanism module 7;

[0127] The second layer of the encoding layer split attention mechanism module 2 → the second layer residual hybrid attention mechanism module 8;

[0128] The second layer 2 of the encoding layer split attention mechanism module, the second layer residual hybrid attention mechanism module 8 → second splicing 9 → the second layer 10 of the decoding layer split attention mechanism module;

[0129] The first layer of the encoding layer split attention mechanism module 1 → the first layer residual hybrid attention mechanism module 11;

[0130] The first layer 1 of the encoding layer split attention mechanism module, the first layer residual mixed attention mechanism module 11 → second splicing 12 → the third layer of the decoding layer split attention mechanism module 13 → 1x1 convolution classification 14.

[0131] The first layer 1 of the encoding layer split attention mechanism module, the second layer 2 of the encoding layer split attention mechanism module, the third layer 3 of the encoding layer split attention mechanism module, the fourth layer 4 of the encoding layer split attention mechanism module, and the first layer 7 of the decoding layer split attention mechanism module, the second layer 10 of the decoding layer split attention mechanism module, and the third layer 13 of the decoding layer split attention mechanism module all contain two multi-scale combined split attention mechanism modules. Each multi-scale combined split attention mechanism module divides the input image into four branches according to the channel, that is, four different feature maps, and then extracts multi-scale information through convolution with different void coefficients, and then adds an attention mechanism to the feature map obtained by each branch. The structure of each multi-scale combined split attention mechanism module is:

[0132] Split by channel 15 → Feature map 1 after splitting 16, Feature map 2 after splitting 17, Feature map 3 after splitting 18, Feature map 4 after splitting 19;

[0133] The first feature map after splitting 16 → the first multi-scale feature extraction 20; the second feature map after splitting 17 → the second multi-scale feature extraction 21;

[0134] The third feature map after splitting 18 → the third multi-scale feature extraction 22; the fourth feature map after splitting 19 → the fourth multi-scale feature extraction 23;

[0135] First multi-scale feature extraction 20, second multi-scale feature extraction 21 → first superposition 24;

[0136] First stacking 24, third multi-scale feature extraction 22 → second stacking 25;

[0137] Second stacking 25, fourth multi-scale feature extraction 23 → third stacking 26;

[0138] The third stacking 26 → global average pooling 27 → generate attention weights 28 → r-softmax layer 29 → first attention ratio feature map 30, second attention ratio feature map 31, third attention ratio feature map 32, fourth attention ratio feature map 33;

[0139] The split feature map is 16, the first attention weight feature map is 30 → the first multiplication is 34;

[0140] The split feature map 2 is 17, the second attention ratio feature map is 31 → the second multiplication is 35;

[0141] The split feature map 3 is 18, the third attention ratio feature map is 32 → the third multiplication is 36;

[0142] The fourth feature map after splitting is 19, the fourth attention ratio feature map is 33 → the fourth multiplication is 37;

[0143] The first multiplication 34, the second multiplication 35, the third multiplication 36, the fourth multiplication 37 → the fourth superposition 38 → 1x1 convolution 39 → residual connection 40.

[0144] In the multi-scale combined split attention mechanism module, extracting multi-scale feature information and adding the attention mechanism specifically include the following steps:

[0145] Step 1: Split the input feature map into 4 different feature maps by channel;

[0146] Step 2: Perform multi-scale feature extraction on the results of step 1;

[0147] Step 3: Perform feature aggregation on the results of step 2;

[0148] Step 4: Apply attention mechanism to the result of step 3;

[0149] In step 2, the multi-scale feature extraction is achieved by using three 3x3 asymmetric convolutions in succession, and the dilation coefficient of each asymmetric convolution is different. The settings are shown in the following table:

[0150]

[0151] Table 1: Cavity coefficient setting table

[0152] In step 3, after feature extraction, the output of each branch is aggregated, and the formula is as follows:

[0153]

[0154] Among them, M i is the dilated convolution operation of the i-th branch, and the output F∈R H×W×C ;

[0155] In step 4, the attention mechanism includes the following steps:

[0156] Step 4.1: Perform global average pooling on the result of step 3 so that the feature size is compressed to 1x1xC. The formula is as follows:

[0157]

[0158] F(i,j) represents the value of the i-th and j-th pixel points of the feature map, i∈H, j∈W, H is the length of the feature map, W is the width of the feature map, S∈R C , C is the number of channels;

[0159] Step 4.2: Pass the result of step 4.1 through two fully connected layers to obtain the attention weight. Use the r-softmax activation function to convert the weight into a 0-1 ratio, multiply it by the output of each branch and add the attention weight, and then add them together. The formula is as follows:

[0160]

[0161] The value of i is 1, 2, 3, 4, F1, F2, F3, F4 are the outputs of each branch, and a i The formula is as follows:

[0162]

[0163] in is the split weight of the Cth channel in the i-th branch determined by S, and the value of i is 1, 2, 3, 4, is the split weight of the Cth channel in the jth branch determined by S;

[0164] The residual mixed attention mechanism module is used to add a mixed attention mechanism to the feature map, which specifically includes the following steps:

[0165] Step 1: Assign channel attention weights;

[0166] Step 2: Assign spatial attention weights;

[0167] In step 1, the channel attention weights are assigned by performing global maximum pooling and global average pooling on the input features to obtain two feature maps of shape 1x1xC. The attention weights are then obtained by performing two 1x1 convolutions. The obtained weights are added together and activated by the sigmoid function to obtain the final channel attention feature. The formula is as follows:

[0168] F′=M C (F)×F (5)

[0169] Among them, F is the input feature map, F∈C×H×W,M C It is the channel attention mechanism, and its formula is as follows:

[0170]

[0171] Among them, W0 is the weight matrix of the first fully connected layer, W0∈R C / r×C , W1 is the weight matrix of the second fully connected layer, W1∈RC×C / r ,σ is the sigmoid function, are the global average pooling features and global maximum pooling features in the channel attention mechanism, and r is the reduction rate;

[0172] In step 2, the results of step 1 are subjected to global maximum pooling and global average pooling in the channel dimension to obtain two HxWx1 feature maps. After splicing these two feature maps, they are reduced in dimension through a 7x7 convolution kernel and then passed through the sigmoid function to obtain the spatial attention feature. The final result is added to the input F. The formula is as follows:

[0173] F″=M S (F′)×F′+F (7)

[0174] Where F′ is the result of adding channel attention weight, M S It is the spatial attention mechanism, and its formula is as follows:

[0175]

[0176] Where σ is the sigmoid function, f 7×7 is a 7x7 convolution, They are the global average pooling features and global maximum pooling features in the spatial attention mechanism respectively;

[0177] The maximum pooling is used for downsampling in the encoding layer, and bilinear interpolation is used for upsampling in the decoding layer.

[0178] The multi-scale split attention mechanism network works by taking the following steps:

[0179] Step 1: Pass the input feature map through the encoder part of the network for feature extraction;

[0180] Step 2: Pass the output of each encoder layer in step 1 through the residual mixed attention mechanism;

[0181] Step 3: Feed the output of steps 1 and 2 into the decoder part of the network;

[0182] In step 1, the feature extraction of the encoder part specifically includes the following steps:

[0183] Step 1.1: Split the input through the encoding layer to the first layer of the attention mechanism module;

[0184] Step 1.2: Downsample the result of step 1.1 and split the second layer of the attention mechanism module through the encoding layer;

[0185] Step 1.3: Downsample the result of step 1.2 and split the third layer of the attention mechanism module through the encoding layer;

[0186] Step 1.4: Downsample the result of step 1.3 and split the fourth layer of the attention mechanism module through the encoding layer;

[0187] In step 2, the output of each encoder layer is passed through the residual mixed attention mechanism, which specifically includes the following steps:

[0188] Step 2.1: Pass the result of step 1.1 through the first layer of residual hybrid attention mechanism module;

[0189] Step 2.2: Pass the result of step 1.2 through the second layer of residual hybrid attention mechanism module;

[0190] Step 2.3: Pass the result of step 1.3 through the third layer of residual hybrid attention mechanism module;

[0191] In step 3, the decoder part specifically includes the following steps:

[0192] Step 3.1: Upsample the result of step 1.4 and concatenate it with the result of step 2.3, then split the first layer of the attention mechanism module through the decoding layer;

[0193] Step 3.2: Upsample the result of step 3.1 and concatenate it with the result of step 2.2, then split the second layer of the attention mechanism module through the decoding layer;

[0194] Step 3.2: Upsample the result of step 3.2 and concatenate it with the result of step 2.3, then split the third layer of the attention mechanism module through the decoding layer.

[0195] A multi-scale split attention mechanism network in the present invention has the following structure: the first layer 1 of the coding layer split attention mechanism module → the second layer 2 of the coding layer split attention mechanism module → the third layer 3 of the coding layer split attention mechanism module → the fourth layer 4 of the coding layer split attention mechanism module;

[0196] The third layer of the encoding layer split attention mechanism module 3 → the third layer residual hybrid attention mechanism module 5;

[0197] The third layer of the encoding layer split attention mechanism module 3, the third layer residual hybrid attention mechanism module 5 → first splicing 6 → the first layer of the decoding layer split attention mechanism module 7;

[0198] The second layer of the encoding layer split attention mechanism module 2 → the second layer residual hybrid attention mechanism module 8;

[0199] The second layer 2 of the encoding layer split attention mechanism module, the second layer residual hybrid attention mechanism module 8 → second splicing 9 → the second layer 10 of the decoding layer split attention mechanism module;

[0200] The first layer of the encoding layer split attention mechanism module 1 → the first layer residual hybrid attention mechanism module 11;

[0201] The first layer 1 of the encoding layer split attention mechanism module, the first layer residual mixed attention mechanism module 11 → second splicing 12 → the third layer of the decoding layer split attention mechanism module 13 → 1x1 convolution classification 14.

[0202] The first layer 1 of the encoding layer split attention mechanism module, the second layer 2 of the encoding layer split attention mechanism module, the third layer 3 of the encoding layer split attention mechanism module, the fourth layer 4 of the encoding layer split attention mechanism module, and the first layer 7 of the decoding layer split attention mechanism module, the second layer 10 of the decoding layer split attention mechanism module, and the third layer 13 of the decoding layer split attention mechanism module all contain two multi-scale combined split attention mechanism modules. Each multi-scale combined split attention mechanism module divides the input image into four branches according to the channel, that is, four different feature maps, and then extracts multi-scale information through convolution with different void coefficients, and then adds an attention mechanism to the feature map obtained by each branch. The structure of each multi-scale combined split attention mechanism module is:

[0203] Split by channel 15 → Feature map 1 after splitting 16, Feature map 2 after splitting 17, Feature map 3 after splitting 18, Feature map 4 after splitting 19;

[0204] The first feature map after splitting 16 → the first multi-scale feature extraction 20; the second feature map after splitting 17 → the second multi-scale feature extraction 21;

[0205] The third feature map after splitting 18 → the third multi-scale feature extraction 22; the fourth feature map after splitting 19 → the fourth multi-scale feature extraction 23;

[0206] First multi-scale feature extraction 20, second multi-scale feature extraction 21 → first superposition 24;

[0207] First stacking 24, third multi-scale feature extraction 22 → second stacking 25;

[0208] Second stacking 25, fourth multi-scale feature extraction 23 → third stacking 26;

[0209] The third stacking 26 → global average pooling 27 → generate attention weights 28 → r-softmax layer 29 → first attention ratio feature map 30, second attention ratio feature map 31, third attention ratio feature map 32, fourth attention ratio feature map 33;

[0210] The split feature map is 16, the first attention weight feature map is 30 → the first multiplication is 34;

[0211] The split feature map 2 is 17, the second attention ratio feature map is 31 → the second multiplication is 35;

[0212] The split feature map 3 is 18, the third attention ratio feature map is 32 → the third multiplication is 36;

[0213] The fourth feature map after splitting is 19, the fourth attention ratio feature map is 33 → the fourth multiplication is 37;

[0214] The first multiplication 34, the second multiplication 35, the third multiplication 36, the fourth multiplication 37 → the fourth superposition 38 → 1x1 convolution 39 → residual connection 40.

[0215] The multi-scale split attention mechanism network works by taking the following steps:

[0216] Step 1: Pass the input feature map through the encoder part of the network for feature extraction;

[0217] Step 2: Pass the output of each encoder layer in step 1 through the residual mixed attention mechanism;

[0218] Step 3: Feed the output of steps 1 and 2 into the decoder part of the network;

[0219] In step 1, the feature extraction of the encoder part specifically includes the following steps:

[0220] Step 1.1: Split the input through the encoding layer to the first layer of the attention mechanism module;

[0221] Step 1.2: Downsample the result of step 1.1 and split the second layer of the attention mechanism module through the encoding layer;

[0222] Step 1.3: Downsample the result of step 1.2 and split the third layer of the attention mechanism module through the encoding layer;

[0223] Step 1.4: Downsample the result of step 1.3 and split the fourth layer of the attention mechanism module through the encoding layer;

[0224] In step 2, the output of each encoder layer is passed through the residual mixed attention mechanism, which specifically includes the following steps:

[0225] Step 2.1: Pass the result of step 1.1 through the first layer of residual hybrid attention mechanism module;

[0226] Step 2.2: Pass the result of step 1.2 through the second layer of residual hybrid attention mechanism module;

[0227] Step 2.3: Pass the result of step 1.3 through the third layer of residual hybrid attention mechanism module;

[0228] In step 3, the decoder part specifically includes the following steps:

[0229] Step 3.1: Upsample the result of step 1.4 and concatenate it with the result of step 2.3, then split the first layer of the attention mechanism module through the decoding layer;

[0230] Step 3.2: Upsample the result of step 3.1 and concatenate it with the result of step 2.2, then split the second layer of the attention mechanism module through the decoding layer;

[0231] Step 3.2: Upsample the result of step 3.2 and concatenate it with the result of step 2.3, then split the third layer of the attention mechanism module through the decoding layer.

Claims

1. A skin disease image segmentation method based on a multi-scale split attention mechanism, characterized in that: It includes the following steps: Step 1: Read the skin disease image dataset and perform segmentation and preprocessing; Step 2: Construct a multi-scale split attention mechanism network; Step 3: Send the training set and validation set of skin disease images in step 1 to the network constructed in step 2 for training; Step 4: Send the test set divided in step 1 to the network trained in step 3 for prediction; Step 5: Output the final predicted image; In step 2, the constructed multi-scale split attention mechanism network is: Coding layer split attention mechanism module first layer (1) → Coding layer split attention mechanism module second layer (2) → Coding layer split attention mechanism module third layer (3) → Coding layer split attention mechanism module fourth layer (4); The third layer of the encoding layer split attention mechanism module (3) → the third layer residual hybrid attention mechanism module (5); The third layer of the encoding layer split attention mechanism module (3), the third layer residual hybrid attention mechanism module (5) → the first splicing (6) → the first layer of the decoding layer split attention mechanism module (7); The second layer of the encoding layer split attention mechanism module (2) → the second layer residual hybrid attention mechanism module (8); The second layer of the split attention mechanism module of the encoding layer (2), the second layer of the residual hybrid attention mechanism module (8) → the second splicing (9) → the second layer of the split attention mechanism module of the decoding layer (10); The first layer of the encoding layer split attention mechanism module (1) → the first layer of the residual hybrid attention mechanism module (11); The first layer of the encoding layer split attention mechanism module (1), the first layer of the residual hybrid attention mechanism module (11) → the second splicing (12) → the third layer of the decoding layer split attention mechanism module (13) → 1x1 convolution classification (14); The first layer (1) of the encoding layer split attention mechanism module, the second layer (2) of the encoding layer split attention mechanism module, the third layer (3) of the encoding layer split attention mechanism module, the fourth layer (4) of the encoding layer split attention mechanism module, and the first layer (7) of the decoding layer split attention mechanism module, the second layer (10) of the decoding layer split attention mechanism module, and the third layer (13) of the decoding layer split attention mechanism module all contain two multi-scale combined split attention mechanism modules. Each multi-scale combined split attention mechanism module divides the input image into four branches according to the channel, that is, four different feature maps, and then extracts multi-scale information through convolution with different void coefficients, and then adds an attention mechanism to the feature map obtained from each branch. The structure of each multi-scale combined split attention mechanism module is: Split by channel (15) → Feature map 1 after splitting (16), Feature map 2 after splitting (17), Feature map 3 after splitting (18), Feature map 4 after splitting (19); The first feature map after splitting (16) → the first multi-scale feature extraction (20); the second feature map after splitting (17) → the second multi-scale feature extraction (21); The third feature map after splitting (18) → the third multi-scale feature extraction (22); the fourth feature map after splitting (19) → the fourth multi-scale feature extraction (23); First multi-scale feature extraction (20), second multi-scale feature extraction (21) → first superposition (24); First superposition (24), third multi-scale feature extraction (22) → second superposition (25); Second superposition (25), fourth multi-scale feature extraction (23) → third superposition (26); The third stacking (26) → global average pooling (27) → generate attention weights (28) → r-softmax layer (29) → first attention ratio feature map (30), second attention ratio feature map (31), third attention ratio feature map (32), fourth attention ratio feature map (33); The split feature map 1 (16), the first attention weight feature map (30) → the first multiplication (34); The split feature map 2 (17), the second attention ratio feature map (31) → the second multiplication (35); The split feature map 3 (18), the third attention ratio feature map (32) → the third multiplication (36); The fourth feature map after splitting (19), the fourth attention ratio feature map (33) → the fourth multiplication (37); First multiplication (34), second multiplication (35), third multiplication (36), fourth multiplication (37) → fourth stacking (38) → 1x1 convolution (39) → residual connection (40).

2. The method according to claim 1, characterized in that In step 1, the skin disease image is preprocessed, specifically including the following steps: Step 1-1) Divide the skin disease dataset into training set, validation set, and test set; Step 1-2) Normalize the divided data set, that is, calculate the image by channel, first calculate its variance and mean, then subtract the mean from each data in each channel and divide it by the variance to obtain the normalized result.

3. The method according to claim 1, characterized in that In the multi-scale combined split attention mechanism module, extracting multi-scale feature information and adding the attention mechanism specifically include the following steps: Step s1: Split the input feature map into 4 different feature maps by channel; Step s2: Perform multi-scale feature extraction on the results of step s1; Step s3: perform feature aggregation on the results of step s2; Step s4: Apply attention mechanism to the result of step s3; In step s3, after feature extraction, the output of each branch is aggregated, and the formula is as follows: (1)； in For the i The output F is obtained by performing a dilated convolution operation on the branches. .

4. The method according to claim 3, characterized in that In step s4, the attention mechanism is used, which specifically includes the following steps: Step s4.1: Perform global average pooling on the result of step s3 so that the feature size is compressed to 1x1xC. The formula is as follows: (2)； Represents the feature map i , No. j The value of the pixel, i H, j W, H is the length of the feature map, W is the width of the feature map, S , C is the number of channels; Step s4.2: Pass the result of step s4.1 through two fully connected layers to obtain attention weights. Use the r-softmax activation function to convert the weights into a 0-1 ratio, multiply them by the output of each branch, add the attention weights, and then add them together. The formula is as follows: (3)； The value of i is 1, 2, 3, 4, 、 、 、 For the output of each branch, The formula is as follows: (4)； in for the reason Determine the split weight of the Cth channel in the i-th branch, where the value of i is 1, 2, 3, or 4. for the reason Determine the split weight of the Cth channel in the jth branch.

5. The method according to any one of claims 1 to 4, characterized in that The residual mixed attention mechanism module is used to add a mixed attention mechanism to the feature map, which specifically includes the following steps: Step (1) assign channel attention weights; Step (2) assign spatial attention weights; In step (1), the channel attention weight is assigned by performing global maximum pooling and global average pooling operations on the input features to obtain two feature maps with a shape of 1x1xC. Then, the attention weight is obtained by two 1x1 convolutions. The obtained weights are added together and activated by the sigmoid function to obtain the final channel attention feature. The formula is as follows: （5）； Among them, F is the input feature map, F C H W, It is the channel attention mechanism, and its formula is as follows: (6)； in, is the weight matrix of the first fully connected layer, , is the weight matrix of the second fully connected layer, , is the sigmoid function, 、 are the global average pooling features and global maximum pooling features in the channel attention mechanism, and r is the reduction rate; In step (2), the results of step (1) are subjected to global maximum pooling and global average pooling in the channel dimension to obtain two HxWx1 feature maps. After these two feature maps are spliced, they are reduced in dimension through a 7x7 convolution kernel and then passed through the sigmoid function to obtain the spatial attention feature. The final result is added to the input F. The formula is as follows: +F（7）； in To increase the result of channel attention weight, It is the spatial attention mechanism, and its formula is as follows: (8)； in is the sigmoid function, It is a 7x7 convolution; 、 They are the global average pooling features and global maximum pooling features in the spatial attention mechanism respectively; The maximum pooling is used for downsampling in the encoding layer, and bilinear interpolation is used for upsampling in the decoding layer.

6. The method according to claim 1 or 2, characterized in that The multi-scale split attention mechanism network works by taking the following steps: Step 2.1: Pass the input feature map through the encoder part of the network for feature extraction; Step 2.2: Pass the output of each encoder layer in step 2.1 through the residual mixed attention mechanism; Step 2.3: Feed the output of steps 2.1 and 2.2 into the decoder part of the network. In step 2.1, the feature extraction of the encoder part specifically includes the following steps: Step 2.1.1: Split the input through the encoding layer to the first layer of the attention mechanism module; Step 2.1.2: Downsample the result of step 2.1.1 and split the second layer of the attention mechanism module through the encoding layer; Step 2.1.3: Downsample the result of step 2.1.2 and split the third layer of the attention mechanism module through the encoding layer; Step 2.1.4: Downsample the result of step 2.1.3 and split the fourth layer of the attention mechanism module through the encoding layer; In step 2.2, passing the output of each encoder layer through the residual mixed attention mechanism specifically includes the following steps: Step 2.2.1: Pass the result of step 2.1.1 through the first layer of residual hybrid attention mechanism module; Step 2.2.2: Pass the result of step 2.1.2 through the second layer of residual hybrid attention mechanism module; Step 2.2.3: Pass the result of step 2.1.3 through the third layer residual hybrid attention mechanism module; In step 2.3, the decoder part specifically includes the following steps: Step 2.3.1: Upsample the result of step 2.1.4 and concatenate it with the result of step 2.2.3, then split the first layer of the attention mechanism module through the decoding layer; Step 2.3.2: Upsample the result of step 2.3.1 and concatenate it with the result of step 2.2.2, then split the second layer of the attention mechanism module through the decoding layer; Step 2.3.3: Upsample the result of step 2.3.2 and concatenate it with the result of step 2.2.3, then split the third layer of the attention mechanism module through the decoding layer.

Images