Efficient skin disease segmentation method based on multi-scale and mixed attention mechanism

Abstract

The invention discloses an efficient skin disease segmentation method based on multiple scales and a mixed attention mechanism, and the method comprises the steps: firstly carrying out the preprocessing of a skin mirror image through data enhancement, and then inputting the skin mirror image into a convolutional neural network for feature extraction, thereby obtaining a skin lesion segmentation region prediction map. According to the method, the convolutional neural network is applied, multi-scale features of local information and global information can be extracted from a skin mirror image, and the feature representation capability is enhanced through feature fusion. The objective of the invention is to solve the problems of changeable lesion areas, irregular shapes and the like of skin disease images, and improve the accuracy and robustness of skin disease image segmentation.

Description

Technical Field

[0001] The present invention relates to the fields of computer-aided diagnosis and medical image processing, and in particular to an efficient skin disease segmentation method based on multi-scale and hybrid attention mechanisms. Technical Background

[0002] Skin diseases are very common around the world. They not only cause physical discomfort to patients, but also affect their mental health and social life. As a branch of skin diseases, skin cancer has the fastest growth rate in the world. Melanoma, in particular, is considered one of the most deadly skin cancers and the main cause of death for most skin cancer patients. According to statistics, there were approximately 1,329,779 newly confirmed cases in 2018, of which the global incidence of melanoma exceeded 287,700 and the number of deaths was close to 60,700. Although the mortality rate of melanoma is high, the cure rate of early malignant melanoma is over 95%, while the survival rate of late-stage melanoma drops to 15%. Therefore, efficient and accurate early diagnosis of skin diseases is crucial to improving the survival rate of patients.

[0003] For a long time, the diagnosis of skin diseases has mainly relied on dermoscopy, especially for the early detection of early melanoma. Dermoscopy is a non-invasive imaging diagnostic technology that enables doctors to detect lesions that are difficult to detect with the human eye by eliminating surface reflections and enhancing visualization of deep skin. Although dermoscopy has been widely recognized, manual inspection of images is still a time-consuming and skill-intensive task. Relying solely on human vision to identify melanoma can lead to subjective, inaccurate, and poorly reproducible results. To improve this problem, the introduction of computer-aided diagnosis (CAD) technology has become a key means to help dermatologists in their daily practice. CAD technology aims to improve the efficiency of detection and analysis of skin diseases, especially in the establishment of a skin cancer CAD system. The system focuses on the segmentation and classification of skin lesions to accurately detect lesion locations and boundaries and assist in the diagnosis of different types of lesions, such as melanoma, nevus, seborrheic keratosis, etc.

[0004] Image segmentation technology is a key technical field in the study of computer vision processing, especially in computer-aided diagnosis systems. This technology not only helps doctors focus directly on the lesion area, but also its processing results directly affect the accuracy of subsequent image recognition. In skin lesion images, segmentation usually involves detecting pigment blocks that are different from normal skin color. However, due to the influence of multiple factors such as color, texture, lesion size, lesion location and shape, and artifacts, the skin disease segmentation task is still extremely challenging. Early skin disease segmentation methods were mainly based on optimal thresholds, region growing, and edge detection algorithms. However, these methods usually have difficulty in obtaining accurate segmentation results when processing skin lesion images with uneven colors and blurred boundaries. In addition, these traditional methods often require manual intervention or contain a large number of hyperparameters for fine-tuning to achieve good segmentation results. However, with the rise of deep learning, this dilemma has been solved. At the same time, network segmentation algorithms based on deep learning have gradually replaced traditional algorithms and become the mainstream hotspot of current research. As the earliest deep learning algorithm introduced in the field of image segmentation, the fully convolutional neural network (FCN) has pioneered a new segmentation method. Many optimization algorithms have emerged based on the FCN structure, and the introduction of U-Net has established its important position in the field of medical image segmentation. At present, most medical image segmentation networks are built on the basis of the U-Net structure. The overall structure of U-Net adopts an encoding-decoding structure. In the encoding part, features are extracted by downsampling. In the decoding part, the image size is restored by upsampling operations, and finally a segmentation prediction map is generated. The structure also designs a skip connection operation to make up for the part of the information lost by the downsampling operation in the feature extraction stage of high-level features, thereby improving the accuracy and robustness of segmentation. Many network structures are inspired by U-Net and improved on its basis. For example, FAT-Net is based on the U-Net structure and introduces a dual encoder integrated convolutional neural network (CNN) and transformer branches to integrate local and global context information to enhance the accuracy of segmentation boundaries. On the other hand, H2Former combines convolution and transformer modules to not only integrate local and global context information, but also incorporate multi-scale information, further improving segmentation performance. Summary of the invention

[0005] In view of the problems of variable lesion areas and irregular shapes in skin disease images in the above-mentioned prior art, the present invention provides an efficient skin disease segmentation method based on network coding, multi-scale and hybrid attention mechanism. The purposes of the present invention are: (1) to solve the problems of variable lesion areas and irregular shapes in skin disease images; (2) to reduce the number of model parameters and improve the prediction efficiency while ensuring the prediction accuracy; (3) to improve the accuracy and robustness of skin disease image segmentation by effectively integrating global context and local information as well as multi-scale information.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] An efficient skin disease segmentation method based on multi-scale and hybrid attention mechanism. This method first uses data enhancement strategies such as random rotation of images, random changes in image brightness and contrast, and random changes in image hue and saturation values ​​to increase sample diversity to ensure the generalization ability of the model. In the encoder stage, convolutional neural networks and global and local attention modules are used to extract global context and local and multi-scale information, and then feature fusion is performed through convolution and other methods to enhance the information extraction ability of the model and improve segmentation prediction accuracy. In the decoder stage, a lightweight decoder is built using transposed convolution to restore the image size, and simple jump connections are used to compensate for lost information. Finally, a prediction map of the skin lesion area is output to achieve efficient skin lesion segmentation.

[0008] As a preferred embodiment of the present invention, the method specifically comprises the following steps:

[0009] 1) Construction of skin disease segmentation network based on multi-scale and hybrid attention mechanism

[0010] 1.1) The segmentation network consists of three key components: encoder, decoder and jump connection. The encoder includes two parts: downsampling and feature extraction layer. In the downsampling part, the image features are initially extracted through convolution and maximum pooling operations. This process not only helps to reduce the spatial dimension of the feature map and improve computational efficiency, but also enables the model to better understand the structure and semantic information of the image by expanding the receptive field and retaining important features. The feature extraction layer consists of a channel attention module, a global and local attention module, and a multi-scale feature extraction module. The role of the channel attention module is to adjust the feature responses of different channels by learning the weights between channels to highlight the channels that are more important to the current task, thereby enhancing the performance of the network on specific tasks. The global and local attention modules are mainly used to help the convolutional neural network extract global context and local information. Because the convolutional neural network has inherent limitations and its ability to extract global context information is weak, a global attention module is used to make up for its shortcomings. At the same time, adding a local attention module can further enhance the convolutional neural network's ability to extract local features. After passing through the global and local attention modules, the extracted features are subjected to multi-scale feature extraction. The multi-scale feature extraction module extracts features at different scales. , so that the model has the ability to adapt to different object sizes, improve translation invariance and anti-deformation capabilities; at the same time, it can better capture the hierarchical structure and details of the image, avoiding the loss of information at a single scale; in addition, multi-scale feature extraction also improves the robustness of the model, making it more suitable for processing skin disease image data with variable lesion areas and irregular shapes; finally, the multi-scale features extracted from local information and global information are effectively fused through the convolution module; the main purpose of the jump connection is to achieve key functions such as information transmission, detail retention, anti-gradient vanishing, accelerated convergence and reduced parameter transmission distance by connecting the bottom and high-level features, effectively improving the network performance and generalization ability; the decoder is mainly composed of simple convolution operations, whose function is to convert the high-level semantic feature map extracted by the encoder into the same resolution as the input image, and gradually restore the detail information through upsampling and jump connection operations; the decoder converts the low-dimensional feature map into a high-resolution feature map through convolution and upsampling operations, and uses jump connections to combine the features of the corresponding layer of the encoder with the corresponding layer of the decoder, which helps to retain the detailed information of the bottom and high layers; this process enables the network to generate fine segmentation results, while making full use of the semantic information extracted by the encoder to achieve accurate and comprehensive segmentation of the input image;

[0011] 1.2) The segmentation network specifically includes: 1 4x downsampling initial feature extraction module, 4 encoders, 3 skip connections, 3 decoders, and a 4x upsampling module; each encoder includes 1 downsampling operation and a feature extraction layer, and the first encoder has no downsampling operation; each feature extraction layer includes 1 local attention, 1 global attention module, 1 channel attention module, and 1 multi-scale feature extraction module;

[0012] 1.3) The network input is a skin disease image after data enhancement, with a size of 224×224 and a total of 3 channels. Two images are input each time, and the dimension is [2, 3, 224, 224]; the network output is 1 channel, with the same size as the input image, and its dimension is [2, 1, 224, 224];

[0013] 1.4) The initial 4x downsampling feature extraction module consists of a convolution with a kernel size of 7×7, a stride of 2, and a padding of 3, layer normalization, a Relu activation function, and a maximum pooling layer with a kernel size of 3×3, a stride of 2, and a padding of 1. After mapping the original input to a high dimension, the encoder is used to extract feature information. The number of modules in the feature extraction layer of each encoder is [1,1,1,1], and the number of channels is 64, 128, 256, and 512 respectively. The decoder is constructed by 3 simple convolutional layers. The model finally restores the original image size through two consecutive 2x upsampling, and then uses the convolutional layer to predict and output the segmentation prediction map.

[0014] 2) Network model training and parameter optimization

[0015] 2.1) Network parameter initialization: Kaiming initialization method is used to initialize the parameters of the skin disease segmentation network model based on multi-scale and hybrid attention mechanism;

[0016] 2.2) Dataset construction: The dataset uses the public dataset provided by the International Skin Imaging Collaboration (ISIC);

[0017] 2.3) Data preprocessing: First, the dataset resolution is uniformly adjusted to 224×224. Then, in order to increase the diversity of image samples, a variety of data augmentation strategies are adopted, including vertically flipping the input image around the x-axis or y-axis, rotating the input image by a randomly selected angle of -15 to 15 degrees, randomly changing the brightness and contrast of the input image at a ratio of -3% to 3%, and randomly changing the hue and saturation values ​​of the input image at a ratio of -3% to 3%.

[0018] 2.4) The image after data preprocessing is used as the input of the network, and the feature vector is generated through network forward calculation;

[0019] 2.5) The standard binary cross entropy loss function and Dice loss function are used as the segmentation prediction network target optimization function. The definition of Dice loss function is as follows:

[0020]

[0021] The cross entropy loss function is defined as follows:

[0022]

[0023] Where i represents the index of all pixels in the probability distribution map, P i is the probability that the i-th pixel belongs to the segmented region, G i is the true label of the i-th pixel;

[0024] The total loss function is defined as follows:

[0025] L=λ1L bce +λ2L dice

[0026] Among them, λ1 and λ2 are set to 0.6 and 0.4 respectively;

[0027] 2.6) To optimize the model performance, the model parameters need to be optimized to the minimum value of L;

[0028] 3) Evaluation indicators

[0029] Five widely used metrics are adopted to evaluate the accuracy performance of the method, including sensitivity (SE), specificity (SP), accuracy (ACC), intersection over union (IoU) and dice coefficient (Dice)

[0030]

[0031] Among them, TP (True Positive) and TN (True Negative) represent the number of correctly segmented skin lesion pixels and background pixels, respectively; FP (False Positive) represents background pixels that are incorrectly marked as skin lesion pixels, and FN (False Negative) represents skin lesion pixels that are incorrectly predicted as background pixels.

[0032] As a preferred embodiment of the present invention, in step 2.2), the public data sets are the ISIC2017 data set (Codella et al., 2018) and the ISIC2018 data set (Codella et al., 2019). The ISIC2017 data set includes a total of 2750 annotated dermoscopic images, which are divided into a training set, a validation set and a test set; specifically, the training set contains 2000 images, the validation set contains 150 images, and the test set contains 600 images; ISIC2018 contains 2594 RGB skin lesion images, of which 1815 images are randomly selected for training, 259 images are used for validation, and 520 images are used for testing.

[0033] As a preferred solution of the present invention, in step 2.4), the forward calculation of the network includes:

[0034] Convolution operation: The output feature map corresponding to any convolution kernel is calculated using the following formula:

[0035]

[0036] Where f represents the nonlinear activation function, b i represents the bias term corresponding to the i-th convolution kernel, r represents the input channel index number, k represents the number of input channels, and W ir represents the rth channel weight matrix of the i-th convolution kernel, is the convolution operation, X r Represents the rth input channel image;

[0037] Self-attention operation: Self-attention helps the model better understand the global structure and pixel-level correlation of the image by learning the long-range dependencies and spatial relationships between different regions in the image, thereby improving the accuracy of image segmentation; it enables the model to dynamically adjust the size of the receptive field to adapt to objects of different scales, and at the same time, it can more flexibly focus on areas related to the segmentation target in processing complex scenes; the calculation formula of the self-attention mechanism is as follows:

[0038] Query = XW Q ,Key=XW K ,Value=XW v (2)

[0039]

[0040] Where X is the input feature matrix, Query, Key, Value are the representations of X after linear transformation, representing query, key, and value respectively, and W Q ,W k ,W Vare three learnable feature weight matrices, and d is the channel dimension of Query or Key;

[0041] Batch Normalization (BN) is a method that makes neural network training faster and more stable. It calculates the mean and variance of each mini-batch and pulls it back to a standard normal distribution with a mean of 0 and a variance of 1.

[0042] Layer Normalization: LayerNormalization (LN) is a neural network regularization method, similar to batch normalization, which calculates the mean and variance of all features of each sample in the same layer and adjusts them to a standard normal distribution to improve training speed and stability;

[0043] Nonlinear excitation: Use the rectified linear unit ReLU as the nonlinear excitation function to perform nonlinear transformation on each normalized value. The rectified linear unit ReLU is defined as follows:

[0044] ReLU(x)=max(0,x) (4)

[0045] Among them, max means finding the maximum value, and x is an input value;

[0046] Probability value conversion: Use the Sigmoid function to convert the predicted score into a probability distribution. The Sigmoid function is defined as follows:

[0047]

[0048] Among them, Y j is the probability that the jth pixel belongs to the segmented region, O j It is the predicted score at the jth pixel output by the label predictor.

[0049] As a preferred solution of the present invention, in step 2.6), the Adam optimizer algorithm and the back propagation algorithm are used to iteratively update the network parameters θ to obtain the optimal network parameters; the Adam optimizer refers to an adaptive optimization algorithm that can adjust the learning rate according to historical gradient information; it combines the ideas of the two traditional gradient descent and momentum update optimization algorithms, and normalizes the parameter updates so that each parameter update has a similar magnitude, thereby improving the training effect; the Adam optimizer can adaptively adjust the learning rate according to the historical gradient information, and there is no need to manually adjust the learning rate, thereby improving the convergence speed and generalization ability of the model, while reducing the amount of calculation, so that it can perform well on large-scale data sets.

[0050] As a preferred solution of the present invention, when the Adam optimizer is used for iteration in this step, the number of samples used in each iteration, i.e., the batch size, is set to 2, the initial learning rate is set to 1e-4, and the cosine regression learning rate decay technique is used, and the period is consistent with the total number of training rounds.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. Through the local and global attention modules and the multi-scale feature extraction module, not only the global context and local information are extracted, but also the information of different scales is obtained. The convolution is used for fusion, so that the network can effectively deal with the problems of variable image lesion areas and irregular shapes.

[0053] 2. The model's ability to distinguish the boundaries of lesion areas is enhanced, improving the accuracy and robustness of skin disease image segmentation.

[0054] 3. By grouping convolution and reducing convolution from 2D calculation to linear calculation, not only the number of model parameters is reduced, but also the prediction efficiency is improved while ensuring the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a diagram of the skin disease segmentation network model based on multi-scale and hybrid attention mechanism;

[0056] Figure 2 This is the structure diagram of the local attention and global attention modules;

[0057] Figure 3 This is the structural diagram of the channel attention and multi-scale feature extraction module. DETAILED DESCRIPTION

[0058] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0059] An efficient skin disease segmentation method based on multi-scale and hybrid attention mechanism. This method first uses data enhancement strategies such as random rotation of images, random changes in image brightness and contrast, and random changes in image hue and saturation values ​​to increase sample diversity to ensure the generalization ability of the model. In the encoder stage, convolutional neural networks and global and local attention modules are used to extract global context and local and multi-scale information, and then feature fusion is performed through convolution and other methods to enhance the information extraction ability of the model and improve the segmentation prediction accuracy. In the decoder stage, a lightweight decoder is built using transposed convolution to restore the image size, and simple jump connections are used to compensate for lost information. Finally, a prediction map of the skin lesion area is output to achieve efficient skin lesion segmentation.

[0060] The method specifically comprises the following steps:

[0061] 1) Construction of skin disease segmentation network based on multi-scale and hybrid attention mechanism, such as Figure 1 shown.

[0062] 1.1) The segmentation network consists of three key components: encoder, decoder and skip connection. The encoder consists of two parts: downsampling and feature extraction layer. In the downsampling part, the image features are initially extracted through convolution and maximum pooling operations. This process not only helps to reduce the spatial dimension of the feature map and improve computational efficiency, but also enables the model to better understand the structure and semantic information of the image by expanding the receptive field and retaining important features. The feature extraction layer consists of a channel attention module, a global and local attention module, and a multi-scale feature extraction module. The structure of the local attention and global attention modules is as follows: Figure 2 As shown in Figure 2, the channel attention and multi-scale feature extraction module structure is as follows: Figure 3As shown in the figure. The role of the channel attention module is to adjust the feature responses of different channels by learning the weights between channels to highlight the channels that are more important to the current task, thereby enhancing the performance of the network on specific tasks. The global and local attention modules are mainly used to help the convolutional neural network extract global context and local information. Because the convolutional neural network has inherent limitations and is weak in extracting global context information, a global attention module is used to make up for its shortcomings. At the same time, adding a local attention module can further enhance the convolutional neural network's ability to extract local features. After passing through the global and local attention modules respectively, multi-scale feature extraction is performed on the extracted features. Specifically, the multi-scale feature extraction module extracts features at different scales, enabling the model to adapt to different object sizes, improve translation invariance and anti-deformation capabilities. At the same time, it can better capture the hierarchical structure and details of the image, avoiding the loss of information at a single scale. In addition, multi-scale feature extraction also improves the robustness of the model, making it more suitable for processing skin disease image data with variable lesion areas and irregular shapes. Finally, the multi-scale features extracted from local information and global information are effectively fused through the convolution module. The main purpose of the jump connection is to achieve key functions such as information transmission, detail preservation, anti-gradient vanishing, accelerated convergence, and reduced parameter transfer distance by connecting the features of the bottom layer and the top layer, which effectively improves the network performance and generalization ability. The decoder is mainly composed of simple convolution operations, which converts the high-level semantic feature map extracted by the encoder into the same resolution as the input image, and gradually restores the detail information through upsampling and jump connection operations. The decoder converts the low-dimensional feature map into a high-resolution feature map through convolution and upsampling operations, and uses jump connections to combine the features of the corresponding layer of the encoder with the corresponding layer of the decoder, which helps to retain detailed information of the bottom layer and the top layer. This process enables the network to generate fine segmentation results, while making full use of the semantic information extracted by the encoder to achieve accurate and comprehensive segmentation of the input image.

[0063] 1.2) The segmentation network specifically includes: 1 4x downsampling initial feature extraction module, 4 encoders, 3 skip connections, 3 decoders, and a 4x upsampling module. Each encoder includes 1 downsampling operation and a feature extraction layer, and the first encoder has no downsampling operation. Each feature extraction layer includes 1 local attention, 1 global attention module, 1 channel attention module, and 1 multi-scale feature extraction module.

[0064] 1.3) The network input is a skin disease image after data enhancement, with a size of 224×224, a total of 3 channels, and 2 images are input each time, with a dimension of [2,3,224,224]. The network output is 1 channel, with the same size as the input image, and a dimension of [2,1,224,224].

[0065] 1.4) The initial 4x downsampling feature extraction module consists of a convolution with a kernel size of 7×7, a step size of 2, and a padding of 3, layer normalization, a Relu activation function, and a maximum pooling layer with a kernel size of 3×3, a step size of 2, and a padding of 1. After mapping the original input to a high dimension, the encoder is used to extract the feature information. The number of modules in the feature extraction layer of each encoder is [1,1,1,1], and the number of channels is 64, 128, 256, and 512 respectively. The decoder is constructed by 3 simple convolutional layers. The model finally restores the original image size through two consecutive 2x upsampling, and then uses the convolutional layer to predict and output the segmentation prediction map. See the table below for detailed model parameters.

[0066] Table 1 Parameters of skin disease segmentation network based on multi-scale and hybrid attention mechanism (Padding = 1, part = 3)

[0067]

[0068] Table 2 Feature extraction layer parameter table (Padding = 1, taking feature extraction layer 1 as an example)

[0069]

[0070]

[0071] Table 3 Multi-scale feature extraction module parameters (taking feature extraction layer 1 as an example)

[0072]

[0073] Table 4 Local attention module parameters (taking feature extraction layer 1 as an example)

[0074]

[0075] Table 5 Global attention module parameters (taking feature extraction layer 1 as an example)

[0076]

[0077] 2) Network model training and parameter optimization

[0078] 2.1) Network parameter initialization: The Kaiming initialization method is used to initialize the parameters of the skin disease segmentation network model based on multi-scale and hybrid attention mechanism.

[0079] 2.2) Dataset construction: The datasets used are public datasets provided by the International Skin Imaging Collaboration (ISIC), namely the ISIC2017 dataset (Codella et al, 2018) and the ISIC2018 dataset (Codella et al, 2019). The ISIC2017 dataset includes a total of 2750 annotated dermoscopic images, which are divided into training set, validation set and test set. Specifically, the training set contains 2000 images, the validation set contains 150 images, and the test set contains 600 images. ISIC2018 contains 2594 RGB skin lesion images, of which 1815 images are randomly used for training, 259 images are used for validation, and 520 images are used for testing.

[0080] 2.3) Data preprocessing: First, the dataset resolution is uniformly adjusted to 224×224. Then, in order to increase the diversity of image samples, a variety of data augmentation strategies are adopted, including vertically flipping the input image around the x-axis or y-axis, rotating the input image by a randomly selected angle from -15 to 15 degrees, randomly changing the brightness and contrast of the input image at a ratio of -3% to 3%, and randomly changing the hue and saturation values ​​of the input image at a ratio of -3% to 3%.

[0081] 2.4) The image after data preprocessing is used as the input of the network, and a feature vector is generated through network forward calculation. As a specific embodiment, the network forward calculation includes:

[0082] Convolution operation: The output feature map corresponding to any convolution kernel is calculated using the following formula:

[0083]

[0084] Where f represents the nonlinear activation function, b i represents the bias term corresponding to the i-th convolution kernel, r represents the input channel index number, k represents the number of input channels, and W ir represents the rth channel weight matrix of the i-th convolution kernel, is the convolution operation, X r Represents the rth input channel image.

[0085] Self-attention operation: Self-attention helps the model better understand the global structure and pixel-level correlation of the image by learning the long-range dependencies and spatial relationships between different regions in the image, thereby improving the accuracy of image segmentation. It enables the model to dynamically adjust the size of the receptive field to adapt to objects of different scales, and to more flexibly focus on areas related to the segmentation target in processing complex scenes. The calculation formula of the self-attention mechanism is as follows:

[0086] Query = XWQ ,Key=XW K ,Value=XW v (2)

[0087]

[0088] Where X is the input feature matrix, Query, Key, Value are the representations of X after linear transformation, representing query, key, and value respectively, and W Q ,W k ,W V are three learnable feature weight matrices, and d is the channel dimension of Query or Key.

[0089] Batch Normalization (BN) is a method to make neural network training faster and more stable. It calculates the mean and variance of each mini-batch and pulls it back to a standard normal distribution with a mean of 0 and a variance of 1.

[0090] Layer Normalization: LayerNormalization (LN) is a neural network regularization method, similar to batch normalization, which calculates the mean and variance of all features of each sample in the same layer and adjusts them to a standard normal distribution to improve training speed and stability.

[0091] Nonlinear excitation: Use the rectified linear unit ReLU as the nonlinear excitation function to perform nonlinear transformation on each normalized value. The rectified linear unit ReLU is defined as follows:

[0092] ReLU(x)=max(0,x) (4)

[0093] Among them, max means finding the maximum value, and x is an input value.

[0094] Probability value conversion: Use the Sigmoid function to convert the predicted score into a probability distribution. The Sigmoid function is defined as follows:

[0095]

[0096] Among them, Y j is the probability that the jth pixel belongs to the segmented region, O j It is the predicted score at the jth pixel output by the label predictor.

[0097] 2.5) The standard binary cross entropy loss function and Dice loss function are used as the segmentation prediction network target optimization function. The definition of Dice loss function is as follows:

[0098]

[0099] The cross entropy loss function is defined as follows:

[0100]

[0101] Where i represents the index of all pixels in the probability distribution map. i is the probability that the i-th pixel belongs to the segmented region, G i is the true label of the i-th pixel.

[0102] The total loss function is defined as follows:

[0103] L=λ1L bce +λ2L dice

[0104] Among them, λ1 and λ2 are set to 0.6 and 0.4 respectively.

[0105] 2.6) In order to optimize the model performance, the model parameters need to be optimized to the minimum value of L. To this end, the present invention uses the Adam optimizer algorithm and the back propagation algorithm to iteratively update the network parameters θ to obtain the optimal network parameters; Adam optimizer refers to an adaptive optimization algorithm that can adjust the learning rate according to historical gradient information. It combines the ideas of the two optimization algorithms of traditional gradient descent and momentum update, and normalizes the update of the parameters so that the update of each parameter has a similar magnitude, thereby improving the training effect. The Adam optimizer can adaptively adjust the learning rate according to the historical gradient information, and does not need to manually adjust the size of the learning rate, thereby improving the convergence speed and generalization ability of the model, while reducing the amount of calculation, so that it can have good performance on large-scale data sets. As a specific embodiment, when the Adam optimizer is used for iteration in this step, the number of samples used in each iteration, that is, the batch size, is set to 2, the initial learning rate is set to 1e-4, and the cosine return learning rate decay technology is used, and the cycle is consistent with the total number of training rounds.

[0106] 3) Evaluation indicators

[0107] We adopt five widely used metrics to evaluate the accuracy performance of our proposed method, including sensitivity (SE), specificity (SP), accuracy (ACC), intersection over union (IoU) and dice coefficient (Dice)

[0108]

[0109] TP (True Positive) and TN (True Negative) represent the number of correctly segmented skin lesion pixels and background pixels, respectively. FP (False Positive) represents background pixels that are incorrectly marked as skin lesion pixels, and FN (False Negative) represents skin lesion pixels that are incorrectly predicted as background pixels.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. An efficient skin disease segmentation method based on multi-scale and hybrid attention mechanism, characterized in that: This method first uses random image rotation, random changes in image brightness and contrast, and random changes in image hue and saturation values ​​to increase sample diversity to ensure the generalization ability of the model; in the encoder stage, convolutional neural networks and global and local attention modules are used to extract global context and local and multi-scale information, and then feature fusion is performed through convolution to enhance the information extraction ability of the model and improve segmentation prediction accuracy; in the decoder stage, transposed convolution is used to build a lightweight decoder for image size recovery, and simple jump connections are used to compensate for lost information, and finally a prediction map of the skin lesion area is output to achieve efficient skin lesion segmentation.

2. The efficient skin disease segmentation method based on multi-scale and hybrid attention mechanism according to claim 1 is characterized in that the method specifically comprises the following steps: 1) Construction of skin disease segmentation network based on multi-scale and hybrid attention mechanism 1.1) The segmentation network consists of three key components: encoder, decoder and jump connection. The encoder consists of two parts: downsampling and feature extraction layer. In the downsampling part, image features are initially extracted through convolution and maximum pooling operations. This process not only helps to reduce the spatial dimension of the feature map and improve computational efficiency, but also enables the model to better understand the structure and semantic information of the image by expanding the receptive field and retaining important features. The feature extraction layer consists of a channel attention module, a global and local attention module, and a multi-scale feature extraction module. The role of the channel attention module is to adjust the feature responses of different channels by learning the weights between channels to highlight the channels that are more important to the current task, thereby enhancing the performance of the network on specific tasks. The global and local attention modules are mainly used to help the convolutional neural network extract global context and local information. After passing through the global and local attention modules respectively, multi-scale feature extraction is performed on the extracted features. The multi-scale feature extraction module extracts features at different scales, enabling the model to adapt to different object sizes, improve translation invariance and anti-deformation capabilities, and better capture the hierarchical structure of the image. and details, avoiding the loss of information at a single scale; in addition, multi-scale feature extraction also improves the robustness of the model, making it more suitable for processing skin disease image data with variable lesion areas and irregular shapes; finally, the multi-scale features extracted from local information and global information are effectively fused through the convolution module; the main purpose of the jump connection is to achieve key functions such as information transmission, detail retention, anti-gradient vanishing, accelerated convergence and reduced parameter transfer distance by connecting the bottom and high-level features, effectively improving the network performance and generalization ability; the decoder is mainly composed of simple convolution operations, whose function is to convert the high-level semantic feature map extracted by the encoder into the same resolution as the input image, and gradually restore the detail information through upsampling and jump connection operations; the decoder converts the low-dimensional feature map into a high-resolution feature map through convolution and upsampling operations, and uses jump connections to combine the features of the corresponding layer of the encoder with the corresponding layer of the decoder, which helps to retain detailed information at the bottom and high levels; this process enables the network to generate fine segmentation results, while making full use of the semantic information extracted by the encoder to achieve accurate and comprehensive segmentation of the input image; 1.2) The segmentation network specifically includes: 1 4x downsampling initial feature extraction module, 4 encoders, 3 skip connections, 3 decoders, and a 4x upsampling module; each encoder includes 1 downsampling operation and a feature extraction layer, and the first encoder has no downsampling operation; each feature extraction layer includes 1 local attention, 1 global attention module, 1 channel attention module, and 1 multi-scale feature extraction module; 1.3) The network input is a skin disease image after data enhancement, with a size of 224×224 and a total of 3 channels. Two images are input each time, and the dimension is [2, 3, 224, 224]; the network output is 1 channel, with the same size as the input image, and its dimension is [2, 1, 224, 224]; 1.4) The initial 4x downsampling feature extraction module consists of a convolution with a kernel size of 7×7, a stride of 2, and a padding of 3, layer normalization, a Relu activation function, and a maximum pooling layer with a kernel size of 3×3, a stride of 2, and a padding of 1. After mapping the original input to a high dimension, the encoder is used to extract feature information. The number of modules in the feature extraction layer of each encoder is [1,1,1,1], and the number of channels is 64, 128, 256, and 512 respectively. The decoder is constructed by 3 simple convolutional layers. The model finally restores the original image size through two consecutive 2x upsampling, and then uses the convolutional layer to predict and output the segmentation prediction map. 2) Network model training and parameter optimization 2.1) Network parameter initialization: Kaiming initialization method is used to initialize the parameters of the skin disease segmentation network model based on multi-scale and hybrid attention mechanism; 2.2) Dataset construction: The dataset uses a public dataset provided by the International Skin Imaging Collaboration; 2.3) Data preprocessing: First, the dataset resolution is uniformly adjusted to 224×224. Then, in order to increase the diversity of image samples, a variety of data augmentation strategies are adopted, including vertically flipping the input image around the x-axis or y-axis, rotating the input image by a randomly selected angle of -15 to 15 degrees, randomly changing the brightness and contrast of the input image at a ratio of -3% to 3%, and randomly changing the hue and saturation values ​​of the input image at a ratio of -3% to 3%. 2.4) The image after data preprocessing is used as the input of the network, and the feature vector is generated through network forward calculation; 2.5) The standard binary cross entropy loss function and Dice loss function are used as the segmentation prediction network target optimization function. The definition of Dice loss function is as follows: The cross entropy loss function is defined as follows: Where i represents the index of all pixels in the probability distribution map, P i is the probability that the i-th pixel belongs to the segmented region, G i is the true label of the i-th pixel; The total loss function is defined as follows: L=λ1L bce +λ2L dice Among them, λ1 and λ2 are set to 0.6 and 0.4 respectively; 2.6) To optimize the model performance, the model parameters need to be optimized to the minimum value of L; 3) Evaluation indicators Five widely used metrics are adopted to evaluate the accuracy performance of the method, including sensitivity (SE), specificity (SP), accuracy (ACC), intersection over union (IoU) and dice coefficient (Dice) Among them, TP (True Positive) and TN (True Negative) represent the number of correctly segmented skin lesion pixels and background pixels, respectively; FP (False Positive) represents background pixels that are incorrectly marked as skin lesion pixels, and FN (False Negative) represents skin lesion pixels that are incorrectly predicted as background pixels.

3. The efficient skin disease segmentation method based on multi-scale and hybrid attention mechanism according to claim 2 is characterized in that: In step 2.2), the public data sets are the ISIC2017 data set and the ISIC2018 data set. The ISIC2017 data set includes a total of 2750 annotated dermoscopic images, which are divided into training set, validation set and test set; specifically, the training set contains 2000 images, the validation set contains 150 images, and the test set contains 600 images; ISIC2018 contains 2594 RGB skin lesion images, of which 1815 images are randomly used for training, 259 images are used for validation, and 520 images are used for testing.

4. The efficient skin disease segmentation method based on multi-scale and hybrid attention mechanism according to claim 2 is characterized in that: In step 2.4), the forward calculation of the network includes: Convolution operation: The output feature map corresponding to any convolution kernel is calculated using the following formula: Where f represents the nonlinear activation function, b i represents the bias term corresponding to the i-th convolution kernel, r represents the input channel index number, k represents the number of input channels, and W ir represents the rth channel weight matrix of the i-th convolution kernel, is the convolution operation, X r Represents the rth input channel image; Self-attention operation: Self-attention helps the model better understand the global structure and pixel-level correlation of the image by learning the long-range dependencies and spatial relationships between different regions in the image, thereby improving the accuracy of image segmentation; it enables the model to dynamically adjust the size of the receptive field to adapt to objects of different scales, and at the same time, it can more flexibly focus on areas related to the segmentation target in processing complex scenes; the calculation formula of the self-attention mechanism is as follows: Query=XW Q ,Key=XW K ,Value=XW v (2) Where X is the input feature matrix, Query, Key, Value are the representations of X after linear transformation, representing query, key, and value respectively, and W Q ,W k ,W V are three learnable feature weight matrices, and d is the channel dimension of Query or Key; Batch Normalization (BN) is a method that makes neural network training faster and more stable. It calculates the mean and variance of each mini-batch and pulls it back to a standard normal distribution with a mean of 0 and a variance of 1. Layer Normalization: LayerNormalization (LN) is a neural network regularization method, similar to batch normalization, which calculates the mean and variance of all features of each sample in the same layer and adjusts them to a standard normal distribution to improve training speed and stability; Nonlinear excitation: Use the rectified linear unit ReLU as the nonlinear excitation function to perform nonlinear transformation on each normalized value. The rectified linear unit ReLU is defined as follows: ReLU(x)=max(0,x) (4) Among them, max means finding the maximum value, and x is an input value; Probability value conversion: Use the Sigmoid function to convert the predicted score into a probability distribution. The Sigmoid function is defined as follows: Among them, Y j is the probability that the jth pixel belongs to the segmented region, O j It is the predicted score at the jth pixel output by the label predictor.

5. The efficient skin disease segmentation method based on multi-scale and hybrid attention mechanism according to claim 2 is characterized in that: In step 2.6), the Adam optimizer algorithm and the back propagation algorithm are used to iteratively update the network parameters θ to obtain the optimal network parameters; the Adam optimizer refers to an adaptive optimization algorithm that can adjust the learning rate according to historical gradient information; it combines the ideas of the traditional gradient descent and momentum update optimization algorithms, and normalizes the parameter updates so that each parameter update has a similar magnitude, thereby improving the training effect; the Adam optimizer can adaptively adjust the learning rate according to historical gradient information, without the need to manually adjust the learning rate, thereby improving the convergence speed and generalization ability of the model, while reducing the amount of computation, so that it can perform well on large-scale data sets.

6. The efficient skin disease segmentation method based on multi-scale and hybrid attention mechanism according to claim 5, characterized in that: In this step, when the Adam optimizer is used for iteration, the number of samples used in each iteration, i.e., the batch size, is set to 2, the initial learning rate is set to 1e-4, and the cosine regression learning rate decay technique is used, and the cycle is consistent with the total number of training rounds.

Images