Skin lesion segmentation method based on error positioning and adaptive optimization

Abstract

The invention relates to a skin lesion segmentation method based on error positioning and adaptive optimization, and the method comprises the steps: dividing a skin lesion image shot by a dermatoscope and a corresponding segmentation label into a training set, a verification set and a test set; then constructing a segmentation network, training the constructed segmentation network by adopting a training set, continuously monitoring the generalization ability of the segmentation network through a verification set in the training process, and finally obtaining a trained segmentation network; and finally, inputting the test set into the trained segmentation network, and outputting a segmentation result of the skin lesion. The skin lesion segmentation method based on error localization and adaptive optimization effectively focuses on a key area needing to be optimized in actual training, and helps the model to more accurately understand and distinguish the features of lesion and normal skin. Therefore, the segmentation capability of the method in processing complex lesions such as fuzzy boundaries, complex shapes and background noise is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of skin lesion diagnosis, and relates to a skin lesion segmentation method based on error positioning and adaptive optimization. Background Art

[0002] Skin lesion diagnosis usually includes image preprocessing, lesion segmentation, feature extraction and classification, among which lesion segmentation is the most basic and crucial step. Accurate segmentation of the lesion area not only helps to clarify the boundary, morphology and distribution of the lesion, but also lays the foundation for subsequent feature analysis and classification tasks.

[0003] Computer-aided diagnosis can actively assist dermatologists in clinical diagnosis and has broad prospects in helping doctors diagnose skin cancer earlier and more accurately, while reducing the workload of doctors. In recent years, with the development of deep learning technology, more and more studies have used deep learning methods such as convolutional neural networks to improve the performance of skin cancer segmentation. Compared with traditional medical image segmentation algorithms, deep learning image segmentation methods are more effective and stable.

[0004] In the past, most skin lesion segmentation methods were mainly based on improvements in encoder-decoder structures, such as introducing Transformer structures in the UNet framework, designing multi-scale feature extraction architectures, or adding spatial and channel attention mechanisms. However, the following difficulties in skin lesion samples lead to unsatisfactory overall segmentation effects of these models. First, the morphology of skin lesions is complex and diverse, with highly heterogeneous shapes, textures, and distributions, and lesion characteristics may vary significantly due to lesion types or individual differences. Secondly, the color and contrast inside the lesions are often uneven, and color gradients or low-contrast areas may appear, interfering with visual recognition. At the same time, the boundary between lesions and normal skin is usually blurred and difficult to separate clearly. In addition, since skin lesion image annotation requires professional medical knowledge and is usually performed by dermatologists, it is time-consuming and labor-intensive, which results in the relatively small scale of existing skin lesion image datasets, making it difficult to support high-quality model training and improve generalization performance.

[0005] To address these challenges, some methods attempt to introduce prior knowledge such as lesion contours and boundary key points to enhance the model's ability to perceive key information. This prior knowledge helps the model to segment more accurately under complex shapes and fuzzy boundary conditions by providing additional supervision signals. For example, the literature (Boundary-aware transformers for skinlesion segmentation [C] / / Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24. Springer International Publishing, 2021: 206-216. ) uses boundary key points extracted before training to supervise the network to focus on key lesion boundary areas, thereby improving the model's segmentation performance in boundary areas. However, such methods usually rely on pre-calculated fixed focus areas and lack the flexibility to dynamically adjust according to the actual performance of the model during training, which may ignore subtle changes in local features. In addition, existing models often fail to effectively detect and exclude potential errors when dealing with difficult-to-segment lesions. This neglect of error accumulation may lead to deviations in segmentation results in details, further limiting the performance and robustness of the model.

[0006] Therefore, it is of great significance to study a skin lesion segmentation method based on error localization and adaptive optimization, so as to effectively learn the subtle differences between lesions and normal skin with limited data, dynamically adjust the area of ​​focus of the model, and suppress the influence of potential errors on the segmentation results. Summary of the invention

[0007] The purpose of the present invention is to solve the problems existing in the prior art and to provide a skin lesion segmentation method based on error localization and adaptive optimization.

[0008] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0009] A skin lesion segmentation method based on error localization and adaptive optimization comprises the following steps:

[0010] (1) Divide the skin lesion images taken by the dermatoscope and the corresponding segmentation labels into training set, validation set and test set;

[0011] (2) Construct a segmentation network;

[0012] The segmentation network consists of an encoder and a decoder; the encoder contains 5 encoding stages, the first encoding stage consists of a CNN (convolutional neural network) module, the second to fourth encoding stages consist of a CNN module and a Transformer module, and the fifth encoding stage consists of a Transformer module; the first encoding stage only executes the CNN module and downsampling to perform preliminary feature extraction and dimensionality reduction on the original image, the feature maps of the second to fourth encoding stages are respectively input into the CNN module and the Transformer module to complete local and global information extraction, and the fifth encoding stage uses only the Transformer module due to its small resolution. The nsformer module processes the feature map encoded by the encoder and finally inputs it into the decoder; the decoder contains three decoding stages, and the feature map input in each decoding stage is processed by the error optimization unit, which includes the functions of error location and feature optimization; the feature map output by the fifth encoding stage is input to the first decoding stage, and the feature map output by the first decoding stage is input to the second decoding stage, and the feature map output by the second decoding stage is output to the third decoding stage. After the feature map output by the third decoding stage and the output feature map of the second encoding stage are jump-connected, the predicted segmentation map is output through the segmentation head. (i.e. the final segmentation result);

[0013] (3) Use the training set to train the constructed segmentation network, and continuously monitor the generalization ability of the segmentation network through the validation set during the training process, and finally obtain a trained segmentation network;

[0014] (4) Input the test set into the trained segmentation network and output the segmentation results of skin lesions.

[0015] Unlike traditional methods that simply rely on initial features or fixed feature extraction, this invention transforms conventional segmentation tasks into an adaptive error correction process, and makes full use of the error area information in the feedback prediction to dynamically adjust the focus and weight of feature extraction, and use accurate category feature information to help the model enhance feature representation. This design can not only effectively eliminate potential error areas in existing features, but also avoid the interference of error accumulation on subsequent decisions through iterative optimization, significantly improving the robustness and generalization ability of the model in complex scenarios.

[0016] As the preferred technical solution:

[0017] In the above-mentioned skin lesion segmentation method based on error localization and adaptive optimization, in step (1), the data in the divided training set is preprocessed and enhanced, and the data in the divided validation set and test set are preprocessed; the preprocessing includes adjusting the image size to Pixels; augmentation includes random rotation of 90 degrees with 50% probability, horizontal or vertical flipping, and random scaling operations with a scaling factor between 0.9 and 1.1.

[0018] As described above, in a skin lesion segmentation method based on error localization and adaptive optimization, after the segmentation network is constructed in step (2), first, for the input original image X, the encoder uses the CNN module and downsampling operation in the first encoding stage to extract image features and reduce the resolution to obtain a feature map , then Input to the CNN module of the second encoding stage; in the second encoding stage, the CNN module is first used to extract the local information map Then use the Transformer module to extract Global infographic of , and then and Fusion obtains the output feature map of the second encoding stage ; After downsampling, it is input to the CNN module of the third encoding stage. In the third encoding stage, the CNN module is first used to extract the local information map , then and The feature map formed by the fusion of downsampled features is input into the Transformer module to obtain the global information map , and then and The output feature map of the third encoding stage is obtained by fusion ; After downsampling, it is input into the CNN module of the fourth encoding stage. In the fourth encoding stage, the CNN module is first used to extract the local information map , then and The feature map formed by the fusion of downsampled features is input into the Transformer module to obtain the global information map , and then and The output feature map of the fourth encoding stage is obtained by fusion ; In the fifth encoding stage, Feature map after downsampling and The feature map formed by the fusion of downsampled features is input into the Transformer module to obtain the global information map , and then and Fusion is performed to obtain the output feature map of the fifth encoding stage ;

[0019] The calculation method of CNN module feature extraction is as follows:

[0020] ;

[0021] Where conv represents the convolution kernel size. convolution, BN means batch normalization, ReLU is the activation function, and down means downsampling;

[0022] The Transformer module in the second encoding stage first uses the multi-head self-attention mechanism to capture The global dependencies are obtained , and then use the feedforward network to enhance the nonlinear expression ability to obtain the final output of the Transformer module , the Transformer module is calculated as follows:

[0023] ;

[0024] ;

[0025] Among them, LN represents layer normalization, FFN represents a feedforward network containing two linear transformations and a ReLU activation function, and MHSA represents multi-head self-attention;

[0026] The Transformer module calculation method for the 3rd to 5th encoding stages is as follows:

[0027] ;

[0028] ;

[0029] Among them, concat means channel connection, conv1 means the convolution kernel size is Convolution;

[0030] The 2nd to 5th encoding stage fusion and Output feature map The calculation method is as follows:

[0031] ;

[0032] The inputs of the 1st to 3rd decoding stages in the decoder are feature maps , and ; Equal to the output feature map of the 5th encoding stage ; Feature map By feature map Feature map and feature map after processing by the error optimization unit Perform skip connection to obtain feature map By feature map Feature map and feature map after processing by the error optimization unit Perform skip connection to obtain feature map By feature map Feature map and feature map after processing by the error optimization unit Perform skip connection to obtain;

[0033] The process of inputting the feature map of each decoding stage into the error optimization unit for processing is as follows:

[0034] The feature map Input error optimization unit, in which error location is first performed, and the feature map Predicted coarse segmentation map Connect the original image X according to the channel dimension and input it into the UNet network to predict the error of the coarse segmentation map and output the predicted error area map ; Then, using the predicted error area map , coarse segmentation map and feature map Perform feature optimization to enhance feature maps separately The category features corresponding to the foreground lesions and the background normal skin are output, and the feature map processed by the error optimization unit is output; among them, .

[0035] As mentioned above, a skin lesion segmentation method based on error localization and adaptive optimization is used. The error localization in the error optimization unit is to predict the segmentation error area in the segmentation result through a UNet network trained synchronously with the overall segmentation network. The label used in the UNet network training is the coarse segmentation map. The sum of the areas that actually belong to the target category but are not predicted, and the non-target areas that are incorrectly predicted as the target category;

[0036] The feature optimization in the error optimization unit first refers to the predicted error area map Extracting foreground category maps and background category map ; Then the category map , , respectively, with the feature map Multiply pixel by pixel, retain the features of the matching category, and then average to get the category prototype ; Then, use Convolutional compression category prototype and feature map The channel dimension is and ; By combining two learnable weight matrices and Perform a linear transformation to obtain the key vector Sum value vector ; Through Multiply pixel by pixel to weight the error area, and then combine the weighted features with the learnable weight matrix Perform linear transformation to obtain the query vector ; Calculate the attention matrix according to the calculation formula of the attention mechanism , and and Point product to get the feature map of category enhancement ; Finally, And the feature maps of two categories enhancement and Connect with channels and After convolution fusion, the feature map processed by the error optimization unit is output;

[0037] Category diagram The calculation process is as follows: First, extract the coarse segmentation map In the area where the median value is greater than 0.5, the value of each pixel represents the confidence of the prediction as the foreground category; then extract In the area where the median value is less than or equal to 0.5, the value of each pixel represents the confidence of the prediction as the background category; then, according to the predicted error area map , reduce the prediction confidence of these areas by a weight of 0.1, and obtain the category map , .

[0038] As mentioned above, a skin lesion segmentation method based on error localization and adaptive optimization is and The specific calculation formula is:

[0039] ;

[0040] ;

[0041] in, , Indicates pixel-by-pixel multiplication;

[0042] The specific calculation formula is:

[0043] ;

[0044] in, is the indicator function, M represents the category map The number of pixels, Representation category diagram The mth pixel of Representation feature map The mth pixel of

[0045] , , , , , and The specific calculation formulas are:

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] ;

[0052] ;

[0053] Among them, e is and All-1 matrices of the same size, is a vector Dimension.

[0054] As mentioned above, a skin lesion segmentation method based on error localization and adaptive optimization, the feature map output by the third decoding stage is output through the segmentation head to predict the segmentation map The process is as follows: the segmentation head first transforms the feature map through the convolution layer The number of channels is mapped to a single-channel output, and then the single-channel feature map is upsampled to the original image size using transposed convolution. Finally, the sigmoid activation function is used to convert the value of each pixel into the probability of belonging to the foreground category, thereby obtaining the final segmentation map. .

[0055] In the above skin lesion segmentation method based on error localization and adaptive optimization, the training process of the segmentation network in step (3) is as follows:

[0056] First, the parameters of each layer of the segmentation network are randomly initialized, the Adam optimizer is selected, the initial learning rate is set to 0.0001, and 100 rounds of training are performed with a small batch size of 8;

[0057] In each training round, forward propagation is first performed to input the images in the training set into the segmentation network, and the predicted coarse segmentation map is generated in the decoding stage. The decoded features are used to predict the final segmentation map through the segmentation head ;

[0058] Coarse segmentation map predicted at each decoding stage , calculate the error area label of the UNet network in the training error positioning part :

[0059] ;

[0060] Then calculate the total loss , The total loss including the predicted segmentation map and the total loss of the predicted error area map ; Since the constructed segmentation network uses deep supervision, the total loss It is the weighted sum of the losses of the predicted segmentation maps of four different scales according to the size of the coarse segmentation map; The calculation formula is as follows:

[0061] ;

[0062] Among them, Focal Loss is used to calculate the loss ; The calculation formula is as follows:

[0063] ;

[0064] Using binary cross entropy loss Sum intersection loss Calculate the segmentation loss of each predicted segmentation map, and weight its loss value according to the size of the predicted segmentation map at different scales, and then sum it up to get ; The calculation formula is as follows:

[0065] ;

[0066] ;

[0067] ;

[0068] in, is the i-th wrong region label of the k-th sample, is the error area map of the i-th prediction of the k-th sample, N is the batch size, N=8; α is set to 0.25 to balance the number of categories, and γ is set to 2 to balance the degree of error; represents the i-th segmentation label of the k-th sample, represents the i-th predicted segmentation map of the k-th sample, Set to 1, 0.5, 0.4, and 0.3, indicating the weight added when calculating the loss for predictions of different resolutions, and ϵ is set to 1e-6 to prevent the denominator from being zero;

[0069] Finally, according to The chain rule is used to calculate the gradients of each layer in the network, and the gradients are passed to each layer of the network through back propagation. Then, the Adam optimizer is used to update the parameters in the network based on the gradients obtained through back propagation.

[0070] After every 10 rounds of training, the generalization ability of the segmentation network is checked using the validation set, and its performance indicators on the validation set are calculated, including DSC, IoU, and Acc; if these indicators do not improve or show a downward trend on the validation set, the learning rate is halved to further optimize the segmentation network.

[0071] Beneficial effects:

[0072] (1) The present invention provides a skin lesion segmentation method based on error localization and adaptive optimization, and proposes an encoder that extracts local and global feature information at different resolution levels to provide a richer feature expression for the decoder.

[0073] (2) The present invention proposes a skin lesion segmentation method based on error localization and adaptive optimization. An error localization process is proposed and integrated into the network decoder, which can identify the areas where the model prediction is wrong to reveal the current performance limitations.

[0074] (3) The present invention proposes a skin lesion segmentation method based on error localization and adaptive optimization, proposes a feature optimization process and integrates it into the network decoder; combines the segmentation error map generated by the error localization process, analyzes reliable category areas, and provides support for subsequent predictions by aggregating more accurate category feature prototypes, thereby reducing the impact of errors and refining the segmentation results.

[0075] (4) The present invention provides a skin lesion segmentation method based on error localization and adaptive optimization. By performing error localization and feature optimization at each decoding stage, the method can not only identify error areas at multiple scales, but also use error maps as feedback on learning effects to further optimize category understanding. This mechanism effectively focuses on key areas that need to be optimized in actual training, helping the model to more accurately understand and distinguish the characteristics of lesions from normal skin, thereby improving its segmentation ability when dealing with complex lesions such as fuzzy boundaries, complex shapes, and background noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 Figure 1. Network structure diagram for skin lesion segmentation based on error localization and adaptive optimization.

[0077] Figure 2 It is the structure diagram of Transformer module and CNN module;

[0078] Figure 3 Optimize unit structure diagram for errors. DETAILED DESCRIPTION

[0079] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0080] A skin lesion segmentation method based on error localization and adaptive optimization comprises the following steps:

[0081] (1) The skin lesion images taken by the dermatoscope and the corresponding segmentation labels are divided into a training set, a validation set, and a test set. The data in the divided training set is then preprocessed and enhanced, and the data in the validation set and the test set are preprocessed;

[0082] (2) Construct a segmentation network;

[0083] The segmentation network consists of an encoder and a decoder; the encoder contains 5 encoding stages, the first encoding stage consists of a CNN module, the second to fourth encoding stages consist of a CNN module and a Transformer module, and the fifth encoding stage consists of a Transformer module; the decoder contains 3 decoding stages, and the feature map input to each decoding stage is processed by an error optimization unit, which includes the functions of error location and feature optimization; the feature map output by the fifth encoding stage is input to the first decoding stage, the feature map output by the first decoding stage is input to the second decoding stage, and the feature map output by the second decoding stage is output to the third decoding stage. After the feature map output by the third decoding stage and the output feature map of the second encoding stage are jump-connected, the predicted segmentation map is output through the segmentation head. ;

[0084] The feature map output by the third decoding stage is used to output the predicted segmentation map through the segmentation head The process is as follows: the segmentation head first transforms the feature map through the convolution layer The number of channels is mapped to a single-channel output, and then the single-channel feature map is upsampled to the original image size using transposed convolution. Finally, the sigmoid activation function is used to convert the value of each pixel into the probability of belonging to the foreground category, thereby obtaining the final segmentation map. ;

[0085] After constructing the segmentation network, Figure 1 As shown in Figure 1, first, for the input original image X, the encoder uses the CNN 1 module and downsampling operation in the first encoding stage to extract image features and reduce the resolution to obtain the feature map , then Input to the CNN 2 module of the second encoding stage; in the second encoding stage, the CNN 2 module is first used to extract the local information map , and then use the Transformer 1 module to extract Global infographic of , and then and Fusion obtains the output feature map of the second encoding stage ; After downsampling, it is input to the CNN 3 module in the third encoding stage. In the third encoding stage, the CNN 3 module is first used to extract the local information map , then and The feature map formed by the fusion of downsampled features is input into the Transformer 2 module to obtain the global information map , and then and The output feature map of the third encoding stage is obtained by fusion ; After downsampling, it is input to the CNN 4 module of the fourth encoding stage. In the fourth encoding stage, the CNN 4 module is first used to extract the local information map , then and The feature map formed by the fusion of downsampled features is input into the Transformer 3 module to obtain the global information map , and then and The output feature map of the fourth encoding stage is obtained by fusion ; In the fifth encoding stage, Feature map after downsampling and The feature map formed by the fusion of downsampled features is input into the Transformer 4 module to obtain the global information map , and then and Fusion is performed to obtain the output feature map of the fifth encoding stage ;

[0086] like Figure 2 As shown, the calculation method of CNN module feature extraction is as follows:

[0087] ;

[0088] Where conv represents the convolution kernel size. convolution, BN means batch normalization, ReLU is the activation function, and down means downsampling;

[0089] The Transformer module in the second encoding stage first uses the multi-head self-attention mechanism to capture The global dependencies are obtained , and then use the feedforward network to enhance the nonlinear expression ability to obtain the final output of the Transformer module , the Transformer module is calculated as follows:

[0090] ;

[0091] ;

[0092] Among them, LN (Layer Norm) represents layer normalization, FFN represents a feedforward network containing two linear transformations and a ReLU activation function, and MHSA represents multi-head self-attention;

[0093] The Transformer module calculation method for the 3rd to 5th encoding stages is as follows:

[0094] ;

[0095] ;

[0096] Among them, concat means channel connection, conv1 means the convolution kernel size is Convolution;

[0097] The 2nd to 5th encoding stage fusion and Output feature map The calculation method is as follows:

[0098] ;

[0099] The inputs of the 1st to 3rd decoding stages in the decoder are feature maps , and ; Equal to the output feature map of the 5th encoding stage ; Feature map By feature map Feature map and feature map after processing by the error optimization unit Perform skip connection to obtain feature map By feature map Feature map and feature map after processing by the error optimization unit Perform skip connection to obtain feature map By feature map Feature map and feature map after processing by the error optimization unit Perform skip connection to obtain;

[0100] The process of inputting the feature map of each decoding stage into the error optimization unit for processing is as follows:

[0101] (2.1) If Figure 3 As shown, the feature map Input error optimization unit, in the error optimization unit, first use the feature map Predict the rough segmentation map ,right Perform error localization;

[0102] Error localization is done by predicting the segmentation error areas in the segmentation results through a UNet network trained synchronously with the overall segmentation network. The labels used in UNet network training are the coarse segmentation maps. The sum of the areas that actually belong to the target category but are not predicted, and the non-target areas that are incorrectly predicted as the target category;

[0103] (2.2) The coarse segmentation map Connect the original image X according to the channel dimension and input it into the UNet network to predict the error of the coarse segmentation map and output the predicted error area map ;

[0104] (2.3) Using the predicted error area map , coarse segmentation map and feature map Perform feature optimization to enhance feature maps separately The category features corresponding to the foreground lesions and the background normal skin are output, and the feature map processed by the error optimization unit is output; among them, ;

[0105] The feature optimization in the error optimization unit first refers to the predicted error area map Extracting foreground category maps and background category map , f represents the foreground, b represents the background, and then the category map , , respectively, with the feature map Multiply pixel by pixel, retain the features of the matching category, and then average to get the category prototype ; Then, use Convolutional compression category prototype and feature map The channel dimension is and ; By combining two learnable weight matrices and Perform a linear transformation to obtain the key vector Sum value vector , K represents the key vector, V represents the value vector; Through Multiply pixel by pixel to weight the error area, and then combine the weighted features with the learnable weight matrix Perform linear transformation to obtain the query vector ; Calculate the attention matrix according to the calculation formula of the attention mechanism , and and Point product to get the feature map of category enhancement ; Finally, And the feature maps of two categories enhancement and Connect with channels and After convolution fusion, the feature map processed by the error optimization unit is output;

[0106] and The specific calculation formula is:

[0107] ;

[0108] ;

[0109] in, , Indicates pixel-by-pixel multiplication;

[0110] The specific calculation formula is:

[0111] ;

[0112] in, is the indicator function, M represents the category map The number of pixels, Representation category diagram The mth pixel of Representation feature map The mth pixel of

[0113] , , , , , and The specific calculation formulas are:

[0114] ;

[0115] ;

[0116] ;

[0117] ;

[0118] ;

[0119] ;

[0120] ;

[0121] Among them, e is and All-1 matrices of the same size, is a vector Dimensions;

[0122] Category diagram The calculation process is as follows: First, extract the coarse segmentation map In the area where the median value is greater than 0.5, the value of each pixel represents the confidence of the prediction as the foreground category; then extract In the area where the median value is less than or equal to 0.5, the value of each pixel represents the confidence of the prediction as the background category; then, according to the predicted error area map , reduce the prediction confidence of the wrong prediction area according to the weight of 0.1, and get the category map , ;

[0123] (3) Use the training set to train the constructed segmentation network, and continuously monitor the generalization ability of the segmentation network through the validation set during the training process, and finally obtain a trained segmentation network;

[0124] The training process of the segmentation network is as follows:

[0125] First, the parameters of each layer of the segmentation network are randomly initialized, the Adam optimizer is selected, the initial learning rate is set to 0.0001, and 100 rounds of training are performed with a small batch size of 8;

[0126] In each training round, forward propagation is first performed to input the images in the training set into the segmentation network, and the predicted coarse segmentation map is generated in the decoding stage. The decoded features are used to predict the final segmentation map through the segmentation head ;

[0127] Coarse segmentation map predicted at each decoding stage , calculate the error area label of the UNet network in the training error positioning part :

[0128] ;

[0129] Then calculate the total loss , The total loss including the predicted segmentation map and the total loss of the predicted error area map ; The calculation formula is as follows:

[0130] ;

[0131] Among them, Focal Loss is used to calculate the loss ; The calculation formula is as follows:

[0132] ;

[0133] Using binary cross entropy loss Sum intersection loss Calculate the segmentation loss of each predicted segmentation map, and weight its loss value according to the size of the predicted segmentation map at different scales, and then sum it up to get ; The calculation formula is as follows:

[0134] ;

[0135] ;

[0136] ;

[0137] in, is the i-th wrong region label of the k-th sample, is the error area map of the i-th prediction of the k-th sample, N is the batch size, N=8; α is set to 0.25 to balance the number of categories, and γ is set to 2 to balance the degree of error; represents the i-th segmentation label of the k-th sample, represents the i-th predicted segmentation map of the k-th sample, Set to 1, 0.5, 0.4, and 0.3, indicating the weight added when calculating the loss for predictions of different resolutions, and ϵ is set to 1e-6 to prevent the denominator from being zero;

[0138] Finally, according to The chain rule is used to calculate the gradients of each layer in the network, and the gradients are passed to each layer of the network through back propagation. Then, the Adam optimizer is used to update the parameters in the network based on the gradients obtained through back propagation.

[0139] After every 10 rounds of training, the generalization ability of the segmentation network is checked using the validation set, and its performance indicators on the validation set are calculated, including DSC (Dice Similarity Coefficient), Acc (Accuracy), and IoU (Intersection over Union). If these indicators do not improve or show a downward trend on the validation set, the learning rate is halved to further optimize the segmentation network.

[0140] (4) Input the test set into the trained segmentation network and output the segmentation results of skin lesions.

[0141] Specifically, the present invention conducts experiments on two datasets, ISIC 2017 and ISIC 2018. The ISIC 2017 dataset is randomly divided into 1250 training samples, 150 validation samples, and 600 test samples. The ISIC 2018 dataset is randomly divided into 1815 training samples, 259 validation samples, and 520 test samples. The preprocessing includes resizing the image to Pixels; enhancement includes random rotation of 90 degrees with a probability of 50%, horizontal or vertical flipping, and random scaling operations with a scaling ratio between 0.9 and 1.1. The comparison of the effects of the present invention and other skin lesion segmentation methods in the prior art is shown in Table 1, and statistical analysis is performed on the three indicators of DSC, Acc, and IoU. Assume and are the foreground and background areas predicted by the model, and They are the segmentation labels of the real foreground area and the background area respectively. The calculation formulas of these three indicators are as follows:

[0142] ;

[0143] ;

[0144] ;

[0145] DSC focuses more on measuring the segmentation performance of the model for lesion categories. Acc considers the prediction accuracy of all pixels, which reflects the accuracy of the overall segmentation. IoU focuses on boundary segmentation by evaluating the spatial overlap quality of the predicted area and the true area.

[0146] It can be seen from Table 1 that the present invention achieves the best comprehensive results on both data sets.

[0147] Table 1

[0148] .

[0149] The specific literature information of the prior art [1]~[5] is as follows:

[0150] [1] U-net: Convolutional networks for biomedical image segmentation[C] / / Medical image computing and computer-assisted intervention–MICCAI 2015:18th international conference, Munich, Germany, October 5-9, 2015,proceedings, part III 18. Springer International Publishing, 2015: 234-241.

[0151] [2] Attention gated networks: Learning to leverage salient regions inmedical images[J]. Medical image analysis, 2019, 53: 197-207.

[0152] [3] Transunet: Transformers make strong encoders for medical imagesegmentation[J]. arXiv preprint arXiv:2102.04306, 2021.

[0153] [4] Medical transformer: Gated axial-attention for medical imagesegmentation[C] / / Medical image computing and computer assisted intervention–MICCAI 2021: 24th international conference, Strasbourg, France, September 27–October 1, 2021, proceedings, part I 24. Springer International Publishing,2021: 36-46.

[0154] [5] Hiformer: Hierarchical multi-scale representations usingtransformers for medical image segmentation[C] / / Proceedings of the IEEE / CVFwinter conference on applications of computer vision. 2023: 6202-6212.

Claims

1. A skin lesion segmentation method based on error localization and adaptive optimization, characterized in that The steps include: (1) Divide the skin lesion images taken by the dermatoscope and the corresponding segmentation labels into training set, validation set and test set; (2) Construct a segmentation network; The segmentation network consists of an encoder and a decoder; the encoder contains 5 encoding stages, the first encoding stage consists of a CNN module, the second to fourth encoding stages consist of a CNN module and a Transformer module, and the fifth encoding stage consists of a Transformer module; the decoder contains 3 decoding stages, and the feature map input to each decoding stage is processed by an error optimization unit, which includes the functions of error location and feature optimization; the feature map output by the fifth encoding stage is input to the first decoding stage, the feature map output by the first decoding stage is input to the second decoding stage, and the feature map output by the second decoding stage is output to the third decoding stage. After the feature map output by the third decoding stage and the output feature map of the second encoding stage are jump-connected, the predicted segmentation map is output through the segmentation head. ; (3) Use the training set to train the constructed segmentation network, and continuously monitor the generalization ability of the segmentation network through the validation set during the training process, and finally obtain a trained segmentation network; (4) Input the test set into the trained segmentation network and output the segmentation results of skin lesions.

2. The skin lesion segmentation method based on error localization and adaptive optimization according to claim 1, characterized in that: In step (1), the data in the training set is preprocessed and enhanced, and the data in the validation set and the test set are preprocessed.

3. The skin lesion segmentation method based on error localization and adaptive optimization according to claim 1, characterized in that: Step (2) After the segmentation network is constructed, first, for the input original image X, the encoder uses the CNN module and downsampling operation in the first encoding stage to extract image features and reduce the resolution to obtain the feature map , then Input to the CNN module of the second encoding stage; in the second encoding stage, the CNN module is first used to extract the local information map Then use the Transformer module to extract Global infographic of , and then and Fusion obtains the output feature map of the second encoding stage ; After downsampling, it is input to the CNN module of the third encoding stage. In the third encoding stage, the CNN module is first used to extract the local information map , then and The feature map formed by the fusion of downsampled features is input into the Transformer module to obtain the global information map , and then and The output feature map of the third encoding stage is obtained by fusion ; After downsampling, it is input into the CNN module of the fourth encoding stage. In the fourth encoding stage, the CNN module is first used to extract the local information map , then and The feature map formed by the fusion of downsampled features is input into the Transformer module to obtain the global information map , and then and The output feature map of the fourth encoding stage is obtained by fusion ; In the fifth encoding stage, Feature map after downsampling and The feature map formed by the fusion of downsampled features is input into the Transformer module to obtain the global information map , and then and Fusion is performed to obtain the output feature map of the fifth encoding stage ; The calculation method of CNN module feature extraction is as follows: ； Where conv represents the convolution kernel size. Convolution, BN means batch normalization, ReLU is the activation function, and down means downsampling; The Transformer module in the second encoding stage first uses the multi-head self-attention mechanism to capture The global dependencies are obtained , and then use the feedforward network to enhance the nonlinear expression ability to obtain the final output of the Transformer module , the Transformer module is calculated as follows: ； ； Among them, LN represents layer normalization, FFN represents a feedforward network containing two linear transformations and a ReLU activation function, and MHSA represents multi-head self-attention; The Transformer module calculation method for the 3rd to 5th encoding stages is as follows: ； ； Among them, concat means channel connection, conv1 means the convolution kernel size is Convolution; The 2nd to 5th encoding stage fusion and Output feature map The calculation method is as follows: ； The inputs of the 1st to 3rd decoding stages in the decoder are feature maps , and ; Equal to the output feature map of the 5th encoding stage ; Feature map By feature map Feature map and feature map after processing by the error optimization unit Perform skip connection to obtain feature map By feature map Feature map and feature map after processing by the error optimization unit Perform skip connection to obtain feature map By feature map Feature map and feature map after processing by the error optimization unit Perform skip connection to obtain; The process of inputting the feature map of each decoding stage into the error optimization unit for processing is as follows: The feature map Input error optimization unit, in which error location is first performed, and the feature map Predicted coarse segmentation map Connect the original image X according to the channel dimension and input it into the UNet network to predict the error of the coarse segmentation map and output the predicted error area map ; Then, using the predicted error area map , coarse segmentation map and feature map Perform feature optimization to enhance feature maps separately The category features corresponding to the foreground lesions and the background normal skin are output, and the feature map processed by the error optimization unit is output; among them, .

4. The skin lesion segmentation method based on error localization and adaptive optimization according to claim 3, characterized in that: Error localization in the error optimization unit is done by predicting the segmentation error areas in the segmentation results through a UNet network trained synchronously with the overall segmentation network. The labels used in the UNet network training are the coarse segmentation maps. The sum of the areas that actually belong to the target category but are not predicted, and the non-target areas that are incorrectly predicted as the target category; The feature optimization in the error optimization unit first refers to the predicted error area map Extracting foreground category maps and background category map ; Then the category map , , respectively, with the feature map Multiply pixel by pixel, retain the features of the matching category, and then average to get the category prototype ; Then, use Convolutional compression category prototype and feature map The channel dimension is and ; By combining two learnable weight matrices and Perform a linear transformation to obtain the key vector Sum value vector ; Through Multiply pixel by pixel to weight the error area, and then combine the weighted features with the learnable weight matrix Perform linear transformation to obtain the query vector ; Calculate the attention matrix according to the calculation formula of the attention mechanism , and and Point product to get the feature map of category enhancement ; Finally, And the feature maps of two categories enhancement and Connect with channels and After convolution fusion, the feature map processed by the error optimization unit is output; Category diagram The calculation process is as follows: First, extract the coarse segmentation map In the area where the median value is greater than 0.5, the value of each pixel represents the confidence of the prediction as the foreground category; then extract In the area where the median value is less than or equal to 0.5, the value of each pixel represents the confidence of the prediction as the background category; then, according to the predicted error area map , reduce the prediction confidence of these areas by a weight of 0.1, and obtain the category map , .

5. The skin lesion segmentation method based on error localization and adaptive optimization according to claim 4, characterized in that: and The specific calculation formula is: ； ； in, , Indicates pixel-by-pixel multiplication; The specific calculation formula is: ； in, is the indicator function, M represents the feature map The number of pixels, Representation category diagram The mth pixel of Representation feature map The mth pixel of , , , , , and The specific calculation formulas are: ； ； ； ； ； ； ； Among them, e is and All-1 matrices of the same size, is a vector Dimension.

6. The skin lesion segmentation method based on error localization and adaptive optimization according to claim 5, characterized in that: The feature map output by the third decoding stage is used to output the predicted segmentation map through the segmentation head The process is as follows: the segmentation head first transforms the feature map through the convolution layer The number of channels is mapped to a single-channel output, and then the single-channel feature map is upsampled to the original image size using transposed convolution. Finally, the sigmoid activation function is used to convert the value of each pixel into the probability of belonging to the foreground category, thereby obtaining the final segmentation map. .

7. The skin lesion segmentation method based on error localization and adaptive optimization according to claim 6, characterized in that: The training process of the segmentation network in step (3) is as follows: First, the parameters of each layer of the segmentation network are randomly initialized, the Adam optimizer is selected, the initial learning rate is set to 0.0001, and 100 rounds of training are performed with a small batch size of 8; In each training round, forward propagation is first performed to input the images in the training set into the segmentation network, and the predicted coarse segmentation map is generated in the decoding stage. The decoded features are used to predict the final segmentation map through the segmentation head ; Coarse segmentation map predicted at each decoding stage , calculate the error area label of the UNet network in the training error positioning part : ； Then calculate the total loss , The total loss including the predicted segmentation map and the total loss of the predicted error area map ; The calculation formula is as follows: ； ； ； ； ； in, is the i-th wrong region label of the k-th sample, is the error area map of the i-th prediction of the k-th sample, N is the batch size, N=8; α is set to 0.25 to balance the number of categories, and γ is set to 2 to balance the degree of error; represents the i-th segmentation label of the k-th sample, represents the i-th predicted segmentation map of the k-th sample, Set to 1, 0.5, 0.4, and 0.3, indicating the weight added when calculating the loss for predictions of different resolutions, and ϵ is set to 1e-6 to prevent the denominator from being zero; Finally, according to The chain rule is used to calculate the gradients of each layer in the network, and the gradients are passed to each layer of the network through back propagation. Then, the Adam optimizer is used to update the parameters in the network based on the gradients obtained through back propagation. After every 10 rounds of training, the generalization ability of the segmentation network is checked using the validation set, and its performance indicators on the validation set are calculated; if these indicators do not improve or show a downward trend on the validation set, the learning rate is halved to further optimize the segmentation network.

Images