Skin Melanoma Image Segmentation Network Structure and Method Based on Improved MultiResUNet

Abstract

The present invention provides a skin melanoma image segmentation network structure and method based on an improved MultiResUNet. The network structure includes an encoder module and a decoder module, and replaces the MultiRes Block in the original MultiResUNet network with a new MultiRes Block to form a new MultiResUNet network; wherein, the new MultiRes Block includes a first residual layer and three stacked Conv2d layers with different dilation rates; after the outputs of the three Conv2d layers are subjected to a concat operation, a feature addition operation is performed with the output result of the first residual layer. The present invention expands the receptive field, which helps to segment targets of different sizes; the addition of a dual attention mechanism to readjust the feature weights helps to segment targets at different positions and with different shapes, improving the segmentation accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and particularly relates to a skin melanoma image segmentation network structure and method based on an improved MultiResUNet. Background Art

[0002] Cutaneous melanoma is a highly malignant tumor derived from melanocyte lesions. Melanoma has a high fatality rate and often metastasizes. At present, there is no specific treatment method and means for melanoma except for early surgical resection. Therefore, early diagnosis and treatment are very important.

[0003] With the continuous development of deep learning, the method of using deep learning to assist medicine has become increasingly popular. Among them, the U-Net model proposed by Olaf Ronneberger et al. in 2015 (Reference 1: Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation [C] / / International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 2015: 234-241.) has become one of the popular segmentation network models in the field of medical image segmentation. However, the shapes, sizes, and locations of cutaneous melanomas are different, and the U-Net cannot fully meet the requirements of cutaneous melanoma image segmentation. Aiming at the small receptive field of the U-Net and the differences in the fusion of deep and shallow network features, Ibtehaz N et al. proposed MultiResUNet (Reference 2: Ibtehaz N, Rahman MS. MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation [J]. Neural Networks, 2020, 121: 74-87.), which has achieved good results in the task of cutaneous melanoma image segmentation. However, the network still has problems such as a small receptive field and an unclear relationship between spatial and channel features. Summary of the Invention

[0004] To solve or partially solve the problems of small receptive field and unclear relationship between spatial and channel features existing in the existing MultiResUNet model, the present invention provides a skin melanoma image segmentation network structure and method based on an improved MultiResUNet.

[0005] In a first aspect, the present invention provides a skin melanoma image segmentation network structure based on an improved MultiResUNet, including: an encoder module and a decoder module, replacing the MultiRes Block in the original MultiResUNet network with a new MultiRes Block to form a new MultiResUNet network; wherein, the new MultiRes Block includes a first residual layer and three stacked Conv2d layers with different dilation rates; after the outputs of the three Conv2d layers are concatenated, a feature addition operation is performed with the output result of the first residual layer.

[0006] Further, the dilation rates of the three Conv2d layers are 1, 2, and 5 in sequence from shallow to deep.

[0007] Further, the encoder module includes 5 new MultiResBlocks in sequence from shallow to deep; the decoder module includes 4 new MultiRes Blocks in sequence from shallow to deep; wherein, an att-Block module is arranged between each MultiRes Block in the first 4 layers of the encoder module and the corresponding layer MultiResBlock in the decoder module; the att-Block module is used to reassign weights to the features output by each MultiRes Block in the encoder module.

[0008] Further, an upsampling module is arranged between the last layer MultiRes Block in the encoder module and the first layer MultiRes Block in the decoder module, and between adjacent two layers MultiRes Blocks in the decoder module;

[0009] A residual module is arranged between each MultiRes Block in the first 4 layers of the encoder module and its corresponding att-Block module;

[0010] The att-Block module reallocates weights for the features output by the corresponding layer of the MultiRes Block in the encoder module according to the output of the upsampling module and the output of the residual module.

[0011] Furthermore, the number of residual modules corresponding to each layer of the MultiRes Block in the first 4 layers of the encoder module is 4, 3, 2, and 1 in sequence from the shallow layer to the deep layer.

[0012] Furthermore, the att-Block module includes a channel attention CA module and a spatial attention SA module; after the same input data passes through the CA module and the SA module respectively, the outputs of the CA module and the SA module are added together.

[0013] Furthermore, the CA module includes two CA units with the same structure, and each CA unit includes an adaptive global average pooling layer and a convolutional layer connected in sequence;

[0014] One of the CA units receives and processes the low-level features from the encoder module in the input data, and the other CA unit receives and processes the high-level features from the decoder module;

[0015] After the outputs of the two CA units are added together and passed through the Softmax activation function, channel weights are obtained; the low-level features from the encoder module in the input data are multiplied by the channel weights and then added to the high-level features from the decoder module in the input data to obtain the output features, which are denoted as the output of the CA module.

[0016] Furthermore, the SA module includes three convolutional layers;

[0017] Two of the convolutional layers each receive and process the low-level features from the encoder module and the high-level features from the decoder module in the input data;

[0018] After the outputs of the two convolutional layers are added together and passed through the third convolutional layer, and then passed through the BN layer and the Sigmoid activation function in sequence, spatial weights are obtained; the low-level features from the encoder module in the input data are multiplied by the spatial weights to obtain the output features, which are denoted as the output of the SA module.

[0019] Furthermore, the residual module includes an ordinary convolutional layer and a second residual layer; the ordinary convolutional layer includes a convolutional layer, a BN layer, and a LeakyReLU layer connected in sequence; the second residual layer includes a convolutional layer and a BN layer connected in sequence.

[0020] Second aspect, the present invention provides a skin melanoma image segmentation method based on an improved MultiResUNet, including:

[0021] Construct and train the above-mentioned skin melanoma image segmentation network based on the improved MultiResUNet;

[0022] Segment the input image using the trained skin melanoma image segmentation network.

[0023] Advantages of the present invention:

[0024] The skin melanoma image segmentation network structure and method based on the improved MultiResUNet provided by the present invention address the problems of the original MultiResUNet network having a small receptive field or unclear channel and spatial relationships. By replacing the ordinary convolutional kernels with convolutional kernels of different dilation rates in its original MultiRes Block, the receptive field is expanded, which helps to segment targets of different sizes; the dual attention mechanism is added to readjust the feature weights, which helps to segment targets at different positions and of different shapes, improving the segmentation accuracy; on the basis of the entire framework, the parameters during network training are adjusted to train the model to the optimal state, optimizing the segmentation of lesions with different morphologies, sizes, and blurred boundaries. Description of the Drawings

[0025] Figure 1 One of the structural schematic diagrams of the new MultiRes Block provided by the embodiment of the present invention;

[0026] Figure 2 One of the network structure diagrams of the skin melanoma image segmentation model based on the improved MultiResUNet provided by the embodiment of the present invention;

[0027] Figure 3 Another network structure diagram of the skin melanoma image segmentation model based on the improved MultiResUNet provided by the embodiment of the present invention;

[0028] Figure 4 The structural schematic diagram of the att-Block module provided by the embodiment of the present invention;

[0029] Figure 5 Provided by the embodiment of the present invention Figure 4 One of the structural schematic diagrams of the CA module and the SA module;

[0030] Figure 6 Another network structure diagram of the skin melanoma image segmentation network based on the improved MultiResUNet provided by the embodiment of the present invention;

[0031] Figure 7Schematic diagram II of the new MultiRes Block provided by the embodiment of the present invention;

[0032] Figure 8 Provided by the embodiment of the present invention Figure 4 Schematic diagram II of the CA module and SA module in;

[0033] Figure 9 Comparison of the segmentation result map and mask based on the existing MultiResUNet network;

[0034] Figure 10 Comparison of the segmentation result map and mask based on the improved MultiResUNet provided by the embodiment of the present invention. Detailed implementation manners

[0035] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0036] Embodiment 1

[0037] Aiming at the problem that the existing MultiResUNet network has too small receptive field, the embodiment of the present invention provides a skin melanoma image segmentation network structure based on an improved MultiResUNet. The main difference from the original MultiResUNet network is that the MultiRes Block therein is improved in the embodiment of the present invention, and then the improved new MultiResBlock is used to replace the original MultiRes Block, thereby forming a new MultiResUNet network.

[0038] As Figure 1 shown, the new MultiRes Block includes a first residual layer and three stacked Conv2d layers with different dilation rates; after the outputs of the three Conv2d layers are subjected to a concat operation, a feature addition operation is performed with the output result of the first residual layer.

[0039] As an implementable manner, the dilation rates of the three Conv2d layers are 1, 2, and 5 in sequence from the shallow layer to the deep layer.

[0040] In the MultiRes Block of the MultiResUNet backbone network, this segmentation network uses convolutional layers with different dilation rates to give full play to the advantage of dilated convolution in effectively expanding the receptive field. By expanding the receptive field, the target image is segmented, which helps to segment targets of different sizes.

[0041] Embodiment 2

[0042] As Figure 2 shown, according to the structural composition of the original MultiResUNet network, the new MultiResUNet network also mainly includes two parts: an encoder module and a decoder module; on the basis of the above embodiment, in order to further solve the problem that the channel and spatial relationships in the existing MultiResUNet network are not obvious, the embodiment of the present invention adds an att-Block module with a dual attention mechanism to re-integrate the channel and spatial feature weights. The specific structural design is as follows:

[0043] In the embodiment of the present invention, the encoder module includes 5 new MultiRes Blocks from shallow to deep; the decoder module includes 4 new MultiRes Blocks from shallow to deep; among them, an att-Block module is arranged between each MultiRes Block in the first 4 layers of the encoder module and the corresponding layer MultiRes Block in the decoder module; the att-Block module is used to re-allocate the weights of the features output by each MultiRes Block in the encoder module.

[0044] By adding the att-Block module, the embodiment of the present invention re-integrates the channel and spatial feature weights, improves the performance of the network, and strengthens the extraction of effective features of skin melanoma images, providing further possibilities for computer-aided diagnosis and treatment of skin melanoma lesions.

[0045] Embodiment 3

[0046] As Figure 3As shown, on the basis of Embodiment 2, an upsampling module is provided between the last layer of the MultiRes Block in the encoder module and the first layer of the MultiRes Block in the decoder module, and between adjacent two layers of the MultiRes Block in the decoder module, so as to unify the deep feature resolution size and the number of channels with the corresponding module in the encoder to facilitate the input of the dual attention module to reallocate the feature weights. Since there are large differences in the semantic information contained in the high-level and low-level features, directly splicing the features will cause information loss. Therefore, it is necessary to perform a convolution operation on the shallow features to reduce the differences between the shallow and deep features, and the residual structure can avoid feature loss caused during the convolution process. Therefore, a residual module is added before splicing the shallow and deep features. A residual module is provided between each layer of the MultiRes Block in the first 4 layers of the encoder module and its corresponding att-Block module; the att-Block module reallocates the weights of the features output by the corresponding layer of the MultiRes Block in the encoder module according to the output of the upsampling module and the output of the residual module.

[0047] As an implementable manner, since the UNet structure is a symmetric structure, the degree of difference between the shallow and deep features corresponding to each layer in the UNet is different, so the set residual modules are also different. The difference in the first layer is the largest and multiple residual modules are required. The difference in the second layer is small and fewer residual modules are required than in the first layer, and so on. The number of residual modules corresponding to each layer of the MultiRes Block in the first 4 layers of the encoder module is 4, 3, 2, and 1 in sequence from shallow to deep.

[0048] As an implementable manner, the residual module includes an ordinary convolution layer and a second residual layer; the ordinary convolution layer includes a convolution layer, a BN layer, and a LeakyReLU layer connected in sequence; the second residual layer includes a convolution layer and a BN layer connected in sequence.

[0049] Embodiment 4

[0050] As Figure 4 shown, an embodiment of the present invention provides an att-Block module, which includes a channel attention CA module and a spatial attention SA module; after the same input data passes through the CA module and the SA module respectively, the outputs of the CA module and the SA module are added.

[0051] As Figure 5 shown, as an implementable manner, the CA module includes two CA units with the same structure, and each CA unit includes an adaptive global average pooling layer and a convolution layer connected in sequence;

[0052] One of the CA units receives and processes the low-level features from the encoder module in the input data, and the other CA unit receives and processes the high-level features from the decoder module; wherein, the pooling layer performs a pooling operation, and the convolutional layer performs a convolutional operation.

[0053] The outputs of the two CA units are added and then passed through the Softmax activation function to obtain the channel weights; the low-level features from the encoder module in the input data are multiplied by the channel weights and then added to the high-level features from the decoder module in the input data to obtain the output features, denoted as the output of the CA module.

[0054] As Figure 5 shown, as an implementable manner, the SA module includes three convolutional layers;

[0055] Two of the convolutional layers each receive and process the low-level features from the encoder module and the high-level features from the decoder module in the input data;

[0056] The outputs of the two convolutional layers are added and then passed through the third convolutional layer and then successively through the BN layer and the Sigmoid activation function to obtain the spatial weights; the low-level features from the encoder module in the input data are multiplied by the spatial weights to obtain the output features, denoted as the output of the SA module.

[0057] In the embodiment of the present invention, the channel attention CA module and the spatial attention SA module are used to re-integrate the spatial and channel feature weights to segment the target image, improving the performance of the entire network when segmenting skin melanoma images.

[0058] Embodiment 5

[0059] The embodiment of the present invention provides a specific skin melanoma image segmentation network based on the improved MultiResUNet and gives the parameters of each module. As Figure 6As shown, the network structure includes an encoder module and a decoder module. The encoder module includes five new MultiRes Blocks from the shallow layer to the deep layer of the network, numbered 1, 2, 3, 4, and 5. Max-pooling layers are used for downsampling between every two MultiRes Blocks. The decoder module includes four new MultiRes Blocks from the shallow layer to the deep layer of the network, numbered 6, 7, 8, and 9. An upsampling layer is included between every two MultiRes Blocks. The output of the encoder module is input into the decoder module through an upsampling layer. The output features of the MultiResBlocks numbered 5, 6, 7, and 8 are first subjected to a 2x upsampling operation, and then are fused with the features after reassigning weights through the att-Block module from the output features of the MultiRes Blocks numbered 4, 3, 2, and 1 through a Concat operation. Among them, the output features of the MultiRes Blocks numbered 4, 3, 2, and 1 need to pass through 1, 2, 3, and 4 residual modules respectively (such as the Res Block in Figure 6 ).

[0060] In the embodiment of the present invention, the upsampling layer is composed of an upsampling module with a multiple of 2 and a Conv2d layer with a convolution kernel size of 1×1, a stride of 1, and a padding of 0.

[0061] In the embodiment of the present invention, as shown in Figure 7 the parameters of the three Conv2d layers of the new MultiRes Block from the shallow layer to the deep layer are as follows: the first Conv2d layer includes a convolutional layer with a convolution kernel size of 3×3, a stride of 1, a padding of 1, and a dilation rate of 1, a BN layer, and a LeakyReLU activation function; the second Conv2d layer includes a convolutional layer with a convolution kernel size of 3×3, a stride of 1, a padding of 2, and a dilation rate of 2, a BN layer, and a LeakyReLU activation function; the third Conv2d layer includes a convolutional layer with a convolution kernel size of 3×3, a stride of 1, a padding of 5, and a dilation rate of 5, a BN layer, and a LeakyReLU activation function; the first residual layer includes a convolutional layer with a convolution kernel size of 1×1, a stride of 1, and a padding of 0 and a BN layer.

[0062] In the embodiment of the present invention, as shown in Figure 8As shown in the figure, in the CA module, two adaptive global average pooling layers are used to perform pooling operations on the features from the encoder module and the decoder module respectively, compressing the feature map size to 1×1. Then, two convolutional layers are used to perform convolutional operations with a convolutional kernel size of 1×1, a stride of 1, and a padding of 0 on the outputs of the above pooling layers respectively. Finally, after adding the respective results and passing through the Softmax activation function, the channel weights are obtained, multiplied by the features from the encoder module, and then added to the features from the decoder module to obtain the final output features.

[0063] As Figure 8 shown in the figure, in the SA module, two convolutional layers with a convolutional kernel size of 1×1, a stride of 1, and a padding of 0 are used to perform convolutional operations on the features from the encoder module and the decoder module respectively. After adding the outputs and activating through the LeakyReLU activation function, the output is then passed through a convolutional layer with a convolutional kernel size of 1×1, a stride of 1, and a padding of 0 to compress the channels to 1. Then, it passes through a BN layer and a Sigmoid activation function in sequence to obtain the spatial weights. Finally, the features of the encoder module are multiplied by the spatial weights to obtain the final output features.

[0064] In the embodiment of the present invention, in the residual module, the ordinary convolutional layer consists of a convolutional layer with a convolutional kernel size of 3×3, a stride of 1, and a padding of 1, a BN layer, and a LeakyReLU layer; the second residual layer consists of a convolutional layer with a convolutional kernel size of 1×1, a stride of 1, and a padding of 0 and a BN layer; then the outputs of the ordinary convolutional layer and the second residual layer pass through a LeakyReLU layer.

[0065] Embodiment 6

[0066] Based on the skin melanoma image segmentation network structure based on the improved MultiResUNet in the above embodiments, the embodiment of the present invention provides a skin melanoma image segmentation method, including the following steps:

[0067] S601: Divide the data set into a training set and a validation set according to a certain ratio;

[0068] S602: Use the training set to train the skin melanoma image segmentation network based on the improved MultiResUNet to obtain the optimal segmentation model;

[0069] S603: Use the optimal segmentation model obtained by training to segment the pictures in the validation set;

[0070] S604: Evaluate the test results of the validation set, and the evaluation indicators include the Jaccard Index.

[0071] To verify the effectiveness of the segmentation network and method provided by the present invention, the present invention also provides the following experiments, which are specifically as follows:

[0072] 1. Prepare the dataset, randomly divide the ISIC-2018 dataset into a training set and a validation set at a ratio of 8:2; uniformly adjust the original image size to 256×192;

[0073] 2. Combine Figure 6 , set the specific parameters of the encoder module. The input and output of the MultiRes Block are represented in the format of "number of channels × width × height", and the parameters of Maxpool are represented in the format of "kernel size × stride" as (2×2)×2; specifically as shown in Table 1.

[0074] Combine Figure 7 , the MultiRes Block internally includes three Conv2d layers. The coefficient of the output channels of the convolution kernel of the first Conv2d layer is 0.167 of the output channels of the MultiRes Block module, the coefficient of the output channels of the convolution kernel of the second Conv2d layer is 0.333, and the coefficient of the output channels of the convolution kernel of the third Conv2d layer is 0.5; specifically as shown in Table 2.

[0075] Table 1 Specific parameters of the encoder module

[0076] Module Name Input Output MultiRes Block1 3×256×192 51×256×192 MultiRes Block2 51×128×96 105×128×96 MultiRes Block3 105×64×48 212×64×48 MultiRes Block4 212×32×24 426×32×24 MultiRes Block5 426×16×12 853×16×12 Maxpool1 51×256×192 51×128×96 Maxpool2 105×128×96 105×64×48 Maxpool3 212×64×48 212×32×24 Maxpool4 426×32×24 426×16×12

[0077] Table 2 Specific internal parameters of the MultiRes Block

[0078]

[0079]

[0080] Combine Figure 6 , set the specific parameters of the decoder module. The factor parameter of Upsample is 2, and the size of the Conv2d convolution kernel is represented in the format of "number of input channels × number of output channels × (kernel height × kernel width)" as number of input channels × number of output channels × (3×3), the convolution stride is 1, and the number of 0 rows (columns) filled at the edge of the feature map during convolution is 1; specifically as shown in Table 3. Then set the specific parameters of the residual module, as shown in Table 4.

[0081] Table 3 Specific parameters of the decoder module

[0082]

[0083] Table 4 Specific parameters of the residual module

[0084]

[0085] Among them, the Res Block includes a common convolutional layer and a second residual layer. The common convolutional layer includes Conv2d, BN, and LeakyReLU layers; the parameters of the convolutional kernel are expressed in the format of "number of input channels × (convolution kernel height × convolution kernel width)" as "number of input channels × (3 × 3)", the stride is 1, and the padding is 1. The second residual layer includes Conv2d and BN layers, and the parameters of the convolutional kernel are expressed in the format of "number of input channels × (convolution kernel height × convolution kernel width)" as "number of input channels × (1 × 1)", the stride is 1, and the padding is 0.

[0086] Combined with Figure 8 as shown, the specific parameters of the att-Block module are set as shown in Table 5:

[0087] Table 5 Specific Parameters of the att-Block Module

[0088]

[0089] Combined with Figure 8 , the parameters of Conv2d are expressed in the format of "number of input channels × (convolution kernel height × convolution kernel width)" as "number of input channels × (1 × 1)", the stride is 1, and the padding is 0.

[0090] 3. Set the training parameters: Use the Adam algorithm to train the network on the training set, and set the hyperparameters in the neural network: the initial learning rate is 0.001, the learning rate decay rate weight is 0.1, the learning rate decay strategy is that the learning rate decays if the evaluation index does not improve after 10 rounds of training, the batchsize is set to 8, and the total number of training rounds is 150; save the weight file after training is completed.

[0091] 4. Use the model weights after training to segment the images in the validation set, and evaluate the segmentation results of the validation set. The calculation process of the evaluation index Jaccard Index is as follows:

[0092]

[0093] In the above formula, A represents the predicted region of the network, and B represents the segmented region annotated in the mask.

[0094] Through the above evaluation index, the method of the present invention is compared with the UNet and the original MultiResUNet network respectively, and the segmentation results are shown in Table 6:

[0095] Table 6

[0096] Model Jaccard Index UNet 76.4277 MultiResUNet 80.2988 Improved MultiResUNet 81.79

[0097] As can be seen from Table 6, the network model proposed by the present invention has a relatively high improvement in performance on the ISIC-2018 dataset compared to the original MultiResUNet network and the UNet.

[0098] In addition, as shown in combination with Figure 9 and Figure 10 , it can also be seen that for the same mask image, the segmentation result map of the skin melanoma image segmentation network based on the improved MultiResUNet provided by the present invention has higher segmentation accuracy.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image segmentation method for skin melanoma based on improved MultiResUNet, characterized in that Including: Construct and train a skin melanoma image segmentation network based on the improved MultiResUNet; Use the trained skin melanoma image segmentation network to segment the input image; Among them, the skin melanoma image segmentation network includes an encoder module and a decoder module, and replaces the MultiRes Block in the original MultiResUNet network with a new MultiRes Block to form a new MultiResUNet network; wherein, the new MultiRes Block includes a first residual layer and three stacked Conv2d layers with different dilation rates; after the outputs of the three Conv2d layers are subjected to a concat operation, a feature addition operation is performed with the output result of the first residual layer; the dilation rates of the three Conv2d layers are 1, 2, and 5 from shallow to deep in sequence; The encoder module includes 5 new MultiRes Blocks from shallow to deep in sequence; the decoder module includes 4 new MultiRes Blocks from shallow to deep in sequence; wherein, an att-Block module is arranged between each layer of MultiRes Block in the first 4 layers of the encoder module and the corresponding layer of MultiRes Block in the decoder module; the att-Block module is used to reassign weights to the features output by each layer of MultiRes Block in the encoder module; the att-Block module includes a channel attention CA module and a spatial attention SA module; after the same input data passes through the CA module and the SA module respectively, the outputs of the CA module and the SA module are subjected to an addition operation; The CA module includes two CA units with the same structure, and each CA unit includes an adaptive global average pooling layer and a convolutional layer connected in sequence; one of the CA units receives and processes the low-level features from the encoder module in the input data, and the other CA unit receives and processes the high-level features from the decoder module; after the outputs of the two CA units are subjected to an addition operation, a channel weight is obtained through a Softmax activation function; the low-level features from the encoder module in the input data are multiplied by the channel weight and then added to the high-level features from the decoder module in the input data to obtain the output features, denoted as the output of the CA module; The SA module includes three convolutional layers; two of the convolutional layers each receive and process the low-level features from the encoder module and the high-level features from the decoder module in the input data; the outputs of the two convolutional layers are added together, then passed through the third convolutional layer, and then through the BN layer and the Sigmoid activation function in sequence to obtain the spatial weights; the low-level features from the encoder module in the input data are multiplied by the spatial weights to obtain the output features, which are denoted as the output of the SA module.

2. The method for segmenting skin melanoma images based on the improved MultiResUNet according to claim 1, characterized in that, An upsampling module is provided between the last layer MultiRes Block of the encoder module and the first layer MultiRes Block of the decoder module, and between adjacent two layer MultiRes Blocks in the decoder module; A residual module is provided between each layer MultiRes Block in the first 4 layers of the encoder module and its corresponding att-Block module; The att-Block module re-weights the features output by the corresponding layer MultiRes Block in the encoder module according to the output of the upsampling module and the output of the residual module.

3. The skin melanoma image segmentation method based on the improved MultiResUNet according to claim 2, characterized in that, The number of residual modules corresponding to each layer MultiRes Block in the first 4 layers of the encoder module is 4, 3, 2, and 1 in sequence from the shallow layer to the deep layer.

4. The skin melanoma image segmentation method based on the improved MultiResUNet according to claim 2, wherein The residual module includes an ordinary convolutional layer and a second residual layer; the ordinary convolutional layer includes a convolutional layer, a BN layer, and a LeakyReLU layer connected in sequence; the second residual layer includes a convolutional layer and a BN layer connected in sequence.

Images