A U-shaped neural network skin lesion segmentation method and device

Abstract

The present invention provides a U-shaped neural network skin lesion segmentation method and device, which relates to the field of image segmentation technology, including: S1: obtaining a skin lesion image set, preprocessing and data normalization of the skin lesion image set, and obtaining a training data set; S2: constructing a U-shaped neural network, training the U-shaped neural network through the training data set, and obtaining a trained U-shaped neural network; S3: inputting the skin lesion image to be segmented into the trained U-shaped neural network to obtain the segmented skin lesion image. The U-shaped neural network of the present invention is composed of an encoder and a decoder composed of a fused convolutional mamba module, which can extract rich features, and can achieve better segmentation effects for skin lesions with blurred boundaries and low contrast. At the same time, the combination of a spatial attention module and a channel attention module can realize multi-level and multi-scale information fusion, and realize high-precision skin lesion segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a U-shaped neural network skin lesion segmentation method and device. Background Art

[0002] Skin cancer and various pigmented skin diseases are seriously threatening human health. Traditional skin lesion image segmentation methods are affected by impurities such as hair and blisters in the image, and the segmentation effect is not ideal and time-consuming and laborious. It is necessary to study automatic skin lesion image segmentation. Automatic skin lesion image segmentation is one of the key elements of computer vision-assisted diagnosis. Automatic segmentation can not only avoid human differences but also quickly find the lesion area. In recent years, skin lesion image segmentation based on deep learning has become a research hotspot. CNN models have been widely used in skin lesion image segmentation, but due to limited receptive fields and inherent inductive biases, CNNs have limitations in long-range modeling capabilities. They cannot build global context and long-range dependencies in images, and usually ignore the global information of the image, resulting in mis-segmentation, which limits their performance in skin lesion image segmentation. Transformer and Mamba can model global context information, but Transformer is limited by its quadratic computational complexity and requires more computing resources. In addition, due to factors such as the large scale variations of diseased skin, the irregular shape of the lesion area, the low contrast between the lesion and the surrounding skin, the blurred lesion boundaries, and the image background interference, the existing technology is far from sufficient to locate skin lesions and outline their fine boundaries. It is still very difficult to accurately segment skin lesions in dermoscopic images. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a U-shaped neural network skin lesion segmentation method and device to solve the problem of low accuracy of existing skin image segmentation.

[0004] The present invention provides a U-shaped neural network skin lesion segmentation method, comprising:

[0005] S1: obtaining a skin lesion image set, preprocessing and data normalization of the skin lesion image set, and obtaining a training data set;

[0006] S2: construct a U-shaped neural network, train the U-shaped neural network through a training data set, and obtain a trained U-shaped neural network;

[0007] S3: Input the skin lesion image to be segmented into the trained U-shaped neural network to obtain the segmented skin lesion image.

[0008] Preferred:

[0009] The U-shaped neural network includes: a first convolutional layer, a first encoding layer, a second encoding layer, a third encoding layer, a fourth encoding layer, a first decoding layer, a second decoding layer, a third decoding layer, a fourth decoding layer, a second convolutional layer, a spatial attention module, and a channel attention module;

[0010] The first convolution layer, the first encoding layer, the second encoding layer, the third encoding layer, the fourth encoding layer, the first decoding layer, the second decoding layer, the third decoding layer, the fourth decoding layer, and the second convolution layer are connected in sequence;

[0011] The first convolutional layer, the first encoding layer, the second encoding layer and the third encoding layer are connected to the spatial attention module, the spatial attention module is connected to the channel attention module, and the channel attention module is connected to the second convolutional layer, the fourth decoding layer, the third decoding layer and the second decoding layer.

[0012] Preferred:

[0013] The encoding layer includes a fused convolutional Mamba module and a downsampling module, and the output of the fused convolutional Mamba module is used as the input of the downsampling module.

[0014] Preferred:

[0015] The decoding layer includes a fused convolutional Mamba module and an upsampling module, and the output of the fused convolutional Mamba module is used as the input of the upsampling module.

[0016] Preferred:

[0017] The fused convolutional Mamba module includes: dual parallel visual Mamba module and fusion module;

[0018] The dual parallel visual Mamba module includes: a first visual state space module and a second visual state space module;

[0019] The fusion module includes a first 1×1 convolution module, a residual block, and a second 1×1 convolution module connected in sequence;

[0020] The residual block includes a first 3×3 convolutional module and a second 3×3 convolutional module connected in sequence;

[0021] An input end of the fused convolutional mamba module is connected to the first visual state space module, the second visual state space module and the first 1×1 convolution module;

[0022] The first visual state space module and the second visual state space module are connected to the first 3×3 convolution module;

[0023] The input end of the fused convolutional Mamba module is residually connected to the output end of the fused convolutional Mamba module;

[0024] The input of the first 3×3 convolutional module is residually connected to the input of the second 1×1 convolutional module.

[0025] Preferred:

[0026] The output feature map F of the fused convolutional Mamba module out The expression is:

[0027]

[0028] Among them, F represents the input image, TPVMblock() represents the processing by the dual parallel visual Mamba module, conv1×1() represents the processing by the 1×1 convolution module, Indicates that it has been processed by the residual block.

[0029] Preferably, step S2 specifically comprises:

[0030] S21: Get the true label set and true value;

[0031] S22: input the training data set into the U-shaped neural network, and calculate and obtain the predicted label set and the predicted value;

[0032] S23: Calculate the total loss through the real label set, the real value, the predicted label set and the predicted value;

[0033] S24: The adaptive moment estimation optimizer adjusts the parameters of the U-shaped neural network according to the total loss;

[0034] S25: Repeat steps S22-S24 until the total loss is less than a preset value, and obtain a trained U-shaped neural network.

[0035] Preferred:

[0036] Total loss The calculation formula is:

[0037]

[0038] Among them, L Bce and L Dice They represent binary cross entropy loss and Dice loss respectively, N is the total number of training data, i is the number of training data, y i is the i-th true label, p i is the i-th predicted label, |X| and |Y| represent the true value and predicted value respectively, λ1 and λ2 represent L Bce and L Dice The weight of .

[0039] Preferably, step S3 specifically comprises:

[0040] S31: The skin lesion image to be segmented passes through the first convolution layer, the first coding layer, the second coding layer, the third coding layer and the fourth coding layer to obtain 32, 64, 128, 256 and 512 channel segmentation images respectively;

[0041] S32: The 32, 64, 128 and 256 channel segmented images are input to the second convolutional layer, the fourth decoding layer, the third decoding layer and the second decoding layer respectively through the spatial attention module and the channel attention module. The 512 channel segmented image is input to the first decoding layer, and the second convolutional layer outputs the segmented skin lesion image.

[0042] A U-shaped neural network skin lesion segmentation device comprises: a processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement the U-shaped neural network skin lesion segmentation method.

[0043] The present invention has the following beneficial effects:

[0044] The U-shaped neural network of the present invention is composed of an encoder and a decoder composed of a fused convolutional Mamba module, which can extract rich features and achieve better segmentation effects for skin lesions with blurred boundaries and low contrast. At the same time, the combination of the spatial attention module and the channel attention module can realize multi-level and multi-scale information fusion, thereby achieving high-precision skin lesion segmentation.

[0045] The fused convolutional Mamba module has a significant positive impact on the overall segmentation performance, especially in extracting global and local features. Global information helps to understand the overall structure of the lesion and its relationship with the background, so as to more accurately locate the lesion area. Local information helps to capture the edge features and corner details of the lesion, improve the model performance, and effectively solve the problem of similarity between the target and the background and blurred boundaries in the skin lesion image, thereby achieving accurate segmentation of skin lesions. The fused convolutional Mamba module contains a dual parallel visual Mamba module, which can expand the receptive field of the model and thus enhance its sensitivity to targets of different scales, further improving the accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of a method according to an embodiment of the present invention;

[0047] Figure 2 It is a U-shaped neural network structure diagram;

[0048] Figure 3 This is the structure diagram of the fused convolutional Mamba module;

[0049] Figure 4 A comparison diagram of the segmented skin lesion images of the present invention and other models;

[0050] Figure 5This is a system structure diagram of an embodiment of the present invention;

[0051] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0053] Reference Figure 1 The present invention provides a U-shaped neural network skin lesion segmentation method, comprising:

[0054] S1: obtaining a skin lesion image set, preprocessing and data normalization of the skin lesion image set, and obtaining a training data set;

[0055] Specifically, the skin lesion image dataset is derived from the International Skin Open Challenge dataset (ISIC 2018), which contains 2694 original skin lesion images of different resolutions, where the true labels of the original images are binary images manually annotated by dermatology hospitals; the skin lesion image dataset is divided into: 1886 for training and 808 for testing; image preprocessing is to use bilinear interpolation to uniformly scale the original images and the true labels of the images to 256×256 pixels, and image data normalization is to use the conventional method min-max normalization to perform linear transformation on the skin lesion image sample data so that the processed skin lesion image sample data falls into the [0, 1] interval.

[0056] S2: construct a U-shaped neural network, train the U-shaped neural network through a training data set, and obtain a trained U-shaped neural network;

[0057] Furthermore, the structure of the U-shaped neural network is as follows Figure 2 As shown;

[0058] The U-shaped neural network includes: a first convolutional layer, a first encoding layer, a second encoding layer, a third encoding layer, a fourth encoding layer, a first decoding layer, a second decoding layer, a third decoding layer, a fourth decoding layer, a second convolutional layer, a spatial attention module, and a channel attention module;

[0059] The first convolution layer, the first encoding layer, the second encoding layer, the third encoding layer, the fourth encoding layer, the first decoding layer, the second decoding layer, the third decoding layer, the fourth decoding layer, and the second convolution layer are connected in sequence;

[0060] The first convolutional layer, the first encoding layer, the second encoding layer and the third encoding layer are connected to the spatial attention module, the spatial attention module is connected to the channel attention module, and the channel attention module is connected to the second convolutional layer, the fourth decoding layer, the third decoding layer and the second decoding layer.

[0061] Specifically, the skin lesion image is input into a 3×3 convolutional layer to preliminarily extract image features. The features are used as the input of the first encoding layer and added to the output of the fourth decoding layer through a skip connection operation as the input of the 1×1 convolutional layer. Then, the output of the first encoding layer is used as the input of the second encoding layer and added to the output of the third decoding layer through a skip connection operation as the input of the fourth decoding layer. Then, the output of the second encoding layer is used as the input of the third encoding layer and added to the output of the second decoding layer through a skip connection operation as the input of the third decoding layer. Similarly, the output of the third encoding layer is used as the input of the fourth encoding layer and added to the output of the first decoding layer through a skip connection operation as the input of the second decoding layer. Finally, the output of the fourth encoding layer is directly used as the input of the first decoding layer. The 3×3 convolution layer and each encoding layer are followed by a downsampling layer, and each decoding layer and 1×1 convolution layer are followed by an upsampling layer; the output of the 1×1 convolution layer is the probability map of each pixel belonging to the target. By introducing the threshold T, the segmentation result can be obtained; the 3×3 convolution layer and four encoders are divided into five stages, and the number of channels in each stage is [32, 64, 128, 256, 512];

[0062] The spatial attention module is mainly composed of average pooling, maximum pooling and shared dilated convolution, and the channel attention module is composed of global average pooling, cascade operation, one-dimensional convolution operation, fully connected layer and sigmoid activation function; the spatial attention module and the channel attention module can accelerate the convergence of the model, realize multi-level and multi-scale information fusion, and improve the segmentation ability of lesion areas of different sizes.

[0063] Furthermore, the encoding layer includes a fused convolutional Mamba module and a downsampling module, and the output of the fused convolutional Mamba module is used as the input of the downsampling module.

[0064] Specifically, each encoding layer is composed of two fused convolutional mamba modules connected together, where each fused convolutional mamba module is composed of a dual parallel visual mamba module and a fusion module. The output of the previous fused convolutional mamba module enters the dual parallel visual mamba module and the fusion module of the next one respectively, and the output of the dual parallel visual mamba module enters the fusion module for fusion; in order to downsample the image, a maximum pooling layer of size 2×2 is added at the end of the 3×3 convolutional layer and all encoding layers.

[0065] Furthermore, the decoding layer includes a fused convolutional Mamba module and an upsampling module, and the output of the fused convolutional Mamba module is used as the input of the upsampling module.

[0066] Specifically, the first, second and third decoding layers are composed of two fused convolutional Mamba modules, and the fourth decoding layer is composed of one fused convolutional Mamba module; in order to upsample the image, the end of all decoding layers and 1×1 convolutional layers contains a bilinear interpolation layer with a scale factor of 2.

[0067] Furthermore, the structure of the fused convolutional Mamba module is as follows Figure 3 As shown;

[0068] The fused convolutional Mamba module includes: dual parallel visual Mamba module and fusion module;

[0069] The dual parallel visual Mamba module includes: a first visual state space module and a second visual state space module;

[0070] The fusion module includes a first 1×1 convolution module, a residual block, and a second 1×1 convolution module connected in sequence;

[0071] The residual block includes a first 3×3 convolutional module and a second 3×3 convolutional module connected in sequence;

[0072] An input end of the fused convolutional mamba module is connected to the first visual state space module, the second visual state space module and the first 1×1 convolution module;

[0073] The first visual state space module and the second visual state space module are connected to the first 3×3 convolution module;

[0074] The input end of the fused convolutional Mamba module is residually connected to the output end of the fused convolutional Mamba module;

[0075] The input of the first 3×3 convolutional module is residually connected to the input of the second 1×1 convolutional module.

[0076] Specifically, the fused convolutional mamba block is called ConvMambaBlock, the dual parallel vision mamba block is called TPVMblock (Two Parallel Vision Mamba Block), the fusion module is called Fusionblock, and the visual state space module is called VSSBlock.

[0077] The dual parallel visual Mamba module consists of two parallel visual state space modules. The input F is evenly divided into two visual state space modules for parallel processing in the channel dimension, and the two outputs are combined into F in the channel dimension. mamba The fusion module consists of two 1×1 convolution modules and a residual block. The residual block includes two batch normalization operations and two 3×3 convolution modules. The input F undergoes a 1×1 convolution and is then convolved with F mambaAdd, then input a residual block, then go through a 1×1 convolution, and finally, add it to a residual connection F to get F out ;

[0078] The input of the visual state space module is divided into two branches. In the first branch, the input passes through a linear layer and then a SiLU activation function. In the second branch, the input is processed through a linear layer, a depth-wise separable convolution, and a SiLU activation function, and then input into a selective scan (SSM) module for further feature extraction. Subsequently, the features are normalized using layer normalization, and then a Hadamard product is performed using the output of the first branch to merge the two paths. Finally, the features are mixed using a linear layer to form the output of the visual state space module.

[0079] Furthermore, the output feature map F of the fused convolutional Mamba module out The expression is:

[0080]

[0081] Among them, F represents the input image, TPVMblock() represents the processing by the dual parallel visual Mamba module, conv1×1() represents the processing by the 1×1 convolution module, Indicates that it has been processed by the residual block.

[0082] Furthermore,

[0083] Step S2 is specifically as follows:

[0084] S21: Get the true label set and true value;

[0085] S22: input the training data set into the U-shaped neural network, and calculate and obtain the predicted label set and the predicted value;

[0086] S23: Calculate the total loss through the real label set, the real value, the predicted label set and the predicted value;

[0087] Furthermore, the total loss The calculation formula is:

[0088]

[0089] Among them, L Bce and L Dice They represent binary cross entropy loss and Dice loss respectively, N is the total number of training data, i is the number of training data, y i is the i-th true label, p i is the i-th predicted label, |X| and |Y| represent the true value and predicted value respectively, λ1 and λ2 represent LBce and L Dice The weight of .

[0090] Specifically, in the present invention, both λ1 and λ2 are 1;

[0091] S24: The adaptive moment estimation optimizer adjusts the parameters of the U-shaped neural network according to the total loss;

[0092] S25: Repeat steps S22-S24 until the total loss is less than a preset value, and obtain a trained U-shaped neural network.

[0093] Specifically, in order to obtain the best model performance, the present invention uses AdamW as the optimizer; the initial learning rate size is set to 1e-3, and the cosine annealing learning rate scheduler is used, with a maximum of 50 iterations, and the minimum learning rate is set to 1e-5; the training epochs is set to 300, and when the verification loss reaches the minimum, the model parameters at this time are saved.

[0094] S3: Input the skin lesion image to be segmented into the trained U-shaped neural network to obtain the segmented skin lesion image.

[0095] Furthermore, step S3 is specifically as follows:

[0096] S31: The skin lesion image to be segmented passes through the first convolution layer, the first coding layer, the second coding layer, the third coding layer and the fourth coding layer to obtain 32, 64, 128, 256 and 512 channel segmentation images respectively;

[0097] S32: The 32, 64, 128 and 256 channel segmented images are input to the second convolutional layer, the fourth decoding layer, the third decoding layer and the second decoding layer respectively through the spatial attention module and the channel attention module. The 512 channel segmented image is input to the first decoding layer, and the second convolutional layer outputs the segmented skin lesion image.

[0098] Specifically, after training is completed, the skin lesion image to be segmented is directly input into the network, and the learned network is used to predict the skin lesion image to be tested. After the test image passes through the network, the distribution probability map of the target area is output, and its value range is 0-1. The threshold T is set to 0.5. Values ​​greater than or equal to 0.5 are considered as targets to be segmented, and values ​​less than 0.5 are considered as backgrounds. Then the target is set to 1 and the background is set to 0. Finally, the segmentation result of the diseased skin target is obtained. The evaluation indicators of skin lesion segmentation are mean intersection over union (mIoU), F1-Score coefficient, accuracy, specificity, and sensitivity. The values ​​of each indicator are obtained through this test.

[0099] Furthermore, comparative experiments were conducted with existing advanced methods to further verify the effectiveness of the model:

[0100] The present invention is implemented under the pytorch deep learning framework. The computer configuration adopts: Intel(R) Xeon(R) Silver 4210R processor, 64G memory, NVIDIA Geforce RTX 3090 graphics card with 24 GB video memory, and Linux operating system; the software platform is: Python3 and PyTorch2.0.

[0101] The segmented skin lesion image obtained in step S3 is compared with the segmentation results of advanced methods such as U-Net, Att-UNet, MALUNet, HSH-UNet and VM-UNet. First, the comparison of indicators is carried out. The method with higher indicators is better. Second, the lesion segmentation result diagrams of each method are compared. The segmentation results are compared with the real mask of the lesion, and the area and boundary of the lesion segmentation are observed to verify whether the method of the present invention is effective for skin lesion images with similar segmentation targets and backgrounds, blurred boundaries, hair influence, and irregular lesion area shapes.

[0102] The method of the present invention is used to conduct skin lesion segmentation experiments on the ISIC2018 skin lesion public segmentation dataset. There are two segmentation categories: background and skin lesions. This embodiment selects 1886 skin lesion images and corresponding skin lesion manual segmentation masks from the skin lesion public segmentation dataset as the training dataset, and selects 808 skin lesion images as the test dataset. The experiment uses mean intersection over union (mIoU), F1-Score coefficient, accuracy (Accuracy), specificity (Specificity), and sensitivity (Sensitivity) as evaluation criteria. The segmentation method of the present invention achieved 81.77%, 89.97%, 95.12%, 96.82% and 89.83% in the skin lesion segmentation task. The segmentation results of some images in the test set are shown in the figure. Figure 4 (c)~ Figure 4 As shown in (h), the area indicated by the white closed curve is the skin lesion segmentation result obtained in this embodiment. It can be seen that the skin lesion area in the figure is accurately segmented, and there is no missed segmentation and over-segmentation phenomenon at the segmentation boundary. The segmentation accuracy is high and close to the actual skin lesion contour.

[0103] 1. Model performance evaluation:

[0104] Since 2015, U-Net has been widely used in the field of biomedical image segmentation. U-Net adopts an encoder-decoder structure and performs well in various biomedical image segmentation tasks. At present, U-Net has derived many variants. Although many new convolutional neural network designs have emerged in recent years, many designs still inherit the core idea of ​​U-Net, while adding new modules or incorporating other design concepts. Among them, Att-UNet introduces an attention mechanism based on U-Net. Before the features at each resolution of the encoder are spliced ​​with the corresponding features of the decoder, the attention module is used to readjust the output features of the encoder; MALUNet is a lightweight model based on a U-shaped architecture and introduces four attention modules; HSH-UNet designs a hybrid selective high-order interaction module HSHB embedded in the U-shaped model. Under the calculated guided features, HSHB adaptively selects the appropriate order of interaction operations channel by channel; VM-UNet introduces the Visual State Space (VSS) module as a basic building block to effectively capture rich contextual information; Ours is the method of the present invention.

[0105] Table 1 Performance comparison of the method of the present invention and the existing method

[0106]

[0107] As shown in Table 1, the performance of the method of the present invention is compared with the above algorithms. The evaluation indicators in the table are mean intersection over union (mIoU), F1-Score coefficient, accuracy, specificity, and sensitivity. The bold in the table indicates the method with the best performance in this indicator. It can be clearly seen from Table 1 that compared with the previous methods, the method of the present invention has achieved the best results in the above performance indicators.

[0108] 2. Segmentation results display:

[0109] like Figure 4As shown in the figure, the segmentation results of the method of the present invention compared with the existing methods are shown. The first column is the input image of 256×256 pixels obtained by preprocessing the original image in the above step 1; the second column is the real mask of the size corresponding to the input image; the third column is the segmentation result of the neural network method U-net for biomedical image segmentation proposed by Ronneberger et al. in 2015. From the segmentation result diagram, it can be seen that the U-net method is also affected by hair in the presence of over-segmentation and under-segmentation, and the hair is segmented into it; the fourth column is the Att-UNet method for CT pancreas segmentation proposed by Oktay et al. in 2018. From the segmentation result diagram, it can be seen that this method is not very good at the overall segmentation of the diseased skin during prediction, and also incompletely segmenting the skin lesions with low contrast; the fifth and sixth columns are the multi-attention lightweight UNet method MALUNet for skin lesion segmentation proposed by Jiacheng Ruan et al. and the hybrid selective high-order interactive U-net for automatic skin lesion segmentation proposed by Renkai Wu et al. From the segmentation results, we can see that these two methods have poor segmentation effects on lesion images with similar foreground and background. They also segment non-lesion areas and easily misjudge similar background interference as the target itself. The seventh column is Vision Mamba proposed by Jiacheng Ruan et al. for medical image segmentation. The UNet method VM-UNet is the best existing method; it can be seen from the segmentation result diagram that this method is not very good for target edge segmentation; the last column is the method of the present invention. It can be seen from the segmentation result diagram that the method of the present invention has certain improvements in background interference, targets of different scales and edge details compared with the previous methods, and can relatively well achieve the segmentation of diseased skin in dermoscopic images; the segmentation indicators of the method of the present invention, mean intersection over union (mIoU), F1-Score coefficient, accuracy (Accuracy), specificity (Specificity), sensitivity (Sensitivity) reached 81.77%, 89.97%, 95.12%, 96.82% and 89.83% respectively. Compared with the best existing method VMUNet, each indicator increased by 1.08%, 0.66%, 0.31%, 0.19% and 0.70% respectively.

[0110] See also Figure 5 , Figure 5 It is a working diagram of the hardware device of an embodiment of the present invention, and the hardware device specifically includes: a U-shaped neural network skin lesion segmentation device 401, a processor 402 and a storage medium 403.

[0111] A U-shaped neural network skin lesion segmentation device 401: The U-shaped neural network skin lesion segmentation device 401 implements the U-shaped neural network skin lesion segmentation method.

[0112] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the U-shaped neural network skin lesion segmentation method.

[0113] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the U-shaped neural network skin lesion segmentation method.

[0114] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0115] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.

[0116] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A U-shaped neural network skin lesion segmentation method, characterized in that: include: S1: obtaining a skin lesion image set, preprocessing and data normalization of the skin lesion image set, and obtaining a training data set; S2: construct a U-shaped neural network, train the U-shaped neural network through a training data set, and obtain a trained U-shaped neural network; S3: inputting the skin lesion image to be segmented into the trained U-shaped neural network to obtain the segmented skin lesion image; The U-shaped neural network includes a fused convolutional Mamba module; The fused convolutional Mamba module includes: dual parallel visual Mamba module and fusion module; The dual parallel visual Mamba module includes: a first visual state space module and a second visual state space module; The fusion module includes a first 1×1 convolution module, a residual block, and a second 1×1 convolution module connected in sequence; The residual block includes a first 3×3 convolutional module and a second 3×3 convolutional module connected in sequence; An input end of the fused convolutional mamba module is connected to the first visual state space module, the second visual state space module and the first 1×1 convolution module; The first visual state space module and the second visual state space module are connected to the first 3×3 convolution module; The input end of the fused convolutional Mamba module is residually connected to the output end of the fused convolutional Mamba module; The input of the first 3×3 convolutional module is residually connected to the input of the second 1×1 convolutional module; The output feature map F of the fused convolutional Mamba module out The expression is: Among them, F represents the input image, TPVMblock() represents the processing by the dual parallel visual Mamba module, conv1×1() represents the processing by the 1×1 convolution module, Indicates that it has been processed by the residual block.

2. The U-shaped neural network skin lesion segmentation method according to claim 1, characterized in that: The U-shaped neural network includes: a first convolutional layer, a first encoding layer, a second encoding layer, a third encoding layer, a fourth encoding layer, a first decoding layer, a second decoding layer, a third decoding layer, a fourth decoding layer, a second convolutional layer, a spatial attention module, and a channel attention module; The first convolution layer, the first encoding layer, the second encoding layer, the third encoding layer, the fourth encoding layer, the first decoding layer, the second decoding layer, the third decoding layer, the fourth decoding layer, and the second convolution layer are connected in sequence; The first convolutional layer, the first encoding layer, the second encoding layer and the third encoding layer are connected to the spatial attention module, the spatial attention module is connected to the channel attention module, and the channel attention module is connected to the second convolutional layer, the fourth decoding layer, the third decoding layer and the second decoding layer.

3. The U-shaped neural network skin lesion segmentation method according to claim 2, characterized in that: The encoding layer includes the fused convolutional Mamba module and a downsampling module, and the output of the fused convolutional Mamba module is used as the input of the downsampling module.

4. The U-shaped neural network skin lesion segmentation method according to claim 2, characterized in that: The decoding layer includes the fused convolutional Mamba module and an upsampling module, and the output of the fused convolutional Mamba module is used as the input of the upsampling module.

5. The U-shaped neural network skin lesion segmentation method according to claim 1, characterized in that: Step S2 is specifically as follows: S21: Get the true label set and true value; S22: input the training data set into the U-shaped neural network, and calculate and obtain the predicted label set and the predicted value; S23: Calculate the total loss through the true label set, the true value, the predicted label set and the predicted value; S24: The adaptive moment estimation optimizer adjusts the parameters of the U-shaped neural network according to the total loss; S25: Repeat steps S22-S24 until the total loss is less than a preset value, and obtain a trained U-shaped neural network.

6. The U-shaped neural network skin lesion segmentation method according to claim 5, characterized in that: Total loss The calculation formula is: Among them, L Bce and L Dice They represent binary cross entropy loss and Dice loss respectively, N is the total number of training data, i is the number of training data, y i is the i-th true label, p i is the i-th predicted label, |X| and |Y| represent the true value and predicted value respectively, λ1 and λ2 represent L Bce and L Dice The weight of .

7. The U-shaped neural network skin lesion segmentation method according to claim 2, characterized in that: Step S3 is specifically as follows: S31: The skin lesion image to be segmented passes through the first convolution layer, the first coding layer, the second coding layer, the third coding layer and the fourth coding layer to obtain 32, 64, 128, 256 and 512 channel segmentation images respectively; S32: The 32, 64, 128 and 256 channel segmented images are input to the second convolutional layer, the fourth decoding layer, the third decoding layer and the second decoding layer respectively through the spatial attention module and the channel attention module. The 512 channel segmented image is input to the first decoding layer, and the second convolutional layer outputs the segmented skin lesion image.

8. A U-shaped neural network skin lesion segmentation device, characterized by: include: Processor and storage medium; the processor loads and executes instructions and data in the storage medium to implement the U-shaped neural network skin lesion segmentation method described in any one of claims 1 to 7.

Images