Image Segmentation Method for Uterine Squamous Epithelial Lesion Areas Based on the DSA-DoubleUNet Model

Abstract

The present invention relates to an image segmentation method for uterine squamous epithelial lesion regions based on the DSA-DoubleUNet model, which solves the defect that it is difficult to accurately and efficiently segment images of uterine squamous epithelial lesion regions compared with the prior art. The present invention includes the following steps: acquisition and preprocessing of cervical acetic acid images; construction of the DSA-DoubleUNet model; training of the DSA-DoubleUNet model; acquisition and preprocessing of images to be segmented; segmentation of images of uterine squamous epithelial lesion regions. The present invention improves the segmentation ability of uterine squamous epithelial lesion regions, is more accurate in segmenting the edges of lesion regions, and at the same time, on the premise of ensuring the segmentation accuracy, greatly reduces the training parameters of the model, shortens the training time of the model, has low requirements for hardware devices, and increases the practicability of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and specifically to an image segmentation method for uterine squamous epithelial lesion regions based on the DSA-DoubleUNet model. Background Art

[0002] Multiple studies have shown that high-risk (Human Papilloma Virus, HPV) infection is a definite cause of cervical cancer. The combined screening method of early HPV detection and liquid-based cytology testing can greatly improve Cervical Intraepithelial Neoplasia (CIN). Therefore, early detection of CIN is very important for disease prevention. Colposcopy-guided cervical biopsy is the gold standard for clinical diagnosis of cervical cancer and cervical intraepithelial neoplasia, and is a very important link in the screening of cervical cancer and precancerous lesions. It is particularly important for the early detection, early diagnosis, and early treatment of cervical cancer.

[0003] In medical image diagnosis, an important step is to identify the lesion area of the patient. The segmentation of suspicious lesion areas plays an important role in assisting clinicians in diagnosis. Manual segmentation of medical images requires the operator to have rich professional knowledge and experience. In addition, the accuracy of segmentation is also affected by external factors such as the time, location, and personal status of the doctor. Computer image processing technology can objectively and accurately provide a lesion segmentation result for medical images to a great extent, thereby helping doctors improve the diagnostic accuracy and reading efficiency. In the detection of cervical squamous epithelial cells, an accurate lesion segmentation result can provide the HSIL biopsy site for patients who need biopsy.

[0004] In recent years, deep learning has developed and grown in the field of computer vision. After the Fully Convolutional Networks (FCN) model was proposed in 2015, end-to-end semantic segmentation networks have been successively proposed. In the field of medical image segmentation, the UNet network is the most classic. UNet adopts an encoder-decoder form, extracts high-level features in the encoding part of the network, and restores the spatial information of the image in the decoding part. After the UNet network, various improvements to the performance of the UNet network have emerged in an endless stream.

[0005] DoubleUNet is composed of cascading multiple UNet structures, enabling the entire network to have a more powerful feature extraction ability. Compared with the traditional UNet model, DoubleUNet has excellent segmentation performance for lesion areas. However, due to the further complexity of the network, compared with UNet, this model has more parameters, which not only increases the training time of the entire model but also poses higher requirements for the hardware devices used to train the model, resulting in DoubleUNet being unable to be used on a large scale in practice.

[0006] During colposcopy, due to the particularity of vaginal examinations, the obtained acetic acid images and iodine images contain a lot of information that is not required for segmentation, including some equipment, text, and non-cervical tissues. These irrelevant information will have a great impact on using deep learning algorithms to automatically segment the lesion area and interfere with the accuracy of the segmentation results.

[0007] Therefore, how to accurately and efficiently achieve the image segmentation of uterine squamous epithelial lesion areas has become an urgent technical problem to be solved. Summary of the Invention

[0008] The purpose of the present invention is to solve the defect that it is difficult to accurately and efficiently segment the images of uterine squamous epithelial lesion areas in the prior art, and to provide an image segmentation method for uterine squamous epithelial lesion areas based on the DSA-DoubleUNet model to solve the above problems.

[0009] To achieve the above purpose, the technical solution of the present invention is as follows:

[0010] An image segmentation method for uterine squamous epithelial lesion areas based on the DSA-DoubleUNet model includes the following steps:

[0011] Obtaining and preprocessing the acetic acid images of the cervix: Obtain the acetic acid images of the cervix during colposcopy, perform high-brightness point removal on the obtained images, and expand the processed data set;

[0012] Constructing the DSA-DoubleUNet model: Using DoubleUNet as the basic model structure, replacing ordinary convolutions with depthwise separable convolutions, introducing the channel attention module SE and the ASPP module, and replacing the transposed convolutions used in the decoder part with bilinear interpolation to construct the DSA-DoubleUNet model;

[0013] Training the DSA-DoubleUNet model: Input the expanded data set into the DSA-DoubleUNet model for training;

[0014] Obtaining and preprocessing the image to be segmented: Obtain the acetic acid image of the cervix to be segmented and preprocess it;

[0015] Segmentation of the image of the uterine squamous epithelial lesion area: Input the preprocessed acetic acid image of the cervix into the trained DSA-DoubleUNet model to obtain the segmentation result of the image of the uterine squamous epithelial lesion area.

[0016] The acquisition and preprocessing of the acetic acid image of the cervix include the following steps: ]>

[0017] Obtain the acetic acid image of the cervix and convert it from the RGB form to the YUV form. The expression is as follows:

[0018]

[0019] Among them, Y represents the brightness of the image; U represents the hue of the image; V represents the saturation of the image;

[0020] Detect the highlighted part by setting the threshold of the highlighted part. The expression is as follows:

[0021]

[0022] Among them, p represents each pixel point, and Y(p) represents the brightness of each pixel point;

[0023] Increase the saturation rate of the highlighted part. The expression is as follows:

[0024] fill(p) = mean(N8(p))

[0025] Among them, N8 represents the 8 pixel points around the pixel point p, mean(N8(p)) represents taking the average value of the brightness of the 8 pixel points around the pixel point p, and fill(p) represents filling the value after taking the average into the pixel point p;

[0026] Use the transformer module in torchvision to augment the dataset:

[0027] Augment the dataset by means of cropping, flipping, and mirroring,

[0028] Among them, the cropping function is.resize, the flipping function is.hfilp or.vfilp, and the mirroring function is.crop.

[0029] The construction of the DSA-DoubleUNet model includes the following steps:

[0030] Set the DSA-DoubleUNet network model to connect two Unet networks in series, adopting a U-shaped encoder-decoder structure;

[0031] Set the first Unet model of the DSA-DoubleUNet network as the U-shaped structure of the encoder-decoder;

[0032] Construct the SE module,

[0033] Set the ASPP module to sample the given input in parallel with dilated convolutions with different sampling coefficients r, and then fuse the sampled features;

[0034] Set the encoder part of the first Unet model to adopt the network structure of VGG19, delete the fully connected layer of VGG19, retain the feature extraction layer, replace the convolution operation in the feature extraction layer with depthwise separable convolution, and add the SE module after each depthwise separable convolution, BN, and Relu operation;

[0035] Among them, the depthwise separable convolution is divided into two processes: depthwise convolution DW and pointwise convolution PW,

[0036] One convolution kernel of DW is responsible for one channel, and the number of channels of the feature map generated in this process is the same as the number of channels of the input feature map;

[0037] The size of the convolution kernel of PW is 1×1×M, where M is the number of channels of the output of DW convolution, and the number of channels of the feature map generated after PW convolution is the same as the number of convolution kernels used;

[0038] The encoder part of the first Unet model performs four downsampling operations, and then sends the output result to the ASPP module to obtain multi-scale object information. The sampling coefficients r used in the ASPP module are 0, 6, 12, and 18 respectively;

[0039] Send the output of the ASPP module to the decoder end of the first Unet network. In the decoder, the upsampling part uses bilinear interpolation instead of transposed convolution. After the first upsampling, the decoder end fuses the features output by the fourth downsampling, and then uses the fused result as the input of the second upsampling; after the second upsampling, the decoder end fuses the features output by the third downsampling, and then uses the fused result as the input of the third upsampling; after the third upsampling, the decoder end fuses the features output by the second downsampling, and then uses the fused result as the input of the fourth upsampling; after the fourth upsampling, the decoder end fuses the features output by the first downsampling, and then uses the fused result as the result output;

[0040] Multiply the output features of the decoder of the first Unet network and the original input features as the input of the second Unet network and send it to the encoder end of the second Unet network,

[0041] Among them, the encoder of the second Unet network adopts the encoder structure of the original UNet network, replaces the convolutional operation of the encoder with depthwise separable convolution, and adds an SE module after each depthwise separable convolution, BN, and Relu operation;

[0042] Send the output of the encoder of the second UNet network to the second ASPP module, where the sampling rates of the ASPP module are 0, 6, 12, and 18;

[0043] Send the output of the second ASPP module to the decoder end of the second Unet network. The upsampling part of the decoder uses bilinear interpolation instead of transposed convolution. After the first upsampling, the decoder end fuses the features output by the fourth downsampling, and then uses the fused result as the input of the second upsampling; after the second upsampling, the decoder end fuses the features output by the third downsampling, and then uses the fused result as the input of the third upsampling; after the third upsampling, the decoder end fuses the features output by the second downsampling, and then uses the fused result as the input of the fourth upsampling; after the fourth upsampling, the decoder end fuses the features output by the first downsampling, and then uses the fused result as the result output;

[0044] Finally, fuse the output of the first Unet decoder and the output of the second Unet decoder to obtain the final output of the DSA-DoubleUNet model.

[0045] The training of the DSA-DoubleUNet model includes the following steps:

[0046] Input the augmented dataset into the DSA-DoubleUNet network model for forward propagation;

[0047] During network training, set the batch size batch_size to 8, the number of iterations epoch to 200, the downsampling multiples of the two feature extraction networks to 16, use the SGD optimizer to optimize the model, and set the initial learning rate to 0.01;

[0048] Calculate the loss after obtaining the output result of the network. The loss function adds DiceLoss to the cross-entropy loss function to form a mixed loss function. The formula of the loss function is as follows:

[0049] Loss = 0.5 * CE + 0.5 * DiceLoss,

[0050] Among them, CE is the cross-entropy loss, and the calculation formula is:

[0051]

[0052] Among them, N is the number of image pixels, and y i represents the label of the i-th pixel, where the lesion area is 1 and the background is 0, and p i represents the probability value that the i-th pixel is predicted as the lesion area;

[0053] The calculation formula of DiceLoss is as follows:

[0054]

[0055] Among them, X is the ground-truth image, and Y is the predicted image;

[0056] According to the loss function, backpropagation is performed to update the parameters in the network;

[0057] Repeat the above steps until the loss converges to obtain the weight value with the best prediction effect.

[0058] The construction of the SE module includes the following steps:

[0059] It is set that the SE module is divided into two key operations: Squeeze and Excitation;

[0060] In Squeeze, global average pooling is adopted, that is, the average value of pixels is calculated on each channel of the input feature map, so as to compress the size of the input feature map X∈R H×W×C to 1×1×C, and its principle of action is as follows:

[0061]

[0062] Among them, z c is the output value of the c-th channel, H and W are the length and width of the input image respectively, and u c (i,j) is each pixel value of the c-th channel;

[0063] In Excitation, a fully connected layer is used to scale the feature map output by Squeeze to 1×1×C / r, where r is the sampling coefficient and C is the number of channels of the feature map output by Squeeze;

[0064] Subsequently, a fully connected layer restores the number of channels to C, and then passes through the Sigmoid layer to obtain the weights between each channel;

[0065] Finally, through the Scale operation, the weights of each channel obtained are weighted to the features of each channel output by the depthwise separable layer.

[0066] Beneficial effects

[0067] The method for segmenting the image of the uterine squamous epithelial lesion area based on the DSA-DoubleUNet model of the present invention improves the segmentation ability of the uterine squamous epithelial lesion area compared with the prior art, is more accurate in segmenting the edge of the lesion area, and at the same time, on the premise of ensuring the segmentation accuracy, greatly reduces the training parameters of the model, shortens the training time of the model, has lower requirements for hardware devices, and increases the practicability of the model.

[0068] The method of the present invention is used for the segmentation of uterine epithelial squamous cells. Compared with the application of the DoubleUNet network in the segmentation of uterine epithelial squamous cells, without losing too much accuracy, the number of network parameters is reduced, and the training speed of the network is improved. Brief Description of the Drawings

[0069] Figure 1 It is the sequence diagram of the method of the present invention;

[0070] Figure 2 It is the schematic diagram of the DSA-DoubleUNet network model of the present invention;

[0071] Figure 3 It is the schematic diagram of the depthwise separable network model in the present invention;

[0072] Figure 4 It is the schematic diagram of the SE module in the depthwise separable convolution of the present invention;

[0073] Figure 5 It is the schematic diagram of the ASPP module in the present invention;

[0074] Figure 6a It is the image of the uterine squamous epithelial lesion area;

[0075] Figure 6b It is the segmentation result diagram using the UNet network model;

[0076] Figure 6c It is the segmentation result diagram using the DoubleUNet network model;

[0077] Figure 6d It is the segmentation result diagram using the DSA-DoubleUNet network model. Detailed Embodiment

[0078] To have a further understanding and recognition of the structural features and achieved effects of the present invention, the following is a detailed description in conjunction with preferred embodiments and drawings:

[0079] The acetic acid and iodine maps of the cervix obtained by the colposcope may contain a lot of information that is not required for segmentation, including some equipment, text, and non-cervical tissues. These irrelevant information will have a great impact on the segmentation results. In order to improve the suppression of the interference of these irrelevant regions on the final segmentation results, the attention mechanism SE is proposed. At the same time, the fusion depthwise separable convolution proposed by this model solves the problems of large parameter quantity and difficult training of traditional convolutional networks.

[0080] As Figure 1 shown, a method for segmenting images of uterine squamous epithelial lesion regions based on the DSA-DoubleUNet model described in this invention includes the following steps:

[0081] The first step is the acquisition and preprocessing of cervical acetic acid images. Obtain the cervical acetic acid images of the colposcopy examination. In the colposcopy examination, in order to observe the vagina and cervix more intuitively, a cold light source is often used, which will cause a large number of high-brightness parts in the obtained colposcopy images. These high-brightness parts will have a great interference on the training of the model. Therefore, it is necessary to remove the high-brightness points from the obtained images to improve the accuracy of the model. Since the images of uterine squamous epithelial lesions are few, overfitting will occur in the training of deeper networks. Therefore, it is necessary to expand the processed data set to prevent overfitting during training and improve the generalization ability of the model.

[0082] (1) Obtain the cervical acetic acid image and convert it from the RGB form to the YUV form. The expression is as follows:

[0083]

[0084] Among them, Y represents the brightness of the image; U represents the hue of the image; V represents the saturation of the image.

[0085] (2) Detect the high-brightness part by setting the threshold of the high-brightness part. The expression is as follows:

[0086]

[0087] Among them, p represents each pixel point, and Y(p) represents the brightness of each pixel point.

[0088] (3) Increase the saturation rate of the high-brightness part. The expression is as follows:

[0089] fill(p) = mean(N8(p))

[0090] Among them, N8 represents the 8 pixel points around the pixel point p, and mean(N8(p)) represents taking the average value of the brightness of the 8 pixel points around the pixel point p. fill(p) represents filling the value after taking the average value to the pixel point p.

[0091] (4) Augment the dataset using the transformer module in torchvision:

[0092] Augment the dataset by means of cropping, flipping, and mirroring.

[0093] Among them, the cropping function is.resize, the flipping functions are.hfilp or.vfilp, and the mirroring function is.crop.

[0094] In the second step, construct the DSA-DoubleUNet model.

[0095] As Figure 2 shown, use DoubleUNet as the basic model structure, replace ordinary convolutions with depthwise separable convolutions, introduce the channel attention module SE and the ASPP module, and replace the transposed convolutions used in the decoder part with bilinear interpolation to construct the DSA-DoubleUNet model.

[0096] In previous uterine squamous epithelial lesion segmentation models based on convolutional operations, in order to improve the segmentation accuracy, the method of deepening the network is often adopted. As the network depth increases, the parameters of the network will increase greatly, which puts higher requirements on the hardware equipment for training the model. And such models are also difficult to embed in colposcopy equipment due to their complexity. To solve this problem, the present invention proposes to use depthwise separable convolution operations to replace traditional convolution operations, which greatly reduces the parameters and computational amount of the model while ensuring the segmentation accuracy, reduces the storage requirements, and thus improves the computing speed.

[0097] The acetic acid map and iodine map of the cervix obtained by colposcopy may contain a lot of information that is not needed during segmentation, including some equipment, text, and non-cervical tissues. These irrelevant information will have a great impact on the segmentation results. To improve the suppression of the interference of these irrelevant regions on the final segmentation results, the attention mechanism SE module is proposed, which will make the network pay more attention to the information between channels, automatically learn the importance degree of channel features, so that the model pays more attention to the lesion regions with a large amount of information, and suppresses the expression of unimportant information such as equipment, text, and some non-cervical tissues.

[0098] The ASPP module in the present invention will extract the multi-scale information of the input features of uterine squamous epithelial lesions and then fuse them, so as to capture the context information of the lesion regions at multiple scales, increase the receptive field of the feature map without introducing additional parameters.

[0099] In the upsampling stage of the model, the bilinear interpolation method is used to replace the transposed convolution operation, which reduces the training parameters of the model while ensuring the segmentation accuracy of uterine squamous epithelial lesions and improves the efficiency of the uterine squamous epithelial lesion segmentation model.

[0100] The specific steps are as follows:

[0101] (1) Set the DSA-DoubleUNet network model to concatenate two Unet networks, adopting a U-shaped encoder-decoder structure.

[0102] (2) Set the first Unet model of the DSA-DoubleUNet network to a U-shaped structure of encoder-decoder.

[0103] (3) As Figure 4 shown, construct the SE module as follows:

[0104] A1) Set the SE module to be divided into two key operations: Squeeze and Excitation;

[0105] A2) In Squeeze, adopt global average pooling, that is, calculate the average value of pixels on each channel of the input feature map, so as to compress the size of the input feature map X∈R H×W×C to 1×1×C. The principle of its action is as follows:

[0106]

[0107] where, z c is the output value of the c-th channel, H and W are the length and width of the input image respectively, and u c (i,j) is each pixel value of the c-th channel;

[0108] A3) In Excitation, use a fully connected layer to scale the feature map output by Squeeze to 1×1×C / r, where r is the sampling coefficient and C is the number of channels of the feature map output by Squeeze;

[0109] A4) Subsequently, a fully connected layer restores the number of channels to C, and then passes through the Sigmoid layer to obtain the weights between each channel;

[0110] A5) Finally, through the Scale operation, the weights of each obtained channel are weighted to the features of each channel output by the depthwise separable layer.

[0111] As Figure 5 shown, set the ASPP module to sample the given input in parallel with dilated convolutions with different sampling coefficients r, and then fuse the sampled features.

[0112] (4) It is set that the encoder part of the first Unet model adopts the network structure of VGG19. The fully connected layers of VGG19 are deleted, and the feature extraction layer is retained. The convolutional operations in the feature extraction layer are replaced with depthwise separable convolutions, and an SE module is added after each depthwise separable convolution, BN, and Relu operation.

[0113] Since the VGG19 network is used for classification tasks, the fully connected layers of VGG19 will be deleted in this network model, and the feature extraction layer will be retained. At the same time, in order to reduce the number of parameters in the network, the convolutional operations in the feature extraction layer are replaced with depthwise separable convolutions.

[0114] In theory, the computational complexity of ordinary convolution is 8 to 9 times that of depthwise separable convolution. Suppose the input is D F ×D F ×M, and the output is D F ×D F ×N. The size of the convolutional kernel is D K ×D K . The computational complexity of traditional convolution is:

[0115] D K ×D K ×M×N×D F ×D F

[0116] The computational complexity of depthwise separable convolution is:

[0117] D K ×D K ×M×D F ×D F +M×N×D F ×D F .

[0118] Among them, as Figure 3 shown, depthwise separable convolution is divided into two processes: depthwise convolution DW and pointwise convolution PW.

[0119] One convolutional kernel of DW is responsible for one channel. In this process, the number of channels of the generated feature map is the same as that of the input feature map.

[0120] The size of the convolutional kernel of PW is 1×1×M, where M is the number of channels output by DW convolution. The number of channels of the feature map generated after PW convolution is the same as the number of convolutional kernels used.

[0121] (5) The encoder part of the first Unet model performs four downsampling operations, and then sends the output result to the ASPP module to obtain multi-scale object information. The sampling coefficients r used in the ASPP module are 0, 6, 12, and 18 respectively.

[0122] (6) Send the output of the ASPP module to the decoder end of the first Unet network. The upsampling part of the decoder uses bilinear interpolation instead of transposed convolution. After each upsampling is completed, the decoder end fuses the output of the corresponding feature layer at the encoder end and uses it as the input of the next layer. Four upsamplings will be performed at the corresponding decoder end.

[0123] (7) Multiply the output features of the decoder of the first Unet network and the original input features as the input of the second Unet network and send it to the encoder end of the second Unet network.

[0124] Among them, the encoder of the second Unet network adopts the encoder structure of the original UNet network, replaces the convolution operation of the encoder with depthwise separable convolution, and adds an SE module after each depthwise separable convolution, BN, and Relu operation.

[0125] (8) Send the output of the encoder of the second UNet network to the second ASPP module, where the sampling rates of the ASPP module are 0, 6, 12, and 18;

[0126] (9) Send the output of the second ASPP module to the decoder end of the second Unet network. The upsampling part of the decoder uses bilinear interpolation instead of transposed convolution. After each upsampling is completed, the decoder end fuses the output of the corresponding feature layer at the encoder end and uses it as the input of the next layer. Four upsamplings will be performed at the corresponding decoder end.

[0127] (10) Finally, fuse the output of the first Unet decoder and the output of the second Unet decoder to obtain the final output of the DSA-DoubleUNet model.

[0128] Among them, the encoder of the second Unet network adopts the encoder structure of the original UNet network. It is the same as the encoder part of the first UNet, and also replaces the convolution operation of the encoder with depthwise separable convolution. And an SE module is added after each depthwise separable convolution, BN, and Relu operation.

[0129] The third step, training of the DSA-DoubleUNet model: Input the augmented dataset into the DSA-DoubleUNet model for training. The specific steps are as follows:

[0130] (1) Input the augmented dataset into the DSA-DoubleUNet network model for forward propagation.

[0131] (2) During network training, set the batch size batch_size to 8, the number of epochs epoch to 200, the downsampling factor of both feature extraction networks to 16, and use the SGD optimizer to optimize the model with an initial learning rate of 0.01.

[0132] (3) Calculate the loss after obtaining the output of the network. The loss function is a mixed loss function composed of adding DiceLoss to the cross-entropy loss function. The formula for the loss function is as follows:

[0133] Loss = 0.5 * CE + 0.5 * DiceLoss,

[0134] where CE is the cross-entropy loss, and the calculation formula is:

[0135]

[0136] where N is the number of image pixels, y i represents the label of the i-th pixel, the lesion area is 1, the background is 0, and p i represents the probability value that the i-th pixel is predicted as the lesion area;

[0137] The calculation formula for DiceLoss is;

[0138]

[0139] where X is the ground-truth image and Y is the predicted image.

[0140] (4) According to the loss function, perform backpropagation to update the parameters in the network.

[0141] (5) Repeat the above steps until the loss converges to obtain the weight value with the best prediction effect.

[0142] Fourth step, acquisition and preprocessing of the image to be segmented: Acquire the acetic acid image of the cervix to be segmented and preprocess it.

[0143] Fifth step, segmentation of the image of the uterine squamous epithelial lesion area: Input the preprocessed acetic acid image of the cervix into the trained DSA-DoubleUNet model to obtain the segmentation result of the image of the uterine squamous epithelial lesion area.

[0144] Here, a comparative experiment was designed. The network model in the present invention was effectively verified using the performance evaluation metrics Dice and mIou of semantic segmentation and the time Time spent on network training, and was compared with the traditional UNet network model and the DoubleUNet network model. As Figure 6a shown, it is the acetic acid image obtained by colposcopy, as Figure 6b shown, it is the segmentation result based on the UNet network model, as Figure 6c shown, it is the segmentation result based on the DoubleUNet network model, as Figure 6d shown, it is the segmentation result based on the DSA-DoubleUNet network model.

[0145] From the above comparison charts, it can be concluded that the segmentation effect of the UNet network model is poor, and there will be missegmentation phenomena in some regions; the DoubleUNet network model has a relatively blurred segmentation at the edge regions of the lesion area; the DSA-DoubleUNet network model proposed in the present invention has an excellent edge segmentation effect compared to DoubleUNet, and there is no missegmentation phenomenon in the UNet network model.

[0146] At the same time, in order to compare the training time, the three models were all trained in the same hardware environment.

[0147] Table 1 Comparison table of segmentation metrics of three networks

[0148] UNet DoubleUNet DSA-DoubleUNet mIou 85.2％ 90.9％ 88.4％ Dice 0.881 0.919 0.889 Time 2 hours 32 minutes 3 hours 28 minutes 2 hours 48 minutes

[0149] From Table 1, it can be seen that the training accuracy of the DSA-DoubleUNet network is almost the same as that of DoubleUNet, but the training time of the DSA-DoubleUNet network is about 1.3 times shorter than that of the DoubleUNet network model.

[0150] Thus, it can be concluded that the present invention reduces the number of parameters of the DoubleUNet network model and shortens the training time of the model without sacrificing the segmentation accuracy, and is more practical in practical applications.

[0151] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art of this industry should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification is only the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for image segmentation of uterine squamous epithelial lesion region based on the DSA-DoubleUNet model, characterized in that: The following steps are involved: 11) Acquisition and preprocessing of cervical acetic acid images: Acquire cervical acetic acid images from colposcopy, remove highlights from the images, and expand the processed dataset; 12) Constructing the DSA-DoubleUNet model: Using DoubleUNet as the basic model structure, replacing ordinary convolution with depthwise separable convolution, introducing the channel attention module SE and ASPP module, and replacing the transposed convolution used in the decoder with bilinear interpolation to construct the DSA-DoubleUNet model; The construction of the DSA-DoubleUNet model includes the following steps: 121) The DSA-DoubleUNet network model is set to connect two Unet networks in series, using a U-shaped encoder-decoder structure; 122) Set the first Unet model of the DSA-DoubleUNet network to an encoder-decoder U-shaped structure; 123) Build the SE module, The ASPP module is set to sample the given input in parallel using dilated convolutions with different sampling coefficients r, and then fuse the sampled features; The construction of the SE module includes the following steps: 1231) Set the SE module to be divided into two key operations: Squeeze and Excitation; 1232) In Squeeze, global average pooling is used, that is, the average value of pixels in each channel of the input feature map is calculated, so that the input feature map X∈R H×W×C The size is compressed to 1×1×C, and its working principle is as follows: Among them, z c is the output value of the cth channel, H and W are the length and width of the input image respectively, u c (i, j) is the value of each pixel in the cth channel; 1233) In Excitation, a fully connected layer is used to scale the feature map output by Squeeze to 1×1×C / r, where r is the sampling coefficient and C is the number of channels of the feature map output by Squeeze; 1234) Then a fully connected layer restores the number of channels to C, and then passes through the Sigmoid layer to obtain the weights between each channel; 1235) Finally, the weights of each channel obtained are weighted to the features of each channel output by the depth separable layer through the Scale operation; 124) Set the encoder part of the first Unet model to adopt the network structure of VGG19, delete the fully connected layer of VGG19, retain the feature extraction layer, replace the convolution operation in the feature extraction layer with depthwise separable convolution, and add SE modules after each layer of depthwise separable convolution, BN, and ReLU operations; Among them, the depth separable convolution is divided into two processes: channel-by-channel convolution DW and point-by-point convolution PW. One convolution kernel of DW is responsible for one channel. The number of channels of the feature map generated in this process is the same as the number of channels of the input feature map. The size of the PW convolution kernel is 1×1×M, where M is the number of channels output by the DW convolution. The number of channels of the feature map generated after the PW convolution is the same as the number of convolution kernels used; 125) The encoder part of the first Unet model performs four downsampling operations, and then sends the output results to the ASPP module to obtain multi-scale object information. The sampling coefficients r used in the ASPP module are 0, 6, 12, and 18 respectively; 126) The output of the ASPP module is sent to the decoder end of the first Unet network, wherein the upsampling part of the decoder uses bilinear interpolation instead of transposed convolution. After the first upsampling, the decoder end fuses the features of the fourth downsampling output, and then uses the fused result as the input of the second upsampling; after the second upsampling, the decoder end fuses the features of the third downsampling output, and then uses the fused result as the input of the third upsampling; after the third upsampling, the decoder end fuses the features of the second downsampling output, and then uses the fused result as the input of the fourth upsampling; after the fourth upsampling, the decoder end fuses the features of the first downsampling output, and then outputs the fused result as the result; 127) The output features of the first Unet network decoder and the original input features are multiplied as the input of the second Unet network to the encoder end of the second Unet network. Among them, the encoder of the second UNet network adopts the encoder structure of the original UNet network, replaces the convolution operation of the encoder with depth-wise separable convolution, and adds SE modules after each layer of depth-wise separable convolution, BN, and ReLU operations; 128) Sending the encoder output of the second UNet network to the second ASPP module, where the sampling rates of the ASPP module are 0, 6, 12, and 18; 129) The output of the second ASPP module is sent to the decoder end of the second Unet network, wherein the upsampling part of the decoder uses bilinear interpolation instead of transposed convolution. After the first upsampling, the decoder end fuses the features of the fourth downsampling output, and then uses the fused result as the input of the second upsampling; after the second upsampling, the decoder end fuses the features of the third downsampling output, and then uses the fused result as the input of the third upsampling; after the third upsampling, the decoder end fuses the features of the second downsampling output, and then uses the fused result as the input of the fourth upsampling; after the fourth upsampling, the decoder end fuses the features of the first downsampling output, and then outputs the fused result as the result; 1210) Finally, the output of the first Unet decoder and the output of the second Unet decoder are fused to obtain the final output of the DSA-DoubleUNet model; 13) Training the DSA-DoubleUNet model: Input the expanded dataset into the DSA-DoubleUNet model for training; 14) Acquisition and preprocessing of the image to be segmented: Acquiring the cervical acetic acid image to be segmented and preprocessing it; 15) Segmentation of uterine squamous epithelial lesion area images: The preprocessed cervical acetic acid image is input into the trained DSA-DoubleUNet model to obtain the image segmentation results of the uterine squamous epithelial lesion area.

2. The method for image segmentation of uterine squamous epithelial lesion region based on the DSA-DoubleUNet model according to claim 1, characterized in that: The acquisition and preprocessing of the cervical acetic acid image comprises the following steps: 21) Obtain the cervical acetic acid image and convert it from RGB format to YUV format. The expression is as follows: Among them, Y represents the brightness of the image, U represents the hue of the image, and V represents the saturation of the image; 22) The highlight part is detected by setting the threshold of the highlight part, and its expression is as follows: Where p represents each pixel, and Y(p) represents the brightness of each pixel; 23) Increase the saturation rate of the highlight part. The expression is as follows: fill(p)=mean(N8(p)) Where N8 represents the eight pixels around pixel p, mean(N8(p)) means taking the average brightness of the eight pixels around pixel p, and fill(p) means filling the pixel p with the average value. 24) Use the transformer module in torchvision to expand the dataset: Expand the dataset by cropping, flipping, and mirroring. Among them, the cropping function is .resize, the flipping function is .hfilp or .vfilp, and the mirroring function is .crop.

3. The method for image segmentation of uterine squamous epithelial lesion region based on the DSA-DoubleUNet model according to claim 1, characterized in that: The training of the DSA-DoubleUNet model includes the following steps: 31) Input the expanded dataset into the DSA-DoubleUNet network model for forward propagation; 32) During network training, the batch size batch_size is set to 8, the number of epochs is set to 200, the downsampling multiples of the two feature extraction networks are both 16, the SGD optimizer is used to optimize the model, and the initial learning rate is set to 0.01; 33) After obtaining the output of the network, the loss is calculated. The loss function is a mixed loss function composed of DiceLoss and the cross entropy loss function. The formula of the loss function is as follows: Loss=0.5*CE+0.5*DiceLoss, Among them, CE is the cross entropy loss, and the calculation formula is: Where N is the number of image pixels, y i Indicates the label of the i-th pixel, the lesion area is 1, the background is 0, and p i Represents the probability value of the i-th pixel being predicted as the lesion area; The calculation formula of DiceLoss is: Where X is the ground-truth image and Y is the predicted image; 34) According to the loss function, back propagate and update the parameters in the network; 35) Repeat the above steps until the loss converges and the weight value with the best prediction effect is obtained.

Images