Cervical cell nucleus image segmentation method and system based on residual network and hollow convolution

Abstract

The application provides a cervical cell nucleus image segmentation method and system based on a residual network and a hollow convolution, a low-layer feature extraction network of sequential convolution is reused, low-layer feature information is added to high-layer information, the low-layer information and the high-layer information are complementary, the utilization rate of a low-layer feature layer with more detailed information is improved, and the learning ability of the network for cells is enhanced.

Description

Technical Field

[0001] This invention belongs to the field of image segmentation technology, and particularly relates to a method and system for cervical cell nucleus image segmentation based on residual networks and dilated convolution. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Cervical cancer is a common gynecological disease, causing approximately 40,000 to 80,000 deaths annually in my country, posing a significant threat to women's health. The progression of cervical cancer from early lesions to obvious symptoms and then to rapid deterioration takes about 8-10 years. Medical research indicates that cervical cancer is almost completely curable if detected early. However, early-stage cervical cancer often presents with no obvious symptoms, making it easy to miss the optimal treatment window. Therefore, early screening and treatment of cervical cancer have become a hot topic.

[0004] In recent years, with the continuous development of artificial intelligence, computer-aided medical diagnosis has become a hot topic in scientific research. Cervical cell image segmentation methods can be divided into two categories: traditional machine learning algorithms and deep learning algorithms. Traditional algorithms mainly include thresholding, edge detection, and watershed algorithms. Thresholding algorithms segment for cases where the foreground and background have significant differences, but they lack structural information in the target space, resulting in poor segmentation performance for images with complex backgrounds. Edge detection is suitable for extracting contours of cellular and non-cellular regions, but it is sensitive to noise. While the watershed algorithm does not require much preprocessing, it does require prior knowledge of seed points. Further improving the accuracy of traditional algorithms is difficult. In contrast, deep learning methods have shone brightly in cell segmentation, demonstrating stronger generalization ability in solving complex cell segmentation problems. Initially, fully convolutional neural networks (FCNs) were applied to cervical cell image segmentation. However, due to insufficient sensitivity to image details and a lack of consideration for pixel relationships and spatial consistency, the accuracy of cell nucleus segmentation was low. Mask-RCNN instance segmentation combines object detection and semantic segmentation. This network can achieve high-quality image segmentation while accurately detecting objects. However, when extracting features from small objects, images are prone to unavoidable and irreversible information loss. Reducing the loss of image information is the key to improving segmentation accuracy. Cell nucleus segmentation plays a decisive role in the classification of cancer, and using deep neural networks for accurate segmentation has important application value. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention provides a method and system for cervical cell nucleus image segmentation based on residual networks and dilated convolutions. By reusing the low-level feature extraction network that is convolved sequentially, the low-level feature information is added to the high-level information. The low-level information and the high-level information complement each other, improve the utilization rate of the low-level feature layer with more detailed information, and enhance the network's learning ability of cells.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solution: a cervical cell nucleus image segmentation method based on residual networks and dilated convolution, comprising:

[0007] Cervical cell images were acquired and preprocessed, and the preprocessed cervical cell images were divided into training and testing sets.

[0008] A segmentation model is constructed, which includes a feature extraction network that reuses sequential convolutions and an instance segmentation network; the feature extraction network that reuses sequential convolutions is used to add low-level feature information to high-level feature information.

[0009] The segmentation model is trained using cervical cell images from the training set to obtain a trained segmentation model;

[0010] The cervical cell images from the test set are used to input the trained segmentation model to output the segmentation results.

[0011] A second aspect of the present invention provides a cervical cell nucleus image segmentation system based on residual networks and dilated convolution, comprising:

[0012] The image acquisition and processing module is configured to: acquire cervical cell images and perform preprocessing, and divide the preprocessed cervical cell images into training and testing sets;

[0013] The model building module is configured to: build a segmentation model, the segmentation model including a feature extraction network that reuses sequential convolutions and an instance segmentation network; the feature extraction network that reuses sequential convolutions is used to add low-level feature information to high-level feature information;

[0014] The training module is configured to: train the segmentation model using cervical cell images from the training set to obtain a trained segmentation model;

[0015] The output module is configured to output the cervical cell images from the test set to the trained segmentation model and output the segmentation results.

[0016] A third aspect of the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the steps described in the above method.

[0017] A fourth aspect of the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the steps described in the above method.

[0018] The above one or more technical solutions have the following beneficial effects:

[0019] This invention employs a low-level feature extraction network that reuses sequential convolutions in the feature extraction of cervical cell images. By adding low-level feature information to high-level information, the low-level and high-level information complement each other, improving the utilization rate of the low-level feature layer with more detailed information and enhancing the network's learning ability for cells.

[0020] This invention expands the receptive field of convolution by introducing a dilated convolution algorithm, thereby further improving the accuracy of mask prediction.

[0021] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0023] Figure 1 This is a schematic diagram of the overall method in Embodiment 1 of the present invention;

[0024] Figure 2 This is a schematic diagram of the overall segmentation model network structure in Embodiment 1 of the present invention;

[0025] Figure 3 This is a schematic diagram of the low-level feature extraction network structure that reuses sequential convolutions in Embodiment 1 of the present invention;

[0026] Figure 4 This is a schematic diagram of the dilated convolution process in Embodiment 1 of the present invention;

[0027] Figure 5 This is a schematic diagram illustrating the verification of segmentation accuracy in this invention;

[0028] Figure 6 This is a diagram illustrating the segmentation mask of the present invention. Detailed Implementation

[0029] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0030] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0031] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0032] Example 1

[0033] like Figures 1-6 As shown, this embodiment discloses a method for cervical cell nucleus image segmentation based on residual networks and dilated convolution, including:

[0034] Cervical cell images were acquired and preprocessed, and the preprocessed cervical cell images were divided into training and testing sets.

[0035] A segmentation model is constructed, which includes a feature extraction network that reuses sequential convolutions and an instance segmentation network; the feature extraction network that reuses sequential convolutions is used to add low-level feature information to high-level feature information.

[0036] The segmentation model is trained using cervical cell images from the training set to obtain a trained segmentation model;

[0037] The cervical cell images from the test set are used to input the trained segmentation model to output the segmentation results.

[0038] In this embodiment, ISBI2014 is selected as the experimental dataset. The data is preprocessed to generate the required COCO dataset, which is then divided into a test set and a training set.

[0039] In this embodiment, the imbalance of sample data is compensated by preprocessing the cervical cell nucleus images through methods such as flipping, rotating, cropping, and contrast adjustment.

[0040] In this embodiment, the segmentation model is based on Mask RCNN, with improvements made to its backbone network to achieve accurate segmentation of cervical cell nuclei. The segmentation model uses a ResNet residual network and a Feature Pyramid Network (FPN) as its backbone network, and is improved by reusing a low-level feature extraction network with sequential convolutions and introducing dilated convolutions into the FPN.

[0041] like Figure 2As shown, specifically, we reuse the low-level feature extraction network with sequential convolutions, using the ResNet residual network as the backbone, and improve upon the ResNet residual network. The existing ResNet has 5 convolutional groups. The input size of the first convolutional group is 224x224, and the output size of the fifth convolutional group is 7x7, a reduction of 32 (2^5) times. Each reduction is 2 times, for a total of 5 reductions, and the convolutional results from low to high are denoted as C1, C2, C3, C4, and C5, respectively. To improve the utilization of low-level feature information and achieve specific segmentation of cell nuclei in cervical cell images, we improve the initial compressed convolution result C1.

[0042] In the low-level feature extraction network that reuses sequential convolutions, the first compressed feature map is defined as C1. Feature map C1 is then improved by performing 256 convolutions with a stride of 1 and a size of 3×3 to obtain feature map C1C2. Then, feature map C1 is subjected to 256 convolutions with a stride of 2 and a size of 3×3 to obtain feature map C1C3. Similarly, feature map C1 is subjected to 256 convolutions with a stride of 4 and a size of 3×3 to obtain feature map C1C4. The width and height of feature map C1 are then compressed two, three, four, and five times to obtain feature maps C2-C5. Feature maps C2-C5 are then used to construct the feature pyramid. The feature pyramid fully utilizes the extracted feature maps C2-C5, enabling the rapid construction of a feature pyramid with strong semantic information at all scales from a single input image at a single scale, without incurring significant costs.

[0043] The smallest compressed feature layer C5 is convolved once to obtain P5, and then max-pooled to obtain P6. Simultaneously, C5 is convolved and then upsampled once. The upsampled result is accumulated with the channel-unified C4 and C1C4, and the accumulated result is convolved with 256 channels to obtain P4. This accumulated result is then upsampled again and accumulated with the channel-unified C3 and C1C3, and the result is convolved with 256 channels to obtain P3. Similarly, the accumulated result is upsampled again and accumulated with the channel-unified C2 and C1C2 to obtain P2. Through the construction of the feature pyramid in the above manner, five effective feature layers are obtained, and the extracted {P2, P3, P4, P5, P6} are used as the effective feature layers of the RPN network.

[0044] like Figure 3 As shown, in this embodiment, dilated convolution is added to the feature pyramid structure FPN. Dilated convolution reduces the network loss and improves the network's segmentation ability without increasing the amount of computation.

[0045] When performing instance segmentation tasks, Convolutional Neural Networks (CNNs) often result in feature maps that are significantly smaller than the input image size. This leads to coarse-grained segmentation masks due to oversampling. Dilated convolution algorithms, by controlling the kernel rate, can achieve receptive fields of varying sizes.

[0046] like Figure 3 As shown, the left image illustrates the receptive field of a traditional 3×3 convolution kernel, which is the same as the receptive field of a 3×3 dilated convolution kernel with rate=1, covering a 3×3 area each time. The right image corresponds to the receptive field of a 3×3 dilated convolution kernel with rate=1. This kernel is still 3×3, but its receptive field is increased to 7×7.

[0047] In this embodiment, two different rates of dilated convolution kernels are added to the Feature Pyramid Network (FPN) to extract features. Dilated convolution operations (including two different dilated convolution kernels) are performed on the last layer output features of each pyramid stage P2, P3, P4, P5, and P6. This effectively preserves high-level feature information in high-level features with small spatial dimensions.

[0048] The classification network uses feature maps and candidate regions generated by the RPN network to detect target categories. P2-P6 extracted from the backbone network are used as input to the effective feature layers of the RPN to select candidate targets. A softmax classifier is used to determine the category of the candidate targets, while a bounding box regressor is used to refine the position of the candidate targets, generating candidate target regions. Finally, the FCN uses the feature maps to achieve pixel-level precise segmentation of the targets.

[0049] The experimental environment and parameter settings of this embodiment are as follows: The network uses a GPU, each GPU trains two images, and the training model with 24 training epochs uses the stochastic gradient descent algorithm with a momentum factor set to 0.9

[38] . The selected optimizer is SGD, the batch size is set to 2, the initial learning rate is 0.001, and the initial learning rate is reduced in the 8th and 11th epochs. The weights are decayed to 0.0001 to avoid overfitting.

[0050] Example 2

[0051] The purpose of this embodiment is to provide a cervical cell nucleus image segmentation system based on residual networks and dilated convolution, characterized by comprising:

[0052] The image acquisition and processing module is configured to: acquire cervical cell images and perform preprocessing, and divide the preprocessed cervical cell images into training and testing sets;

[0053] The model building module is configured to: build a segmentation model, the segmentation model including a feature extraction network that reuses sequential convolutions and an instance segmentation network; the feature extraction network that reuses sequential convolutions is used to add low-level feature information to high-level feature information;

[0054] The training module is configured to: train the segmentation model using cervical cell images from the training set to obtain a trained segmentation model;

[0055] The output module is configured to output the cervical cell images from the test set to the trained segmentation model and output the segmentation results.

[0056] Example 3

[0057] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0058] Example 4

[0059] The purpose of this embodiment is to provide a computer-readable storage medium.

[0060] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0061] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0062] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0063] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for cervical cell nucleus image segmentation based on residual networks and dilated convolution, characterized in that, include: Cervical cell images were acquired and preprocessed, and the preprocessed cervical cell images were divided into training and testing sets. A segmentation model is constructed, which includes a feature extraction network that reuses sequential convolutions and an instance segmentation network; the feature extraction network that reuses sequential convolutions is used to add low-level feature information to high-level feature information. The segmentation model is trained using cervical cell images from the training set to obtain a trained segmentation model; The cervical cell images from the test set are used to input the trained segmentation model to output the segmentation results; The feature extraction network that reuses sequential convolutions uses a ResNet residual network as its backbone. The preprocessed cervical cell image, after being compressed, is convolved through three convolutional blocks to obtain feature maps C1C2, C1C3, and C1C4. Feature map C1 is compressed 2, 3, 4, and 5 times respectively to obtain feature maps C2, C3, C4, and C5; Convolution is performed on feature map C5 to obtain feature map P5, and feature map P5 is then subjected to max pooling to obtain feature map P6. Feature map P4 is obtained by convolving and upsampling feature map C5, and then summing it with feature map C4 and feature map C1C4. After upsampling feature map P4, it is accumulated with feature map C3 and feature map C1C3, and then convolved with 256 channels to obtain feature map P3. After upsampling feature map P3, it is added to feature map C2 and feature map C1C2 to obtain feature map P2; A feature pyramid network based on P2, P3, P4, P5, and P6 is constructed to achieve multi-scale fusion. The instance segmentation network uses a backbone network and a feature pyramid network as the main network, and adds dilated convolutions to the last layer of the feature pyramid network P2, P3, P4, P5, and P6.

2. The cervical cell nucleus image segmentation method based on residual networks and dilated convolution as described in claim 1, characterized in that, The preprocessing includes: flipping, rotating, cropping, and contrast adjustment.

3. A cervical cell nucleus image segmentation system based on residual networks and dilated convolution, characterized in that, include: The image acquisition and processing module is configured to: acquire cervical cell images and perform preprocessing, and divide the preprocessed cervical cell images into training and testing sets; The model building module is configured to: build a segmentation model, the segmentation model including a feature extraction network that reuses sequential convolutions and an instance segmentation network; the feature extraction network that reuses sequential convolutions is used to add low-level feature information to high-level feature information; The feature extraction network that reuses sequential convolutions uses a ResNet residual network as its backbone. The preprocessed cervical cell image, after being compressed, is convolved through three convolutional blocks to obtain feature maps C1C2, C1C3, and C1C4. Feature map C1 is compressed 2, 3, 4, and 5 times respectively to obtain feature maps C2, C3, C4, and C5; Convolution is performed on feature map C5 to obtain feature map P5, and feature map P5 is then subjected to max pooling to obtain feature map P6. Feature map P4 is obtained by convolving and upsampling feature map C5, and then summing it with feature map C4 and feature map C1C4. After upsampling feature map P4, it is accumulated with feature map C3 and feature map C1C3, and then convolved with 256 channels to obtain feature map P3. After upsampling feature map P3, it is added to feature map C2 and feature map C1C2 to obtain feature map P2; A feature pyramid network based on P2, P3, P4, P5, and P6 is constructed to achieve multi-scale fusion. The instance segmentation network uses a feature pyramid network as its backbone, and dilated convolutions are added to the last layer of the feature pyramid network P2, P3, P4, P5, and P6. The training module is configured to: train the segmentation model using cervical cell images from the training set to obtain a trained segmentation model; The output module is configured to output the cervical cell images from the test set to the trained segmentation model and output the segmentation results.

4. The cervical cell nucleus image segmentation system based on residual networks and dilated convolution as described in claim 3, characterized in that, In the image acquisition and processing module, the preprocessing includes: flipping, rotating, cropping, and contrast adjustment.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the cervical cell nucleus image segmentation method based on residual networks and dilated convolution as described in any one of claims 1-2.

6. A processing apparatus, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the cervical cell nucleus image segmentation method based on residual networks and dilated convolution as described in any one of claims 1-2.

Images