A weakly supervised method for locating abnormal regions of cells

Abstract

The application relates to a weakly-supervised cell abnormal region positioning method, which comprises the following steps: collecting a cell image, forming a training set, and pre-processing training set data based on a ResNet50 neural network model; training the ResNet50 neural network model based on the pre-processed training set until a termination condition is met, so that a trained ResNet50 neural network model is obtained; inputting a cell image to be positioned into the trained ResNet50 neural network model, and outputting an abnormal cell region positioning result. Compared with traditional manual screening of abnormal cells, the abnormal cell positioning method of the application labels training set data based on weakly-supervised learning, avoids the need for a large amount of manual labeling in full-supervised learning, greatly reduces the threshold and time of training set data labeling, effectively improves the labeling efficiency, better realizes the learning of the neural network model, improves the training effect, then preliminarily screens abnormal cells through the trained neural network model, and labels the abnormal cell regions in a cytological section image, so that subsequent review is facilitated.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, specifically relating to a method for locating abnormal cell regions based on weak supervision. Background Technology

[0002] Cervical cancer is the second leading cause of cancer death among women. Early screening of women of appropriate age can effectively reduce its incidence and mortality. Studies have shown that liquid-based cytology and human papillomavirus (HPV) testing are the main screening methods for cervical cancer.

[0003] Currently, most cytology screening still relies on manual labeling of abnormal cells by labelers based on cytology slides and their experience. This method requires labelers to possess both specialized medical knowledge and extensive experience. While the widespread adoption of deep learning and research in pathological artificial intelligence have improved the efficiency of abnormal cell screening to some extent, traditional fully supervised learning still requires a massive amount of manual labeling work and places high demands on the labelers' medical knowledge and experience. Given the huge volume of cervical cytology screenings and the severe shortage of cytology pathologists, cervical cytology labeling and intelligent analysis based on weakly supervised learning are essential. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a method for locating abnormal cell regions based on weak supervision. The method involves labeling a dataset used for training a neural network based on weak supervision, and then training the network on the labeled dataset to locate abnormal cells.

[0005] The technical solution for achieving the objective of this invention is as follows:

[0006] A method for locating abnormal cellular regions based on weak supervision includes the following steps:

[0007] Collect cell images to form a training set;

[0008] Preprocessing of the training set data based on the ResNet50 neural network model;

[0009] The ResNet50 neural network model is trained based on the preprocessed training set until the termination condition is met, resulting in the trained ResNet50 neural network model.

[0010] The cell image to be located is input into the trained ResNet50 neural network model, which outputs the localization results of abnormal cell regions.

[0011] As a preferred embodiment, the process of acquiring cell images is as follows:

[0012] Cell localization and segmentation are performed on cytology slide images with multiple known diagnostic results. The position of each cell is recorded, and the cytology slide images are divided into multiple image blocks, so that only one cell is retained in each image block.

[0013] As a preferred embodiment, the preprocessing process is as follows:

[0014] The segmented image patch is input into the ResNet50 neural network model for forward inference, and the probability of cell abnormalities within the image patch is output.

[0015] All segmented image blocks of each cytology slide image are sorted according to the probability of abnormality, and image blocks are selected to represent the cytology slide image based on the abnormality diagnosis results of the cytology slide image.

[0016] Select an image patch representing the cytology slice image from all the cytology slice images in the training set.

[0017] In a preferred embodiment, the basis for selecting an image block to represent the cytological section image based on the abnormal diagnosis result of the cytological section image is as follows:

[0018] When the abnormal diagnosis result of the cytology slide image is positive, the image block with the highest abnormal probability is selected as the representative of the slide.

[0019] When the abnormal diagnosis result of the cytology slide image is negative, the image block with the lowest abnormal probability is selected as the representative of the slide.

[0020] As a preferred implementation method, the process of training the ResNet50 neural network model is as follows:

[0021] All selected image patches representing each cytology slice are used as the training set and input into the ResNet50 neural network model for iterative training until the training meets the set termination condition, thus obtaining the final ResNet50 neural network model.

[0022] In a preferred implementation, the number of iterations for the iterative training is set to 200, the Adam optimizer is selected for the iterative training, and the initial learning rate for the iterative training is set to 0.001.

[0023] In a preferred embodiment, the termination condition is set as follows:

[0024] (1) The accuracy of the output results of the ResNet50 neural network model is higher than the set threshold;

[0025] (2) The number of training iterations of the ResNet50 neural network model exceeds the set threshold;

[0026] When any of the above conditions are met, the training is considered to have met the termination condition, and the training ends.

[0027] As a preferred embodiment, the process of locating the abnormal cell region is as follows:

[0028] Cell localization and segmentation are performed on the cytology slide image to be located, the position of each cell is recorded, and the cytology slide image is divided into multiple image blocks so that only one cell is retained in each image block;

[0029] The segmented image blocks are input into the trained ResNet50 neural network model. Based on the abnormal cell probability within the image block output by the ResNet50 neural network model, the image block is considered to be an abnormal image block when the abnormal probability is greater than a set threshold.

[0030] Based on the location of cells in the abnormal image block in the cytological slide image to be located, all abnormal cell regions are marked in the cytological slide image to be located, thus completing the localization of abnormal cell regions.

[0031] As a preferred implementation, when segmenting the cytology slide image, if the cytology slide image is scanned using a 10x eyepiece and a 20x objective lens, the position of each cell is recorded and a 128*128dpi image block is cut from the corresponding position on the cytology slide to complete the localization and segmentation of the cytology slide image.

[0032] Compared with the prior art, the significant advantages and beneficial effects of this invention are as follows:

[0033] (1) Compared with traditional artificial screening of abnormal cells, the abnormal cell localization method of the present invention can perform preliminary screening of abnormal cells through a trained neural network model and mark the abnormal cell area in the cytological slide image for easy subsequent verification.

[0034] (2) The technical solution of the present invention is based on weakly supervised learning to label training set data, avoiding the need for a large number of manual labels in fully supervised learning, greatly reducing the threshold and time for labeling training set data, effectively improving the efficiency of labeling, better realizing the learning of neural network models, and improving training effect;

[0035] (3) As part of the artificial intelligence-assisted diagnostic system for pathological cytology, it is more suitable for assisting clinical diagnosis based on external technical means than existing technical solutions.

[0036] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0037] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0038] Figure 1 This is a flowchart of the steps of the cell abnormality region localization method based on weak supervision of the present invention.

[0039] Figure 2 This is a schematic diagram of the overall structure of the ResNet50 neural network used in this invention. Detailed Implementation

[0040] It is readily understood that, based on the technical solution of this invention, various embodiments of the invention can be conceived by those skilled in the art without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention. Rather, these embodiments are provided to enable those skilled in the art to gain a more thorough understanding of the invention. Preferred embodiments of the invention are described below in conjunction with the accompanying drawings, which form part of this application and, together with the embodiments of the invention, serve to illustrate the innovative concept of the invention.

[0041] like Figure 1 The diagram shows the steps of the weakly supervised cell abnormality region localization method of the present invention, which includes the following steps:

[0042] Cell images were collected to form a training set, specifically as follows:

[0043] First, obtain cytological slide images with known diagnostic results (i.e., positive or negative). In this embodiment, cervical liquid-based cytology slide images are obtained. All cervical liquid-based cytology slide images have known diagnostic results, that is, the case corresponding to the slide image is positive or negative for cervical cancer.

[0044] Next, cell localization and segmentation were performed on the cytology slide images, the position of each cell was recorded, and the cytology slide images were divided into multiple image blocks so that only one cell was retained in each image block.

[0045] The size of the segmented image patch here is determined by the scanning magnification of the cytology slide image, with the standard that the segmented image patch contains only one cell;

[0046] Taking this embodiment as an example, the cytology slide image in this embodiment uses a scanning imaging magnification of 10x eyepiece × 20x objective lens. Under this magnification, the size of the segmented image block is approximately 128 × 128 dpi.

[0047] The training set data is preprocessed based on the ResNet50 neural network model, specifically as follows:

[0048] The segmented image blocks are adjusted to 224×224dpi and input into the ResNet50 neural network model for forward inference, outputting the probability of cell abnormalities within the image blocks.

[0049] All segmented image blocks of each cytology slide image are sorted according to the probability of abnormality, and image blocks are selected to represent the cytology slide image based on the abnormality diagnosis results of the cytology slide image.

[0050] The rules for selecting representative image patches are as follows:

[0051] When the abnormal diagnosis result of the cytology slide image is positive, the image block with the highest abnormal probability is selected as the representative of the slide.

[0052] When the abnormal diagnosis result of the cytology slide image is negative, the image block with the lowest abnormal probability is selected as the representative of the slide.

[0053] For all cytology slide images in the training set, select an image patch to represent that cytology slide image. At this time, an image patch will be selected to represent each cytology slide image. The number of image patches is equal to the number of cytology slides, and each image patch has the same category as its corresponding cytology slide image. Here, the category refers to whether it is abnormal.

[0054] At this point, all the image patches obtained based on forward inference have been labeled, and these labeled image patches are used as the new training set.

[0055] The ResNet50 neural network model is trained based on the preprocessed training set until the termination condition is met, resulting in the trained ResNet50 neural network model, specifically:

[0056] All selected image patches representing each cytology slice are used as new training sets and input into the ResNet50 neural network model for iterative training until the training meets the set termination condition, thus obtaining the final ResNet50 neural network model.

[0057] The termination condition is set as follows:

[0058] (1) The accuracy of the output results of the ResNet50 neural network model is higher than the set threshold;

[0059] (2) The number of training iterations of the ResNet50 neural network model exceeds the set threshold;

[0060] When any of the above conditions are met, the training is considered to have met the termination condition, and the training ends.

[0061] In this embodiment, the ResNet50 neural network model is trained for a total of 200 rounds using the Adam optimizer with an initial learning rate of 0.001. After training, the final cell identification model is obtained.

[0062] The cell image to be located is input into the trained ResNet50 neural network model, which outputs the abnormal cell region localization result, specifically:

[0063] Cell localization and segmentation are performed on the cytology slide image to be located, the position of each cell is recorded, and the cytology slide image is divided into multiple image blocks so that only one cell is retained in each image block;

[0064] The segmented image blocks are input into the trained ResNet50 neural network model. Based on the abnormal cell probability within the image block output by the ResNet50 neural network model, the image block is considered to be an abnormal image block when the abnormal probability is greater than a set threshold.

[0065] Based on the location of cells in the abnormal image block in the cytological slide image to be located, all abnormal cell regions are marked in the cytological slide image to be located, thus completing the localization of abnormal cell regions.

[0066] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

[0067] It should be understood that, in order to simplify the present invention and help those skilled in the art understand its various aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes described in a single embodiment or with reference to a single figure. However, the present invention should not be construed as including all features in the exemplary embodiments as essential technical features of the claims of this patent.

[0068] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0069] It should be understood that the modules, units, components, etc., included in the device of one embodiment of the present invention can be adaptively changed to be placed in a device different from that embodiment. Different modules, units, or components included in the device of the embodiment can be combined into a single module, unit, or component, or they can be divided into multiple sub-modules, sub-units, or sub-components.

[0070] The modules, units, or components in the embodiments of the present invention can be implemented in hardware, in software running on one or more processors, or in a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement the embodiments of the present invention. The present invention can also be implemented on a computer program product or computer-readable medium for performing some or all of the methods described herein.

Claims

1. A weakly supervised based cell abnormal region localization method, characterized in that, The method comprises the following steps: collecting cell images to form a training set; preprocessing training set data based on a ResNet50 neural network model; training the ResNet50 neural network model based on the preprocessed training set until a termination condition is met, to obtain a trained ResNet50 neural network model; inputting a cell image to be positioned into the trained ResNet50 neural network model to output an abnormal cell region positioning result; the process of collecting cell images comprises: performing cell positioning and segmentation on a plurality of cytological slice images with known diagnostic results, recording the position of each cell, and segmenting the cytological slice images into a plurality of image blocks so that only one cell is retained in each image block; the preprocessing process comprises: selecting an image block representing each cytological slice image from all training set cytological slice images; the process of positioning an abnormal cell region comprises: performing cell positioning and segmentation on a cytological slice image to be positioned, recording the position of each cell, and segmenting the cytological slice image into a plurality of image blocks so that only one cell is retained in each image block; inputting the segmented image blocks into the trained ResNet50 neural network model, and determining that an image block is an abnormal image block when the abnormal probability of the cell in the image block output by the ResNet50 neural network model is greater than a set threshold value; labeling all abnormal cell regions in the cytological slice image to be positioned according to the positioning of the abnormal image blocks in the cytological slice image to be positioned, to complete the positioning of the abnormal cell regions; the cells are cervical cells.

2. The method for locating abnormal cell regions as described in claim 1, characterized in that, The preprocessing process further comprises: inputting the segmented image blocks into the ResNet50 neural network model for forward inference to output the abnormal probability of the cell in the image block; sorting all segmented image blocks of each cytological slice image according to the abnormal probability, and selecting an image block to represent the cytological slice image according to the abnormal diagnostic result of the cytological slice image.

3. The method of claim 2, wherein the step of determining the abnormal region of the cell comprises the steps of: determining a first abnormal region of the cell; and determining a second abnormal region of the cell. The basis for selecting an image block to represent a cytological slice image according to the abnormal diagnostic result of the cytological slice image is: when the abnormal diagnostic result of the cytological slice image is positive, the image block with the highest abnormal probability is selected as the representative of the slice; when the abnormal diagnostic result of the cytological slice image is negative, the image block with the lowest abnormal probability is selected as the representative of the slice.

4. The method for locating abnormal cell regions as described in claim 2, characterized in that, The process of training the ResNet50 neural network model comprises: inputting all selected image blocks representing each cytological slice image into the ResNet50 neural network model for iterative training until the training meets a set termination condition, to obtain a final ResNet50 neural network model.

5. The method of claim 4, wherein the step of determining the abnormal region of the cell comprises the steps of: determining a first abnormal region of the cell; and determining a second abnormal region of the cell. The number of iterations of the iterative training is set to 200, the Adam optimizer is selected for the iterative training, and the initial learning rate of the iterative training is set to 0.

001.

6. The method for locating abnormal cell regions as described in claim 4, characterized in that, The termination condition is set as: (1) the accuracy of the output result of the ResNet50 neural network model is higher than a set threshold value; (2) ResNet50 neural network model training iteration number is greater than the set threshold value; When any of the above conditions are met, it is considered that the training meets the end condition, and the training is ended.

7. The method for locating abnormal cell regions as described in claim 1, characterized in that, When the cytological section image is segmented, when the cytological section image is scanned and imaged by using a scanning magnification of 10 times objective lens and 20 times objective lens, the position of each cell is recorded and a 128*128 dpi image block is cut at the corresponding position on the cytological section, so as to complete the positioning and segmentation of the cytological section image.

Images