Cell detection method and system based on deep learning and machine vision

Abstract

The application discloses a cell detection method and system based on deep learning and machine vision, which comprises the following steps: cutting a cervical liquid-based cell pathology digital image into a plurality of sub-images to form a sub-image set; inputting the sub-image set into a normal cell detection model and a lesion cell detection model respectively, and identifying normal cells and lesion cells and their categories; selecting an overlapping area of the normal cells and the lesion cells, and performing secondary classification on the overlapping area through a classification model; performing threshold segmentation on granulocytes in the cervical liquid-based cell pathology digital image to obtain all possible granulocyte targets, filtering to determine final granulocyte targets, and counting and determining the number of the granulocytes. The application solves the problem that the detection model in the prior art has poor detection effect due to the imbalance of normal cell and lesion cell categories, improves the accuracy of cell identification, and realizes accurate detection and identification of cervical liquid-based cells in combination with accurately identified granulocytes.

Description

Technical Field

[0001] This invention relates to the field of medical image-assisted diagnostic technology, and in particular to a cell detection method and system based on deep learning and machine vision. Background Technology

[0002] Cervical cancer is a very common gynecological malignancy, ranking second in incidence among female malignancies, after breast cancer. In its development, cervical cancer often progresses from primary to advanced stages; therefore, early prevention and diagnosis can effectively prevent more serious cancerous changes.

[0003] As one of the three pillars of pathological diagnosis, cytopathology plays a crucial role. It involves sampling the lesion site, preparing cell smears, and performing cytopathological analysis under a microscope to obtain the final diagnostic result. Currently, the commonly used method for cervical cancer screening is liquid-based cytopathology (LC-C). Pathologists observe all fields of view on the slide under a microscope to determine whether lesions have occurred in each area. This method is highly dependent on the pathologist's experience and cytological knowledge. Furthermore, a single slide provides a wide field of view, often requiring 5-10 minutes of diagnostic time, resulting in a heavy workload, diagnostic fatigue, and consequently, misdiagnosis and missed diagnosis. Moreover, the current number of practicing pathologists is insufficient to meet the current screening needs.

[0004] With the development of digital pathology technology, pathological slides can be scanned into digital images, and computers can be used to identify and detect cervical cancer cells, assisting doctors in diagnosis and screening.

[0005] Currently, there are various liquid-based cytology screening tools and systems available for cervical cancer screening. However, most of these systems can only detect common abnormal squamous cell lesions such as HSIL, LSIL, and ASC-US. Normal squamous cells (including surface cells, middle cells, and basal cells) and granulocytes (including neutrophils and lymphocytes) are also of reference value for physicians in assisting diagnosis and assessing inflammation. Existing screening systems cannot screen for normal squamous cells and granulocytes.

[0006] Moreover, in a typical cervical liquid-based cytology digital slide, the number of normal cells is often much larger than the number of diseased cells. On the one hand, this difference in the number of normal and diseased cells cannot be changed when training the detection model. Therefore, due to class imbalance, the detection model is more likely to learn more normal targets (i.e., normal cells) and fewer abnormal targets (i.e., diseased cells) are difficult to detect effectively. On the other hand, granulocytes and other abnormal cells are often densely distributed and very numerous, making it difficult to label the image data. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a cell detection method and system based on deep learning and machine vision. This method automates the detection of cervical liquid-based cytology digital images to assist physicians in diagnosis. Utilizing a diseased cell detection model and a normal cell detection model, combined with a classification model, it quickly and accurately locates diseased and normal cells, accurately detecting their types and quantities. This solves the problem of poor detection performance in existing technologies due to an imbalance between normal and diseased cell categories, improving the accuracy of cell identification. Simultaneously, machine vision technology is used to accurately segment and count granulocytes in the pathological image, thereby enabling the detection and identification of cervical liquid-based cytology cells.

[0008] In one aspect, this disclosure provides a cell detection method based on deep learning and machine vision:

[0009] A cell detection method based on deep learning and machine vision includes:

[0010] The digital image of cervical liquid-based cytology is cropped into several sub-images to form a sub-image set, and the relative position coordinates of each sub-image in the whole digital image of cervical liquid-based cytology are recorded.

[0011] The sub-image sets are input one by one into the normal cell detection model and the lesion cell detection model to identify normal cells and lesion cells, and to determine the category, region coordinates and confidence level of normal cells and lesion cells.

[0012] Select the overlapping area between the normal cell area and the diseased cell area, and perform secondary classification on the overlapping area using a classification model to determine whether the cells in the overlapping area are normal cells or diseased cells.

[0013] Threshold segmentation was performed on granulocytes in liquid-based cytology digital images of cervical cells to obtain all possible granulocyte targets. The final granulocyte targets were then determined by filtering, and the number of granulocytes was counted and determined.

[0014] A further technical solution is that the training process of the diseased cell detection model is as follows:

[0015] Select multiple liquid-based cytology digital images of the cervix and pre-label all types of lesion cells and their categories using rectangular boxes;

[0016] Randomly crop the liquid-based cytology digital images of the cervix to obtain several image regions of the same size, which are then used as training sample images for the lesion cell detection model.

[0017] Data augmentation is performed on the training sample images, and the augmented training sample images are input into the lesion cell detection model for iterative training. The lesion cell detection model is completed after the model loss converges.

[0018] A further technical solution is that the random cropping specifically involves: taking the lesion cell as the center, randomly generating at least two random offsets for cropping to obtain several image regions of the same size.

[0019] A further technical solution is that the training process of the normal cell detection model is as follows:

[0020] Multiple liquid-based cytology digital images of the cervix were cropped to obtain several image regions of the same size. Some of these image regions were selected as training sample images for the normal cell detection model.

[0021] Use bounding boxes to label all categories of normal cells and their categories in the training sample images;

[0022] Data augmentation is performed on the training sample images, and the augmented training sample images are input into the normal cell detection model for iterative training. Once the model loss converges, the normal cell detection model training is complete.

[0023] A further technical solution is that the training process of the classification model is as follows: all categories of lesion cells and normal cells in the cervical liquid-based cytology digital image are pre-labeled with rectangular boxes and the target area image is cropped out as a training sample image. After data augmentation, the training sample image is input into the classification model for training. The training is completed after the model loss converges.

[0024] A further technical solution involves thresholding granulocytes in cervical liquid-based cytology digital images to obtain all possible granulocyte targets, including:

[0025] The digital images of cervical liquid-based cytology were converted to grayscale to obtain grayscale images.

[0026] Thresholding is applied to the grayscale image to obtain a binary image;

[0027] Perform a target contour search operation on the binary image to obtain the contour information of all targets in the foreground;

[0028] Based on the contour information of all targets, calculate the area, perimeter, roundness, and solidity of each target.

[0029] Targets are filtered based on the area, perimeter, sphericity, and firmness of each target to obtain all possible granulocyte targets.

[0030] A further technical solution involves filtering to determine the final granulocyte target. Specifically, among all possible target contours of granulocytes, targets within diseased cell regions and normal cell regions are filtered out, while granulocyte targets outside the cell regions are retained to obtain the final granulocyte target.

[0031] Secondly, this disclosure provides a cell detection system based on deep learning and machine vision, comprising:

[0032] The image preprocessing module is used to cut the cervical liquid-based cytology digital image into several sub-images to form a sub-image set, and record the relative position coordinates of each sub-image in the entire cervical liquid-based cytology digital image;

[0033] The normal cell detection module is used to input sub-image sets one by one into the normal cell detection model, identify normal cells, and determine the category, region coordinates, and confidence level of normal cells.

[0034] The lesion cell detection module is used to input the sub-image set into the lesion cell detection model one by one, identify lesion cells, and determine the category, region coordinates and confidence level of lesion cells;

[0035] The secondary classification module is used to select the overlapping area between normal cell area and diseased cell area, and to perform secondary classification on the overlapping area through the classification model to determine whether the cells in the overlapping area are normal cells or diseased cells.

[0036] The granulocyte detection module is used to perform threshold segmentation of granulocytes in cervical liquid-based cytology digital images to obtain all possible granulocyte targets, filter to determine the final granulocyte targets, and count and determine the number of granulocytes.

[0037] Thirdly, this disclosure also provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the steps of the method described in the first aspect.

[0038] Fourthly, this disclosure also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the steps of the method described in the first aspect.

[0039] The above one or more technical solutions have the following beneficial effects:

[0040] 1. This invention provides a cell detection method and system based on deep learning and machine vision, which automates the detection of digital images of cervical liquid-based cytology to assist physician diagnosis. Utilizing a diseased cell detection model and a normal cell detection model, combined with a classification model, it quickly and accurately locates diseased and normal cells, accurately detecting their types and quantities. This solves the problem of poor detection performance in existing technologies due to an imbalance between normal and diseased cell categories, improving the accuracy of cell identification. Simultaneously, machine vision technology is used to accurately segment and count granulocytes in the pathological image, thereby achieving cervical liquid-based cytology-assisted diagnosis.

[0041] 2. This disclosure utilizes deep learning target detection technology, which can not only quickly locate all lesion areas in cervical liquid-based cytology digital images, but also accurately detect the area and number of normal cells, solving the problem of poor detection effect and incomplete cell category identification caused by the imbalance between normal and lesion cell categories in existing detection models; it uses image classification technology to perform secondary precise classification of overlapping areas, improving the recognition accuracy; and it uses machine vision technology to segment dense granulocytes, which is of great significance for the judgment of inflammation and solves the problem of difficulty in labeling dense targets. Attached Figure Description

[0042] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0043] Figure 1 This is an overall flowchart of the cell detection method based on deep learning and machine vision as described in Embodiment 1 of the present invention;

[0044] Figure 2 This is a flowchart of granulocyte detection as described in Embodiment 1 of the present invention. Detailed Implementation

[0045] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0046] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0047] To address the shortcomings of existing cervical liquid-based cytology screening methods, such as poor detection results and inability to detect normal and granulocytes, this invention provides a cell detection method and system based on deep learning and machine vision. This method provides automated assisted diagnosis of cervical liquid-based cytology pathological digital images. Utilizing diseased cell detection models and normal cell detection models, combined with a classification model, it quickly and accurately locates diseased and normal cells, detecting their types and quantities. Simultaneously, machine vision technology is used to accurately segment and count granulocytes in the pathological images, thereby achieving the detection and identification of cervical liquid-based cytology and assisting physicians in subsequent diagnosis.

[0048] Example 1

[0049] This embodiment provides a cell detection method based on deep learning and machine vision, such as... Figure 1 As shown, it includes:

[0050] The liquid-based cytology digital image of the cervix is ​​cropped into several sub-images to form a sub-image set, and the relative position coordinates of each sub-image in the whole liquid-based cytology digital image are recorded.

[0051] The sub-image sets are input into the normal cell detection model and the lesion cell detection model respectively to detect the regions where normal cells and lesion cells are located, as well as the types of normal cells and lesion cells;

[0052] Calculate the overlapping area between normal cells and diseased cells, and then use a classification model to perform secondary classification of the overlapping area to determine the cell category in the overlapping area;

[0053] Threshold segmentation was performed on granulocytes in liquid-based cytology digital images of cervical cells to obtain all possible granulocyte targets. The final granulocyte targets were then determined by filtering, and the number of granulocytes was counted and determined.

[0054] In this embodiment, firstly, a cervical liquid-based cytology slide is scanned using a scanner to obtain a whole-slide image (WSI), i.e., a digital image of the cervical liquid-based cytology. During the scanning process, the scanner can achieve different resolutions such as 40x, 20x, and 5x. In this embodiment, although the 40x image has higher resolution, it occupies more memory, resulting in slower processing speed. 20x resolution is sufficient for the required clarity and offers faster processing speed. Therefore, this embodiment selects the whole-slide WSI image at 20x scanner resolution and crops this image into several 1000×1000 pixel image regions, thus obtaining several sub-images, which constitute a sub-image set. The resolution of slide images is typically tens of thousands of pixels × tens of thousands of pixels. In this embodiment, the slide image resolution is 50,000 pixels × 50,000 pixels. The image is too large to be directly fed into the model; therefore, the slide image is cropped into small image blocks for processing.

[0055] While cropping the image into multiple sub-images, the relative position coordinates of each sub-image within the entire cervical liquid-based cytology digital image are recorded to facilitate subsequent statistics and processing.

[0056] Then, the obtained sub-image sets are input image by image into the lesion cell detection model and the normal cell detection model for target detection, respectively, to obtain the category, region coordinates, and confidence score of all lesion cells and the category, region coordinates, and confidence score of all normal cells. To further clarify the cell location, the region coordinates of lesion cells and normal cells in the sub-images output by the detection models are combined with the relative position coordinates of the sub-images within the entire slice image recorded in the previous steps to obtain the coordinates of lesion cells and normal cells in the sub-image set within the entire slice image.

[0057] The training process for the above-mentioned disease cell detection model is as follows:

[0058] Select multiple liquid-based cytology slides at 20x magnification and pre-label all types of lesion cells and their categories using rectangles;

[0059] Using a random cropping method, three random offsets are randomly generated with the lesion cell as the center for cropping, resulting in several image regions of 1000×1000 pixels, which are used as training sample images for the lesion cell detection model.

[0060] Data augmentation of training sample images is performed using conventional methods such as decoding, flipping, and normalization. The augmented training sample images are then input into a lesion cell detection model using the Faster R-CNN model structure for several iterations of training. Once the model loss converges, the lesion cell detection model training is complete.

[0061] The training process for the above normal cell detection model is as follows:

[0062] Multiple liquid-based cytology slides of the cervix were cropped at 20x magnification to obtain several image regions of 1000×1000 pixels. Some of these image regions were selected as training sample images for the normal cell detection model.

[0063] Use bounding boxes to label all categories of normal cells and their categories in the training sample images;

[0064] Data augmentation of training sample images is performed using conventional methods such as decoding, flipping, and normalization. The augmented training sample images are then input into a normal cell detection model using the Faster R-CNN model structure for several iterations of training. Once the model loss converges, the normal cell detection model training is complete.

[0065] The training processes for the aforementioned lesion cell detection model and normal cell detection model differ in that, during the training of the lesion cell detection model, lesion cells are labeled first and then cropped according to the labeled positions. This is because the relative position of the target (i.e., lesion cells) in the slice is fixed. In contrast, during the training of the normal cell detection model, normal cells are cropped first and then a portion of the image is selected for labeling. This is because there are a large number of normal cells in each slice, making it difficult to label all normal cells in each slice. Instead, only a portion of the image area in each slice is selected for labeling. This ensures the accuracy of the final detection results while reducing labeling time and the waste of manpower and resources.

[0066] The above scheme utilizes a lesion cell detection model to detect lesion cells in pathological digital images, and a normal cell detection model to detect normal cells in the same images. However, since the cell regions detected by the lesion cell detection model and the normal cell detection model are not precise—for example, a certain area of ​​cells might be detected as a lesion cell by the lesion cell detection model, while the normal cell detection model might detect it as a normal cell—this embodiment further includes selecting the overlapping area of ​​normal cells and lesion cells, and using a classification model to perform secondary classification on the overlapping area to determine the cell category within it.

[0067] First, for the coordinates of the lesion cell region and the normal cell region in the obtained whole pathological digital slide, calculate the IOU (Intersection over Union) between each lesion cell region and any surrounding normal cell region, and filter out cell regions with an overlap greater than 0.5.

[0068] Then, for cell regions with an overlap exceeding a threshold, the overlapping region image is cropped and input into the classification model for further classification.

[0069] Specifically, the classification model uses the ResNet34 model structure. Its training process is as follows: all categories of lesion cells and normal cells in the cervical liquid-based cytology digital image are pre-labeled with rectangular boxes and the target area image is cropped as a training sample image. After data augmentation methods such as rotation, scaling, and translation are applied to the training sample image, it is input into the ResNet34 classification model for training. The training is completed after the model loss converges, thereby achieving secondary accurate classification of cell types in overlapping areas.

[0070] If the classification model detects that the cells in the overlapping region are normal cells, the result identified by the normal cell detection model is retained; if the classification model detects that the cells in the overlapping region are diseased cells, the result identified by the diseased cell detection model is retained.

[0071] After detecting and identifying normal and diseased cells in the liquid-based cytology digital image of the cervix, granulocytes in the pathology digital image are then detected and identified.

[0072] Specifically, such as Figure 2 As shown, firstly, the entire liquid-based cytology digital image of the cervix is ​​converted to grayscale, transforming the RGB three-channel color image into a grayscale image with only one channel.

[0073] OTU thresholding is performed on the grayscale image to convert the image containing pixel values ​​in the range of 0-255 into a binary image consisting only of 0 and 255, thus segmenting the foreground and background containing the target.

[0074] Perform a target contour search operation on the binary image to obtain the contour information of all targets in the foreground.

[0075] Based on the contour information of all targets, the area, perimeter, convex hull, convex hull area, circumcircle, and circumcircle area of ​​each target are calculated, with the unit of measurement being pixels. The roundness and solidity of each target are then calculated using the following formulas:

[0076]

[0077]

[0078] In the above formula, Area object For the target area, Area cjrcumcircle Area is the area of ​​the circumcircle. convexhull This represents the area of ​​the convex hull.

[0079] Targets are filtered based on their area, perimeter, granularity, and firmness to obtain possible target contours for all granulocytes, i.e., all possible granulocyte regions. Specifically, targets meeting the filtering criteria are selected as possible target contours for all granulocytes. These filtering criteria include: target area greater than 50 pixels and less than 500 pixels; target perimeter greater than 20 pixels and less than 200 pixels; target granularity less than 0.3; and target firmness greater than 0.5.

[0080] From all possible target contours of granulocytes, targets within diseased cell regions and normal cell regions are filtered out, retaining granulocyte targets outside the cell regions. This yields the final granulocyte targets, and the number of final granulocyte targets is counted. Granulocyte information is stored in a list data structure; the granulocyte count can be obtained by directly obtaining the length of this list.

[0081] Using the above method, the regional coordinates, categories, and confidence levels of all diseased cells and all normal cells in the liquid-based cytology digital image of the cervix were obtained, as well as the number of all granulocytes. Physicians made a diagnosis based on this accurate data and obtained the final diagnostic result.

[0082] In summary, the solution described in this embodiment utilizes deep learning target detection technology, which can not only quickly locate all lesion areas in cervical liquid-based cytology digital images, but also accurately detect the area and number of normal cells. This solves the problem of poor detection performance and incomplete cell category identification caused by the imbalance between normal and lesion cell types in existing detection models. Image classification technology is used to perform secondary precise classification of overlapping areas, improving the recognition accuracy. Machine vision technology is used to segment dense granulocytes, which is of great significance for the judgment of inflammation and solves the problem of difficulty in labeling dense targets.

[0083] Example 2

[0084] This embodiment provides a cell detection system based on deep learning and machine vision, including:

[0085] The image preprocessing module is used to cut the cervical liquid-based cytology digital image into several sub-images to form a sub-image set, and record the relative position coordinates of each sub-image in the entire cervical liquid-based cytology digital image;

[0086] The normal cell detection module is used to input sub-image sets one by one into the normal cell detection model, identify normal cells, and determine the category, region coordinates, and confidence level of normal cells.

[0087] The lesion cell detection module is used to input the sub-image set into the lesion cell detection model one by one, identify lesion cells, and determine the category, region coordinates and confidence level of lesion cells;

[0088] The secondary classification module is used to select the overlapping area between normal cell area and diseased cell area, and to perform secondary classification on the overlapping area through the classification model to determine whether the cells in the overlapping area are normal cells or diseased cells.

[0089] The granulocyte detection module is used to perform threshold segmentation of granulocytes in cervical liquid-based cytology digital images to obtain all possible granulocyte targets, filter to determine the final granulocyte targets, and count and determine the number of granulocytes.

[0090] In this embodiment, firstly, a cervical liquid-based cytology slide is scanned using a scanner to obtain a whole-slide image (WSI), i.e., a digital image of the cervical liquid-based cytology. During the scanning process, the scanner can achieve different resolutions such as 40x, 20x, and 5x. In this embodiment, although the 40x image has higher resolution, it occupies more memory, resulting in slower processing speed. 20x resolution is sufficient for the required clarity and offers faster processing speed. Therefore, this embodiment selects the whole-slide WSI image at 20x scanner resolution and crops this image into several 1000×1000 pixel image regions, thus obtaining several sub-images, which constitute a sub-image set. The resolution of slide images is typically tens of thousands of pixels × tens of thousands of pixels. In this embodiment, the slide image resolution is 50,000 pixels × 50,000 pixels. The image is too large to be directly fed into the model; therefore, the slide image is cropped into small image blocks for processing.

[0091] While cropping the image into multiple sub-images, the relative position coordinates of each sub-image within the entire cervical liquid-based cytology digital image are recorded to facilitate subsequent statistics and processing.

[0092] Then, the obtained sub-image sets are input image by image into the lesion cell detection module and the normal cell detection module for target detection, respectively, to obtain the category, region coordinates, and confidence score of all lesion cells and the category, region coordinates, and confidence score of all normal cells. The lesion cell detection module includes a lesion cell detection model, which is trained using the training method described in Example 1; the normal cell detection module includes a normal cell detection model, which is trained using the training method described in Example 1.

[0093] Images labeled with diseased cell categories, region coordinates, and confidence levels, and images labeled with normal cell categories, region coordinates, and confidence levels, respectively, output by the diseased cell detection module and the normal cell detection module, are input into the secondary classification module. The secondary classification module calculates the Intersection over Union (IOU) between diseased cell regions and normal cell regions, filtering out cell regions with an overlap greater than 0.5. For cell regions with an overlap exceeding a threshold, the target region image is cropped and input into the classification model for further classification to determine whether the cells in the overlapping region are diseased cells or normal cells. This classification model is trained using the method described in Example 1.

[0094] After detecting and identifying normal and diseased cells in the liquid-based cytology digital image of the cervix, granulocytes in the pathology digital image are then detected and identified.

[0095] First, the entire liquid-based cytology digital image of the cervix is ​​converted to grayscale, transforming the RGB three-channel color image into a grayscale image with only one channel.

[0096] OTU thresholding is performed on the grayscale image to convert the image containing pixel values ​​in the range of 0-255 into a binary image consisting only of 0 and 255, thus segmenting the foreground and background containing the target.

[0097] Perform a target contour search operation on the binary image to obtain the contour information of all targets in the foreground.

[0098] Based on the contour information of all targets, the area, perimeter, convex hull, convex hull area, circumcircle, and circumcircle area of ​​each target are calculated, and the granularity and solidity of each target are calculated accordingly.

[0099] Targets are filtered based on their area, perimeter, granularity, and firmness to obtain possible target contours for all granulocytes, i.e., all possible granulocyte regions. Specifically, targets meeting the filtering criteria are selected as possible target contours for all granulocytes. These filtering criteria include: target area greater than 50 pixels and less than 500 pixels; target perimeter greater than 20 pixels and less than 200 pixels; target granularity less than 0.3; and target firmness greater than 0.5.

[0100] From all possible target contours of granulocytes, targets within diseased cell regions and normal cell regions are filtered out, retaining granulocyte targets outside the cell regions. This yields the final granulocyte targets, and the number of final granulocyte targets is counted. Granulocyte information is stored in a list data structure; the granulocyte count can be obtained by directly obtaining the length of this list.

[0101] Through the aforementioned system, the identification system ultimately outputs the regional coordinates, categories, and confidence levels of all diseased cells and all normal cells in the cervical liquid-based cytology digital image, as well as the number of all granulocytes. Physicians can make diagnoses based on this accurate data, assisting in obtaining the final diagnostic results.

[0102] Example 3

[0103] This embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the processor executes the computer instructions, it performs the steps in the cell detection method based on deep learning and machine vision as described above.

[0104] Example 4

[0105] This embodiment also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps in the cell detection method based on deep learning and machine vision as described above.

[0106] The steps and methods involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0107] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0108] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0109] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A cell detection method based on deep learning and machine vision, characterized in that, include: The digital image of cervical liquid-based cytology is cropped into several sub-images to form a sub-image set, and the relative position coordinates of each sub-image in the whole digital image of cervical liquid-based cytology are recorded. The sub-image sets are input one by one into the normal cell detection model and the lesion cell detection model to identify normal cells and lesion cells, and to determine the category, region coordinates and confidence level of normal cells and lesion cells. The overlapping regions of normal and diseased cell regions are selected, and a classification model is used to perform secondary classification on the overlapping regions to determine whether the cells in the overlapping regions are normal or diseased cells. First, for the coordinates of the diseased cell regions and the coordinates of the normal cell regions in the obtained whole pathological digital slice, the overlap ratio (IOU) between each diseased cell region and any surrounding normal cell region is calculated, and cell regions with an overlap ratio greater than 0.5 are filtered out. For cell regions with an overlap ratio exceeding a threshold, the overlapping region image is cropped and input into the classification model for further classification. The classification model uses the ResNet34 model structure. Its training process is as follows: all categories of lesion cells and normal cells in the cervical liquid-based cytology digital image are pre-labeled with rectangular boxes and the target area image is cropped out as training sample image. After data augmentation, the training sample image is input into the classification model for training. The training is completed after the model loss converges. Threshold segmentation was performed on granulocytes in liquid-based cytology digital images of cervical cells to obtain all possible granulocyte targets. The final granulocyte targets were then determined by filtering, and the number of granulocytes was counted and determined. The final granulocyte target is determined by filtering. Specifically, among all possible granulocyte target profiles, targets within diseased cell regions and normal cell regions are filtered out, while granulocyte targets outside the cell regions are retained to obtain the final granulocyte target.

2. The cell detection method based on deep learning and machine vision as described in claim 1, characterized in that, The training process of the diseased cell detection model is as follows: Select multiple liquid-based cytology digital images of the cervix and pre-label all types of lesion cells and their categories using rectangular boxes; Randomly crop the liquid-based cytology digital images of the cervix to obtain several image regions of the same size, which are then used as training sample images for the lesion cell detection model. Data augmentation is performed on the training sample images, and the augmented training sample images are input into the lesion cell detection model for iterative training. The lesion cell detection model is completed after the model loss converges.

3. The cell detection method based on deep learning and machine vision as described in claim 1, characterized in that, The training process of the normal cell detection model is as follows: Multiple liquid-based cytology digital images of the cervix were cropped to obtain several image regions of the same size. Some of these image regions were selected as training sample images for the normal cell detection model. Use bounding boxes to label all categories of normal cells and their categories in the training sample images; Data augmentation is performed on the training sample images, and the augmented training sample images are input into the normal cell detection model for iterative training. Once the model loss converges, the normal cell detection model training is complete.

4. The cell detection method based on deep learning and machine vision as described in claim 1, characterized in that, Threshold segmentation was performed on granulocytes in liquid-based cytology digital images of the cervix to obtain all possible granulocyte targets, including: The digital images of cervical liquid-based cytology were converted to grayscale to obtain grayscale images. Thresholding is applied to the grayscale image to obtain a binary image; Perform a target contour search operation on the binary image to obtain the contour information of all targets in the foreground; Based on the contour information of all targets, calculate the area, perimeter, roundness, and solidity of each target. Targets are filtered based on the area, perimeter, sphericity, and firmness of each target to obtain all possible granulocyte targets.

5. A cell detection system based on deep learning and machine vision, employing a cell detection method based on deep learning and machine vision as described in any one of claims 1-4, characterized in that, include: The image preprocessing module is used to cut the cervical liquid-based cytology digital image into several sub-images to form a sub-image set, and record the relative position coordinates of each sub-image in the entire cervical liquid-based cytology digital image; The normal cell detection module is used to input sub-image sets one by one into the normal cell detection model, identify normal cells, and determine the category, region coordinates, and confidence level of normal cells. The lesion cell detection module is used to input the sub-image set into the lesion cell detection model one by one, identify lesion cells, and determine the category, region coordinates and confidence level of lesion cells; The secondary classification module is used to select the overlapping area between normal cell area and diseased cell area, and to perform secondary classification on the overlapping area through the classification model to determine whether the cells in the overlapping area are normal cells or diseased cells. The granulocyte detection module is used to perform threshold segmentation of granulocytes in cervical liquid-based cytology digital images to obtain all possible granulocyte targets, filter to determine the final granulocyte targets, and count and determine the number of granulocytes.

6. The cell detection system based on deep learning and machine vision as described in claim 5, characterized in that, The secondary classification module includes a classification model. The training process of the classification model is as follows: all categories of lesion cells and normal cells in the cervical liquid-based cytology digital image are pre-labeled with rectangular boxes and the target area image is cropped out as a training sample image. After data augmentation, the training sample image is input into the classification model for training. The training is completed after the model loss converges.

7. An electronic device, characterized in that: It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, complete the steps of a cell detection method based on deep learning and machine vision as described in any one of claims 1-4.

8. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of a cell detection method based on deep learning and machine vision as described in any one of claims 1-4.

Images