Cervical cancer screening support system, cervical cancer screening support method, cervical cancer screening support program, and smartphone application

Abstract

A cervical cancer screening assistance system 3 comprises: an image acquisition unit 10 for acquiring a micrograph of cells sampled from the cervical region; a cell aggregate recognition unit 60 for recognizing cell aggregates in the micrograph; and an output unit 70 for outputting a test classification applicable to the cells belonging to the cell aggregate.

Description

[Technical Field]

[0001] The present invention relates to a cervical cancer screening support system, a cervical cancer screening support method, a cervical cancer screening support program, and a smartphone application. [Background technology]

[0002] In cytology for cervical cancer screening, a technology has been proposed that uses a general object detection method based on deep learning to realize screening such as automatic detection and discrimination of abnormal cells (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Jith, OUN; Harinarayanan, KK; Gautam, S.; Bhavsar, A.; Sao, AK DeepCerv: Deep Neural Network for Segmentation Free Robust Cervical Cell Classification. In Computational Pathology and Ophthalmic Medical Image Analysis; Lecture Notes in Computer Science; Springer: Berlin, Germany, 2018; pp. 86.94. [Non-patent document 2] Gautam S, K. ​​HK, Jith N, Sao AK, Bhavsar A, Natarajan A. "Considerations for a PAP Smear Image Analysis System with CNN Features", 2018. [Non-patent document 3] K. Bora, M. Chowdhury, LB Mahanta, MK Kundu and AK Das, “Pap Smear Image Classification Using Convolutional Neural Network”, Tenth Indian Conference on Computer Vision, Graphics and Image Processing , 2016. [Non-patent document 4] Zhang, Hang, et al. ResNeSt: Split-Attention Networks. 2020, http: / / arxiv.org / abs / 2004.08955. [Non-patent document 5] Saso Dzeroski, Bernard Zenko, Machine Learning, 54, 255-273, 2004 Summary of the Invention [Problem to be solved by the invention]

[0004] The method described in Non-Patent Document 1 enables the detection of cells with findings and classification of malignancy in multiple cytology images (images containing multiple cells in a single image). However, because this method has an average recall rate of less than 70%, it is expected that positive results will be overlooked, i.e., false negatives will increase. In other words, although the application of such general object detection methods to cytology is useful, its low accuracy makes it insufficient as a screening method.

[0005] The present invention has been made in view of the above circumstances, and its object is to provide a highly accurate screening method for cytological diagnosis in cervical cancer screening. [Means for solving the problem]

[0006] In order to solve the above problem, one embodiment of the cervical cancer screening support system of the present invention comprises an image acquisition unit that acquires a microscopic photograph of cells collected from the cervix, a cell clump recognition unit that recognizes cell clumps in the microscopic photograph, and an output unit that outputs the test classification to which cells belonging to the cell clumps belong.

[0007] In the above-described cervical cancer screening support system, the region containing the cell clump may also include the background surrounding the cell clump.

[0008] In the above-described cervical cancer screening support system, the cell clump recognition unit may recognize cell clumps using the YOLO algorithm.

[0009] In the above-described cervical cancer screening support system, the cell clump recognition unit may recognize cell clumps in real time.

[0010] The above-mentioned cervical cancer screening support system may further include an estimation unit that, when a micrograph acquired by the image acquisition unit is input, estimates and outputs the positions of potentially abnormal cells in the micrograph and the test result classification to which each of the potentially abnormal cells falls, using an estimation model generated by machine learning with an object detection algorithm using, as learning data, markings on cell clumps that contain atypical cells among the cell clumps recognized by the cell clump recognition unit and the atypical classification of the cells contained in the cell clumps.

[0011] Another embodiment of the present invention is also a cervical cancer screening support system. This system includes: an image acquisition unit that acquires micrographs of cells collected from the cervix; a first estimation unit that, when a micrograph acquired by the image acquisition unit is input, estimates and outputs the positions of the potentially abnormal cells in the micrograph and the test result classifications to which each of the potentially abnormal cells corresponds, using a first estimation model generated by machine learning with an object detection algorithm, using the micrographs, the positions of potentially abnormal cells in the micrographs, and the test result classifications to which each of the potentially abnormal cells corresponds as learning data; an image conversion unit that extracts images of the cells at the positions estimated by the first estimation unit from the micrograph and converts each of the extracted images into a converted image in a predetermined format; and a second estimation unit that, when a converted image is input, estimates and outputs the probability that the cells in the converted image correspond to each of the test result classifications, using a second estimation model generated by machine learning with an image classification algorithm, using the images of the potentially abnormal cells and the probabilities that the potentially abnormal cells correspond to each of the test result classifications as learning data.

[0012] In the above-described cervical cancer screening support system, the test result classification may be the Bethesda classification.

[0013] In the above-mentioned cervical cancer screening support system, the object detection algorithm may be the YOLO algorithm.

[0014] In the above-mentioned cervical cancer screening support system, the image classification algorithm may be a convolutional neural network.

[0015] The above-mentioned cervical cancer screening support system may further include a third estimation unit that estimates and outputs the probability that a potentially abnormal cell falls into each of the test result classifications using a third estimation model created by integrating the test result classification to which the potentially abnormal cell falls, estimated using the first estimation model, and the probability that a cell in the converted image falls into each of the test result classifications, estimated using the second estimation model.

[0016] In the above-described cervical cancer screening support system, the third estimation model may be created using ensemble learning by stacking.

[0017] The above-mentioned cervical cancer screening support system may include a smartphone having an image acquisition unit, and a data processing device that is network-connected to the smartphone and has a first estimation unit, an image conversion unit, and a second estimation unit.

[0018] In the above-described cervical cancer screening support system, the data processing device may be connectable to an external device via a network.

[0019] In the above-mentioned cervical cancer screening support system, the data processing device may include a database that stores the positions of potentially abnormal cells estimated by the first estimation unit, the test result classifications to which the potentially abnormal cells belong, and the probability that the cells in the converted image estimated by the second estimation unit belong to each of the test result classifications.

[0020] The above-mentioned cervical cancer screening support system may be network-connected to an external system equipped with the above-mentioned cervical cancer screening support system, and the database may store the locations of potentially abnormal cells estimated by a first estimation unit of the external system, the test result classifications to which the potentially abnormal cells belong, and the probability that the cells in the converted image estimated by a second estimation unit of the external system belong to each of the test result classifications.

[0021] The above-mentioned cervical cancer screening support system may include the above-mentioned cell clump recognition unit.

[0022] Another aspect of the present invention is a cervical cancer screening support method, which includes the steps of acquiring a micrograph of cells collected from the cervix using an image acquisition unit, recognizing cell clumps in the micrograph, and outputting a test classification to which cells belonging to the cell clumps are classified.

[0023] Another aspect of the present invention is a cervical cancer screening support method, which includes the steps of: acquiring a micrograph of cells collected from the cervix using an image acquisition unit; a first estimation step of estimating, when the micrograph acquired by the image acquisition unit is input, the positions of potentially abnormal cells in the micrograph and the test result classifications to which each of the potentially abnormal cells corresponds, using a first estimation model generated by machine learning with an object detection algorithm using the micrograph, the positions of potentially abnormal cells in the micrograph, and the test result classifications to which each of the potentially abnormal cells corresponds as learning data; an image conversion step of extracting images of each cell located at the positions estimated in the first estimation step from the micrograph and converting each extracted image into a converted image in a predetermined format; and a second step of estimating, when the converted image is input, the probability that cells in the converted image correspond to each of the test result classifications, using a second estimation model generated by machine learning with an image classification algorithm using the images of the potentially abnormal cells and the probabilities that the potentially abnormal cells correspond to each of the test result classifications as learning data.

[0024] Yet another aspect of the present invention is a cervical cancer screening support program that causes a computer to execute a method including the steps of acquiring a micrograph of cells collected from the cervix using an image acquisition unit, a cell clump recognition step of recognizing cell clumps in the micrograph, and an output step of outputting a test classification to which cells belonging to the cell clumps belong.

[0025] Yet another aspect of the present invention is a cervical cancer screening support program, which causes a computer to execute a method including the steps of: acquiring a micrograph of cells collected from the cervix using an image acquisition unit; a first estimation step of estimating, when the micrograph acquired by the image acquisition unit is input, the positions of potentially abnormal cells in the micrograph and the test result classifications to which each of the potentially abnormal cells corresponds, using a first estimation model generated by machine learning with an object detection algorithm using the micrograph, the positions of potentially abnormal cells in the micrograph, and the test result classifications to which each of the potentially abnormal cells corresponds, as learning data; an image conversion step of extracting from the micrograph images of each cell located at the positions estimated in the first estimation step and converting each extracted image into a converted image in a predetermined format; and a second step of estimating, when the converted image is input, the probability that cells in the converted image correspond to each of the test result classifications, using a second estimation model generated by machine learning with an image classification algorithm using the images of potentially abnormal cells and the probabilities that the potentially abnormal cells correspond to each of the test result classifications as learning data.

[0026] Yet another aspect of the present invention is a smartphone application that includes the above-described cervical cancer screening support program.

[0027] In addition, any combination of the above components, or mutual substitution of the components or expressions of the present invention between methods, devices, programs, temporary or non-temporary storage media on which programs are recorded, systems, etc., are also valid aspects of the present invention. [Effects of the Invention]

[0028] According to the present invention, a highly accurate screening method can be provided for cytological diagnosis in cervical cancer screening. [Brief explanation of the drawings]

[0029] [Figure 1] 1 is a functional block diagram of a cervical cancer screening support system according to a first embodiment. [Figure 2] FIG. 10 is a functional block diagram of a cervical cancer screening support system according to a second embodiment. [Figure 3] 1 is a schematic diagram showing an example of construction of a cervical cancer screening support system according to a first embodiment. [Figure 4] FIG. 2 is a schematic diagram showing another example of construction of the cervical cancer screening support system according to the first embodiment. [Figure 5] 10 is a flowchart showing the process of a cervical cancer screening support method according to a third embodiment. [Figure 6] FIG. 10 is a schematic diagram illustrating an application for a smartphone according to a fifth embodiment. [Figure 7] FIG. 13 is a functional block diagram of a cervical cancer screening support system according to a sixth embodiment. [Figure 8] FIG. 13 is a functional block diagram of a cervical cancer screening support system according to a seventh embodiment. [Figure 9] 13 is a flowchart showing the processing of a cervical cancer screening support method according to an eighth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0030] The present invention will be described below based on preferred embodiments with reference to the drawings. In the embodiments and modifications, identical or equivalent components and parts are designated by the same reference numerals, and redundant description will be omitted where appropriate. The dimensions of the components in the drawings are enlarged or reduced as appropriate for ease of understanding. Some elements that are not important for explaining the embodiments are omitted from the drawings. Terms including ordinal numbers such as "first" and "second" are used to describe various components, but these terms are used only to distinguish one component from another and do not limit the components.

[0031] Before describing specific embodiments, some basic findings will be explained. Cervical cancer is the most common gynecological malignant tumor, ranking first in incidence and third in mortality among cancers affecting women aged 0 to 49 in Japan. From the perspective of prioritizing life extension, hysterectomy is currently the mainstream treatment for cervical cancer. If cervical cancer is detected early, it is possible to preserve the uterus through procedures such as partial conization. However, if tumor infiltration is confirmed, a total hysterectomy is required, and if metastasis to other sites occurs, radiation therapy or anticancer drug treatment is required. For these reasons, early detection through cervical cancer screening is strongly desired.

[0032] Cervical cancer screening is generally performed using a two-stage diagnostic process called cytoscreening, based on a scraping cytology test. In the first stage (hereinafter referred to as "primary screening"), a cytotechnologist examines the cells collected by scraping the cervix under a microscope and makes a presumptive diagnosis of which of the following Bethesda classifications the cells fall into. In the second stage (hereinafter referred to as "secondary screening"), a cytology specialist makes a final diagnosis based on the microscopic observation and the presumptive diagnosis results mentioned above.

[0033] The Bethesda classification is a method of presenting cytology results, classifying them according to malignancy (but only for squamous epithelial cells) as follows: NILM (negative) ASC-US (suspected mild squamous intraepithelial lesion) LSIL (mild squamous intraepithelial lesion) ASC-H (suspected high-grade squamous intraepithelial lesion) HSIL (high-grade squamous intraepithelial lesion) SCC (suspected squamous cell carcinoma) Of these, NILM is judged to be normal, but ASC-US requires further examination such as HPV testing, and LSIL-SCC requires further examination such as colposcopy-guided targeted biopsy.

[0034] The problem with site screening is that it places a heavy burden on specialists and cytotechnologists. Site screening requires advanced diagnostics that maximize sensitivity to ensure no cancer patients are overlooked, while maximizing specificity to avoid detecting positive results in normal patients. Furthermore, to accommodate the increase in patients due to the recent increase in opportunities for mass cervical cancer screening, there is also a need to speed up screening. These demands are placing an ever-increasing burden on specialists and cytotechnologists.

[0035] Another problem with site screening is that clear, mechanically identifiable diagnostic criteria have not been defined. While there are important features in site screening, there is a great deal of ambiguity regarding the quantitative thresholds for determining whether a cell is abnormal based on those features. As a result, diagnostic ability depends on the experience of the examiner, and diagnostic results vary depending on the hospital and examiner.

[0036] Systems that support site screening include products such as Becton Dickinson's "FocalPoint" (registered trademark) and HOLOGIC's "ThinPrep" (registered trademark). These recognize the size and color intensity of cell nuclei and present multiple fields of view where abnormal cells are likely to be present, but they are not involved in diagnosis at all. In other words, these products can present images that include suspicious areas, but they cannot indicate the type of disease suspected. In this respect, they are insufficient to solve the above-mentioned problems related to site screening. Furthermore, these products have the problem of being very expensive in terms of both initial and running costs.

[0037] Realizing automated screening using information processing technology is expected to reduce the burden on specialists and enable consistent diagnoses. Prior art for automated cytology image classification has proposed numerous methods for classifying single cytology images (images containing a single cell). These include a two-class classification method (normal / abnormal) (e.g., Non-Patent Document 2) and a multi-class classification method (e.g., Non-Patent Document 3) for determining malignancy, both of which achieve highly accurate classification. However, these methods target single cytology images, which are relatively low-noise, and therefore require the ability to accurately capture single-cell features. For this reason, they cannot be directly applied to images containing multiple cells, which are used in cytology screening.

[0038] Similarly to the above, there are two-class classification methods for multiple cytology images: normal / abnormal (e.g., Non-Patent Document 4), and multi-class classification methods for determining the degree of malignancy (e.g., Non-Patent Document 1). Most two-class classification methods use segmentation or deep learning to separate multiple cytology images into single cytology images. The separated single cytology images are then classified using existing single-cell features. In this case, accuracy is lower than that of single cytology image classification due to the imperfect accuracy of the single-cell separation process, but accuracy of 95% or more has been reported.

[0039] In contrast, the accuracy of multi-class classification methods for multiple cytology images is still insufficient. Methods that apply generic object detection techniques to detect cells with findings from multiple cytology images have been proposed (e.g., Non-Patent Document 1). These methods, by using generic object detection techniques, can omit preprocessing such as cell separation, artifact removal, and segmentation. Furthermore, these methods can be said to better meet the requirements for cytology screening by enabling the detection of cells with findings and classification of malignancy level from multiple cytology images. However, because these methods have an average recall rate of less than 70%, it is expected that positive results will be overlooked, i.e., false negatives will increase. Therefore, although the application of such generic object detection techniques to cytology is useful, their low accuracy makes them insufficient as screening methods.

[0040] [First embodiment] 1 shows a functional block diagram of a cervical cancer screening support system 1 according to the first embodiment. The cervical cancer screening support system 1 includes an image acquisition unit 10, a first estimation unit 20, an image conversion unit 30, and a second estimation unit 40.

[0041] The image acquisition unit 10 acquires a microscopic image of cells collected from the cervix. These cells are typically collected by scraping them from the area to be examined with a brush or spatula. The image acquisition unit 10 may be any camera, such as a microscope camera, a smartphone camera, or a commercially available digital camera that can be attached to a microscope using an adapter. The image acquisition unit 10 inputs the image data of the acquired microscopic image to the first estimation unit 20 and the image conversion unit 30.

[0042] When image data of a micrograph acquired by the image acquisition unit 10 is input, the first estimation unit 20 uses the first estimation model to estimate and output the locations of potentially abnormal cells in the micrograph and the test result classification of the potentially abnormal cells (i.e., which of the predetermined test result classifications each of the potentially abnormal cells falls into). For example, consider a case where the first estimation model determines that there are 10 potentially abnormal cells in the micrograph and estimates the locations of these 10 cells. In this case, the first estimation unit 20 calculates the x-coordinate, y-coordinate, height, and width of a rectangular area containing the 10 cells as position information for each of the 10 cells using floating-point numbers. In this case, the first estimation model estimates which of the aforementioned Bethesda classifications each of these 10 cells falls into: ASC-US, LSIL, ASC-H, HSIL, or SCC (NILM is excluded because it represents "no abnormality"). The first estimation unit 20 calculates, as the test result classification information for each of the 10 cells, an integer value that identifies the Bethesda classification to which the cell belongs. In this way, the first estimation unit 20 outputs the positions of the possibly abnormal cells and the test result classifications of the possibly abnormal cells, and inputs them to the second estimation unit 40.

[0043] The first estimation model is generated by machine learning using an object detection algorithm with training data consisting of a micrograph, the locations of potentially abnormal cells in the micrograph, and the test result classifications to which each of the potentially abnormal cells corresponds. Potentially abnormal cells, particularly tumors, differ from healthy cells in terms of image characteristics such as shape, size, color, saturation, shading, texture, and inversion. Furthermore, these image characteristics can sometimes be made more prominent by applying processes to the raw image data, such as image inversion (horizontal and vertical directions), blurring, noise removal, gamma correction, and filtering. Therefore, by using an appropriate object detection algorithm with a micrograph containing these image characteristics as training data, the locations of potentially abnormal cells and their test result classifications can be estimated with high accuracy. Note that, from the perspective of balancing detection speed and detection accuracy, it is preferable to use, for example, the YOLO algorithm as the machine learning algorithm used to generate the first estimation model. However, this is not limiting, and any suitable object detection algorithm may be used.

[0044] The image conversion unit 30 first extracts images of each cell located at the position estimated by the first estimation unit from the micrograph. As described above, in this example, the first estimation model estimated the positions of 10 potentially abnormal cells, so the image conversion unit 30 extracts images of the cells located at these 10 positions. Next, the image conversion unit 30 converts each of the extracted images into converted images in a predetermined format. This will be explained. The extracted 10 cell images are not necessarily suitable as images to be input to the second estimation unit 40. In other words, the format of these 10 extracted images does not necessarily match the input format of the second estimation unit 40. Therefore, the image conversion unit 30 converts the 10 images into images that match the input format of the second estimation unit 40 (hereinafter referred to as "converted images"). Specifically, the image conversion unit 30 converts the shape, size, aspect ratio, number of pixels, etc. of each extracted image into converted images that match the input format of the second estimation unit 40. The image conversion unit 30 inputs the converted images thus obtained into the second estimation unit 40.

[0045] When the converted image converted by the image conversion unit 30 is input, the second estimation unit 40 uses the second estimation model to estimate and output the probability that cells in the converted image correspond to each of the test result classifications. Following the example described above, consider the case where 10 converted images are input. In this case, the second estimation unit 40 estimates the probability that the cell in the first converted image corresponds to ASC-US in the Bethesda classification as XA%, the probability that it corresponds to LSIL as XL%, ..., the probability that it corresponds to SCC as XS%, the probability that the cell in the second converted image corresponds to ASC-US as YA%, ..., the probability that the cell in the tenth converted image corresponds to ASC-US as ZS%, and so on. As a result, the second estimation unit 40 calculates a 10-by-5 ​​matrix as information on the estimated probability that each of the 10 cells corresponds to one of the five classes of the Bethesda classification. In this way, the second estimation unit 40 outputs the probability that a potentially abnormal cell corresponds to each of the test result classifications.

[0046] The second estimation model is generated by machine learning using an image classification algorithm with training data consisting of images of potentially abnormal cells and the probability that the potentially abnormal cells correspond to each of the test result classifications. As described above, potentially abnormal cells, such as tumors, have different image features, such as shape, size, color, saturation, shading, and texture, depending on their malignancy. Furthermore, these image features can sometimes be made more prominent by applying processes to raw image data, such as image inversion (horizontal and vertical directions), blurring, noise removal, gamma correction, and filtering. Therefore, by using an appropriate image classification algorithm with micrographs containing these image features as training data, it is possible to estimate with high accuracy the probability that the potentially abnormal cells correspond to each of the test result classifications. Note that, for example, a convolutional neural network is preferably used as the machine learning algorithm for generating the second estimation model, given its superior image classification capabilities. However, this is not limiting, and any suitable image classification algorithm may be used.

[0047] According to this embodiment, in the cytology test for cervical cancer screening, it is possible to estimate with high accuracy the location of potentially abnormal cells and the probability that the potentially abnormal cells fall into each of the test result classifications, thereby improving the accuracy of the screening and reducing the burden on specialists and cytotechnologists.

[0048] [Second embodiment] In the above-described embodiment, the first estimation model was used to estimate the location of potentially abnormal cells and the test result classification of the potentially abnormal cells. Then, the second estimation model was used to estimate the probability that cells in the converted image (potentially abnormal cells) correspond to each of the test result classifications. As a result, the learning result of the first estimation model (the test result classification estimated by the first estimation model) was overwritten with the learning result of the second estimation model (the test result classification estimated by the second estimation model), and this was used as the final result. However, this is not limited to this, and the final result may be obtained by using both the learning results of the first estimation model and the second estimation model. Such an embodiment (second embodiment) will be described below.

[0049] In machine learning, a method (ensemble learning) is known that improves estimation ability for unlearned data by combining learning results from different estimation models. By using this method, it is possible to correct variance (variation in estimated values) that occurs in individual learning models and avoid overlearning. The second embodiment utilizes this knowledge.

[0050] FIG. 2 shows a functional block diagram of a cervical cancer screening support system 2 according to the second embodiment. The cervical cancer screening support system 2 includes an image acquisition unit 10, a first estimation unit 20, an image conversion unit 30, a second estimation unit 40, and a third estimation unit 50. That is, the cervical cancer screening support system 2 includes the third estimation unit 50 in addition to the components of the cervical cancer screening support system 1 in FIG. 1. The other components of the cervical cancer screening support system 2 are common to those of the cervical cancer screening support system 1. Below, the third estimation unit 50 will be explained, and explanations of the common components will be omitted.

[0051] The third estimation unit 50 receives the test result classification of the potentially abnormal cell from the first estimation unit and the probability that the potentially abnormal cell corresponds to each of the test result classifications from the second estimation unit. The third estimation unit 50 uses a third estimation model to estimate and output the probability that the potentially abnormal cell corresponds to each of the test result classifications. The third estimation model is created by integrating the test result classification to which the potentially abnormal cell corresponds, estimated using the first estimation model, and the probability that the cell in the converted image corresponds to each of the test result classifications, estimated using the second estimation model. The third estimation model estimates the probability that the potentially abnormal cell corresponds to each of the test result classifications. A preferred method for generating the third estimation model is, for example, ensemble learning using stacking, from the viewpoint of improving estimation accuracy (see, for example, Non-Patent Document 5). However, this is not limited to this, and any suitable ensemble learning may be used.

[0052] In the above-described embodiment, the image acquisition unit may be provided in a smartphone. The first estimation unit, the image conversion unit, and the second estimation unit may be provided in a data processing device connected to the smartphone via a network. In this case, the image acquisition unit is typically a camera attached to the smartphone. The data processing device is, for example, a server or a cloud server installed in a data center. The network connecting the smartphone and the data processing device may be a wired network, a wireless network, the Internet, an intranet, a public line, a dedicated line, or the like. However, from the viewpoint of high-speed transmission of image data and convenience, a high-speed mobile communication network such as 5G is preferable.

[0053] Figure 3 shows a schematic diagram of this type of cervical cancer screening support system. The smartphone camera takes microscopic photographs of the cells of the subject (1). The captured microscopic images are sent to a remote data processing device via a network (2). The data processing device estimates the location of potentially abnormal cells and the probability that the potentially abnormal cells fall into each of the test result categories from the received microscopic images (3), and sends the estimated results to the smartphone via the network (4). The estimated results received by the smartphone are presented to doctors and cytotechnologists to assist them in their diagnosis (5).

[0054] By constructing a cervical cancer screening support system in this way, microscopic photographs taken in medical settings can be taken with compact, readily available smartphones, while machine learning and screening processes, which require computer resources, can be performed on high-performance data processing devices, achieving an ideal division of functions.

[0055] In the above-described embodiment, the data processing device may be connectable to an external device via a network. By constructing the cervical cancer screening support system in this manner, the system can be shared by devices at multiple medical institutions, for example.

[0056] In the above-described embodiment, the data processing device may include a database that stores the positions of potentially abnormal cells estimated by the first estimation unit, the test result classifications to which the potentially abnormal cells belong, and the probabilities that the cells in the converted image estimated by the second estimation unit belong to each of the test result classifications. By constructing the cervical cancer screening support system in this way, the learning results of machine learning can be stored in a database, thereby improving usability.

[0057] In the above-described embodiment, the cervical cancer screening support system may be connected to an external system equipped with a similar cervical cancer screening support system via a network. In this case, the database of this system may store the positions of potentially abnormal cells estimated by a first estimation unit of the external system, the test result classifications to which the potentially abnormal cells belong, and the probabilities that the cells in the converted image belong to each of the test result classifications estimated by a second estimation unit of the external system.

[0058] Figure 4 shows a schematic diagram of this type of cervical cancer screening support system. The smartphone camera captures microscopic photographs of cells in the subject (1). The captured microscopic images are sent to a remote data processing device via a network (2). The data processing device estimates the location of potentially abnormal cells and the probability that the potentially abnormal cells fall into each of the test result classifications from the received microscopic images (3), and sends the estimated results to the smartphone via the network (4). The estimated results received by the smartphone are presented to doctors and cytotechnologists to assist them in their diagnoses (5). The locations of potentially abnormal cells estimated by the first estimation unit, the test result classifications to which the potentially abnormal cells fall, and the probability that the cells in the converted image fall into each of the test result classifications estimated by the second estimation unit are stored in a database (6). Meanwhile, machine learning is also performed in an external system connected to this system via a network, and the positions of potentially abnormal cells estimated by the first estimation unit of the external system, the test result classification to which the potentially abnormal cells belong, and the probability that the cells in the converted image estimated by the second estimation unit of the external system belong to each of the test result classifications are also stored in the system's database (7).

[0059] By constructing a cervical cancer screening support system in this way, the results of machine learning performed on multiple systems can be combined, enabling estimation models to be generated more quickly and with greater accuracy.

[0060] [Third embodiment] 5 is a flowchart showing the process of a cervical cancer screening support method according to a third embodiment. This method includes step S10 of acquiring a microscopic image, step S20 of estimating the location of potentially abnormal cells and the test result classification to which each potentially abnormal cell belongs, step S30 of extracting and converting images of potentially abnormal cells, and step S40 of estimating the probability that each potentially abnormal cell belongs to each test result classification.

[0061] In step S10, the method uses an image capture unit to capture a micrograph of cells taken from the cervix.

[0062] In step S20, the method uses a first estimation model generated by machine learning using an object detection algorithm, using as learning data the microscopic photograph, the positions of potentially abnormal cells in the microscopic photograph, and the test result classification to which each of these potentially abnormal cells applies, to estimate the positions of potentially abnormal cells in the microscopic photograph and the test result classification to which each of the potentially abnormal cells applies when a microscopic photograph acquired by the image acquisition unit is input.

[0063] In step S30, the method extracts an image of each cell located at the position estimated in the first estimation step from the micrograph, and converts each extracted image into a converted image in a predetermined format.

[0064] In step S40, the method uses a second estimation model generated by machine learning using an image classification algorithm, using images of potentially abnormal cells and the probabilities that the potentially abnormal cells fall into each of the test result classifications as learning data, to estimate the probability that cells in the converted image will fall into each of the test result classifications when the converted image is input.

[0065] This method makes it possible to estimate with high accuracy the location of potentially abnormal cells and the probability that the potentially abnormal cells fall into each test result classification in cytology tests for cervical cancer screening, thereby improving the accuracy of screening and reducing the burden on specialists and cytologists.

[0066] [Fourth embodiment] A computer program according to the fourth embodiment causes a computer to execute the processing flow of Fig. 5. That is, the program causes a computer to execute step S10 of acquiring microscopic images, step S20 of estimating the positions of potentially abnormal cells and the test result classifications to which each of the potentially abnormal cells falls, step S30 of extracting and converting images of the potentially abnormal cells, and step S40 of estimating the probability that the potentially abnormal cells fall into each of the test result classifications.

[0067] According to this embodiment, the cervical cancer screening support program can be implemented on software, so that cervical cancer screening can be supported at high speed and with high accuracy using a computer.

[0068] [Fifth embodiment] The smartphone application according to the fifth embodiment includes the above-mentioned cervical cancer screening support program.

[0069] FIG. 6 shows a schematic diagram of an embodiment of such a smartphone application.

[0070] By implementing the cervical cancer screening support program as a smartphone application, cervical cancer screening can be supported using compact, easily available smartphones.

[0071] Hereinafter, a collection of cells with connectivity is referred to as a "cell clump." Furthermore, "cells" herein encompasses both "isolated and scattered cells" and "cell clumps." To accommodate the large number of cases, research and development of automated cytological diagnostic systems (automated cervical cancer screening devices) has been progressing in the field of cervical cancer diagnosis. Liquid-based cytology (LBC) is a key step in these systems. In this case, improved pretreatment methods to minimize cell overlap have been required to enable automated systems to better identify abnormal cells. For this reason, conventional cytological diagnostic systems have focused on pre-test cell processing methods to avoid the formation of cell clumps and cell invaginations. For example, significant cell accumulation or cell lysis can make it difficult to estimate the number of squamous epithelial cells. This is because it is difficult to evaluate individual cell morphology during cell accumulation. Given this background, the development process for conventional automated diagnostic support devices has focused on making judgments at the single-cell level by using LBC to avoid cell stacking and to disperse cells relatively finely.

[0072] However, there are also cases where characteristics of each cell cluster are identified. For example, HSIL cell clusters are not usually spherical like endometrial cell spheres and have less regular peripheries. Furthermore, in intraepithelial adenocarcinoma, palisade-like or feather-like clusters are formed, with protruding nuclei. Furthermore, atypical cells often form clusters due to altered intercellular connections, compared with normal cells. In the field of urology, when large cell clusters are observed in natural urine, they are often considered to be neoplastic lesions. Thus, the observed cells may exhibit characteristic cluster formation depending on the grade and type of atypia. Based on these trends, the inventors conceived the idea that, contrary to the conventional approach of "avoiding the formation of cell clusters and intussusceptions," actively utilizing the presence of cell clusters to assess the grade of atypia could actually improve the accuracy of assessment.

[0073] Cell clumps are extremely diverse, making it nearly impossible to manually input their characteristics into a diagnostic support system. In other words, clumps and intussusceptions can be flat or form three-dimensional structures, and their size and variation are infinite. They are not as simple as the nuclear structure of a single cell, which is the basis of current distribution systems. To address these challenges, the inventors proposed "automatic feature extraction from cell clump images using deep learning," which yielded excellent results. Specifically, the inventors developed a cervical cancer screening support system that enables highly accurate screening by using deep learning to classify cell clumps containing atypical cells as training data and then marking them.

[0074] [Sixth embodiment] 7 shows a functional block diagram of a cervical cancer screening support system 3 according to the sixth embodiment. The cervical cancer screening support system 3 includes an image acquisition unit 10, a cell clump recognition unit 60, and an output unit 70.

[0075] The image acquisition unit 10 acquires a micrograph of cells collected from the cervix. The cell clump recognition unit 60 recognizes cell clumps in the acquired micrograph. Any machine learning or deep learning method may be used to recognize the cell clumps. The output unit 70 outputs the test classification to which the cells belonging to the cell clumps belong.

[0076] According to this embodiment, the presence of cell clumps can be actively utilized in determining the degree of atypia, thereby improving the accuracy of screening.

[0077] As an example, the region containing the cell clump may include the background surrounding the cell clump. In this case, the cell clump recognition unit 60 may use a frame of a predetermined shape to cut out the image of the cell clump and its surroundings, including the background. The shape of such a frame may be any shape, such as a rectangle, square, any polygon, circle, or ellipse. According to this embodiment, the feature amount can include information about not only the cell clump itself but also necrotic material contained in the background, thereby further improving the accuracy of the screening.

[0078] As an example, the cell clump recognition unit may recognize cell clumps using the YOLO algorithm. Research by the present inventors has shown that the YOLO algorithm is effective in recognizing cell clumps. Therefore, the use of the YOLO algorithm can further improve the accuracy of screening.

[0079] As an example, the cell clump recognition unit may recognize cell clumps in real time. According to this embodiment, no pre-processing is required, which can further improve the efficiency of the examination.

[0080] [Seventh embodiment] FIG. 8 shows a functional block diagram of a cervical cancer screening support system 4 according to the seventh embodiment. The cervical cancer screening support system 4 includes an image acquisition unit 10, a cell clump recognition unit 60, an output unit 70, and an estimation unit 80. That is, the cervical cancer screening support system 4 includes the estimation unit 80 in addition to the configuration of the cervical cancer screening support system 3 in FIG. 7. The other configuration of the cervical cancer screening support system 4 is common to that of the cervical cancer screening support system 3.

[0081] The estimation unit 80 uses, as learning data, markings on cell clumps that contain atypical cells among the cell clumps recognized by the cell clump recognition unit and the atypical classification of the cells contained in the cell clumps, and uses this to generate an estimation model through machine learning using an object detection algorithm.When a microscopic photograph acquired by the image acquisition unit is input, the estimation unit 80 estimates and outputs the location of potentially abnormal cells in the microscopic photograph and the test result classification to which each potentially abnormal cell falls.

[0082] Specifically, when the location of the cell clumps recognized by the cell clump recognition unit 60 and the area containing the cell clumps are input, the estimation unit 80 uses the estimation model to estimate the location of the potentially abnormal cells and the test result classification of the potentially abnormal cells (i.e., which of the predetermined test result classifications each of the potentially abnormal cells falls into). For example, consider a case where the estimation model determines that there are 10 potentially abnormal cells and estimates the locations of these 10 cells. In this case, the estimation unit 80 calculates the x-coordinate, y-coordinate, height, and width of the rectangular area containing the cells as position information for each of the 10 cells using floating-point numbers. The estimation model then estimates which of the aforementioned Bethesda classifications each of these 10 cells falls into: ASC-US, LSIL, ASC-H, HSIL, or SCC (NILM is excluded because it represents "no abnormality"). The estimation unit 80 calculates, as the test result classification information for each of the 10 cells, an integer value that identifies the Bethesda classification to which the cell applies. In this way, the estimation unit 80 outputs the location of the possibly abnormal cell and the test result classification of the possibly abnormal cell.

[0083] The estimation model is generated by machine learning using an object detection algorithm with training data consisting of micrographs containing cell clumps, the positions of the cell clumps and the areas containing the cell clumps, and cell clumps recognized by the cell clump recognition unit that contain atypical cells. Potentially abnormal cells, particularly those in tumors, differ from healthy cells in terms of image characteristics such as shape, size, color, saturation, shading, texture, and inversion. Furthermore, these image characteristics can sometimes be made more prominent by applying processes to the raw image data, such as image inversion (horizontal and vertical directions), blurring, noise removal, gamma correction, and filtering. Therefore, by using an appropriate object detection algorithm with micrographs containing these image characteristics as training data, the location of potentially abnormal cells and the classification of their test results can be estimated with high accuracy.

[0084] According to this embodiment, the presence of cell clumps in cytology for cervical cancer screening can be actively utilized to determine the degree of atypia, making it possible to estimate with high accuracy the location of potentially abnormal cells and the test result classification to which each potentially abnormal cell applies.

[0085] [Eighth embodiment] 9 is a flowchart showing the processing of a cervical cancer screening support method according to the eighth embodiment. This method includes step S10 of acquiring a microscopic image using an image acquisition unit, cell clump recognition step S50 of recognizing cell clumps in the microscopic image, and output step S60 of outputting the test classification to which cells belonging to the cell clumps belong.

[0086] In step S10, the method uses an image capture unit to capture a micrograph of cells taken from the cervix.

[0087] In step S50, the method recognizes cell clumps in the micrograph acquired in step S10. Any machine learning or deep learning method may be used to recognize cell clumps.

[0088] In step S60, the method outputs the test classification to which the cells belonging to the cell clump fall.

[0089] According to this method, the presence of cell clumps can be actively utilized in determining the degree of atypia, thereby improving the accuracy of screening.

[0090] [Ninth embodiment] A computer program according to the ninth embodiment causes a computer to execute the processing flow of Fig. 9. That is, this program causes a computer to execute step S10 of acquiring a microscopic image using an image acquisition unit, cell clump recognition step S50 of recognizing cell clumps in the microscopic image, and output step S60 of outputting the test classification to which cells belonging to the cell clumps belong.

[0091] According to this embodiment, the cervical cancer screening support program can be implemented on software, so that a computer can be used to support cervical cancer screening by actively utilizing the presence of cell clumps to determine the degree of atypia.

[0092] The present invention has been described above based on the embodiments. The embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of the respective components and treatment processes, and that such modifications are also within the scope of the present invention. [Industrial Applicability]

[0093] The present invention relates to a cervical cancer screening support system, a cervical cancer screening support method, a cervical cancer screening support program, and a smartphone application. [Explanation of symbols]

[0094] 1... Cervical cancer screening support system, 2... Cervical cancer screening support system, 3... Cervical cancer screening support system, 4... Cervical cancer screening support system, 10...Image acquisition unit, 20...first estimation part, 30...Image conversion unit, 40…Second estimation part, 50...Third estimation part, 60...Cell cluster recognition unit, 70...output section, 80...estimation section, S10: A step of acquiring a microscope image; S20: A step of estimating the location of the potentially abnormal cells and the test result classification to which each of the potentially abnormal cells falls; S30: Extracting and converting images of potentially abnormal cells; S40: Estimating the probability that a potentially abnormal cell falls into each of the test result categories; S50: a cell clump recognition step for recognizing cell clumps in a micrograph; S60: An output step of outputting the test classification to which the cells belonging to the cell clump belong.

Claims

1. An image acquisition unit that acquires a microscopic photograph of a cell sample for cervical cytology; a cell clump recognition unit that recognizes cell clumps in the micrograph in order to classify the degree of atypia based on the cell clumps in the cell specimen; A cervical cancer screening support system comprising: an output unit that outputs a test classification including the degree of atypia to which cells belonging to the cell clump belong.

2. The cervical cancer screening support system described in Claim 1, characterized in that the cell clump recognition unit performs LBC on the collected cell sample.

3. The cervical cancer screening support system described in Claim 1 or 2, characterized in that the output unit uses deep learning to automatically extract features according to the degree of atypia or type from the cell clumps, thereby outputting a test classification including the degree of atypia.

4. 4. The cervical cancer screening support system according to claim 1, wherein the region containing the cell clumps includes the background surrounding the cell clumps.

5. 5. The cervical cancer screening support system according to claim 1, wherein the cell clump recognition unit recognizes cell clumps using a YOLO algorithm.

6. 6. The cervical cancer screening support system according to claim 1, wherein the cell clump recognition unit recognizes cell clumps in real time.

7. The cervical cancer screening support system of any one of claims 1 to 6 further comprises an estimation unit that, when a microscopic photograph acquired by the image acquisition unit is input, estimates and outputs the position of potentially abnormal cells in the microscopic photograph and the test result classification to which each of the potentially abnormal cells falls, using an estimation model generated by machine learning with an object detection algorithm, using as learning data markings on cell clumps that contain atypical cells among the cell clumps recognized by the cell clump recognition unit and the atypia classification of the cells contained in the cell clumps.

8. an image acquisition unit for acquiring a microscopic image of cells collected from the cervix; a first estimation unit that, when a micrograph acquired by the image acquisition unit is input, estimates and outputs the positions of potentially abnormal cells in the micrograph and the test result classification to which each of the potentially abnormal cells falls, using a first estimation model generated by machine learning using an object detection algorithm, with the micrograph, the positions of potentially abnormal cells in the micrograph, and the test result classification to which each of the potentially abnormal cells falls as learning data; an image conversion unit that extracts an image of each cell located at the position estimated by the first estimation unit from the micrograph and converts each extracted image into a converted image in a predetermined format; A cervical cancer screening support system comprising: a second estimation unit that, when the converted image is input, estimates and outputs the probability that cells in the converted image fall into each of the test result classifications using a second estimation model generated by machine learning with an image classification algorithm, using images of potentially abnormal cells and the probability that the potentially abnormal cells fall into each of the test result classifications as learning data.

9. The cervical cancer screening support system according to claim 8 , wherein the test result classification is the Bethesda classification.

10. The cervical cancer screening support system according to claim 8 or 9, wherein the object detection algorithm is the YOLO algorithm.

11. The cervical cancer screening support system according to any one of claims 8 to 10, wherein the image classification algorithm is a convolutional neural network.

12. A cervical cancer screening support system as described in any one of claims 8 to 11, further comprising a third estimation model created by integrating the test result classification to which potentially abnormal cells, estimated using the first estimation model, fall and the probability that cells in the converted image fall into each of the test result classifications, estimated using the second estimation model, and estimating and outputting the probability that the potentially abnormal cells fall into each of the test result classifications.

13. The cervical cancer screening support system according to claim 12 , wherein the third estimation model is created using ensemble learning by stacking.

14. A smartphone equipped with the image acquisition unit, and A cervical cancer screening support system as described in any one of claims 8 to 13, characterized in that it comprises a data processing device that is network-connected to the smartphone and has the first estimation unit, the image conversion unit, and the second estimation unit.

15. 15. The cervical cancer screening support system according to claim 14, wherein the data processing device can be connected to an external device via a network.

16. The data processing device includes: the location of the possibly abnormal cell estimated by the first estimation unit, and a test result classification to which the possibly abnormal cell falls; The cervical cancer screening support system described in claim 14 or 15, characterized in that it is further characterized by a database that stores the probability that cells in the converted image estimated by the second estimation unit fall into each of the test result classifications.

17. The cervical cancer screening support system according to claim 8 is connected to an external system via a network, The database includes the positions of the possibly abnormal cells estimated by the first estimation unit of the external system, and the test result classifications to which the possibly abnormal cells belong. The cervical cancer screening support system of claim 16, characterized in that it accumulates the probability that cells in the converted image estimated by the second estimation unit of the external system fall into each of the test result classifications.

18. A cervical cancer screening support system according to any one of claims 8 to 17, comprising the cell clump recognition unit according to claim 1.

19. acquiring a micrograph of a cytological specimen for cervical cytology using an image acquisition unit; a cell clump recognition step of recognizing cell clumps in the micrograph in order to classify the degree of atypia based on cell clumps in the cell specimen; A cervical cancer screening support method comprising an output step of outputting a test classification including the degree of atypia to which cells belonging to the cell clump belong.

20. A cervical cancer screening support method as described in Claim 19, further comprising a step of performing LBC on the collected cell sample.

21. The cervical cancer screening support method described in Claim 19 or 20, characterized in that the output step uses deep learning to automatically extract features according to the degree of atypia or type from the cell clumps, thereby outputting a test classification including the degree of atypia.

22. acquiring a micrograph of cells collected from the cervix using an image acquisition unit; a photomicrograph, the location of potentially abnormal cells in the photomicrograph, and a test result classification to which each potentially abnormal cell falls; a first estimation step of estimating, when a micrograph acquired by the image acquisition unit is input, the positions of potentially abnormal cells in the micrograph and the test result classification to which each potentially abnormal cell falls, using a first estimation model generated by machine learning using an object detection algorithm using the above as learning data; an image conversion step of extracting an image of each cell located at the position estimated in the first estimation step from the micrograph and converting each extracted image into a converted image in a predetermined format; a second step of estimating the probability that cells in the converted image correspond to each of the test result classifications when the converted image is input, using a second estimation model generated by machine learning with an image classification algorithm, using images of potentially abnormal cells and the probability that the potentially abnormal cells correspond to each of the test result classifications as learning data.

23. acquiring a micrograph of a cytological specimen for cervical cytology using an image acquisition unit; a cell clump recognition step of recognizing cell clumps in the micrograph in order to classify the degree of atypia based on cell clumps in the cell specimen; and an output step of outputting a test classification including the degree of atypia to which cells belonging to the cell clump fall.

24. The cervical cancer screening support program described in Claim 23, characterized in that the method further comprises a step of performing LBC on the collected cell sample.

25. The cervical cancer screening support program described in Claim 23 or 24, characterized in that the output step uses deep learning to automatically extract features according to the degree of atypia or type from the cell clump, thereby outputting a test classification including the degree of atypia.

26. acquiring a micrograph of cells collected from the cervix using an image acquisition unit; a first estimation step of estimating, when a micrograph acquired by the image acquisition unit is input, the positions of potentially abnormal cells in the micrograph and the test result classification to which each of the potentially abnormal cells falls, using a first estimation model generated by machine learning using an object detection algorithm, with the micrograph, the positions of potentially abnormal cells in the micrograph, and the test result classification to which each of the potentially abnormal cells falls as learning data; an image conversion step of extracting an image of each cell located at the position estimated in the first estimation step from the micrograph and converting each extracted image into a converted image in a predetermined format; a second step of estimating the probability that cells in the converted image correspond to each of the test result classifications when the converted image is input, using a second estimation model generated by machine learning with an image classification algorithm, using images of potentially abnormal cells and the probability that the potentially abnormal cells correspond to each of the test result classifications as learning data.

27. A smartphone application comprising the cervical cancer screening support program according to any one of claims 23 to 25.

Images