Cell classification device, cell classification method, and program

Abstract

In order to classify sample cells as being benign or malignant without being limited to a specific procedure, a cell classification device (1) comprises: an acquisition means (10) that acquires an image including sample cells as a subject; a determination means (20) that determines whether each sample cell is an individual cell or a cluster of cells; and a classification means (30) that performs at least one of classifying a sample cell determined to be an individual cell as being benign or malignant and classifying a sample cell determined to be a cluster of cells as being benign or malignant.

Description

Technical Field

[0001] The present invention relates to a cell sorter, a cell sorting method, and a program.

Background Art

[0002] When a tumor is suspected by an examination such as an X-ray or an X-ray CT, a definitive diagnosis is generally made by a pathological diagnosis. For example, a part of an organ tissue of the human body is collected as a specimen by puncture or the like, and the diagnosis is made by microscopic observation of the specimen by an expert. In the process of specimen collection, it may be necessary to quickly observe whether the specimen contains tissue cells suitable for pathological diagnosis in the treatment room where the specimen collection is being performed. This is because if the specimen does not contain tissue cells suitable for pathological diagnosis, it is necessary to collect the specimen again. However, specimen collection by puncture or the like is invasive, and it is not preferable to perform specimen collection many times.

[0003] Intraoperative rapid cytodiagnosis (ROSE, Rapid On-Site Evaluation) is a technique for quickly observing in the treatment room where the specimen is collected whether the collected specimen contains cells suitable for pathological diagnosis and whether the specimen contains malignant cells. For example, it is used when collecting a specimen by puncture using an endoscope of a respiratory organ.

[0004] However, specialized knowledge and experience are required to determine whether the collected cells are benign or malignant, and there is a limit to such human resources of skilled personnel. For this reason, although ROSE has been reported to be useful in reducing the number of punctures and the risk of complications, there is a problem that its implementation is limited. Therefore, attempts have been made to enable a non-expert to make a diagnosis close to that of an expert by assisting such a determination with a mechanical device.

[0005] For example, Patent Document 1 discloses a technique for calculating morphological features of cells from images of specimens stained with molecular target dyes that visualize predetermined target molecules, and for determining the degree of abnormality of the cells. This technique involves determining whether each recognized cell or cell cluster region is normal or abnormal. Patent Document 2 also discloses determining whether a sample is a solitary, scattered single cell (solitary scattered cell) or a cell cluster consisting of multiple cells, determining whether a solitary scattered cell is normal or abnormal based on a determination table using at least one of shape feature parameters related to the cell image shown in the overall specimen vs. image, and image feature parameters (such as brightness values) related to the cell image, and performing abnormality determination processing for cell clusters by changing the table used for determination. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2011-179924 [Patent Document 2] Japanese Patent Publication No. 2009-175334 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, the technique described in Patent Document 1 involves applying counterstaining to a specimen to stain the cell nuclei, and then evaluating whether or not a desired antigen is expressed around the cell nuclei stained by this counterstaining. This technique is limited to observing whether or not an antigen is expressed. Furthermore, the technique described in Patent Document 2 is limited to an analysis method for virtual slide images of a specimen, which are constructed by combining multiple microscope images acquired by taking pictures each time the objective lens and the specimen are relatively moved in a direction perpendicular to the optical axis.

[0008] One aspect of the present invention has been made in view of the above-mentioned problems, and one example of its objective is to provide a technology that enables improved accuracy in classifying sample cells as benign or malignant, without being limited to specific purposes or methods. [Means for solving the problem]

[0009] A cell classification device according to one aspect of the present invention comprises: acquisition means for acquiring an image including sample cells as a subject; determination means for determining whether each of the sample cells is an individual cell or a clump of cells; and classification means for performing at least one of the following: a process for classifying the sample cells determined to be individual cells as benign or malignant, and a process for classifying the sample cells determined to be clump of cells as benign or malignant.

[0010] A cell classification method according to one aspect of the present invention includes at least one processor acquiring an image containing sample cells as a subject, determining whether each of the sample cells is an individual cell or a clump of cells, performing a process to classify the sample cells determined to be individual cells as benign or malignant, and performing a process to classify the sample cells determined to be clump of cells as benign or malignant.

[0011] A cell classification program according to one aspect of the present invention causes a computer to perform at least one of the following processes: acquiring an image containing sample cells as the subject; determining whether each of the sample cells is an individual cell or a clump of cells; performing benign or malignant classification on the sample cells determined to be individual cells; and performing benign or malignant classification on the sample cells determined to be a clump of cells. [Effects of the Invention]

[0012] According to one aspect of the present invention, it is possible to classify sample cells as benign or malignant without being limited to a specific method. [Brief explanation of the drawing]

[0013] [Figure 1]It is a block diagram showing the configuration of the cell sorter 1 according to Exemplary Embodiment 1 of the present invention. [Figure 2] It is a schematic diagram showing an example of a method for detecting cells from a captured image. [Figure 3] It is a flowchart showing the flow of the cell sorting method S1 according to Exemplary Embodiment 1. [Figure 4] It is a block diagram showing the configuration of the cell sorter 1A according to Exemplary Embodiment 2. [Figure 5] It is an example of teacher data used to construct a learned determination model. [Figure 6] It is a block diagram showing the configuration of the cell sorter 1B according to Exemplary Embodiment 3. [Figure 7] It is a schematic diagram showing an example of constructing a learned single-cell classification model using teacher images showing benign and malignant cells of individual cells. [Figure 8] It is a schematic diagram showing an example of constructing a learned mass-cell classification model using teacher images showing benign and malignant cells of mass cells. [Figure 9] It is a schematic diagram showing an example of a procedure for generating input data for performing benign / malignant classification using both the learned single-cell classification model and the learned mass-cell classification model. [Figure 10] [[ID=H]]It is a configuration diagram for realizing an information processing apparatus by software.

BEST MODE FOR CARRYING OUT THE INVENTION

[0014] 〔Exemplary Embodiment 1〕 The first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form for the exemplary embodiments described later.

[0015] (Configuration of the cell sorter 1) The configuration of the cell classification device 1 according to this exemplary embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the configuration of the cell classification device 1 according to Exemplary Embodiment 1. The cell classification device 1 is a device that classifies whether cells (specimen cells) contained in a specimen are benign cells or malignant cells (benign / malignant classification) for pathological diagnosis. Pathological diagnosis refers to observing a specimen collected from the human body under a microscope and diagnosing the presence or absence of a lesion and the type of the lesion. The cell classification device 1 is, as an example, a device for performing intraoperative rapid cytodiagnosis during specimen collection by puncture using an endoscope of a respiratory organ.

[0016] As shown in FIG. 1, the cell classification device 1 includes an acquisition unit 10, a determination unit 20, and a classification unit 30. The acquisition unit 10 acquires an image including specimen cells as a subject. In this exemplary embodiment, the specimen cells are cells contained in a specimen, and the specimen usually contains a plurality of specimen cells. The specimen may be, as an example, a specimen of an organ tissue collected by puncture. Puncture is a specimen collection method in which a doctor inserts a collection needle into an organ tissue to collect cells of the organ tissue. The puncture may be performed by the doctor while using a respiratory endoscope. Thereby, for example, a tissue suspected of having a lung tumor can be collected. The collected organ tissue is, for example, made into a specimen by a medical technician, and an image (microscope camera image) including specimen cells (hereinafter, also simply referred to as "cells") is taken using a microscope with a digital camera. The medical technician records the taken image in a memory (not shown) of the cell classification device 1, for example. The acquisition unit 10 acquires the image recorded in the memory as digital data. The acquisition unit 10 is one form of the "acquisition means" described in the claims.

[0017] The medical technician creates one or more slide specimens using the collected specimen, stains the cells, air-dries them, and then takes one or more images using a microscope. These operations are generally performed operations, and it is possible to determine at this point whether sufficient cells have been collected. If sufficient cells have not been collected, puncture can be performed again to collect the specimen.

[0018] The determination unit 20 determines whether each of the sample cells is an individual cell or a cluster of cells. An individual cell is a cell that exists as a single, independent cell. A cluster of cells (hereinafter also referred to as "cluster cells") is a cell in which multiple cells are clustered together in a dense state. Cells in which multiple individual cells overlap in the depth direction are also included in cluster cells. The determination unit 20 is one form of the "determination means" described in the claims.

[0019] To determine whether a sample consists of individual cells or a cell aggregate, cell detection is performed first. The cell detection method will be explained with reference to the diagram. Figure 2 is a schematic diagram showing an example of a method for detecting cells from an image 201 containing sample cells as the subject. First, the determination unit 20 generates a stain intensity image 202 from image 201 in Figure 2 by regression analysis. Specifically, the determination unit 20 sets the brightness of unstained background areas (including areas with weak staining, such as red blood cells) to 0, and generates a grayscale image 202 in which stained areas (stained regions) are assigned a brightness range of 1 to 255 for each pixel according to the stain intensity (Reference: US2022 / 0028068 A1, Applicant: NEC Laboratories America). This image 202 is the stain intensity image.

[0020] The determination unit 20 may determine whether a sample cell is an individual cell or a cluster of cells by referring to at least one of the size and roundness of the sample cells. For example, the determination unit 20 determines whether the area of ​​a continuous stained region in the staining intensity image is greater than or equal to a predetermined threshold. The determination unit 20 determines (detects) the region where the area of ​​the continuous stained region is greater than or equal to the predetermined threshold as a cluster of cells. Image 203 in Figure 2 is an image in which only cluster of cells remain.

[0021] On the other hand, the determination unit 20 determines that any region where the area of ​​the continuous staining region is smaller than a predetermined threshold is a candidate region for an individual cell. The determination unit 20 may then use this region as a candidate and ultimately detect an individual cell from among the candidate regions. Image 204 in Figure 2 is an image in which only the candidate regions for individual cells remain.

[0022] The reason for determining whether a cell is an individual cell or a clump of cells is that, in the case of clumps of cells, the criteria for determining whether they are individual cells differ from those for individual cells. Therefore, when performing ROSE using the cell classification device 1, it is preferable to detect cells, determine whether the detected cells are individual cells or clumps of cells, and then determine whether the cells are benign or malignant depending on whether they are individual cells or clumps of cells.

[0023] The classification unit 30 performs at least one of the following: a process to classify sample cells determined to be individual cells as benign or malignant, and a process to classify sample cells determined to be clump-like cells as benign or malignant.

[0024] For example, the classification unit 30 may perform benign / malignant classification only on cells that have been determined to be individual cells. The machine learning model that performs benign / malignant classification is trained on cells that can be visually determined to be benign or malignant at the individual cell level, in other words, cells that can be labeled as benign or malignant. Therefore, the benign / malignant classification of individual cells is more accurate than that of aggregate cells. Thus, applying benign / malignant classification only to cells that have been determined to be individual cells improves the accuracy of the classification compared to classifying individual cells and aggregate cells together. In addition, since it becomes unnecessary to classify aggregate cells as benign or malignant, the time required for benign / malignant classification can be shortened compared to when benign / malignant classification is performed including aggregate cells.

[0025] Furthermore, the classification unit 30 may perform benign / malignant classification only on cells determined to be aggregate cells. Alternatively, the classification unit 30 may perform benign / malignant classification on both sample cells determined to be individual cells and sample cells determined to be aggregate cells. Note that "benign / malignant classification" is not limited to classifying cells as benign or malignant, but may also involve calculating the probability of malignancy, etc.

[0026] The classification unit 30 classifies one or more individual cells and one or more aggregate cells, or both, as benign or malignant. The classification unit 30 may further aggregate the results of the benign / malignant classification. For example, the classification unit 30 may aggregate data such as the number of cells determined to be benign and the number of cells determined to be malignant. The classification unit 30 may refer to the aggregated results to determine whether the specimen is benign or malignant. The determination of the specimen can be made, for example, by comprehensively analyzing the aggregated results of multiple cells that have been classified as benign or malignant. The classification unit 30 is one form of the "classification means" described in the claims.

[0027] Although Figure 1 shows the acquisition unit 10, determination unit 20, and classification unit 30 as being physically located together in a single enclosure, this is not necessarily required. That is, these units may be distributed across multiple physically separated enclosures, and these units may be connected to each other via wired or wireless means to enable information communication. Furthermore, at least some of these units may be located on a cloud. This is also true in the exemplary embodiments described below.

[0028] As described above, the cell classification device 1 according to this exemplary embodiment employs a configuration comprising: an acquisition unit 10 that acquires an image including sample cells as the subject; a determination unit 20 that determines whether each sample cell is an individual cell or a clump of cells; and a classification unit 30 that performs at least one of the following: a process of classifying sample cells determined to be individual cells as benign or malignant, and a process of classifying sample cells determined to be clumps as benign or malignant. Therefore, the cell classification device 1 according to this exemplary embodiment has the effect of improving the accuracy of classifying sample cells as benign or malignant without being limited to a specific purpose or method. Furthermore, when classifying cells as benign or malignant only when they are determined to be individual cells, the accuracy of the classification is improved compared to when individual cells and clumps of cells are classified together as benign or malignant.

[0029] ( Cell classification method S1 (Flow) The flow of the cell classification method S1 according to this exemplary embodiment will be explained with reference to Figure 3. Figure 3 is a flowchart showing the flow of the cell classification method S1. As shown in Figure 3, the cell classification method S1 includes steps S11 to S13.

[0030] Step S11 is a step in which at least one processor (e.g., acquisition unit 10) acquires an image containing sample cells as the subject. The sample cells and image are as described in the configuration of the cell classification device 1.

[0031] Step S12 is a step in which at least one processor (e.g., determination unit 20) determines whether each of the sample cells is an individual cell or a clump of cells. Individual cells and clumps of cells, and the method of determination, are as described in the configuration of the cell classification device 1.

[0032] Step S13 is a step in which at least one processor (e.g., classification unit 30) performs at least one of the following: classifying sample cells determined to be individual cells as benign or malignant, and classifying sample cells determined to be clumps as benign or malignant. The benign / malignant classification is as described in the configuration of the cell classification device 1.

[0033] As described above, the cell classification method S1 according to this exemplary embodiment employs a configuration that includes at least one of the following: acquiring an image containing sample cells as the subject; determining whether each sample cell is an individual cell or a clump of cells; classifying the sample cells determined to be individual cells as benign or malignant; and classifying the sample cells determined to be clumps as benign or malignant. Therefore, the cell classification method S1 according to this exemplary embodiment has the effect of improving the accuracy of classifying sample cells as benign or malignant without being limited to a specific purpose or method.

[0034] [Exemplary Embodiment 2] A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Parts having the same function as those described in Exemplary Embodiment 1 will be denoted by the same reference numerals, and their descriptions will be omitted as appropriate.

[0035] (Configuration of Cell Classification Device 1A) Figure 4 is a block diagram showing the configuration of a cell classification device 1A according to exemplary embodiment 2. As shown in the figure, the cell classification device 1A comprises an acquisition unit 10, a determination unit 20A, a classification unit 30, an output unit 40, and a control unit 50.

[0036] The function of the acquisition unit 10 is the same as that of the acquisition unit 10 described in Exemplary Embodiment 1, however, in this Exemplary Embodiment, the acquisition unit 10 acquires images of organ tissue that have been pre-recorded in the memory 52 of the control unit 50.

[0037] The determination unit 20A includes a trained determination model 21. The trained determination model 21 may be a determination model trained using training images representing individual cells and training images representing cluster cells. Figure 5 shows an example of training data used to construct the trained determination model 21. As shown, the trained determination model 21 is trained using multiple images of individual cells (training data) and multiple images of cluster cells (training data), and can output whether the cells in the image are individual cells or cluster cells. The determination unit 20A inputs an image to the trained determination model 21 and determines whether each cell in the image is an individual cell or a cluster cell by referring to the output of the trained determination model 21. As an example, the trained determination model 21 can be a CNN (Convolutional Neural Network). Alternatively, non-neural network models such as random forests or support vector machines may be used.

[0038] Returning to Figure 4, the determination unit 20A may, after detecting each of the sample cells as an individual cell candidate or a cluster cell candidate, determine the detected sample cells using the trained determination model 21. For example, the determination unit 20A may detect candidate regions of individual cells and candidate cluster cells using the method described in Exemplary Embodiment 1. The determination unit 20A may then perform the following processing on the candidate regions of individual cells to detect individual cell candidates. That is, the determination unit 20A searches for the local maximum brightness within a predetermined range, for example, a circular region with a diameter of n (mm) or more and less than m (mm). This local maximum brightness point (pixel) is defined as the cell center. Subsequently, the determination unit 20A extracts regions from the cell center with a predetermined brightness whose circularity is above a threshold using the Hough transform. The determination unit 20A detects the extracted regions as individual cell candidates. As described above, typically multiple individual cell candidates and cluster cell candidates are detected from a single sample. The determination unit 20A uses the trained determination model 21 to finally determine whether each of the multiple individual cell candidates and aggregate cell candidates detected as described above is an individual cell or an aggregate cell.

[0039] The classification unit 30, similar to the classification unit 30 described in Exemplary Embodiment 1, performs at least one of the following processes: a process of classifying cells determined to be individual cells as benign or malignant, and a process of classifying cells determined to be aggregate cells as benign or malignant. The classification unit 30 may also transmit the results of the benign / malignant classification to the output unit.

[0040] The output unit 40 may output the results of the classification unit 30's benign / malignant classification to an external device. The output benign / malignant classification results may be displayed on a display device (not shown), for example.

[0041] The control unit 50 provides overall control of the cell classification device 1A. The control unit 50 includes at least one processor and a memory 52. ​​The processor 51 can be configured using, for example, at least one general-purpose processor such as an MPU (Micro Processing Unit) or CPU (Central Processing Unit). The memory 52 may include multiple types of memory, such as ROM (Read Only Memory) and RAM (Random Access Memory). As an example, the processor 51 realizes the functions of the acquisition unit 10, the determination unit 20A, the classification unit 30, and the output unit 40 by loading various control programs recorded in the ROM of the memory 52 into the RAM and executing them. The processor 51 may also include a processor configured as an ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or PLD (Programmable Logic Device).

[0042] (Effects of cell classification device 1A) As described above, in the cell classification device 1A according to this exemplary embodiment, the determination unit 20A employs a configuration in which it determines whether each cell is an individual cell or a cluster of cells using a trained determination model 21 that has been learned using training images showing individual cells and training images showing clusters of cells. Therefore, in addition to the effects of the cell classification device 1 according to exemplary embodiment 1, the cell classification device 1A according to this exemplary embodiment can efficiently determine individual cells, thus providing the effect of efficiently classifying individual cells as benign or malignant.

[0043] Furthermore, in the cell classification method, determining whether each sample cell is an individual cell or a cluster of cells may be done using a classification model 21 trained with training images representing individual cells and training images representing clusters of cells. This allows for obtaining effects similar to those of the cell classification device 1A described above.

[0044] [Exemplary Embodiment 3] A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Parts having the same function as those described in Exemplary Embodiment 1 will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0045] (Configuration of cell classification device 1B) Figure 6 is a block diagram showing the configuration of the cell classification device 1B according to Exemplary Embodiment 3. As shown in the figure, the cell classification device 1B comprises an acquisition unit 10, a determination unit 20, a classification unit 30A, an output unit 40, and a control unit 50. The acquisition unit 10, output unit 40, and control unit 50 have the same functions as the acquisition unit 10, output unit 40, and control unit 50 described in Exemplary Embodiment 2, so their description is omitted. Note that the cell classification device 1B may also include a determination unit 20A, as described in Exemplary Embodiment 2, instead of the determination unit 20.

[0046] In this exemplary embodiment 3, the classification unit 30A includes a trained classification model 31. The trained classification model 31 may be a classification model trained using at least one of the following: training images showing benign and malignant individual cells, and training images showing benign and malignant cells in a clump. As an example, the trained classification model 31 can be a CNN (Convolutional Neural Network). Alternatively, a non-neural network type model such as a random forest or a support vector machine may be used. Similar to the classification unit 30 described in exemplary embodiment 1, the classification unit 30A uses the trained classification model 31 to perform at least one of the following processes: classifying cells determined to be individual cells as benign or malignant, and classifying cells determined to be clump cells as benign or malignant. In other words, the classification unit 30A inputs an image of the cell to be classified into the trained classification model 31 and performs benign or malignant classification of the cell by referring to its output.

[0047] The trained classification model 31 may be a trained individual cell classification model that has been trained using training images that show benign and malignant individual cells. In that case, the classification unit 30A acquires only the cells that the determination unit 20 has determined to be individual cells from the determination unit 20 and performs benign / malignant classification using the trained individual cell classification model. Figure 7 is a schematic diagram showing an example of constructing a trained individual cell classification model using training images that show benign and malignant individual cells. As shown in Figure 7, the user inputs images that show benign individual cells into the individual cell classification model to train it on the characteristics of benign individual cells, and inputs images that show malignant individual cells to train it on the characteristics of malignant individual cells. In this way, the trained individual cell classification model 31 can be constructed.

[0048] Furthermore, the trained classification model 31 may consist of two trained classification models: a trained individual cell classification model 31A trained as described above, and a trained aggregate cell classification model 31B trained using training images showing benign and malignant aggregate cells, as shown in Figure 8 (not shown). As shown in Figure 8, the user inputs images showing benign aggregate cells into the aggregate cell classification model to train it on the characteristics of benign aggregate cells, and inputs images showing malignant aggregate cells to train it on the characteristics of malignant aggregate cells. In this way, the trained aggregate cell classification model 31B can be constructed.

[0049] Even if the trained classification model 31 consists of two models, a trained individual cell classification model 31A and a trained aggregate cell classification model 31B, the classification unit 30A may use the trained individual cell classification model 31A to perform benign / malignant classification only for cells that have been determined to be individual cells. Alternatively, the classification unit 30A may use the trained aggregate cell classification model 31B to perform benign / malignant classification only for cells that have been determined to be aggregate cells.

[0050] The classification unit 30A may perform benign / malignant classification using both the trained individual cell classification model 31A and the trained aggregate cell classification model 31B. In this case, the classification unit 30A performs benign / malignant classification of the sample cells by referring to both the output results of the trained individual cell classification model 31A and the output results of the trained aggregate cell classification model 31B. For example, the classification unit 30A may perform benign / malignant classification by referring to the data of individual cells classified as malignant and the data of aggregate cells classified as malignant. For example, the classification unit 30A may calculate the classification result using the following formula (1).

[0051]

number

[0052] In equation (1) above, p is the probability that the sample is malignant. w1 is the output weight of the trained individual cell classification model 31A. w2 is the output weight of the trained aggregate cell classification model 31B. Σp1 is the sum of the probabilities that individual cells are malignant. Σp2 is the sum of the probabilities that a mass cell is malignant. n is the number of individual cells. m represents the number of clump cells.

[0053] Even in the case of aggregate cells, information that can help determine whether a sample is benign or malignant may be present. Therefore, classifying aggregate cells as benign or malignant can provide such information. However, it is generally known that determining the benign or malignant nature of aggregate cells is more difficult than determining the benign or malignant nature of individual cells, meaning the classification accuracy is lower. For this reason, w2 may be set to a smaller value than w1.

[0054] Furthermore, the classification unit 30A may transmit to the output unit 40 at least one of the following: data on individual cells classified as malignant, data on aggregate cells classified as malignant, and the results of classifying the specimen as benign or malignant.

[0055] Figure 9 is a schematic diagram illustrating an example of the procedure for generating input data for malignant / benign classification using both the trained individual cell classification model 31A and the trained aggregate cell classification model 31B.

[0056] First, the acquisition unit 10 acquires an image 901 containing the sample cells as the subject, as shown in Figure 9. Next, the determination unit 20 generates a staining intensity image 902 from the image 901. Next, the determination unit 20 generates a candidate aggregate cell image 903 and a candidate individual cell region image 905 from the staining intensity image 902. Furthermore, the determination unit 20 detects aggregate cells from the candidate aggregate cell image 903 and detects individual cell candidates from the candidate individual cell region image 905. An example of these methods is as described in Exemplary Embodiment 1. Next, the determination unit 20 may generate an image 904 cropped to a predetermined size containing the individual cells from the image of the cells determined to be individual cells. Alternatively, the determination unit 20 may crop an image 907 of an arbitrary size (any range from the center of the aggregate, or the nearest neighbor rectangle, etc.) containing the cells from the image 906 of the cells determined to be aggregate cells (detected) and generate an image normalized to a predetermined size. This is because the images input to the trained classification model need to be standardized to a predetermined size. The determination unit 20 transmits the images generated in this manner to the classification unit 30A. The classification unit 30A inputs these acquired images into either a trained individual cell classification model 31A or a trained aggregate cell classification model 31B, depending on their type, and performs benign or malignant classification by referring to their outputs.

[0057] (Effects of Cell Classification Device 1B) As described above, in the cell classification device 1B according to this exemplary embodiment, the classification unit 30A is configured to include a classification model learned using at least one of the training images showing benign and malignant individual cells and training images showing benign and malignant aggregate cells. Therefore, according to the cell classification device 1B according to this exemplary embodiment, in particular, when performing benign / malignant classification using both the trained individual cell classification model 31A and the trained aggregate cell classification model 31B, in addition to the effects of the cell classification device 1 according to exemplary embodiment 1, it becomes possible to perform benign / malignant classification including information contained in aggregate cells, thereby achieving a higher accuracy in classifying the benign / malignant nature of the specimen.

[0058] Furthermore, in the cell classification method, benign and malignant classification may be performed using a classification model trained with at least one of the following: training images showing benign and malignant individual cells, and training images showing benign and malignant aggregate cells. In particular, when benign and malignant classification is performed using both the trained individual cell classification model 31A and the trained aggregate cell classification model 31B, the same effect as that of the cell classification device 1B described above can be obtained.

[0059] [Examples of implementation using software] Some or all of the functions of the cell classification devices 1, 1A, and 1B may be implemented by hardware such as integrated circuits (IC chips) or by software.

[0060] In the latter case, the cell classification devices 1, 1A, and 1B are implemented by a computer that executes instructions for a program, which is software that implements each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 10. Computer C comprises at least one processor C1 and at least one memory C2. The memory C2 stores a program P that causes computer C to operate as cell classification devices 1, 1A, and 1B. In computer C, the processor C1 reads program P from memory C2 and executes it, thereby realizing each function of the cell classification devices 1, 1A, and 1B.

[0061] For processor C1, for example, a CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof can be used. For memory C2, for example, flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof can be used.

[0062] Computer C may also be equipped with RAM (Random Access Memory) for loading program P at runtime and for temporarily storing various data. Furthermore, computer C may be equipped with communication interfaces for sending and receiving data with other devices. Additionally, computer C may be equipped with input / output interfaces for connecting input / output devices such as keyboards, mice, displays, and printers.

[0063] Furthermore, program P can be recorded on a non-temporary, tangible recording medium M that is readable by computer C. Such a recording medium M could be, for example, tape, disk, card, semiconductor memory, or programmable logic circuitry. Computer C can acquire program P via such a recording medium M. Program P can also be transmitted via a transmission medium. Such a transmission medium could be, for example, a communication network or broadcast waves. Computer C can also acquire program P via such a transmission medium.

[0064] [Additional Note 1] The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the embodiments described above are also included in the technical scope of the present invention.

[0065] [Additional Note 2] Some or all of the embodiments described above may also be described as follows. However, the present invention is not limited to the embodiments described below.

[0066] (Note 1) A cell classification device characterized by comprising: acquisition means for acquiring an image containing sample cells as a subject; determination means for determining whether each of the sample cells is an individual cell or a clump of cells; and classification means for performing at least one of the following: a process for classifying the sample cells determined to be individual cells as benign or malignant, and a process for classifying the sample cells determined to be clump of cells as benign or malignant. With the above configuration, it is possible to classify sample cells as benign or malignant without being limited to a specific method.

[0067] (Note 2) The cell classification device described in Appendix 1, wherein the classification means performs benign / malignant classification only on the sample cells that have been determined to be individual cells. According to the above configuration, by classifying samples as benign or malignant after removing clump cells, the accuracy of classifying samples as benign or malignant can be improved.

[0068] (Note 3) The cell classification device according to Appendix 1, wherein the classification means performs benign or malignant classification on both the sample cells determined to be individual cells and the sample cells determined to be aggregate cells. According to the above configuration, even if cells appear to be clumps of cells, the accuracy of classifying them as benign or malignant can be improved by including individual cells that can be classified as benign or malignant.

[0069] (Note 4) The cell classification device described in Appendix 3 performs benign / malignant classification by referring to data of individual cells classified as malignant and data of aggregate cells classified as malignant. According to the above configuration, it becomes possible to classify cells as benign or malignant, including the information contained in the aggregate cells, thereby improving the accuracy of the benign / malignant classification of specimens.

[0070] (Note 5) The determination means is a cell classification device according to any one of the appendices 1 to 4, which determines whether a sample cell is an individual cell or a clump of cells by referring to at least one of the size and roundness of the sample cell. The above configuration allows for effective determination of whether a cell is an individual cell or a cluster of cells.

[0071] (Note 6) The cell classification device according to any one of the appendices 1 to 4, wherein the determination means includes a determination model learned using training images showing individual cells and training images showing aggregated cells. With the above configuration, individual cells and aggregate cells can be distinguished using a trained classification model, thereby improving the accuracy of classifying samples as benign or malignant.

[0072] (Note 7) The cell classification device described in Appendix 6, wherein the determination means detects each of the sample cells as an individual cell candidate or a cluster cell candidate, and then determines the detected sample cells using the learned determination model. According to the above configuration, it becomes possible to classify cells as benign or malignant, including the information contained in the aggregate cells, thereby improving the accuracy of the benign / malignant classification of specimens.

[0073] (Note 8) The cell classification device according to any one of Appendix 1 to 7, wherein the classification means includes a classification model learned using at least one of training images showing benign and malignant individual cells and training images showing benign and malignant cells in clumps. With the above configuration, benign and malignant cells can be classified using a trained classification model, thereby improving the accuracy of the classification of samples into benign and malignant states.

[0074] (Note 9) The cell classification device according to Appendix 8, wherein the determination means generates an image of a predetermined size containing the individual cells from an image of the sample cells determined to be individual cells, or an image of an arbitrary size containing the aggregate cells from an image of the sample cells determined to be aggregate cells, and then normalizes the image to the predetermined size. According to the above configuration, it is possible to generate images that can be input into a pre-trained classification model.

[0075] (Note 10) A cell classification device according to any one of the appendices 1 to 9, further comprising an output means for outputting the results of the classification means for benign or malignant cells. According to the above configuration, the classification results can be output to a display device.

[0076] (Note 11) A cell classification method comprising at least one processor performing at least one of the following: acquiring an image containing sample cells as a subject; determining whether each of the sample cells is an individual cell or a clump of cells; classifying the sample cells determined to be individual cells as benign or malignant; and classifying the sample cells determined to be clump of cells as benign or malignant. According to the above configuration, the same effect as described in Appendix 1 can be obtained.

[0077] (Note 12) The cell classification method described in Appendix 11, wherein the benign / malignant classification is performed only on the sample cells that have been determined to be individual cells. According to the above configuration, the same effect as described in Appendix 2 can be obtained.

[0078] (Note 13) The cell classification method described in Appendix 11, wherein the benign / malignant classification is performed on both the sample cells determined to be individual cells and the sample cells determined to be aggregate cells. According to the above configuration, the same effect as described in Appendix 3 can be obtained.

[0079] (Note 14) The cell classification method described in Appendix 13, wherein the classification of benign and malignant is performed by referring to the data of the individual cells classified as malignant and the data of the aggregate cells classified as malignant. According to the above configuration, the same effect as described in Appendix 4 can be obtained.

[0080] (Note 15) The cell classification method described in any one of Appendix 11 to 14, wherein the determination is made by referring to at least one of the size and roundness of the sample cells to determine whether they are individual cells or aggregate cells. According to the above configuration, the same effect as described in Appendix 5 can be obtained.

[0081] (Note 16) The cell classification method described in any one of the appendices 11 to 14, wherein the determination is made using a determination model trained with training images showing individual cells and training images showing aggregated cells. According to the above configuration, the same effect as described in Appendix 6 can be obtained.

[0082] (Note 17) The cell classification method described in Appendix 16, wherein the determination involves detecting each of the sample cells as an individual cell candidate or a cluster of cells candidate, and then determining the detected sample cells using the learned determination model. According to the above configuration, the same effect as that described in Appendix 7 can be obtained.

[0083] (Note 18) The cell classification method described in any one of the appendices 11 to 17, wherein the benign / malignant classification is performed using a classification model trained with at least one of the following: training images showing benign and malignant cells of individual cells and training images showing benign and malignant cells of a mass. According to the above configuration, the same effect as that described in Appendix 8 can be obtained.

[0084] (Note 19) The cell classification method described in Appendix 18, wherein the determination is made by cutting out an image of a predetermined size containing the individual cell from an image of the sample cells determined to be individual cells, or by cutting out an image of any size containing the aggregate cells from an image of the sample cells determined to be aggregate cells and normalizing it to the predetermined size, and then making a determination using the image. According to the above configuration, the same effect as that described in Appendix 9 can be obtained.

[0085] (Note 20) A cell classification method according to any one of appendices 11 to 19, further comprising outputting the results of the benign / malignant classification. According to the above configuration, the same effect as described in Appendix 10 can be obtained.

[0086] (Note 21) A cell classification program that causes a computer to perform at least one of the following processes: acquiring an image containing sample cells as the subject; determining whether each of the sample cells is an individual cell or a clump of cells; classifying the sample cells determined to be individual cells as benign or malignant; and classifying the sample cells determined to be clump of cells as benign or malignant.

[0087] [Additional Note 3] Some or all of the embodiments described above can also be expressed as follows: A cell classification device comprising at least one processor, wherein the processor performs an acquisition process to acquire an image containing sample cells as a subject; a determination process to determine whether each of the sample cells is an individual cell or a clump of cells; a process to perform benign or malignant classification on the sample cells determined to be individual cells; and a classification process to perform benign or malignant classification on the sample cells determined to be a clump of cells.

[0088] Furthermore, this cell classification device may also be equipped with memory, and this memory may store a program that causes the processor to execute the acquisition process, the determination process, and the classification process. This program may also be recorded on a computer-readable, non-temporary, tangible recording medium. [Explanation of symbols]

[0089] 1,1A,1B…Cell sorting device 10…Acquisition part 20,20A…judgment part 21...Trained Classification Model 30,30A…Classification section 31…Trained classification model 40…Output section 50…Control Unit 51… Processor 52...Memory

Claims

1. An acquisition means for acquiring an image containing multiple sample cells as subjects, A determination means for determining whether each of the plurality of sample cells is an individual cell or a clump of cells, based on a determination criterion for determining an individual cell and a determination criterion for determining a clump of cells that differs from the determination criterion for determining an individual cell. Image generation means for generating an image that is cropped to a predetermined size including the individual cell from an image of the sample cell determined to be an individual cell by the determination means, or an image of any size including the aggregate cell from an image of the sample cell determined to be a aggregate cell, and then normalized to the predetermined size. A classification means that performs at least one of the following: a process of classifying the benign or malignant state of an image cropped to a predetermined size including the individual cells generated by the image generation means; and a process of classifying the benign or malignant state of an image including the aggregate of cells normalized to the predetermined size by the image generation means. A cell classification device equipped with the following features.

2. The cell classification device according to claim 1, wherein the classification means performs benign or malignant classification only on the sample cells that have been determined to be individual cells.

3. The classification means performs a benign or malignant classification on both the sample cells determined to be individual cells and the sample cells determined to be aggregate cells, by referring to the data of the individual cells classified as malignant and the data of the aggregate cells classified as malignant, respectively. The cell classification device according to claim 1.

4. The determination means determines whether the sample cells are individual cells or clustered cells by referring to at least one of the size and roundness of the sample cells. A cell classification device according to any one of claims 1 to 3.

5. The determination means includes a determination model trained using training images representing individual cells and training images representing clusters of cells. A cell classification device according to any one of claims 1 to 3.

6. The determination means detects each of the sample cells as a candidate individual cell or a candidate cluster of cells, and then determines the detected sample cells using the learned determination model. The cell classification device according to claim 5.

7. The cell classification device according to any one of claims 1 to 3, wherein the classification means includes a classification model learned using at least one of training images showing benign and malignant cells of individual cells and training images showing benign and malignant cells in clumps.

8. An image output means that outputs at least one of the following: an image in which only sample cells determined to be individual cells by the determination means are detected as the subject, and an image in which only sample cells determined to be aggregate cells by the determination means are detected as the subject, The cell classification device according to claim 1, further comprising:

9. At least one processor, To obtain an image that includes multiple sample cells as the subject, For each of the aforementioned plurality of sample cells, it is determined whether they are individual cells or aggregate cells, based on a determination criterion for determining individual cells and a determination criterion for determining aggregate cells that differs from the determination criterion for determining individual cells. The process involves generating an image of a predetermined size, which includes the individual cells, from an image of the sample cells determined to be individual cells, or generating an image of an arbitrary size, which includes the aggregated cells, from an image of the sample cells determined to be aggregated cells, and then normalizing it to the predetermined size. At least one of the following: classifying the generated individual cells as benign or malignant in an image cut to a predetermined size, and classifying the aggregated cells normalized to the predetermined size. A cell classification method that includes this.

10. A computer, A process to acquire an image containing multiple sample cells as the subject, A process for determining whether each of the aforementioned plurality of sample cells is an individual cell or a clump of cells, based on a determination criterion for determining an individual cell and a determination criterion for determining a clump of cells that differs from the determination criterion for determining an individual cell. A process to generate an image of a predetermined size, which is cropped from an image of a sample cell determined to be an individual cell by the above determination process, and which includes the individual cell, or an image of an arbitrary size, which includes the aggregate cell, from an image of a sample cell determined to be a aggregate cell, and which is then normalized to the predetermined size. A process of classifying the benign or malignant state of an image cropped to a predetermined size containing the individual cells generated by the image generation process, and a process of classifying the benign or malignant state of an image containing the aggregate of cells normalized to the predetermined size by the image generation process, A cell classification program that performs this task.

Images