Data augmentation system for tumor recognition from cell images based on DAPI Staining

Abstract

The cell image classification system based on DAPI staining and multiple instance learning according to the present invention comprises: a shooting unit for capturing a DAPI-stained cell image; a patch segmentation unit for dividing the cell image captured by the shooting unit into a plurality of patch regions; a machine learning unit for learning whether each patch divided by the machine learning unit contains cancer cells through a Multiple Instance Learning (MIL) method for each patch divided by the patch segmentation unit; and a determination unit for determining whether each patch contains cancer cells through a learning module learned by the machine learning unit. According to the present invention, the nucleus of a cell is stained through DAPI staining, and the nucleus is classified into malignant and benign based on the stained pattern of the nucleus, and in the case of malignancy, a deep learning-based multi-entity learning among machine learning techniques can be used to determine it as cancer.

Description

Technology Field

[0001] The present invention relates to a data augmentation system for tumor recognition from DAPI staining-based cell images, and more specifically, to a system for automatically generating AI training data to improve the accuracy of DAPI staining-based cell image recognition obtained through tissue biopsy. Background Technology

[0002] Although there are many methods for classifying diseases, over the past two centuries, the morphology and function of normal tissues have been successfully used as a reference point to define various diseases of the same tissue, and these attempts have been used to effectively classify hematopoietic tumors such as lymphoma and leukemia.

[0003] Despite major successes in the rational classification and treatment of hematopoietic malignancies, the use of normal cell types to classify other types of solid tumors has not been widely replicated. The primary reason for this lies in the lack of understanding of cell type diversity in most solid tissues.

[0004] Meanwhile, DAPI (4',6-diamidino-2-phenylindole, Dihydrochloride) refers to a substance primarily used for staining cell images. DAPI staining refers to a DNA staining method using the fluorescent stain DAPI (4',6-diamidino-2-phenylindole), which allows the stained region to be identified by detecting blue fluorescence. The problem to be solved

[0005] The purpose of the present invention is to automatically generate AI training data to improve the accuracy of cell image recognition based on DAPI staining obtained through tissue biopsy.

[0006] In addition, the present invention provides a system that improves learning efficiency by augmenting learning data for determining the presence of a tumor through DAPI staining of cell images obtained through tissue biopsy. means of solving the problem

[0007] A data augmentation system for tumor recognition from DAPI staining-based cell images according to the present invention comprises: a shooting unit for capturing DAPI-stained cell images; a machine learning unit for learning whether cancer cells are included in the captured images through a Multiple Instance Learning (MIL) method; a learning data generation unit for artificially generating pre-learning data to be pre-learned by the machine learning unit; and a determination unit for determining whether cancer cells are included in each patch through a learning module learned by the machine learning unit.

[0008] In addition, the above-mentioned training data generation unit can generate fake images using a combination of images excluding cancer cells, a combination of images excluding cancer cells and images including cancer cells, and artificially formed images, and provide them as training data to the above-mentioned machine learning unit.

[0009] In addition, the machine learning unit can learn by determining whether cancer cells are included from training data including fake images and images containing cancer cells generated by the training data generation unit.

[0010] The above-mentioned training data generation unit performs training to generate the fake image in a direction that increases the error rate of the above-mentioned judgment unit, and the above-mentioned judgment unit performs training from training data including the fake image generated by the above-mentioned training data generation unit in a direction that increases the accuracy rate, thereby each performing machine learning in an adversarial manner.

[0011] In addition, the above training data generation unit may utilize the Generative Adversarial Network (GAN) technique. Effects of the invention

[0012] According to the present invention, the nucleus of a cell is stained through DAPI staining, and a machine learning technique is used to classify it as malignant or benign based on the stained pattern of the nucleus and to determine it as cancer in the case of malignancy, and the efficiency can be improved by augmenting the training data for using such a machine learning technique. Brief explanation of the drawing

[0013] FIG. 1 is a block diagram illustrating a cell image classification system based on multi-entity learning and DAPI staining according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating a cell image classification process based on multi-entity learning and DAPI staining according to one embodiment. Figure 3 is a photograph showing an example of a cell image according to DAPI staining. FIG. 4 is a schematic diagram sequentially illustrating the process by a cell image classification system based on multi-entity learning and DAPI staining according to one embodiment. FIG. 5 is a block diagram illustrating the process of generating augmented data according to one embodiment. Figure 6 is a photograph showing fake images generated through the augmented data generation process. Specific details for implementing the invention

[0014] Embodiments of the present invention will be described below with reference to the attached drawings. Unless otherwise specifically defined or mentioned, terms indicating direction used in this description are based on the state shown in the drawings. Additionally, throughout each embodiment, the same reference numeral indicates the same component. Meanwhile, the thickness or dimensions of each component shown in the drawings may be exaggerated for convenience of explanation and do not imply that it must actually be constructed according to the corresponding dimensions or ratios between components.

[0016] With reference to FIGS. 1 to 4, a machine learning system for cell image classification based on DAPI staining according to one embodiment and a process according to the same will be described. FIG. 1 is a block diagram showing a machine learning system for cell image classification based on DAPI staining based on entity learning according to one embodiment of the present invention, FIG. 2 is a flowchart for explaining a machine learning process for cell image classification based on DAPI staining based on entity learning according to one embodiment, FIG. 3 is a photograph showing an example of a cell image based on DAPI staining, and FIG. 4 is a schematic diagram sequentially showing a process by a cell image classification system based on multiple entity learning based on DAPI staining.

[0017] A machine learning system for cell image classification based on DAPI staining according to one embodiment of the present invention includes a shooting unit (100), a patch segmentation unit (150), a machine learning unit (200), and a judgment unit (300) as shown in FIG. 1.

[0018] The imaging unit (100) is a component that captures DAPI-stained cell images, and the patch segmentation unit (150) is a component that divides the cell images captured by the imaging unit into multiple patch regions. The machine learning unit (200) performs machine learning for each patch segmented by the patch segmentation unit to form a learning module, and the determination unit (300) uses the aforementioned learning module to determine whether the entire cell image contains cancer cells through a Multiple Instance Learning (MIL) method for each patch.

[0019] Additionally, the judgment unit (300) can provide information on the determination result of whether cancer cells are included in the cell image captured by the imaging unit (100) and the position occupied by the patch determined to contain cancer cells in the cell image, and can also provide data such as the number of patches determined to contain cancer cells in the cell image and data derived from the number of patches determined to contain cancer cells, i.e., the state of progression of cancer.

[0020] Meanwhile, the method of distinguishing by patch as described above is implemented as one embodiment, but is not limited to such a method and can be implemented by introducing various determination methods.

[0021] The data augmentation system for tumor recognition from DAPI staining-based cell images according to the present invention is characterized by a computer automatically generating tumor data and applying it to machine learning.

[0022] That is, the data augmentation system for tumor recognition from DAPI staining-based cell images according to the present embodiment is characterized by a training data generation unit that artificially generates training data and a machine learning unit that performs training using the same.

[0023] In other words, to improve the performance of deep learning, a large amount of data is required, and both normal data and tumor data are needed. However, while normal data is relatively easy to obtain, tumor data is relatively difficult to collect, so we intend to automatically generate it using an artificial intelligence algorithm.

[0024] The training data generation unit (400) can generate fake images using a combination of images excluding cancer cells, a combination of images excluding cancer cells and images including cancer cells, and artificially formed images, and provide them as training data to the machine learning unit and the judgment unit (300).

[0025] In addition, the machine learning unit can learn by determining whether cancer cells are included from the training data, which includes a fake image and an image containing cancer cells generated by the training data generation unit. The machine learning unit (200) performs learning on both sides of the training data generation unit (400) and the determination unit (300), and the specific learning method will be described below.

[0026] As illustrated in FIG. 2, a machine learning process for cell image classification based on entity learning and DAPI staining according to the present embodiment is described.

[0027] First, cell DAPI staining is performed (S100). DAPI penetrates the cell membrane and binds to DNA, and at this time absorbs light in the wavelength range of about 358 nm (Max.) and emits blue to cyan fluorescence.

[0028] The principles by which DAPI binds to DNA can be broadly categorized into two types: weak groove-binding, which primarily acts on AT-rich regions, and intercalative binding, which acts on DNA sequences containing GC or a mixture of AT and GC. Among these, binding by interlacing between DNA strands accounts for the majority of cases. When DAPI binds to DNA, a modification occurs in its directional ring structure, causing the absorption wavelength to shift to the visible light region.

[0029] Subsequently, the stained cells are photographed in the imaging unit (S200). An example of the photographed stained cells is shown in Fig. 3. In Fig. 3, the left side shows the state before staining, and the right side shows the image taken after DAPI staining.

[0030] When stained with DAPI, the nucleus of the cell is stained, and the cell is classified as malignant or benign based on the stained pattern of the nucleus. If it is malignant, it can be determined as cancer, and to automatically determine this, machine learning techniques are used as follows.

[0031] Referring to FIG. 4, to apply a multi-entity-based machine learning method, a cell image is divided into multiple patch regions (S300), multi-entity-based machine learning is performed for each patch (S400), and then a final tumor identification result is output using this (S500).

[0032] Conventionally, when a document is given as input, a feature vector is generated using a network such as an RNN or CNN, and document classification is performed on top of it. On the other hand, the multiple instance learning according to the present embodiment performs a judgment on each of the patches constituting the document when a document is given as input, and combines the results to obtain an overall document judgment result.

[0033] This multi-entity-based machine learning method has the advantage that it can have a judgment value for each patch and can derive judgment results for the entire document in various ways depending on the combination method of each patch.

[0034] As in the present embodiment, if it is determined that cancer cells are contained in any one patch, it can be determined that cancer cells are contained in the entire image, whereas when determining the degree of progression, etc., the determination can be made by utilizing the number of patches determined to contain cancer cells.

[0035] Meanwhile, the generation of augmented data or training data (S350) and machine learning using it (S360) are performed as separate processes and are used in the tumor determination machine learning (S400) step.

[0036] In this regard, the following is explained in detail with reference to FIGS. 5 and FIG. 6. FIG. 5 is a block diagram illustrating an augmented data generation process according to one embodiment, and FIG. 6 is a photograph showing fake images generated through the augmented data generation process.

[0037] Referring to FIG. 5, machine learning is performed in parallel in two ways. First, when the training data generation unit (400) generates a fake image as described above, the judgment unit (300) performs a judgment using training data containing an image containing actual cancer cells, and the result of this judgment is evaluated by the machine learning unit (200) as described above, and learning proceeds by reinforcing the neural network in the training data generation unit (400) and the judgment unit (300).

[0038] At this time, the learning data generation unit (400) performs learning to generate fake images in a direction that increases the error rate of the evaluation result judgment unit (300), and the judgment unit (300) performs learning in a direction that increases the accuracy rate of selecting actual cancer cells from the learning data including the fake images generated by the learning data generation unit (400), thereby performing machine learning in an adversarial manner.

[0040] Although preferred embodiments of the present invention have been described above, the technical concept of the present invention is not limited to the preferred embodiments described above and can be implemented in various ways within the scope that does not depart from the technical concept of the present invention as embodied in the claims. Explanation of the symbols

[0041] 100: Filming Department 150: Patch split section 200: Machine Learning Department 300: Judgment Division 400: Training data generation section

Claims

Claim 1 A capturing unit for capturing images of DAPI-stained cells; a machine learning unit for learning whether cancer cells are included in the captured images through a Multiple Instance Learning (MIL) method; and a training data generation unit for artificially generating pre-training data for the machine learning unit to pre-train. A data augmentation system for tumor recognition from DAPI staining-based cell images, comprising: a determination unit that determines whether each patch contains cancer cells through a learning module learned by the machine learning unit; wherein the learning data generation unit generates fake images from combinations of images excluding cancer cells, combinations of images excluding cancer cells and images containing cancer cells, and artificially formed images, and provides them as learning data to the machine learning unit; wherein the machine learning unit learns by determining whether cancer cells are included from the learning data including the fake images generated by the learning data generation unit and images containing cancer cells; wherein the learning data generation unit performs learning to generate the fake images in a direction that increases the error rate of the determination unit, and the determination unit performs learning to increase the accuracy rate from the learning data including the fake images generated by the learning data generation unit, thereby performing machine learning in an adversarial manner. Claim 2 delete Claim 3 delete Claim 4 A data augmentation system for tumor recognition from DAPI staining-based cell images, wherein, in claim 1, the learning data generation unit utilizes a Generative Adversarial Network (GAN) technique.

Images