Three-dimensional tomography fast regularization method and device using machine learning algorithm

Abstract

A three-dimensional tomography fast regularization method and apparatus using a machine learning algorithm are proposed. A regularization method of three-dimensional tomography using a machine learning algorithm according to an embodiment can include a step of obtaining a raw tomogram of a cell by measuring a three-dimensional tomogram of the cell; a step of obtaining a regularized tomogram using a regularization algorithm; and a step of learning a relationship between the raw tomogram and the regularized tomogram by a machine learning algorithm.

Description

TECHNICAL FIELD

[0001] Various embodiments below relate to a three-dimensional tomographic imaging fast regularization method and apparatus using a machine learning algorithm. BACKGROUND

[0002] Currently, due to the physical limit of an optical lens, a three-dimensional tomographic imaging technique cannot measure spatial frequency information in the direction of an optical axis, and thus the resolution in the corresponding direction is degraded. In order to overcome this, a regularization algorithm based on iteration is used to fill in the unmeasured spatial frequency information, and the regularization algorithm uses known information or assumptions about a test piece.

[0003] Commonly used regularization algorithms include non-negativity, which uses an assumption that the background of a test piece has a value above a certain value, and total variation, which uses an assumption that a three-dimensional tomographic image of a test piece has a small spatial variation rate.

[0004] In particular, in the case of a regularization algorithm such as total variation, since the resolution in the direction of the optical axis of the three-dimensional image can be effectively improved, it is widely used, but since the regularization three-dimensional tomographic image is obtained by iteratively calculating using gradient descent, there is a disadvantage that the calculation time takes several minutes to several hours. Also, since the algorithmic parameters suitable for each test piece are different, there is a problem that the user can only find the best result in a trial-and-error manner by inputting various variables.

[0005] Non-Patent Document 1: Kim, K., et al. (2016). "Optical diffraction tomography techniques for the study of cell pathophysiology." arXiv preprint arXiv:1603.00592.

[0006] Non-Patent Document 2: Wolf, E. (1969). "Three-dimensional structure determination of semi-transparent objects from holographic data." Optics Communications 1(4): 153-156.

[0007] Non-Patent Document 3: Lim, J. (2015). "Comparative study of iterative reconstruction algorithms for missing cone problems in optical diffraction tomography." Optics Express 23(13): 16933-16948.

[0008] Non-Patent Document 4: Park, Y. (2018). "Quantitative phase imaging in biomedicine." Nature Photonics 12(10): 578-589. SUMMARY

[0009] TECHNICAL PROBLEM

[0010] Various embodiments describe a three-dimensional tomography fast regularization method and apparatus using a machine learning algorithm, and more particularly, provide a technology for providing fast regularization of tomography images through a deep learning algorithm that has learned features of a regularization algorithm such as total variation, without an optimization algorithm or a search variable.

[0011] Various embodiments provide a three-dimensional tomography fast regularization method and apparatus using a machine learning algorithm, which learns a nonlinear relationship between a three-dimensional tomogram first optically photographed and a regularized three-dimensional tomogram thereof through deep learning, and regularizes a newly photographed three-dimensional tomogram within several seconds to improve resolution.

[0012] SOLUTION TO THE PROBLEM

[0013] A regularization method of three-dimensional tomography using a machine learning algorithm of an embodiment can include a step of obtaining a raw tomogram of a cell by measuring a three-dimensional tomogram of the cell, a step of obtaining a regularized tomogram using a regularization algorithm, and a step of learning a relationship between the raw tomogram and the regularized tomogram through a machine learning algorithm.

[0014] And, it can further include a step of regularizing a measured three-dimensional tomogram of a cell using the learned machine learning algorithm.

[0015] In the step of obtaining the original tomographic image of the cell by measuring the three-dimensional tomographic image of the cell, the three-dimensional refractive index image of the cell can be measured using a method of irradiating incident light from a plurality of angles while rotating, or the three-dimensional refractive index image of the cell can be obtained by using a plurality of two-dimensional images measured while rotating (sample rotation) or translating the cell.

[0016] In the step of obtaining the regularized tomographic image using the regularization algorithm, the regularized refractive index image corresponding to each image can be obtained by total variation regularization, forming a paired dataset of the original tomographic image and the regularized tomographic image.

[0017] In the step of learning the relationship between the original tomographic image and the regularized tomographic image by the machine learning algorithm, a nonlinear relationship between the paired dataset of the original tomographic image and the regularized tomographic image can be learned by the machine learning algorithm to extract a specific feature of the cell species.

[0018] In the step of regularizing the measured three-dimensional tomographic image of the cell using the machine learning algorithm, the measured original tomographic image can be input to a Convolutional Neural Network (CNN) algorithm to output a regularized tomographic image, and the regularized tomographic image can be applied to distinguish the species of the cell.

[0019] The step of regularizing the measured three-dimensional tomographic image of the cell using the machine learning algorithm can include a contraction step of extracting a specific feature from the original tomographic image data of the cell by sequentially and stepwise applying convolution and subsampling, and an expansion step of outputting again the regularized tomographic image data of the same size as the input value by sequentially and stepwise applying convolution and subsampling.

[0020] The regularization apparatus for three-dimensional tomography using a machine learning algorithm according to other embodiments can include a three-dimensional tomographic image measurement unit for obtaining an original tomographic image of a cell by measuring a three-dimensional tomographic image of the cell, a regularization algorithm for obtaining a regularized tomographic image, and a machine learning algorithm for learning a relationship between the original tomographic image and the regularized tomographic image.

[0021] Also, the present application can further include a regularization unit for regularizing a measured three-dimensional tomographic image of a cell using the learned machine learning algorithm.

[0022] The regularization unit can be configured to input the measured original tomographic image into a convolutional neural network algorithm to output a regularized tomographic image, and to apply the regularized tomographic image to distinguish the types of the cells.

[0023] Effects of the Invention

[0024] According to various embodiments, a three-dimensional tomographic imaging fast regularization method and apparatus using a machine learning algorithm can be provided, and a deep learning algorithm that has learned features of a total variation or the like regularization algorithm can be used to provide a deep learning-based regularization method that is 10 times faster than an existing regularization method used in a three-dimensional tomographic imaging technique.

[0025] According to various embodiments, a three-dimensional tomographic imaging fast regularization method and apparatus using a machine learning algorithm can be provided, and a deep learning algorithm that has learned features of a total variation or the like regularization algorithm can be used to provide a deep learning-based regularization method that is 10 times faster than an existing regularization method used in a three-dimensional tomographic imaging technique. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 A diagram for briefly explaining a regularization method of three-dimensional tomographic imaging using a machine learning algorithm according to an embodiment.

[0027] Figure 2 A flowchart for illustrating a regularization method of three-dimensional tomographic imaging using a machine learning algorithm according to an embodiment.

[0028] Figure 3 A block diagram for illustrating a regularization apparatus of three-dimensional tomographic imaging using a machine learning algorithm according to an embodiment.

[0029] Figure 4a A diagram for explaining a method of measuring a three-dimensional refractive index of a cell using an incident light rotation method according to an embodiment.

[0030] Figure 4b A diagram for explaining a method of measuring a three-dimensional refractive index of a cell using a cell rotation method according to an embodiment.

[0031] Figure 4c A diagram for explaining a method of measuring a three-dimensional refractive index of a cell using a sample translation method according to an embodiment.

[0032] Figure 5 A diagram for explaining a regularization method of three-dimensional tomographic imaging using a convolutional neural network according to an embodiment.

[0033] Figure 6FIG. 1 is a diagram showing an optical diffraction tomography (ODT) and regularization flow using a machine learning algorithm according to an embodiment.

[0034] Figure 7 FIG. 2 is a diagram showing a structure of a machine learning algorithm according to an embodiment. DETAILED DESCRIPTION

[0035] Various embodiments will be described with reference to the accompanying drawings. However, the various embodiments described can be modified in various other forms, and the scope of the present invention is not limited to the various embodiments described below. Also, various embodiments are provided in order to describe the present invention more completely to those skilled in the art to which the present invention pertains. For a clearer description, the shape and size of elements in the accompanying drawings can be exaggerated.

[0036] Three-dimensional tomographic imaging techniques widely used in life sciences and medicine are physically unable to obtain spatial frequency information (kz) corresponding to the direction of the optical axis (generally, the z coordinate on the x, y, z coordinate system) because the angle at which the specimen is physically illuminated and the angle at which light scattered from the specimen is obtained by the lens are limited by the light acquisition limit (e.g., numerical aperture) of the lens. z Therefore, there is a problem of resolution degradation in the direction of the optical axis and distortion of the result of reconstructing a three-dimensional image in the z direction, which is often referred to as a missing cone problem.

[0037] In order to solve these missing cone problems, it is very important to perform post-processing with a computational regularization algorithm (Non-Patent Literature 3). However, since existing regularization algorithms generally take several minutes to several hours, real-time three-dimensional visualization of a specimen cannot be achieved, and therefore, a fast regularization algorithm needs to be developed.

[0038] The various embodiments below provide a three-dimensional tomographic imaging fast regularization method and apparatus using a machine learning algorithm, and provide a deep learning-based regularization method that is more than 10 times faster than an existing regularization method used in three-dimensional tomographic imaging techniques.

[0039] More specifically, various embodiments learn a non-linear relationship between a first optical photograph of a three-dimensional tomographic image and a regularized three-dimensional tomographic image thereof through deep learning such as a convolutional neural network algorithm, and regularize a newly photographed three-dimensional tomographic image in seconds to improve resolution, rather than regularizing a missing cone that causes resolution reduction in an optical axis direction in a three-dimensional tomographic imaging technique using an iterative algorithm based on total variation that takes several minutes to several hours.

[0040] Figure 1 A diagram for a regularization method of three-dimensional tomographic imaging using a machine learning algorithm of an embodiment.

[0041] An object of the present embodiment is to provide fast regularization of a tomographic image through a deep learning algorithm that has learned characteristics of a regularization algorithm such as total variation without an optimization algorithm or a search variable.

[0042] A three-dimensional tomographic imaging technique can make a three-dimensional image by synthesizing a plurality of two-dimensional images obtained by irradiating a sample from a plurality of angles, rotating the sample, or translating the sample.

[0043] Since an angle allowed in a lens cannot exceed a certain range, an angle range of a light source incident to a sample and an angle range of light scattered from the sample are physically limited by a numerical aperture (NA) of the lens, respectively. Due to these missing cone problems, all spatial frequency information in an optical axis direction cannot be obtained in reality, and resolution is correspondingly reduced. For these missing cone problems, a subsequent computational regularization algorithm is used to fill, but many existing regularization methods iteratively search for a regularized image, and thus take a very long time. Therefore, a high-spec computing power or an optimization algorithm is required to reduce regularization running time, but a level of several seconds is not possible with the current technology level.

[0044] Reference Figure 1According to an embodiment, a three-dimensional tomogram of a cell can be measured by the three-dimensional tomogram measuring unit 110, and a regularized tomogram can be obtained using a conventional regularization algorithm 120 such as a total variation. At this time, a database 111 of original tomograms of cells can be constructed by measuring three-dimensional tomograms of cells by the three-dimensional tomogram measuring unit 110, and a database of regularized tomograms can be constructed by the regularization algorithm 120.

[0045] Then, a complex relationship (mapping) between the original tomogram and the regularized tomogram is learned by a machine learning algorithm, and as a result, a newly measured three-dimensional tomogram of a cell can be quickly regularized without further optimization.

[0046] According to various embodiments, a three-dimensional refractive index distribution (image) of a test piece such as a living cell can be measured without using staining or labeling, and a kind of cell can be distinguished using the same. When the three-dimensional refractive index distribution is used, a kind of cell can be distinguished using specific morphological and biochemical characteristics of the kind of cell. In particular, by measuring a three-dimensional refractive index of a cell and applying the measured value to a machine learning algorithm, a kind of cell can be simply and accurately distinguished.

[0047] Figure 2 A flowchart of a regularization method of three-dimensional tomography using a machine learning algorithm according to an embodiment is shown.

[0048] Referring to Figure 2 A regularization method of three-dimensional tomography using a machine learning algorithm according to an embodiment can include a step 210 of obtaining original tomograms of cells by measuring three-dimensional tomograms of cells, a step 220 of obtaining regularized tomograms using a regularization algorithm, and a step 230 of learning a relationship between the original tomograms and the regularized tomograms by a machine learning algorithm.

[0049] Also, a step 240 of regularizing a measured three-dimensional tomogram of a cell using the learned machine learning algorithm can be further included.

[0050] According to various embodiments, by learning a non-linear relationship between a three-dimensional tomogram first optically photographed and a regularized three-dimensional tomogram thereof through deep learning, a newly photographed three-dimensional tomogram can be regularized to improve resolution in a few seconds.

[0051] Each step of a regularization method of three-dimensional tomography using a machine learning algorithm according to an embodiment will be described in more detail below.

[0052] A regularization method of three-dimensional tomography using a machine learning algorithm of one embodiment can be described by way of an example of a regularization device of three-dimensional tomography using a machine learning algorithm of one embodiment.

[0053] Figure 3 A block diagram of a regularization device of three-dimensional tomography using a machine learning algorithm of one embodiment is shown.

[0054] Reference Figure 3 A regularization device of three-dimensional tomography using a machine learning algorithm 300 of one embodiment can include a three-dimensional tomographic image measuring unit 310, a regularization algorithm 320, and a machine learning algorithm 330. According to various embodiments, the regularization device of three-dimensional tomography using a machine learning algorithm 300 can further include a regularization unit 340.

[0055] In step 210, the three-dimensional tomographic image measuring unit 310 can obtain a raw tomographic image of a cell by measuring a three-dimensional tomographic image of the cell.

[0056] The three-dimensional tomographic image measuring unit 310 can optically measure a three-dimensional refractive index distribution of a cell. For example, the three-dimensional refractive index measuring unit 310 can be an optical system including a light source and a camera, and can have various forms such as a reflection type and a transmission type.

[0057] The three-dimensional refractive index measuring unit 310 can include a light source, an interferometer, and a measuring unit. The light source can emit light to a cell. For example, a laser can be used as the light source, and the light source can emit a laser beam to a sample such as a cell to be measured. The light source can use a single-wavelength laser. The light source can measure a three-dimensional refractive index in each wavelength by using a multi-wavelength laser, and thus can use a larger amount of information for distinguishing cells.

[0058] Here, the cell can represent a sample representing an object to be measured, and can be not only a cell but also bacteria or microorganisms, or an object including a cell.

[0059] The interferometer can obtain a plurality of two-dimensional holograms by measuring transmitted light diffracted from the cell after light incident from the light source is incident on the cell. The interferometer is a measuring instrument that uses an interference phenomenon of light, and is an instrument that divides light from the same light source into two or more, makes the travel paths different, and observes an interference phenomenon generated when the light meets again.

[0060] The measuring unit can measure a three-dimensional refractive index distribution of a cell using the plurality of two-dimensional holograms obtained from the interferometer. For example, as the measuring unit, a camera that is an imaging device for imaging can be used.

[0061] The three-dimensional refractive index measurement section 310 can measure the three-dimensional refractive index distribution of the cell by at least one of optical measurements in optical diffraction tomography and optical projection tomography.

[0062] The three-dimensional tomographic image measurement section 310 can measure the three-dimensional refractive index image of the cell in a manner of rotating incident light irradiated from a plurality of angles, or obtain the three-dimensional refractive index image of the cell by using a plurality of two-dimensional images measured while rotating or translating the cell.

[0063] In step 220, a regularized tomographic image can be obtained using a regularization algorithm 320. The regularization algorithm 320 can obtain a regularized refractive index image corresponding to each image by regularization such as total variation, forming a paired dataset of the original tomographic image and the regularized tomographic image.

[0064] In step 230, a machine learning algorithm 330 can learn the relationship between the original tomographic image and the regularized tomographic image. The machine learning algorithm 330 can extract a specific feature of the cell species by learning a non-linear relationship between the paired dataset of the original tomographic image and the regularized tomographic image.

[0065] The machine learning algorithm 330 can be composed of a deep learning algorithm such as a deep neural network (DNN) or a convolutional neural network (CNN) algorithm. Therefore, a refractive index feature specific to the cell species can be extracted using the measured cell original tomographic image by the deep learning algorithm or the convolutional neural network algorithm.

[0066] In step 240, a regularization section 340 can regularize the measured three-dimensional tomographic image of the cell using the learned machine learning algorithm.

[0067] The regularization section 340 can input the measured original tomographic image to the deep learning algorithm or the convolutional neural network algorithm, output the regularized tomographic image, and apply the regularized tomographic image to distinguish the species of the cell. In particular, the regularization section 340 can regularize the measured three-dimensional tomographic image of the cell by a contraction step and an expansion step, the contraction step extracts a specific feature from the original tomographic image data of the cell by sequentially and gradually applying convolution and subsampling, and the expansion step outputs the regularized tomographic image data of the same size as the input value again by sequentially and gradually applying convolution and subsampling.

[0068] Hereinafter, a method of measuring a three-dimensional tomographic image (refractive index) of a cell will be described.

[0069] Figure 4aFig. 1 is a diagram for explaining a method of measuring a three-dimensional refractive index of a cell using an incident light rotation method according to an embodiment. Also, Figure 4b Fig. 2 is a diagram for explaining a method of measuring a three-dimensional refractive index of a cell using a cell rotation method according to an embodiment. Also, Figure 4c Fig. 3 is a diagram for explaining a method of measuring a three-dimensional refractive index of a cell using a sample translation method according to an embodiment.

[0070] The present embodiment can be applied to all kinds of three-dimensional tomographic images. Here, a refractive index, which all objects have in a three-dimensional tomographic image, will be explained. The refractive index is an optical physical quantity inherent to a substance itself, which describes the degree of reduction in the speed of light when the light passes through the substance.

[0071] In order to measure the three-dimensional refractive index of the cell 401, an optical diffraction tomography method or an optical projection tomography method (tomographic phase microscopy, three-dimensional digital holographic microscopy, three-dimensional quantitative phase imaging) (non-patent document 1) can be used. Figures 4a to 4c Various possible optical measurement implementations are shown.

[0072] As shown in Figure 4a , the optical diffraction tomography method and the optical projection tomography method can use the same optical implementation (non-patent document 2 and non-patent document 4). Light from a coherent light source 410 is incident on the cell 401, and a hologram of transmitted light diffracted from the cell 401 is measured using an interferometer 420.

[0073] At this time, the three-dimensional refractive index distribution 440 of the cell 401 can be measured using a plurality of two-dimensional holograms measured while the angle of the above light rotation (scanning) incident on the cell 401 is measured. However, the difference between the diffraction tomography method and the projection tomography method is the reconstruction algorithm 430 that considers whether the light is diffracted in the sample.

[0074] Referring to Figure 4b , the method of measuring the three-dimensional refractive index distribution of the cell 401 using the incident light rotation method described in Figure 4a , the three-dimensional refractive index distribution 440 can be measured by directly rotating the cell 401, instead of rotating the incident light.

[0075] Also, Figure 4c Referring to, in Figure 4aIn the method for measuring the three-dimensional refractive index distribution of the cell 401 by rotating the incident light described in , the three-dimensional refractive index distribution can also be measured from multiple two-dimensional images measured while directly translating the cell 401 in the optical axis direction, instead of rotating the incident light.

[0076] In addition, the method of measuring cells 401 can be to place cells 401 on a slide glass at a low concentration in vitro, or to form cells 401 on a slide glass at a high concentration in vitro in a single layer or multiple layers, or to cut a biological tissue slide into a tissue slide with a thickness of 5 microns to 200 microns, or to place cells 401 on a multi-well plate in vitro, or for high-throughput screening, the above cells are in the form of passing through a microfluidic channel in vitro.

[0077] The light source used can utilize a single-wavelength laser. Alternatively, a low-coherence light source with a wavelength width (full-width half-maximum of the spectral bandwidth) of 1 nm to 100 nm can be used. Furthermore, by utilizing a combination of lasers or low-coherence light sources with multiple central wavelengths to measure the three-dimensional refractive index at each central wavelength, a greater amount of information can be used to differentiate cells 401.

[0078] Figure 5 A diagram illustrating a regularization method for three-dimensional tomography using a convolutional neural network according to an embodiment.

[0079] Can be used Figures 4a to 4c The method described in was used to measure the 3D refractive index images of the sorted cells. Figure 5The regularized tomographic image 530 corresponding to each image can be obtained by regularization such as total variation, and a paired dataset of "original tomographic image 510-regularized tomographic image 530" is formed. At this time, after measuring a large number (> 100) of test pieces for each type, a deep learning algorithm or a convolutional neural network algorithm 520 can be used to learn the complex nonlinear relationship between the paired datasets, thereby extracting specific features. Among them, the original tomographic image 510 can be represented as a three-dimensional refractive index image 511 or a spatial spectrum 512, and the regularized tomographic image 530 can also be represented as a three-dimensional refractive index image 531 or a spatial spectrum 532.

[0080] Specifically, the convolutional neural network algorithm 520 can be used based on the measured three-dimensional refractive index information. At this time, the information input to the convolutional neural network algorithm 520 is the original tomographic image 510, which is the three-dimensional refractive index information of each cell, and the predicted value generated as a result of machine learning becomes a regularized tomographic image 530, which is regularized three-dimensional refractive index information used to fill the spatial frequency information (k z ) that could not be physically measured.

[0081] For three-dimensional refractive index tomographic image data (i.e., original tomographic image) 510 of cells, specific feature extraction is sequentially and stepwise applied by convolution and subsampling in the contraction path 521, and in the expanding path 522, the same size of regularized three-dimensional refractive index information as the input value is output again by sequentially and stepwise applying convolution and subsampling, i.e., the regularized tomographic image 530 is output.

[0082] At this time, if the refractive index information composed of three-dimensional forms is n(x, y, z), specific features can be extracted directly in the form of a three-dimensional matrix, or the location (x, y, z) can be transformed into a spatial frequency location (k x , k y , k z ) using Fourier transform.

[0083] The specific features of the cell type obtained by the above method can be applied to the three-dimensional refractive index distribution of a specific cell to perform regularization. At this time, the efficiency of deep learning can be improved by using the real part of the refractive index related to the deceleration of light and the imaginary part of the refractive index related to light absorption in the substance. Spatial frequency information can also improve the efficiency of deep learning by inputting values using algorithms.

[0084] According to various embodiments, three-dimensional tomographic images can be rapidly regularized in seconds. This technique, unlike existing regularization methods using iterative optimization based on gradient descent that takes several minutes to several hours, can achieve regularization in seconds. Thus, a large number of three-dimensional tomographic images can be regularized in real time.

[0085] Although the present application is explained by applying to optical tomography techniques for measuring three-dimensional refractive index distribution, it can be applied to a wide range of tomography techniques such as x-ray, electron microscopy, computed tomography (CT), magnetic resonance imaging (MRI), etc. Then, a fast regularization method can be implemented by deep learning using a pair of physically measured three-dimensional tomographic images and tomographic images regularized by existing algorithms. Also, the deep learning algorithm that learns the regularization algorithm variable initially found by the user can achieve fast regularization without additional variable search or optimization when accepting new tomographic image input values.

[0086] Optical diffraction tomography (ODT) can visualize biochemical phenomena in a non-destructive manner at the nanometer level by measuring a three-dimensional refractive index (RI) map of a specimen. One major drawback of optical diffraction tomography (ODT) is poor axial resolution due to limited access to 3D light transport capabilities. These missing cone problems can be solved by using regularization algorithms using priority information such as non-negativity and specimen smoothness. However, it cannot be visualized in real time due to iterative characteristics and parameter dependency.

[0087] Therefore, in an embodiment, a machine learning algorithm such as a deep neural network can be provided, and a regularization method of three-dimensional tomography using the machine learning algorithm can rapidly improve the resolution of a 3D refractive index map.

[0088] Here, a 3D-based convolutional neural network can learn a transformation between two tomographic image domains through a paired data set (regularized tomographic images that improve resolution through original tomographic images / iterative total variation algorithms).

[0089] Figure 6 FIG. 1 shows an optical diffraction tomography (ODT) and regularization flow using a machine learning algorithm according to an embodiment.

[0090] Referring to Figure 6 , an optical diffraction tomography and regularization flow using a 3D deep neural network as a machine learning algorithm and an existing iterative total variation (TV) algorithm are shown. The optical diffraction tomography uses a scattering field of microbeads at multiple angles to reconstruct a 3D refractive index (RI) by Fourier diffraction theory.

[0091] The regularization method of three-dimensional tomography using a machine learning algorithm according to an embodiment regularizes the original tomographic image 610 by a machine learning algorithm, and the regularized tomographic image 620 can be regularized in 5 seconds. In contrast, the optimization of the existing iterative total variation algorithm 630 takes about 1 minute.

[0092] Figure 7 FIG. 2 shows a structure of a machine learning algorithm according to an embodiment.

[0093] Referring to Figure 7 The learned 3D deep neural network can regularize a tomographic image block (64x64x64) and connect all the regularized blocks with the entire tomographic image in a few seconds. Four units (reduction, encoder normal, decoder normal, and expansion) are used alternatively in a downsampling and upsampling path. Each unit is composed of a 3D convolution operation (64x64x64), a skip connection, concatenation, average pooling (3x3x3), and an expansive convolution (3x3x3, rate=2 or 3). A standard convolution is used for the reduction unit and the encoder normal unit in the downsampling path, and an expansive convolution is used for the decoder normal unit and the expansion unit in the upsampling path to preserve feature information and effectively expand the feature dimension.

[0094] The above-described devices can be implemented as hardware components, software components, and / or combinations of hardware components and software components. For example, the devices and components illustrated in the embodiments can be implemented using one or more general-purpose computers, special-purpose computers, microprocessors, arithmetic logic units (ALUs), digital signal processors (DSPs), microcontrollers, field programmable gate arrays (FPGAs), programmable logic units (PLUs), microprocessors, or any other devices that can execute and respond to instructions, as well as combinations of the foregoing. The processing devices can execute an operating system (OS) and one or more software applications running on the operating system. Also, the processing devices can access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing devices are illustrated as using one element, but the person of ordinary skill in the art will understand that the processing devices include a plurality of processing elements and / or a plurality of types of processing elements. For example, the processing devices can include a plurality of processors or include one processor and one controller. Also, other processing configurations, such as parallel processors, are possible.

[0095] The software can include a computer program, code, instructions, or a combination of one or more of the foregoing, and can configure the processing devices to operate as desired, or to command the processing devices independently or collectively. The software and / or data can embody in any type of machine, component, physical device, virtual device, computer storage medium, or means, so as to be interpreted by or to provide instructions or data to the processing devices. The software can be distributed over a networked computer system and stored or executed in a distributed manner. The software and data can be stored in one or more computer-readable recording media.

[0096] The method according to various embodiments can be implemented in the form of program instructions, which can be executed through various computer means, and recorded in computer readable media. The above computer readable media can include a single or plural program instructions, data files, data structures, etc. The above program instructions recorded on the media can be specially designed and configured for the present embodiments, or can be known and available to those skilled in the computer software field. Examples of the computer readable media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, etc. Examples of the program instructions include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by computers using an interpreter, etc.

[0097] As described above, although the description has been made with reference to limited embodiments and drawings, various modifications and improvements can be made by those skilled in the art according to the above description. For example, the described technology can be performed in a different order from the described method, and / or the components of the described system, structure, device, circuit, etc. can be combined or integrated in a different form from the described method, or replaced or substituted by other components or equivalent technical solutions, and a proper result can be achieved.

[0098] Therefore, other embodiments, other examples, and technical solutions equivalent to the claimed scope of the invention also belong to the claimed scope of the invention described hereinafter.

Claims

1. A regularization method for three-dimensional tomographic imaging with machine learning algorithms, characterized in that, comprises: a step of obtaining a raw tomographic image of cells by measuring a three-dimensional tomographic image of the cells; a step of obtaining a regularized tomographic image using a regularization algorithm; a step of learning a relationship between the raw tomographic image and the regularized tomographic image by a machine learning algorithm before learning; and a step of regularizing a three-dimensional tomographic image of measured cells using the machine learning algorithm after learning, in the step of learning a relationship between the raw tomographic image and the regularized tomographic image by the machine learning algorithm before learning, a non-linear relationship between the raw tomographic image and the regularized tomographic image is learned by the machine learning algorithm before learning to extract a specific feature of a cell type, i.e., a refractive index feature, in the step of regularizing a three-dimensional tomographic image of measured cells using the machine learning algorithm after learning, the three-dimensional tomographic image of measured cells is regularized using the extracted refractive index feature of the cell type and applied to a three-dimensional refractive index distribution of a specific cell, and the raw tomographic image of measured cells is input to a convolutional neural network algorithm to output a regularized tomographic image, the regularized tomographic image is applied to distinguish the type of the cell, the step of regularizing a three-dimensional tomographic image of measured cells using the machine learning algorithm after learning comprises: a contraction step of extracting a specific feature from the raw tomographic image data of cells by sequentially and gradually applying convolution and subsampling; and a dilation step of outputting again the regularized tomographic image data of the same size as the input value by sequentially and gradually applying convolution and subsampling. 2.The regularization method of three-dimensional tomographic imaging using a machine learning algorithm according to claim 1, wherein, in the step of obtaining a raw tomographic image of cells by measuring a three-dimensional tomographic image of the cells, a three-dimensional refractive index image of cells is measured using a method of rotating incident light irradiated from a plurality of angles, or a three-dimensional refractive index image of cells is obtained by using a plurality of two-dimensional images measured while rotating or translating the cells. 3.The regularization method of three-dimensional tomographic imaging using a machine learning algorithm according to claim 1, wherein, in the step of obtaining a regularized tomographic image using a regularization algorithm, a regularized refractive index image corresponding to each image is obtained by total variation regularization to form a paired data set of the raw tomographic image and the regularized tomographic image. 4.A regularization apparatus of three-dimensional tomographic imaging using a machine learning algorithm, comprising: a three-dimensional tomographic image measurement unit for obtaining a raw tomographic image of cells by measuring a three-dimensional tomographic image of the cells; a regularization algorithm for obtaining a regularized tomographic image; a machine learning algorithm for learning a non-linear relationship between the raw tomographic image and the regularized tomographic image to extract a specific feature of a cell type, i.e., a refractive index feature; and a regularization unit for regularizing a three-dimensional tomographic image of measured cells using the machine learning algorithm after learning. ​ ​ ​ ​ ​ ​ ​ ​ The above regularization unit is configured to apply the extracted refractive index characteristics of the above cell types to the three-dimensional refractive index distribution of a specific cell to regularize the three-dimensional tomographic image of the measured cell, and input the above measured raw tomographic image to a convolutional neural network algorithm to output a regularized tomographic image, and apply the above regularized tomographic image to distinguish the types of the above cells, The above regularization unit is configured to extract specific features from the above raw tomographic image data of the cell by sequential step-by-step application of convolution and subsampling, and then output the above regularized tomographic image data of the same size as the input value again by sequential step-by-step application of convolution and subsampling.

Images