Method for reconstructing phase image based on unsupervised learning using gabor hologram and system therefor

Abstract

The present invention relates to a method for reconstructing a phase image based on unsupervised learning using a Gabor hologram and a system therefor. According to the present invention, the method for reconstructing a phase image based on unsupervised learning using a Gabor hologram comprises the steps of: receiving, by an input unit, a digital hologram as an input; and reconstructing, by a reconstruction unit, a phase image by applying the digital hologram to a pre-trained reconstruction model, wherein the digital hologram may comprise an original phase image and an original inline hologram.

Description

Unsupervised learning-based topological image reconstruction method and system using Gabor holograms

[0001] The present invention relates to a method and system for restoring a phase image based on unsupervised learning using a Gabor hologram, and more specifically, to a method and system for restoring a phase image based on unsupervised learning using a Gabor hologram that restores a phase image by applying an unsupervised diffusion model to a Gabor single-shot digital hologram.

[0002] Cells are transparent or translucent, so imaging is performed using off-axis quantitative phase digital holography. Off-axis quantitative phase digital holography stores unlabeled holographic images of living cells at low radiation, and single holograms are processed to reconstruct a 3D image corresponding to the phase distribution and volume of the cells.

[0003] There is a limitation in that precise control of optical elements is required because the reference wave and the object wave must be aligned before capturing a hologram of a biological sample, such as a cell.

[0004] To compensate for this, Garbor holographic microscopy is used, but there is a problem where in-focus images and out-of-focus images overlap.

[0005] In addition, reconstructing phase images from Gabor hologram images through supervised learning is labor-intensive and costly because a large number of pair datasets must be acquired, and reconstructing phase images from Gabor holograms through unsupervised learning is difficult to judge the accuracy of the results because there are no labels to verify.

[0006] Therefore, there is a need for a technology to reconstruct phase images using unsupervised learning-based Gabor single-shot digital holograms.

[0007] The technology forming the background of the present invention is described in Korean Published Patent No. 10-2024-0119854 (published August 6, 2024).

[0008] As such, the present invention aims to provide an unsupervised learning-based phase image restoration method and system using a Gabor hologram, which restores a phase image by applying an unsupervised diffusion model to a Gabor single-shot digital hologram.

[0009] According to an embodiment of the present invention for achieving such technical challenges, a method for restoring a phase image based on unsupervised learning using a Gabor hologram comprises: a step in which an input unit receives a digital hologram; and a step in which a restoration unit applies the digital hologram to a previously learned restoration model to restore a phase image, wherein the digital hologram may include an original phase image and an original inline hologram.

[0010] The above restoration model may include a phase image generation module that generates a phase image by applying a first forward process and a first reverse process to the digital hologram; and a hologram generation module that generates an inline hologram by applying a second forward process and a second reverse process to the digital hologram.

[0011] The first forward process above generates the original phase image and the original inline hologram with added noise by repeatedly adding noise to the original phase image and the original inline hologram at each time step for time steps from 0 to T, and T is a natural number greater than or equal to 2.

[0012] The above first reverse process can generate a noise-removed phase image by inputting the original inline hologram and random noise into a first generator at time T, and generate a noise-removed phase image in a subsequent step by applying posterior sampling to the noise-removed phase image.

[0013] The above first reverse process can generate a noise-removed phase image by inputting the original inline hologram and the noise-removed phase image of the subsequent step generated in the previous time step (T to 2) into the first generator at the time step T-1 to 1, and generate a noise-removed phase image of the subsequent step of the current time step by applying post-sampling to the noise-removed phase image.

[0014] The second forward process above generates the original phase image and the original inline hologram with added noise by repeatedly adding noise to the original phase image and the original inline hologram at each time step for time steps from 0 to T, and T is a natural number greater than or equal to 2.

[0015] The second reverse process described above can generate a noise-removed inline hologram by inputting the original phase image and random noise into a second generator at time T, and generate a noise-removed inline hologram in a subsequent step by applying post-sampling to the noise-removed inline hologram.

[0016] The second reverse process above can generate a noise-removed inline hologram by inputting the original phase image and the noise-removed inline hologram of the subsequent step generated in the previous time step (T to 2) into the second generator at the time step T-1 to 1, and generate a noise-removed inline hologram of the subsequent step of the current time step by applying post-sampling to the noise-removed inline hologram.

[0017] In an unsupervised learning-based phase image restoration system using a Gabor hologram according to another embodiment of the present invention, the system comprises: an input unit that receives a digital hologram; and a restoration unit that applies the digital hologram to a pre-trained restoration model to restore a phase image, wherein the digital hologram may include an original phase image and an original inline hologram.

[0018] As such, according to the present invention, a high-quality quantitative phase image can be obtained cost-effectively from a Gabor single-shot digital hologram.

[0019] In addition, biological samples can be analyzed with high accuracy in a miniaturized holographic setup.

[0020] FIG. 1 is a drawing illustrating a phase image restoration system according to one embodiment of the present invention.

[0021] FIG. 2 is a flowchart of an unsupervised learning-based phase image restoration method using a Gabor hologram according to another embodiment of the present invention.

[0022] FIG. 3 is a diagram illustrating the process of a phase image generation module according to another embodiment of the present invention.

[0023] FIG. 4 is a diagram illustrating an inference process for restoring a phase image using a first generator according to another embodiment of the present invention.

[0024] FIG. 5 is a diagram illustrating the process of a hologram generation module according to another embodiment of the present invention.

[0025] FIG. 6 is a drawing illustrating an example of photographing red blood cells and cancer cells using an inline holographic microscope and a non-optical holographic microscope according to another embodiment of the present invention.

[0026] FIG. 7 is a diagram illustrating the result of a reconstruction unit, learned using only red blood cell holograms according to another embodiment of the present invention, generating a final phase image.

[0027] FIG. 8 is a diagram illustrating the result of a reconstruction unit, learned using only cancer cell holograms according to another embodiment of the present invention, generating a final phase image.

[0028] FIG. 9 is a diagram illustrating quantitative cell results of a final phase image generated according to another embodiment of the present invention.

[0029] FIG. 10 is a diagram comparing the performance of a conventional model based on unsupervised learning and a restoration model learned according to another embodiment of the present invention.

[0030] FIG. 11 is a diagram comparing the results of a supervised learning model, a conventional unsupervised learning model, and a restoration model learned according to another embodiment of the present invention.

[0031] Preferred embodiments according to the present invention will be described in detail below with reference to the attached drawings. In this process, the thickness of lines or the size of components shown in the drawings may be exaggerated for clarity and convenience of explanation.

[0032] Furthermore, the terms described below are defined in consideration of their functions within the present invention, and these may vary depending on the intent or practice of the user or operator. Therefore, the definitions of these terms should be based on the content throughout this specification.

[0033] In the embodiment described below, an unsupervised learning-based phase image restoration system (100) using a Gabor hologram is performed by a computing device and can restore a phase image using an unsupervised learning restoration model using an in-line hologram and phase image used in training (unpaired) that are not matched in pairs.

[0034] FIG. 1 is a drawing illustrating a phase image restoration system according to one embodiment of the present invention.

[0035] As illustrated in FIG. 1, the phase image restoration system (100) may include an input unit (110) that receives a digital hologram and a restoration unit (120) that restores a phase image by applying the input digital hologram to a pre-trained restoration model. At this time, the digital hologram is a Gabor single-shot digital hologram and includes an original inline hologram and an original phase image.

[0036] Here, the restoration model is an unsupervised learning-based diffusion model using a generative model (e.g., Generative adversarial networks (GAN)), and includes a phase image generation module (121) that generates a phase image from an input digital hologram and a hologram generation module (122) that generates an inline hologram from an input digital hologram.

[0037] Additionally, the restoration unit (120) can restore the phase image based on the generated first phase image and the first inline hologram to generate a final phase image.

[0038] First, the phase image generation module (121) can generate a phase image by applying a first forward process and a first reverse process to the input digital hologram. At this time, the phase image is an off-axis phase image.

[0039] Next, the hologram generation module (122) can generate an inline hologram by applying a second forward process and a second reverse process to the input digital hologram. At this time, the restoration unit (120) can perform the phase image generation module (121) and the hologram generation module (122) simultaneously.

[0040] Here, the restoration unit (120) can add noise to the original phase image of the digital hologram and the original inline hologram using the following [Equation 1] through the first and second forward processes.

[0041]

[0042] Here, t is the time step And, is the original phase image or original inline hologram with noise added at step t-1, (i,j) is the coordinates of the original phase image or original inline hologram, ε is noise sampled independently of the Gaussian distribution N(0,1), and β is a noise schedule in which noise is added at each time step.

[0043] In addition, the restoration unit (120) can adopt the discretization of continuous-time extension as shown in [Equation 2] below to maintain the noise schedule at a preset level regardless of the number of diffusion steps and perform noise diffusion quickly.

[0044]

[0045] Here, class It controls the progression of noise spreading according to a preset exponential schedule. In another embodiment of the present invention described below, T is 4, and is 0.1, and Noise spreading was performed by setting it to 20.

[0046] Additionally, the restoration unit (120) can generate a phase image or an inline hologram by repeatedly removing noise from the original phase image with added noise and the original inline hologram from time step T to 0 using [Equation 3] below through the first and second reverse processes.

[0047]

[0048] Here, t is the time step, and is a first generator that generates a noise-removed phase image using an inline hologram and a phase image generated in the previous step as input, and is a second generator that generates a noise-removed inline hologram using the phase image and inline hologram generated in the previous step as input, and and is the original phase image and original inline hologram with randomly added noise, h is the original inline hologram, and p is the original phase image.

[0049] In addition, the restoration unit (120) uses the following [Equation 4] in the first and second reverse processes to remove noise from the previous time step through posterior sampling of the digital hologram ( ) can be derived.

[0050]

[0051] Here, is a digital hologram generated in the previous time step (t-1), which is the original phase image with added noise generated in the previous time step ( ) and original inline hologram( Includes ), is a noise-removed phase image ( ) and inline hologram( Includes ), , , is noise sampled in the same way as the forward process, except for the final noise removal step.

[0052] Furthermore, the restoration unit (120) can restore the digital hologram using the generated inline hologram and phase image.

[0053] Hereinafter, a phase image restoration method performed by an unsupervised learning-based phase image restoration system (100) using a Gabor hologram is described in more detail using FIGS. 2 to 11.

[0054] FIG. 2 is a flowchart of an unsupervised learning-based phase image restoration method using a Gabor hologram according to another embodiment of the present invention.

[0055] As shown in FIG. 2, the input unit (110) can receive a digital hologram (S210).

[0056] Next, the restoration unit (120) can restore the phase image by applying the input digital hologram to a previously learned restoration model (S220).

[0057] Specifically, the phase image generation module (121) can generate a phase image by applying a first forward process and a first reverse process to an input digital hologram (S221). At this time, the phase image is a non-optical phase image.

[0058] Specifically, the phase image generation module (121) can generate a phase image by applying a first forward process that adds noise to an original phase image and an original inline hologram that are not matched as pairs of input digital holograms, and a first reverse process that generates a phase image from the original phase image (p') with added noise and the original inline hologram (h).

[0059] FIG. 3 is a diagram illustrating the process of a phase image generation module according to another embodiment of the present invention.

[0060] As illustrated in FIG. 3, the phase image generation module (121) repeatedly adds noise (ε) to the original phase image and the original inline hologram at each time step through a first forward process to create an original phase image with added noise ( ) and the original inline hologram with added noise( Can generate ).

[0061] At time step T of the first reverse process, the phase image generation module (121) generates the original inline hologram (h) and random noise from the first generator ( Noise-removed phase image input into ) Generates ) and the generated noise-removed phase image( Phase image with noise removed in subsequent step (T-1) by applying post-sampling to ) Can generate ).

[0062] Also, at each time step (T,…,1) of the first reverse process, the phase image generation module (121) generates the original phase image (p') and the generated noise-removed phase image ( ) is the 2-1 discriminator( It is possible to determine whether the input phase image is the original phase image (Real) or the generated noise-removed phase image (False) by inputting it into ). At this time, the 2-1 discriminator was trained to distinguish whether the input phase image is the original phase image or the generated noise-removed phase image.

[0063] According to one embodiment of the present invention, the 2-1 discriminator can determine whether the phase image is the original phase image or the generated phase image from which noise has been removed.

[0064] Also, at time step T of the first reverse process, the phase image generation module (121) generates the original phase image with added noise at time step T ( ) and the original phase image with added noise at the T-1 time step ( The phase image generated through ) and the generated post-sampling and random noise are used as the 2-2 discriminator ( The input phase image is input into ) and is the original phase image at time step T-1 ( Phase image with generated noise removed (Real) Phase image generated through post-sampling on ) It can determine whether ) is (False). In this case, the 2-2 discriminator was trained to distinguish whether the input phase image is the original phase image with added noise or the phase image generated through post-sampling.

[0065] According to one embodiment of the present invention, the 2-2 discriminator can determine whether the phase image is an original phase image with added noise or a phase image generated through post-sampling of a generated phase image with noise removed.

[0066] Also, in the time step T-1 to 1 of the first reverse process, the phase image generation module (121) generates the original inline hologram (h) and the noise-removed phase image ( generated in the previous time step (T,…,2) ,… , ) is the first constructor( Noise-removed phase image input into ) Generates ) and the generated noise-removed phase image( Phase image with noise from subsequent steps (T-2,…,0) removed by applying post-sampling to ) ,… , Can generate ).

[0067] Also, in the time step T-1 to 1 of the first reverse process, the phase image generation module (121) generates the original phase image with added noise at the corresponding time step ( ,… , Original phase image with noise added from ) and subsequent steps (T-2,…,0) ( ,… ,p') and the noise-removed phase image of the generated corresponding time step ( ,… , ) and the noise-removed phase image of the previous time step (T,…,2) ,… , ) is the 2-2 discriminator( By inputting into ), it is possible to determine whether the input phase image is an original phase image with noise added to the original phase image of the subsequent stage, or a phase image generated through post-sampling of a generated phase image with noise removed. At this time, the 2-2 discriminator is trained to distinguish whether the input phase image is an original phase image with noise added or a phase image generated through post-sampling.

[0068] At each time step (T,…,1), the 2-1 discriminator and the 2-2 discriminator can be made to generate more precisely similar phase images using the adversarial loss calculated through [Equation 5] below.

[0069]

[0070] Here, is the adversarial loss of the 2-1 discriminator at time step t, and D(p') is the value of the original phase image identified by the 2-1 discriminator, and is the value determined by the 2-1 discriminator of the generated noise-removed phase image, and is the adversarial loss of the 2-2 discriminator at time step t, and is a value determined by the 2-2 discriminator of the original phase image of the current time step (t) and the original phase image of the subsequent step (t-1), and is the value determined by the 2-2 discriminator of the phase image containing noise used as input in the previous time step (t) and the phase image with one step of noise removed generated through post-sampling, and is the adversarial loss for the phase image at the current time step (t).

[0071] Additionally, the phase image generation module (121) has an inline hologram with added noise at time step T ( ) and the generated noise-removed phase image( ) is the second constructor( The first inline hologram with noise removed by inputting into ) Can generate ).

[0072] At this time, the phase image generation module (121) can generate an inline hologram that is more precisely similar to the original inline hologram by using the cycle-cosistency loss calculated through [Equation 6] below.

[0073]

[0074] Here, is the cyclic coherence loss for the inline hologram, and h is the original inline hologram, and is the first inline hologram with generated noise removed.

[0075] Additionally, the phase image generation module (121) has an inline hologram with added noise at time step T ( ) and the original inline hologram(h) to the second constructor( Input into ) to the second inline hologram ( Can generate ).

[0076] At this time, the phase image generation module (121) can preserve the essential features of the input original inline hologram using the identity loss calculated through [Equation 7] below.

[0077]

[0078] Here, is the identity loss for the inline hologram, and h is the input original inline hologram, and is an inline hologram with added noise at the current time point (T) The original inline hologram(h) in the second constructor( It is a second inline hologram generated by inputting into ).

[0079] FIG. 4 is a diagram illustrating an inference process for restoring a phase image using a first generator according to another embodiment of the present invention.

[0080] As illustrated in FIG. 4, the phase image generation module (121) is random noise ( A noise-removed phase image () by applying the first generator to the original inline hologram (h) The process of generating ) can be repeated for a time step (T) to finally generate a noise-removed phase image.

[0081] Next, the hologram generation module (122) can generate an inline hologram by applying a second forward process and a second reverse process to the input digital hologram (S222).

[0082] Specifically, the hologram generation module (122) can generate an inline hologram by applying a second forward process that adds noise to an original phase image and an original inline hologram that are not matched as pairs of input digital holograms, and a second reverse process that generates an inline hologram from the original inline hologram (h') with added noise and the original phase image (p).

[0083] FIG. 5 is a diagram illustrating the process of a hologram generation module according to another embodiment of the present invention.

[0084] As illustrated in FIG. 5, the hologram generation module (122) repeatedly adds noise (ε) to the original inline hologram and the original phase image at each time step through a second forward process to create an original phase image with added noise ( ) and the original inline hologram with added noise( Can generate ).

[0085] In the time step T of the second reverse process, the hologram generation module (122) generates the original phase image (p) and random noise from the second generator ( Noise-removed inline hologram input into ) Generates ) and the generated noise-removed inline hologram( Inline hologram with noise from subsequent step (T-1) removed by applying post-sampling to ) Can generate ).

[0086] Also, at each time step (T,…,1) of the second reverse process, the hologram generation module (122) generates the original inline hologram (h') and the generated noise-removed inline hologram ( ) is the 1-1 discriminator( Input into ) to determine whether the input inline hologram is the original inline hologram (Real) or the generated noise-removed inline hologram ( It can determine whether ) is (False). In this case, the 1-1 discriminator was trained to distinguish whether the input inline hologram is the original inline hologram or the generated noise-removed inline hologram image.

[0087] According to one embodiment of the present invention, the 1-1 discriminator can determine whether an inline hologram is an original inline hologram or an inline hologram from which generated noise has been removed.

[0088] Also, at time T of the second reverse process, the hologram generation module (122) generates the original inline hologram with added noise at time T ( ) and the original inline hologram with added noise at the T-1 time step ( ) and the inline hologram and random noise generated through the generated post-sampling are used as the 1st-2nd discriminator ( Input into ) so that the input inline hologram is the original inline hologram at time step T-1 ( Inline hologram with generated noise removed (Real) Inline hologram generated through post-sampling on ) It can determine whether ) is (False). In this case, the first and second discriminators were trained to distinguish whether the input inline hologram is the original inline hologram with added noise or an inline hologram generated through post-sampling.

[0089] According to one embodiment of the present invention, the first and second discriminators can determine whether an inline hologram is an original inline hologram with added noise or an inline hologram generated through post-sampling of an inline hologram from which noise has been removed.

[0090] Also, in the time step T-1 to 1 of the second reverse process, the hologram generation module (122) generates an original phase image (p) and an inline hologram with noise removed generated in the previous time step (T,…,2). ,… , ) is the second constructor( Noise-removed inline hologram input into ) Generates ) and the generated noise-removed inline hologram( Inline hologram with noise removed in subsequent steps (T-2,…,0) by applying post-sampling to ) ,… , Can generate ).

[0091] Also, in the time step T-1 to 1 of the second reverse process, the hologram generation module (122) generates the original inline hologram with added noise at the corresponding time step ( ,… , The original inline hologram with noise added from ) and subsequent steps (T-2,…,0) ( ,… ,h') and the noise-removed inline hologram of the generated corresponding time step ( ,… , ) and the inline hologram with noise removed from the previous time step (T,…,2) ,… , ) is the 1st-2nd discriminator( By inputting into ), it is possible to determine whether the input inline hologram is an original inline hologram with noise added to the original inline hologram of the subsequent stage, or an inline hologram generated through post-sampling of a generated inline hologram with noise removed. At this time, the 1st and 2nd discriminators are trained to distinguish whether the input inline hologram is an original inline hologram with noise added or an inline hologram generated through post-sampling.

[0092] At each time step (T,…,1), the 1-1 discriminator and the 1-2 discriminator can be made to generate more precisely similar inline holograms using the adversarial loss calculated through [Equation 8] below.

[0093]

[0094] Here, is the adversarial loss of the 1-1 discriminator at time step t, and D(h') is the value of the original inline hologram identified by the 1-1 discriminator, and is the value determined by the 1-1 discriminator of the generated noise-removed inline hologram, and is the adversarial loss of the 1st and 2nd discriminators at time step t, and is the value determined by the first and second discriminators for the original inline hologram of the current time step (t) and the original inline hologram of the subsequent step (t-1), and is the value determined by the first and second discriminators for the inline hologram containing noise used as input in the previous time step (t) and the inline hologram with one step of noise removed generated through post-sampling, and is the adversarial loss for the inline hologram at the current time step (t).

[0095] In addition, the hologram generation module (122) has a phase image with noise added at time step T ( Noise-removed inline hologram generated in ) ) is the first constructor( The first phase image with noise removed by inputting into ) Can generate ).

[0096] At this time, the hologram generation module (122) can generate a phase image that is more precisely similar to the original phase image by using the cyclic coherence loss calculated through [Equation 9] below.

[0097]

[0098] Here, is the cyclic coherence loss for the phase image, p is the original phase image, and is the first phase image with the generated noise removed.

[0099] In addition, the hologram generation module (122) has a phase image with noise added at time step T ( The original phase image (p) in the first generator ( Input into ) to the second phase image ( Can generate ).

[0100] At this time, the hologram generation module (122) can preserve the essential features of the input original phase image using the identity loss calculated through [Equation 10] below.

[0101]

[0102] Here, is the identity loss for the phase image, and p is the input original phase image, and is a phase image with added noise at the current time point (T) The original phase image (p) in the first generator ( It is a second inline hologram generated by inputting into ).

[0103] Additionally, the first and second generators are built based on a U-Net comprising five downsampling blocks, one channel-specific self-attention block, and five upsampling blocks, each block comprising two residual sub-blocks and a convolutional layer, and the residual sub-blocks include time embeddings generated by projecting a 128-dimensional sinusoidal position encoding through a multilayer perceptron.

[0104] According to one embodiment of the present invention, the convolutional layer of the downsampling block reduces the resolution of the feature map by half and doubles the channel dimension, the convolutional layer of the upsampling block doubles the resolution of the feature map and reduces the channel dimension by half, and the remaining sub-block receives a time embedding obtained by projecting a 128-dimensional sine wave position encoding through a 2-layer multilayer perceptron and receives a 256-dimensional random latent value from a 3-layer multilayer perceptron and modulates the feature map through adaptive normalization.

[0105] In addition, the first and second generators can be trained using the total loss calculated using [Equation 11] below.

[0106]

[0107] Here, is the total loss of the first or second generator, and is a pre-set adversarial loss weight (e.g., 1), and is an adversarial loss of the first or second generator, and is a pre-set cyclic consistency loss weight (e.g., 10), and is the cyclic consistency loss of the first or second generator, and is a pre-set identity loss weight (e.g., 10), and is the identity loss of the first or second generator.

[0108] Additionally, the 1-1 and 2-1 discriminators each comprise five blocks consisting of a convolutional layer, batch normalization, and a rectified linear unit (e.g., LeakyReLU) activation, and the 1-2 and 2-2 discriminators each comprise six blocks consisting of two convolutional layers, batch normalization, and a rectified linear unit (e.g., LeakyReLU) activation.

[0109] Below, an unsupervised learning-based phase image restoration method using Gabor holograms is explained in more detail using FIGS. 6 to 9.

[0110] Experimental method

[0111] <1-1> Dataset

[0112] Inline holographic microscopes and non-optical holographic microscopes were used to image red blood cells and cancer cells using a digital holographic microscope.

[0113] FIG. 6 is a drawing illustrating an example of photographing red blood cells and cancer cells using an inline holographic microscope and a non-optical holographic microscope according to another embodiment of the present invention.

[0114] As shown in Fig. 6, the non-optical holographic microscope can record using non-optical Mach-Zehnder interferometry and reconstruct the phase image through numerical calculation of Fresnel diffraction.

[0115] In addition, the inline holographic microscope blocked the reference wave in a non-optical Mach-Zehnder interferometer and recorded using only the object wave.

[0116] The size of the hologram and phase image recorded through Fig. 6 is 900×900 pxl, with an area of ​​129.68㎛×129.68㎛ for red blood cells and an area of ​​256.38㎛×256.38㎛ for cancer cells.

[0117] In addition, a dataset containing 3,600 holograms of red blood cells and 3,000 holograms of cancer cells was generated by applying data augmentation to 900 holograms of red blood cells and 750 holograms of cancer cells, and 200 images were selected from each dataset of red blood cells and cancer cells to be used as a test dataset, and the remainder were used as a training dataset.

[0118] <1-2> Standard Evaluation Indicators

[0119] The Structural Similarity Index (SSIM) measure was used to evaluate the quality of image deformation.

[0120] The structural similarity index represents the perceptual distance between the generated image and the reference image, and the closer the structural similarity index is to 1, the higher the luminance, contrast, and structural similarity between the two images.

[0121] Experimental results

[0122] <2-1> Results of a restoration model trained on red blood cells or cancer cells

[0123] FIG. 7 is a diagram illustrating the result of a reconstruction unit, learned using only red blood cell holograms according to another embodiment of the present invention, generating a final phase image.

[0124] Figure 7 (a) is a phase image of a red blood cell, Figure 7 (b) is a 3D surface plot of multiple red blood cells, Figure 7 (c) is an enlarged image of one of the red blood cells indicated in Figure 7 (b), and Figure 7 (d) shows the line profile of the red blood cell in Figure 7 (c).

[0125] As shown in Figure 7, it can be seen that the final phase image generated by the learned reconstruction model using only the hologram of red blood cells is very similar to the reference image (Ground truth) located on the left.

[0126] FIG. 8 is a diagram illustrating the result of a reconstruction unit, learned using only cancer cell holograms according to another embodiment of the present invention, generating a final phase image.

[0127] Figure 8(a) is a phase image of a cancer cell, Figure 8(b) is a 3D surface plot of multiple cancer cells, Figure 8(c) is an enlarged image of one of the cancer cells indicated in Figure 8(b), and Figure 8(d) shows the line profile of the cancer cell in Figure 8(c).

[0128] As shown in Fig. 8, the reconstruction model trained using only cancer cell holograms has lower performance than the reconstruction model trained using only red blood cell holograms, but the final phase image generated by the reconstruction unit is very similar to the reference image (Ground truth) located on the left.

[0129] <2-2> Single-cell analysis of the reconstructed phase image of the reconstructed model

[0130] FIG. 9 is a diagram illustrating quantitative cell results of a final phase image generated according to another embodiment of the present invention.

[0131] Figure 9(a) shows the process of creating a segmented image from an input phase image, Figure 9(b) shows a single-cell analysis of red blood cells, Figure 9(c) shows a single-cell analysis of cancer cells, and in Figure 9, orange is the ground truth image and yellow is the result of a reconstruction model trained only on red blood cells or cancer cells.

[0132] As shown in Figure 9, in the case of red blood cells, the projection area calculated from the restored phase image is similar to the actual projection area because the shape of each cell is similar, and in the case of cancer cells, the restoration model shows high similarity to the dry mass, indicating that the accuracy of the restored phase image is high.

[0133] <2-2> Comparison with Conventional Models Based on Unsupervised Learning

[0134] FIG. 10 is a diagram comparing the performance of a conventional model based on unsupervised learning and a restoration model learned according to another embodiment of the present invention.

[0135] In Figure 10, for each cell type (red blood cells and cancer cells), the first and second rows are full images of the cells, and the third row is an enlarged image of some of the cells in the second row (indicated by red squares).

[0136] Referring to FIG. 10, in the case of red blood cells, conventional models (CycleGAN, UNIT, Syndiff) have limitations in that the shape of the red blood cells is distorted, they fail to generate phase images for dark areas, or the reconstructed phase image is more similar to an inline hologram than the ground truth image due to noise in the background. However, it can be seen that the reconstructed model trained according to another embodiment of the present invention generates a phase image very similar to the ground truth image, accurately reconstructs all cell regions without adding background noise, and expresses a texture similar to the ground truth image.

[0137] In the case of cancer cells, conventional models (CycleGAN, UNIT, Syndiff) have limitations, such as failing to accurately maintain the structure of the cancer cells, deleting cells in areas where they should be present, or generating blurry images. However, it can be seen that the restoration model trained according to another embodiment of the present invention is superior to conventional models in its ability to distinguish cell regions in various cell shapes and inline holograms by generating a texture representation of the cancer cells that is very similar to the ground truth image.

[0138] [Table 1] below shows the results of evaluating the similarity between the phase image generated using the structural similarity index and the ground truth image.

[0139] Model Red Blood Cell Cancer Cell CycleGAN 0.72 0.82 UNIT 0.74 0.80 Syndiff 0.76 0.84 Proposed Model (Reconstruction Model) 0.74 0.83

[0140] Referring to Figure 10 and [Table 1], it can be seen that comparing the generated phase image with the ground truth image is important because there is a difference between evaluating by comparing the generated phase image with the ground truth image and judging by the structural similarity index.

[0141] FIG. 11 is a diagram comparing the results of a supervised learning model, a conventional unsupervised learning model, and a restoration model learned according to another embodiment of the present invention.

[0142] In Fig. 11, the upper side of each cell type (red blood cell, cancer cell) is the full phase image, and the lower side is an enlarged image of a part (red square) of the full phase image.

[0143] Referring to Figure 11, it can be seen that in the case of red blood cells, the supervised learning model and the conventional unsupervised learning model (Syndiff) retain important features of the ground truth image. In particular, it can be seen that even when the target data is limited, the topological image is restored and generated in a manner similar to that obtained by training using all target data.

[0144] Referring to Fig. 11, it can be seen that in the case of cancer cells, a supervised learning model trained using limited target data fails to properly reconstruct the topological image of the background region compared to a model trained using all target data. Additionally, while a conventional unsupervised learning model and a reconstruction model trained according to another embodiment of the present invention perform topological image reconstruction well regardless of the amount of target data, it can be seen that the reconstruction model trained according to another embodiment of the present invention expresses texture better than the conventional unsupervised learning model and is more similar to the ground truth image.

[0145] [Table 2] below is a table comparing the performance of the supervised learning model of Fig. 11, the conventional unsupervised learning model, and the restoration model using the structural similarity index.

[0146] Model Red Blood Cell Cancer Cell Supervised learning model for the entire target data 0.89 0.91 Supervised learning model for limited target data 0.87 0.80 Unsupervised learning model for the entire target data 0.76 0.84 Unsupervised learning model for limited target data 0.73 0.83 Reconstruction model for the entire target data 0.74 0.83 Reconstruction model for limited target data 0.75 0.82

[0147] Referring to [Table 2] above, it can be seen that for a reconstruction model trained according to another embodiment of the present invention, labeled phase images maintain at least excellent performance. In particular, it can be seen that a reconstruction model trained according to another embodiment of the present invention can effectively generalize to complex cell types, such as cancer cells.

[0148] Table 3 below is a diagram comparing the learning complexity of the conventional unsupervised learning model of FIG. 11 and the restoration model learned according to another embodiment of the present invention.

[0149] Learning Complexity Conventional Unsupervised Learning Model Reconstruction Model Average time required to train once 3.56 1.76 Number of model parameters 36 1.173 Memory load 139 1.71 166 8.07

[0150] Referring to [Table 3] above, it can be seen that the restoration model has higher learning efficiency by performing learning with about half the time, parameters, and memory load compared to conventional unsupervised learning models.

[0151] According to the above-described embodiment of the present invention, a high-quality quantitative phase image can be obtained cost-effectively from a Gabor single-shot digital hologram.

[0152] In addition, biological samples can be analyzed with high accuracy in a miniaturized holographic setup.

[0153] The present invention has been described with reference to the embodiments illustrated in the drawings, but this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the present invention should be determined by the technical spirit of the following claims.

[0154] [Explanation of the symbol]

[0155] 100: Unsupervised learning-based topological image reconstruction system using Gabor holograms

[0156] 110: Input section

[0157] 120: Restoration section

[0158] 121: Phase Image Generation Module 122: Hologram Generation Module

Claims

1. A step in which an input unit receives a digital hologram; and The restoration unit includes the step of restoring a phase image by applying the digital hologram to a pre-trained restoration model, and The above digital hologram is an unsupervised learning-based phase image restoration method using a Gabor hologram comprising an original phase image and an original inline hologram.

2. In Paragraph 1, The above restoration model is, A phase image generation module that generates a phase image by applying a first forward process and a first reverse process to the digital hologram; and An unsupervised learning-based phase image restoration method using a Gabor hologram comprising a hologram generation module that generates an inline hologram by applying a second forward process and a second reverse process to the digital hologram.

3. In Paragraph 2, The above first forward process is, For time steps from 0 to T, noise is repeatedly added to the original phase image and the original inline hologram at each time step to generate the original phase image with added noise and the original inline hologram with added noise, and An unsupervised learning-based topological image restoration method using a Gabor hologram where T is a natural number greater than or equal to 2.

4. In Paragraph 3, The above first reverse process is, An unsupervised learning-based phase image restoration method using a Gabor hologram, wherein the original inline hologram and random noise are input into a first generator at time T to generate a noise-removed phase image, and posterior sampling is applied to the noise-removed phase image to generate a noise-removed phase image in a subsequent step.

5. In Paragraph 4, The above first reverse process is, An unsupervised learning-based phase image restoration method using a Gabor hologram, wherein the original inline hologram and the noise-removed phase image of the subsequent step generated in the previous time step (T to 2) are input into a first generator to generate a noise-removed phase image, and post-sampling is applied to the noise-removed phase image to generate a noise-removed phase image of the subsequent step of the current time step.

6. In Paragraph 2, The above second forward process is, For time steps from 0 to T, noise is repeatedly added to the original phase image and the original inline hologram at each time step to generate the original phase image with added noise and the original inline hologram with added noise, and An unsupervised learning-based topological image restoration method using a Gabor hologram where T is a natural number greater than or equal to 2.

7. In Paragraph 6, The above second reverse process is, An unsupervised learning-based phase image restoration method using a Gabor hologram, wherein the original phase image and random noise are input into a second generator at time T to generate a noise-removed inline hologram, and post-sampling is applied to the noise-removed inline hologram to generate a noise-removed inline hologram in a subsequent step.

8. In Paragraph 7, The above second reverse process is, An unsupervised learning-based phase image restoration method using a Gabor hologram, wherein the original phase image and the noise-removed inline hologram of the subsequent step generated in the previous time step (T to 2) are input into a second generator to generate a noise-removed inline hologram, and post-sampling is applied to the noise-removed inline hologram to generate a noise-removed inline hologram of the subsequent step of the current time step.

9. Input unit for receiving a digital hologram; and It includes a restoration unit that restores a phase image by applying the above digital hologram to a pre-trained restoration model, and The above digital hologram is an unsupervised learning-based phase image restoration system using a Gabor hologram comprising an original phase image and an original inline hologram.

10. In Paragraph 9, The above restoration model is, A phase image generation module that generates a phase image by applying a first forward process and a first reverse process to the digital hologram; and An unsupervised learning-based phase image restoration system using a Gabor hologram, comprising a hologram generation module that generates an inline hologram by applying a second forward process and a second reverse process to the digital hologram.

Images