Fourier ptychography microscopy using multiple ROIs and method for using thereof
The Fourier typography microscope addresses the limitations of existing live cell imaging by generating high-resolution images through synthesizing low-resolution images with varying illumination angles, enabling efficient label-free live cell imaging within a miniaturized cell culture system.
Patent Information
- Application Number
- KR1020210137126
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-10-15
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2041-10-15
AI Technical Summary
Existing live cell imaging equipment is expensive, limited by imaging speed, and requires cell fixation, making it unsuitable for continuous observation of living cells, and there is a need for label-free methods that can quantify morphological and physical information.
A Fourier typography microscope with a lighting display generating lights at different angles, a transparent sample container, and an image restoration module that synthesizes low-resolution images to generate high-resolution images, allowing for miniaturized live cell imaging within a cell culture system.
Enables high-resolution, label-free live cell imaging within a miniaturized cell culture system, overcoming limitations of existing equipment by resolving varying illumination angles and providing time-series data on live cells.
Smart Images

Figure 112021118055917-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a Fourier Typography microscope and a method of using the same. Specifically, the present invention relates to a Fourier Typography microscope capable of live cell imaging when mounted in a cell culture system and a method of using the same. Background Technology
[0002] Recently, there has been an increasing demand for live cell imaging equipment that can be utilized in cell experiments across fields such as biology, medicine, and pharmacy. By utilizing live cell imaging equipment, researchers can elucidate the mechanisms by which drugs act in vivo to treat diseases and assess the efficacy and toxicity of drug candidates. Furthermore, it is possible to detect and analyze molecular reactions related to intracellular signal transduction and to utilize it for studying the characteristics of stem cells. In particular, during the new drug development process, screening of new drug candidates can be effectively implemented by using live cell imaging to monitor and quantitatively analyze the long-term response of cell lines to target drugs.
[0003] Currently commercialized equipment for cell-based High Content Analysis (HCA) and High Content Screening (HCS) primarily utilizes fluorescence imaging to measure various target molecules within cells, tissues, and living organisms. These devices mainly employ laser scanning or spinning disc confocal microscopy methods and are capable of imaging various fluorescent labels at high resolution.
[0004] However, equipment using these fluorescent labels is very expensive and has limitations in imaging speed because it utilizes high-magnification lenses, precision optics, and laser light sources, and requires the mechanical, precise movement of samples for wide-area observation. Furthermore, most immunofluorescence labels used in fluorescence imaging require cell fixation during the labeling process, which may make them unsuitable for the continuous observation of living cell lines.
[0005] In addition, even in the case of fluorescent labels capable of live-cell imaging, further investigation is needed to determine whether they are suitable for HCS experiments because they affect cell growth. Therefore, imaging methods that continuously and quantitatively measure the morphological and physical information of cell lines to evaluate drug responses by imaging live cells without fluorescent labels are expected to be useful in cell-based / imaging-based drug evaluation systems.
[0006] For label-free live cell imaging, a quantitative phase imaging (QPI) method may be required to obtain refractive index and thickness information of light-nonabsorbing transparent samples from the sample's phase information. Fourier Ptychography Microscopy (FPM), a type of QPI technology, is a computational microscopy imaging technique that computationally reconstructs quantitative phase information through structural illumination and simultaneously increases the microscope's observation area and resolution using a massive space-bandwidth product. Prior art literature
[0007] Published Patent Application No. 2016-0065742 (Date of publication: June 9, 2016) The problem to be solved
[0008] The objective of the present invention is to provide a Fourier tychography microscope that solves the problem of varying illumination angles within the imaging FOV.
[0009] In addition, another objective of the present invention is to provide a method for using a Fourier tychography microscope that solves the problem of varying illumination angles within the imaging FOV.
[0010] The objects of the present invention are not limited to those mentioned above, and other unmentioned objects and advantages of the present invention may be understood from the following description and will be more clearly understood by the embodiments of the present invention. Furthermore, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims. means of solving the problem
[0011] A Fourier tychography microscope according to some embodiments of the present invention for solving the above problem comprises: a lighting display comprising a first light including a first incident light and a second incident light, and a first pixel generating the first incident light and a second pixel generating the second incident light; a transparent sample container disposed on the lighting display and accommodating a live cell, wherein the angle of incidence of the first incident light reaching the sample container and the angle of incidence of the second incident light are different; a second light disposed on the sample container and generated by the first light passing through the sample container is received in an imaging FOV (field of view) to generate a first low-resolution image and a second low-resolution image, wherein the first low-resolution image and the second low-resolution image each comprise an optical system corresponding to the first incident light and the second incident light, respectively, and an image restoration module that synthesizes the first low-resolution image and the second low-resolution image each in the frequency domain and inversely transforms them back into the spatial domain to generate a high-resolution image, wherein the image restoration module [represents] the imaging FOV The area is divided into multiple regions of interest (ROIs), the illumination angle of the first light is measured for each of the multiple ROIs, and the images of the multiple ROIs are restored according to the illumination angle.
[0012] Additionally, the area of the illumination FOV of the first pixel measured in the first plane is smaller than the area of the imaging FOV measured in the first plane, and the first plane can be defined perpendicular to the straight line connecting the illumination display and the optical system, and spaced apart by the first distance on the illumination display.
[0013] Additionally, the illumination angle includes a first illumination angle of the first incident light and a second illumination angle of the second incident light, wherein the first illumination angle of the first incident light differs for each of the plurality of ROIs, and the second illumination angle of the second incident light may differ for each of the plurality of ROIs.
[0014] In addition, the size of the plurality of ROIs may correspond to the size of the pixels of the lighting display.
[0015] Additionally, the cell culture housing includes a sealed culture chamber inside, and further includes a cell culture control unit that controls the culture environment inside the culture chamber to culture the live cell, and the lighting display, the transparent sample container, and the optical system may be located inside the culture chamber.
[0016] Additionally, the high-resolution image may include a first high-resolution image and a second high-resolution image corresponding to different points in time, and may further include a time-series data generation module that generates live cell time-series data over time using the first high-resolution image and the second high-resolution image.
[0017] In addition, the live cell time series data may include at least one of a time-lapse live cell image, a cell count graph, a cell confluence graph, a 3D cell motility image, an image sequence, and 2D live cell tracking data.
[0018] Additionally, the optical system may include an objective lens that magnifies an image of the sample container, a tube lens that is used together with the objective lens to help form an image of the sample container, and an image sensor that detects an image generated by the tube lens.
[0019] Additionally, the optical system further includes a filter between the objective lens, the tube lens, and the image sensor, and the filter may include at least one of a bandpass filter, a prism filter, and a polarizing filter.
[0020] Additionally, the cell culture housing includes a sealed culture chamber inside, and further includes a cell culture control unit that controls the culture environment inside the culture chamber to culture the live cell, and the lighting display, the transparent sample container, and the optical system are located inside the culture chamber, and the positions of the objective lens, the tube lens, and the image sensor, and the arrangement of the objective lens and the tube lens can be designed to be accommodated inside the culture chamber through an optical simulation program.
[0021] A method for using a Fourier tychography microscope according to some embodiments of the present invention for solving the above other problems comprises: an optical system receiving a second light generated by a first light passing through a sample container; detecting the second light to generate a low-resolution image; dividing the low-resolution image into a plurality of ROIs; measuring the illumination angle of the second light for each of the plurality of ROIs; generating a high-resolution image segment for each of the plurality of ROIs considering the illumination angle; and generating a high-resolution image using the high-resolution image segment.
[0022] Additionally, the second light includes a first transmitted light and a second transmitted light having different angles of incidence, and the low-resolution image includes a first low-resolution image corresponding to the first transmitted light and a second low-resolution image corresponding to the second transmitted light, and generating the high-resolution image segment may include measuring the illumination angle for each of the plurality of ROIs of the first and second low-resolution images, and generating the high-resolution image segment corresponding to each of the plurality of ROIs using the illumination angle for each of the plurality of ROIs and the first and second low-resolution images.
[0023] Additionally, generating the high-resolution image segments corresponding to each of the plurality of ROIs using the illumination angles for each of the plurality of ROIs may include: performing a Fourier transform on each of the images divided into the plurality of ROIs of the first low-resolution image into a first low-resolution frequency spectrum in the frequency domain; performing a Fourier transform on each of the images divided into the plurality of ROIs of the second low-resolution image into a second low-resolution frequency spectrum in the frequency domain; correcting the first low-resolution frequency spectrum according to the illumination angle for each of the plurality of ROIs to generate a first corrected spectrum; correcting the second low-resolution frequency spectrum according to the illumination angle for each of the plurality of ROIs to generate a second corrected spectrum; synthesizing the first corrected spectrum and the second corrected spectrum in the frequency domain to generate high-resolution spectrum segments for each of the plurality of ROIs; and inversely transforming the high-resolution spectrum segments to generate the high-resolution image segments.
[0024] Additionally, the high-resolution image may include a first high-resolution image and a second high-resolution image generated at different first and second timestamps, respectively, and may further include generating live cell time series data using the first and second high-resolution images.
[0025] In addition, the sample container accommodates a live cell, and the live cell can be continuously cultured between the first and second time points. Effects of the invention
[0026] The Fourier typography microscope and the method of use thereof according to the present invention can be mounted inside a cell culture vessel through miniaturization, thereby enabling the measurement of data on changes over time in live cells.
[0027] In addition, the small pixel size of the lighting display makes it easy to maintain spatial coherence, allowing for the acquisition of higher resolution images.
[0028] In addition, the problem of varying illumination angles within the imaging FOV due to miniaturization can be resolved, allowing for the acquisition of higher resolution images.
[0029] In addition to the above, the specific effects of the present invention are described together with the specific details for implementing the invention below. Brief explanation of the drawing
[0030] FIG. 1 is a conceptual diagram illustrating a Fourier tychography microscope according to some embodiments of the present invention. Figure 2 is an enlarged view illustrating the positional relationship between the illumination display, sample container, and objective lens of Figure 1. Figure 3 is a conceptual diagram illustrating the illumination FOV (Field of View) of a pixel of the illumination display of Figure 1 and the imaging FOV of an optical system. Figure 4 is a planar conceptual diagram for comparing the illumination FOV and imaging FOV of Figure 3. Figure 5 is a conceptual diagram illustrating the illumination angle of light reaching within the imaging FOV of Figure 3. Figure 6 is a conceptual diagram for explaining the lighting display of Figure 1 in detail. Figure 7 is an example diagram illustrating a lighting pattern by the lighting display of Figure 1. Figure 8 shows various examples of lighting patterns by the lighting display of Figure 1. Figure 9 is a block diagram for explaining the image restoration module of Figure 1 in detail. FIG. 10 is a conceptual diagram for explaining in detail how a low-resolution image is divided according to the division module of FIG. 9. Figure 11 is a conceptual diagram illustrating the image change process of the image restoration module of Figure 9. Fig. 12 is an example of a high-resolution image of Fig. 1. FIG. 13 is a conceptual diagram illustrating a Fourier tychography microscope according to some embodiments of the present invention. FIGS. 14 to 19 are example diagrams for explaining the live cell time series data of FIG. 13. FIG. 20 is a flowchart illustrating a method of using a Fourier tychography microscope according to some embodiments of the present invention. FIG. 21 is a flowchart for detailing the steps for generating high-resolution image segments of FIG. 20. FIG. 22 is a flowchart illustrating a method of using a Fourier tychography microscope according to some embodiments of the present invention. Specific details for implementing the invention
[0031] Terms and words used in this specification and claims shall not be interpreted as being limited to their general or dictionary meanings. In accordance with the principle that an inventor may define the concept of a term or word to best describe their invention, they shall be interpreted in a meaning and concept consistent with the technical spirit of the invention. Furthermore, since the embodiments described in this specification and the configurations illustrated in the drawings are merely one embodiment of the invention and do not represent the entire technical spirit of the invention, it should be understood that various equivalents, modifications, and applicable examples capable of replacing them may exist at the time of filing this application.
[0032] The terms first, second, A, B, etc., as used in this specification and claims may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.
[0033] The terms used in this specification and claims are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" should be understood as not precluding the existence or addition of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification.
[0034] In this specification and claims, where it is stated that one component is "connected" to another component, this should be understood to include cases where the components are connected through an intermediate component, as well as cases where they are directly connected. Conversely, only when it is stated that a component is "directly connected" or "directly connected" should it be understood that a component is connected to another component without any intermediate components. Likewise, other expressions describing the relationships between components should be understood in the same sense.
[0035] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this invention pertains.
[0036] Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.
[0037] In addition, each component, process, procedure, or method included in each embodiment of the present invention may be shared within a scope that is not technically contradictory to one another.
[0039] Hereinafter, with reference to FIGS. 1 to 9, a Fourier tychography microscope according to several embodiments of the present invention will be described.
[0040] FIG. 1 is a conceptual diagram illustrating a Fourier tychography microscope according to some embodiments of the present invention.
[0041] Referring to FIG. 1, a Fourier typography microscope according to some embodiments of the present invention includes an illumination display (100), a sample container (200), an optical system (300), and an image restoration module (400).
[0042] The lighting display (100) can generate light. The light generated by the lighting display (100) can pass through the sample container (200) and be transmitted to the optical system (300). The lighting display (100) can irradiate light having different angles of incidence onto the sample container (200) so that the optical system (300) and the image restoration module (400) can subsequently measure a high-resolution image (HRIm).
[0043] The sample container (200) may be positioned on the lighting display (100). The sample container (200) may be transparent so that light generated from the lighting display (100) can pass through. The sample container (200) may contain a sample inside. In this case, the sample may be a live cell, that is, a living cell. The sample container (200) may serve to fix the live cell in order to observe the live cell.
[0044] The optical system (300) can receive light generated from the lighting display (100) and transmitted through the sample container (200). The optical system (300) can receive the transmitted light to generate a low-resolution image (LRIm). There are multiple low-resolution images (LRIm), and each low-resolution image (LRIm) may be an image with different spatial frequency information due to light having different angles of incidence.
[0045] At this time, the optical system (300) may be positioned on the sample container (200). However, the present embodiment is not limited thereto. That is, if the lighting display (100), the sample container (200), and the optical system (300) are positioned sequentially along the path of light, it includes cases where the relationship is not vertical as in FIG. 1.
[0046] The optical system (300) may include an objective lens (310), a tube lens (320), and an image sensor (330). The objective lens (310) may be the component of the optical system (300) located closest to the sample container (200). The objective lens (310) may serve to magnify the image of the sample container (200). The objective lens (310) may include a large number of lenses, but is not limited thereto.
[0047] The objective lens (310) may use a lens with a relatively low magnification or a relatively low numerical aperture (NA) to enable large-area imaging. Generally, using a lens with a low magnification or a low numerical aperture allows for the acquisition of a low-resolution image over a large area. Conversely, using a lens with a high magnification or a high numerical aperture allows for the acquisition of a high-resolution image, but it is limited to a narrow area, so a large-area image cannot be acquired.
[0048] This embodiment enables large-area imaging by using an objective lens (310) with low magnification or a low numerical aperture, and subsequently allows the low-resolution image (LRIm) to be restored into a high-resolution image (HRIm) by an image restoration module (400), thereby enabling high image resolution to be obtained. Accordingly, the low-resolution image (LRIm) may be an image that displays a relatively wide area at a relatively low resolution as a large-area image.
[0049] The tube lens (320) can be used in conjunction with the objective lens (310) to assist in image formation. Parallel light generated through the objective lens (310) can form an image through the tube lens (320).
[0050] The image sensor (330) can detect an image generated by the tube lens (320). The image sensor (330) can generate current through a photoelectric element and process the current through a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor). However, the present embodiment is not limited thereto.
[0051] The optical system (300) may additionally include various filters between the designed lenses. For example, at least one of a bandpass filter, a prism, and a polarizing filter may be added to the optical system (300). Through this, the present embodiment may have at least one function of a fluorescence microscope, a hyperspectral microscope, and a birefringence microscope. However, the present embodiment is not limited thereto.
[0052] The image restoration module (400) receives a low-resolution image (LRIm) and restores it to generate a high-resolution image (HRIm) with better resolution than the low-resolution image (LRIm). At this time, the low-resolution image (LRIm) is a plurality of images having different spatial frequency information, and the image restoration module (400) can generate a high-resolution image (HRIm) by synthesizing these low-resolution images (LRIm) in the frequency domain. A specific method for generating the high-resolution image (HRIm) will be explained in more detail later.
[0053] Figure 2 is an enlarged view illustrating the positional relationship between the illumination display, sample container, and objective lens of Figure 1.
[0054] Referring to FIG. 2, the lighting display (100) may include a first pixel (Px1) and a second pixel (Px2). The first pixel (Px1) and the second pixel (Px2) may be placed at different locations on the lighting display (100). Although FIG. 2 describes only two pixels, the present embodiment is not limited thereto. That is, it is entirely possible for one or three or more pixels to generate light at different locations.
[0055] A lighting display (100) can irradiate a first light (L1) onto a sample container (200), and the first light (L1) may include a first incident light (Li1) and a second incident light (Li2). At this time, a first pixel (Px1) may generate the first incident light (Li1). A second pixel (Px2) may generate the second incident light (Li2). At this time, the first incident light (Li1) and the second incident light (Li2) may be irradiated onto the sample container (200) simultaneously, or may be irradiated onto the sample container (200) with a time difference.
[0056] The first incident light (Li1) and the second incident light (Li2) may have different angles of incidence depending on the positions of the first pixel (Px1) and the second pixel (Px2). That is, the angle of incidence at which the first incident light (Li1) is irradiated onto the sample container (200) and the angle of incidence at which the second incident light (Li2) is irradiated onto the sample container (200) may be different from each other.
[0057] The first light (L1) can be converted into the second light (L2) by passing through the sample container (200). The second light (L2) may include the first transmitted light (Lt1) and the second transmitted light (Lt2). In this case, the first transmitted light (Lt1) may be generated by the first incident light (Li1) passing through the sample container (200). Similarly, the second transmitted light (Lt2) may be generated by the second incident light (Li2) passing through the sample container (200). The second light (L2) may be received by the objective lens (310).
[0058] The first distance (D1) between the illumination display (100) and the sample container (200) may be smaller than the second distance (D2) between the sample container (200) and the objective lens (310). Conventional Fourier Ptychographic Microscopy (FPM) uses an LED array for illumination. In the case of such an LED array, spatial coherence can be obtained only if the size of the LED elements is sufficiently small and the distance between the sample container (200) and the LED array is sufficiently far. This is because, given the nature of FPM which obtains high-resolution images through low-resolution images with spatial frequency differences, spatial coherence is necessary to obtain low-resolution images with spatial frequency differences caused by the LED array.
[0059] Therefore, in the case of conventional FPMs, spatial coherence can be achieved by separating the sample and the light source by more than 100mm, which can result in a very long distance between the sample and the light source, and naturally, this distance between the sample and the light source can be greater than the distance between the sample and the optical system. This limitation on the distance between the sample and the light source has been the biggest obstacle to the miniaturization of FPMs.
[0060] In contrast, in some embodiments of the present invention, the first distance (D1) between the illumination display (100) and the sample container (200) in the Fourier Typography microscope may be significantly reduced to be smaller than or equal to 10 mm, which is 1 / 10 of the existing FPM. That is, the first distance (D1) may be greater than 0 and smaller than or equal to 10 mm. Accordingly, the first distance (D1) may be smaller than the second distance (D2), and the size of the Fourier Typography microscope according to some embodiments of the present invention may be manufactured on a greatly reduced scale.
[0061] This may be due to the fact that, as the size of the pixels of the lighting display (100) is small and the spacing between pixels is narrow, it is sufficiently possible to generate a low-resolution image (LRIm) having a spatial frequency difference according to the angle of incidence even if the size of the first distance (D1) is not increased.
[0062] Accordingly, the Fourier typography microscope according to some embodiments of the present invention can be designed and used in a relatively small size, and thus can be easily mounted even in equipment with limited size, such as a cell culture system. This will be explained in more detail later.
[0063] FIG. 3 is a conceptual diagram for explaining the illumination FOV (Field of View) of a pixel of the illumination display of FIG. 1 and the imaging FOV of an optical system, and FIG. 4 is a planar conceptual diagram for comparing the illumination FOV and imaging FOV of FIG. 3.
[0064] Referring to FIGS. 1, 3, and 4, the area of the illumination FOV (FOVil) of the first pixel (Px1) of the illumination display (100) may be smaller than the area of the imaging FOV (FOVim) of the optical system (300). In this case, the area of the illumination FOV (FOVil) and the area of the imaging FOV (FOVim) may be defined on a first plane (PL1). The first plane (PL1) may be a virtual plane defined to face the optical system (300) and the illumination display (100) at the location of the sample container (200). That is, the first plane (PL1) may be positioned perpendicular to the straight line connecting the illumination display (100) and the optical system (300).
[0065] This may be a phenomenon that occurs as the size of the pixels of the lighting display (100) decreases and the first distance (D1) decreases. At this time, the area of the lighting FOV of a pixel other than the first pixel (Px1) may also be smaller than the area of the imaging FOV (FOVim) of the optical system (300).
[0066] In the case of an LED array of a conventional FPM, as the LEDs are sufficiently large, the area of the illumination FOV is larger than the area of the imaging FOV (FOVim), so the difference in relative illumination angles at different locations within the imaging FOV (FOVim) area can be ignored.
[0067] In contrast, in a Fourier typography microscope according to some embodiments of the present invention, as the area of the illumination FOV (FOVil) becomes smaller than the area of the imaging FOV (FOVim), a difference in relative illumination angle may occur at different locations within the imaging FOV (FOVim) area. Accordingly, in a Fourier typography microscope according to some embodiments of the present invention, this difference in illumination angle can be calculated and transmitted to an image restoration module (400). Through this, the image restoration module (400) can generate a high-resolution image (HRIm) with greater accuracy and higher resolution.
[0068] Figure 5 is a conceptual diagram illustrating the illumination angle of light reaching within the imaging FOV of Figure 3.
[0069] Referring to FIGS. 1 and 5, even light generated from a single pixel of the lighting display (100) may have different illumination angles at various points of the imaging FOV (FOVim) located on the first plane (PL1). Specifically, light irradiated by the first pixel (Px1) may be divided into light incident perpendicularly to the first illumination angle (θ1) and the second illumination angle (θ2), which are acute angles, at various points within the imaging FOV (FOVim). This may be due to the fact that the first distance (D1) is smaller than the second distance (D2) and the illumination FOV (FOVil) is smaller than the imaging FOV (FOVim).
[0070] Accordingly, the image restoration module (400) can subsequently divide the low-resolution image (LRIm) into multiple ROIs (Regions of Interest) and restore it by considering the illumination angle for each ROI. This will be explained in more detail later.
[0071] Figure 6 is a conceptual diagram for explaining the lighting display of Figure 1 in detail.
[0072] Referring to FIG. 6, the lighting display (100) may further include a third pixel (Px3). The third pixel (Px3) may be the pixel closest to the first pixel (Px1). Here, "closest" may mean that they are placed close together without any other components between them.
[0073] The distance (G) between the first pixel (Px1) and the third pixel (Px3) may be much shorter than the distance between optical elements of a conventional FPM. Accordingly, even though the first distance (D1) is shortened, a large number of illumination FOVs (FOVil) can be located within the imaging FOV (FOVim), thereby enabling the generation of multiple low-resolution images (LRIm) and contributing to the miniaturization of the entire device. For example, the distance (G) between the first pixel (Px1) and the third pixel (Px3) may be greater than 0 and less than 50 micrometers. However, the present embodiment is not limited thereto.
[0074] Pixels including the first pixel (Px1), the second pixel (Px2), and the third pixel (Px3) may be arranged at regular intervals. However, the present embodiment is not limited thereto.
[0075] The diameters of the first pixel (Px1), the second pixel (Px2), and the third pixel (Px3) can be formed to be smaller than the diameter of the optical element of the conventional FPM, i.e., the LED. For example, the diameters of the first pixel (Px1), the second pixel (Px2), and the third pixel (Px3) can be greater than 0 and less than 20 micrometers. However, the present embodiment is not limited thereto.
[0076] The first pixel (Px1), the second pixel (Px2), and the third pixel (Px3) may all have the same size. However, the present embodiment is not limited thereto. That is, the diameters of the first pixel (Px1), the second pixel (Px2), and the third pixel (Px3) may be formed differently from each other.
[0077] As the pixel size of the lighting display (100) decreases, spatial coherence can be better maintained even though the first distance (D1) decreases. That is, in the case of conventional FPM, the distance between the sample and the lighting had to be kept long to obtain a low-resolution image in which spatial coherence occurs through light of various incident angles, but in this embodiment, even though the first distance (D1) is short, as the pixel size decreases, it is still possible to obtain a low-resolution image through spatial coherence.
[0078] Figure 7 is an example diagram illustrating a lighting pattern by the lighting display of Figure 1.
[0079] Referring to FIG. 7, the lighting display (100) can generate a desired pattern of lighting across the entire display. The lighting display (100) may be used as lighting as a whole, but only a specific area of the lighting display (100) may be used as lighting. The lighting display (100) may include a pattern area (PB), and the pattern area (PB) may be a part intended to be used as actual lighting. However, the present embodiment is not limited thereto.
[0080] The lighting display (100) may include at least one light pattern (Lp) and at least one dark pattern (Dp) within a pattern area (PB). The light pattern (Lp) is a portion that emits light within the pattern area (PB), and the dark pattern (Dp) may be a portion that is darkened within the pattern area (PB).
[0081] The light pattern (Lp) may be displayed by one or more light-emitting pixels. However, the light pattern (Lp) may not consist solely of light-emitting pixels. The brightness and size of the pattern of the light pattern (Lp) can be controlled through a combination of light-emitting pixels and dark pixels.
[0082] Similarly, the dark pattern (Dp) may be represented by one or more dark pixels, but the dark pattern (Dp) may not consist solely of dark pixels. The brightness and size of the pattern of the dark pattern (Dp) can also be controlled through a combination of light-emitting pixels and dark pixels.
[0083] The lighting display (100) may be any one of a display including at least one of an LCD method, an OLED method, a micro-LED method, and a mini-LED method.
[0084] Figure 8 shows various examples of lighting patterns by the lighting display of Figure 1.
[0085] Referring to FIG. 8, the lighting display (100) can freely implement and change lighting patterns, unlike an LED array. That is, an LED array determines the position of LED elements through the arrangement of hardware, and once the position of an LED element is determined, it cannot be changed again.
[0086] In contrast, the lighting display (100) of the present embodiment is <a1>class <a2>As such, free lighting patterns can be implemented through the emission and blackout of pixels without the need for changes in hardware configuration. Accordingly, it is possible to implement not only a simple Fourier Typography microscope, but also various types of microscopes such as a dark-field microscope using a donut-shaped pattern, a phase-contrast microscope using a semicircular pattern, a bright-field microscope using a circular pattern, and a structural illumination microscope using a fringe pattern.
[0087] FIG. 9 is a block diagram for explaining in detail the image restoration module of FIG. 1, and FIG. 10 is a conceptual diagram for explaining in detail how a low-resolution image is divided according to the division module of FIG. 9. FIG. 11 is a conceptual diagram for explaining the image change process of the image restoration module of FIG. 9.
[0088] Referring to FIGS. 1, 9 to 11, the image restoration module (400) may include a segmentation module (410), a Fourier transform module (420), a correction module (430), a stitch module (440), an inverse transform module (450), and an integration module (460).
[0089] The splitting module (410) can receive a low-resolution image (LRIm). The low-resolution image (LRIm) may include a first low-resolution image (LRIm1) and a second low-resolution image (LRIm2). In this case, the first low-resolution image (LRIm1) may be imaged by a first incident light (Li1) and a first transmitted light (Lt1), and the second low-resolution image (LRIm2) may be imaged by a second incident light (Li2) and a second transmitted light (Lt2). That is, the first low-resolution image (LRIm1) and the second low-resolution image (LRIm2) may be images having different spatial frequency information.
[0090] The splitting module (410) can split the first low-resolution image (LRIm1) and the second low-resolution image (LRIm2) into multiple ROIs. At this time, each ROI may be an area of a preset size. The reason the splitting module (410) splits the first low-resolution image (LRIm1) and the second low-resolution image (LRIm2) is that the imaging FOV (FOVim) of the optical system (300) is larger than the illumination FOV (FOVil) of the illumination display (100), so light irradiated from the same pixel has different illumination angles at each point of the imaging FOV (FOVim). Accordingly, it is necessary to divide the first low-resolution image (LRIm1) and the second low-resolution image (LRIm2) into the maximum unit area where it can be assumed that the illumination angles are the same and perform calculations.
[0091] To this end, the size of the ROI may correspond to the size of the light-emitting element constituting each pixel of the lighting display (100). Here, "correspondence" may mean a degree of similarity such that the sizes can be compared. For example, the size of the ROI may be 50% to 150% of the size of the light-emitting element constituting each pixel. However, the present embodiment is not limited thereto.
[0092] Accordingly, in each ROI area, light generated by a single pixel can have different illumination angles depending on the location. In the case of conventional FPMs, the LED array is far apart and the illumination FOV is larger than the imaging FOV, so the same illumination angle can be had in all areas of the imaging FOV, but this embodiment is not like that.
[0093] Referring to FIG. 10, the splitting module (410) can generate a first split low-resolution image (LRIm1) by splitting a first low-resolution image (LRIm1) into a plurality of ROIs. Additionally, the splitting module (410) can generate a second split low-resolution image (LRImz2) by splitting a second low-resolution image (LRIm2) into a plurality of ROIs. The first split low-resolution image (LRImz1) may include a first ROI (ROI1) and a second ROI (ROI2). The second split low-resolution image (LRImz2) may include a third ROI (ROI3) and a fourth ROI (ROI4).
[0094] The splitting module (410) can measure the illumination angle for each ROI of the first split low-resolution image (LRImz1). That is, the splitting module (410) can measure the illumination angle of the first ROI (ROI1) and the illumination angle of the second ROI (ROI2). Similarly, the splitting module (410) can measure the illumination angle for each ROI of the second split low-resolution image (LRImz2). That is, the splitting module (410) can measure the illumination angle of the third ROI (ROI3) and the illumination angle of the fourth ROI (ROI4). At this time, since the illumination angle is a result based on the position of each ROI, the illumination angle may be the same even for different images at the same position. That is, the illumination angle of the first ROI (ROI1) and the third ROI (ROI3) may be the same, and the illumination angle of the second ROI (ROI2) and the fourth ROI (ROI4) may be the same.
[0095] The Fourier transform module (420) can transform the first divided low-resolution image (LRImz1) and the second low-resolution image (LRIm2) into the first divided low-resolution frequency spectrum (LRSpz1) and the second divided low-resolution frequency spectrum (LRSpz2), respectively. The first divided low-resolution frequency spectrum (LRSpz1) may be the result of Fourier transforming the first divided low-resolution image (LRImz1). Similarly, the second divided low-resolution frequency spectrum (LRSpz2) may be the result of Fourier transforming the second divided low-resolution image (LRImz2). That is, the first divided low-resolution frequency spectrum (LRSpz1) and the second divided low-resolution frequency spectrum (LRSpz2) may be images in the frequency domain.
[0096] At this time, the first ROI (ROI1) can be converted into the first frequency ROI (ROIs1), and the third ROI (ROI3) can be converted into the third frequency ROI (ROIs3). That is, the first segmented low-resolution frequency spectrum (LRSpz1) is the result of performing a Fourier transform on the first segmented low-resolution image (LRImz1) for each ROI, and the second segmented low-resolution frequency spectrum (LRSpz2) is the result of performing a Fourier transform on the second segmented low-resolution image (LRImz2) for each ROI. Therefore, the first segmented low-resolution frequency spectrum (LRSpz1) may include the first frequency ROI (ROIs1), and the second segmented low-resolution frequency spectrum (LRSpz2) may include the third frequency ROI (ROIs3).
[0097] The correction module (430) can receive a first divided low-resolution frequency spectrum (LRSpz1) and a second divided low-resolution frequency spectrum (LRSpz2). The correction module (430) can correct the first divided low-resolution frequency spectrum (LRSpz1) using the measured illumination angle to generate a first corrected low-resolution frequency spectrum (LRSpc1). Additionally, the correction module (430) can correct the second divided low-resolution frequency spectrum (LRSpz2) using the measured illumination angle to generate a second corrected low-resolution frequency spectrum (LRSpc2).
[0098] Accordingly, the first frequency ROI (ROIs1) and the third frequency ROI (ROIs3) can be corrected into the first correction frequency ROI (ROIsc1) and the third correction frequency ROI (ROIsc3), respectively. That is, the correction module (430) can perform correction for each ROI. At this time, the correction can be performed in a direction that eliminates the effect according to the illumination angle.
[0099] The stitch module (440) can receive a first corrected low-resolution frequency spectrum (LRSpc1) and a second corrected low-resolution frequency spectrum (LRSpc2). The stitch module (440) can synthesize the first corrected low-resolution frequency spectrum (LRSpc1) and the second corrected low-resolution frequency spectrum (LRSpc2). At this time, the stitch module (440) can identify the overlapping portions of the first corrected low-resolution frequency spectrum (LRSpc1) and the second corrected low-resolution frequency spectrum (LRSpc2) and synthesize them together. Accordingly, the stitch module (440) can generate a high-resolution segmented frequency spectrum (HRSpz). The high-resolution segmented frequency spectrum (HRSpz) may be an image of the frequency domain in which the first corrected low-resolution frequency spectrum (LRSpc1) and the second corrected low-resolution frequency spectrum (LRSpc2) are synthesized.
[0100] Specifically, synthesis can be performed on a per-ROI basis. That is, the first correction frequency ROI (ROIsc1) and the third correction frequency ROI (ROIsc3), which are ROIs at the same location, can be synthesized together. Similarly, the first correction low-resolution frequency spectrum (LRSpc1) and the second correction low-resolution frequency spectrum (LRSpc2) can be synthesized at the same location of other ROIs, respectively. Accordingly, the first high-resolution frequency ROI (ROIs1_HR) can be generated. The first high-resolution frequency ROI (ROIs1_HR) can be included in the high-resolution segmented frequency spectrum (HRSpz).
[0101] The inverse transform module (450) can receive a high-resolution divided frequency spectrum (HRSpz). The inverse transform module (450) can inverse transform the high-resolution divided frequency spectrum (HRSpz) into a spatial domain to generate a high-resolution image segment (HRImsg). The high-resolution image segment (HRImsg) may be an image segment having a higher resolution than the first low-resolution image (LRIm1) and the second low-resolution image (LRIm2). The high-resolution image segment (HRImsg) may be an image segment corresponding to each ROI of the first low-resolution image (LRIm1) and the second low-resolution image (LRIm2). The high-resolution image segment (HRImsg) may include the first high-resolution ROI (ROI1_HR).
[0102] The integration module (460) can receive high-resolution image segments (HRImsg). The integration module (460) can merge all the high-resolution image segments (HRImsg) to generate a single large-area high-resolution image (HRIm).
[0103] Although the above description was made using two low-resolution images, the present embodiment is not limited thereto. That is, there may be three or more low-resolution images.
[0104] The image restoration module (400) may include a processor for computation. It may include a server composed of a CPU and / or a GPU. However, the present embodiment is not limited thereto.
[0105] Fig. 12 is an example of a high-resolution image of Fig. 1.
[0106] Referring to FIG. 12, the high-resolution image (HRIm) is a large-area image of a wide area and may have a resolution higher than the resolution according to the basic performance of the optical system (300). Through this, the present embodiment can observe the shape of the live cell over a large area with high resolution, thereby enabling very efficient research.
[0107] This embodiment can be miniaturized by utilizing the characteristics of the lighting display (100). Accordingly, although the problem of the same light illumination angle changing within the imaging FOV (FOVim) of the optical system (300) occurs, image restoration can be performed taking this into account to obtain a high-resolution image without problems.
[0108] Hereinafter, with reference to FIGS. 13 to 19, a Fourier tychography microscope according to several embodiments of the present invention will be described. Parts that overlap with the embodiments described above will be simplified or omitted.
[0109] FIG. 13 is a conceptual diagram illustrating a Fourier tychography microscope according to some embodiments of the present invention.
[0110] Referring to FIG. 13, a Fourier typography microscope according to some embodiments of the present invention may further include a cell culture device (600) and a time series data generation module (500).
[0111] The cell culture device (600) may include a cell culture device housing (610), a culture chamber (620), and a cell culture device control unit (630).
[0112] The cell culture housing (610) may be an outer wall that isolates the culture chamber (620) from the outside to create a cell culture environment. The cell culture housing (610) may surround a lighting display (100), a sample container (200), and an optical system (300).
[0113] The culture chamber (620) may be defined as a space enclosed by a cell culture housing (610). The culture chamber (620) may include a lighting display (100), a sample container (200), and an optical system (300) inside. An image restoration module (400) may be located inside the culture chamber (620), but may also be located outside the culture chamber (620). The image restoration module (400) may receive a low-resolution image (LRIm) via a data transmission line connected from the optical system (300).
[0114] The cell culture control unit (630) can control the environment of the culture chamber (620). The cell culture control unit (630) can control at least one of temperature, supply gas, and humidity to ensure that the live cells in the sample container (200) are properly cultured. The sample container (200) is fixed so that its position does not change within the cell culture unit (600) to ensure that the live cells are properly cultured.
[0115] Since a general cell culture device (600) is equipment for culturing small-scale cells, the size of the internal chamber may not be large. Therefore, in order for the Fourier Tichography microscope of this embodiment to be positioned inside the cell culture device (600), it is essential to adjust the size of the optical system (300).
[0116] In this embodiment, the first distance (D1) between the sample container (200) and the lighting display (100) can be adjusted so that it can be mounted inside the cell culture vessel (600). At this time, a magnification that satisfies an appropriate Nyquist rate can be set by considering the magnification of the objective lens (310) and the tube lens (320), and the pixel size and numerical aperture of the image sensor (330). In order to set such a magnification, the position and arrangement of the lenses must be adjusted, and in this embodiment, the optical system (300) can be designed through an optical simulation program that adjusts the position and arrangement of the lenses according to the size of the cell culture vessel (600). In this embodiment, through the optical system (300) designed in this way, a device of a size suitable for the size of the cell culture vessel (600) can be implemented.
[0117] The time series data generation module (500) can receive high-resolution images (HRIm) from the image restoration module (400). The time series data generation module (500) can receive multiple high-resolution images (HRIm) at different points in time. The time series data generation module (500) can generate live cell time series data (Dts) using multiple high-resolution images (HRIm).
[0118] A live cell located inside a sample container (200) can be cultured by a cell culture device (600). Accordingly, the present embodiment can generate live cell time series data (Dts) that can observe changes in the live cell.
[0119] FIGS. 14 to 19 are example diagrams for explaining the live cell time series data of FIG. 10.
[0120] Referring to Figures 13 and 14, this is an example of live cell time series data that can be used to check the cell confluence status. The cell confluence status from 0h to 48h can be checked through high-resolution images (HRIm), and a graph of the cell count over time can also be provided.
[0121] Referring to FIGS. 13 and FIGS. 15, a time-lapse live cell image can be observed. Time-lapse is a method of periodically extracting some frames from video image frames captured in real time to rapidly examine changes in the image. Through the time-lapse live cell image of FIG. 15 and the cell confluency graph over time, the Fourier Tichography microscope of the present embodiment can provide live cell time-series data (Dts) that can rapidly confirm changes in the live cell.
[0122] Figures 16 and 17 illustrate the intermediate process of measuring cell motility. As shown in Figure 16, the boundaries of the live cells are first detected to identify the live cells and verify their number. Subsequently, as shown in Figure 17, the boundaries of each live cell are colored, and their movement over time can be displayed in three dimensions. Through this live cell time-series data (Dts), the movement of each live cell can be intuitively understood.
[0123] Figure 18 shows an image sequence. That is, high-resolution images (HRIm) generated periodically or non-periodically are arranged in sequence to allow for detailed observation of changes in the cells. Figure 19 is live cell time series data (Dts) that tracks live cells by displaying the movement of each cell in different colors on a two-dimensional plane and indicates the motility of the live cells.
[0124] The Fourier typography microscope according to the present embodiment can image live cells without fluorescent labeling, enabling continuous measurement of morphological and physical information of cell lines without the influence of cell growth. Furthermore, the use of small pixels facilitates the maintenance of spatial coherence, thereby enabling easy high-resolution imaging. Additionally, the time-series changes of live cells can be generated as various data, which can be usefully applied in various fields.
[0125] Hereinafter, a method of using a Fourier Tichography microscope according to several embodiments of the present invention will be described with reference to FIGS. 1 to 22. Parts that overlap with the above-described embodiments will be simplified.
[0126] FIG. 20 is a flowchart for explaining a method of using a Fourier Typography microscope according to some embodiments of the present invention, and FIG. 21 is a flowchart for explaining in detail the step of generating a high-resolution image segment of FIG. 20. FIG. 22 is a flowchart for explaining a method of using a Fourier Typography microscope according to some embodiments of the present invention.
[0127] First, referring to FIG. 20, a low-resolution image is generated (S100).
[0128] Specifically, referring to FIGS. 1 and 2, the optical system (300) can receive light generated from the illumination display (100) and transmitted through the sample container (200). The optical system (300) can receive the transmitted light to generate a low-resolution image (LRIm). There are multiple low-resolution images (LRIm), and each low-resolution image (LRIm) may be an image with different spatial frequency information due to light having different angles of incidence. At this time, multiple low-resolution images (LRIm) generated at a first time point are defined as multiple first time point low-resolution images.
[0129] Again, referring to FIG. 20, a low-resolution image is divided into multiple ROIs (S200).
[0130] Specifically, referring to FIGS. 9 and 10, the splitting module (410) can split a first low-resolution image (LRIm1) and a second low-resolution image (LRIm2) into multiple ROIs. At this time, each ROI may be an area of a preset size. The splitting module (410) can generate a first split low-resolution image (LRImz1) by splitting the first low-resolution image (LRIm1) into multiple ROIs. Additionally, the splitting module (410) can generate a second split low-resolution image (LRImz2) by splitting the second low-resolution image (LRIm2) into multiple ROIs.
[0131] Again, referring to FIG. 20, the illumination angle is measured for each of the multiple ROIs (S300).
[0132] Specifically, referring to FIGS. 9 and FIGS. 10, the splitting module (410) can measure the illumination angle for each ROI of the first split low-resolution image (LRImz1). That is, the splitting module (410) can measure the illumination angle of the first ROI (ROI1) and the illumination angle of the second ROI (ROI2). Likewise, the splitting module (410) can measure the illumination angle for each ROI of the second split low-resolution image (LRImz2). That is, the splitting module (410) can measure the illumination angle of the third ROI (ROI3) and the illumination angle of the fourth ROI (ROI4).
[0133] Again, referring to FIG. 20, high-resolution image segments are generated for each of the multiple ROIs considering the illumination angle (S400).
[0134] Referring specifically to FIG. 21, first and second divided low-resolution frequency spectra are generated (S410).
[0135] Specifically, referring to FIGS. 9 to 11, the Fourier transform module (420) can transform the first divided low-resolution image (LRImz1) and the second low-resolution image (LRIm2) into the first divided low-resolution frequency spectrum (LRSpz1) and the second divided low-resolution frequency spectrum (LRSpz2), respectively. The first divided low-resolution frequency spectrum (LRSpz1) may be the result of Fourier transforming the first divided low-resolution image (LRImz1). Likewise, the second divided low-resolution frequency spectrum (LRSpz2) may be the result of Fourier transforming the second divided low-resolution image (LRImz2). That is, the first divided low-resolution frequency spectrum (LRSpz1) and the second divided low-resolution frequency spectrum (LRSpz2) may be images in the frequency domain.
[0136] Referring again to FIG. 21, first and second corrected low-resolution frequency spectra are generated (S420).
[0137] Specifically, with reference to FIGS. 9 to 11, the correction module (430) can correct the first segmented low-resolution frequency spectrum (LRSpz1) using the measured illumination angle to generate the first corrected low-resolution frequency spectrum (LRSpc1). Additionally, the correction module (430) can correct the second segmented low-resolution frequency spectrum (LRSpz2) using the measured illumination angle to generate the second corrected low-resolution frequency spectrum (LRSpc2).
[0138] Referring again to FIG. 21, a high-resolution divided frequency spectrum is generated (S430).
[0139] Specifically, referring to FIGS. 9 to 11, the stitch module (440) can synthesize a first corrected low-resolution frequency spectrum (LRSpc1) and a second corrected low-resolution frequency spectrum (LRSpc2). At this time, the stitch module (440) can identify the overlapping portions of the first corrected low-resolution frequency spectrum (LRSpc1) and the second corrected low-resolution frequency spectrum (LRSpc2) and synthesize them together. Accordingly, the stitch module (440) can generate a high-resolution segmented frequency spectrum (HRSpz). The high-resolution segmented frequency spectrum (HRSpz) may be an image of the frequency domain in which the first corrected low-resolution frequency spectrum (LRSpc1) and the second corrected low-resolution frequency spectrum (LRSpc2) are synthesized.
[0140] Referring again to FIG. 21, a high-resolution image segment is generated (S440).
[0141] Specifically, referring to FIGS. 9 to 11, the inverse transform module (450) can inverse transform the high-resolution divided frequency spectrum (HRSpz) into the spatial domain to generate a high-resolution image segment (HRImsg). The high-resolution image segment (HRImsg) may be an image segment having a higher resolution than the first low-resolution image (LRIm1) and the second low-resolution image (LRIm2). The high-resolution image segment (HRImsg) may be an image segment corresponding to each ROI of the first low-resolution image (LRIm1) and the second low-resolution image (LRIm2).
[0142] Referring again to FIG. 20, a high-resolution image is generated using a high-resolution image segment (S500).
[0143] Specifically, with reference to FIGS. 9 to 11, the integration module (460) can receive high-resolution image segments (HRImsg). The integration module (460) can merge all high-resolution image segments (HRImsg) to generate a high-resolution image (HRIm), which is a single large-area image.
[0144] Referring to Fig. 22, a method for generating live cell time series data is explained.
[0145] Referring to FIG. 22, a first high-resolution image is generated at a first time point (S1000).
[0146] At this time, the first high-resolution image can be generated at a first time point through the steps described in FIG. 20. That is, the first high-resolution image may be a high-resolution image generated at a first time point.
[0147] Next, a second high-resolution image is generated at a second time point (S2000).
[0148] At this time, the second time point may be a time point later than the first time point. The second high-resolution image may be generated at the second time point through the steps described in FIG. 20. That is, the second high-resolution image may be a high-resolution image generated at the second time point.
[0149] Next, live cell time series data is generated (S3000).
[0150] Specifically, with reference to FIGS. 13 to 19, the time series data generation module (500) can generate live cell time series data (Dts) using a plurality of high-resolution images (HRIm). The live cell time series data (Dts) can be visualized and represented by at least one of cell confluence, cell count, time-lapse image, and cell motility.
[0151] The above description is merely an illustrative explanation of the technical concept of the present embodiment, and a person skilled in the art to which the present embodiment belongs would be able to make various modifications and variations within the scope of the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment. Explanation of the symbols
[0152] 100: Light display 200: Sample container 300: Optical system 310: Objective lens 320: Tube lens 330: Image sensor 400: Image Restoration Module 410: Split Module 420: Fourier Transform Module 430: Calibration Module 440: Stitch Module 450: Inverse transformation module 460: Integrated Module 500: Time Series Data Generation Module 600: Cell culture device 610: Cell culture housing 620: Culture chamber 630: Cell culture control unit
Claims
Claim 1 A lighting display comprising a first pixel generating a first incident light and a second pixel generating a second incident light; a transparent sample container disposed on the lighting display and accommodating a live cell, wherein the angle of incidence of the first incident light reaching the sample container and the angle of incidence of the second incident light are different; and an optical system disposed on the sample container, wherein the first light generated from the lighting display passes through the sample container and the second light generated is received in an imaging FOV (field of view) to generate a first low-resolution image and a second low-resolution image, wherein the first low-resolution image and the second low-resolution image each correspond to the first incident light and the second incident light, respectively. A Fourier Tichography microscope comprising an image restoration module that synthesizes the first low-resolution image and the second low-resolution image in the frequency domain, respectively, and inversely transforms them back into the spatial domain to generate a high-resolution image, wherein the image restoration module divides the imaging FOV into a plurality of ROIs (regions of interest), measures the illumination angle of the first light for each of the plurality of ROIs, restores the image of the plurality of ROIs according to the illumination angle, and the area of the illumination FOV of the first pixel measured in the first plane is smaller than the area of the imaging FOV measured in the first plane, and the first plane is spaced by a first distance on the illumination display and is defined perpendicular to a straight line connecting the illumination display and the optical system. Claim 2 delete Claim 3 In claim 1, the illumination angle includes a first illumination angle of the first incident light and a second illumination angle of the second incident light, wherein the first illumination angle of the first incident light is different for each of the plurality of ROIs and the second illumination angle of the second incident light is different for each of the plurality of ROIs, a Fourier Typography microscope. Claim 4 In claim 1, the size of the plurality of ROIs corresponds to the size of the pixels of the illumination display in a Fourier Typography microscope. Claim 5 In claim 1, the cell culturer housing including a sealed culture chamber inside, and the cell culturer control unit for controlling the culture environment inside the culture chamber to culture the live cell, further comprising the lighting display, the transparent sample container, and the optical system a Fourier Typography microscope located inside the culture chamber. Claim 6 A Fourier tychography microscope according to claim 5, wherein the high-resolution image includes a first high-resolution image and a second high-resolution image corresponding to different points in time, and further includes a time-series data generation module that generates live cell time-series data over time using the first high-resolution image and the second high-resolution image. Claim 7 In claim 6, the live cell time series data comprises at least one of a time-lapse live cell image, a cell count graph, a cell confluence graph, a three-dimensional cell motility image, an image sequence, and two-dimensional live cell tracking data, in a Fourier tychography microscope. Claim 8 A Fourier Typography microscope according to claim 1, wherein the optical system comprises an objective lens that magnifies an image of the sample container, a tube lens that is used together with the objective lens to help form an image of the sample container, and an image sensor that detects an image generated by the tube lens. Claim 9 In claim 8, the optical system further comprises a filter between the objective lens, the tube lens, and the image sensor, and the filter comprises at least one of a bandpass filter, a prism filter, and a polarizing filter, in a Fourier tyography microscope. Claim 10 A Fourier Typography microscope according to claim 8, further comprising a cell culture housing including a culture chamber sealed inside, and a cell culture control unit for controlling a culture environment inside the culture chamber to culture the live cell, wherein the lighting display, the transparent sample container, and the optical system are located inside the culture chamber, and the positions of the objective lens, the tube lens, and the image sensor, and the arrangement of the objective lens and the tube lens are designed to be accommodated inside the culture chamber through an optical simulation program. Claim 11 A method for using a Fourier Tichography microscope comprising: an optical system receiving a second light generated by a first light passing through a sample container; detecting the second light to generate a low-resolution image; dividing the low-resolution image into a plurality of ROIs; measuring the illumination angle of the second light for each of the plurality of ROIs; generating a high-resolution image segment for each of the plurality of ROIs considering the illumination angle; and generating a high-resolution image using the high-resolution image segment. Claim 12 A method for using a Fourier Tichography microscope according to claim 11, wherein the second light comprises a first transmitted light and a second transmitted light having different angles of incidence, the low-resolution image comprises a first low-resolution image corresponding to the first transmitted light and a second low-resolution image corresponding to the second transmitted light, and generating the high-resolution image segment comprises measuring the illumination angle for each of the plurality of ROIs of the first and second low-resolution images, and generating the high-resolution image segment corresponding to each of the plurality of ROIs using the illumination angle for each of the plurality of ROIs and the first and second low-resolution images. Claim 13 A method for using a Fourier Tichography microscope according to claim 12, wherein generating the high-resolution image segments corresponding to each of the plurality of ROIs using the illumination angles for each of the plurality of ROIs comprises: Fourier transforming each of the images divided into the plurality of ROIs of the first low-resolution image into a first low-resolution frequency spectrum in the frequency domain; Fourier transforming each of the images divided into the plurality of ROIs of the second low-resolution image into a second low-resolution frequency spectrum in the frequency domain; generating a first corrected spectrum by correcting the first low-resolution frequency spectrum according to the illumination angle for each of the plurality of ROIs; generating a second corrected spectrum by correcting the second low-resolution frequency spectrum according to the illumination angle for each of the plurality of ROIs; generating a high-resolution spectrum segment for each of the plurality of ROIs by synthesizing the first corrected spectrum and the second corrected spectrum in the frequency domain; and generating the high-resolution image segments by inverse transforming the high-resolution spectrum segments. Claim 14 A method of using a Fourier tychography microscope according to claim 11, wherein the high-resolution image includes a first high-resolution image and a second high-resolution image generated at different first and second timestamps, respectively, and further comprises generating live cell time series data using the first and second high-resolution images. Claim 15 A method of using a Fourier Typography microscope in which, in claim 14, the sample container accommodates a live cell, and the live cell is continuously cultured between the first and second time points.
Citation Information
Patent Citations
Illumination system and equipment for Fourier ptychography imaging
JP2018503059A
Array-level Fourier ptychography imaging
JP2018511815A
Microscopy system with single shot Fourier ptychography
KR1020190088277A
