Imaging device and operating method thereof

Abstract

This imaging device may control the intensity of a laser beam emitted from a light source to be a saturation intensity or greater at which an excited state of a fluorophore is saturated, and control a pulse width or exposure time of the laser beam. The imaging device may modulate, by using a light modulation module, the laser beam having the beam intensity of the saturation intensity or greater and the controlled pulse width or exposure time into a plurality of structured light beams having different pattern directions and pattern phases, and irradiate a biological sample with the plurality of structured light beams. The imaging device may acquire, by using an image sensor, a plurality of fluorescence images corresponding to the pattern directions and pattern phases of the plurality of structured light beams. The imaging device may acquire a result image by synthesizing the plurality of fluorescence images in a frequency space.

Description

Imaging device and method of operation thereof

[0001] The present disclosure relates to an imaging device and a method of operating the same. Specifically, the present disclosure discloses an imaging device and a method of operating the same for performing DNA sequencing imaging, which images a fluorescent signal excited from a DNA sample. Furthermore, the present disclosure discloses an imaging device and a method of operating the same for imaging a multi-omics including at least one of genomics, transcriptomics, proteomics, and metabolomics within a cell by imaging a fluorescent signal excited from a cell sample.

[0002] DNA sequencing imaging is a method of identifying the type of base by distinguishing and identifying the fluorescent signals of fluorescent substances (fluorophores) that emit different wavelengths (colors) bound to four types of bases. Since the performance of the imaging device (e.g., a fluorescent imaging device) is highly relevant in terms of sequencing accuracy, measurement time, and productivity, the imaging device needs to acquire precise and accurate fluorescent images.

[0003] Recently, techniques for Structured Illumination Microscopy (SI), which utilizes structured light-based imaging methods to acquire high-resolution fluorescence images, are being developed.

[0004] One aspect of the present disclosure provides an imaging device. The imaging device may include a light source that irradiates a laser beam. The imaging device may include a diffraction grating or a spatial light modulator and may include a first light modulation module that modulates the laser beam into structured light. The imaging device may include an image sensor that receives a fluorescence signal emitted by a fluorophor coupled to a biological sample as the structured light is irradiated. The imaging device may include at least one processor that includes processing circuitry. The imaging device may include a memory that stores one or more instructions.

[0005] By executing one or more commands individually or collectively by at least one processor, the imaging device can adjust the intensity of the laser beam irradiated by the light source (10) to a saturation intensity greater than that which saturates the excited state of the phosphor, and can adjust the pulse width or exposure time of the laser beam. By executing one or more commands individually or collectively by at least one processor, the imaging device can use the first light modulation module (50) to modulate the laser beam, which has a beam intensity greater than the saturation intensity and has a pulse width or exposure time adjusted, into a plurality of structured lights having different pattern directions and pattern phases, and irradiate the plurality of structured lights toward a biological sample. By executing one or more commands individually or collectively by at least one processor, the imaging device can use the image sensor (20) to acquire a plurality of fluorescent images corresponding to the pattern directions and pattern phases of the plurality of structured lights. By executing one or more instructions individually or collectively by at least one processor, the imaging device can synthesize multiple fluorescence images in the frequency space to obtain a result image.

[0006] Another aspect of the present disclosure provides a method of operating an imaging device. The method may include the step of adjusting the intensity of a laser beam irradiated by a light source (10) to a saturation intensity greater than that which saturates the excited state of a phosphor bound to a biological sample, and adjusting the pulse width or exposure time of the laser beam. The method may include the step of using a first light modulation module (50) to modulate the laser beam, which has a beam intensity greater than the saturation intensity and has a pulse width or exposure time greater than that, into a plurality of structured lights having different pattern directions and pattern phases, and irradiating the plurality of structured lights toward a biological sample. The method may include the step of acquiring a plurality of fluorescent images corresponding to the pattern directions and pattern phases of the plurality of structured lights by receiving a fluorescent signal excited and emitted by a phosphor bound to a biological sample as the plurality of different structured lights are irradiated using an image sensor (20). The method may include the step of synthesizing the plurality of fluorescent images in the frequency space to acquire a result image.

[0007] The present disclosure can be easily understood from the combination of the following detailed description and the accompanying drawings, where reference numerals denote structural elements.

[0008] FIG. 1 illustrates the operation of an imaging device according to one embodiment of the present disclosure imaging a fluorescent signal excited from a biological sample.

[0009] FIG. 2 is a flowchart illustrating the operation method of an imaging device according to one embodiment of the present disclosure.

[0010] FIG. 3 is a block diagram illustrating the components of an imaging device according to one embodiment of the present disclosure.

[0011] FIG. 4 is a diagram showing the operation method of an imaging device according to one embodiment of the present disclosure.

[0012] FIG. 5 is a drawing for explaining structured light according to one embodiment of the present disclosure.

[0013] Figure 6 is a diagram showing graphs visualizing frequency transfer characteristics according to the lighting used in the imaging device.

[0014] Figure 7 is a diagram showing graphs visualizing frequency transfer characteristics according to the lighting used in the imaging device.

[0015] FIG. 8 is a flowchart illustrating an example of a method for controlling a laser beam irradiated as a light source by an imaging device according to one embodiment of the present disclosure.

[0016] FIG. 9 is a flowchart illustrating an example of a method for controlling a laser beam irradiated as a light source by an imaging device according to one embodiment of the present disclosure.

[0017] Figure 10 is a graph showing the laser output characteristics over time for a continuous wave laser and a time-modulated continuous wave laser, respectively.

[0018] FIG. 11 is a drawing illustrating examples of periodic grid patterns of structured light according to one embodiment of the present disclosure.

[0019] FIG. 12 is a flowchart illustrating an example of a method in which an imaging device according to one embodiment of the present disclosure irradiates a biological sample with a plurality of structured lights.

[0020] FIG. 13 is a schematic diagram of an imaging device in which a plurality of structured lights are irradiated in a coaxial illumination manner according to one embodiment of the present disclosure.

[0021] FIG. 14 is a schematic diagram of an imaging device in which a plurality of structured lights are irradiated in a transmitted illumination manner according to one embodiment of the present disclosure.

[0022] FIG. 15 is a schematic diagram of an imaging device in which a plurality of structured lights are irradiated in a light sheet illumination manner according to one embodiment of the present disclosure.

[0023] FIG. 16 is a flowchart illustrating a method for an imaging device according to one embodiment of the present disclosure to acquire a result image based on a plurality of fluorescent images.

[0024] FIG. 17 is a flowchart illustrating a method for an imaging device according to one embodiment of the present disclosure to acquire a plurality of fluorescent images containing three-dimensional information.

[0025] FIG. 18 is a drawing for illustrating an example of a method in which an imaging device according to one embodiment of the present disclosure irradiates a plurality of structured lights to each of different focal positions along the Z-axis direction corresponding to the depth direction of a biological sample.

[0026] FIG. 19 is a drawing for illustrating an example of a method in which an imaging device according to one embodiment of the present disclosure irradiates a plurality of structured lights to each of different focal positions along the Z-axis direction corresponding to the depth direction of a biological sample.

[0027] FIG. 20 is a flowchart illustrating a method for an imaging device according to one embodiment of the present disclosure to acquire a plurality of fluorescent images containing three-dimensional information.

[0028] FIG. 21 is a drawing for explaining an example of a method in which an imaging device according to one embodiment of the present disclosure irradiates a plurality of structured lights for each of different incident angles.

[0029] FIG. 22 is a flowchart illustrating a method for an imaging device to acquire a result image according to one embodiment of the present disclosure.

[0030] FIG. 23 is a drawing illustrating a three-dimensional (3D) image of intracellular multi-omics obtained by an imaging device according to one embodiment of the present disclosure.

[0031] FIG. 24 is a schematic diagram of an imaging device including a plurality of light sources and a plurality of image sensors according to one embodiment of the present disclosure.

[0032] The terms used in the embodiments of this specification have been selected to be as widely used as possible, taking into account the functions of the present disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description section of the relevant embodiments. Therefore, terms used in this specification should be defined not merely by their names, but based on their meanings and the overall content of the present disclosure.

[0033] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art as described in this specification.

[0034] In this document, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0035] The term “and / or” includes a combination of multiple related described components or any of the multiple related described components.

[0036] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another component and do not limit the components in other aspects (e.g., importance or order).

[0037] Throughout this disclosure, when a part is described as “comprising” a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as “...part,” “...module,” etc., as used in this specification refer to a unit that processes at least one function or operation, and may be implemented in hardware or software, or as a combination of hardware and software.

[0038] The expression “configured to” as used in this disclosure may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware. Instead, in some situations, the expression “system configured to” may mean that the system is “capable of” together with other devices or components. For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing said operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or an application processor) capable of performing said operations by executing one or more software programs stored in memory.

[0039] In addition, when a component is described in the present disclosure as being “connected” or “connected” to another component, it should be understood that the component may be directly connected to or directly connected to the other component, but unless otherwise specifically stated, it may also be connected or connected through another component in between.

[0040] All functions or operations described in this disclosure may be processed individually by a single processor and / or collectively by a plurality of processors. A single processor or a combination of a plurality of processors may include circuitry that performs processing, such as an Application Processor (AP), Communication Processor (CP), Graphical Processing Unit (GPU), Neural Processing Unit (NPU), Microprocessor Unit (MPU), System on Chip (SoC), Integrated Chip (IC), etc.

[0041] It should be understood that the blocks and combinations of flowcharts in the flowcharts illustrated in the present disclosure may be performed by one or more computer programs comprising computer-executable instructions. The one or more computer programs may be stored all in a single memory or may be divided and stored in multiple different memories.

[0042] One embodiment of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a specific function. Additionally, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as algorithms executed on one or more processors. Furthermore, the present disclosure may employ prior art for electronic configuration, signal processing, and / or data processing, etc.

[0043] In the present disclosure, 'DNA sequencing imaging' refers to an imaging technique that visually displays the base sequence (A, T, C, G) of DNA. In one embodiment of the present disclosure, DNA sequencing imaging may include a Fluorescence In Situ Hybridization (FISH) method, which involves binding a fluorescent dye to a specific DNA sequence among A, T, C, and G to image the corresponding DNA region in real time using an imaging device such as a microscope.

[0044] In the present disclosure, 'multi-omics' refers to a method for analyzing a biological system by integrating various omics data, and specifically refers to a method of imaging by analyzing data such as intracellular genomics, transcriptomics, proteomics, and metabolomics together. In one embodiment of the present disclosure, multi-omics may refer to an imaging method that senses and images at least one of intracellular DNA, RNA, or protein.

[0045] In the present disclosure, 'fluorophore' refers to a compound that absorbs light of a specific wavelength and emits light of another wavelength. In the present disclosure, the fluorophore emits fluorescent signals of different wavelengths depending on the DNA base sequence, thereby enabling an imaging device to identify and image the DNA base sequence through the fluorescent signals.

[0046] In the present disclosure, a 'biological sample' may be a sample or specimen observed by an imaging device (100) and may be an object subject to testing and / or analysis. The biological sample may contain at least one gene base. For example, the biological sample may be a genetic biological tissue such as a cell, protein, or tissue. In one embodiment of the present disclosure, the biological sample may be a sample treated with a plurality of fluorophores. A fluorophor may be a substance that emits a fluorescent signal when irradiated with light of a predetermined wavelength or a predetermined range of wavelengths. A predetermined fluorescent substance may be pre-treated on the biological sample before the imaging device (100) is used.

[0047] In the present disclosure, 'optical resolution' may be defined as the minimum distance at which an optical device can distinguish two objects or points. Optical resolution may represent the ability of an optical device to capture fine images. Optical resolution may be influenced by the wavelength of light, numerical aperture (NA), etc., and optical devices with higher optical resolution can output images with higher resolution and finer details.

[0048] In the present disclosure, 'NA (numerical aperture)' is a physical parameter that determines the light collection capability and resolution of an optical device (objective lens, microscope, or image sensor, etc.). NA can be determined by the following formula.

[0049]

[0050] In the above mathematical formula 1, n is the refractive index of the medium, and θ represents the maximum half-angle that the optical device can collect. That is, NA is determined by the refractive index of the medium and the maximum angle at which light can be incident on the lens. As NA increases, the size of the point formed by the lens becomes smaller, and as a result, the resolution increases.

[0051] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0052] Embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0053] FIG. 1 illustrates the operation of an imaging device (100) according to one embodiment of the present disclosure imaging a fluorescent signal excited from a biological sample (200).

[0054] In one embodiment of the present disclosure, the imaging device (100) may perform DNA sequencing imaging. The imaging device (100) may be an optical device that performs DNA sequencing imaging by using a light source (10) to irradiate a beam toward a biological sample (200) and imaging the emitted light emitted from the biological sample (200) by the irradiated beam to obtain an image representing a DNA base sequence (A, T, C, G). For example, the biological sample (200) may be a DNA sample. In one embodiment of the present disclosure, the imaging device (100) may be implemented as an optical device such as an optical microscope, a fluorescence microscope, a super-resolution imaging system, or a multispectral analysis system.

[0055] In FIG. 1, only essential components for explaining the function and / or operation of the imaging device (100) are illustrated, and the components included in the imaging device (100) are not limited to those illustrated in FIG. 1. The components included in the imaging device (100) will be described in detail with reference to FIG. 3 and FIG. 4.

[0056] Referring to FIG. 1, the imaging device (100) may include a light source (10), a light modulation module (50), and an image sensor (20). The imaging device (100) may irradiate a beam toward a biological sample (200) using the light source (10). The imaging device (100) may receive a fluorescence signal emitted from the biological sample (200) by the irradiated beam using the image sensor (20). When the beam is irradiated onto the biological sample (200), the image sensor (20) may acquire a fluorescence signal generated as a fluorescent substance bound to the biological sample (200) is excited. The imaging device (100) may acquire a fluorescence image by imaging the fluorescence signal (or converting the fluorescence signal into an electrical signal) using the image sensor (20).

[0057] In one embodiment of the present disclosure, the imaging device (100) can irradiate a laser beam using a light source (10).

[0058] In one embodiment of the present disclosure, the light source (10) may be configured as a pulsed laser light source that irradiates a pulsed laser. A pulsed laser is a laser that emits light in the form of pulses rather than continuously emitting light. An imaging device (100) can adjust the pulse width of the pulsed laser irradiated by the light source (10).

[0059] In one embodiment of the present disclosure, the light source (10) may be configured as a continuous-wave laser light source that irradiates a continuous-wave laser (CW laser). A continuous-wave laser is a laser that continuously emits light with a constant output. An imaging device (100) may control the exposure time of the continuous-wave laser irradiated by the light source (10). By controlling the exposure time of the continuous-wave laser, the imaging device (100) may control at least one of the pulse period and pulse width of the imaging device (100). In the present disclosure, a continuous-wave laser with controlled exposure time may be referred to as a time-modulated continuous-wave laser. By controlling the exposure time of the continuous-wave laser, the imaging device (100) may modulate the continuous-wave laser irradiated from the light source (10) into a continuous-wave laser having a predetermined pulse period or a predetermined pulse width. In other words, by controlling the exposure time of the continuous-wave laser, the imaging device (100) may modulate the continuous-wave laser irradiated from the light source (10) into a continuous-wave laser that is output in a pulse form.

[0060] In one embodiment of the present disclosure, the imaging device (100) may adjust the intensity of a laser beam emitted from a light source (10) to a saturation intensity greater than that which saturates the excited state of a phosphor bound to a biological sample (200). When the intensity of the laser beam irradiated onto the biological sample (200) is less than or equal to the saturation intensity, the intensity of the laser beam irradiated onto the biological sample (200) and the amount of fluorescence generated from the biological sample (200) may be proportional to each other. When the intensity of the laser beam irradiated onto the biological sample (200) is greater than or equal to the saturation intensity, the phosphor may respond non-linearly to the intensity of the laser beam as it reaches a saturation state. Reaching a saturation state means that the amount of fluorescence does not increase proportionally to the intensity of the laser beam but converges to a constant value.

[0061] In one embodiment of the present disclosure, a laser beam adjusted to have a beam intensity greater than or equal to the saturation intensity may be a non-sine wave laser beam containing higher-order harmonics components. For example, the higher-order harmonics components may include frequency components such as two, three, or five times the fundamental frequency of light.

[0062] In one embodiment of the present disclosure, the imaging device (100) can adjust the intensity of a laser beam irradiated from a light source (10) to a saturation intensity greater than that which saturates the excited state of a phosphor, while simultaneously adjusting the pulse width or exposure time of the laser beam. The imaging device (100) can irradiate a laser beam having a pulsed output with a beam intensity greater than or equal to the saturation intensity toward a light modulation module (50).

[0063] An imaging device (100) can modulate a laser beam irradiated from a light source (10) into structured light using an optical modulation module (50). A laser beam having a beam intensity greater than or equal to the saturation intensity and having a pulse width or exposure time adjusted can be irradiated to the optical modulation module (50). Accordingly, the optical modulation module (50) can modulate a laser beam having a beam intensity greater than or equal to the saturation intensity and having a pulse width or exposure time adjusted into structured light. The optical modulation module (50) may include a diffraction grating or a spatial light modulator.

[0064] The imaging device (100) can modulate a laser beam into a plurality of structured lights having different pattern directions and pattern phases. The structured light is light modulated to have a spatial pattern (e.g., a grid, stripes, dots, etc.) rather than ordinary planar light, and the spatial pattern includes a spatial frequency and can provide an effect that enables the detection of high-frequency information of a biological sample beyond the diffraction limit. The spatial pattern of the structured light can form a Moire pattern when mixed with the biological sample (200), thereby allowing the high-frequency information of the biological sample (200) to be folded in to a low frequency and detected. In the present disclosure, the spatial pattern of the structured light may be referred to as a grid pattern or an illumination pattern.

[0065] As the imaging device (100) modulates a laser beam of saturation intensity or higher into structured light, the structured light irradiated onto the biological sample (200) may be structured light in the form of a non-sine wave containing higher-order harmonics components. In the present disclosure, structured light modulated from a laser beam of less than saturation intensity may be referred to as linear structured light or non-saturated structured light. In the present disclosure, structured light modulated from a laser beam of saturation intensity or higher may be referred to as saturated structured light.

[0066] The imaging device (100) can irradiate a plurality of saturated structured lights, modulated by an optical modulation module (50) to have different pattern directions and pattern phases, toward a biological sample (200). The imaging device (100) can control the saturated structured lights generated (or modulated) from a laser beam emitted from a light source (10) and having a controlled beam intensity and pulse width or exposure time, so that they are ultimately irradiated toward the biological sample (200). At this time, the imaging device (100) can generate (or modulate) a plurality of saturated structured lights having different pattern directions and pattern phases from a laser beam emitted from a light source (10) and having a controlled beam intensity and pulse width or exposure time, and irradiate these plurality of saturated structured lights toward the biological sample (200).

[0067] An imaging device (100) can receive a fluorescent signal excited from a biological sample (200) using an image sensor (20) as a plurality of saturated structured lights are irradiated onto a biological sample (200). The imaging device (100) can obtain a plurality of fluorescent images corresponding to the plurality of structured lights by imaging the received fluorescent signals (or converting the fluorescent signals into electrical signals).

[0068] According to one embodiment of the present disclosure, an imaging device (100) can generate a result image based on a plurality of acquired fluorescent images. The imaging device (100) can generate a result image having a higher spatial resolution than the plurality of fluorescent images by processing the plurality of fluorescent images in the frequency space through an image synthesis device. The imaging device (100) can obtain frequency components corresponding to the plurality of fluorescent images by performing a Fourier transform on the plurality of fluorescent images, and can obtain frequency components corresponding to the result image by merging the frequency components. The imaging device (100) can obtain a result image by performing an inverse Fourier transform on the frequency components corresponding to the result image.

[0069] DNA sequencing imaging is a method of identifying the type of base by distinguishing and identifying the fluorescent signals of fluorescent substances (fluorophores) that emit different wavelengths (colors) bound to four types of bases. Since the performance of the imaging device (100) (e.g., fluorescent imaging device) is highly relevant in terms of sequencing accuracy, measurement time, and productivity, the imaging device (100) needs to acquire a precise and accurate fluorescent image.

[0070] In order to increase the accuracy of the fluorescence image acquired by the imaging device (100), a method using a high numeric aperture (NA) objective lens with high resolution can be considered, but in this case, there may be a trade-off problem with the depth of focus and the shooting range.

[0071] In order to increase the accuracy of the fluorescence image obtained by the imaging device (100), methods using confocal microscopy or two-photon fluorescence excitation microscopy can be considered, but since the focal plane must be scanned, it may not be suitable for sequencing technology that requires analyzing the base sequence in a short time.

[0072] Among high-resolution fluorescence microscopy techniques, structured illumination microscopy (SI) is a method capable of obtaining fluorescence signals in a single step without scanning. Structural illumination microscopy enables the acquisition of high-resolution images by irradiating the focal plane of a biological sample with spatially patterned excitation light and synthesizing the acquired signals in the frequency domain.

[0073] According to one embodiment of the present disclosure, in order to further overcome the resolution limit of a structured light microscope, an imaging device (100) may use saturated structured light. Saturated structured light can improve resolution by creating a non-linear relationship with the amount of fluorescence by saturating the excited state of the phosphor. For example, while resolution can be improved by up to 2 times when using linear structured light, resolution can be improved by up to 5.5 times when using saturated structured light.

[0074] Meanwhile, if the imaging device (100) uses saturated structured light, it is inevitable that the intensity of the beam irradiated onto the biological sample (200) will increase, and thus it may be unsuitable for imaging living cells due to phototoxicity caused by high illumination intensity. Phototoxicity refers to a phenomenon in which the natural state of cells is disrupted or cells die due to light-induced damage. In addition, if the imaging device (100) uses saturated structured light, it is inevitable that the intensity of the beam irradiated onto the biological sample will increase, and thus information loss may occur due to the extinction of fluorescence caused by photobleaching. Photobleaching refers to a phenomenon in which a photochemical change occurs in a dye or fluorescent pigment that can no longer permanently emit fluorescence in optics.

[0075] According to one embodiment of the present disclosure, the imaging device (100) can reduce light exposure of a biological sample (200) to saturated structured light by generating saturated structured light from a pulsed laser or a time-modulated continuous wave laser. By using a plurality of saturated structured lights having various pattern directions and pattern phases while reducing light exposure of the biological sample (200) to saturated structured light, the effect of increased resolution can be maintained while minimizing photothermal damage to the biological sample (200) and photobleaching of the phosphor. Accordingly, the imaging device (100) can observe the biological sample (200) for a long time with improved stability of the phosphor.

[0076] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood from the present disclosure by those skilled in the art to which the present disclosure pertains.

[0077] FIG. 2 is a flowchart illustrating a method of operation of an imaging device (100) according to one embodiment of the present disclosure. Hereinafter, the function and / or operation of the imaging device (100) will be described in detail with reference to the flowchart of FIG. 2 together with the embodiment illustrated in FIG. 1.

[0078] Referring to FIG. 2, the method of operation of the imaging device (100) may include steps S210 to S240. In one embodiment of the present disclosure, steps S210 to S240 may be executed by at least one processor included in the imaging device (100). The method of operation of the imaging device (100) is not limited to that shown in FIG. 2, and in one or more embodiments, may further include steps not shown in FIG. 2.

[0079] In step S210 of FIG. 2, an imaging device (100) according to one embodiment of the present disclosure can adjust the intensity of a laser beam irradiated by a light source (10) to a saturation intensity greater than that which saturates the excited state of a phosphor, and can adjust the pulse width or exposure time of the laser beam.

[0080] A method of operation of an imaging device (100) according to one embodiment of the present disclosure may include the step of adjusting the intensity of a laser beam irradiated by a light source (10) to a saturation intensity greater than that which saturates the excited state of a phosphor, and adjusting the pulse width or exposure time of the laser beam.

[0081] In one embodiment of the present disclosure, the imaging device (100) may control the intensity of the laser beam to a beam intensity greater than the saturation intensity that saturates the excited state of the phosphor. When a laser beam greater than the saturation intensity is irradiated onto a biological sample (200), the amount of fluorescence generated from the biological sample (200) may respond non-linearly to the intensity of the laser beam. The laser beam greater than the saturation intensity may be a non-sine wave type laser beam containing higher-order harmonics components.

[0082] In one embodiment of the present disclosure, the imaging device (100) may irradiate a pulsed laser using a light source (10). Alternatively, in one embodiment of the present disclosure, the imaging device (100) may irradiate a continuous wave laser using a light source (10). In this case, the imaging device (100) may irradiate a continuous wave laser (or a time-modulated continuous wave laser) having a pulsed output by controlling the exposure time of the continuous wave laser. A detailed description thereof will be provided with reference to FIGS. 8 to 10.

[0083] According to one embodiment of the present disclosure, the imaging device (100) can irradiate a laser beam toward a light modulation module (50) having a pulsed output while having a beam intensity greater than or equal to the saturation intensity by controlling the intensity of the laser beam irradiated from the light source (10) and controlling the pulse width or exposure time of the laser beam.

[0084] In step S220 of FIG. 2, an imaging device (100) according to one embodiment of the present disclosure can use a light modulation module (50) to modulate a laser beam, which has a beam intensity greater than or equal to the saturation intensity and has a pulse width or exposure time adjusted, into a plurality of structured lights having different pattern directions and pattern phases, and irradiate the plurality of structured lights toward a biological sample (200).

[0085] A method of operation of an imaging device (100) according to one embodiment of the present disclosure may include the step of using a light modulation module (50) to modulate a laser beam, which has a beam intensity greater than or equal to the saturation intensity and has a pulse width or exposure time adjusted, into a plurality of structured lights having different pattern directions and pattern phases, and irradiating the plurality of structured lights toward a biological sample (200).

[0086] In one embodiment of the present disclosure, an optical modulation module (50) can modulate a laser beam into structured light having a spatial pattern (e.g., a grating, stripes, dots, etc.). A laser beam having a beam intensity greater than or equal to the saturation intensity and having a pulse width or exposure time adjusted may be delivered to the optical modulation module (50), and the optical modulation module (50) can modulate such a laser beam into structured light. Through this, the structured light generated using the optical modulation module (50) may have a pulsed output. Additionally, the structured light generated using the optical modulation module (50) may be non-sine wave structured light (or may be referred to as saturated structured light) containing higher-order harmonics components. A detailed description thereof will be provided later with reference to FIG. 5.

[0087] In one embodiment of the present disclosure, an imaging device (100) may use a light modulation module (50) to modulate a laser beam irradiated from a light source (10) into a plurality of structured lights having different pattern directions and pattern phases, and irradiate the plurality of structured lights toward a biological sample (200). The plurality of structured lights may be implemented as a plurality of grid patterns having different pattern directions and pattern phases. More specifically, the plurality of structured lights may be implemented as a plurality of grid patterns having different pattern phases for each of the different pattern directions.

[0088] In one embodiment of the present disclosure, the imaging device (100) may irradiate a plurality of structured lights toward a biological sample (200) in at least one of an epi-illumination method, a trans-illumination method, or a light sheet illumination method. A detailed description thereof will be provided later with reference to FIGS. 12 to 15.

[0089] In one embodiment of the present disclosure, the imaging device (100) may irradiate a plurality of structured lights toward the biological sample (200) at each of different focal positions along the Z-axis direction corresponding to the depth direction of the biological sample (200). By doing so, the imaging device (100) may acquire a plurality of fluorescent images containing three-dimensional information. A detailed description thereof will be provided later with reference to FIGS. 17 to 19.

[0090] In one embodiment of the present disclosure, the imaging device (100) may irradiate a plurality of structured lights toward a biological sample (200) for each of different incident angles. By doing so, the imaging device (100) may acquire a plurality of fluorescent images containing three-dimensional information. A detailed description thereof will be provided later with reference to FIGS. 20 and 21.

[0091] In step S230 of FIG. 2, an imaging device (100) according to one embodiment of the present disclosure can acquire a plurality of fluorescent images corresponding to the pattern direction and pattern phase of a plurality of patterned lights using an image sensor (20).

[0092] A method of operation of an imaging device (100) according to one embodiment of the present disclosure may include the step of acquiring a plurality of fluorescent images corresponding to the pattern direction and pattern phase of a plurality of patterned light using an image sensor (20).

[0093] In one embodiment of the present disclosure, an imaging device (100) can acquire a plurality of fluorescent images corresponding to a plurality of structured lights by using an image sensor (20) to receive a fluorescent signal emitted as a plurality of structured lights having different pattern directions and pattern phases are irradiated. That is, the imaging device (100) can acquire a plurality of fluorescent images corresponding to a plurality of grid patterns having different pattern directions and pattern phases.

[0094] According to one embodiment of the present disclosure, an imaging device (100) can acquire multiple fluorescent images containing different frequency components by irradiating a biological sample (200) with multiple structured lights having different pattern directions and pattern phases of grid patterns. Through this, the imaging device (100) can obtain various structural information of the biological sample (200). For example, the imaging device (100) can obtain different information regarding the microstructure of the sample and specific molecular distribution through multiple fluorescent images containing different frequency components. Accordingly, the frequency transfer region of the imaging device (100) can be expanded, thereby enabling the acquisition of fluorescent images with improved resolution.

[0095] Additionally, as the imaging device (100) irradiates a biological sample (200) with a plurality of saturated structured lights containing higher-order harmonic components, the frequency transfer region of the imaging device (100) may be extended compared to the case where linear structured lights are irradiated. A detailed explanation regarding this will be provided later with reference to FIGS. 6 and FIGS. 7.

[0096] In step S240 of FIG. 2, an imaging device (100) according to one embodiment of the present disclosure can synthesize a plurality of fluorescent images in the frequency space to obtain a result image.

[0097] A method of operation of an imaging device (100) according to one embodiment of the present disclosure may include the step of synthesizing a plurality of fluorescent images in the frequency space to obtain a result image.

[0098] The imaging device (100) can synthesize a plurality of fluorescent images in the frequency space to obtain a result image. The result image may be an image with improved resolution compared to individual plurality of fluorescent images. By obtaining the result image, the imaging device (100) can observe a biological sample (200) with higher spatial resolution.

[0099] In one embodiment of the present disclosure, the imaging device (100) can obtain frequency components corresponding to a plurality of fluorescent images by performing a Fourier transform on a plurality of fluorescent images. In one embodiment, the imaging device (100) can obtain frequency components corresponding to a result image by merging the frequency components. In one embodiment of the present disclosure, the imaging device (100) can obtain a result image by performing an inverse Fourier transform on the frequency components of the result image. A detailed description regarding this will be provided later with reference to FIG. 16.

[0100] According to one embodiment of the present disclosure, an imaging device (100) can acquire a fluorescent image by using structured light with different pattern directions and pattern phases, thereby expanding the image information of a biological sample in the frequency domain, and thus acquiring a high-resolution image that exceeds the conventional optical resolution limit. In particular, the imaging device (100) can acquire a fluorescent image by using saturated structured light containing higher harmonic components, thereby expanding the image information of a biological sample to an even higher frequency domain, and thus acquiring a super-resolution image.

[0101] According to one embodiment of the present disclosure, an imaging device (100) can acquire a fluorescent image using saturated structured light having different pattern directions and pattern phases, and at the same time, reduce light exposure of a biological sample (200) by using saturated structured light having a pulsed output. Accordingly, according to one embodiment of the present disclosure, the effect of increased resolution can be maintained while minimizing photothermal damage to the biological sample and photobleaching of the fluorescence.

[0102] According to one embodiment of the present disclosure, an imaging device (100) can acquire a plurality of fluorescent images containing three-dimensional information. In this case, the imaging device (100) can acquire a plurality of fluorescent images containing three-dimensional information by using saturated structured light having different pattern directions and pattern phases, so that the image information of the biological sample can also be extended to a higher frequency range, thereby acquiring an image containing super-resolution three-dimensional structured information. In addition, the imaging device (100) can acquire a plurality of fluorescent images containing three-dimensional information by using saturated structured light having a pulsed output, so that the effect of increasing the resolution of the image containing three-dimensional structured information is maintained, while at the same time minimizing photothermal damage to the biological sample and photobleaching problems of the fluorescence.

[0103] FIG. 3 is a block diagram illustrating the components of an imaging device (100) according to one embodiment of the present disclosure.

[0104] Referring to FIG. 3, an imaging device (100) according to one embodiment of the present disclosure may include a light source (10), an image sensor (20), a processor (30), a memory (40), a first light modulation module (50), and a second light modulation module (60). The light source (10), the image sensor (20), the processor (30), the memory (40), the first light modulation module (50), and the second light modulation module (60) may each be electrically and / or physically connected to each other. Meanwhile, in one embodiment of the present disclosure, the second light modulation module (60) may be omitted. FIG. 3 illustrates only essential components for explaining the function and / or operation of the imaging device (100), and the components included in the imaging device (100) are not limited to those illustrated in FIG. 3.

[0105] According to one embodiment of the present disclosure, the light source (10) may be a device or component that emits a laser beam. The light source (10) may be a device that irradiates laser light onto a biological sample (200) on a flow cell.

[0106] According to one embodiment of the present disclosure, the light source (10) may be a pulsed laser light source that emits a pulsed laser. Alternatively, according to one embodiment of the present disclosure, the light source (10) may be a continuous wave laser light source that emits a continuous wave laser.

[0107] In one embodiment of the present disclosure, the light source (10) may emit polychromatic light. The light emitted by the light source (10) may be broadband light or light whose wavelength changes over time. The wavelength band of the broadband light may be controlled and adjusted to within the range of the excitation wavelength of the fluorescent material treated on the biological sample (200). Additionally, the wavelength range of the light whose wavelength changes over time may be controlled and adjusted to within the range of the excitation wavelength of the fluorescent material treated on the biological sample (200).

[0108] In one embodiment of the present disclosure, the light source (10) may emit single-wavelength or multi-wavelength light and may be combined with a filter or a spectrometer to irradiate light having specific spectral characteristics.

[0109] In one embodiment of the present disclosure, the imaging device (100) may include a plurality of light sources (10). An embodiment in which the imaging device (100) includes a plurality of light sources (10) will be described in detail with reference to FIG. 24.

[0110] In one embodiment of the present disclosure, the image sensor (20) can detect a fluorescence signal. The image sensor (20) can obtain information about the composition of the biological sample (200) based on the detected fluorescence signal. For example, the image sensor (20) can obtain a plurality of images containing information about the location where the fluorescence signal was emitted from the biological sample (200) based on at least one beam.

[0111] A phosphor bound to a biological sample (200) can be excited by a laser beam (or saturated structured light modulated from a laser beam) irradiated through a light source (10). The laser beam (or saturated structured light modulated from a laser beam) irradiated from the light source (10) can correspond to the excitation light of the biological sample (200). When the beam corresponding to the excitation light is irradiated onto the biological sample (200), emission light of a wavelength different from the wavelength of the irradiated excitation light may be generated. The emission light may have a longer wavelength than the beam corresponding to the excitation light. An image sensor (20) can receive the emission light emitted from the biological sample (200).

[0112] In one embodiment of the present disclosure, the image sensor (20) may include a plurality of image sensors (20). An embodiment in which the imaging device (100) includes a plurality of image sensors (20) will be described in detail with reference to FIG. 24.

[0113] In one embodiment of the present disclosure, the first light modulation module (50) may be a light modulation module that modulates a laser beam emitted from a light source (10) into structured light. The first light modulation module (50) may correspond to the light modulation module (50) described with reference to FIGS. 1 and FIGS. 2. For example, the first light modulation module (50) may include a diffraction grating or a spatial light modulator.

[0114] A diffraction grating can be composed of periodic lines or groove structures arranged at regular intervals. Through a structure with a periodic pattern, incident light can be diffracted in a specific direction, forming rays of multiple diffraction orders. In this case, structured light can be generated as rays of different diffraction orders interfere with each other.

[0115] A spatial light modulator may be a digital or analog optical device for precisely controlling the phase, amplitude, polarization, or intensity of light spatially within an optical system. For example, a spatial light modulator may include a liquid crystal-based phase modulator (LC-SLM), a digital micromirror device (DMD), etc.

[0116] A liquid crystal-based phase modulator is composed of liquid crystal molecules that are aligned by electrical signals on a pixel-by-pixel basis on a transparent substrate, and can control the phase of polarized light. As the arrangement of liquid crystal molecules changes according to the voltage applied to each pixel, the phase of the passing light can vary depending on the position, thereby enabling the generation of a precise diffraction pattern.

[0117] A digital micromirror device comprises microscopic mirrors arranged in a grid pattern, each of which is electrically controlled independently to adjust the direction in which light is reflected. Each microscopic mirror has two tilt states of ±θ, and light can be projected or blocked by controlling the direction of reflection. Structured light can be formed by directly loading a grid pattern onto the microscopic mirror array, or indirectly by using a programmed diffraction pattern to selectively pass only specific diffraction values ​​(e.g., ±1) through spatial filtering.

[0118] In one embodiment of the present disclosure, the second optical modulation module (60) can control at least one of the pulse width or pulse period of a laser beam emitted from a light source (10). The second optical modulation module (60) can control at least one of the pulse width or pulse period of a pulsed laser emitted from a light source (10).

[0119] In one embodiment of the present disclosure, the second optical modulation module (60) can control the exposure time of a laser beam emitted from a light source (10). The second optical modulation module (60) can control the exposure time of a continuous wave laser emitted from a light source (10). By controlling the exposure time of the continuous wave laser, the second optical modulation module (60) can modulate the output form of the continuous wave laser into a pulse form. At this time, by controlling the exposure time of the continuous wave laser, the second optical modulation module (60) can control at least one of the pulse width or the pulse period. In other words, the second optical modulation module (60) can control the exposure time of the continuous wave laser to control at least one of the pulse width or the pulse period of the continuous wave laser.

[0120] In one embodiment of the present disclosure, the second light modulation module (60) can control the intensity of a laser beam emitted from a light source (10). The second light modulation module (60) can control the intensity of a laser beam (e.g., a pulsed laser or a continuous wave laser) emitted from a light source (10) to be greater than or equal to a saturation intensity that saturates the excited state of a phosphor coupled to a biological sample (200). A detailed description of the second light modulation module (60) will be provided later with reference to FIGS. 9 and FIGS. 10.

[0121] The processor (30) can execute one or more instructions of a program stored in memory (40). The processor (30) may be composed of hardware components that perform arithmetic, logic, and input / output operations and image processing. Although the processor (30) is depicted as a single element in FIG. 3, it is not limited thereto. In one embodiment of the present disclosure, the processor (30) may be composed of one or more elements.

[0122] The processor (30) may include various processing circuits and / or multiple processors. For example, the term 'processor' as used in the present disclosure, including in the claims, may include at least one processor (30) and various processing circuits. In the at least one processor (30), one or more processors may be configured to perform the various functions described herein in a distributed manner, individually and / or collectively. As used in the present disclosure, 'processor', 'at least one processor', and 'one or more processors' may be configured to perform various functions. However, these terms cover, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor can perform all functions. Additionally, the at least one processor (30) may include a combination of processors performing various functions of the disclosed functions in a distributed manner. The at least one processor (30) may execute program instructions to achieve or perform various functions.

[0123] In an embodiment of the present disclosure, the processor (30) may store one or more instructions in an internally provided memory (40) and control the operation of the imaging device (100) to be performed by executing one or more instructions stored in the internally provided memory. That is, the processor (30) may perform a predetermined operation by executing at least one instruction or program stored in the internal memory provided within the processor (30) or in the memory (30).

[0124] The processor (30) may be implemented as a general-purpose processor such as a CPU (Central Processing Unit), AP (Application Processor), DSP (Digital Signal Processor), a graphics-dedicated processor such as a GPU (Graphic Processing Unit) or VPU (Vision Processing Unit), or an artificial intelligence-dedicated processor such as an NPU (Neural Processing Unit). The processor (30) may be controlled to process input data according to predefined operation rules or an artificial intelligence model. Alternatively, if the processor (30) is an artificial intelligence-dedicated processor, the artificial intelligence-dedicated processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0125] The memory (40) may be composed of at least one type of storage medium, such as a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), or an optical disk.

[0126] The memory (40) may store instructions related to functions and / or operations for the imaging device (100) to perform DNA sequencing imaging or multi-omics. In one embodiment of the present disclosure, the memory (40) may store at least one of instructions, an algorithm, a data structure, program code, and an application program that can be read by the processor (30). The instructions, algorithm, data structure, and program code stored in the memory (40) may be implemented in a programming or scripting language such as, for example, C, C++, Java, assembler, etc.

[0127] The processor (30) can be implemented by executing instructions or program code stored in memory (40). Hereinafter, the functions and / or operations performed by the processor (30) by executing instructions or program code stored in memory (40) will be described in detail.

[0128] In one embodiment of the present disclosure, the processor (30) can adjust the intensity of the laser beam irradiated by the light source (10) to a saturation intensity greater than that which saturates the excited state of the phosphor, and can adjust the pulse width or exposure time of the laser beam. The processor (30) can use a first light modulation module (50) to modulate the laser beam, which has a beam intensity greater than the saturation intensity and has a pulse width or exposure time adjusted, into a plurality of structured lights having different pattern directions and pattern phases, and irradiate the plurality of structured lights toward the biological sample (200). The processor (30) can use an image sensor (20) to acquire a plurality of fluorescent images corresponding to the pattern directions and pattern phases of the plurality of structured lights. The processor (30) can synthesize the plurality of fluorescent images in the frequency space to acquire a result image.

[0129] In one embodiment of the present disclosure, the processor (30) can control at least one of the pulse width or pulse period of the pulse laser and the intensity of the beam using a second optical modulation module (60).

[0130] In one embodiment of the present disclosure, the processor (30) controls the exposure time of the continuous wave laser using the second optical modulation module (60) to control at least one of the pulse period or pulse width of the continuous wave laser and can control the intensity of the beam of the continuous wave laser.

[0131] In one embodiment of the present disclosure, the processor (30) may irradiate a plurality of structured lights toward the biological sample (200) at each of different focal positions along the Z-axis direction corresponding to the depth direction of the biological sample (200). The processor (30) may acquire a plurality of fluorescent images containing three-dimensional information.

[0132] In one embodiment of the present disclosure, the processor (30) may irradiate a plurality of structured lights toward a biological sample (200) for each of different incident angles. The processor (30) may acquire a plurality of fluorescent images containing three-dimensional information.

[0133] In one embodiment of the present disclosure, the processor (30) can obtain frequency components corresponding to a plurality of fluorescent images by performing a Fourier transform on a plurality of fluorescent images. The processor (30) can obtain frequency components corresponding to a result image by merging the frequency components. The processor (30) can obtain a result image by performing an inverse Fourier transform on the frequency components of the result image.

[0134] FIG. 4 is a diagram showing the operation method of an imaging device (100) according to one embodiment of the present disclosure.

[0135] Referring to FIG. 4, an imaging device (100) according to one embodiment of the present disclosure may include a light source (10), a lens (410), a dichroic filter (420), an objective lens (430), a band-pass filter (440), a tube lens (450), a first light modulation module (50), an image sensor (20), and an image synthesis device (not shown). However, not all components shown in FIG. 4 are essential components. The imaging device (100) may be implemented with more components than shown, or with fewer components. A detailed description of the components described above with reference to FIG. 1 to 3 is omitted.

[0136] A laser beam irradiated from a light source (10) can be modulated into structured light by a first light modulation module (50). The structured light modulated by the first light modulation module (50) can pass through a lens (410) located between the first light modulation module (50) and the objective lens (430) and be refracted toward the objective lens (430) and the biological sample (200).

[0137] In one embodiment of the present disclosure, the lens (410) may be a Fourier lens and may convert diffraction components generated from the first light modulation module (50) (e.g., a diffraction grating or a spatial light modulator) into the spatial frequency domain so as to place ±1st order diffracted light on the back focal plane of the objective lens (430). By doing so, an interference pattern of a preset period and direction may be formed on the plane of the biological sample (200).

[0138] The dichroic filter (420) may be a component of an optical system having the characteristic of selectively reflecting or transmitting light of a specific wavelength. For example, the dichroic filter (420) may be a dichroic mirror or a polarization splitter.

[0139] In one embodiment of the present disclosure, a dichroic filter (420) can separate structured light irradiated toward a biological sample (200) from emitted light from the biological sample (200). For example, the dichroic filter (420) can irradiate structured light toward the biological sample (200) by transmitting structured light, and the emitted light generated by the irradiation of structured light toward the biological sample (200) can be reflected toward a bandpass filter (440).

[0140] For example, the dichroic filter (420) may reflect light of a wavelength longer than the cutoff wavelength and transmit light of a wavelength shorter than the cutoff wavelength. Alternatively, for example, the dichroic filter (420) may reflect light of a wavelength shorter than the cutoff wavelength and transmit light of a wavelength longer than the cutoff wavelength. In one embodiment of the present disclosure, a fluorescence signal emitted from a biological sample (200) may pass through a bandpass filter (440) as it is reflected by the dichroic filter (420).

[0141] In one embodiment of the present disclosure, a dichroic filter (420) transmits excitation light toward a biological sample (200) and reflects emission light having a relatively long wavelength emitted from the biological sample (200), thereby enabling the acquisition of a fluorescence image of the biological sample (200) through an image sensor (20). In one embodiment of the present disclosure, the excitation light is light that excites a phosphor bound to the biological sample (200) to cause it to emit light, and may have short wavelength (high energy) characteristics. The emission light is light emitted after the phosphor contained in the biological sample (200) is excited, and may have a longer wavelength (lower energy) than the excitation light.

[0142] The objective lens (430) may be a component that performs the function of collecting a fluorescent signal emitted from a biological sample (200). The objective lens (430) may include at least one lens. In one embodiment of the present disclosure, the objective lens (430) may be positioned between a light source (10) and a biological sample (200). The objective lens (430) may condense a beam emitted from the light source (10) onto the biological sample (200). As the objective lens (430), a lens capable of creating a small focusing area may be used. The objective lens (430) may temporarily bring the energy of the biological sample (200) into an excited state by irradiating the biological sample (200) with excitation light. The biological sample (200) may emit emission light, which is a constant fluorescent light, as it returns to a stable state by releasing the absorbed energy again.

[0143] The emitted light has a wavelength that is shifted slightly toward the red direction compared to the excitation light, and this change can be called Stokes Shift. That is, according to Stokes Shift, the emitted light may have a slightly longer wavelength compared to the excitation light (or the beam emitted from the light source (10)).

[0144] In one embodiment of the present disclosure, the bandpass filter (440) can selectively transmit (or pass) only the wavelength specified by the dichroic filter (420) and transmit it to the image sensor (20). The bandpass filter (440) can selectively transmit only light of the emission wavelength band. The bandpass filter (440) can selectively transmit only light of the first emission wavelength band and block (or absorb or reflect) light of the second emission wavelength band.

[0145] A bandpass filter (440) can be placed between the dichroic filter (420) and the image sensor (20). A fluorescent signal emitted from a biological sample (200) can pass through the bandpass filter (440) and be incident on the tube lens (450).

[0146] In one embodiment of the present disclosure, the emitted light emitted from a biological sample (200) and reflected by a dichroic filter (420) may contain some form of optical noise. A bandpass filter (440) can remove the optical noise contained in the emitted light and transmit only the emitted light of a desired wavelength to an image sensor (20). Through this, the resolution of the image obtained by the image sensor (20) can be improved.

[0147] The tube lens (450) can perform the function of converting the intermediate image formed by the objective lens (430) into a focused image. The tube lens (450) can transmit the converted image to the image sensor (20). The tube lens (450) can focus the emitted light that has passed through the bandpass filter (440) to the image sensor (20). The emitted light (e.g., a fluorescent signal) that has passed through the tube lens (450) can be focused to the focal point of the tube lens (450). The tube lens (450) may be a lens for consistently correcting the focus of the emitted light (e.g., a fluorescent signal). In one embodiment, the emitted light (e.g., a fluorescent signal) that has passed through the tube lens (450) can be focused to the focal point of the tube lens (450). The image sensor (20) can detect the emitted light (e.g., a fluorescent signal) that has passed through the tube lens (450).

[0148] In one embodiment of the present disclosure, an image synthesis device may generate a result image based on a plurality of images acquired by an image sensor (20). The image synthesis device may be a component included in the imaging device (100) or may exist as a separate electronic device.

[0149] In one embodiment of the present disclosure, the image synthesis device may be a computer system or part of a computer system connected to an imaging device (100) via a wired or wireless communication network. The image synthesis device may include a processor comprising one or more cores.

[0150] In one embodiment of the present disclosure, when the image synthesis device is configured as an electronic device separate from the imaging device (100), the image synthesis device may represent various types of electronic devices capable of synthesizing images. The image synthesis device may represent various hardware and / or software elements that perform the function of generating a result image based on fluorescent image data. For example, the image synthesis device may include devices capable of synthesizing images, such as a PC, server, smartphone, tablet PC, laptop PC, etc., but is not limited thereto. For example, the image synthesis device may include a communication interface, memory, and a processor, but the components are not limited thereto. In one embodiment of the present disclosure, the image synthesis device may synthesize a plurality of images based on artificial intelligence technology and output a result image.

[0151] In one embodiment of the present disclosure, the image synthesis device may be implemented in the form of a graphics-dedicated processor such as a GPU (Graphics Processing Unit) or VPU (Vision Processing Unit), an artificial intelligence-based processor such as an NPU (neural processing unit), or a Field-Programmable Gate Array (FPGA), but is not limited thereto.

[0152] In one embodiment of the present disclosure, whether the image synthesis device is implemented as a hardware / software element or as an electronic device, the image synthesis device may receive a fluorescent image by being connected to an image sensor (20) via a wired or wireless connection. The image synthesis device may synthesize a result image based on a plurality of fluorescent images obtained from the image sensor (20). In one embodiment of the present disclosure, the result image may represent an image with a higher resolution than the plurality of fluorescent images.

[0153] In one embodiment of the present disclosure, the biological sample (200) may be a sample treated with a plurality of fluorescent materials. The fluorescent material may be a material that emits a fluorescent signal when irradiated with light of a predetermined wavelength or a predetermined range of wavelengths. The predetermined fluorescent material may be pre-treated on the biological sample (200) before the imaging device (100) is used.

[0154] The wavelength range of the fluorescence signal emitted from the biological sample (200) may be independent of the wavelength range of the light irradiated onto the biological sample (200). For example, a fluorescent material bound to the biological sample (200) may absorb light within a specific absorption wavelength range. As light within a specific absorption wavelength range is absorbed by the fluorescent material, the fluorescent material may emit a fluorescence signal within a specific emission wavelength range. The absorption wavelength range of the light absorbed by the fluorescent material and the emission wavelength range of the fluorescence signal emitted by the fluorescent material may be different from each other. In one embodiment of the present disclosure, the absorption wavelength range of the light absorbed by the fluorescent material may be the range of the excitation wavelength of the fluorescent material. The range of the excitation wavelength of the fluorescent material may vary depending on the type of fluorescent material.

[0155] FIG. 5 is a drawing for explaining the non-linear characteristics of structured light according to one embodiment of the present disclosure.

[0156] Referring to FIG. 5, the imaging device (100) may include a first light modulation module (510), a lens (520) (or Fourier lens), and an objective lens (530). FIG. 5 illustrates, as an example, the application of a transmission diffraction grating as the first light modulation module (510). However, the embodiment is not limited thereto, and a radial diffraction grating may be applied as the first light modulation module (510), or a spatial light modulator may be applied.

[0157] In one embodiment of the present disclosure, when a laser beam irradiated from a light source (10) is incident on a first light modulation module (520), it can be separated into a plurality of diffracted lights, including ±1st-order diffracted lights. The ±1st-order diffracted lights can be dispersed at different angles. The ±1st-order diffracted lights can be aligned at different angles as they pass through a lens (520) (or Fourier lens) so as to be focused by an objective lens (530). As the ±1st-order diffracted lights pass through the objective lens (530), they can be focused onto a plane of a biological sample (200). As the ±1st-order diffracted lights interfere with each other on the plane of the biological sample (200), a periodic grid pattern (or illumination pattern) can be formed.

[0158] FIG. 5 illustrates, exemplarily, a grid pattern of linear structured light (501) and a grid pattern of saturated structured light (502). In FIG. 5, linear structured light (501) refers to structured light modulated from a laser beam in which the intensity of the irradiated beam is less than the saturation intensity that saturates the excited state of the phosphor bound to the biological sample (200). In FIG. 5, saturated structured light (502) refers to structured light modulated from a laser beam in which the intensity of the irradiated beam is greater than or equal to the saturation intensity that saturates the excited state of the phosphor bound to the biological sample (200).

[0159] As illustrated in FIG. 5, the linear structured light (501) can correspond to a sinusoidal light. As illustrated in FIG. 5, the saturated structured light (502) can correspond to a non-sinusoidal light by applying high-intensity light to the structured light to induce a saturated phosphor.

[0160] Linear structured light (501) or saturated structured light (502) can excite a fluorescence emitting agent bound to a biological sample (200). Under normal conditions, such as when linear structured light is irradiated, the intensity of the fluorescence is linearly proportional to the intensity of the linear structured light, but when saturated structured light is irradiated, the fluorescence reaches a saturated state, so the intensity of the fluorescence does not increase linearly but can be flattened at a certain level. That is, when saturated structured light (502) is irradiated, a non-linear fluorescence response may occur.

[0161] As the intensity of structured light (or the intensity of the laser beam) increases, the pattern of the structured light can change from a sinusoidal form to a non-sinusoidal form. Since saturated structured light in a non-sinusoidal form contains higher-order harmonic information compared to sinusoidal structured light, using saturated structured light allows for the acquisition of additional high-frequency information compared to using linear structured light, thereby enabling higher resolution. In other words, by utilizing the non-linear fluorescence response caused by fluorescence saturation, high spatial frequency components that cannot be detected by conventional Structured Illumination Microscopy (SIM) can be recovered. This can be implemented in Saturated Structured Illumination Microscopy (SSIM), overcoming the resolution limitations of existing structured illumination microscopy technology and enabling super-resolution imaging.

[0162] Figure 6 is a diagram illustrating graphs visualizing frequency transfer characteristics according to the lighting used in the imaging device. The graphs shown in Figure 6 are graphs visualizing the frequency transfer region. The x-axis of the graph represents the spatial frequency in the horizontal direction, and the y-axis of the graph represents the spatial frequency in the vertical direction.

[0163] The 601 graph shown in FIG. 6(a) illustrates the frequency transfer region in an imaging device using unstructured illumination (e.g., a light-field imaging device). The circles shown in the 601 graph represent the passband of the optical system, i.e., the Optical Transfer Function (OTF). The optical transfer function is a function that indicates the frequency range that the optical system can transmit in the spatial frequency domain, and indicates that it can only accommodate frequency information within a limited radius relative to the center point k0. The frequency transfer region shown in the 601 graph represents the maximum range of spatial frequencies that the optical system can detect. In an imaging device using unstructured illumination, high-frequency components exceeding the spatial frequency region shown in the 601 graph cannot be detected, resulting in the acquisition of a fluorescence image with limited resolution.

[0164] Graph 602 shown in Fig. 6(b) and graph 603 shown in Fig. 6(c) illustrate the frequency transfer region of an imaging device using structured light.

[0165] First, graph 602 illustrates the frequency transfer region in an imaging device that irradiates structured light having a grid pattern in a single direction. For example, the frequency transfer region illustrated in graph 602 may correspond to the frequency transfer region of an imaging device that irradiates multiple structured lights modulated to have different pattern phases (e.g., three different pattern phases) with respect to a grid pattern of striped patterns extended in the 60-degree direction.

[0166] When an imaging device irradiates a biological sample with structured light, it can generate a sideband effect that folds high-frequency information located outside the existing measurable frequency transfer region (i.e., corresponding to the frequency transfer region in a 601 graph) into the measurable frequency transfer region. Fold-in refers to the phenomenon where high-frequency components originally outside the detection range of the optical system move into the detectable range by the structured illumination. For example, the grating pattern of the structured light may have a specific spatial frequency k1. For instance, +k1 may be a frequency component corresponding to the +1st order diffracted light of the structured light illumination, and -k1 may be a frequency component corresponding to the -1st order diffracted light of the structured light illumination. When the structure of the biological sample is multiplied by the grating pattern, convolution occurs in Fourier space, causing the high-frequency information of the biological sample to shift by +k1 and -k1, thereby moving into the existing measurable frequency transfer region.

[0167] A sideband refers to a modified frequency component created when high-frequency information outside the original resolution folds in near the center of the frequency domain due to structured illumination. In the frequency space of the 602 graph, additional sidebands corresponding to +k1 and -k1 can be generated in addition to the center k0. In other words, it can be confirmed that the frequency transfer region (or the region of OTF) has expanded in the 602 graph. When comparing the 601 and 602 graphs, it can be confirmed that the frequency transfer region is expanded when biological samples are irradiated with structured light, and fluorescence images with increased resolution can be obtained compared to when non-structured light is irradiated.

[0168] Graph 603 illustrates a frequency transfer region in an imaging device that irradiates multiple structured lights having a grid pattern in multiple directions (e.g., three directions: 0-degree direction, 60-degree direction, and -60-degree direction). For example, the frequency transfer region illustrated in Graph 603 may correspond to a frequency transfer region in an imaging device that irradiates multiple structured lights modulated to have different pattern phases (e.g., three different pattern phases) for each of the grid patterns of a striped pattern extended in the 0-degree direction, a striped pattern extended in the 60-degree direction, and a striped pattern extended in the -60-degree direction.

[0169] As the imaging device acquires a fluorescence image by irradiating structured light with different pattern directions, sidebands (e.g., +k) in each pattern direction 11 , -k 11 , +k 12 , -k 12 , +k 13 , -k 13 ...can be generated. Accordingly, a wide frequency extension region can be formed, and the more evenly the three directional axes are distributed, the closer the extension can be to a circularly symmetrical shape. The maximum frequency space in the 603 graph can be extended up to twice the maximum spatial frequency in the 601 graph. That is, in the case of an imaging device irradiating structured light, a fluorescence image with up to twice the resolution can be obtained compared to an imaging device irradiating non-structured light.

[0170] In one embodiment of the present disclosure, an imaging device (100) may irradiate a plurality of structured lights having three or more different pattern directions to obtain a plurality of fluorescent images corresponding to different pattern directions. More specifically, the imaging device (100) may irradiate a plurality of structured lights having different pattern phases for each of three or more different pattern directions to obtain a plurality of fluorescent images corresponding to different pattern directions and different pattern phases. As illustrated in the graph 603, when using a plurality of structured lights having three or more pattern directions, there is a high possibility that the frequency transfer region will be extended across all directions within the frequency space.

[0171] According to one embodiment of the present disclosure, an imaging device (100) can obtain a plurality of fluorescent images corresponding to a plurality of structured lights composed of different grid patterns by irradiating a biological sample (200) with structured light having different pattern directions and pattern phases. Through this, the image information of the biological sample (200) can be expanded in the frequency domain, and thus the imaging device (100) can obtain high-resolution images that exceed the conventional optical resolution limit.

[0172] Figure 7 is a diagram illustrating graphs visualizing frequency transfer characteristics according to the lighting used in the imaging device. The graphs shown in Figure 7 are graphs visualizing the frequency transfer region. The x-axis of the graph represents the spatial frequency in the horizontal direction, and the y-axis represents the spatial frequency in the vertical direction.

[0173] The 701 graph shown in FIG. 7(a) illustrates a frequency transfer region in an imaging device that irradiates linear structured light (710) in the form of a sinusoidal wave having a grid pattern in a single direction. For example, the frequency transfer region shown in the 701 graph may correspond to the frequency transfer region of an imaging device that irradiates multiple structured lights modulated to have different pattern phases (e.g., three different pattern phases) with respect to a grid pattern of striped patterns extended in the 60-degree direction.

[0174] As shown in graph 701, in the frequency space, additional sidebands corresponding to +k1 and -k1 can be generated in addition to the center k0, so it can be confirmed that the frequency transmission area has been expanded. Since the same explanation regarding graph 602 in Fig. 6 can be applied to graph 701, a detailed explanation will be omitted.

[0175] The 702 graph shown in FIG. 7(b) illustrates the frequency transfer region of an imaging device that irradiates a non-sine wave type saturated structured light (720) having a grid pattern in a single direction. For example, the frequency transfer region shown in the 702 graph may correspond to the frequency transfer region of an imaging device that irradiates a plurality of saturated structured lights modulated to have different pattern phases (e.g., three different pattern phases) with respect to a grid pattern of striped patterns extended in the 60-degree direction.

[0176] Saturated structured illumination, in which the illumination intensity reaches saturation, may contain higher-order harmonic components. Since higher-order harmonic components include frequency components higher than the fundamental frequency (e.g., frequency components that are 2, 3, or 5 times the fundamental frequency), the frequency transfer region can be further extended along the pattern direction of the saturated structured light. Compared to graph 701, it can be observed that the frequency transfer region in graph 702 is further extended. Therefore, when irradiating biological samples with saturated structured light, fluorescence images with increased resolution can be obtained compared to when irradiating with linear structured light.

[0177] The 703 graph shown in FIG. 7(c) illustrates a frequency transfer region in an imaging device that irradiates saturated structured light (730) in the form of a non-sine wave having a grid pattern in multiple directions (e.g., three directions: 0-degree direction, 60-degree direction, and -60-degree direction). For example, the frequency transfer region shown in the 703 graph may correspond to a frequency transfer region in an imaging device that irradiates multiple saturated structured lights modulated to have different pattern phases (e.g., three different pattern phases) for each of the grid patterns of a striped pattern extended in the 0-degree direction, a striped pattern extended in the 60-degree direction, and a striped pattern extended in the -60-degree direction.

[0178] As the imaging device acquires a fluorescence image by irradiating saturated structured light having different pattern directions, additional sidebands may be generated in each pattern direction. Accordingly, a wide frequency extension region with circular symmetry may be formed. For example, the maximum frequency space in the 703 graph may be extended up to five times the maximum spatial frequency in the 701 graph. That is, in the case of an imaging device that irradiates multiple saturated structured lights having multiple pattern directions, a fluorescence image with up to five times improved resolution can be acquired compared to an imaging device using unstructured light.

[0179] In one embodiment of the present disclosure, an imaging device may irradiate a plurality of saturated structured lights having three or more different pattern directions to acquire a plurality of fluorescence images corresponding to different pattern directions. More specifically, the imaging device may irradiate a plurality of saturated structured lights having different pattern phases for each of the three or more different pattern directions to acquire a plurality of fluorescence images corresponding to different pattern directions and different pattern phases. As illustrated in the graph 703, when using a plurality of saturated structured lights having three or more pattern directions, there is a high possibility that the frequency transfer region will be extended across all directions within the frequency space.

[0180] According to one embodiment of the present disclosure, an imaging device (100) can obtain a plurality of fluorescent images corresponding to a plurality of structured lights composed of different grid patterns by irradiating a biological sample (200) with saturated structured light (730) having different pattern directions and pattern phases. Through this, the image information of the biological sample (200) can be expanded in the frequency domain, and thus the imaging device (100) can obtain high-resolution images that exceed the conventional optical resolution limit. In particular, according to one embodiment of the present disclosure, the imaging device (100) can obtain fluorescent images by using saturated structured light containing higher-order harmonic components, thereby expanding the image information of the biological sample (200) to an even higher frequency domain, and thus obtaining super-resolution images.

[0181] FIG. 8 is a flowchart illustrating an example of a method for controlling a laser beam irradiated by a light source (10) by an imaging device (100) according to one embodiment of the present disclosure.

[0182] Referring to FIG. 8, the method of operation of the imaging device (100) may include steps S810 and S820. In one embodiment of the present disclosure, steps S810 and S820 may be executed by at least one processor (30) included in the imaging device (100). The operation of step S810 and the operation of step S820 illustrated in FIG. 8 embodies the operation of step S210 illustrated in FIG. 2. After the operation of step S820 illustrated in FIG. 8 is performed, the operation of S220 illustrated in FIG. 2 may be performed.

[0183] In step S810 of FIG. 8, the imaging device (100) may irradiate a pulsed laser using a light source (10). The light source (10) may be configured as a pulsed laser light source that irradiates a pulsed laser.

[0184] In one embodiment of the present disclosure, the method of operating the imaging device (100) may include the step of irradiating a pulsed laser using a light source (10).

[0185] In step S820 of FIG. 8, the imaging device (100) can control at least one of the pulse width or pulse period of the pulse laser and the intensity of the beam using the second optical modulation module (60).

[0186] In one embodiment of the present disclosure, the method of operation of the imaging device (100) may include the step of controlling at least one of the pulse width or pulse period of the pulse laser and the intensity of the beam using a second optical modulation module (60).

[0187] In one embodiment of the present disclosure, the imaging device (100) may further include a second optical modulation module (60).

[0188] The second optical modulation module (60) can control at least one of the pulse width or pulse period of the pulse laser. The pulse width and pulse period of the pulse laser can be appropriately set according to the configuration and characteristics of the optical system of the imaging device (100).

[0189] The second optical modulation module (60) can control the intensity of the pulsed laser. The second optical modulation module (60) can control the intensity of the pulsed laser to be greater than the saturation intensity that saturates the excited state of the phosphor coupled to the biological sample (200).

[0190] In one embodiment of the present disclosure, the second optical modulation module (60) may include a first sub-optical modulation module that modulates at least one of the pulse width or pulse period of the pulse laser and a second sub-optical modulation module that modulates the intensity of the pulse laser. That is, at least one of the pulse width or pulse period of the pulse laser and the intensity of the pulse laser may be controlled through separate, independent optical modulation modules. However, the embodiment is not limited thereto, and the imaging device (100) may integrally control at least one of the pulse width or pulse period of the pulse laser and the intensity of the pulse laser by a single optical modulation module.

[0191] Meanwhile, FIGS. 2 and FIGS. 8 illustrate the second optical modulation module (60) as being an independent configuration separated from the light source (10), but the embodiments of the present disclosure are not limited thereto, and the second optical modulation module (60) may be configured to be integrated into the light source (10). In this case, it may be described that a pulsed laser, in which at least one of the pulse width or pulse period and the beam intensity are pre-controlled, is irradiated from the light source (10).

[0192] FIG. 9 is a flowchart illustrating an example of a method for controlling a laser beam irradiated by a light source (10) by an imaging device (100) according to one embodiment of the present disclosure.

[0193] Referring to FIG. 9, the method of operation of the imaging device (100) may include steps S910 and S920. In one embodiment of the present disclosure, steps S910 and S920 may be executed by at least one processor (30) included in the imaging device (100). The operation of step S910 and the operation of step S920 illustrated in FIG. 9 embodies the operation of step S210 illustrated in FIG. 2. After the operation of step S920 illustrated in FIG. 9 is performed, the operation of S220 illustrated in FIG. 2 may be performed.

[0194] In step S910 of FIG. 9, the imaging device (100) can irradiate a continuous wave laser using a light source (10). The light source (10) may be configured as a continuous wave laser light source that irradiates a continuous wave laser.

[0195] In one embodiment of the present disclosure, the method of operation of the imaging device (100) may include the step of irradiating a continuous wave laser using a light source (10).

[0196] In step S920 of FIG. 9, the imaging device (100) can control the intensity of the beam of the continuous wave laser using the second optical modulation module (60) and control the exposure time of the continuous wave laser to adjust at least one of the pulse period and pulse width of the continuous wave laser.

[0197] In one embodiment of the present disclosure, the method of operation of the imaging device (100) may include the step of controlling the intensity of the beam of the continuous wave laser using a second optical modulation module (60), and controlling the exposure time of the continuous wave laser to control at least one of the pulse period and pulse width of the continuous wave laser.

[0198] In one embodiment of the present disclosure, the imaging device (100) may further include a second optical modulation module (60).

[0199] The second optical modulation module (60) can control the exposure time of the continuous wave laser. By controlling the exposure time of the continuous wave laser, the second optical modulation module (60) can control at least one of the pulse width or pulse period of the continuous wave laser. Accordingly, the continuous wave laser with modulated exposure time can be output in the form of pulses. The exposure time of the continuous wave laser (i.e., the pulse width and pulse period of the continuous wave laser) can be appropriately set according to the configuration and characteristics of the optical system of the imaging device (100).

[0200] The second optical modulation module (60) can control the intensity of the continuous wave laser. The second optical modulation module (60) can control the intensity of the continuous wave laser to be greater than the saturation intensity that saturates the excited state of the fluorescent material bound to the biological sample.

[0201] In one embodiment of the present disclosure, the second optical modulation module (60) may include a first sub-optical modulation module that modulates the exposure time of the continuous wave laser and a second sub-optical modulation module that modulates the intensity of the continuous wave laser. That is, the exposure time of the continuous wave laser and the intensity of the continuous wave laser may be controlled through separate, independent optical modulation modules. However, the embodiment is not limited thereto, and the imaging device (100) may integrally control the exposure time of the continuous wave laser and the intensity of the continuous wave laser by a single optical modulation module.

[0202] Meanwhile, FIGS. 2 and 9 illustrate the second light modulation module (60) as being an independent configuration separated from the light source (10), but the embodiments of the present disclosure are not limited thereto, and the second light modulation module (60) may be configured to be integrated into the light source (10). In this case, it may be described that a continuous wave laser, with a pre-controlled exposure time (or at least one of pulse width or pulse period) and beam intensity, is irradiated from the light source (10).

[0203] Figure 10 is a graph showing the laser output characteristics over time for a continuous wave laser and a time-modulated continuous wave laser, respectively.

[0204] Graph 1001 shown in FIG. 10 (a) represents the laser output characteristics over time of linear structured light generated by a continuous wave laser. Graph 1002 shown in FIG. 10 (b) represents the laser output characteristics over time of saturated structured light generated by a continuous wave laser.

[0205] As illustrated in graphs 1001 and 1002, linear structured light or saturated structured light generated by a continuous wave laser has a continuously maintained output, and the output intensity can remain constant without changing over time. Meanwhile, since the saturated structured light is generated from a continuous wave laser having a high output intensity greater than the saturation intensity, it can be seen that the output intensity (1020) of the continuous wave laser in graph 1002 is higher than the output intensity (1010) of the continuous wave laser in graph 1001.

[0206] The 1003 graph shown in Fig. 10 (c) represents the laser output characteristics over time of saturated structured light generated from a time-modulated continuous wave laser.

[0207] As illustrated in graph 1003, the time-modulated continuous wave laser can be output in a pulse form having a predetermined pulse width (1031) and a predetermined pulse period (1032) as the exposure time of the continuous wave laser is controlled. By controlling the exposure time of the continuous wave laser, the pulse width (1031) and pulse period (1032) of the time-modulated continuous wave laser can be controlled. That is, the imaging device (100) can modulate the continuous wave laser, which was output in the form of graph 1002, into a time-modulated continuous wave laser that is output in the form of graph 1003 by controlling the exposure time of the continuous wave laser emitted from the light source (10).

[0208] The 1003 graph can be similarly applied to the laser output characteristics over time of saturated structured light modulated from a pulsed laser.

[0209] According to one embodiment of the present disclosure, an imaging device (100) can acquire a fluorescent image using saturated structured light having different pattern directions and pattern phases, and at the same time, reduce light exposure of a biological sample (200) by using saturated structured light having a pulsed output. Accordingly, according to one embodiment of the present disclosure, the effect of increased resolution associated with using saturated structured light can be maintained, while minimizing photothermal damage to the biological sample and photobleaching of the fluorescence.

[0210] FIG. 11 is a drawing illustrating examples of periodic grid patterns of structured light according to one embodiment of the present disclosure.

[0211] Referring to FIG. 11, in one embodiment of the present disclosure, the structured light may have an illumination distribution in the form of a spatially periodic grid pattern. FIG. 11 illustrates the grid pattern form of saturated structured light as an example.

[0212] First, as illustrated in FIG. 11(a), in one embodiment of the present disclosure, the structured light may have a light distribution in the form of a one-dimensional grid pattern having a one-dimensional period. The one-dimensional grid pattern (1111, 1112, 1113) may be a grid pattern in which periodic brightness / darkness is repeated in one direction. The one-dimensional grid pattern (1111, 1112, 1113) may be a pattern in the form of parallel stripes. For example, the one-dimensional grid pattern (1111) may be a grid pattern in which periodic brightness / darkness is repeated in the Y-axis direction. For example, the one-dimensional grid pattern (1112) may be a grid pattern in which periodic brightness / darkness is repeated in a direction tilted with respect to the X-axis and Y-axis. For example, the one-dimensional grid pattern (1113) may be a grid pattern in which periodic brightness / darkness is repeated in the X-axis direction.

[0213] As illustrated in FIG. 11(b), in one embodiment of the present disclosure, the structured light may have a light distribution in the form of a two-dimensional grid pattern having two-dimensional periodicity. The two-dimensional grid pattern (1121, 1122) may be a grid pattern in which periodic brightness / darkness repeats in two directions. The two-dimensional grid pattern (1121, 1122) may be a grid pattern. For example, the two-dimensional grid pattern (1121, 1122) may be a grid pattern having periodicity in both the X-axis direction and the Y-axis direction. FIG. 11(b) illustrates, by way of example, that the two-dimensional grid pattern (1121, 1122) is a grid pattern having periodicity in two directions, the X-axis direction and the Y-axis direction, but the two directions having periodicity are not limited thereto and may have periodicity in a direction tilted with respect to the X-axis and Y-axis.

[0214] For example, in a two-dimensional grid pattern (1121), the period width in the first direction (e.g., X-axis direction) and the period width in the second direction (e.g., Y-axis direction) may be the same. For example, in a two-dimensional grid pattern (1122), the period width in the first direction (e.g., X-axis direction) and the period width in the second direction (e.g., Y-axis direction) may be different.

[0215] According to one embodiment of the present disclosure, the imaging device (100) can irradiate structured light having various types of patterns, and can be flexibly utilized depending on the sample type and research purpose. However, the shape of the grid pattern of the structured light is not limited to that shown in FIG. 11, and the structured light can be implemented as various types of grid patterns having spatially periodic patterns.

[0216] FIG. 12 is a flowchart illustrating an example of a method in which an imaging device (100) according to one embodiment of the present disclosure irradiates a plurality of structured lights onto a biological sample (200).

[0217] Referring to FIG. 12, the method of operation of the imaging device (100) may include step S1210. In one embodiment of the present disclosure, step S1210 may be executed by at least one processor (30) included in the imaging device (100). The operation of step S1210 illustrated in FIG. 12 is a specific embodiment of the operation of step S220 illustrated in FIG. 2. The operation of step S1220 illustrated in FIG. 12 may be performed after the operation of S210 illustrated in FIG. 2 has been performed. After the operation of step S1220 illustrated in FIG. 12 has been performed, the operation of S230 illustrated in FIG. 2 may be performed.

[0218] In step S1210 of FIG. 12, the imaging device (100) can irradiate a plurality of structured lights in at least one of an epi-illumination method, a trans-illumination method, or a light sheet illumination method.

[0219] In one embodiment of the present disclosure, the method of operation of the imaging device (100) may include the step of irradiating a plurality of structured lights in at least one of an epi-illumination method, a trans-illumination method, or a light sheet illumination method.

[0220] According to one embodiment of the present disclosure, an imaging device (100) may be configured to irradiate a plurality of structured lights from various directions. Accordingly, the imaging device (100) of the present disclosure can be flexibly utilized depending on the type of biological sample and the purpose of the study. Hereinafter, with reference to FIGS. 13 to 15, an epi-illumination method, a trans-illumination method, and a light sheet illumination method will each be described.

[0221] FIG. 13 is a schematic diagram of an imaging device (1300) in which a plurality of structured lights are irradiated in a coaxial illumination manner according to one embodiment of the present disclosure.

[0222] The imaging device (1300) described with reference to FIG. 13 may correspond to the imaging device (100).

[0223] Since the description of the imaging device (100) described above with reference to FIGS. 1 to 12 can be applied to the operation of the imaging device (1300), the description of FIG. 13 will focus on the operation of irradiating multiple structured lights in a coaxial illumination manner, and further detailed descriptions will be omitted below.

[0224] In one embodiment of the present disclosure, the imaging device (1300) may include a light source (1310), a beam expander (1341), a linear polarizer (1342), a spatial light modulator (1330), a lens (1343), a dichroic filter (1344), an objective lens (1345), a mirror (1351), a band-pass filter (1352), a tube lens (1353), and an image sensor (1320). Since the dichroic filter (1344), objective lens (1345), band-pass filter (1352), tube lens (1353), and image sensor (1320) illustrated in FIG. 13 correspond to the dichroic filter (420), objective lens (430), band-pass filter (440), tube lens (450), and image sensor (20) described with reference to FIG. 4, a detailed description thereof will be omitted below. Meanwhile, FIG. 13 exemplarily illustrates that an imaging device (1300) includes a mirror (1351) and that light reflected from a dichroic filter (1344) is reflected by the mirror (1351) and received by an image sensor (1320); however, as shown in FIG. 4, in one embodiment of the present disclosure, the mirror (1351) may be omitted.

[0225] In one embodiment of the present disclosure, the imaging device (1300) can adjust the intensity of a laser beam irradiated by a light source (1310) to a saturation intensity greater than that which saturates the excited state of a phosphor, and can adjust the pulse width or exposure time of the laser beam. By doing so, the imaging device (1300) can provide a laser beam in which the intensity of the beam is greater than or equal to the saturation intensity. Additionally, the imaging device (1300) can provide a pulsed laser or a time-modulated continuous wave laser.

[0226] The imaging device (1300) can control (enlarge or reduce) the diameter of the laser beam by passing the laser beam emitted from the light source (1310) through the beam expander (1341). For example, the laser beam can have its diameter expanded by the beam expander (1341), and as the divergence angle is reduced, its collimation can be improved. The imaging device (1300) can control the beam diameter so that it is expanded to less than the clear aperture of the spatial light modulator (1330). The imaging device (1300) can pass only the beam that is polarized (or linearly polarized) in a specific direction by passing the laser beam that has passed through the beam expander (1341) through the linear polarizer (1342).

[0227] An imaging device (1300) can modulate a laser beam emitted from a light source (1310) and passed through a beam expander (1341) and a linear polarizer (1342) into structured light using a spatial light modulator (1330). Although a reflective spatial light modulator (1330) is illustrated exemplarily in FIG. 13, the embodiment is not limited thereto, and a transmissive spatial light modulator may be applied as a light modulation module for forming structured light. Additionally, although a spatial light modulator (1330) is illustrated exemplarily in FIG. 13, the embodiment is not limited thereto, and a diffraction grating (e.g., a transmissive diffraction grating or a reflective diffraction grating) may be applied as a light modulation module for forming structured light.

[0228] In one embodiment of the present disclosure, an imaging device (1300) may form saturated structured light using a spatial light modulator (1330). The imaging device (1300) may form a plurality of saturated structured lights having different pattern directions and pattern phases. More specifically, the imaging device (1300) may form a plurality of saturated structured lights having different pattern phases for each of the different pattern directions. In one embodiment of the present disclosure, the plurality of saturated structured lights may include structured lights having three or more different pattern directions. More specifically, the plurality of saturated structured lights may include structured lights having three or more different pattern directions and different pattern phases for each of the three or more different pattern directions. The imaging device (1300) may acquire a plurality of fluorescence images corresponding to each of the different pattern directions and pattern phases of the plurality of saturated structured lights by irradiating a biological sample (200) with the plurality of saturated structured lights.

[0229] Coaxial illumination is an illumination method in which illumination light irradiated onto a biological sample (200) travels along the same optical axis as emission light (or observation light) emitted from the biological sample (200). Coaxial illumination may be an illumination method in which illumination light and emission light are incident on the biological sample (200) through the same objective lens (1345) and received toward an image sensor (1320). Coaxial illumination may irradiate illumination light from above the biological sample (200). According to one embodiment of the present disclosure, an imaging device (1300) may irradiate saturated structured light modulated from a pulsed laser or a time-modulated continuous wave laser from above the biological sample (200).

[0230] FIG. 14 is a schematic diagram of an imaging device (1400) in which a plurality of structured lights are irradiated in a transmitted illumination manner according to one embodiment of the present disclosure.

[0231] The imaging device (1400) described with reference to FIG. 14 may correspond to the imaging device (100).

[0232] Since the description of the imaging device (100, 1300) described above with reference to FIGS. 1 to 13 can be applied to the operation of the imaging device (1400), the description of FIG. 14 will focus on the operation of irradiating multiple structured lights in a transmitted illumination manner, and further detailed descriptions will be omitted below.

[0233] In one embodiment of the present disclosure, the imaging device (1400) may include a light source (1410), a beam expander (1441), a linear polarizer (1442), a spatial light modulator (1430), a lens (1443), a dichroic filter (1444), an objective lens (1445), a mirror (1451), a band-pass filter (1452), a tube lens (1453), and an image sensor (1420). Since the lens (1443), dichroic filter (1444), band-pass filter (1452), tube lens (1453), and image sensor (1420) illustrated in FIG. 14 correspond to the dichroic filter (420), band-pass filter (440), tube lens (450), and image sensor (20) described with reference to FIG. 4, a detailed description thereof will be omitted below. The light source (1410), beam expander (1441), linear polarizer (1442), spatial light modulator (1430), and mirror (1451) illustrated in FIG. 14 correspond to the light source (1310), beam expander (1341), linear polarizer (1342), spatial light modulator (1330), and mirror (1351) described with reference to FIG. 13, respectively, so a detailed description of these is omitted below.

[0234] Transmission illumination is an illumination method that images by receiving emission light (or observation light) emitted as illumination light irradiated onto a biological sample (200) passes through the biological sample (200). Transmission illumination may be an illumination method that irradiates illumination light from the lower side of the biological sample (200) and receives emission light (or observation light) emitted from the biological sample (200) from the upper side. According to one embodiment of the present disclosure, an imaging device (1400) may irradiate saturated structured light modulated from a pulsed laser or a time-modulated continuous wave laser from the lower side of the biological sample (200).

[0235] In one embodiment of the present disclosure, when a plurality of structured lights are irradiated in a transmitted illumination manner, the imaging device (1400) may include a first objective lens (1445a) positioned below the biological sample (200) to focus the illumination light onto the biological sample, and a second objective lens (1445b) positioned above the biological sample (200) to collect the emitted light emitted from the biological sample (200) and transmit it to an image sensor (1420).

[0236] FIG. 15 is a schematic diagram of an imaging device (1500) in which a plurality of structured lights are irradiated in a light sheet illumination manner according to one embodiment of the present disclosure.

[0237] The imaging device (1500) described with reference to FIG. 15 may correspond to the imaging device (100).

[0238] Since the operation of the imaging device (1500) can be applied as described above with reference to FIGS. 1 to 13, the description of FIG. 15 will focus on the operation of irradiating multiple structured lights in a transmitted illumination manner, and further detailed descriptions will be omitted below.

[0239] In one embodiment of the present disclosure, the imaging device (1500) may include a light source (1510), a beam expander (1541), a linear polarizer (1542), a spatial light modulator (1530), a lens (1543), an objective lens (1545), a band-pass filter (1551), a tube lens (1552), and an image sensor (1520). Since the lens (1543), band-pass filter (1551), tube lens (1552), and image sensor (1520) illustrated in FIG. 15 correspond to the band-pass filter (440), tube lens (450), and image sensor (20) described with reference to FIG. 4, a detailed description thereof will be omitted below. The light source (1510), beam expander (1541), linear polarizer (1542), and spatial light modulator (1530) illustrated in FIG. 15 correspond to the light source (1310), beam expander (1341), linear polarizer (1342), and spatial light modulator (1330) described with reference to FIG. 13, respectively, so a detailed description of these is omitted below.

[0240] The light sheet illumination method may be an illumination method that irradiates illumination light from the side of the biological sample (200) and receives emission light (or observation light) emitted from the biological sample (200) from the upper side. According to one embodiment of the present disclosure, the imaging device (1500) may irradiate saturated structured light modulated from a pulsed laser or a time-modulated continuous wave laser from the side of the biological sample (200).

[0241] In one embodiment of the present disclosure, when a plurality of structured lights are irradiated in a transmitted illumination manner, the imaging device (1500) may include a first objective lens (1545a) positioned above the biological sample (200) to collect emitted light emitted from the biological sample (200) and transmit it to an image sensor (1520), and a second objective lens (1545b) positioned on the side of the biological sample (200) to focus the illumination light onto the biological sample (200). However, the second objective lens (1545b) may be omitted.

[0242] In one embodiment of the present disclosure, the imaging device (1500) may irradiate a thin structured light sheet with a light-sheer structure onto a biological sample (200). In one embodiment of the present disclosure, the imaging device (1500) may further include a cylindrical lens to irradiate the light of the light-sheer structure. The imaging device (1500) may use the cylindrical lens to expand the illumination light into a thin sheet of light to create a light sheet. Accordingly, the imaging device (1500) may realize a final three-dimensional image by acquiring fluorescent images containing three-dimensional information by utilizing the light sheet.

[0243] FIG. 16 is a flowchart illustrating a method in which an imaging device (100) according to one embodiment of the present disclosure obtains a result image based on a plurality of fluorescent images.

[0244] Referring to FIG. 16, the method of operation of the imaging device (100) may include steps S1610 and S1630. In one embodiment of the present disclosure, steps S1610 and S1630 may be executed by at least one processor (30) included in the imaging device (100). The operation of steps S1610 through S1630 illustrated in FIG. 16 is an embodiment of the operation of step S240 illustrated in FIG. 2. The operation of step S1610 illustrated in FIG. 16 may be performed after the operation of S230 illustrated in FIG. 2 has been performed.

[0245] In step S1610 of FIG. 16, the imaging device (100) can obtain frequency components corresponding to a plurality of fluorescent images by performing a Fourier transform on a plurality of fluorescent images.

[0246] In one embodiment of the present disclosure, the method of operating the imaging device (100) may include the step of obtaining frequency components corresponding to a plurality of fluorescent images by performing a Fourier transform on a plurality of fluorescent images.

[0247] In one embodiment of the present disclosure, a plurality of fluorescent images may correspond to images obtained by irradiating a biological sample with a plurality of structured lights having different pattern angles and pattern phases. Each of the plurality of fluorescent images may correspond to a different pattern angle and pattern phase. Accordingly, each of the plurality of fluorescent images may include different frequency components.

[0248] In one embodiment of the present disclosure, an imaging device (100) can perform a Fourier Transform (FT) or Fast Fourier Transform (FFT) on a plurality of fluorescent images to extract frequency components or spatial frequency information (spectral bandwidth) corresponding to a plurality of fluorescent images.

[0249] In step S1620 of FIG. 16, the imaging device (100) can merge frequency components to obtain frequency components corresponding to the resulting image.

[0250] In one embodiment of the present disclosure, the method of operating the imaging device (100) may include the step of merging frequency components to obtain a frequency component corresponding to a result image.

[0251] In one embodiment of the present disclosure, the imaging device (100) can obtain a frequency component corresponding to a result image by merging at least one frequency component corresponding to a plurality of fluorescent images.

[0252] In one embodiment of the present disclosure, the imaging device (100) can generate an extended synthetic aperture that exceeds the resolution of a conventional lens system. In one embodiment of the present disclosure, the imaging device (100) can perform registration by correcting phase information corresponding to a plurality of fluorescence images so that data lost in the frequency domain is minimized.

[0253] In step S1630 of FIG. 16, the imaging device (100) can obtain a result image by performing an inverse Fourier transform on the frequency components of the result image.

[0254] In one embodiment of the present disclosure, the method of operating the imaging device (100) may include the step of obtaining a result image by performing an inverse Fourier transform on the frequency components of the result image.

[0255] Each fluorescence image can be represented (or mapped) in the frequency space through a Fourier transform. Frequency components corresponding to each fluorescence image are distributed in the frequency space. The frequency components can represent frequency components corresponding to multiple structured lights having different pattern directions and pattern phases.

[0256] The imaging device (100) can generate a result image containing wider frequency information by merging frequency components corresponding to each fluorescence image and converting the merged frequency components back into the spatial domain. The result image, obtained by combining multiple fluorescence images in Fourier space and then undergoing an inverse transform, can display an image containing the characteristics of a biological sample that is clearer than before.

[0257] For example, the imaging device (100) can obtain a result image by combining multiple fluorescence images using a Fourier tychography algorithm to restore high-frequency components. Fourier tychography is a technique that can be understood by a person skilled in the art to expand the overall frequency band of a sample by merging frequency information of fluorescence images obtained at different angles of incidence.

[0258] FIG. 17 is a flowchart illustrating a method in which an imaging device (100) according to one embodiment of the present disclosure acquires a plurality of fluorescent images containing three-dimensional information.

[0259] Referring to FIG. 17, the method of operation of the imaging device (100) may include steps S1710 and S1720. In one embodiment of the present disclosure, steps S1710 and S1720 may be executed by at least one processor (30) included in the imaging device (100). The operation of step S1720 illustrated in FIG. 17 is a specific embodiment of the operation of step S230 illustrated in FIG. 2. The operation of step S1710 illustrated in FIG. 17 may be performed after the operation of S220 illustrated in FIG. 2 has been performed. After the operation of step S1720 illustrated in FIG. 17 has been performed, the operation of S240 illustrated in FIG. 2 may be performed.

[0260] In step S1710 of FIG. 17, the imaging device (100) can irradiate a plurality of structured lights toward the biological sample (200) at each of different focal positions along the Z-axis direction corresponding to the depth direction of the biological sample (200).

[0261] In one embodiment of the present disclosure, the method of operation of the imaging device (100) may include the step of irradiating a plurality of structured lights toward the biological sample (200) at each of different focal positions along the Z-axis direction corresponding to the depth direction of the biological sample (200).

[0262] In one embodiment of the present disclosure, the imaging device (100) may irradiate a plurality of structured lights by varying the focal positions irradiated to the biological sample (200). The imaging device (100) may irradiate a plurality of structured lights having different pattern directions and pattern phases at each of the different focal positions. For example, when the imaging device (100) intends to irradiate structured lights at a first focal position and at a second focal position different from the first focal position, it may irradiate a plurality of structured lights having different pattern directions and pattern phases at the first focal position and irradiate a plurality of structured lights having different pattern directions and pattern phases at the second focal position. For example, it may irradiate a plurality of structured lights having different pattern phases for each of three or more pattern directions at the first focal position and irradiate a plurality of structured lights having different pattern phases for each of three or more pattern directions at the second focal position.

[0263] In step S1720 of FIG. 17, the imaging device (100) can acquire a plurality of fluorescent images containing three-dimensional information.

[0264] In one embodiment of the present disclosure, the method of operating the imaging device (100) may include the step of acquiring a plurality of fluorescent images containing three-dimensional information.

[0265] According to one embodiment of the present disclosure, an imaging device (100) can acquire fluorescent images at different depths of a biological sample (200) by irradiating a plurality of structured lights to different focal positions along the Z-axis direction, thereby acquiring a plurality of fluorescent images including three-dimensional information.

[0266] According to one embodiment of the present disclosure, an imaging device (100) can improve the resolution of a fluorescence image for all focal positions by irradiating pattern structured light, which has a different pattern direction and pattern phase, to each of the different focal positions. Accordingly, as the imaging device (100) improves the resolution of each of the plurality of fluorescence images containing three-dimensional information, the resolution of a three-dimensional result image obtained by synthesizing the plurality of fluorescence images in the frequency space can also be improved.

[0267] Additionally, the imaging device (100) can acquire multiple fluorescent images containing three-dimensional information using saturated structured light having a pulsed output, thereby maintaining the effect of increasing the resolution of the images containing three-dimensional structured information while minimizing photothermal damage to the biological sample (200) and photobleaching of the fluorescent material.

[0268] Hereinafter, with reference to FIGS. 18 and 19, examples of a method in which an imaging device (1800, 1900) irradiates a plurality of structured lights to each of different focal positions along the Z-axis direction corresponding to the depth direction of a biological sample (200) will be described.

[0269] FIG. 18 is a drawing for illustrating an example of a method in which an imaging device (1800) according to one embodiment of the present disclosure irradiates a plurality of structured lights to each of different focal positions along the Z-axis direction corresponding to the depth direction of a biological sample (1802).

[0270] Referring to FIG. 18, in one embodiment of the present disclosure, an imaging device (1800) can provide a plurality of structured lights to a biological sample (200) in a coaxial illumination manner. Since the imaging device (1800) of FIG. 18 corresponds to the imaging device (1300) of FIG. 13, and the description of the imaging device (1300) of FIG. 13 can be applied as is, the operation of irradiating a plurality of structured lights to each of different focal positions will be described mainly.

[0271] In one embodiment of the present disclosure, the imaging device (1800) may irradiate a plurality of structured lights by varying the focal position, i.e., the depth of focus, along the Z-axis direction corresponding to the depth direction of the biological sample (200). The depth direction of the biological sample (200) and the corresponding Z-axis direction may correspond to a direction orthogonal to the plane defined by the substrate of the flow cell on which the biological sample (200) is placed. The depth direction of the biological sample (200) and the corresponding Z-axis direction may also be described as the observation direction of the optical system of the imaging device (100).

[0272] In one embodiment of the present disclosure, the imaging device (1800) can move the position of the flow cell where the biological sample (200) is located in the Z-axis direction. By doing so, the distance between the objective lens (1810) and the biological sample (200) is changed, and the focal plane where the structured light passing through the objective lens (1810) is focused can be moved in the Z-axis direction corresponding to the depth direction of the biological sample (200). Meanwhile, FIG. 18 illustrates an example of moving the position of the flow cell in the Z-axis direction, but is not limited thereto.

[0273] In one embodiment of the present disclosure, the imaging device (1800) may move the position of the objective lens (1810) in the Z-axis direction. By doing so, the distance between the objective lens (1810) and the biological sample (200) is changed, and the focal plane where the structured light passing through the objective lens (1810) is focused can be moved in the Z-axis direction corresponding to the depth direction of the biological sample (200).

[0274] In one embodiment of the present disclosure, the imaging device (1800) may change the numerical aperture of the objective lens (1810) by controlling the degree of opening of the aperture of the objective lens (1810). By doing so, the focal plane where the structured light passing through the objective lens (1810) is focused may be moved in the Z-axis direction corresponding to the depth direction of the biological sample (200).

[0275] Meanwhile, although not shown in the present disclosure, the method described with reference to FIG. 18 can be similarly applied to an imaging device (1800) that provides a plurality of structured lights to a biological sample (200) in a transmitted illumination manner.

[0276] FIG. 19 is a drawing for illustrating an example of a method in which an imaging device (1900) according to one embodiment of the present disclosure irradiates a plurality of structured lights to each of different focal positions along the Z-axis direction corresponding to the depth direction of a biological sample (200).

[0277] Referring to FIG. 19, in one embodiment of the present disclosure, an imaging device (1900) can provide a plurality of structured lights to a biological sample (200) in a light sheet illumination manner. Since the imaging device (1900) of FIG. 19 corresponds to the imaging device (1500) of FIG. 15, and the description of the imaging device (1500) of FIG. 15 can be applied as is, the operation of irradiating a plurality of structured lights to each of different focal positions will be described mainly.

[0278] The imaging device (1900) can irradiate multiple structured lights by varying different focal positions along the Z-axis direction corresponding to the depth direction of the biological sample (200). That is, the imaging device (1900) can move the focal positions formed on the biological sample (200) by multiple structured lights irradiated from the side of the biological sample (200) along the Z-axis direction.

[0279] In one embodiment of the present disclosure, the imaging device (1900) can move the position of the flow cell where the biological sample (200) is located in the Z-axis direction. By doing so, the focal position of a plurality of structured lights can be moved in the Z-axis direction corresponding to the depth direction of the biological sample (200). Meanwhile, FIG. 19 illustrates an example in which the position of the flow cell is moved in the Z-axis direction, but is not limited thereto.

[0280] In one embodiment of the present disclosure, the imaging device (1900) may move the position of the first objective lens (1910) (or cylindrical lens) in the Z-axis direction. By doing so, the focal position of a plurality of structured lights may be moved in the Z-axis direction corresponding to the depth direction of the biological sample (200).

[0281] FIG. 20 is a flowchart illustrating a method in which an imaging device (100) according to one embodiment of the present disclosure acquires a plurality of fluorescent images containing three-dimensional information.

[0282] Referring to FIG. 20, the method of operation of the imaging device (100) may include steps S2010 and S2020. In one embodiment of the present disclosure, steps S2010 and S2020 may be executed by at least one processor (30) included in the imaging device (100). The operation of step S2020 illustrated in FIG. 20 is a specific embodiment of the operation of step S230 illustrated in FIG. 2. The operation of step S2010 illustrated in FIG. 20 may be performed after the operation of S220 illustrated in FIG. 2 is performed. After the operation of step S2020 illustrated in FIG. 20 is performed, the operation of S240 illustrated in FIG. 2 may be performed.

[0283] In step S2010 of FIG. 20, the imaging device (100) can irradiate a plurality of structured lights toward the biological sample (200) for each of different incident angles.

[0284] In one embodiment of the present disclosure, the method of operating the imaging device (100) may include the step of irradiating a plurality of structured lights toward a biological sample (200) for each of different incident angles.

[0285] In one embodiment of the present disclosure, the imaging device (100) may irradiate a plurality of structured lights by varying the incident angle of the structured light irradiated onto a biological sample (200). At this time, the incident angle of the structured light may include an oblique angle with respect to at least the optical axis (i.e., the Z-axis direction). The imaging device (100) may irradiate a plurality of structured lights having different pattern directions and pattern phases for each of the different incident angles. For example, when the imaging device (100) irradiates structured light at a first incident angle and intends to irradiate structured light at a second incident angle different from the first incident angle, it may irradiate a plurality of structured lights having different pattern directions and pattern phases at the first incident angle and irradiate a plurality of structured lights having different pattern directions and pattern phases at the second incident angle. For example, a plurality of structured lights having different pattern phases for each of three or more pattern directions can be irradiated at a first incident angle, and a plurality of structured lights having different pattern phases for each of three or more pattern directions can be irradiated at a second incident angle.

[0286] In step S2020 of FIG. 20, the imaging device (100) can acquire a plurality of fluorescent images containing three-dimensional information.

[0287] In one embodiment of the present disclosure, the method of operating the imaging device (100) may include the step of acquiring a plurality of fluorescent images containing three-dimensional information.

[0288] According to one embodiment of the present disclosure, an imaging device (100) may irradiate at least a portion of a plurality of structured lights toward a biological sample (200) at an oblique angle with respect to the Z-axis. Accordingly, at least a portion of the plurality of structured lights may acquire a fluorescent image corresponding to an oblique cross-section of the biological sample (200). Thus, the imaging device (100) may acquire a three-dimensional (3D) image containing information at multiple depths, rather than a two-dimensional (2D) image at a single depth of the biological sample (200). Accordingly, the imaging device (100) may acquire a plurality of fluorescent images containing three-dimensional information by irradiating a plurality of structured lights for each of different incident angles.

[0289] According to one embodiment of the present disclosure, an imaging device (100) can improve the resolution of a fluorescence image for all incident angles (or all inclined cross-sections) by irradiating pattern structured light, which has a different pattern direction and pattern phase, for each of each of different incident angles. Accordingly, as the imaging device (100) improves the resolution of each of a plurality of fluorescence images containing three-dimensional information, the resolution of a three-dimensional result image obtained by synthesizing a plurality of fluorescence images in the frequency space can also be improved.

[0290] Additionally, the imaging device (100) can acquire multiple fluorescent images containing three-dimensional information using saturated structured light having a pulsed output, thereby maintaining the effect of increasing the resolution of images containing three-dimensional structured information while minimizing photothermal damage to the biological sample (200) and photobleaching of the fluorescent material.

[0291] Hereinafter, with reference to FIG. 21, examples of methods for an imaging device (2100) to irradiate a plurality of structured lights for each of different incident angles will be described.

[0292] FIG. 21 is a drawing for explaining an example of a method in which an imaging device (2100) according to one embodiment of the present disclosure irradiates a plurality of structured lights for each of different incident angles.

[0293] Referring to FIG. 21, in one embodiment of the present disclosure, an imaging device (2100) can provide a plurality of structured lights to a biological sample in a coaxial illumination manner. Since the imaging device (2100) of FIG. 21 corresponds to the imaging device (1300) of FIG. 13, and the description of the imaging device (1300) of FIG. 13 can be applied as is, the operation of irradiating a plurality of structured lights for each of different incident angles will be described mainly.

[0294] In one embodiment of the present disclosure, the imaging device (2100) may further include a rotary drive mirror. The imaging device (2100) may irradiate a plurality of structured lights in an oblique illumination manner using the rotary drive mirror. The imaging device (2100) may change the incident angle of the illumination light incident on the biological sample (200) by changing the angle of the rotary drive mirror. For example, the rotary drive mirror may be implemented as a galvo mirror. Through this, the imaging device (2100) may control the incident angle of a plurality of structured lights using the rotary drive mirror and irradiate a plurality of structured lights at an incident angle oblique with respect to the Z-axis direction.

[0295] FIG. 22 is a flowchart illustrating a method for an imaging device (100) according to one embodiment of the present disclosure to acquire a result image.

[0296] Referring to FIG. 22, the method of operation of the imaging device (100) may include step S2210. In one embodiment of the present disclosure, step S2210 may be executed by at least one processor (30) included in the imaging device (100). The operation of step S2210 illustrated in FIG. 22 is a specific embodiment of the operation of step S240 illustrated in FIG. 2. The operation of step S2220 illustrated in FIG. 22 may be performed after the operation of S230 illustrated in FIG. 2 has been performed. More specifically, the operation of step S2220 illustrated in FIG. 22 may be performed after the operation of S1720 illustrated in FIG. 17 has been performed or after the operation of S2020 illustrated in FIG. 20 has been performed.

[0297] In step S2210 of FIG. 22, the imaging device (100) can acquire a result image containing multi-omics information of a biological sample (200).

[0298] In one embodiment of the present disclosure, the method of operating the imaging device (100) may include the step of acquiring a result image containing multi-omics information of a biological sample.

[0299] As described with reference to FIGS. 17 to 21, an imaging device (100) according to one embodiment of the present disclosure can acquire a plurality of fluorescent images containing three-dimensional information.

[0300] In one embodiment of the present disclosure, as described with reference to FIGS. 17 to 19, an imaging device (100) can acquire a plurality of fluorescent images containing three-dimensional information by irradiating a plurality of structured lights toward a biological sample (200) at each of different focal positions along the Z-axis direction corresponding to the depth direction of the biological sample (200).

[0301] In one embodiment of the present disclosure, as described with reference to FIGS. 20 to 21, an imaging device (100) can acquire a plurality of fluorescent images containing three-dimensional information by irradiating a plurality of structured lights toward a biological sample (200) for each of different incident angles.

[0302] In one embodiment of the present disclosure, a plurality of fluorescent images containing three-dimensional information may include at least one of DNA, RNA, or protein within a biological sample (200). Each of the plurality of fluorescent images may be a two-dimensional image representing at least one of RNA, DNA, or protein. An imaging device (100) can obtain three-dimensional spatial information of the biological sample (200) through the plurality of fluorescent images and can obtain information regarding where and how RNA, DNA, and protein exist within a cell based on the three-dimensional spatial information.

[0303] In one embodiment of the present disclosure, the imaging device (100) may continue to perform experiments in which different fluorescent dyes are chemically attached and detached according to RNA, DNA, and proteins within a biological sample (200) (e.g., a cell sample) to obtain images containing information regarding RNA, DNA, and proteins distributed in three-dimensional space. However, it is not limited thereto, and the imaging device (100) may also obtain separate data regarding molecules regarding where and how each of RNA, DNA, and proteins within the biological sample (200) is distributed. Accordingly, the imaging device (100) may obtain a result image containing multi-omics information of the biological sample (200).

[0304] In the present disclosure, "multi-omics" refers to a method of analyzing biological systems by integrating various omics data, specifically a method of imaging by analyzing data such as intracellular genomics, transcriptomics, proteomics, and metabolomics together. In one embodiment of the present disclosure, multi-omics may refer to an imaging method that senses and images at least one of RNA, DNA, or protein contained in a biological sample. Through multi-omics, by imaging and analyzing RNA, DNA, and proteins within a cell, a basic understanding of the molecular hierarchy structure in the genome of an individual cell can be obtained. Since the identification and treatment of diseases are possible through complex research at the RNA, DNA → protein → cell level, if information at the molecular level constituting the cell can be identified, it is possible to track how those molecules change.

[0305] According to one embodiment of the present disclosure, an imaging device (100) can improve the resolution of a fluorescence image for all focal positions by irradiating pattern structured light with different pattern directions and pattern phases to each of different focal positions. Alternatively, according to one embodiment of the present disclosure, an imaging device (100) can improve the resolution of a fluorescence image for all incident angles (or all inclined cross-sections) by irradiating pattern structured light with different pattern directions and pattern phases for each of different incident angles.

[0306] Accordingly, the imaging device (100) can improve the resolution of each of the plurality of fluorescent images containing three-dimensional information, and thereby improve the resolution of the three-dimensional result image obtained by synthesizing the plurality of fluorescent images in the frequency space. The imaging device (100) can obtain a super-resolution three-dimensional result image, thereby improving inspection precision in the multi-omics field.

[0307] Additionally, according to one embodiment of the present disclosure, the imaging device (100) can reduce light exposure of a biological sample (200) to saturated structured light by generating saturated structured light from a pulsed laser or a time-modulated continuous wave laser. By using a plurality of saturated structured lights having various pattern directions and pattern phases while reducing light exposure of the biological sample (200) to saturated structured light, the effect of inspection precision in the multi-omics field can be maintained, while minimizing photothermal damage to the biological sample and photobleaching of the fluorescence.

[0308] FIG. 23 is a drawing illustrating a three-dimensional (3D) image of intracellular multi-omics obtained by an imaging device (100) according to one embodiment of the present disclosure.

[0309] In one embodiment of the present disclosure, an imaging device (100) can acquire a plurality of fluorescent images (2300) containing three-dimensional information by imaging a fluorescent signal excited from a biological sample (200) while moving the focal position in the Z-axis direction of structured light incident on the biological sample (200). At this time, the plurality of fluorescent images (2300) acquired by the imaging device (100) may correspond to a multi-omic image (2300) representing at least one of RNA, DNA, or protein. That is, the imaging device (100) can acquire a multi-omic image (2300) representing at least one of RNA, DNA, or protein.

[0310] In one embodiment of the present disclosure, an imaging device (100) can acquire a plurality of fluorescent images (2300) containing three-dimensional information by imaging a fluorescent signal excited from a biological sample (200) while varying the incident angle of structured light incident toward the biological sample (200). At this time, the plurality of fluorescent images (2300) acquired by the imaging device (100) may correspond to a multi-omic image (2300) representing at least one of RNA, DNA, or protein. That is, the imaging device (100) can acquire a multi-omic image (2300) representing at least one of RNA, DNA, or protein.

[0311] In one embodiment of the present disclosure, the multi-omic image (2300) may include a first image (2310) showing the distribution of proteins, a second image (2320) showing the distribution of RNA, and a third image (2330) showing the distribution of DNA.

[0312] FIG. 24 is a schematic diagram of an imaging device (2400) including a plurality of light sources and a plurality of image sensors according to one embodiment of the present disclosure.

[0313] The imaging device (2400) described with reference to FIG. 24 may correspond to the imaging device (100) and the imaging devices (1300, 1400, 1500) described with reference to FIG. 13 to 15. According to one embodiment of the present disclosure, the imaging device (2400) may include a plurality of light sources (2411, 2412). For example, the light source (10) described with reference to FIG. 3 may include a plurality of light sources (2411, 2412). According to one embodiment of the present disclosure, the imaging device (2400) may include a plurality of image sensors (2421, 2422, 2423, 2424). For example, the image sensor (20) described with reference to FIG. 3 may include a plurality of image sensors (2421, 2422, 2423, 2424).

[0314] Since the operation of the imaging device (2400) can be similarly applied to the imaging device (100) described above with reference to FIGS. 1 to 23, the description of FIG. 24 will focus on the operation of the imaging device (2400) including a plurality of light sources (2411, 2412) and a plurality of image sensors (2421, 2422, 2423, 2424), and a more detailed description will be omitted below.

[0315] In one embodiment of the present disclosure, the imaging device (2400) may emit beams of different wavelengths through a first light source (2411) and a second light source (2412), respectively. The first laser beam emitted by the imaging device (2400) through the first light source (2411) and the second laser beam emitted through the second light source (2412) may be emitted simultaneously from the first light source (2411) and the second light source (2412), may be emitted sequentially with a predetermined time difference, or may be emitted in a time-slicing manner.

[0316] The imaging device (2400) can control the intensity of the first laser beam to have a beam intensity greater than the saturation intensity that saturates the excited state of the phosphor coupled to the biological sample (200). The imaging device (2400) can control the pulse width or exposure time of the first laser beam, and the first laser beam may be a pulsed laser or a time-modulated continuous wave laser.

[0317] The imaging device (2400) can control the intensity of the second laser beam to have a beam intensity greater than the saturation intensity that saturates the excited state of the phosphor coupled to the biological sample (200). The imaging device (2400) can control the pulse width or exposure time of the second laser beam, and the second laser beam may be a pulsed laser or a time-modulated continuous wave laser.

[0318] The first laser beam can be irradiated onto a biological sample (200) by passing through a first beam expander (2441), a first dichroic filter (2443), a mirror (2444), a spatial light modulator (2430), a lens (2445), a second dichroic filter (2446), and an objective lens (2447) in sequence. Meanwhile, although omitted in FIG. 24, the first laser beam may further pass through a first linear polarizer between the first beam expander (2441) and the first dichroic filter (2443).

[0319] The second laser beam can be irradiated onto a biological sample (200) by passing through a second beam expander (2442), a first dichroic filter (2443), a mirror (2444), a spatial light modulator (2430), a lens (2445), a second dichroic filter (2446), and an objective lens (2447) in sequence. Meanwhile, although omitted in FIG. 24, the second laser beam may further pass through a second linear polarizer between the second beam expander (2442) and the first dichroic filter (2443).

[0320] In one embodiment of the present disclosure, the first dichroic filter (2443) can transmit the first laser beam and reflect the second laser beam. In one embodiment of the present disclosure, the second dichroic filter (2446) can transmit the first laser beam and the second laser beam, transmit excitation light shorter than the first threshold wavelength, and reflect emission light longer than the first threshold wavelength.

[0321] The first dichroic filter (2443), the second dichroic filter (2446), the third-1 dichroic filter (2451), the third-2 dichroic filter (2452), and the third-3 dichroic filter (2457) can separate light based on their respective threshold wavelengths. For example, at least one of the first dichroic filter (2443), the second dichroic filter (2446), the third-1 dichroic filter (2451), the third-2 dichroic filter (2452), and the third-3 dichroic filter (2457) can reflect light shorter than the threshold wavelength and transmit light longer than the threshold wavelength. For example, at least one of the first dichroic filter (2443), the second dichroic filter (2446), the third-1 dichroic filter (2451), the third-2 dichroic filter (2452), and the third-3 dichroic filter (2457) may reflect light longer than a critical wavelength and transmit light longer than a critical wavelength. The critical wavelengths of the first dichroic filter (2443), the second dichroic filter (2446), the third-1 dichroic filter (2451), the third-2 dichroic filter (2452), and the third-3 dichroic filter (2457) may be different from each other or the same from each other. The critical wavelength of the first dichroic filter (2443) is referred to as the first critical wavelength, the critical wavelength of the second dichroic filter (2446) as the second critical wavelength, the critical wavelength of the third-1 dichroic filter (2451) as the third critical wavelength, the critical wavelength of the third-2 dichroic filter (2452) as the fourth critical wavelength, and the critical wavelength of the third-3 dichroic filter (2457) as the fifth critical wavelength.

[0322] In one embodiment of the present disclosure, when a first laser beam emitted by an imaging device (2400) through a first light source (2411) is irradiated onto a biological sample (200), a first emitted light may be generated as a fluorescent material bound to (or treated with) the biological sample (200) is excited. The first emitted light may pass through an objective lens (2447) and be incident on a second dichroic filter (2446).

[0323] The second dichroic filter (2446) transmits the first excitation light (e.g., the first laser beam) shorter than the second threshold wavelength and reflects the first emission light shorter than the second threshold wavelength. The first emission light may be incident on the third-1 dichroic filter (2451) and reflected toward the third-2 dichroic filter (2452). The third-2 dichroic filter (2452) may transmit the first-1 emission light among the first emission light, which is longer than the fourth threshold wavelength of the third-2 dichroic filter (2452). The imaging device (2400) may receive the first-1 emission light through the fourth image sensor (2424) to generate a first wavelength image. The third-2 dichroic filter (2452) may reflect the first-2 emission light among the first emission light, which is shorter than the fourth threshold wavelength of the third-2 dichroic filter (2452). The imaging device (1400) can generate a second wavelength image by receiving first-second emitted light through a third image sensor (2423).

[0324] For example, the first-1 emission light may be emitted as the first-1 fluorescent material bound to (or treated with) the biological sample (200) is excited, and the first-2 emission light may be emitted as the first-2 fluorescent material bound to (or treated with) the biological sample (200) is excited. The first-1 fluorescent material and the first-2 fluorescent material may be excited by a first laser beam emitted through an excitation light of the same wavelength, but may emit emission light having different wavelengths (e.g., the first-1 emission light and the first-2 emission light).

[0325] In one embodiment of the present disclosure, when a second laser beam emitted by an imaging device (2400) through a second light source (2412) is irradiated onto a biological sample (200), a second emitted light may be generated as a fluorescent material bound to (or treated with) the biological sample (200) is excited. The second emitted light may pass through an objective lens (2447) and be incident on a second dichroic filter (2446).

[0326] The second dichroic filter (2446) can transmit a second excitation light (e.g., a second laser beam) shorter than the second threshold wavelength and reflect a second emission light shorter than the second threshold wavelength. The second emission light can be incident on the third-1 dichroic filter (2451) and transmitted toward the third-3 dichroic filter (2457). The third-3 dichroic filter (2457) can transmit the second-1 emission light among the second emission light, which is longer than the fifth threshold wavelength of the third-3 dichroic filter (2457). The imaging device (2400) can receive the second-1 emission light through the second image sensor (2422) to generate a third wavelength image. The third-3 dichroic filter (2457) can transmit the second-2 emission light among the second emission light, which is shorter than the fifth threshold wavelength of the third-3 dichroic filter (2457). The imaging device (2400) can generate a fourth wavelength image by receiving the second-2 emission light through the first image sensor (2491).

[0327] For example, the second-1 emission light may be emitted as the second-1 fluorescent material bound to (or treated with) the biological sample (200) is excited, and the second-2 emission light may be emitted as the second-1 fluorescent material bound to (or treated with) the biological sample (200) is excited. The second-1 fluorescent material and the second-2 fluorescent material may be excited by a second laser beam emitted through an excitation light of the same wavelength, but may emit emission light having different wavelengths (e.g., the second-1 emission light and the second-2 emission light). In summary, according to one embodiment of the present disclosure, the imaging device (2400) may be designed to detect fluorescent signals of different wavelengths.

[0328] For example, when a first laser beam emitted from a first light source (2411) irradiates a biological sample (200), a specific fluorescent substance present in the biological sample (200) is excited and emits a fluorescent signal. An imaging device (2400) can detect fluorescent signals of different wavelengths (first-1 emission light and first-2 emission light) using a third image sensor (2423) and a fourth image sensor (2424). An imaging device (100) can detect the first-1 emission light emitted by the first-1 fluorescent substance through the fourth image sensor (2424), and the first-2 emission light emitted by the first-2 fluorescent substance through the third image sensor (2423).

[0329] Likewise, when a second laser beam emitted from a second light source (2412) irradiates a biological sample (200), another fluorescent material is excited and emits a fluorescent signal. An imaging device (2400) can detect fluorescent signals of different wavelengths (second-1 emission light and second-2 emission light) using a first image sensor (2421) and a second image sensor (2422). The imaging device (2400) can detect the second-1 emission light emitted by the second-1 fluorescent material through the second image sensor (2422), and the second-2 emission light emitted by the second-2 fluorescent material through the first image sensor (2421).

[0330] In this way, the imaging device (2400) can detect fluorescent signals emitted from a plurality of fluorescent substances contained in a biological sample (200) and analyze the distribution, structure, or presence of biomaterials (e.g., gene bases, proteins, cell structures, etc.) corresponding to each fluorescent substance.

[0331] For example, if a sample contains four types of gene bases (A, T, G, C) and a specific fluorescent substance is attached to each base, each fluorescent substance will have different excitation and emission wavelengths. At this time, the imaging device (2400) can identify the distribution, structure, or presence of each gene base contained in the biological sample (200) by using two light sources (1412, 1412) and a plurality of image sensors (2421, 2422, 2423, 2424).

[0332] In one embodiment of the present disclosure, the imaging device (2400) can acquire a plurality of first wavelength images corresponding to different pattern directions and pattern phases by irradiating a plurality of saturated structured lights having different pattern directions and pattern phases while acquiring a first wavelength image.

[0333] In one embodiment of the present disclosure, the imaging device (2400) can acquire a plurality of second wavelength images corresponding to different pattern directions and pattern phases by irradiating a plurality of saturated structured lights having different pattern directions and pattern phases while acquiring a second wavelength image.

[0334] In one embodiment of the present disclosure, the imaging device (2400) can acquire a plurality of third wavelength images corresponding to different pattern directions and pattern phases by irradiating a plurality of saturated structured lights having different pattern directions and pattern phases while acquiring a third wavelength image.

[0335] In one embodiment of the present disclosure, the imaging device (2400) can acquire a plurality of fourth wavelength images corresponding to different pattern directions and pattern phases by irradiating a plurality of saturated structured lights having different pattern directions and pattern phases while acquiring a fourth wavelength image.

[0336] Accordingly, the imaging device (2400) according to one embodiment of the present disclosure can accurately separate and analyze fluorescent signals, so it can be utilized in various life science and medical diagnostic fields such as gene sequence analysis, protein detection, and cell structure research.

[0337] Meanwhile, the number of light sources and / or image sensors included in the imaging device (2400) is not limited to that shown in FIG. 24. In one embodiment of the present disclosure, the imaging device (2400) may include three or more light sources. In this case, each light source may emit a beam of the same wavelength or a different wavelength.

[0338] In one embodiment of the present disclosure, the imaging device (2400) may include two, three, five, or more image sensors. For example, the imaging device (2400) may include two image sensors (e.g., a first image sensor and a second image sensor). In this case, the imaging device (2400) may acquire a plurality of first fluorescent images by detecting a first emission light of a first wavelength generated as a plurality of structured lights are irradiated onto a biological sample bound to a first fluorescent material using the first image sensor. The imaging device (2400) may acquire a plurality of second fluorescent images by detecting a second emission light of a second wavelength generated as a plurality of structured lights are irradiated onto a biological sample bound to a second fluorescent material using the second image sensor.

[0339] In one embodiment of the present disclosure, the number of image sensors included in the imaging device (2400) may be determined in correspondence with the number of fluorescent materials having different wavelengths that are bound to (or processed) the biological sample. In one embodiment of the present disclosure, the number of at least one of the other components included in the imaging device (2400) may also be determined in correspondence with the number of fluorescent materials having different wavelengths that are bound to (or processed) the biological sample.

[0340] In one embodiment of the present disclosure, the imaging device (2400) includes one light source and may include a plurality of image sensors.

[0341] In one embodiment of the present disclosure, the imaging device (2400) may include a plurality of light sources and one image sensor. For example, the bandpass filter included in the imaging device (2400) may be a multi-bandpass filter capable of passing light in a plurality of wavelength ranges. For example, the multi-bandpass filter may selectively transmit light in a first wavelength range and light in a second wavelength range different from the first wavelength range.

[0342] The imaging device (2400) can acquire multiple wavelength images corresponding to each wavelength and synthesize (or combine) the multiple wavelength images to acquire multiple result images with improved resolution. By acquiring result images with improved resolution for each wavelength, the imaging device (2400) can more accurately identify different gene bases, specific proteins, or structures within a cell. Additionally, the imaging device (2400) can minimize damage to biological samples or loss of information due to fluorescence extinction during the process of irradiating structured light and acquiring fluorescence images.

[0343] According to one embodiment of the present disclosure, an imaging device (100) may be provided.

[0344] According to one embodiment of the present disclosure, the imaging device (100) may include a light source (10) that irradiates a laser beam. According to one embodiment of the present disclosure, the imaging device (100) may include a first light modulation module (50) that modulates the laser beam into structured light, comprising a diffraction grating or a spatial light modulator. According to one embodiment of the present disclosure, the imaging device (100) may include an image sensor (20) that receives a fluorescent signal emitted by a fluorescent material coupled to a biological sample as the structured light is irradiated. According to one embodiment of the present disclosure, the imaging device (100) may include. According to one embodiment of the present disclosure, the imaging device (100) may include at least one processor (30) that includes processing circuitry. According to one embodiment of the present disclosure, the imaging device (100) may include a memory (40) that stores one or more instructions.

[0345] According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can adjust the intensity of a laser beam irradiated by a light source (10) to a saturation intensity greater than that which saturates the excited state of a phosphor, and can adjust the pulse width or exposure time of the laser beam. According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can use a first light modulation module (50) to modulate a laser beam, which has a beam intensity greater than the saturation intensity and has a pulse width or exposure time adjusted, into a plurality of structured lights having different pattern directions and pattern phases, and irradiate the plurality of structured lights toward a biological sample. According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can acquire a plurality of fluorescent images corresponding to the pattern direction and pattern phase of a plurality of structured lights using an image sensor (20). According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can acquire a result image by synthesizing a plurality of fluorescent images in the frequency space.

[0346] According to one embodiment of the present disclosure, the light source (10) may be characterized as being composed of a pulsed laser light source that irradiates a pulsed laser. According to one embodiment of the present disclosure, the imaging device (100) may further include a second light modulation module (60) that controls at least one of the pulse width or pulse period of the pulsed laser and the intensity of the beam.

[0347] According to one embodiment of the present disclosure, the light source (10) may be characterized as being composed of a continuous wave laser light source that irradiates a continuous wave laser. According to one embodiment of the present disclosure, the imaging device (100) may further include a second light modulation module (60) that controls the exposure time of the continuous wave laser to adjust at least one of the pulse period or pulse width of the continuous wave laser and controls the intensity of the beam of the continuous wave laser.

[0348] According to one embodiment of the present disclosure, a plurality of structured lights may be characterized as being non-sine wave type structured lights containing higher-order harmonics components.

[0349] According to one embodiment of the present disclosure, a plurality of structured lights may be characterized by having different pattern phases for each of three or more different pattern directions.

[0350] According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can irradiate a plurality of structured lights in at least one of an epi-illumination method, a trans-illumination method, or a light sheet illumination method.

[0351] According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can irradiate a plurality of structured lights toward a biological sample at each of different focal positions along the Z-axis direction corresponding to the depth direction of the biological sample. According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can acquire a plurality of fluorescent images containing three-dimensional information.

[0352] According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can irradiate a plurality of structured lights toward a biological sample for each of different incident angles. According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can acquire a plurality of fluorescent images containing three-dimensional information.

[0353] According to one embodiment of the present disclosure, a plurality of fluorescent images may be characterized by comprising at least one of DNA, RNA, or protein within a biological sample. According to one embodiment of the present disclosure, the resulting image may include multi-omics information of the biological sample.

[0354] According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can obtain frequency components corresponding to a plurality of fluorescent images by performing a Fourier transform on a plurality of fluorescent images. According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can obtain frequency components corresponding to a plurality of fluorescent images by merging the frequency components corresponding to a plurality of fluorescent images. According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) can obtain a result image by performing an inverse Fourier transform on the frequency components of the result image.

[0355] According to one embodiment of the present disclosure, the image sensor (20) may be characterized by including a first image sensor and a second image sensor. According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) may detect a first emission light of a first wavelength generated as a plurality of structured lights are irradiated onto a biological sample bound to a first fluorescent material using the first image sensor, thereby acquiring a plurality of first fluorescent images. According to one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (30), the imaging device (100) may detect a second emission light of a second wavelength generated as a plurality of structured lights are irradiated onto a biological sample bound to a second fluorescent material using the second image sensor, thereby acquiring a plurality of second fluorescent images.

[0356] According to one embodiment of the present disclosure, a method of operating an imaging device (100) may be provided.

[0357] According to one embodiment of the present disclosure, the method may include the step (S210) of adjusting the intensity of a laser beam irradiated by a light source (10) to a saturation intensity greater than that which saturates the excited state of a phosphor bound to a biological sample, and adjusting the pulse width or exposure time of the laser beam. According to one embodiment of the present disclosure, the method may include the step (S220) of using a first light modulation module (50) to modulate a laser beam, which has a beam intensity greater than the saturation intensity and has a pulse width or exposure time adjusted, into a plurality of structured lights having different pattern directions and pattern phases, and irradiating the plurality of structured lights toward a biological sample. According to one embodiment of the present disclosure, the method may include the step (S230) of acquiring a plurality of fluorescent images corresponding to the pattern directions and pattern phases of the plurality of structured lights by receiving a fluorescent signal excited and emitted by a phosphor bound to a biological sample as the plurality of different structured lights are irradiated using an image sensor (20). According to one embodiment of the present disclosure, the method may include the step (S240) of synthesizing a plurality of fluorescent images in the frequency space to obtain a result image.

[0358] According to one embodiment of the present disclosure, the step (S210) of adjusting the intensity of a laser beam to be greater than or equal to the saturation intensity and adjusting the pulse width or exposure time of the laser beam may include the step (S810) of irradiating a pulsed laser using a light source (10). According to one embodiment of the present disclosure, the step (S210) of adjusting the intensity of a laser beam to be greater than or equal to the saturation intensity and adjusting the pulse width or exposure time of the laser beam may include the step (S820) of controlling at least one of the pulse width or pulse period of the pulsed laser and the intensity of the beam using a second optical modulation module (60).

[0359] According to one embodiment of the present disclosure, the step (S210) of adjusting the intensity of a laser beam to be greater than or equal to the saturation intensity and adjusting the pulse width or exposure time of the laser beam may include the step (S910) of irradiating a continuous wave laser using a light source (10). According to one embodiment of the present disclosure, the step (S210) of adjusting the intensity of a laser beam to be greater than or equal to the saturation intensity and adjusting the pulse width or exposure time of the laser beam may include the step (S920) of controlling the exposure time of the continuous wave laser using a second optical modulation module (60) to adjust at least one of the pulse period or pulse width of the continuous wave laser and controlling the intensity of the beam of the continuous wave laser.

[0360] According to one embodiment of the present disclosure, the method may include the step (S1710) of irradiating a plurality of structured lights toward a biological sample at each of different focal positions along the Z-axis direction corresponding to the depth direction of the biological sample. According to one embodiment of the present disclosure, the step (S230) of acquiring a plurality of fluorescent images may include the step (S1720) of acquiring a plurality of fluorescent images including three-dimensional information.

[0361] According to one embodiment of the present disclosure, the method may include the step (S2010) of irradiating a plurality of structured lights toward a biological sample for each of different incident angles. According to one embodiment of the present disclosure, the step (S230) of acquiring a plurality of fluorescent images may include the step (S2020) of acquiring a plurality of fluorescent images including three-dimensional information.

[0362] According to one embodiment of the present disclosure, the step of acquiring a result image (S240) may include the step of acquiring a result image (S2310) including multi-omics information of a biological sample.

[0363] According to one embodiment of the present disclosure, the step of acquiring a result image (S240) may include the step (S1610) of acquiring frequency components corresponding to a plurality of fluorescent images by performing a Fourier transform on a plurality of fluorescent images. According to one embodiment of the present disclosure, the step of acquiring a result image (S240) may include the step (S1620) of acquiring frequency components corresponding to a plurality of fluorescent images by merging them to obtain frequency components corresponding to a result image. According to one embodiment of the present disclosure, the step of acquiring a result image (S240) may include the step (S1630) of acquiring a result image by performing an inverse Fourier transform on the frequency components of the result image.

[0364] A program executed by the imaging device (100) described in the present disclosure may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. The program may be executed by any system capable of executing computer-readable instructions.

[0365] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively.

[0366] Software can be implemented as a computer program containing instructions stored on a computer-readable storage medium. Examples of computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROMs, DVDs (Digital Versatile Discs)). Computer-readable recording media can be distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner. The medium is readable by a computer, stored in memory, and can be executed by a processor.

[0367] Computer-readable storage media may be provided in the form of non-transitory storage media. Here, 'non-transitory' means only that the storage medium does not contain a signal and is tangible, and does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium. For example, a 'non-transitory storage medium' may include a buffer in which data is stored temporarily.

[0368] In addition, the program according to the embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product.

[0369] A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, the computer program product may include a product in the form of a software program (e.g., a downloadable application) that is distributed electronically through the manufacturer of the imaging device (100) or an electronic market (e.g., Samsung Galaxy Store™). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily created. In this case, the storage medium may be a server of the manufacturer of the imaging device (100), a server of the electronic market, or a storage medium of a relay server that temporarily stores the software program.

[0370] A computer program product may include a storage medium of a server or a storage medium of an imaging device (100) in a system composed of an imaging device (100) and / or a server. Alternatively, if there is a third device that is communicationally connected to the imaging device (100), the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include a software program itself that is transmitted from the imaging device (100) to the third device or from the third device to the imaging device.

[0371] In this case, either the imaging device (100) or one of the third devices may execute a computer program product to perform the method according to the disclosed embodiments. Alternatively, at least one of the imaging device (100) and the third device may execute a computer program product to perform the method according to the disclosed embodiments in a distributed manner.

[0372] For example, the imaging device (100) can execute a computer program product stored in memory (140, see FIG. 3) to control another imaging device that is communicationally connected to the imaging device (100) to perform a method according to the disclosed embodiments.

[0373] As another example, a third device may execute a computer program product to control an imaging device that is communicationally connected to the third device to perform the method according to the disclosed embodiment.

[0374] When the third device executes a computer program product, the third device may download the computer program product from the imaging device (100) and execute the downloaded computer program product. Alternatively, the third device may execute a computer program product provided in a pre-loaded state to perform the method according to the disclosed embodiments.

[0375] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, appropriate results can be achieved even if the described techniques are performed in a different order than described, and / or components such as the described computer system or module are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

Claims

1. In an imaging device (100), A light source (10) that emits a laser beam; A first light modulation module (50) comprising a diffraction grating or a spatial light modulator and modulating the laser beam into structured light; An image sensor (20) that receives a fluorescent signal emitted by a fluorescent material coupled to a biological sample as the above structured light is irradiated; At least one processor (30) including processing circuitry; and It includes memory (40) for storing one or more instructions, and By executing the above one or more instructions individually or collectively by the at least one processor (30), the imaging device (100) The intensity of the laser beam irradiated by the light source (10) is adjusted to be greater than the saturation intensity that saturates the excited state of the phosphor, and the pulse width or exposure time of the laser beam is adjusted. Using the above first light modulation module (50), the laser beam having a beam intensity greater than the saturation intensity and having the pulse width or the exposure time adjusted is modulated into a plurality of structured lights having different pattern directions and pattern phases, and the plurality of structured lights are irradiated toward the biological sample. Using the image sensor (20), a plurality of fluorescent images corresponding to the pattern direction and pattern phase of the plurality of structured lights are obtained, and An imaging device (100) that synthesizes the above plurality of fluorescent images in the frequency space to obtain a result image.

2. In Paragraph 1, The light source (10) is composed of a pulsed laser light source that irradiates a pulsed laser, and The imaging device (100) above is, An imaging device (100) further comprising a second optical modulation module (60) that controls at least one of the pulse width or pulse period of the pulse laser and the intensity of the beam.

3. In Paragraph 1, The light source (10) is composed of a continuous wave laser light source that irradiates a continuous wave laser, and The imaging device (100) above is, An imaging device (100) further comprising a second optical modulation module (60) that controls the exposure time of the continuous wave laser to adjust at least one of the pulse period or pulse width of the continuous wave laser and controls the beam intensity of the continuous wave laser.

4. In any one of paragraphs 1 to 3, An imaging device (100), wherein the plurality of structured lights are non-sine wave type structured lights containing higher-order harmonics components.

5. In any one of paragraphs 1 through 4, By executing the above one or more instructions individually or collectively by the at least one processor (30), the imaging device (100) An imaging device (100) that irradiates the above plurality of structured lights in at least one of an epi-illumination method, a trans-illumination method, or a light sheet illumination method.

6. In any one of paragraphs 1 through 5, By executing the above one or more instructions individually or collectively by the at least one processor (30), the imaging device (100) The plurality of structured lights are irradiated toward the biological sample at each of different focal positions along the Z-axis direction corresponding to the depth direction of the biological sample, and An imaging device (100) for acquiring the plurality of fluorescent images including three-dimensional information.

7. In any one of paragraphs 1 through 6, By executing the above one or more instructions individually or collectively by the at least one processor (30), the imaging device (100) For each of the different incident angles, the plurality of structured lights are irradiated toward the biological sample, and An imaging device (100) for acquiring the plurality of fluorescent images including three-dimensional information.

8. In any one of paragraphs 6 to 7, The plurality of fluorescent images above include at least one of DNA, RNA, or protein in the biological sample, and The above result image is an imaging device (100) containing multi-omics information of the biological sample.

9. In the method of operating the imaging device (100), A step (S210) of adjusting the intensity of a laser beam irradiated by a light source (10) to a saturation intensity greater than that which saturates the excited state of a fluorescent material bound to a biological sample, and adjusting the pulse width or exposure time of the laser beam; A step (S220) of using a first light modulation module (50) to modulate the laser beam, which has a beam intensity greater than the saturation intensity and has a pulse width or exposure time adjusted, into a plurality of structured lights having different pattern directions and pattern phases, and irradiating the plurality of structured lights toward the biological sample; A step (S230) of acquiring a plurality of fluorescent images corresponding to the pattern direction and pattern phase of the plurality of structured lights by receiving a fluorescent signal excited and emitted by the fluorescent material coupled to the biological sample as the plurality of different structured lights are irradiated using an image sensor (20); and A method comprising the step (S240) of synthesizing the plurality of fluorescent images in the frequency space to obtain a result image.

10. In Paragraph 9, The step (S210) of adjusting the intensity of the laser beam above the saturation intensity and adjusting the pulse width or exposure time of the laser beam is, A step (S810) of irradiating a pulsed laser using the light source (10); and A method comprising the step (S820) of controlling at least one of the pulse width or pulse period of the pulse laser and the intensity of the beam using a second optical modulation module (60).

11. In Paragraph 9, The step (S210) of adjusting the intensity of the laser beam above the saturation intensity and adjusting the pulse width or exposure time of the laser beam is, A step (S910) of irradiating a continuous wave laser using the light source (10); and A method comprising the step (S920) of controlling the exposure time of the continuous wave laser using a second optical modulation module (60) to control at least one of the pulse period or pulse width of the continuous wave laser and controlling the beam intensity of the continuous wave laser.

12. In any one of paragraphs 9 through 11, A method in which the plurality of structured lights are non-sine wave type structured lights containing higher-order harmonic components.

13. In any one of paragraphs 9 through 12, The above method is, The method further includes the step (S1710) of irradiating the plurality of structured lights toward the biological sample at each of different focal positions along the Z-axis direction corresponding to the depth direction of the biological sample. The step (S230) of acquiring the plurality of fluorescent images above is, A method comprising the step (S1720) of acquiring a plurality of fluorescent images including three-dimensional information.

14. In any one of paragraphs 9 through 13, The above method is, The method further includes the step (S2010) of irradiating the plurality of structured lights toward the biological sample for each of the different incident angles, The step (S230) of acquiring the plurality of fluorescent images above is, A method comprising the step (S2020) of acquiring a plurality of fluorescent images including three-dimensional information.

15. In any one of paragraphs 9 through 14, The plurality of fluorescent images above include at least one of DNA, RNA, or protein in the biological sample, and The step of obtaining the above result image (S240) is, A method comprising the step (S2310) of obtaining the result image including multi-omics information of the biological sample.

Images