High-resolution oblique back illumination microscopy and image restoration method thereof

Abstract

A high-resolution backward-slanted microscope and a method for reconstructing images thereof are disclosed. The high-resolution backward-slanted microscope minimizes image distortion caused by optical aberrations and can obtain high-resolution images even in various scattering media.

Description

Technology Field

[0001] The present invention relates to a high-resolution backward-slanted microscope and a method for reconstructing images thereof. Background Technology

[0002] Oblique back illumination microscopy (OBM) is a representative technique capable of imaging specimens in two or three dimensions using a reflective method, such as thick or reflective specimens that are difficult to access with transmission light microscopes. Other reflective imaging techniques, such as optical coherence tomography (OCT) and reflective confocal microscopy (RCM), can also image such specimens, but this is only possible if the material has a significant difference in refractive index along the depth direction. However, OBM is a technique that can image materials with relatively weak scattering characteristics, such as biological samples, without such limitations. For this reason, OBM technology has been utilized for the analysis of various biological specimens within scattering media. However, OBM technology has disadvantages, such as degradation of resolution and image quality due to optical aberrations and the need for additional simulations to predict light source patterns, which need to be overcome.

[0003] In OBM imaging, problems arise where resolution is degraded and the image of the specimen is distorted due to optical aberrations inherent in the optical system itself and sample-induced aberrations caused by scattering within the biological specimen. A representative method to address this is hardware-based optical aberration correction (Hardware adaptive optics), which measures wavefront distortion using a wavefront sensor and directly corrects it using equipment such as a deformable mirror. However, this method has the disadvantage of high system complexity and the need for expensive equipment. On the other hand, computational optical aberration correction (Computational adaptive optics) expresses the relationship between the specimen and the image as an optical aberration function and corrects optical aberrations through an arithmetic algorithm; compared to hardware-based methods, this approach has the advantage of allowing for a simpler and more affordable optical system configuration.

[0004] Conventional OBM technology requires performing simulations such as Monte-Carlo to predict light irradiation patterns, but the present invention does not require additional simulations because it directly predicts the light source pattern using an image sensor in the pupil area. Furthermore, conventional light source pattern prediction simulations require optical information regarding the scattering medium, making it difficult to apply OBM technology when such information is unavailable; however, the light source pattern image acquisition method of the present invention overcomes this limitation, enabling imaging of specimens located within any scattering medium.

[0005] This invention proposes two directions of advancement from existing OBM technology for high-resolution imaging using a reflective back-tilt microscope. First, high-resolution images are acquired by combining an arithmetic optical aberration restoration method. Second, specimens within any scattering medium can be imaged by directly acquiring the light source pattern in the pupil area. This enables the application of OBM imaging technology in a wider range of environments and allows for the acquisition of high-resolution images of improved quality through optical aberration correction. The problem to be solved

[0006] The problem that this invention aims to solve is to minimize image quality degradation caused by optical aberrations and to stably obtain high-resolution images in various scattering media. This enables the acquisition of clear and precise images even from complex specimens, making it useful for the precise analysis of biological tissues or industrial materials. Furthermore, since light source patterns can be directly acquired with a simple configuration, rapid application is possible without a separate simulation process, and it can help improve image quality in various environments. means of solving the problem

[0007] In one aspect, the present invention provides a high-resolution backward-slant microscope comprising: a specimen forming part in which a specimen complex comprising a specimen to be observed and a scatterer formed around said specimen may be disposed; a light source capable of irradiating light onto said specimen part; a light processing part comprising at least one beam splitter and configured such that light irradiated from said light source is scattered from said specimen complex and emitted to proceed; a specimen area image sensor for sensing light emitted in a first direction from said beam splitter; a perforated area image sensor for sensing light emitted from said beam splitter and emitted in a second direction which is different from said first direction; and a computational part for imaging said specimen to be observed from an image obtained from said specimen area image sensor and said perforated area image sensor.

[0008] In one embodiment, the operation unit can perform a first operation to derive a light source pattern from an image obtained from the face area image sensor using the following formula (A).

[0009] (A)

[0010] Here, u is a 2D spatial frequency coordinate, I_pupil() is an image obtained from the pupil area image sensor, S() is a light source pattern, const is a constant, and |P()| is an ideal pupil function expressed through the following formula (B),

[0011] (B)

[0012] Here, NA is the numerical aperture of the objective lens, and λ is the wavelength of the light source.

[0013] In one embodiment, the operation unit may perform a second operation to derive the transmission function of the specimen from the derived light source pattern through the following equation (C).

[0014] (C)

[0015] Here, t() is the transfer function of the specimen, u' is the 2D spatial frequency coordinate, r is the 2D spatial coordinate, F[] is the Fourier transform, j represents the imaginary part, and P() is the pupil function modeled through the following equation (D):

[0016] (D)

[0017] Here, z(m) is the weight (Zernike coefficient) of the m-th Zernike function, and Zm() is the m-th Zernike function.

[0018] In one embodiment, the operation unit can perform a third operation to derive the light absorption of the specimen and the light phase of the specimen from the derived transfer function of the specimen through the following equation (E).

[0019] (E)

[0020] Here, μ() is the light absorption of the specimen, and Φ() is the optical phase of the specimen.

[0021] In one embodiment, the operation unit can perform a fourth operation to derive a predicted acquired image through the following equation (F) from the light absorption of the derived specimen and the light phase of the specimen.

[0022] (F)

[0023] Here, ^I_l() is a predicted acquired image predicted under specific light irradiation conditions l, and H_l^μ() and H_l^Φ() are the light absorption transfer function and light phase transfer function, respectively, under specific light irradiation conditions l.

[0024] In one embodiment, the operation unit can perform a fifth operation to derive the weight (Zernike coefficient) of the Zernike function of the pupil function through an optimization technique expressed by the following equation (G) from the derived predicted acquisition image and the image obtained from the specimen area image sensor.

[0025] (G)

[0026] Here, I_l() is the image sensor image of the specimen area obtained under specific light irradiation condition l, and argmin is the argument of the minimum.

[0027] In one embodiment, the optimization technique may use Gradient descent or the Nelder-Mead simplex method.

[0028] In one embodiment, the operation unit updates the Purfill function through the weights of the Zernike function obtained through the fifth operation, and can repeat the fourth operation and the fifth operation at least once.

[0029] In one embodiment, the specimen complex, the beam splitter, and the specimen area image sensor may be formed on a first axis.

[0030] In one embodiment, the light processing unit may include one or more lenses formed on the first axis.

[0031] In one embodiment, at least some of the components of the specimen complex, the specimen area image sensor, or the light processing unit may be movable forward and backward along the first axis.

[0032] In one embodiment, the computation unit can image the specimen to be observed by performing computations on two or more different phases obtained by moving at least some of the components of the specimen complex, the specimen area image sensor, or the light processing unit forward and backward along the first axis.

[0033] In one embodiment, the computation unit can visualize three-dimensional physical quantity information for the specimen. Effects of the invention

[0034] The advantage of this invention is that it minimizes image distortion caused by optical aberrations and enables the acquisition of high-resolution images even in various scattering media. This allows for the provision of clearer and more accurate images, even in complex specimen structures or biological tissues, thereby contributing to increased reliability in research and diagnosis. Furthermore, since it can be applied with a simple equipment configuration, it can help improve cost efficiency in experimental environments and equipment setup. Brief explanation of the drawing

[0035] Figure 1 shows (a) a schematic diagram of an optical system and a structure composed of a light source and a pupil modulation imaging system. (b1-b2) illustrates a method in which the light source irradiates light onto a specimen, allowing for a detailed explanation of how light is irradiated onto the specimen. Figure 2 illustrates the process of light irradiation into a scattering medium and visually explains the process in which multiple scattered light within the scattering medium is scattered backward to illuminate the specimen in a transmissive manner. Figure 3 illustrates axial scanning in a 3D OBM image and shows that multiple components can move in the manner of ①-④ to obtain data from various focal planes of the specimen. Figure 4 shows a schematic diagram of the optical aberration restoration algorithm to visually explain the process of aberration restoration. Figure 5 shows the results of optical aberration restoration and correction in OBM images to which the present technology is applied. (a) shows the results of aberration correction in a 2D OBM image of a mouse retinal specimen, and (b) shows the results of aberration correction in a 3D OBM image of a Siemens phase target specimen, visually demonstrating that this technology can help improve image quality by correcting optical aberrations. Specific details for implementing the invention

[0036] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. Since the present invention is susceptible to various modifications and may take various forms, specific embodiments are illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to the specific disclosed forms, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the present invention. Similar reference numerals have been used for similar components in the description of each drawing. In the attached drawings, the dimensions of the structures are shown enlarged compared to the actual dimensions for the clarity of the present invention.

[0037] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as “comprising” or “having” are intended to specify the presence of the features, numbers, steps, actions, components, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, or combinations thereof. In the context of this specification, terms such as “about” may mean about ± 1%, about ± 2%, about ± 3%, about ± 4%, about ± 5%, about ± 6%, about ± 7%, about ± 8%, about ± 9%, or about ± 10% of the figures described in the specification.

[0038] In addition, the description of one aspect of the present invention may be applied identically or similarly to the description of other aspects for identical or similar configurations or terms.

[0039] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0040] A high-resolution backward-slanted microscope according to an embodiment of the present invention may include: a specimen forming part in which a specimen complex comprising a specimen to be observed and a scatterer formed around the specimen can be disposed; a light source capable of irradiating light onto the specimen part; a light processing part comprising at least one beam splitter and configured so that light irradiated from the light source is scattered from the specimen complex and emitted, and then propagates; a specimen area image sensor for sensing light emitted in a first direction from the beam splitter; a perforation area image sensor for sensing light emitted from the beam splitter and emitted in a second direction which is different from the first direction; and a computation part for imaging the specimen to be observed from the images obtained from the specimen area image sensor and the perforation area image sensor.

[0041] In the context of this specification, the term "high-resolution backward-oblique microscope" refers to equipment capable of obtaining high-resolution images by illuminating specimens that cannot be effectively observed by conventional transmission optical systems with light at an oblique angle from the rear. This microscope is a system capable of providing sharper images by analyzing light that has passed through various scattering media and correcting optical aberrations. Through this, precise analysis can be enabled even for complex structures or biological samples where conventional methods had limitations.

[0042] In the context of this specification, the term "observation target specimen" refers to a material or structure intended for optical analysis through a microscope. This specimen may have various forms, such as biological tissues or industrial materials, and may be located inside or outside a scattering medium. The observation target specimen plays a central role in the process where light is scattered or reflected by the backslanted illumination method and captured by an image sensor, and serves as the primary objective of imaging and analysis.

[0043] The role of the aforementioned "observation target specimen" is to provide various optical signals through the microscope based on the reflection, scattering, or absorption of light. This enables the optical analysis of the specimen's internal structure or surface details and serves as an essential element for obtaining high-resolution images. The method of interaction with the light source may vary depending on the characteristics of the specimen, and data based on this can be processed by the computational unit to be reconstructed into a final image.

[0044] In the context of this specification, the term "scatterer" refers to a medium or material located around a specimen under observation that serves to allow light irradiated from a light source to pass through or be scattered. This scatterer can play an important role in the imaging process by altering the path of light interacting with the specimen. The scatterer can help to more clearly reveal the structural or physical characteristics of the specimen through the reflection and refraction of light.

[0045] The role of the aforementioned "scatterer" is to scatter light irradiated by a light source before or after it reaches the specimen. This allows the various optical properties of the specimen to be revealed more clearly and serves as an important factor in providing data for high-resolution imaging. The scatterer also plays a role in enabling a better understanding of the internal structure of complex samples and can have a significant impact on the process of correcting optical aberrations.

[0046] In the context of this specification, the term "specimen complex" refers to a complex structure in which a specimen under observation and a scatterer existing in its vicinity are combined. This complex plays a crucial role in the process of generating and processing optical signals, signifying that the specimen and the scatterer operate as a single system. The specimen complex serves as an important subject of analysis for maximizing the optical performance of a backward-tilting microscope.

[0047] The role of the aforementioned "specimen complex" is to generate various optical signals through the interaction between the specimen and the scatterer. This enables the analysis of the fine structure and physical properties within the specimen, while the scatterer controls the light path to facilitate high-resolution imaging. The specimen complex serves as a crucial input element for processing and imaging data collected by the computational unit, contributing to accurate and detailed image reconstruction.

[0048] In the context of this specification, the term "optical system" refers to a concept that includes all components for illuminating a specimen with light and collecting light reflected or scattered therefrom. An optical system can play an important role in controlling the path of light and minimizing optical distortion occurring along that path. Furthermore, the design of the optical system can be optimized to suit the characteristics of the specimen, which can help improve the accuracy and efficiency of the experiment.

[0049] In the context of this specification, the term "light source" refers to a device for irradiating a specimen with light, which emits light at an appropriate wavelength and intensity to contribute to the analysis of the structural characteristics of the material under observation. The characteristics of the light source can be adjusted according to experimental conditions and the type of specimen, allowing it to adapt to various experimental situations. This enables a clearer understanding of the optical properties occurring in a specific specimen and plays an important role in obtaining high-resolution images.

[0050] The role of the aforementioned "light source" is to irradiate light onto the specimen to convert its structural characteristics into reflected or scattered light. The proper setting of the light source can significantly affect the accuracy and quality of the image, and clearer images can be obtained by adjusting the light intensity and wavelength according to the characteristics of the specimen.

[0051] In the context of this specification, the term "beam splitter" refers to a device that splits irradiated light into multiple paths and transmits them in different directions. A beam splitter enables the efficient dispersion and transmission of light within an optical system and can play a crucial role in facilitating the detection of light emitted in multiple directions by individual sensors. This provides an opportunity to analyze the optical properties of complex specimens from various angles.

[0052] The role of the aforementioned "beam splitter" is to separate the irradiated light into different paths, facilitating its arrival at multiple image sensors. This allows for the efficient processing of light scattered from the specimen and enables the collection of information from different angles at each sensor. This process allows for a more precise understanding of the specimen's overall structure and provides an essential step for obtaining high-resolution images.

[0053] The role of the aforementioned "light processing unit" is to detect light scattered from the specimen and process it into a form suitable for imaging. The light processing unit can generate an accurate signal tailored to the characteristics of the specimen by adjusting the light path and intensity, thereby improving image quality. This process can help minimize optical distortion and provide clearer and more precise images.

[0054] The role of the aforementioned "specimen area image sensor" is to detect light scattered from the specimen complex and thereby collect information regarding the shape and structure of the specimen. This sensor contributes to the detailed imaging of the specimen and, in particular, plays a crucial role in obtaining high-resolution images by accurately capturing optical signals emitted from scatterers surrounding the specimen. This enables the clear observation of fine structures or complex shapes within the specimen and helps improve image quality.

[0055] In the context of this specification, the term "pupil region" refers to a spatial area within an optical system where light is formed after passing through a specimen, reflecting the specimen's specific pattern and optical properties. This region is where information formed through the interaction between the light source and the specimen is concentrated, playing a crucial role in accurate imaging. In particular, the information collected from this region can be used to analyze and correct optical aberrations, ultimately aiding in the restoration of high-resolution images.

[0056] The role of the aforementioned "peer region image sensor" is to detect light collected in the peer region and gather information regarding the refractive index differences and optical characteristics of the specimen. This sensor precisely measures the scattering pattern of the specimen to provide data necessary for light source pattern analysis and aberration correction. Through this, the complex structure of the specimen can be clearly identified, and it can play an important role in the process of restoring high-resolution images.

[0057] In the context of this specification, the meaning of "imaging a specimen" refers to a process of irradiating a specimen with light, collecting the light reflected or scattered therefrom, and reproducing or upgrading the shape and structure of the specimen based on this. Through this, the complex internal structure or fine material properties of the specimen can be expressed in high resolution and can be utilized for the analysis and evaluation of specimens in various research environments. In particular, it can serve as a very useful tool for research involving biological specimens or complex materials.

[0058] Contrary to its intended meaning, the visualization of the aforementioned specimen does not necessarily have to be 'visual.' For instance, the results can be expressed by quantifying the specimen's physical or chemical properties or analyzing them through data. In such cases, rather than directly observing the shape or structure of the specimen, the approach can be to reconstruct or interpret the specimen's characteristics based on the analyzed data.

[0059] The role of the aforementioned "processing unit" is to reconstruct an image of the specimen by processing data collected from the specimen area image sensor and the face area image sensor. Based on the collected data, the processing unit analyzes the optical characteristics, scattering patterns, and light source patterns of the specimen, and thereby assists in generating a high-resolution image. Additionally, it can perform the role of improving image quality by correcting optical aberrations that may occur during the image reconstruction process.

[0060] The specific method by which the above-mentioned operation unit includes or is composed of an operation unit is not particularly limited, and any known device may be used as long as it performs the role identified through this specification. For example, computers, processors, microcontrollers, microprocessors, digital signal processors (DSPs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), artificial intelligence computing units (AI processors), network servers, cloud computing devices, distributed computing systems, edge computing devices, quantum computers, parallel computing devices, application-specific integrated circuits (ASICs), memory modules, data acquisition devices, neural network processors, high-speed computing devices, high-performance computing (HPC) systems, embedded systems, laptop computers, workstations, smartphones, tablets, portable computing devices, sensor processors, machine learning accelerators, IoT devices, industrial control computers, simulation devices, virtual machines, hybrid computing systems, general-purpose computing devices, signal analyzers, electronic device control systems, cluster computing devices, smart embedded devices, ultra-high-speed data processing devices, image processors, deep learning dedicated hardware, data center devices, distributed data processing devices, system-on-chip (SoC), database servers, superconducting computers, server farms, real-time processing systems, multi-threaded processors, network-attached storage (NAS), etc. may be used. As long as it helps to restore or visualize the image of the specimen as described above, the function of the above-described operation unit is not particularly limited.

[0061] In one embodiment, the operation unit can perform a first operation to derive a light source pattern from an image obtained from the face area image sensor using the following formula (A).

[0062] (A)

[0063] Here, u is a 2D spatial frequency coordinate, I_pupil() is an image obtained from the pupil area image sensor, S() is a light source pattern, const is a constant, and |P()| is an ideal pupil function expressed through the following formula (B), and

[0064] (B)

[0065] Here, NA is the numerical aperture of the objective lens, and λ is the wavelength of the light source.

[0066] In the context of this specification, the term "light source pattern" refers to information indicating how the intensity and distribution of light emitted from a light source are spatially arranged. This significantly affects how light is irradiated onto a specimen in a microscope or optical device and can serve as an important factor in reproducing an image of the specimen.

[0067] In the context of this specification, the role of the "first operation" is to derive a light source pattern based on optical data collected in the perfill region. This allows for a more accurate determination of the characteristics of the light irradiated onto the specimen and provides basic data necessary for correcting the specimen's transfer function and optical aberrations in subsequent steps.

[0068] In the context of this specification, "spatial frequency coordinates" refers to a coordinate system representing the spatial frequency components of an object, primarily used in mathematical transformations such as the Fourier transform. This is useful for analyzing fine frequency components of images or signals and can help to optically represent the detailed structure of a specimen more clearly.

[0069] In the context of this specification, the term "ideal pupil function" refers to a function representing the propagation of light passing through the pupil region under ideal conditions of an optical system, defining the distribution of light based on the numerical aperture of the optical system. This is a function that assumes the ideal propagation of light in a perfect optical system without optical aberrations.

[0070] In the context of this specification, "numerical aperture" refers to a numerical value indicating the ability of an optical system to collect light; a larger numeric aperture collects more light, allowing for clearer observation of the fine structure of a specimen. The numeric aperture serves as an important factor in determining the performance of an objective lens.

[0071] In one embodiment, the operation unit may perform a second operation to derive the transmission function of the specimen from the derived light source pattern through the following equation (C).

[0072] (C)

[0073] Here, t() is the transfer function of the specimen, u' is the 2D spatial frequency coordinate, r is the 2D spatial coordinate, F[] is the Fourier transform, j represents the imaginary part, and P() is the pupil function modeled through the following equation (D):

[0074] (D)

[0075] Here, z(m) is the weight (Zernike coefficient) of the m-th Zernike function, and Zm() is the m-th Zernike function.

[0076] In the context of this specification, the role of the "second operation" is to calculate the transfer function of the specimen based on the derived light source pattern. This operation contributes to estimating the transfer function that explains the intensity and phase change of light passing through the specimen, and can provide basic data for precisely analyzing the physical properties and optical changes of the specimen. Through this, complex internal structures or differences in refractive index can be identified, and it can play an important role in the subsequent image reconstruction process.

[0077] In the context of this specification, the term "transmission function" refers to a function representing the intensity and phase of light passing through a specimen, which explains how the optical properties of the specimen change during the passage of light. This function is determined by physical properties of the specimen, such as its shape, refractive index, and thickness, and can ultimately play an important role in reconstructing or analyzing images of the specimen.

[0078] In the context of this specification, the term "Zernike function" refers to a function used to mathematically express optical aberrations, particularly for modeling distortions in the furfill function. The Zernike function is, for example, a set of functions having circular symmetry, and is useful for describing various types of aberrations (e.g., spherical aberration, coma aberration) occurring in optical systems, thereby enabling precise analysis and correction of optical aberrations.

[0079] In one embodiment, the operation unit can perform a third operation to derive the light absorption of the specimen and the light phase of the specimen from the derived transfer function of the specimen through the following equation (E).

[0080] (E)

[0081] Here, μ() is the light absorption of the specimen, and Φ() is the optical phase of the specimen.

[0082] In the context of this specification, the role of the "third operation" is to calculate the light absorption (μ) and optical phase (Φ) of the specimen based on the derived transfer function of the specimen. This operation plays a crucial role in analyzing how much light passing through the specimen is absorbed within the specimen and how its phase changes. This enables a precise determination of the physical properties and optical changes of the specimen and can help in gaining a deeper understanding of the specimen's internal structure and material properties.

[0083] In one embodiment, the operation unit can perform a fourth operation to derive a predicted acquired image through the following equation (F) from the light absorption of the derived specimen and the light phase of the specimen.

[0084] (F)

[0085] Here, ^I_l() is a predicted acquired image predicted under specific light irradiation conditions l, and H_l^μ() and H_l^Φ() are the light absorption transfer function and light phase transfer function, respectively, under specific light irradiation conditions l.

[0086] In the context of this specification, the role of the "fourth operation" is to calculate a predicted acquired image under specific light irradiation conditions based on the light absorption (μ) and light phase (Φ) information of the derived specimen. This operation is a process of predicting optical changes after passing through the specimen by utilizing light absorption and light phase transfer functions, and thereby producing an image that can be observed in an actual experiment.

[0087] In one embodiment, the operation unit can perform a fifth operation to derive the weight (Zernike coefficient) of the Zernike function of the pupil function through an optimization technique expressed by the following equation (G) from the derived predicted acquisition image and the image obtained from the specimen area image sensor.

[0088] (G)

[0089] Here, I_l() is the image sensor image of the specimen area obtained under specific light irradiation condition l, and argmin is the argument of the minimum.

[0090] In the context of this specification, the role of the "fifth operation" is to compare the derived predicted acquisition image with the actual image collected from the specimen area image sensor to optimize the Zernike function weight (z) of the purfill function. This minimizes the difference between the predicted image and the actual image and allows for a more accurate reflection of the physical characteristics of the specimen. This process can provide important basic data for image restoration and optical aberration correction.

[0091] In the context of this specification, the term "optimization technique" refers to a mathematical method that varies the value of the target variable (z) in a given function so that the output of the function reaches a minimum (or maximum) value. The goal of this process is to derive optimal results by reducing the error between experimental data and predicted data, and it can be utilized here to minimize the difference between the predicted acquired image and the actual measured image.

[0092] In the context of this specification, "argmin" or "argument of the minimum" refers to the value of a variable that produces the minimum value when a specific function has a minimum value. That is, it is a process of finding the input value at which the result of a given function is minimized, thereby enabling the solution of optimization problems. In the context of the present invention, it is used to find the weights of a Zernike function that minimize the difference between the predicted acquired image and the actual image.

[0093] As long as the functions described above are performed, there are no particular limitations on how the optimization technique is executed. In one embodiment, the optimization technique may use Gradient descent or the Nelder-Mead simplex method. "Gradient descent" refers to an optimization algorithm that iteratively moves in the direction where the function value is minimized by adjusting the target variable in a given function. This method calculates the gradient of the function and reaches the minimum value by gradually moving in the direction with the smallest gradient. Gradient descent is widely used, particularly for continuous and differentiable functions, and is applicable to large data or complex functions. "Nelder-Mead simplex method" refers to an optimization method that finds the minimum value of a function using polygons in a multidimensional space called simplexes, without calculating the gradient. This method can be used for non-differentiable or non-linear functions and approaches the optimal solution by gradually improving the function value. In particular, the Nelder-Mead method has the advantage of not requiring gradient calculation, making it useful for optimization problems involving complex functions.

[0094] In one embodiment, the operation unit updates the perfill function using the weights of the Zernike function obtained through the fifth operation, and can repeat the fourth operation and the fifth operation at least once. Through the iterative operation, the accuracy of the perfill function can be gradually improved, and accordingly, the light absorption and optical phase information of the specimen can be derived more precisely. By performing the iterative fourth and fifth operations, the error between the predicted acquired image and the actual image is reduced, and the analysis results regarding the physical and optical characteristics of the specimen are continuously improved. This process can ultimately help obtain a high-resolution image and contribute to image quality improvement, such as optical aberration correction.

[0095] In one embodiment, the specimen complex, the beam splitter, and the specimen area image sensor may be formed on a first axis. That is, the specimen complex, the beam splitter, and the specimen area image sensor may be formed coaxially on the first axis. In one embodiment, the light processing unit may include one or more lenses formed on the first axis. In one embodiment, at least some of the components of the specimen complex, the specimen area image sensor, or the light processing unit may be movable forward and backward on the first axis. The reason for being designed to be movable forward and backward by being formed coaxially is to obtain images at various focal planes to acquire three-dimensional physical quantity information of the specimen. As the components of the specimen complex, the specimen area image sensor, or the light processing unit move forward and backward along the first axis, various information of the specimen can be collected at different depths. Through this, the refractive index distribution, structural depth changes, or complex internal shapes of the specimen can be analyzed more precisely, and it can assist in high-resolution three-dimensional image reconstruction and physical property analysis. This movement can be effectively utilized, especially even when there are large differences in specimen thickness or refractive index.

[0096] In one embodiment, the computation unit can image the specimen to be observed by performing computations on two or more different images obtained by moving at least some of the components of the specimen complex, the specimen area image sensor, or the light processing unit forward and backward along the first axis. In one embodiment, the computation unit can image three-dimensional physical quantity information regarding the specimen. Through this, instead of a simple two-dimensional image of the specimen, three-dimensional information including the refractive index distribution or thickness change of the specimen can be obtained. In particular, by combining data obtained from various focal planes, the internal structure of the specimen can be precisely reconstructed, and based on this, a high-resolution three-dimensional image is made possible that allows for a more in-depth analysis of the physical characteristics of the specimen.

[0097] Embodiments of the present invention are described below. However, the embodiments described below are merely partial embodiments of the present invention, and the scope of the present invention is not limited to the following embodiments.

[0098] [[Theoretical Background of the Present Invention]]

[0099] [Fig. 1] is a schematic diagram of an optical system for high-resolution OBM imaging used in the present invention. The developed optical system is broadly divided into a light source section and a pupil modulation imaging system section, wherein the light source section serves to irradiate light onto the specimen. To implement oblique back illumination, the present invention irradiates light directly onto a scattering medium through the light source section of [Fig. 1]. The irradiated light undergoes multiple scattering or backscattering processes within the scattering medium, thereby enabling the specimen to be irradiated in a transmissive manner. Through this process, specimens with relatively weak scattering characteristics can be imaged, and it can help obtain high-resolution images for various types of specimens.

[0100] 3D image reconstruction can be constructed using the same optical system as in [Fig. 1], and to obtain high-resolution 3D images, additional axial scanning of the specimen is required. Axial scanning can be acquired by moving the specimen itself or by moving the lens or image sensor, as shown in [Fig. 3]. By collecting information about the specimen at various depths, this method can help to more accurately analyze structural changes or physical properties in the depth direction. This provides the possibility to effectively reconstruct 3D images even from specimens with complex internal structures or multilayer structures.

[0101] [Signal Processing 1: Light Source Pattern Image Acquisition]

[0102] In this technique, instead of complex methods such as conventional Monte Carlo simulations to predict light source patterns, a method of directly acquiring the light source in the pupil region is adopted. This approach is advantageous for most thin biological samples with weak scattering characteristics. For such samples, the specimen's transmission function t() can be linearized by reflecting the specimen's light absorption and optical phase information, which is expressed as follows.

[0103] (Equation 1; E)

[0104] Here, r represents a 2D spatial coordinate, μ() represents the light absorption of the specimen, and Φ() represents the optical phase of the specimen.

[0105] At this time, the light scattered by interacting with the light source pattern S() can be represented in the Pupil region as follows.

[0106] (Equation 2; C)

[0107] Here, u represents the 2D spatial frequency coordinate, F[] represents the 2D Fourier transform, and P(u) represents the Pupil function of the optical system.

[0108] By combining [Equation 1] and [Equation 2], information on the light source pattern S() can be obtained from the light intensity I_Pupil() acquired in the Pupil region as shown below.

[0109] (Equation 3; A)

[0110] Here, const is a constant.

[0111] Here, the absolute value of the pupil function |P(u)| is a known function expressed by [Equation 4] below. Therefore, light source pattern information can be obtained from the image I_Pupil(u) acquired from the pupil region.

[0112] (Equation 4; B)

[0113] Here, NA is the numerical aperture of the lens, and λ is the wavelength of the light source.

[0114] [Signal Processing 2: Optical Aberration Restoration and Correction in 2D OBM Images]

[0115] The inventors present an arithmetic optical aberration restoration and correction technique that can correct optical aberrations without requiring expensive equipment or complex optical system configurations. Through this, information on light absorption, optical phase, and optical aberrations can be predicted simultaneously, and optical aberrations caused by the optical system or specimen can be effectively overcome to restore high-resolution images of light absorption and optical phase. In this technique, the Pupil function of the optical system containing optical aberration information is modeled as a linear sum of Zernike functions as shown in [Equation 5], and the optical aberration and image restoration process is performed by constructing an inverse problem based on images acquired from multiple Pupil regions and specimen regions. This process is explained through the schematic diagram of the algorithm presented in [Fig. 4].

[0116] (Equation 5; D)

[0117] {Iterative Arithmetic Algorithm}

[0118] 1) Optical aberration correction is performed by calculating the predicted image (^I_l()) obtained under specific light irradiation conditions (l) in Fourier space using the light absorption (μ) and light phase (Φ) information extracted in the previous iteration process based on [Equation 6]. The predicted image is expressed through the transfer function of the optical system and the light absorption / light phase information, and the transfer function is determined by the light source pattern and the Pupil function.

[0119]

[0120] (Equation 6; F)

[0121] Here, H_l^μ() and H_l^Φ() are the light absorption transfer function and the light phase transfer function, respectively, under specific light irradiation condition l.

[0122] 2) The weights (z) of the Zernike function are calculated in a direction that reduces the difference between the predicted image (^I_l()) and the actual image (I_l()) obtained from the experiment by applying optimization techniques such as Gradient descent or the Nelder-Mead simplex method. This process is carried out using [Equation 7].

[0123] (Equation 7; G)

[0125] 3) A Pupil function is constructed based on the weights of the predicted Zernike function, and this is applied to the transfer function of the optical system to restore the light absorption and optical phase information of the specimen.

[0126] 4) By repeatedly performing this process, more precise light absorption and optical phase information with progressively corrected optical aberrations can be restored. Through repetition, the accuracy of the results is improved, and finally, a high-resolution aberration-corrected image can be obtained.

[0127] [Signal Processing 3: Optical Aberration Restoration and Correction in 3D OBM Images]

[0128] A three-dimensional specimen can be represented by a scattering potential containing refractive index distribution information within three-dimensional space. In this case, the scattering potential (V(r3D)) can be expressed by the following Equation 8:

[0129] (Equation 8)

[0130] Here, r3D represents three-dimensional spatial coordinates, n(r3D) represents the three-dimensional refractive index distribution of the specimen, n_m represents the refractive index of the medium surrounding the specimen, and k0 represents the wave number of the irradiated light.

[0131] When the specimen causes weak scattering, the scattering potential can be approximated and expressed again.

[0132] (Equation 9)

[0133] Here, Δn represents the difference in refractive index between the surrounding medium and the specimen, and is defined as Δn(r3D)=n(r3D)-n_m.

[0134] Three-dimensional refractive index information (or scattering potential information) can be derived from the Helmholtz equation (Equation 10).

[0135] (Equation 10)

[0136] Here, ∇ is the Laplacian operator, and U(r3D) is the electric field after interacting with the specimen.

[0137] Here, using the first Born approximation with a single scattering assumption, the solution to the above equation can be expressed in terms of the Helmholtz Green function (G(r_3D)) and the electric field information of the incident light (U_i (r_3D)) (Equation 11).

[0138] (Equation 11)

[0139] To summarize, the 3D acquired images obtained by various axial scanning methods in [Fig. 3] can be expressed by linearizing them into the transfer function of the optical system in Fourier space as shown in Equation 12 below. In this case, the transfer function of the optical system is determined by the pupil function of the optical system.

[0140]

[0141] (Equation 12)

[0142] Here, u_3D is a 3D spatial frequency coordinate, V_Re(r_3D) and V_Im(r_3D) are the real / imaginary parts of the scattering potential, respectively, and H_Re(u_3D;P(u)) and H_Im(u_3D;P(u)) represent the transfer functions of the optical system for the real / imaginary parts of the scattering potential.

[0143] The optical aberration restoration method operates based on the image transfer model of Equation 12, and the detailed restoration method is identical to the iterative arithmetic algorithm of Signal Processing 2: Optical Aberration Restoration and Correction in 2D OBM Images described above.

[0144] [[Experimental Proof]]

[0145] To demonstrate the utility of the present technology, the results of optical aberration restoration in actual 2D and 3D OBM images are included in [Fig. 5]. In [Fig. 5 (a)], it can be seen that optical aberrations were restored and corrected in a 2D OBM image of a mouse retina specimen, resulting in improved contrast and image quality in the optical phase image. Additionally, in [Fig. 5 (b)], optical aberrations were restored and corrected in a 3D OBM image of a Siemens star phase target specimen defocused in the z-direction, resulting in corrected defocus in the z-direction of the 3D refractive index image. Through this, it can be proven that the present technology can effectively restore optical aberrations occurring in 2D and 3D images.

[0146] Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as set forth in the following claims.

Claims

Claim 1 A high-resolution backward-slanted microscope comprising: a specimen forming part in which a specimen complex including a specimen to be observed and a scatterer formed around said specimen can be disposed; a light source capable of irradiating light onto said specimen forming part; a light processing part comprising at least one beam splitter and configured so that light irradiated from said light source is scattered and emitted from said specimen complex and propagates; a specimen area image sensor for sensing light emitted in a first direction from said beam splitter; a perforation area image sensor for sensing light emitted from said beam splitter and emitted in a second direction which is different from said first direction; and a computation part for imaging said specimen to be observed from an image obtained from said specimen area image sensor and said perforation area image sensor. Claim 2 In claim 1, the above-described operation unit performs a first operation to derive a light source pattern from an image obtained from the above-described fill area image sensor using the following formula (A), a high-resolution backward-slanted microscope: (A) Here, u is a 2D spatial frequency coordinate, I_pupil() is an image obtained from the pupil area image sensor, S() is a light source pattern, const is a constant, and |P()| is an ideal pupil function expressed by the following formula (B), (B) Here, NA is the numerical aperture of the objective lens, and λ is the wavelength of the light source. Claim 3 In paragraph 2, the above-described operation unit performs a second operation to derive the transmission function of the specimen from the derived light source pattern through the following equation (C), a high-resolution backward-slanted microscope: (C) Here, t() is the transfer function of the specimen, u' is the 2D spatial frequency coordinate, r is the 2D spatial coordinate, F[] is the Fourier transform, j represents the imaginary part, and P() is the pupil function modeled through the following equation (D): (D) Here, z(m) is the weight (Zernike coefficient) of the m-th Zernike function, and Zm() is the m-th Zernike function. Claim 4 In paragraph 3, the above-described operation unit performs a third operation to derive the light absorption of the specimen and the light phase of the specimen from the derived transfer function of the specimen through the following equation (E), a high-resolution backward-slanted microscope: (E) Here, μ() is the light absorption of the specimen, and Φ() is the optical phase of the specimen. Claim 5 In claim 4, the above-described operation unit performs a fourth operation to derive a predicted acquisition image through the following equation (F) from the light absorption of the derived specimen and the light phase of the specimen, a high-resolution backward-slanted microscope: (F) Here, ^I_l() is the predicted acquired image predicted under specific light irradiation condition l, and H_l^μ() and H_l^Φ() are the light absorption transfer function and light phase transfer function, respectively, under specific light irradiation condition l. Claim 6 In claim 5, the above-described operation unit performs a fifth operation to derive the weight (Zernike coefficient) of the Zernike function of the pupil function through an optimization technique expressed by the following equation (G) from the above-described predicted acquisition image and the image obtained from the specimen area image sensor, a high-resolution backward-slanted microscope: (G) Here, I_l() is the image sensor image of the specimen area obtained under specific light irradiation condition l, and argmin is the argument of the minimum. Claim 7 In claim 6, the optimization technique is a high-resolution backward-slanted microscope using Gradient descent or the Nelder-Mead simplex method. Claim 8 A high-resolution backward-slanted microscope according to claim 7, wherein the operation unit updates the perfill function through the weights of the Zernike function obtained through the fifth operation, and repeats the fourth operation and the fifth operation at least once. Claim 9 In claim 8, a high-resolution backward-tilting microscope in which the specimen complex, the beam splitter, and the specimen area image sensor are formed on a first axis. Claim 10 In claim 9, the light processing unit comprises one or more lenses formed on the first axis, a high-resolution backward-slanted microscope. Claim 11 In claim 9, a high-resolution rearward tilting microscope in which at least some of the constituent members of the specimen complex, the specimen area image sensor, or the light processing unit are movable forward and backward along the first axis. Claim 12 A high-resolution backward-slanted microscope according to claim 11, wherein the computational unit performs computation from two or more different phases obtained by moving at least a part of the specimen complex, the specimen area image sensor, or the constituent member of the light processing unit forward and backward along the first axis to image the specimen to be observed. Claim 13 In claim 12, the above-described processing unit is a high-resolution backward-slanted microscope that visualizes three-dimensional physical quantity information for the specimen.

Images