METHOD FOR DIGITAL CORRECTION OF AN OPTICAL IMAGE OF A SAMPLE USING A MICROSCOPE AND MICROSCOPE

Description

[0001] The invention relates to a method for digitally correcting an optical image of a sample using a microscope, wherein the microscope has a cover glass covering the sample. The invention further relates to a microscope with a device for digitally correcting an optical image of a sample.

[0002] The primary influence on the quality of a light microscopic image of a sample is sample-related aberrations. These can typically be determined using special measuring techniques. For this purpose, state-of-the-art methods for deconvolution of image data acquired in fluorescence microscopy using computer programs are known. These known methods take into account, in particular, spherical aberration induced by refractive index mismatch. A disadvantage of these known methods is that it is left to the operator to determine the information necessary to determine the refractive index mismatch and feed it into the computer program. Furthermore, these known methods do not take coverslip tilt into account.

[0003] The publication DE 102017105928 A1 describes a method for digitally correcting an optical image of a sample using a microscope with a coverslip covering the sample. The microscope can determine the refractive index of an optical medium adjacent to the coverslip, a tilt of the coverslip, and / or the thickness of the coverslip.

[0004] US 2018 / 018758 A1 describes determining the position of an optical interface using a pupil function. US 2017 / 178317 A1 describes the acquisition of multiple microscope images, from whose partial spectral images a distortion map of the overall image is calculated and corrected accordingly. US 2005 / 265621 A1 describes a method for iteratively calculating the point spread function in the blind deconvolution of microscope images.

[0005] DE 102 50 775 A1 discloses a method for determining a point spread function(hereinafter referred to as PSF) and deconvolution of an image of an object using a microscope based on this PSF. In this method, a parameter of an approximate PDF is varied until the comparison between the recorded image and the calculated image is positive. Based on the PSF thus determined, the image of the object can be corrected.

[0006] US 7,489,828 B2 discloses a method for determining a PSF and digitally correcting an image of a sample using a microscope based on this PSF. In this method, the PSF for which a characteristic value for the image quality is maximum is selected from a plurality of PSFs. The spherical aberration used to determine this PSF is then assumed to be the spherical aberration actually present in the microscope.

[0007] Regarding the state of the art, reference is also made to Kam et al.: "Computational adaptive optics for live three-dimensional biological imaging", Proceedings of the Nation Academy of Sciences, Vol. 98, No. 7, March 27, 2001, pages 3790-3795, XP002610823 and EP 1 988 417 B1.

[0008] It is an object of the invention to provide a method for digitally correcting an optical image of a sample by means of a microscope and a microscope with a device for digitally correcting an optical image of a sample, which enable a correction of the image taking into account measurable properties of the sample and the sample space.

[0009] In the method according to the invention for digitally correcting an optical image of a sample by means of a microscope, wherein the microscope has a cover glass covering the sample, the refractive index of an optical medium adjacent to the cover glass, a tilt of the cover glass and / or the thickness of the cover glass are determined by the microscope, an imaging error to be corrected in the form of a pupil function is determined on the basis of the refractive index of the optical medium, the tilt of the cover glass and / or the thickness of the cover glass, the image of the sample is imaged through the microscope and image data acquired by the image of the sample are digitally corrected on the basis of the pupil function.

[0010] According to the invention, the microscope determines properties of the sample to be imaged and the sample chamber. The aberration is then determined from the properties determined by the microscope. This is advantageous over known methods in which the aberration is determined ex situ, for example, using a reference preparation, since in this case, no correlation can be ensured between the aberration determined using the reference preparation and the aberration when imaging the sample.

[0011] The imaging error is determined, for example, by modeling a wavefront error from the properties of the sample and the sample space determined by the microscope. Models for calculating a wavefront error based on properties of the sample and the sample space are known, for example, from P. Török et al., "Electromagnetic diffraction of light focused through a planar interface between materials of mismatched refractive indices: an integral representation," J. Opt. Soc. Am. A 12 (1995), 325-332, or from S. Stallinga, "Compact description of substrate-related aberrations in high numerical-aperture optical disk readout," Appl. Opt. 44 (2005), 849-858.

[0012] From the refractive index of the optical medium adjacent to the cover glass and the thickness of the cover glass, the aberration can be determined as the pupil function of an objective lens: ψ r = z n Spc 2 − r 2 NA 2 + Δ d n Dgl 2 − r 2 NA 2 − z + Δ d n Imm 2 − r 2 NA 2

[0013] Where r is a radial coordinate of the exit pupil of the lens normalized to the unit circle, z a mechanical focal length, n Spc the refractive index of the optical medium adjacent to the cover glass, n Imm the known refractive index of another optical medium placed between the objective and the cover glass, for example air or an immersion oil, Δd a deviation of the cover glass thickness along the optical axis of the objective from the nominal cover glass thickness and NA the numerical aperture of the lens.

[0014] The pupil function contains the essential properties of the image, i.e., the imaging behavior of the optical system formed by the sample, sample chamber, and microscope. Based on the pupil function, sample properties, such as the spatial distribution of fluorophores or a refractive index distribution, can be reconstructed from the image data obtained by the imaging. This allows for image correction.

[0015] In particular, the optical medium is an embedding medium in which the sample is embedded. Furthermore, an immersion medium is arranged between the microscope objective and the coverslip. The imaging error to be corrected is determined based on the known refractive index of the immersion medium.

[0016] In a preferred embodiment, a digital filter is generated based on the pupil function to correct the image data acquired by imaging the sample. This is the simplest way to correct the image data acquired during imaging of the sample. By applying the digital filter to the image data acquired by imaging the sample, the image is inverted by the optical system formed by the sample, sample chamber, and microscope, which allows the imaging errors induced by the properties of the sample and the sample chamber to be removed from the image data acquired by imaging the sample.

[0017] In an advantageous further development, phase information from the detection light emitted by the sample is reconstructed using the digital filter. The pupil function can be used to create so-called inverse filters for quantitative phase reconstruction methods. Quantitative phase reconstruction methods include, for example, imaging techniques based on the transport of intensity equation, as described, for example, in D. Paganin, KA Nugent, "Noninterferometric Phase Imaging with Partially Coherent Light", Phys. Rev. Lett. 80 (1998), 2586; coherent interference microscopy, as described, for example, in M. Mir et al., "Quantitative Phase Imaging", Progress in Optics 57 (2012), 133-217; or differential phase contrast microscopy, as described, for example, in M. Chen et al., "3D differential phase contrast microscopy", Biomed. Opt. Express 7 (2016), 3940-3950.By reconstructing phase information, samples that absorb little light, so-called phase objects, can also be imaged.

[0018] Preferably, the imaging creates an image of a plane within the sample, and the image is corrected by inverting the image based on the pupil function. In a light microscopic imaging of the plane within the sample, the image data is, in particular, a two-dimensional matrix consisting of pixels. A digital filter determined from the pupil function is applied to the image data to invert the image through the microscope.

[0019] In a preferred embodiment, a point spread function is generated based on the pupil function, and a deconvolution operation is performed on the image data acquired by imaging the sample using the point spread function to correct the image. The point spread function specifies how an ideal point-like object is imaged by the microscope. It can therefore be used, in particular, to deconvolve image data acquired in fluorescence microscopy, as described, for example, in Chapter 25, J. Pawley, "Handbook of Biological Confocal Microscopy," 3rd Edition, Springer, ISBN 978-0-387-25921-5. This can be image data from epifluorescence microscopy, confocal microscopy, or multiphoton microscopy, for example. The point spread function can be determined from the pupil function using methods known, for example, from M. Leutenegger et al., "Fast focus field calculations", Opt. Express 14 (2006), 11277-11291.

[0020] Preferably, the imaging of the sample and the correction of the image data acquired by the imaging of the sample are performed repeatedly. This allows either multiple planes within the sample to be acquired to generate a three-dimensional image or multiple images of a single plane within the sample to generate a video.

[0021] In a preferred embodiment, the microscope detects a tilt of the coverslip, and the aberration to be corrected is determined based on the tilt. The tilt of the coverslip induces an asymmetric aberration, which is difficult to correct using purely optical means but can be easily corrected digitally.

[0022] In one embodiment, the sample is imaged in several scanning steps, each of which illuminates a portion of the sample and / or excites fluorescence. In each scanning step, only a point-shaped portion of the sample is illuminated and / or excited to fluorescence, for example, by illumination with laser light.

[0023] The invention further relates to a microscope with a cover glass covering a sample, wherein the microscope comprises a device for digitally correcting an optical image of the sample. The device is designed to determine the refractive index of an optical medium adjacent to the cover glass, a tilt of the cover glass, and / or the thickness of the cover glass, to determine the imaging error to be corrected in the form of a pupil function based on the refractive index of the optical medium, the tilt of the cover glass, and / or the thickness of the cover glass, and to digitally correct image data acquired by the image of the sample based on the pupil function.

[0024] The microscope is, for example, a confocal microscope. Alternatively, the microscope can be a transmitted-light microscope or a reflected-light microscope.

[0025] Further features and advantages of the invention will become apparent from the following description, which explains the invention in more detail using exemplary embodiments in conjunction with the attached figures.

[0026] They show: Figure 1 shows a schematic representation of a microscope as an exemplary embodiment; and Figure 2 shows a flowchart of an embodiment of a method for digitally correcting an optical image of a sample using the microscope according to Figure 1.

[0027] Figure 1shows a microscope 10 as an exemplary embodiment. The microscope 10 has an objective 12 directed onto a cover glass 14. The microscope 10 further comprises a control unit 16, which is connected to the objective 12 via an external cable 18 and which controls the various objective components. An optical medium 20, in which a sample is embedded and which is referred to below as embedding medium 20, is located on the cover glass 14. Furthermore, an immersion medium 22 is introduced between the objective 12 and the cover glass 14.

[0028] Figure 2 shows a flow chart of an embodiment of the method according to the invention for digitally correcting an optical image of a sample by means of the microscope according to Figure 1 .

[0029] In a first step S1, the thickness of the cover glass 14 is determined along the optical axis O of the objective lens 12. For this purpose, a measuring light beam is directed, for example, at oblique incidence through the objective lens 12 onto the cover glass 14. Partial reflections of the measuring light beam at a first optical interface formed by the cover glass 14 and the immersion medium 22 and at a second optical interface formed by the cover glass 14 and the embedding medium 20 generate two spatially separated reflected light beams. The two reflected light beams are received by the objective lens 12 and directed onto a position-sensitive detector. The thickness of the cover glass can be determined based on the detected locations of incidence of the two reflected light beams on the position-sensitive detector.

[0030] Furthermore, in the first step S1, the refractive index of the embedding medium 20 is determined. This can be done, for example, by detecting not only the point of incidence but also the respective intensities of the two reflected light beams using the position-sensitive detector. Based on the detected intensities of the two reflected light beams, the refractive index of the embedding medium 20 can be determined, since the intensities of the two reflected light beams depend on the reflection and transmission of the measuring light beam at the two interfaces defined by the cover glass 14 and the embedding medium 20 or the immersion medium 22. The reflection and transmission processes on which the detected intensities of the two spatially separated reflected light beams are based are thus essentially determined by the refractive indices of the cover glass 14 as well as the embedding medium 20 and the immersion medium 22.

[0031] In a second step S2, a tilt of the cover glass 14 is determined. For this purpose, at least three measuring points spanning a plane are defined, for example, on a surface of the cover glass 14. The following steps are performed for each of the three measuring points: directing a measuring light beam through the objective 12 onto the measuring point; generating a reflected light beam by at least partially reflecting the measuring light beam at the respective measuring point; directing the reflected light beam through the objective onto a position-sensitive detector; detecting the point of incidence of the reflected light beam on the position-sensitive sensor; and determining the distance of the respective measuring point from the objective 12 along its optical axis O based on the detected point of incidence.Subsequently, on the basis of the determined distances, a tilt of the plane spanned by the three measuring points relative to the optical axis O of the objective 12 is determined as a tilt of the cover glass 14.

[0032] In a third step S3, the imaging error in the form of a pupil function is determined from the thickness of the cover glass 14, the refractive index of the embedding medium 20 and the tilt of the cover glass 14 as ψ r φ = Δ z n Spc 2 − r 2 NA 2 − Δ d + Δ z n Imm 2 − r 2 NA 2 + d + Δ d n Dgl 2 − r NA cos α cos φ − φ 0 − sin α n Imm 2 − r 2 NA 2 2 − r 2 NA 2 sin 2 φ − φ 0 − r NA sin α cos φ − φ 0 − cos α n Imm 2 − r 2 NA 2 − d n Dgl 2 − r 2 NA 2 − n Imm 2 − r 2 NA 2

[0033] Where r is a radial coordinate of the exit pupil of the lens 12 normalized to the unit circle, ϕ the angular coordinate of the exit pupil of the lens 12, z a mechanical focal length, n Spc the refractive index of the embedding medium 20 adjacent to the cover glass 14, n Imm the known refractive index of the immersion medium 22 introduced between the objective 12 and the cover glass 14, Δdthe deviation of the thickness of the cover glass 14 along the optical axis O of the objective 12 from the nominal thickness of the cover glass 14, NA the numerical aperture of the lens 12, n Dgl the refractive index of the cover glass 14, α the azimuth of the tilt and ϕ 0 is the polar angle of tilt.

[0034] In a fourth step S4, image data is generated by the microscope 10 by imaging the sample.

[0035] In a fifth step S5, the image data are processed based on the pupil function ψ ( r , ϕ ) according to equation (1) above. For example, the pupil function ψ ( r , ϕ) generates a digital filter that reverses the process of imaging the sample through the microscope 10 in order to correct the imaging error. Alternatively, the image data can be deconvolutioned using a function derived from the pupil function ψ ( r , ϕ ) obtained point spread function. List of reference symbols

[0036] 10Microscope 12Objective 14Cover glass 16Control unit 18Cable 20Embedding medium 22Immersion medium P1Arrow

Claims

1. Method for digitally correcting an optical image of a sample by means of a microscope (10), wherein the microscope (10) has a cover glass (14) covering the sample, wherein the refractive index of an optical medium (20) bordering the cover glass (14), a tilt of the cover glass (14) and / or the thickness of the cover glass (14) are determined by the microscope (10), characterised in that, on the basis of the refractive index of the optical medium (20), the tilt of the cover glass (14) and / or the thickness of the cover glass (14), an imaging error to be corrected in the form of a pupil function is determined, the imaging of the sample is performed by the microscope (10) and image data, acquired by the imaging of the sample, are digitally corrected on the basis of the pupil function.

2. Method according to claim 1, characterised in that, on the basis of the pupil function, a digital filter is generated for correcting the image data acquired by the imaging of the sample.

3. Method according to claim 2, characterised in that, with the aid of the digital filter, phase information of detection light, emitted by the sample, is reconstructed.

4. Method according to any one of claims 1 to 3, characterised in that, by an inversion of the imaging on the basis of the pupil function, the imaging produces an image of a plane, lying in the sample, and the image is corrected.

5. Method according to any one of claims 1 to 4, characterised in that, on the basis of the pupil function, a point spread function is generated, and by means of the point spread function a deconvolution operation is carried out on the image data acquired by imaging the sample in order to correct the imaging.

6. Method according to any one of claims 1 to 5, characterised in that the imaging of the sample and the correction of the image data, acquired by the imaging of the sample, are carried out repeatedly.

7. Method according to any one of claims 1 to 6, characterised in that the imaging of the sample takes place in several raster steps, in each of which a partial area of the sample is illuminated and / or excited to fluorescence.

8. Method according to any one of claims 1 to 7, characterised in that the optical medium (20) is an embedding medium in which the sample is embedded.

9. Method according to any one of claims 1 to 8, characterised in that an immersion medium (22) is arranged between an objective of the microscope (10) and the cover glass (14) and the imaging error to be corrected is determined on the basis of the known refractive index of the immersion medium.

10. Microscope, with a cover glass (14) covering a sample, wherein the microscope (10) comprises a device for digitally correcting an optical image of the sample, and the device is configured to determine the refractive index of an optical medium (20) bordering the cover glass (14), a tilt of the cover glass (14) and / or the thickness of the cover glass (14), characterised in that the device is configured, on the basis of the refractive index of the optical medium, a tilt of the cover glass (14) and / or the thickness of the cover glass (14), to determine the imaging error to be corrected in the form of a pupil function, and the device is configured to digitally correct image data, acquired by the imaging of the sample, on the basis of the pupil function.

11. Microscope according to claim 10, characterised in that the microscope (10) is a confocal microscope.