Method for optically high-resolution raster scanning of an object and use of the method

Abstract

A method which scans an object to be imaged with one or more illumination foci in a scanning process and, using means for scanning and rescanning, automatically generates a rasterized image on an array detector or in the eye from a total emitted signal as a summation of the shifted detection events, with particular consideration that a signal emitted at a given time on the array detector or in the eye generates an instantaneous image which, depending on the proximity of a focus, experiences a different imaging ratio than the rasterized image described by a nominal position or nominal positions of one or more illumination foci.

Description

[0001] What technical problem is your invention intended to solve? Imaging in a confocal system is inefficient and does not allocate photons to the locations in the image where an emitter (or scattering center) is most likely to be found.

[0002] How has the problem been solved so far? What is the current state of the art?

[0003] By deconvolution of the images in the computer something can be gained, but this has

[0004] Disadvantages for the signal-to-noise ratio (SNR). This problem has been solved in principle by capturing individual images at each scan position and calculating the corresponding results [Sheppard et al., Enderlein et al., Shroff et al.]. However, this has the disadvantage that a spatially resolved image in the pinhole plane must be acquired for each scan position, which slows the scan and leads to additional readout noise. What are the disadvantages of the known solutions?

[0005] Image acquisition is slowed down because a spatially resolved image must be captured for each scan position. Additionally, more photons are required to achieve equivalent image quality, as this approach to the problem degrades the SNR. What is the purpose of your invention?

[0006] The task is to create an efficient, high-resolution scanning system that sends the off-axis photons directly to more suitable locations in the image. How does your invention solve this problem?

[0007] By changing the off-axis image scale while maintaining the scale of the scan image. What is fundamentally new about your invention? What is the core of the invention?

[0008] The computer-based calculation of the known method is realized in the new approach directly through clever optics. What essential or additional benefits are achieved by your invention?

[0009] An increase in resolution is achieved with an improved signal-to-noise ratio (SNR) and higher acquisition speed. Which keywords (German and English) can you use to describe your invention?

[0010] High-resolution confocal system Photon reassignment Decoupled scanning Demagnified Rescanning Emission magnification changer

[0011] Explain your invention using a drawing. See below. background

[0012] A number of modern microscopic techniques allow for increased resolution by illuminating with focused light and scanning the focus across the sample (“raster scan method”). There are currently two ways to record the data: an integrating detector measures the signal intensity in rapid succession and assigns it to the nominal scan position. This principle is used in conventional confocal microscopes, where the backscattered light or the fluorescence emission is directed through a pinhole (conjugated to the focal plane in the specimen) to suppress light outside the focal plane. The pinhole can be adjustable in size. To scan the sample, either the sample can be moved or the illumination beam can be appropriately controlled. In the latter case, the light emitted by the sample (e.g., fluorescence or scattered light) is subjected to the reverse process (“descanning”), so that the nominal focus position is retained after descanning. appears stationary again. A pinhole can then simply be placed here. The same optical elements that are used for scanning the sample are typically used for descanning. These include, for example, resonant or non-resonant galvanometric mirrors, rotating polygon mirrors, or acousto-optical deflectors.

[0013] Another data acquisition method involves imaging the emitted light onto a spatially resolved detector ("array detector," e.g., a CCD or CMOS camera). This has proven particularly useful for 2-photon microscopy, as optical sectioning is achieved directly by the excitation light [example: LaVision microscope]. Arrangements are also known in which the light is first descanned and then, often using the same optical elements (or their backs), scanned across the array detector. One such method (as a multi-point version) is known from the "VT-Infinity" device from Visitech International (VTI).

[0014] Well-known publications [Sheppard 1988, Optik 80, 53-54, Müller&Enderlein PRL 104, 198101 (2010), York et al., Nature Methods 8, 327-333, 2011] describe methods that record the light distribution in the raster positions using spatially resolved detectors in the image plane (conjugated to the focal plane in the object). From this data, images can then be calculated that use the emitted light more efficiently than the methods described above, thus leading to an increase in resolution. In particular, it has been shown that it is useful to assign a photon that is not detected exactly on the nominal scan axis neither to the nominal scan position nor to the nominal detected image position, but rather Ideally, a value in between. In previous publications, this value is specified as the mean position between the nominal excitation position and the nominal sample position of the detection point (pixel).

[0015] A number of modern microscopy techniques (e.g., STED) allow light emission to be restricted to a range well below the diffraction limit ("high-resolution scanning"). This is achieved, for example, by deliberately preventing spontaneous emission from the peripheral regions of the excitation light by forcing stimulated emission using a ring-shaped light distribution (a so-called "doughnut mode"). To date, no technique is known that is capable of mapping the resolution of a high-resolution scanning technique onto an array detector or onto the retina of the eye. The new technique would enable direct imaging of the high-resolution images.

[0016] Furthermore, methods are known that scan the sample with multiple beams simultaneously (e.g., Nipkow Disc, Yokagawa Disc, LaVision Multi-beam scanning, VTInfinity) and image the emitted light onto an array detector or the retina. This imaging is usually done either without using the scanning device (the rotating wheel) or by imaging through it (e.g., the rotating hole pattern in Yokagawa). The VT-Infinity system implements a scanning-descaming-rescanning process.

[0017] In DE10024685A1, Czarnetzki and Scherübl describe a laser scanning microscope that can automatically detect and correct the focus position. This task is achieved by introducing three pinhole-based illumination and detection beams, each with a different focus. However, the three detection channels are not imaged using a descan and rescan process, and no image is automatically generated on an array detector, thus no rasterized image is created. Furthermore, there is no change in the imaging ratio.

[0018] Utility model DE202009011701U1 describes a confocal scanning microscope with a built-in zoom device and transmitted-light signal detection. The transmitted-light signal is superimposed on the fluorescence signal. However, there is no change in the magnification ratio, and the image is not generated by the superimposition of a rescanned focus moving on the detector. A rasterized image is not automatically generated on an array detector such as a camera.

[0019] Patent application US 2007 / 0146869 A1 describes a device for rapid scanning of the sample, which uses rotating mirrors to rescan the beam on the retina of the eye or an array detector. [0034, 0045, 0092, 0108] explicitly states that it is important to achieve an image independent of the mirror position. Increased resolution cannot be achieved this way. Problem

[0020] What is needed is a raster scanning method that achieves flexible imaging with high resolution on an array detector (e.g. CCD or CMOS camera) or directly in the human eye and efficiently uses the available light. Solution to the problem

[0021] This object is achieved by a method according to claim 1. Further developments and uses are specified in the subclaims.

[0022] The invention consists in implementing methods analogous to photon reassignment described in the literature [Sheppard 1988, Optik 80, 53-54, Müller&Enderlein PRL 104, 198101 (2010), York et al., Nature Methods 8, 327-333, 2011] in a purely optical manner, so that the summation of the shifted detection events occurs automatically on the detector or in the human eye. This is made possible by a different imaging ratio of the scan position on the detector compared to the imaging ratio of the current image around the respective scan position ( Fig. 1). This can be implemented both for single-beam scanning processes and, e.g., with the help of microlenses for multi-beam scanning processes. One solution according to the invention consists in performing a descanning followed by a rescanning, whereby the nominal position of the rescanning occurs at a different speed (relative to the respective corresponding object position) than that of the scanning. This avoids time-resolved detection of the emission light for each respective scan position, thus allowing better use of the photons than would be achieved by simple imaging onto a less efficient point detector (e.g., a secondary electron multiplier, PMT).

[0023] This can be technically realized according to the invention by using several synchronized scanners that move at different speeds, or in particular by using optics that specifically change the beam diameter after descanning but before rescanning, so that the size of the current image of the sample on the array detector (or the retina) can be influenced independently of the image size described by the scanning. With conventional scanning methods coupled with imaging without scanning or with a conventional rescanning system with a relative scale of one, an emission that occurs at a distance d from the scan axis will appear in the image according to the total magnification M of the system. This point is therefore simply imaged at position Md from the nominal scan position in the image (after convolution with the emission PSF).In contrast, according to the invention, the scanning process is decoupled from the imaging process, and the emission at a distance d from the scan axis can be imaged with a different magnification scale M2 at M2 d from the nominal scan position. Since the probability of a detected emission now depends on the excitation probability and the detection probability, M2 ≠ M can exhibit improved imaging properties of the overall system. The photons treated in this way can better reflect the true position of the emitters than with M2 = M.

[0024] By adjusting the resulting deviating emission magnification, optimal resolution and utilization of the emitted light can be achieved (if the possible emission area in the sample is approximately in the order of magnitude of the specified diffraction limit that applies to the imaging from the sample to the detector). The emitted photons are thus preferentially brought to the positions of the spatially resolving detector that are most likely for their emission position in the sample, taking into account the illumination and detection position [see Sheppard 1988].

[0025] By significantly reducing the size between descanning and rescanning (corresponding to an increase in the beam diameter before rescanning) or, for example, by selecting a suitably fast descan in the opposite direction compared to the scan, it is possible to use array detectors for high-resolution raster scanning methods without a serious loss of resolution, where the size of the possible emission distribution is significantly smaller than the diffraction limit of the detection. Examples include stimulated emission microscopy (STED), ground state depletion microscopy (GSD), and similar methods that narrow the range of possible emission by exploiting photoswitchable dark states. The method is also useful for optical Systems where the illumination aperture is larger than the detection aperture or where the illumination and emission wavelengths differ greatly (e.g. X-ray fluorescence). In this variant of the method, it is sufficient to bring about a sufficiently large change in the relative magnification so that the measured Photon position can essentially be interpreted as the position of the nominal instantaneous scan axis and only weakly depends on the current position of the emitted light. In general, for each definition of the typical volume that can contribute to the emission of signal light determined by the microscope method (i.e., smaller than the diffraction limit for STED) and a size of the diffraction disk of a point object in the image determined by the imaging optics, there is an optimal choice for the magnification M2 compared to the magnification M determined by the scan.

[0026] With sufficiently fast scans, the image can be observed directly on the retina after rescanning, allowing high-resolution techniques to be used for direct observation. Furthermore, array detectors (e.g., CCD, iCCD, CMOS, sCMOS) are often superior to single-pixel detectors (e.g., PMT, APD) in terms of quantum efficiency, dark noise, spectral sensitivity range, readout speed, and especially the maximum permissible photon rate.

[0027] Compared to direct imaging (without descanning) or descanning with other methods (e.g., opposite motion), descanning and rescanning have the advantage that a (variable) pinhole can be inserted to suppress light from outside the focal plane. The pinhole should not be too small to avoid compromising the light efficiency of the process.

[0028] Methods according to the invention can implement the principle in one dimension (e.g., X), in two dimensions (e.g., X and Y), or in three dimensions (X, Y, and Z). Accordingly, combinations of multiple one-, two-, or three-dimensional raster scanning devices can be used. Rescanning can also be performed partially using the same optical scanning elements (e.g., X-galvo mirrors) and partially using an additional scanning element (e.g., Y-galvo) only for rescanning, which is, for example, synchronized with the object movement along Y.

[0029] Scanning the sample with multiple foci simultaneously offers the advantage of increased throughput, meaning more signal can be detected per unit of time. To achieve an advantage within the meaning of the invention, the magnification of the image of each individual focus must be changed in relation to the focal positions for the emitted signal. This can be achieved, for example, by using two multifocus devices with different focal lengths and / or (with different magnification ratios between illumination and detection) different Multifocus distance, each for illumination and detection.

[0030] The use of a 2-photon laser can be helpful for achieving inherent optical cutting. This is particularly useful when no confocal pinhole is used in the imaging beam path. When illuminating microlens arrays (such as Fig. ), it is advisable to choose the pinhole spacing so that only the highest diffraction orders pass through the lens. This has the advantage of increased contrast of the diffraction pattern in the sample at the highest spatial frequency of the illumination. With larger pinhole spacing, the contrast can also be increased by a clever choice of the Hole spacing and the use of, for example, hexagonal arrangement can be improved.

[0031] In Fig. The principle is illustrated according to the main claim using a single focus. Examples of times t1 (top row) and t2 (bottom row) during the scan are shown here. A scan point (blue) is at position d1 in the sample (left column) at time t1. For clarity, only the x-component d1,x is shown. The scan point moves so that at time t2 it is at position d2 in the sample. The scan point is imaged into the image plane at the imaging ratio M1, i.e., at time t1 it is at position M1d1 and at time t2 at M1d2. An emitter (green) in the sample (or any adjacent selected point in the sample) with a distance D1 to the next neighboring scan position at time t1 is now optically imaged with a different imaging ratio M2 so that mapped so that its distance to the neighboring scan point is not M1D1 but M2D1. The same applies at time t2. The origins of the coordinate systems are chosen so that they are mapped to each other at time 0.

[0032] In Fig. A raster scanning method is shown, which represents a prime example within the meaning of the invention. L1 - L6: achromatic lenses with different focal lengths (L1 - 50 mm; L2 - 75 mm; L3 - 60 mm; TL (tubelens) - 400 mm; L4 - 200 mm; L5 - 400 mm; L6 - 200 mm); optional pinhole 1 is used to clean the laser beam. Optional pinhole 2 with adjustable size acts as a confocal aperture. M1 - M5: mirrors. M4 and M5 are preferably D-shaped in order to be able to separate beams with only a small angular difference. The dichroic beam splitter in the fluorescence arrangement shown here reflects the laser excitation light and allows the fluorescent light to pass through. The laser filter suppresses unwanted laser light with high efficiency; the scanning unit can control the laser beam two-dimensionally.It can be realized by one mirror with 2D angle scanning capability, by two closely adjacent scanning mirrors, by three scanning mirrors that mutually correct the pupil position or by two scanning mirrors with. intermediate image.

[0033] In Fig. An implementation with three scanning mirrors is shown, one of which is located in the conjugated pupil plane, while the other two ensure, through suitable control, that the pupil position remains constant.

[0034] In Fig. A realization of the method is shown, which uses a concave mirror to achieve the change in the image scale.

[0035] In Fig. According to the invention, a method is presented in which the scanning and descanning device consists of two individually controllable units. The scanners can run in different directions and at different speeds. The first mirror scans the sample and descans the light from the sample. The light separated by the dichroic mirror is then scanned by scanning unit 2 on the camera (“rescan”). By appropriately selecting the scanning speeds, the desired different imaging ratios M1 and M2 can be achieved. The advantage of this arrangement is that this can be done variably, so that, for example, the influence of Stoks' can optimize shift.

[0036] In Fig. is a system similar to Fig. Shown here, where the pinhole and two lenses are omitted. This may increase optical efficiency.

[0037] In Fig. A setup is shown in which the emission light is decoupled before reaching the "descan" stage, and the scanning position on the camera is then determined using a second scanning unit. A confocal pinhole is not possible here, but the desired resolution effect can still be achieved.

[0038] Fig. An advantageous embodiment that allows the use of an existing scanning system. In the case of fluorescence, the emitted light is subjected to additional reflection before passing through the scanner on its return path. This scans it in the opposite direction to the conventional arrangement, resulting in a relative scanning speed in the image (here directly at the otherwise pinhole plane) of 2x.

[0039] The emitted light takes a different optical path than the incident light before descanning. This results in image reflection, which results in scanning at twice the normal speed of the image instead of the usual descanning. This process has the advantage that no modifications to the scanning system are required. Thus, the desired resolution effect can be achieved by suitable integration into existing systems or by minor modifications to the systems themselves.

[0040] Fig. and Fig. show detailed views of the insert and its operating principle

[0041] Fig.shows, by way of example, how a parallel version similar to a Nipkow disk or Yokagawa system with improved resolution can be realized using microlens arrays. Here, three microlens arrays (e.g. in disk form as in the Yokagawa system) are mechanically connected to each other. The microlens arrays have different focal lengths in order to achieve the desired effect of variable local magnification. It is important that the image of the pinholes is still upright at different intermediate magnifications. This can be achieved by suitable selection of the distances a1, a2 and a3 between the microlens arrays and suitable selection of the focal lengths f1, f2, as described, for example, in http: / / www.livephysics.com / problems-and-answers / optics / lens-system-imagedistance-magnification / The intermediate image in the image plane (from the pinhole array) is then

[0042] The beam splitter (color splitter for fluorescence) is reflected and imaged onto a camera or into the observing eye using conventional optics (not shown here). A third coupled microlens array is provided for illumination, projecting into the intermediate image plane and creating diffraction-limited foci there. With this arrangement, the local magnification can be advantageously varied by shifting the microlens arrays along the axis.

[0043] Furthermore (as known from the Yokagawa system), the mechanically connected discs can be scanned very quickly, so that high frame rates can be achieved. Definitions The emitted signal can be any form of wave that allows imaging. For example, visible light, IR light, UV light, X-rays, or ultrasound. It is irrelevant whether the emission has the same wavelength as the illumination focus, or whether it is coherent or incoherent with it. It can be, for example, fluorescence or scattered light. Array detectors include charge-coupled devices (CCDs), electron multiplying CCDs (emCCDs), intensified CCDs or intensified CMOS, electron bombardment cameras, CMOS, scientific CMOS (sCMOS), spatially resolved single-photon counting (Europhoton, Leibniz Institute for Neurobiology Magdeburg), and film detection. No actual array is required; only spatial resolution is required. Illumination foci are any form of influence on the object that can affect the emitted signal and can be spatially localized. This can be, for example, an illumination light to generate fluorescence in the sample, or a magnetic tip that scans the sample. Image The image can be two-dimensional or one-dimensional.

Claims

[1] Method which scans an object to be imaged with one or more illumination foci in a scanning process and, using means for descanning and rescanning, automatically generates a rasterized image on an array detector or in the eye as a summation of the shifted detection events, with particular consideration of the fact that a signal emitted at a time on the array detector or in the eye generates a momentary image which, from the proximity of a focus, experiences a different imaging ratio than the rasterized image, described by a nominal position or nominal positions of one or more illumination foci. [2] Method according to claim 1, wherein means are introduced between descanning and rescanning which influence the emitted signal in such a way that this leads to the desired change in the imaging ratio. [3] Method according to claim 1, wherein the change in the imaging ratio is achieved by multiple use of the scanning device. [4] Method according to one of claims 1-2, wherein the change in the imaging ratio is realized by a reducing optical imaging between the descanning and the rescanning. [5] Method according to claim 1, wherein spatially separated descan and rescan means are used and the imaging ratio can thus be varied electronically. [6] Method according to claim 1, wherein the scanning process is realized by a microlens array and / or pinhole array and the change in the imaging ratio is realized by a microlens array with a different focal length with or without a pinhole array. [7] Method according to one of claims 1-2, wherein the scanning process is realized by a microlens array and / or pinhole array and the change in the imaging ratio is realized by a suitable intermediate magnification and a microlens array with the same focal length but different lens pitch, with or without pinhole array. [8] Method according to claim 1, wherein the scanning process is realized by a microlens array and / or pinhole array and a microlens array is used behind a pinhole array to realize a magnification change. [9] Use of a method according to any one of claims 1 to 8 for fluorescence microscopy. [10] Use according to claim 9, wherein a light source which enables 2-photon absorption in the sample is used for illumination during the scanning process.

Images