METHOD AND DEVICE FOR DETECTING DISPATCHES OF A SAMPLE VERSUS A LENS

Description

TECHNICAL FIELD OF THE INVENTION

[0001] The invention relates to a method for detecting displacements of a sample relative to an objective. In particular, the invention relates to such a method in which light from at least one reference object connected to the sample is imaged at successive times with the objective into reference object images in an image plane, in which the reference images in the image plane are recorded with a camera, and in which the recorded reference object images are compared with one another.

[0002] Furthermore, the invention relates to a device for carrying out such a method with a lens and with a camera which is arranged in an image plane of an imaging optics comprising the lens.

[0003] Particularly in high-resolution microscopy, where spatial resolutions beyond the diffraction limit are achieved, displacements of a sample under investigation relative to the objective of the respective microscope have a significant effect, even if these displacements do not exceed the nanometer range. A displacement of the sample relative to the objective between two points in time shifts the relative positions of objects in the sample determined at these two points in time. If occurring displacements of the sample relative to the objective are not recorded, they cannot be compensated for, and the effective spatial resolution in the microscopic imaging of a structure of interest is determined by the magnitude of these displacements.

[0004] Since displacements of a sample relative to the objective of a microscope can never be completely avoided, especially over longer measurement times, and limiting occurring displacements to small values ​​also requires considerable technological effort, there is an interest in recording such displacements in order to be able to take them into account, i.e., in particular, to compensate for them. However, recording the displacements should, if possible, not impair the actual use of the objective, but ideally be able to take place parallel to this actual use. STATE OF THE ART

[0005] DE 101 00 246 A1 discloses a microscope, in particular a confocal or double-confocal scanning microscope, and a method for operating such a microscope. A reference object is provided on a slide unit associated with an object to be examined. For calibration, adjustment, and / or setting of the microscope, the reference object is detected by light microscopy to detect and compensate for drift-related changes. The reference object can be a textured or structured surface of a glass plate that is part of the slide unit. The reference object can also comprise microscopic objects attached to the slide unit, such as beads or nanocrystals, which can be stochastically distributed.The reference object is detected by capturing an image or scanning, and the detected image data of the reference object is used to determine its position and / or orientation relative to the detected object area. By comparing detected image data of the reference object with previously detected image data, the drift of the object / slide unit is determined. This drift is then compensated for by moving the slide unit or using digital image processing methods. With the known microscope and the known method, calibration, adjustment, and / or setting interrupts the actual use of the microscope, i.e., it cannot be performed in parallel with the actual use of the microscope.

[0006] EP 1 548 485 B1 discloses a method for calculating the drift of a microscope and a corresponding microscope, which have the features of the preambles of independent patent claims 1 and 12. When carrying out the method, an immobile object is selected, and a first image is taken of a region of interest on the immobile object at a first point in time and a second image is taken at a subsequent second point in time. The first and second images are each divided into blocks using a grid and then compared block by block for displacement. A drift is determined from the comparison and stored to correct the drift of the microscope. This procedure cannot be carried out during normal operation of the microscope either.

[0007] WO 2013 / 016356 A1 discloses a drift correction in a microscope using reference objects arranged on a sample carrier. Both the reference objects and an object of interest are illuminated with one and the same light source, and photons emitted by the reference objects and the object of interest are registered with one and the same image sensor. The positions of the reference objects are extracted from images recorded with the image sensor and used to correct the microscope's drift. To determine the positions of the reference objects, a center of gravity of a corresponding light intensity distribution on the image sensor can be determined. In practice, it proves difficult to find reference objects that can be excited to emit light in the same way as objects of interest and imaged in the same way with the same image sensors in order to carry out this known method.

[0008] Autofocus devices are commercially available for light microscopes. These devices couple an auxiliary laser beam together with illumination light, albeit laterally shifted towards the edge of the pupil, into the objective of the respective light microscope. At an interface between an immersion medium and a coverslip or between a coverslip and an embedding medium, a back reflection of this auxiliary laser beam occurs, which is imaged by the objective onto a position-sensitive detector. The position of the back reflection on the detector depends on the distance of the interface to the objective, so that the position signal from the detector can be used as a measurement signal for a control loop to compensate for any axial drift of a sample stage by controlling a z-drive on the sample stage or an additional actuator. By design, these autofocus devices are sensitive to a change in the refractive index between the immersion medium and the coverslip or between a coverslip and an embedding medium.between the coverslip and the embedding medium, so that with certain combinations of coverslip glass type, immersion medium, and embedding medium, such as those found when using glycerin objectives, the autofocus systems are impaired in their function or fail completely. Furthermore, the known autofocus systems only allow compensation for axial drift in the z-direction, i.e., along the optical axis of the objective, but not in the orthogonal x- and y-directions.

[0009] From GARCÍA-MARTÍNEZ PASCUALA ET AL: "On-axis programmable microscope using liquid crystal spatial light modulator", Proc. of SPIE Vol. 10333, 103330Q, June 26, 2017, a microscope with an objective lens and a camera arranged in an image plane of an imaging optics system encompassing the objective lens is known. A spatial light modulator (SLM) is arranged in a Fourier plane of the imaging optics system to image objects onto the camera using a phase mask. The phase mask can be used for low-pass or high-pass filtering of the spatial frequencies of the images captured by the camera.

[0010] US 2014 / 0 022 373 A1 discloses a method for detecting displacements of a sample relative to an objective having the features of the preamble of independent patent claim 1 and a device for carrying out the method having the features of the preamble of independent patent claim 13, in which gold nanoparticles, quantum dots, or fluorescent beads are introduced into a sample as fixed point markers for drift correction. In a drift correction system for a microscope, which is the subject of US 2014 / 0 022 373 A1, additional infrared light is focused with an objective of the microscope for reflection on a coverslip of a sample, and infrared light reflected from the coverslip is detected with a camera.

[0011] A microscope with three-dimensional drift control is known from US 2012 / 0 194 903 A1. Near-infrared light is directed onto the surface of a coverslip of a sample. Regularly reflected light and scattered light from the near-infrared light are generated, and the regularly reflected light and scattered light pass through the microscope objective to a semi-transparent mirror. The regularly reflected light is used to monitor the focus of the microscope. The scattered light, from which the regularly reflected light is filtered out, is used to detect lateral drift. The scattered light originates from surface irregularities, such as scratches on the coverslip, but a mark or a scattering substance can also be pre-applied to the surface of the coverslip.

[0012] From TOMONOBU M. WATANABE ET AL: "Distinct Modulated Pupil Function System for Real-Time Imaging of Living Cells," PLoS One, Vol. 7, No. 9, September 4, 2012, pages 1-13, it is known to vary the pupil function in the Fourier plane of a microscope's imaging optics using a micromirror array. For example, low spatial frequencies are filtered out using a point-shaped aperture in the center of the pupil. This reduces the diameter of the image of a fluorescent bead.

[0013] From WO 2009 / 085218 A1, the use of a cylindrical lens in an imaging optics of a microscope is known in order to derive the z-position of an imaged object from the resulting z-position-dependent imaging error.

[0014] DE 10 2008 028 490 A1 describes microscopy using total internal reflection fluorescence (TIRF), a special form of fluorescence microscopy. Fluorophores in a sample are excited to fluorescence using an evanescent illumination field exclusively in a thin layer behind the interface between a coverslip and the sample. The evanescent illumination field in the sample is generated by directing excitation radiation within the coverslip onto the interface between the coverslip and the sample at an angle that leads to total internal reflection. By exciting only the thin layer to fluorescence, a particularly high axial resolution can be achieved.A known type of TIRF illumination is achieved using a prism on the side facing away from the objective. This has the disadvantage that the sample to be examined must be prepared on the prism, since the evanescent illumination field is excited at the interface between the prism and the sample. This type of preparation is complex, and in contrast, samples are usually prepared on a thin coverslip. Alternatively, TIRF illumination is applied through the microscope objective, which requires a high numerical aperture and thus a complex objective due to the required large angles of incidence. DE 10 2008 028 490 A1 describes a TIRF illumination system that is intended to enable high axial resolution with minimal effort. For this purpose, a TIRF illumination device is designed as a module and has an optical fiber and collimation optics.The collimation optics are mounted in front of a light exit aperture of the optical fiber in such a way that they collimate the divergent light emerging from the optical fiber into a light beam. The collimated light beam intersects the optical axis of the microscope objective at an angle greater than or equal to the angle of total internal reflection. This allows the excitation light to be directed toward a sample outside the detection beam path. This decouples the numerical aperture of the excitation from the numerical aperture of the detection, allowing a standard microscope objective to be used for detection.

[0015] From SHIH-HAO HUANG ET AL: "Development of a monolithic total internal reflection-based biochip utilizing a microprism array for fluorescence sensing; Development of a monolithic total internal reflection-based biochip utilizing a microprism array", JOURNAL OF MICROMECHANICS AND MICROENGINEERING, INSTITUTE OF PHYSICS PUBLISHING, BRISTOL, UK, Vol. 15, No. 12, December 1, 2005 (2005-12-01), pages 2235–2242, a monolithic total internal reflection-based biochip for fluorescence measurements is known. Excitation light is coupled into a quartz substrate beneath an analyte via a microprism and guided by total reflection within the quartz substrate. The excitation light interacts with the analyte via the resulting evanescent fields. Here the prism is arranged on the side of the quartz substrate facing the analyte.

[0016] US 2009 / 0 052 021 A1 discloses an illumination arrangement for a TIRF microscope in which excitation light is coupled into a sample carrier via a prism and guided therein by total internal reflection. The resulting evanescent fields at individual points on the sample carrier interact with a sample on the sample carrier. Here, the prism is arranged on the side of the sample carrier facing away from the sample. OBJECT OF THE INVENTION

[0017] The invention is based on the object of demonstrating a method and a device for detecting displacements of a sample relative to an objective, which can be carried out or used while using the objective to image or otherwise measure an object of interest, in order to be able to accurately detect and compensate for the displacements of the sample relative to the objective at each time at which the object of interest is imaged or measured by the objective, without extending the measuring times. SOLUTION

[0018] The object of the invention is achieved by a method for detecting displacements of a sample relative to an objective having the features of independent patent claim 1 and by a device for carrying out the method having the features of independent patent claim 13. Preferred embodiments of the method and device according to the invention are defined in the dependent patent claims. DESCRIPTION OF THE INVENTION

[0019] A method for detecting displacements of a sample relative to an objective, wherein light from at least one reference object connected to the sample is imaged at successive times with the objective into reference object images in an image plane, wherein the reference object images in the image plane are recorded with a camera and wherein the recorded reference object images are compared with one another, is characterized according to the invention in that illuminating light is coupled into a waveguide aligned transversely to the optical axis of the objective and that the at least one reference object is illuminated with the coupled-in illuminating light, which causes the light to be emitted by the at least one reference object, wherein a glass plate or another plate of the sample made of an optically transparent material is used as the waveguide and wherein the at least one reference object is located in the plate.

[0020] This method is disclosed here as an invention, even independent of the feature that low spatial frequencies in front of the camera are masked out of the reference object images. The reference object is provided in addition to an object of interest on the sample that is imaged or measured with the lens.

[0021] A glass plate or another plate of the sample made of an optically transparent material is used as the waveguide. This illumination is particularly effective when the at least one reference object is located within the plate. Specifically, the plate can be a microscope slide or a coverslip of the sample. If the microscope slide or coverslip has defects that act as scattering centers, these defects can serve as reference objects. If the illumination light is coherent light, a speckle pattern of the illumination light scattered by the at least one reference object or another part of the sample can be captured with the camera and observed for changes.

[0022] In a plane in front of the camera that is Fourier conjugate to the image plane, low spatial frequencies can be masked out from the reference object images. This aspect of the invention is based on the finding that information about displacements between two images of the at least one reference object is contained exclusively in those regions of the reference object images that have intensity gradients. In contrast, regions of the reference object images with a homogeneous intensity distribution do not contain any information about a displacement of the images of the at least one reference object relative to one another. The information about the displacement of the images is therefore contained in particular in the sharpest possible edges of bright reference objects against a dark background, or vice versa, which lead to large gradients in light intensity in the images.

[0023] In this aspect of the method according to the invention, low spatial frequencies are therefore masked out from the reference object images in a plane in front of the camera that is Fourier conjugate to the image plane, or, in other words, low spatial frequencies in the reference object images are attenuated compared to high spatial frequencies. In this way, edge detection filtering is performed, which suppresses uniform regions compared to regions of the reference object images with strong brightness gradients.

[0024] Because the masked low spatial frequencies, which would correspond to homogeneous, bright image areas of the reference object images, do not reach the camera in the first place, they cannot impair the recording of the higher spatial frequencies with the higher content of information regarding the position of the images through increased signal noise, for example, due to stray light. Furthermore, the dynamic range of the camera can be fully utilized in those areas of the reference object images where the content of information regarding the position of the images of at least one reference object is highest in the reference object images, without significantly and extensively exceeding the camera's saturation intensity.

[0025] Edge detection filtering can be achieved by filtering out portions of the light originating from the central region of the objective's pupil in the plane Fourier-conjugate to the image plane. This suppresses the low spatial frequencies close to the pupil's center. Here and in the following, the objective's pupil refers to the objective's eyepiece-side pupil, also known as the exit pupil. The eyepiece-side pupil is the image of the objective's aperture diaphragm, which appears to the observer from an on-axis image point. In microscope objectives, the eyepiece-side pupil practically coincides with the objective's rear focal plane. This applies exactly to objectives for an infinity beam path, and at least approximately to those for a finite beam path. Therefore, the eyepiece-side pupil is often used, and here as well, as a synonym for the objective's rear focal plane.In addition, the eyepiece-side pupil lies close to a Fourier-conjugate plane to the object plane of the lens, thus resulting in an amplitude distribution of the electromagnetic field in the image plane that corresponds very well to a Fourier transform of the amplitude distribution of the electromagnetic field in the pupil. The portions of the light from the at least one reference object originating from the central region of the lens's pupil are therefore among the low spatial frequencies to be filtered out.

[0026] In order to filter out the portions of light originating from the central area of ​​the lens pupil in the plane Fourier conjugate to the image plane, the lens pupil can be imaged into this Fourier plane.

[0027] In the aspect of the method according to the invention explained here, preferably only high spatial frequencies, and thus portions of the light originating from a peripheral region of the lens's pupil, are transmitted to the camera. It is even more preferred if these high spatial frequencies or portions from the peripheral region of the pupil are transmitted to the camera as completely as possible. The reference object images then contain maximum information about the position of the images of at least one reference object.

[0028] Specifically, the low spatial frequencies, and thus the portions of light originating from the central region of the lens pupil, can be masked out using a circular disk covering the central region. Instead of the circular disk, a mirror covering the central region can also be used for masking. This mirror can then redirect the low spatial frequencies to another detector to evaluate them, for example, with regard to the brightness of the images of at least one reference object. If a spatial light modulator (SLM) is used in front of the camera to mask the low spatial frequencies, the masked region of the Fourier conjugate plane can be quickly varied by controlling the SLM in order to optimize it. In addition, the SLM can also be controlled to image the reference objects onto the camera in the conventional manner or in images complementary to the reference object images.

[0029] A circular disk covering the central region of the pupil in the Fourier-conjugate plane can be realized specifically by a glass substrate with a circular, centrally metal-deposited layer. In the simplest case, edge detection filtering in the method according to the invention is carried out using a thin wire, for example made of metal, which extends through the center of the image of the pupil of the lens in the plane Fourier-conjugate to the image plane. This filter filters the spatial frequencies in a direction-dependent manner, but with a direction dependence that decreases with increasing spatial frequency and in the same way for all images of the at least one reference object. Since the method according to the invention only considers changes in the positions of the images, the direction dependence of the filtering is therefore irrelevant.

[0030] Regardless of how the low spatial frequencies are filtered out in front of the camera in the method according to the invention, these components of the light do not reach the camera. The edge detection filtering according to the invention is therefore not a step in the image processing of an image captured with the camera. Rather, the invention creates reference object images directly on the camera, for which the edge detection filtering according to the invention has already been performed.

[0031] In the method according to the invention, it proves to be advantageous if the at least one reference object has a steep edge, above which the intensity of the light emanating from the reference object drops by at least 90% and whose width parallel to the image plane is smaller than the wavelength of the light. If monochromatic light, i.e. light of one wavelength from the at least one reference object is then imaged into the reference object images in the image plane, a diffraction pattern appears at the edge in the edge-detection-filtered reference object images. Specifically, the light intensity distribution shows a first-order intensity minimum following a zero-order intensity maximum, which is greatly reduced in intensity due to the suppressed low spatial frequencies, which is followed by a first-order intensity maximum.Both intensity increases adjacent to the first-order intensity minimum to the zero-order and first-order intensity maxima exhibit, with respect to the total amount of light, a high information density regarding the respective position of the edge of the at least one reference object. Depending on the upper limit up to which the low spatial frequencies are suppressed and the aperture of the image, a second-order intensity minimum and a second-order intensity maximum can also adjoin the first-order intensity maximum, whereby both intensity increases adjacent to the second-order intensity minimum to the first-order and second-order intensity maxima exhibit a high information density regarding the respective position of the edge of the at least one reference object.

[0032] If the at least one reference object has a dimension equal to or less than the wavelength of the illumination light, it consists practically only of edges, thus ensuring maximum information density regarding the respective position of the reference object in relation to the total amount of light. Specifically, the at least one reference object can be a point-like marking or a bead with a diameter below the wavelength of the light, whereby the edge-detection-filtered monochromatic image of the reference object shows two or more concentric intensity rings in the reference object images. So-called gold nanorods, which are commercially available and can have diameters in the range of 50 nm to 150 nm, for example 75 nm or 100 nm, and lengths in the range of 300 nm to 2,000 nm, for example 500 nm or 1,000 nm, are also very suitable as reference objects.

[0033] When, in the method according to the invention, monochromatic light from the at least one reference object is imaged into the reference object images in the image plane, the light from the at least one reference object can be separated from the measurement light from the sample due to its wavelength, which is imaged by the lens onto a detector other than the camera. In other words, the method according to the invention can be carried out in parallel with the actual use of the lens, with the measurement light being imaged onto the other detector.

[0034] It should be noted here that even if no explicitly coherent monochromatic illumination light is used to illuminate the at least one reference object, a diffraction pattern is visible in the reference object images if the at least one reference object has a sufficiently steep edge and in particular a diameter below the wavelength of the illumination light.

[0035] In any case, in the method according to the invention, it is advantageous if the sample is illuminated with monochromatic illumination light of one wavelength in order to generate the light from the at least one reference object with precisely this wavelength. The term "monochromatic" or "one wavelength" regarding the illumination light is always to be understood here as meaning that the bandwidth of the wavelength of the illumination light is so narrow that the described diffraction pattern appears in a form usable for detecting displacements of the sample relative to the objective. The use of an inexpensive conventional laser, including a laser diode, is sufficient for this purpose.

[0036] Preferably, the at least one reference object is illuminated with illuminating light whose wavelength differs from the wavelengths of the measurement light from the sample and the excitation light generated by the measurement light from the sample by at least 50 nm or preferably at least 100 nm. These wavelength differences between the illuminating light and the light from the at least one reference object, both from the measurement light from the sample and from the excitation light generated by the measurement light, enable not only spectral separation of the light from the at least one reference object from the measurement light, but also interference-free coexistence of measuring the sample with the excitation light and generating the light from the at least one reference object with the illuminating light.

[0037] In the method according to the invention, additional light from at least one optical interface located between the sample and the object or from at least one further reference object arranged on the lens can be recorded with the camera in the reference object images. This additional light contains further information about the relative position of the optical interface with respect to the lens or of the lens with respect to the camera and thus about any additional drifts that need to be compensated. Edge detection filtering can also be performed with respect to the additional light in the method according to the invention.

[0038] In the method according to the invention, the camera is preferably controlled such that the maximum image intensities in the reference object images reach 90% to 1,000% of a saturation intensity of the camera. It is preferred if the maximum light intensities reach 100 to 150% of this saturation intensity. In other words, the camera is controlled such that its dynamic range is fully utilized, while accepting that the maximum light intensities lie above the saturation intensity of the camera and are therefore no longer resolved. This is based on the consideration that even in the edge-detection-filtered reference object images created using the method according to the invention, the highest light intensities contain little information about the positions of interest in the images of the at least one reference object. It is more important to maximize the resolution of the underlying intensity gradients.

[0039] When comparing the recorded reference object images within the scope of the method according to the invention, a shift in images of the at least one reference object between the reference object images can be used to infer a lateral displacement of the sample relative to the objective orthogonal to an optical axis of the objective, i.e., in the x and y directions. This shift can be determined, for example, by calculating correlations between the reference object images. On the other hand, deformations of the images of the at least one reference object between the reference object images can be used to infer a displacement of the sample relative to the objective along the optical axis of the objective.In this case, a similarity between images of the at least one reference object in the reference object images and images of the or other reference objects in reference object comparison images can be used to infer a displacement of the sample relative to the objective along the optical axis of the objective, determined by magnitude and / or direction. Specifically, at the start of the implementation of the method according to the invention, a stack of reference object comparison images can be acquired for various distances of the sample from the objective along the optical axis. The distance associated with the respective reference object image can then be inferred from the similarity between the image of the at least one reference object in one of the reference object images and the images of the reference object in the reference object comparison images.This inference is simplified if, with the aid of a cylindrical lens or other optical element arranged between the camera and the lens, an astigmatism or other aberration is introduced into the image on the camera, which eliminates the symmetry of the deformation of the images of at least one reference object with increasing distance from a focal plane of the lens, so that a direction of this distance becomes recognizable.

[0040] In the method according to the invention, the at least one reference object can be introduced into a glass plate or another plate of the sample made of an optically transparent material using a focused light pulse. It is generally known that even a glass plate can be locally modified by a focused light pulse, for example by evaporation, see, for example, US Pat. No. 9,370,956 B2. This can be used in the method according to the invention to attach the at least one reference object to a particularly suitable location on the sample or to introduce it into the sample. For this purpose, the light pulse can be focused into the respective plate made of optically transparent material using the objective lens. The focused light pulse can be positioned opposite the plate using a scanner, which scans the sample to generate measuring light from the sample with a local light intensity distribution.This refers, in particular, to the scanner of a laser scanning microscope. A laser in such a laser scanning microscope, which provides the localized light intensity distribution for generating the measurement light from the sample, can also be used to provide the light pulse for introducing the reference object into the sample, for example, if the laser is capable of concentrating its laser power into particularly short light pulses. Both a coverslip and a sample carrier can be used as a plate made of optically transparent material of the sample into which the reference object is introduced with a focused light pulse.

[0041] By repeating the light pulse, a permanent pattern encoding information can be introduced into the sample, either in the sample plate or in another plate made of optically transparent material. The encoded information can, for example, document the type, extent, and / or timing of measurements already taken on the sample.

[0042] As already mentioned repeatedly, the method according to the invention can be carried out while the objective is actually being used to measure the sample. Specifically, the reference object images can be recorded with the camera while a structure of interest on the sample is being imaged using confocal, STED, RESOLFT, or MINFLUX microscopy. If displacements of the sample relative to the objective are detected using the method according to the invention, these can be compensated for the microscopic imaging of the sample. This compensation can be achieved by directly compensating for the displacements, by opposing displacements of the sample relative to the objective, or by controlling an adaptive optics system that causes a virtual displacement of the sample relative to the objective, but also by computationally correcting the microscopic image of the sample.

[0043] If the at least one reference object is located in a reference object plane that is noticeably spaced from an object of interest on the sample in the direction of the optical axis of the objective, it is expedient for the at least one reference object to be imaged into the reference object images in the image plane with the aid of at least one lens additional to the objective. This means that the reference object plane can be offset in the direction of the optical axis of the objective relative to an object plane in which the object of interest on the sample imaged by the objective is arranged. In more general terms, it is preferred if the focal length of the imaging optics is adjustable so that a reference object plane imaged by the imaging optics into the image plane can be displaced in the direction of the optical axis of the objective relative to an object plane imaged by the objective.

[0044] In a device according to the invention for carrying out the method according to the invention with an objective and with a camera arranged in an image plane of an imaging optics comprising the objective, an illumination device couples illumination light into a waveguide of the sample aligned transversely to the optical axis of the objective.

[0045] An optical device can be arranged between the lens and the camera in a plane Fourier conjugate to the image plane, which optical device is designed to filter out low spatial frequencies from reference object images that the imaging optics images into the image plane.

[0046] The optical device arranged and designed in this way blocks out portions of light originating from a central area of ​​a pupil of the lens in front of the camera.

[0047] Preferably, the device is arranged in such an image plane which is Fourier conjugated to the image plane into which the pupil of the objective is imaged.

[0048] Specifically, the optical device for filtering out the low spatial frequencies can comprise a circular disk covering an area of ​​the image plane around the axial image point and thus the central region of the pupil, a mirror deflecting the low spatial frequencies toward another detector of the device, or a spatial light modulator (SLM). In the simplest case, the optical device is a wire running in the Fourier conjugate plane through the axial image point or the center of the pupil image.

[0049] The device according to the invention can have a bandpass filter between the lens and the camera, which only allows light of one wavelength from the at least one reference object to pass to the camera.

[0050] An illumination light source of the device for illuminating the at least one reference object can provide monochromatic illumination light, ie illumination light of one wavelength, wherein the wavelength is regularly the same as that which the bandpass filter arranged between the lens and the camera allows to pass to the camera.

[0051] Specifically, the illuminating light may have a wavelength in an infrared range of approximately 800 nm or preferably approximately 900 nm to 1,300 nm.

[0052] As already indicated, a beam path of the device according to the invention between the lens and the camera preferably begins at the lens as an infinity beam path. Accordingly, the lens then also connects to an infinity beam path during the actual measurement of the sample.

[0053] Advantageous developments of the invention emerge from the patent claims, the description, and the drawings. The advantages of features and combinations of several features mentioned in the description are merely exemplary and may be effective alternatively or cumulatively, without necessarily achieving the advantages of embodiments according to the invention. The invention is defined by the patent claims.

[0054] The number of features mentioned in the patent claims and the description is to be understood as meaning that exactly this number or a greater number than the stated number is present, without the need for the explicit use of the adverb "at least." Thus, for example, if reference is made to an optical element, this is to be understood as meaning that exactly one optical element, two optical elements, or more optical elements are present. The features mentioned in the patent claims may be supplemented by other features or may be the only features present in the respective method or device.

[0055] The reference signs contained in the patent claims do not represent a limitation of the scope of the subject-matter protected by the patent claims. They serve solely to make the patent claims easier to understand. BRIEF DESCRIPTION OF THE CHARACTERS

[0056] In the following, the invention is further explained and described with reference to preferred embodiments shown in the figures. Fig. 1 shows a device, not covered by the patent claims as such, for carrying out a method, not covered by the patent claims as such, for detecting displacements of a sample relative to an objective. Fig. 2 shows a detail of the device according to Fig. 1 . Fig. 3 shows a variant of the detail according to Fig. 2 . Fig. 4 shows an edge detection-filtered reference object image in a simplified representation. Fig. 5 shows a reference object image according to Fig. 4 Complementary image from the image parts filtered out in the edge detection-filtered reference object image in a simplified representation. Fig. 6 shows a real edge detection filtered reference object image. Fig. 7 is an intensity profile along a Fig. 6drawn section line through an image of a reference object in the real reference object image. Fig. 8 illustrates a compensation device of the device according to Fig. 1 . Fig. 9 shows a further device not covered by the patent claims as such, also with incident light illumination of the reference objects. Fig. 10 shows a device with transmitted light illumination of the reference objects, which as such does not fall under the patent claims. Fig. 11 shows a device according to the invention with lateral illumination of the reference objects. Fig. 12 shows a detail of a modification of the embodiment of the device according to the invention according to Fig. 11 and Fig. 13 shows a detail of another modification of the device according to the invention according to Fig. 11 . FIGURE DESCRIPTION

[0057] The Fig. 1The device 1 shown serves to detect displacements of a sample 2 relative to an objective 3. The sample 2 is arranged on a sample positioner 4 movable in the x-, y-, and z-directions, wherein the z-direction coincides with an optical axis 44 of the objective 3. The sample 2 comprises a slide 5, a cover glass 6, and an object of interest arranged therebetween. This object is primarily examined with the aid of the objective 3, i.e., for example, microscopically imaged or measured, which in Fig. 1 However, this is not shown in detail. This examination of the object of interest using the objective lens 3 occurs in parallel with the use of the device 1 for detecting the displacements of the sample 2 relative to the objective lens 3 in order to detect and compensate for any drift occurring therebetween.

[0058] The device 1 has an illumination light source 7 which provides a beam of illumination light 8 with one wavelength, i.e. monochromatic illumination light 8. The illumination light 8 is coupled into the objective 3 via a polarization beam splitter 9 and a tube lens 10, specifically into the center of an eyepiece-side pupil 19 of the objective 3, which lies on the optical axis 44 of the objective 3, and illuminates the sample 2 in the wide field. In the infinity beam path illustrated in all figures here, in which light coming from the sample 2 and exiting the objective 3 through the eyepiece-side pupil 19 is a parallel beam or a beam focused at infinity, the eyepiece-side pupil 19 coincides with a rear focal plane of the objective 3, so that the term eyepiece-side pupil is used here as a synonym for the rear focal plane of the objective 3.But even in the case of a finite beam path with a focal point of the light converging and emerging from the objective 3 of a microscope at the finite point, the eyepiece-side pupil can be so close to the rear focal plane of the objective 3 that this does not result in any relevant difference with regard to the following explanations.

[0059] Between the tube lens 10 and the objective 3, as shown in Fig. 1As shown, a λ / 4 retardation plate 11 may be arranged, which right-circularly polarizes the initially s-linearly polarized illumination light 8. Corresponding light 43 scattered by reference objects in the sample 2 remains essentially right-circularly polarized and is then p-linearly polarized after passing through the λ / 4 retardation plate 11 again, so that it is transmitted by the polarization beam splitter 9 to a camera 12. Light reflected at optical interfaces between the polarization beam splitter 9 and the λ / 4 retardation plate 11, on the other hand, continues to have s-linear polarization and is redirected by the polarization beam splitter 9 back to the illumination light source 7.

[0060] If the λ / 4 retardation plate 11 is omitted, all light reflected at optical interfaces from the polarization beam splitter 9 up to and including the sample 2 continues to have s-linear polarization and is redirected by the polarization beam splitter 9 back to the illumination light source 7. This also predominantly applies to the light returning from the reference objects into the objective 3. However, to a certain extent, the illumination light 8 scattered by the reference objects has p-linear polarization and thus passes through the polarization beam splitter 9 to the camera 12. The absolute amount of light 43 thus selectively passing from the reference objects to the camera 12 can be increased if necessary by increasing the intensity of the illumination light 8.

[0061] The light 43 from the reference objects in the sample 2 is imaged by an eyepiece lens 13 and another lens 14 into reference object images in an image plane 15. These reference object images in the image plane 15 are recorded by the camera 12. A bandpass filter 16 is also arranged between the polarization beam splitter 9 and the camera 12, which only transmits the light 43 with the wavelength of the illumination light to the camera 12. Furthermore, an amplitude filter 18 is arranged there, in a plane 17 Fourier-conjugated to the image plane 15, which filters out portions of the light originating from a central region of the pupil 19 of the objective 3 with a circular disk 20 arranged in the central region of an image of the pupil 19 in the Fourier-conjugated plane 17.The amplitude filter 18 thus effects edge detection filtering of the reference object images in the image plane 15, specifically in front of the camera 12, so that the camera 12 records the already edge detection filtered reference object images by registering their light intensity distributions over the image plane 15.

[0062] Since the reference objects are illuminated with monochromatic illumination light 8, first-order intensity minima appear at steep edges of the reference objects in the light intensity distributions of the reference object images following zero-order intensity maxima that are significantly attenuated by edge detection filtering. These first-order intensity minima are followed by first-order intensity maxima. The intensity increases adjacent to the first-order intensity minima leading to the zero-order and first-order intensity maxima exhibit a high information density regarding the positions of the edges of the reference objects in relation to the total amount of light. If the reference objects have dimensions equal to or less than the wavelength of the illumination light 8, they consist practically only of edges, thus ensuring a maximum information density regarding the positions of the reference objects in relation to the total amount of light.

[0063] The device 1 records the reference images at successive times, and the reference object images are compared by a compensation device 21 in order to compensate for displacements of the sample 2 relative to the objective 3, which displacements become visible from changes in the reference object images, by controlling the sample positioner 4.

[0064] Fig. 2 shows the amplitude filter 18 in an axial plan view. The circular disk 20 is formed as a metallic coating of a transparent substrate 22 and is positioned at the center of an image 23 of the objective's pupil in the Fourier-conjugate plane 17, in which the amplitude filter 18 is arranged. The Fourier-conjugate plane 17 is a Fourier plane to the image plane 15, so that the circular disk 20 filters out low spatial frequencies from the reference object images while allowing higher spatial frequencies to pass through.

[0065] In the embodiment of the amplitude filter 18 according to Fig. 3 This is designed as a wire 24, which extends straight through its center over the image 23 of the pupil 19 of the objective 3 in the Fourier conjugate plane 17. The wire 24 also filters out higher spatial frequencies in the Fourier plane, but only to a small extent, while it filters out the low spatial frequencies like the circular disk 20 according to Fig. 2 completely hidden.

[0066] Fig. 4 shows a reference object image 5 resulting from edge detection filtering, in which the images 26 of individual reference objects consist only of edge regions in which the intensity of the light from the reference objects 16 increases. The essentially constant high intensities in the centers of the images 26, which in the reference object image 25 according to Fig. 4 complementary Figure 27 according to Fig. 5are reproduced and which do not contain any usable information about the position of the images 26 in the reference object image 25, are hidden from the reference object image 25. They are therefore not registered by the camera 12 and do not influence the camera 12. Rather, the camera 12 is controlled in such a way that it detects the intensity increase areas of the images 26 in the reference object image 25 according to Fig. 4 maximum resolution.

[0067] An overlay of the reference object image 25 according to Fig. 4 with the complementary image 27 according to Fig. 5 would result in a conventional image of the reference objects. However, such a conventional image of the reference objects is not recorded by the camera 12 of the device 1 and accordingly is not converted into the reference object image 25 by image processing according to Fig. 4 and the complementary figure 27 according to Fig. 5 disassembled.

[0068] While the Fig. 4 and 5simplified representations of the reference object image 25 and the complementary image 27 of quasi-point-shaped reference objects, Fig. 6 a real edge-detection-filtered reference object image 25 of reference objects in the form of so-called gold nanorods, whose diameter is smaller than the wavelength of light. In the transverse direction to the longitudinal extent of the respective gold nanorod, the images 26 of the individual reference objects show diffraction patterns, each with a zero-order intensity maximum 46 and two adjacent first-order intensity maxima 48.

[0069] The course of the light intensity along a Fig. 6 The section line 45 drawn through one of the images 26 of one of the reference objects is in Fig. 7 over the distance in pixels of the camera 12 according to Fig. 1plotted. The light intensity recorded with camera 12 in the region of the zero-order intensity maximum 46 is not only reduced by edge detection filtering to the level of the light intensity in the first-order intensity maxima 48, but is also capped at a maximum value by the control of camera 12. The latter also applies here to the light intensity in the first-order intensity maximum 48 shown on the left. Between the zero-order and first-order intensity maxima 46 and 48 lie first-order intensity minima 47. Intensity increase regions 49 and 50 adjacent to the first-order intensity minima 47 exhibit particular information density with regard to the position of the reference objects. Intensity increase regions 51 from second-order intensity minima 52 to the first-order intensity maxima 48 also contain usable position information.

[0070] Fig. 8illustrates an embodiment of the compensation device 21 according to Fig. 1. First, in a learning mode of the compensation device 21, a series of reference object images with different axial positions zi along the optical axis of the objective 3 are recorded and stored as reference object comparison images in a comparison image stack memory 28. The sample 2 is then moved into its target position using the sample positioner 4, and a compensation mode of the compensation device 21 is started. In this compensation mode, a control loop is run through. In each run of the control loop, a reference object image 25 is recorded with the camera 12. Then, an image correlator 29 calculates a spatial cross-correlation from the recorded reference object image and at least some of the reference object comparison images in the comparison image stack memory 28. The calculations result in a lateral position deviation Δr in the form of the position of a correlation maximum.This position of the correlation maximum corresponds to a shift of the images 26 of the reference objects in the current reference object image relative to the position of the images 26 in one or more of the reference object comparison images in the comparison image stack 28. The lateral position deviation Δr indicates the lateral drift of the sample 2 relative to the optical axis 44 of the objective 3 since the acquisition of the reference object comparison images in the teach-in mode. Furthermore, the image correlator determines the reference object comparison images with which the current reference object image 25 exhibits the highest image similarity, i.e., in which the images 26 of the reference objects are shaped or deformed in the same way.An axial position deviation ΔI is inferred from the axial positions zi of these most similar reference object comparison images by equating the current axial position of sample 2 with the axial position zi of the reference object comparison image with the highest degree of similarity, or by extrapolating the current axial position of sample 2 relative to lens 3 from the axial positions zi of the reference object comparison images with the highest degree of similarity. Control signal formers 30 and 31 generate control signals 32 and 33 to compensate for the lateral position deviation Δr and axial position deviation ΔI, thereby canceling out these position deviations, taking into account the image scale of the image of sample 2 onto camera 12.After controlling the sample positioner 4 with these control signals 32 and 33, the next run of the control loop takes place with the recording of the next reference object image 25 with the camera 12.

[0071] To calculate the cross-correlations between the current reference object image 25 and the reference object comparison images in the comparison image stack 28, instead of considering the entire images, only a portion of the reference object image and the reference object comparison images can be considered in order to suppress potential interference signals. However, the reference object comparison images can be acquired in equidistant axial positions. Rather, it is advantageous to select a very small axial distance between the reference object comparison images around the target position of sample 2 and to increase it with increasing axial distance from the target position. In this way, finely graded reference object comparison images are available in the range of low axial drift, so that even small deviations can be detected.At the same time, a large drift range can be covered without having to record a large number of reference object comparison images and process them in each pass of the control loop.

[0072] Furthermore, in addition to the signals generated by the reference objects in the form of images 26 in the reference object images 25, it is possible to image a back reflection from the objective 3 or a meniscus lens arranged in front of the pupil 19 of the objective 3 onto the camera 12. This back reflection can be used to detect and correct any drift of the optical elements involved in imaging the reference objects in the reference object images 25 separately, i.e., independently of the drift of the sample 2 relative to the objective 3.

[0073] The embodiment of the device 1 according to Fig. 9 differs from that according to Fig. 1by a somewhat more compact design, in which the amplitude filter 18 is also arranged in a plane 17 Fourier conjugate to the image plane 15, but not behind the eyepiece lens 13 and thus not in a plane Fourier conjugate to the object plane, into which the pupil 19 of the lens 3 is imaged. But here too, the circular disk 20 in front of the camera 12 blocks out the low spatial frequencies and also the portions of the light from the central area of ​​the pupil 19 of the lens 3. Fig. 9 no polarization directions of the illumination light 8 and the light from the reference objects are shown, does not mean that the same polarization directions as in Fig. 1 The arrangement of the bandpass filter 16 at a different location in the beam path changes Fig. 1 nothing about its function.

[0074] In the embodiment of the device 1 according to Fig. 10the illumination light 8 is not applied to the sample 2 through the objective 3 in a reflected light configuration. Instead, the illumination light 8 is applied here in a transmitted light configuration from the illumination light source 7 through a collector lens 34, a light field diaphragm 35, an aperture diaphragm 36 and a condenser lens 37 with respect to the objective 3 from behind onto the sample 2. The imaging of the reference objects in the sample 2 illuminated with the illumination light 8 onto the image plane 15, in which the camera 12 is arranged, corresponds in the embodiment of the device 1 according to Fig. 10 that in the embodiment of the device 1 according to Fig. 1 . A Fig. 1 The intermediate image plane 38, which is also present but not shown there, is also drawn in. Even if the illumination light source 7 is in Fig. 10as a light bulb, in this embodiment of the device 1 the light 43 reaching the camera 12 from the reference objects can also have only one wavelength, ie be monochromatic.

[0075] Fig. 11 illustrates an embodiment of the device 1 according to the invention, in which a different illumination of the reference objects in the sample 2 is realized. This illumination is shown in an enlarged detail. Via a prism 39, a beam of the illumination light 8 is coupled laterally into the cover glass 6, which thus serves as a waveguide for the illumination light 8. The reference objects 14 in the sample 2 are illuminated by the illumination light 8 guided in the cover glass 6. The imaging of the light from the reference objects 40 onto the camera 12 corresponds Fig. 1 and Fig. 10 .

[0076] Fig. 12shows how a light guide 41 with a holding device 42 can be attached to the lens 3 in order to guide the illuminating light 8 into the prism 39 according to Fig. 11 to couple.

[0077] Fig. 13 illustrates an alternative embodiment of the lateral illumination in an enlarged detail of Fig. 11corresponding illustration. The light guide 41 couples the illumination light 8 into the prism 39, which is arranged here on the sample carrier 5, so that the sample carrier 5 serves as a waveguide for the illumination light 8 to guide it to the reference objects 40 in the sample 2. The reference objects 40 can be markings introduced into the sample 2 or into the cover glass 6 or the sample carrier 5 using laser light. For this purpose, highly concentrated laser light can be applied in short pulses to the respective location on the sample 2, whereby the damage threshold of the respective glass is reached. In this way, the reference objects 40 can be attached to particularly suitable positions on the respective sample.

[0078] The Fig. 11 to 13The lateral illumination of the reference objects explained above is also disclosed here as an invention, independent of the feature that portions of the light originating from a central area of ​​a pupil of the lens are blocked out in front of the camera. LIST OF REFERENCE SYMBOLS

[0079] 1Device 2Specimen 3Objective 4Specimen positioner 5Slide 6Cover glass 7Illumination light source 8Illumination light 9Polarization beam splitter 10Tube lens 11λ / 4 waveplate 12Camera 13Ocular lens 14Lens 15Image plane 16Bandpass filter 17Fourier conjugate plane 18Amplitude filter 19Pupil 20Circular disk 21Compensation device 22Substrate 23Image of pupil 19 24Wire 25Reference object image 26Image of a reference object 27Complementary image 28Comparative image stack 29Image correlator 30Control signal conditioner 31Control signal conditioner 32Control signal 33Control signal 34Collector lens 35Field diaphragm 36Aperture diaphragm 37Condenser lens 38Intermediate image plane 39Prism 40Reference object 41Light guide 42Holding device 43Light 44Optical axis 45Section line 46Zero-order maximum 47First-order minimum 48First-order maximum 49Intensity increase range 50Intensity increase range 51Intensity increase range 52Second-order minimum Δrlateral position deviation ΔIaxialPosition deviation

Claims

1. Method of detecting movements of a sample (2) with respect to an objective (3), - wherein light (43) from at least one reference object (40) connected to the sample (2) is imaged at consecutive points in time by means of the objective (3) into reference object images (25) in an image plane (15), - wherein the reference object images (25) in the image plane (15) are recorded by a camera (12), and - wherein the recorded reference object images (25) are compared to one another, characterized in - that illumination light is coupled into a waveguide which is oriented transversely to the optical axis (44) of the objective (3), and that the at least one reference object (40) is illuminated with the coupled-in illumination light (8) that induces the irradiation of the light (43) from the at least one reference object (40), - wherein a glass plate or another plate of made of an optically transparent material of the sample (2) is used as the waveguide, wherein the at least one reference object (40) is in the plate.

2. Method of claim 1, characterized in that the plate serves as an object slide (5) or coverslip (6) of the sample (2).

3. Method of claim 1 or 2, characterized in that the reference object is formed in the plate by a focused light pulse.

4. Method of claim 2, characterized in that the at least one reference object (40) is - a defect of the object slide (5) or the coverslip (6), which acts as a scatter center, or - a marker which is formed in the cover slip (6) or the object slide (5) by laser light.

5. Method of claim 1 or 3, characterized in that the at least one reference object (40) has an edge across which the intensity of the light (43) from the reference object (40) drops by at least 90 % and whose width in parallel to the image plane is smaller than the wavelength of the light (43), wherein, optionally, the at least one reference object (40) is a point-shaped marker or a bead or a gold nanorod having a diameter below the wavelength of the light (43).

6. Method of any of the preceding claims, characterized in that the light (43) of the one wavelength from the at least one reference object (40) is, based on its wavelength, separated from measuring light from the sample (2) which is imaged by means of the objective (3) onto another detector than the camera (12).

7. Method of any of the preceding claims, characterized in that the sample (2) is illuminated with illumination light (8) of one wavelength, that induces the light (43) from the at least one reference object (40), wherein the wavelength of the illumination light (8) differs by at least 50 nm from wavelengths both of measuring light from the sample (2) and of excitation light that induces the measuring light from the sample (2), wherein, optionally, the wavelength of the illumination light (8) is equal to or longer than 800 nm.

8. Method of any of the preceding claims, characterized in that further light from at least one optical boundary surface located between the sample (2) and the objective (3) or from at least one further reference object (40) which is arranged at the objective (3) is recorded by the camera (12) in the reference object images (25).

9. Method of any of the preceding claims, characterized in that the camera (12) is controlled such that the maximum light intensities in the reference object images (25) reach 90 % to 1000 % or 100 % to 150 % of a saturation intensity of the camera (12).

10. Method of any of the preceding claims, characterized in - that a lateral movement of the sample (2) with respect to the objective (3) orthogonal to an optical axis (44) of the objective (3) is gathered from a shift of images (26) of the at least one reference object (40) between the reference object images (25), and / or - that a movement of the sample (2) with respect to the objective (3) along the optical axis (44) of the objective (3) is gathered from deformations of images (26) of the at least one reference object (40) between the reference object images (25).

11. Method of any of the preceding claims, characterized in wherein a movement of the sample (2) with respect to the objective (3) along the or an optical axis (44) of the objective (3), which is defined with regard to magnitude and / or direction, is gathered from a similarity of images (26) of the at least one reference object (40) in the reference object images (25) with images (26) of the or other reference objects (40) in reference object comparison images.

12. Method of any of the preceding claims, characterized in that the reference object images (25) are recorded by the camera (12) while a structure of interest of the sample (2) is imaged by confocal, STED, RESOLFT or MINFLUX microscopy, wherein the detected movements of the sample (2) with respect to the objective (3) are compensated.

13. Apparatus for carrying out the method of any of the preceding claims, comprising an objective (3), a camera (12) which is arranged in an image plane (15) of imaging optics that include the objective (3), and an illumination device coupling-in illumination light (8), characterized in - the apparatus further includes a waveguide, and that the illumination device couples the illumination light (8) into the waveguide of the sample (2), which is oriented transversely to the optical axis (44) of the objective (3), and - that the waveguide is a glass plate or another plate of made of an optically transparent material of the sample (2), wherein the reference object (40) is in the plate.

14. Apparatus of claim 13, characterized in that the illumination device couples the illumination light (8), via a prism (39), laterally into a cover slip (6) of the sample (2), which, in this way, serves as the waveguide for the illumination light (8), wherein, optionally, a light guide (41) is attached to the objective (3) by means of a holding device (42) to couple the illumination light via the light guide (41) into the prism (39).

15. Apparatus of claim 13, characterized in that the illumination device couples the illumination light (8), via a light guide (41), into a prism (39) which is arranged on an object slide (5) of the sample (2), such that the object slide (5) serves as the waveguide for the illumination light (8), wherein, optionally, the reference objects (40) are markers which are formed in the object slide (5) by laser light.