Method, computer program, apparatus and microscope for estimating a position of an emitter in a sample

Abstract

The present invention relates to a method, a computer program having instructions and an apparatus for estimating a position of an emitter in a sample. The present invention moreover relates to a microscope which uses a method according to the invention or an apparatus according to the invention. In the method, the sample is scanned (S1) using illumination light. In the process, the illumination light is applied to illumination positions on a scanning path that are determined on the basis of a previously estimated position of the emitter. Light emanating from the emitter is detected (S2) for the illumination positions. An updated estimated position of the emitter is then determined (S3) on the basis of the light detected for the illumination positions. In addition, it is possible to determine (S4) an uncertainty of the determination of the updated estimated position. Moreover, an updated scanning path can be determined (S5) on the basis of the updated estimated position of the emitter.

Description

[0001] Description

[0002] Method, computer program, device and microscope for estimating a position of an emitter in a sample

[0003] The present invention relates to a method, a computer program with instructions, and a device for estimating the position of an emitter in a sample. The present invention also relates to a microscope that carries out a method according to the invention or uses a device according to the invention.

[0004] Fluorescence microscopy is an established tool in numerous fields of research, for example, biology, biochemistry, biophysics, and medicine. One challenge with fluorescence microscopy is so-called photobleaching. Fluorescent molecules cannot be excited indefinitely, as they can be destroyed by the excitation light. It is therefore advisable to implement photon-efficient approaches in fluorescence microscopy.

[0005] A first approach to photon-efficient fluorescence microscopy is CLEM (Controlled Light-Exposure Microscopy) [1]. CLEM is based on the idea of ​​adapting the excitation in an imaging area to the amount of fluorophores with which a structure is labeled. One goal is to acquire good images with as little light as possible in order to reduce sample bleaching. During a scan, therefore, as much light should always be collected from an area as would otherwise be needed for conventional imaging. In areas with high fluorescence, little excitation light is needed, whereas in areas where no usable fluorescence is present, even a lot of excitation light cannot generate a signal. In areas where usable fluorescence is present, but only to a small extent, it is worthwhile to use more excitation light. In this way, the signal can be improved.By correcting the measured brightness values ​​using the known excitation differences, the final image obtained is one that corresponds to a completely normal confocal image.

[0006] Another approach for photon-efficient fluorescence microscopy is MINFLUX (MINimal photon FLUXes) [2]. MINFLUX is a single-molecule-based, super-resolution microscopy or nanoscopy technique. At its core, MINFLUX localization is based on the idea of ​​injecting a reference coordinate into the sample using a structured optical beam of illuminating light, e.g., a donut with a central intensity minimum, i.e., ideally, a zero. The position of the zero in the sample defines the targeted sample coordinate. The MINFLUX concept applies equally to entire sets of reference coordinates, i.e., line- and point-like zeros, as well as to parallelized detection in the wide-field.Coordinate targeting enables controlled and thus photon-efficient localization of fluorescent molecules. The fluorophore coordinate to be determined is no longer determined by determining the center of a weak, diffraction-limited fluorescence spot captured by a camera. Instead, the fluorophore is localized by actively aligning the zero of the excitation light to the fluorophore. Specifically, the zero of the excitation light is moved as close as possible to the molecule in several iterations until the detected fluorescence rate approximately corresponds to that of the background noise. At this closest proximity, only a minimal number of fluorescence photons is required to achieve maximum localization precision, since determining the remaining distance between the coordinate targeted by the zero and the molecule position requires far fewer detected photons.Thus, "injecting" or targeting a reference coordinate with a zero shifts the requirement of many fluorescence photons for localization to the virtually unlimited number of photons in the excitation light.

[0007] US Patent No. 11,255,791 B2 [3] describes the "injection" of a reference coordinate by superimposing an excitation light distribution with a local maximum on a STED light distribution with a local minimum. The sample is scanned by shifting the STED distribution with the STED minimum, and the position of the fluorophore is determined from the measured fluorescence intensity values ​​at various positions of the STED intensity distribution using a fitting algorithm. As with MINFLUX, scanning is carried out particularly in a close range around an emitter to be localized. Like the MINFLUX method, the method can be performed iteratively with smaller close ranges from step to step.

[0008] The published patent application US 2023 / 0101017 A1 [4] describes a method and a device for the parallel or simultaneous determination of positions of several spaced-apart molecules in one or more spatial directions in a sample. In the method, a plurality of light distributions are generated using a first light modulator with a plurality of switchable pixels. The first light modulator is arranged in an image plane of a localization microscope. Each light distribution has a local intensity minimum and adjacent intensity increase regions. Each of the plurality of molecules is illuminated with a respective light distribution. For each of the light distributions, photons emitted by the molecules are detected for different positionings of the light distribution (S4). The light distributions are positioned independently of one another (S3).Based on the photons recorded for the different positions of the light distributions, the positions of the molecules are finally derived.

[0009] Similar to the MINFLUX method is the RASTMIN (single-molecule localization by RASTer scanning a MINimum of light) method [5]. In this method, a rectangular area with dimensions in the range of approximately 100 nm, in which an isolated fluorophore is located, is scanned line by line with a donut-shaped focus of excitation light, and the fluorescence emitted from the sample is detected confocally. The resulting image of the fluorophore is compared with a high-resolution reference image of a single emitter. From the comparison, the position of the fluorophore within the rectangular area is determined using a maximum likelihood estimation. Scanning is carried out using a galvo scanner. The method achieves resolutions equivalent to those of the MINFLUX method, but can be implemented with less equipment effort, in particular without the use of electro-optical deflectors.According to the cited publication, if RASTMIN is to be performed iteratively, with the extent of the rectangular area in which the sample is exposed to excitation light being adjusted accordingly, the aforementioned advantage of reduced equipment costs is largely lost. The publication does not provide any information on how RASTMIN could be performed iteratively.

[0010] It is an object of the invention to provide improved solutions for estimating the position of an emitter in a sample. In particular, solutions are to be provided that enable the adjustment of scanning points at which an emitter, in particular an isolated emitter, is illuminated with illumination light in a sample with minimal equipment complexity, particularly using mechanical scanners.

[0011] It is a further object of the invention to provide improved solutions for the simultaneous estimation of positions of multiple emitters, in particular multiple individual emitters, in a sample. In particular, solutions are to be provided that enable the adaptation of sampling points for scanning multiple individual fluorophores in a sample, at which the respective fluorophore is illuminated with illumination light, with minimal equipment complexity, in particular using mechanical scanners. The invention is set forth in the independent claims. Preferred embodiments of the invention are the subject of the dependent claims.

[0012] According to a first aspect of the invention, a method for estimating a position of an emitter in a sample comprises the steps of:

[0013] - scanning the sample with illuminating light, wherein the illuminating light is switched on at illumination positions of a scanning path determined depending on a previously estimated position of the emitter;

[0014] - detecting light emanating from the emitter for the illumination positions; and

[0015] - Determining an updated estimated position of the emitter based on the light detected for the illumination positions.

[0016] According to a further aspect of the invention, a computer program includes instructions that, when executed by a computer, cause the computer to perform the following steps for estimating a position of an emitter in a sample:

[0017] - scanning the sample with illuminating light, wherein the illuminating light is switched on at illumination positions of a scanning path determined depending on a previously estimated position of the emitter;

[0018] - detecting light emanating from the emitter for the illumination positions; and

[0019] - Determining an updated estimated position of the emitter based on the light detected for the illumination positions.

[0020] The term "computer" should be understood broadly. In particular, it also includes microcontrollers, embedded systems, and other processor-based data processing devices. The execution of these steps can be performed directly by the computer or involve the computer controlling a component intended to execute a step, such as a light source, a scanner, or a detector.

[0021] The computer program may, for example, be made available for electronic retrieval or stored on a computer-readable storage medium.

[0022] According to a further aspect of the invention, an apparatus for estimating a position of an emitter in a sample comprises:

[0023] - a control device for scanning the sample with illuminating light, wherein the illuminating light is switched on at illumination positions of a scanning path determined as a function of a previously estimated position of the emitter;

[0024] - a detection device for detecting light emanating from the emitter for the illumination positions; and

[0025] -an estimator for determining an updated estimated position of the emitter based on the light detected for the illumination positions.

[0026] The emitter whose position is estimated is preferably an isolated emitter. In the context of the application, an emitter is referred to as isolated if the distance to a nearest neighboring emitter, from which it is optically indistinguishable, is at least as large as a distance corresponding to the resolution of the optical arrangement with which the emissions are detected during the method.

[0027] While in the CLEM described above, the amount of excitation light is adjusted to the amount of fluorophores in the scanned areas, in the solution according to the invention, the same emitter is always excited during a localization. To maximize the information provided by the photons emitted by the emitter, the illumination light is switched on at suitable positions along the scan path and switched off again at suitable positions along the scan path. In particular, this can mean that the illumination light is switched on at suitable positions along the scan path and not switched on at other positions along the scan path.In this way, the amount of illumination light required for localization is reduced by not introducing illumination light into the sample at all positions along the scan path, but only at positions particularly suitable for estimating the emitter's position, i.e., the specific illumination positions along the scan path. This, in turn, leads to less stress on the sample overall. The more light is required, the more the emitters suffer that are not currently being localized but are, for example, randomly activated in an outer region of the focus of the illumination light, especially if the emitters are fluorescence emitters. Switching on illumination light can, in particular, also mean that switched-on illumination light is left switched on.

[0028] In this application, emitters are understood to mean objects which, when illuminated with illuminating light, which is excitation light, can be regarded as point light sources with regard to the measurements according to the invention. The light emanating from the object acting as a point light source can, for example, be scattered light resulting from elastic scattering such as Rayleigh scattering or inelastic scattering such as Raman scattering, or it can be luminescent light, in particular fluorescent light. Emitters can therefore also be, in particular, individual fluorophores or individual fluorescent groups of fluorescent dyes, which are also referred to as fluorescence emitters in this application. Accordingly, in the context of this application, excitation light is understood not only to mean fluorescence excitation light, but generally to mean light that causes light to emanate from an emitter.Of particular importance in the context of emitter localization using a microscope are switchable fluorescence emitters, which, in addition to a state in which they can be excited to fluorescence by the excitation light used, can have another state in which they cannot be excited to fluorescence by the excitation light used. Accordingly, the emitter whose position is estimated according to the invention is particularly preferably a switchable fluorescence emitter.

[0029] The illumination light can, for example, be excitation light or emission-preventing light or fluorescence-preventing light, in particular STED light, which suppresses fluorescence emission by exciting stimulated emission. STED light is a special form of emission-preventing light in the context of this application. If the illumination light is emission-preventing light or fluorescence-preventing light, the emitter is excited to emit using additional excitation light. Excitation can in principle also occur in the wide field; preferably, excitation occurs with the focus of an excitation light superimposed on a focus of the illumination light, i.e., the emission-preventing light or fluorescence-preventing light.

[0030] In this application, a scanning path is understood to be the path described by a focal point or an imaginary focal point of a beam path of an illuminating light in a sample over time. An imaginary focal point in a sample plane is a point in the sample at which the focal point of the illuminating light would be formed if the illuminating light were switched on. This scanning path is determined in particular by the temporal change in the setting of a beam deflection device that shifts the focal point or the imaginary focal point in the sample. The scanning path can be predetermined, i.e., the scanning path can correspond to a path specified by the control of the beam deflection device; it can result from a superposition of a deflection specified by the control and a random deflection; or it can result entirely from a random deflection within a range of possible deflections.In contrast to the scans typically used in MINFLUX methods, the scan path does not have any points at which the focal point or the imaginary focal point is at rest. This means that the focal point or the imaginary focal point is constantly shifted in the method according to the invention, i.e. the scan path is continuous. In the context of the present application, a mechanical scanner is a scanner that variably deflects or shifts an illumination light, wherein the variation of the deflection or shift occurs by moving at least one optical component, e.g. a mirror. The movement can in particular be a rotation, in particular a tilt, about one or more axes of rotation. Accordingly, both a galvo scanner or a galvo scanner system and a micro-electromechanically actuated scanner, i.e. a MEMS scanner, are mechanical scanners within the meaning of this application.Likewise, a scanner based on rotatable Risley prisms is a mechanical scanner within the meaning of this application.

[0031] According to one aspect of the invention, an uncertainty in the determination of the updated estimated position is determined, i.e., the uncertainty of the updated estimated position is determined. The estimated, i.e., determined uncertainty can be taken into account when determining the illumination positions of the scanning path. For example, with a greater uncertainty, the illumination positions can be determined such that the illumination light is switched on over a larger section of the scanning path than is the case with a lower uncertainty. In this way, it is ensured that the location of the emitter to be localized is covered by the illumination light; in particular, this can mean that it is ensured that illumination positions in the directions in which the emitter's position is estimated lie on both sides of the emitter.

[0032] According to one aspect of the invention, updated illumination positions at which illumination light is introduced into the sample are determined based on the updated estimated position and / or the estimated uncertainty of the emitter. In particular, the distance between the outermost illumination positions at which illumination light is introduced into the sample can be reduced. In connection with updating the illumination positions, the intensity of the illumination light can also be increased; such a procedure is known as power ramping in the context of iterative MINFLUX methods.

[0033] According to one aspect of the invention, an initial estimated position of the emitter is determined during an initial scan or recording of the sample. Since an estimated position of the emitter is required to determine the illumination positions at which the illumination light is to be switched on, i.e., in particular, at which illumination light is to be introduced into the sample, or the illumination positions at which the illumination light is to be switched off, i.e., in particular, at which no illumination light is to be introduced into the sample, it is expedient to first perform an initial scan or recording of the sample. In this scan, an initial estimated position of the emitter can be determined, which is then used as the basis for further iterations of the position estimation.

[0034] According to one aspect of the invention, the estimated position of the emitter is determined in one, two, or three dimensions. The inventive solution can be used regardless of whether the localization is to be carried out in 1D, 2D, or 3D. For localization in 1D, for example, illumination light with a suitable intensity profile can be placed on both sides of the emitter, as can be generated, for example, with a half-moon phase plate (see, for example, in [6]); such an intensity profile is referred to in this application as a bun mode. Accordingly, with suitable beam profiles, localizations in 2D and 3D are also possible. Localization in 2D can alternatively also be carried out by two sequentially performed or two interleaved localizations in 1D.For example, a 3D localization can be performed by three sequentially performed or by three interleaved localizations in 1D; alternatively, for a 3D localization, one localization can be performed in 2D and one in 1D, whereby the localizations can be performed sequentially or interleaved.

[0035] According to one aspect of the invention, the illumination positions, in particular the illumination positions at which the emitter is illuminated with illumination light, are determined such that they lie on either side of the previously estimated position of the emitter. In this way, data can be obtained in the current iteration that enables a position estimation.

[0036] According to one aspect of the invention, an illumination position or illumination positions in the center or very close to the center of the scan path are determined as the additional illumination position(s) at which the emitter is illuminated with illumination light. The data obtained at this illumination position or these illumination positions can enable an improved determination of the updated estimated position or provide additional information that can be used, for example, in a background determination. Illumination positions are, for example, very close to the center of the scan path if they are less than 10 nm, in particular if they are less than 5 nm or less than 2 nm from the center of the scan path.According to one aspect of the invention, an updated scan path is determined based on the updated estimated position and / or based on the uncertainty of the updated estimated position of the emitter. For example, the spatial extent of the scan path can be reduced in order to limit the scan path to a narrow area around the location of the emitter to be located. The extent of the scan path can be adjusted in one, two, or three spatial directions. For this purpose, for example, amplitudes of deflection elements of the beam deflection device used to generate the scan path can be changed. The course, i.e., the shape, of the scan path can also be changed, for example, by adjusting amplitudes or frequencies of the deflection devices that generate the scan path.

[0037] According to one aspect of the invention, the center of the scan path is shifted based on the updated estimated position of the emitter. This allows the time required to locate an emitter to be reduced. The center of the scan path can be shifted, for example, by appropriately controlling the beam deflection device, which shifts the focal point or the imaginary focal point along the scan path in the sample, or by means of a separate deflection unit, or by shifting the sample.

[0038] According to one aspect of the invention, scanning along the scanning path is carried out with a beam deflection device that has a non-resonant scanner or with a beam deflection device that is constructed from non-resonant scanners, so that different scanning paths can be specifically scanned by means of the control.

[0039] According to one aspect of the invention, scanning along the scanning path is performed with a beam deflection device comprising a resonant scanner. Resonant scanners have the advantage of enabling rapid scanning of samples.

[0040] According to one aspect of the invention, scanning along the scanning path is performed with a beam deflection device having a resonant scanner for a first deflection direction and a non-resonant scanner for a second deflection direction.

[0041] According to one aspect, scanning along the scanning path is carried out with a beam deflection device constructed from mechanical scanners, or with a beam deflection device which, in addition to mechanical scanners causing a displacement of the focal point or the imaginary focal point within a plane, has a displacement device for displacing the focal point or the imaginary focal point in the direction of a normal to the plane.

[0042] According to one aspect of the invention, the sample is scanned line by line. In this embodiment of the invention, the sample is scanned in a raster pattern, specifically quickly along a line, for example, with a resonant galvo scanner, and slowly, i.e., slower than in the line direction, for example, with a non-resonant galvo scanner, in the column direction. Scanning in the column direction can be continuous, i.e., continuously, or discontinuous, i.e., with interruptions. If scanning in the column direction is continuous, i.e., continuously, the scanning along a line is inclined to the row direction, with the inclination depending on the ratio of the scanning speeds to one another.

[0043] A line-by-line scan in two spatial directions can also be combined with a superimposed scan in a third spatial direction perpendicular to the two spatial directions.For example, a sample plane can be scanned with a 3D donut, wherein at positions of the scanning path with predetermined radial distances, for example radial distances of 150 nm to 100 nm, from the presumed position of the emitter, i.e. initially or in a previous iteration step estimated position, illumination light is switched on in order to localize the emitter in two spatial directions, and wherein at illumination positions that have a small radial distance, for example a distance of at most 40 nm or at most 20 nm or at most 10 nm, from the presumed position of the emitter, the position of the local intensity minimum is placed at focus positions shifted along the third spatial direction, so that illumination positions at which the emitter is illuminated with illumination light lie on both sides of the emitter with respect to the third spatial direction in order to localize the emitter in the third spatial direction.In this context, the radial distance refers to the distance that the illumination positions would have without the additional displacement in the third spatial direction. This embodiment is an example of a 3D localization in which a 2D localization is interleaved with a 1D localization in the third spatial direction.Similarly, the 2D localization and the 1D localization in the third spatial direction can be carried out sequentially, wherein in one or more raster scans no displacement in the third spatial direction occurs, in particular with illumination positions of the scanning path selected accordingly as described above, at which the illumination light is switched on, and in further raster scans with scanning positions shifted in the third spatial direction, wherein the illumination positions at which the illumination light is switched on, in particular according to the above-described, have a small radial distance to the presumed position of the emitter.A shift of the focus positions along the third spatial direction during a raster scan or between successive raster scans can be carried out using methods known from the prior art, for example from WO 2023 / 062099 A1 [7], whereby switching times well below one microsecond can be achieved with Pockels cells.

[0044] A line-by-line scan in two spatial directions can also be performed in such a way that two 1D localizations occur, which in turn can be performed either sequentially or preferably interleaved. For this purpose, for example, bun modes with an intensity minimum extending in the row direction can be used for localization in the column direction, and bun modes with an intensity minimum in the column direction can be used for localization in the row direction. During scanning, the illumination light is switched on at specific illumination positions, and in addition, an orientation of the intensity minimum of the bun mode is selected depending on the illumination position with a view to maximizing the information content of the light emitted by the emitter.For example, a bun mode with an intensity minimum extending in the row direction can be selected along the outer rows as viewed in the column direction, and a bun mode with an intensity minimum extending in the column direction can be selected along the inner rows as viewed in the column direction. Sampling with bun modes for 2D localization from two 1D localizations can also be combined with localization in a third spatial direction. For this purpose, illumination light can be switched on at illumination positions that are a small radial distance from the presumed position of the emitter, so that illumination positions at which the emitter is illuminated with illumination light lie on either side of the emitter with respect to the third spatial direction in order to localize the emitter in the third spatial direction. 3D donuts, for example, can be used at these positions.In line with what has been said about entangled and serial localizations, the three 1D localizations can also be performed completely serially or partially, or preferably completely, entangled. 3D donuts shifted in the third spatial direction (or other shifted intensity profiles suitable for localization in the third spatial direction) can be generated, for example, by adjusting phase patterns in the beam path. Rapid switching between four phase patterns in the beam path is known, for example, from patent application US 2024 / 0231066 A9 [8].

[0045] According to one aspect of the invention, the sample is scanned along Lissajous paths, loops, or rosettes, which can, in particular, lie in a single plane. Particularly when using mechanical scanners, such scanning paths have the advantage that a region to be scanned, and thus also emitters, can generally be quickly scanned multiple times, enabling very rapid localization. For example, scanning along Lissajous paths can be performed using a beam deflection device in which a deflection along a first direction and a second direction perpendicular to the first direction is each caused by a resonant galvo scanner, wherein the scanners have different resonant frequencies.When scanning along Lissajous paths, loops or rosettes, i.e. when the scanning path is a Lissajou figure, a loop or a rosette, in a preferred embodiment the illumination positions at which the emitter is illuminated with illumination light can be determined such that they lie within a predetermined radial distance range from the initial estimated position or the position of the emitter estimated in a previous iteration step. Scanning along Lissajous paths, loops or rosettes that lie in a plane can also be combined with 1D localization in a spatial direction normal to the plane. In particular, illumination light can be switched on at illumination positions that have a small radial distance from the presumed position of the emitter, so that illumination positions at which the emitter is illuminated with illumination light arethe third spatial direction on both sides of the emitter in order to localize the emitter in the third spatial direction, whereby 3D donuts, for example, can be used at these positions.

[0046] According to one aspect of the invention, the beam deflection device comprises scanners from the group of galvo scanners, acousto-optical scanners, electro-optical scanners, piezo scanners, MEMS scanners, polygonal scanners, and Risley prism-based scanners. Which scanner(s) is / are used in the beam deflection device in a specific configuration is at the discretion of the person skilled in the art. Depending on their design, galvo scanners can be operated resonantly or non-resonantly. They are advantageously relatively simple in construction and inexpensive. For example, acousto-optical and electro-optical scanners have the advantage of high speed, but are limited to a relatively small field of view. High speed and a large field of view can be achieved, for example, by combining galvo scanners and electro-optical scanners. Some acousto-optical and electro-optical scanners can also be controlled resonantly.Among MEMS, there are solutions with a mirror that can oscillate in two directions, both resonantly and at different frequencies. With such a scanner, as well as with galvo scanners, Lissajous trajectories, loops, rosettes, and similar trajectories can be easily realized. Polygon scanners, on the other hand, have the advantage of being extremely fast.

[0047] According to one aspect of the invention, a current illumination position is determined from control data or measurement data of a beam deflection device or from measurement data of a beam measuring device. Beam deflection devices or the individual components of assembled beam deflection devices often monitor themselves. This monitoring is typically used for closed-loop control. However, the monitoring can also be used to infer the illumination position in the sample from the monitored values. For example, non-resonant galvo scanners practically always have an integrated encoder, the signal from which is required for closed-loop control. In the simplest case, these position signals can be used to infer the position of the beam in the sample and to couple the switching on or off of the illumination light to these signals.Resonant scanners also monitor themselves because they oscillate in a forced resonance. Here, too, the illumination light can be switched on and off depending on the position signals of the resonant scanner, which may only be generated at edge points or at zero crossing. Alternatively, the position can also be calculated from the control data used to control the beam deflectors, provided the parameters of the beam path and the behavior of the beam deflectors are known with sufficient accuracy. It is also possible to monitor the entire section of the optical path in which the beam deflectors are located using a single light beam that passes over all relevant beam deflectors. In this case, a separate measuring device is required. The advantage is that the current illumination position does not have to be derived from many individual values.

[0048] According to one aspect of the invention, the illumination light has an intensity profile in the shape of a donut, i.e., the illumination light has the shape of a donut at the focal point. Such a donut is particularly advantageous for localization in 2D and 3D and is technically easy to implement. In the context of this disclosure, the term "donut" is understood to mean a light distribution with a local intensity minimum that borders on intensity increase regions in at least two spatial directions. This includes, for example, a so-called 2D donut, which can be generated, for example, by phase modulation of the illumination light with a so-called vortex phase pattern (phase ramp), and a so-called 3D donut (also referred to as a bottle beam), which can be generated, for example, by phase modulation of the illumination light with a phase pattern with an annular phase jump.Such light distributions are known from the state of the art, in particular from the field of STED microscopy and MINFLUX microscopy.

[0049] According to one aspect of the invention, the illumination light is switched on by switching a light source that provides the illumination light, or by switching the illumination light from a secondary path to a useful path. Various options are possible for switching the illumination light. For example, pulsed semiconductor lasers can be switched directly. Furthermore, all lasers, regardless of whether they are pulsed or continuous-wave lasers, can be switched using fast acousto-optical or electro-optical elements. With pulsed lasers, depending on the pulse frequency, one illumination can certainly comprise several individual pulses. Finally, it is possible to switch the light from a secondary path to a useful path using acousto-optical or electro-optical elements. Fluorescence lifetimes are in the range of several nanoseconds.Since the switching times with electro-optical modulators can be below nanoseconds, sufficiently fast switching is guaranteed.

[0050] According to one aspect of the invention, the scanning path results from a superposition of a deflection predetermined by the control and a random deflection. In this embodiment, the fact that the illumination light is switched on at certain illumination positions and not at others ensures that, despite the random component of the beam deflection, the sample is not illuminated at illumination positions that are not particularly suitable for obtaining information for localizing the emitter.

[0051] According to one aspect, the scanning path results from a random deflection within a range of possible deflections, i.e., the scanning path corresponds to a fluctuation around a nominal position. In particular, the scanning path can correspond to a fluctuation around a center corresponding to the estimated position of the emitter. In this embodiment, too, the fact that the illumination light is switched on at certain illumination positions and not at others ensures that, despite the random component of the beam deflection, the sample is not illuminated at illumination positions that are not particularly suitable for obtaining information for localizing the emitter.

[0052] Random deflection can occur, especially when the beam path of the illuminating light includes means for influencing the illuminating light, such as beam deflection devices, that do not meet the highest requirements for dynamics, positioning accuracy, and positioning stability (as required by conventional MINFLUX methods). The fluctuation of the position of the illuminating light around a nominal position in the sample can be caused by rapid, irregular, or seemingly irregular movements of the mechanical components of the beam positioning device that shift the beam, for example, tilting mirrors in a galvo scanner system.Galvoscanners and galvoscanner systems operated in a closed control loop, like piezo-actuated sample stages, for example, perform permanent corrective movements due to the continuous tracking by the control loop in order to achieve the desired position of the control loop. These corrective movements are also irregular or appear irregular. Such random deflection around a nominal position, which covers a scanning range sufficiently extensive for localizing an emitter, can occur, for example, when using a beam deflection direction such as that commonly used in confocal microscopy, which uses controlled galvoscanners for scanning comparatively large image fields, or can be induced by selecting appropriate control parameters.

[0053] According to one aspect of the invention, the scanning path covers an area that, in at least one direction, preferably in two mutually perpendicular directions, is a multiple of an extension corresponding to the resolution of the optical arrangement with which the emissions are detected when carrying out the method. This embodiment is particularly suitable for the simultaneous estimation of positions of multiple emitters, in particular multiple individual emitters, in a sample. Simultaneous estimation is understood to mean that the measurement data from which the positions of the multiple individual emitters are estimated are obtained with a temporal overlap.

[0054] In this embodiment, the method for estimating a position of an emitter in a sample may comprise the steps

[0055] - scanning the sample with illuminating light, wherein the illuminating light is switched on at illumination positions of a scanning path determined depending on a previously estimated position of the emitter;

[0056] - detecting light emanating from the emitter for the illumination positions; and

[0057] - Determining an updated estimated position of the emitter based on the light detected for the illumination positions, for multiple emitters, wherein the scanning is performed along a scanning path that sweeps over multiple isolated emitters. The scanning can in principle be performed in the ways described above with regard to locating a single emitter. For example, line-by-line scanning is particularly suitable for this embodiment. Also particularly suitable for this embodiment is scanning along a Lissajous trajectory. As stated above, scanning along Lissajous trajectories can be performed using a beam deflection device in which a deflection along a first direction and a second direction perpendicular to the first direction is each caused by a resonant galvo scanner, wherein the scanners have different resonant frequencies.With regard to the embodiment described here, this has the advantage that the scanning area, whose extent in at least one direction or even in two mutually perpendicular directions is a multiple of an extent corresponding to the resolution of the optical arrangement with which the emissions are detected when carrying out the method, can be swept over several times very quickly, so that several emitters can be located simultaneously. In particular, in this embodiment, the resonance frequencies of the deflection units for the different deflection directions can be in a non-integer relationship to one another. This results in a scanning path along which the illumination positions at which the illumination light is switched on for each emitter densely fill the scanning area, which is advantageous.Combining different localizations, such as a 2D localization combined serially or interleaved with a 1D localization in a third spatial direction, or combining two or three 1D localizations serially or interleaved, can also be advantageously implemented in this embodiment. In contrast to the embodiment in which a single emitter is localized and in which the estimated position of the emitter can be the center of the scanning path, in the embodiment in which multiple emitters are localized, typically none of the emitters is the center of the scanning path. However, the possibility of iteratively localizing each emitter by updating the illumination positions at which the illumination light is switched on is retained.

[0058] According to a further aspect of the invention, the illumination light is switched on at positions along the scanning path that are further away from the emitter(s) currently to be located in order to detect illumination light emanating from emitters in the sample that have recently switched to a fluorescent state, to roughly estimate their position, and to scan them in subsequent steps, i.e., during subsequent scans along the scanning path, with illumination light that is switched on depending on this estimated position. In this context, "further away" means "far enough away" that none of the emitters currently to be located is exposed to a relevant amount of illumination light when the illumination light is switched on.At those positions along the scanning path where switching is not performed to locate the emitters currently being located, but rather to find additional emitters, a different intensity profile can be selected than the one used for locating the emitters, for example, a Gaussian profile. How the necessary switching between different intensity profiles can be achieved has already been explained above. When using a Gaussian profile as the intensity profile for detecting emitters, "further away" can mean, for example, at least one half-width of the Gaussian profile away or at least 70% of one half-width of the Gaussian profile away.

[0059] If the illumination light is switched on at positions along the scanning path that are farther away from the emitter(s) currently to be located, and if the emitters are switchable fluorescence emitters, the illumination light can advantageously be combined with activation light, which converts the switchable fluorescence emitters from a non-fluorescent state to a fluorescent state. Furthermore, the activation light is advantageously switched on selectively in areas where emitters are to be located, for example, because a particularly small number of emitters have been located in those areas or depending on the distance of the areas from the emitters currently to be located.

[0060] According to one aspect of the invention, a plurality of emitters are localized in a temporal sequence using one of the aforementioned methods, in particular such a plurality of emitters that an image of a structure in the sample is obtained.

[0061] According to a further aspect of the invention, particularly when the scanning path covers an area which, in at least one direction, preferably in two mutually perpendicular directions, is a multiple of an extent corresponding to the resolution of the optical arrangement with which the emissions are detected when carrying out the method, the scanning path covers an area in which at least one reference structure, which is preferably a reference emitter, is located, based on which a stabilization of the sample, i.e. a drift correction, is carried out. Reference structures are structures that are stable over the entire measurement duration, during which emitters are localized, so that their positions can be determined repeatedly during the entire measurement duration. Suitable reference emitters, for example, are fluorescent beads or reflective beads, for example gold beads.The reference structure(s) is / are illuminated with illumination light at different times by switching on illumination light at illumination positions assigned to the reference structure(s). If the reference structures are reference emitters, i.e., they act as point light sources within the meaning of this description, the reference emitters are preferably located in the same way as the emitters.

[0062] According to one aspect of the invention, the detection of the emissions from the sample is carried out descanned, in particular with a detector arranged confocally to a scanning plane, for example an array detector, which can be constructed, for example, from an array of several photon-counting avalanche diodes.

[0063] Preferably, a microscope uses a device or method according to the invention for estimating a position of an emitter in a sample.

[0064] Further features of the present invention will become apparent from the following description and the appended claims taken in conjunction with the figures.

[0065] Fig. 1 schematically shows a method for estimating a position of an emitter in a sample;

[0066] Fig. 2 shows a first embodiment of an apparatus for estimating a position of an emitter in a sample;

[0067] Fig. 3 shows a second embodiment of an apparatus for estimating a position of an emitter in a sample;

[0068] Fig. 4 shows a sample with a plurality of emitters;

[0069] Fig. 5 shows a basic structure of a MINFLUX microscope in which a solution according to the invention is implemented; and

[0070] Fig. 6 shows an exemplary structure of a beam deflection device.

[0071] To better understand the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. It is understood that the invention is not limited to these embodiments and that the described features may also be combined or modified without departing from the scope of the invention as defined in the appended claims.

[0072] Fig. 1 shows a schematic of a method for estimating the position of an emitter in a sample. The estimated position of the emitter can be determined in one, two or three dimensions. In the method, the sample is scanned with illumination light S1, e.g. line by line or along Lissajous paths, loops or rosettes. The scanning can be carried out using a resonant scanner or a non-resonant scanner. The illumination light is switched on at illumination positions of a scanning path determined depending on a previously estimated position of the emitter. For this purpose, an initial estimated position of the emitter can first be determined during an initial scan or recording of the sample. A current illumination position can be determined, for example, from control data or measurement data of a beam deflection device or from measurement data of a beam measuring device.The illumination light can be switched on by switching a light source that provides the illumination light, or by switching the illumination light from a secondary path to a useful path. Light emanating from the emitter is detected S2 for the illumination positions. Based on the light detected for the illumination positions, an updated estimated position of the emitter is then determined S3. In addition, an uncertainty in the determination of the updated estimated position can be determined S4. Furthermore, an updated scanning path can be determined S5 based on the updated estimated position of the emitter. For example, the center of the scanning path can be shifted or the course of the scanning path can be changed.

[0073] Fig. 2 shows a simplified schematic representation of a first embodiment of a device 30 for estimating the position of an emitter in a sample. The position of the emitter can be estimated in one, two, or three dimensions. The device 30 has an interface 31 for communication with external components. A control device 32 is configured to scan the sample with illuminating light, e.g., line by line or along Lissajous paths, loops, or rosettes. The scanning can be performed with a resonant scanner or a non-resonant scanner. For this purpose, the control device 32 can, for example, control components of a microscope, e.g., a light source 12 or a beam deflection device 14. The illuminating light is directed to a position L depending on a previously estimated position L nof the emitter at certain illumination positions of a scanning path. For this purpose, an initial estimated position Lo of the emitter can first be determined during an initial scan or recording of the sample. A current illumination position can be determined, for example, from control data or measurement data of a beam deflection device or from measurement data of a beam measuring device. The illumination light can be switched on by the control device 32 by switching the light source 12, which provides the illumination light, or by switching the illumination light from a secondary path to a useful path. A detection device 33 is set up to detect light emanating from the emitter for the illumination positions. For this purpose, the detection device 33 can, for example, evaluate signals from a photodetector 19, which are received via the interface 31.An estimation device 34 is configured to generate an updated estimated position L based on the light detected for the illumination positions. n +i of the emitter. The estimation device 34 can additionally be configured to determine an uncertainty in the determination of the updated estimated position. Furthermore, the control device 32 can be configured to determine an updated scanning path based on the updated estimated position of the emitter. For example, the control device 32 can shift the center of the scanning path or change the course of the scanning path.

[0074] The control device 32, the acquisition device 33, and the estimation device 34 can be controlled by a control unit 35. Settings of the control device 32, the acquisition device 33, the estimation device 34, or the control unit 35 can be changed if necessary via a user interface 37. The data generated in the device 30 can be stored in a memory 36 of the device 30, for example, for later evaluation or for use by the components of the device 30. The control device 32, the acquisition device 33, the estimation device 34, and the control unit 35 can be implemented as dedicated hardware, for example, as integrated circuits. Of course, they can also be partially or completely combined or implemented as software running on a suitable processor, for example, a GPU or a CPU.The interface 31 can also be implemented in the form of separate inputs and outputs.

[0075] Fig. 3 shows a simplified schematic representation of a second embodiment of a device 40 for estimating the position of an emitter in a sample. The device 40 has a processor 42 and a memory 41. For example, the device 40 is a microcontroller, a computer, or an embedded system. Instructions are stored in the memory 41 which, when executed by the processor 42, cause the device 40 to carry out the steps according to one of the described methods. The instructions stored in the memory 41 thus embody a program executable by the processor 42 which implements the method according to the invention. The device 40 has an input 43 for receiving information. Data generated by the processor 42 are provided via an output 44. Furthermore, they can be stored in the memory 41.The input 43 and the output 44 can be combined to form a bidirectional interface.

[0076] The processor 42 may include one or more processor units, such as microprocessors, digital signal processors, or combinations thereof.

[0077] The memories 36, 41 of the described embodiments can have both volatile and non-volatile memory areas and can comprise a wide variety of storage devices and storage media, for example hard disks, optical storage media or semiconductor memories.

[0078] As already explained above, a general idea of ​​the inventive solution is that a new position is estimated from currently collected data and, during further scanning, the illumination light is switched on and off again at positions that are adjusted to the estimated new position. Two concrete implementation options for this general idea are described below as examples.

[0079] In a first implementation option, the sample is scanned in a raster pattern. This occurs quickly along a row, the x-direction, and slowly in the column direction, the y-direction. Scanning in the y-direction can be continuous or discontinuous. Scanning then occurs along one or more rows. If scanning in the y-direction is continuous, scanning along the rows occurs at an angle to the row direction. In particular, scanning occurs along one or more rows within which the previously estimated position of the emitter lies. In this case, scanning begins, so to speak, in the middle. From the data obtained during scanning, the position of the emitter is estimated with respect to the row direction. Preferably, an uncertainty in the position estimate is also estimated.In the subsequent row(s), illumination then occurs at positions that are adjusted to the estimate of the emitter's position in the row direction. The estimate of the emitter's position in the y-direction is updated from the respective recorded rows. Depending on the updated estimate in the y-direction in conjunction with the respectively updated estimates in the x-direction, illumination occurs in further rows and / or not only the illumination but also the scanning in the y-direction is adjusted. In doing so, it is sensibly ensured that the scanning range in the y-direction includes scanning and illumination positions that actually lie on both sides of the emitter in the y-direction. This can then be achieved, for example, if scanning downwards in the y-direction happened to have begun above the emitter by chance, without the scanning direction of the y-scanner being reversed.

[0080] In a second implementation option, the sample is scanned along Lissajous paths, loops, or rosettes. A scan is then performed along such a path, and the illumination light is switched on whenever the focus is at the edge or in the edge region of the scanned area. Optionally, illumination can also be provided when the focus is in the center of the scanned area. A new estimate is then determined from the data obtained during the scan, and the selection of illumination points is adjusted to the new estimate. For example, illumination points further inside, on either side of the new estimated position of the emitter, and optionally at least one additional illumination point in the area close to the estimated position of the emitter are selected. Optionally, the center of the path can be shifted, e.g., by means of a separate deflection unit or by shifting the sample.Alternatively or additionally, the path of the beam can be changed, e.g. by adjusting the amplitudes or frequencies of the beam deflection devices by means of which the paths are generated, or of components of the beam deflection devices.

[0081] Fig. 4 shows a sample 1 with a large number of emitters 2. Five emitters 2 are shown as an example. The emitters 2 can be, for example, fluorophores or molecules labeled with fluorophores. The fluorophores can be excited to emit photons using light of a suitable wavelength. In MINFLUX microscopy, the fluorophores are excited in such a way that a fluorophore to be localized is always placed close to or in a minimum of a light distribution used for excitation, whereby the light distribution must have an intensity increase region adjacent to the minimum. This achieves better utilization of the fluorescence photons with regard to obtaining information about the position of the respective emitting fluorophore. Ideally, the minimum of the excitation light distribution is a zero.

[0082] Fig. 5 shows a simplified basic structure of a MINFLUX microscope 10 in which a solution according to the invention is implemented. The MINFLUX microscope 10 is controlled by a controller 11. In order to estimate the position of emitters in a sample, the controller 11 comprises a device 30 according to the invention for estimating the position of an emitter. The illumination light 13 emitted by a light source 12, in particular a laser, passes through a beam deflection device 14 and a scanning lens 15 before being focused into the focal plane of an objective 16, in which the sample is located. The light 17 emitted by an emitter in the sample is directed by a dichroic mirror 18 toward a photodetector 19 and detected by the latter. Other components of the microscope are known to those skilled in the art and are not shown for the sake of simplicity.

[0083] An exemplary structure of the beam deflection device 14 is shown in Fig. 6. In this example, the beam deflection device 14 comprises an x-scanner 140 and two y-scanners 141, 142, as well as a coupling mirror 143 for the illumination light 13. The scanning lens 15 images the rear focal plane of the objective onto the x-scanner 140. The y-scanners 141, 142 provided for the y-deflection are not located in the same pupil. However, two mirrors can always be adjusted as a pair so that the beam is not shifted in the rear focal plane of the objective when tilted. This has the following background. If a deflection mirror is placed at a location where the rear focal plane of the lens is imaged, i.e. at a pupil, then tilting the mirror causes a tilt of the beam in the rear focal plane of the lens, also a pupil, without the beam being shifted in the pupil.The tilt in the rear focal plane of the objective then leads to a shift in the focus of the sample. If illuminating light with a donut-shaped intensity profile is to be scanned across a sample, it is advantageous if the beam is tilted as little as possible in the rear focal plane of the objective.

[0084] Of course, the beam deflection device 14 can also be designed in a different way. For example, two x-scanners and two y-scanners can be combined, neither of which is located in a pupil. Likewise, one scanner can be provided for each direction, with both positioned as close as possible to a pupil. Another possibility is to place a relay optics between an x-scanner and a y-scanner, so that both scanners are located in one pupil. This embodiment is particularly advantageous when the inventive solution is implemented with two resonant scanners. References

[0085] [1] R.A. Hoebe et al.: „Controlled light-exposure microscopy reduces photobleaching and phototoxicity in fluorescence live-cell imaging”, Nature Biotechnology 25,

[0086] S. 249-253 (2007).

[0087] [2] F. Balzarotti et al.: “Nanometer resolution imaging and tracking of fluorescent molecules with minimal photon fluxes”, Science 355, S. 606-612 (2017).

[0088] [3] US 11 ,255,791 B2

[0089] [4] US 2023 / 0101017 A1

[0090] [5] L.A. Masullo et al.: “An alternative to MINFLUX that enables nanometer resolution in a confocal microscope”, Light Sei Appl 11, 199 (2022)

[0091] [6] P- Hoyer et al.: “Breaking the diffraction limit of light-sheet fluorescence microscopy by RESOLFT”, Proc Natl Acad Sei U S A. 2016 Mar 29; 113(13):3442-6.

[0092] [7] WO 2023 / 062099 A1

[0093] [8] US 2024 / 0231066 A9

[0094] Bezugszeichenliste

[0095] 1 sample

[0096] 2 emitters

[0097] 10 Microscope

[0098] 11 controllers

[0099] 12 Light source

[0100] 13 Illumination light

[0101] 14 Beam deflection device

[0102] 140 x scanner

[0103] 141 y-scanner

[0104] 142 y-scanner

[0105] 143 coupling mirrors

[0106] 15 Scan lens

[0107] 16 lens

[0108] 17 Emitted light

[0109] 18 Dichroic mirror

[0110] 19 Photodetector

[0111] 30 Device

[0112] 31 Interface

[0113] 32 Control device

[0114] 33 Recording device

[0115] 34 Estimating facility

[0116] 35 Control unit

[0117] 36 storage

[0118] 37 User interface

[0119] 40 Device

[0120] 41 storage

[0121] 42 processor

[0122] 43 Entrance

[0123] 44 Exit

[0124] Estimated location

[0125] S1 Scanning a sample with illuminating light

[0126] S2 Detecting light emitted by an emitter

[0127] S3 Determine an updated estimated position of the emitter

[0128] S4 Determine an uncertainty of the position estimate

[0129] S5 Determining an updated scan path

Claims

Patent claims 1. Method for estimating a position of an emitter (2) in a sample (1), comprising the steps: - scanning (S1) the sample with illuminating light (13), wherein the illuminating light (13) is directed at a position (L n ) of the emitter (2) is switched on to certain illumination positions of a scanning path; - detecting (S2) light emanating from the emitter (2) for the illumination positions; and - Determining (S3) an updated estimated position (L n +i) of the emitter (2) based on the light detected for the illumination positions.

2. The method according to claim 1, wherein the scanning path is a path that a focal point or an imaginary focal point of a beam path of an illuminating light in a sample describes over time.

3. Method according to claim 2, wherein the focal point or the imaginary focal point is continuously shifted.

4. Method according to one of the preceding claims, wherein an uncertainty of the determination of the updated estimated position (L n +i) is determined (S4).

5. The method according to any one of the preceding claims, wherein the scan path corresponds to a fluctuation around a center corresponding to the estimated position of the emitter.

6. The method according to any one of claims 1 to 4, wherein an updated scan path is determined based on the updated estimated position (L n +i) and / or the uncertainty in determining the updated estimated position (L n +i) of the emitter (2) is determined (S5).

7. Method according to one of claims 1 to 4 and 6, wherein the scanning (S1) of the sample (1) is carried out line by line.

8. Method according to one of claims 1 to 4 and 6, wherein the scanning (S1) of the sample (1) takes place along Lissajous paths, loops or rosettes.

9. Method according to one of the preceding claims, wherein the illumination positions at which the illumination light is switched on are determined such that they are located on both sides of the previously estimated position (L n ) of the emitter (2).

10. The method according to claim 9, wherein illumination positions in the center of the scanning path are determined as additional illumination positions at which the illumination light is switched on.

11. Method according to one of the preceding claims, wherein on the basis of the updated estimated position (L n +i) of the emitter (2) the center of the scanning path is shifted or the course of the scanning path is changed.

12. Method according to one of the preceding claims, wherein the scanning (S1) of the sample (1) is carried out with a beam deflection device.

13. The method according to claim 12, wherein the beam deflection device comprises a resonant scanner and / or a non-resonant scanner.

14. The method according to claim 12 or 13, wherein the beam deflection device (14) is constructed from mechanical scanners, or wherein the beam deflection device, in addition to mechanical scanners which cause a displacement of the focus point or the imaginary focus point within a plane, comprises a displacement device for displacing the focus point or the imaginary focus point in the direction of a normal to the plane.

15. Method according to one of claims 12 to 14, wherein a current illumination position is determined from control data or measurement data of the beam deflection device (14) or from measurement data of a beam measuring device.

16. Method according to one of the preceding claims, wherein the illumination light (13) has a local intensity minimum in one or in the focal point, in particular corresponding to a donut or a bun mode.

17. Method according to one of the preceding claims, wherein the illumination light (13) is switched on by switching a light source (12) providing the illumination light (13), or by switching the illumination light (13) from a secondary path to a useful path.

18. Method according to one of the preceding claims, wherein the scanning path has a range which, in at least one direction, preferably in two mutually perpendicular directions, is a multiple of an extent which corresponds to the resolution of an optical arrangement with which the emissions are detected when carrying out the method.

19. The method according to claim 18, wherein the method comprises the steps - scanning (S1) the sample with illuminating light (13), wherein the illuminating light (13) is directed at a position (L n ) of the emitter (2) is switched on to certain illumination positions of the scanning path; - detecting (S2) light emanating from the emitter (2) for the illumination positions; and - Determining (S3) an updated estimated position (L n +i) of the emitter (2) is carried out for a plurality of emitters based on the light detected for the illumination positions.

20. Method according to one of the preceding claims, wherein the illumination light and optionally an activation light are switched on at positions of the scanning path which are further away from the emitter or emitters currently to be located in order to detect illumination light emanating from emitters in the sample which have recently switched to a fluorescent state, to roughly estimate their position and to scan them in subsequent steps, that is to say during subsequent scans along the scanning path, with illumination light which is switched on as a function of this estimated position.

21. A computer program comprising instructions which, when executed by a computer, cause the computer to carry out the steps of a method according to any one of claims 1 to 20 to estimate a position of an emitter (2) in a sample (1).

22. Device (30) for estimating a position of an emitter (2) in a sample (1), comprising: - a control device (32) for scanning (S1) the sample with illuminating light (13), wherein the illuminating light (13) is directed at a position (L) estimated beforehand. n ) of the emitter (2) is switched on to certain illumination positions of a scanning path; - a detection device (33) for detecting (S2) light emanating from the emitter (2) for the illumination positions; and -an estimation device (34) for determining (S3) an updated estimated position (L n +i) of the emitter (2) based on the light detected for the illumination positions.

23. Microscope (10), characterized in that the microscope (10) comprises a device (30) according to claim 22 or is configured to carry out a method according to one of claims 1 to 20 for estimating a position of an emitter (2) in a sample (1).

Images