Methods and microscope for three-dimensional, high-resolution microscopy

Abstract

High-resolution microscopy method for the three-dimensional position determination of objects, in particular individual fluorophores, preferably for spatially high-resolution luminescence microscopy of a sample (7) which is labelled with labeling molecules (1) which can be activated or switched with a signal such that they can only be excited to emit certain luminescence radiation in the activated state, wherein the method comprises the following steps: a) Introducing the signal onto the sample (7) such that only a subset of the labeling molecules (1) present in the sample (7) are activated, wherein the sample (7) contains subregions in which activated labeling molecules (1) are at least as far apart from the nearest neighboring activated labeling molecules (1) as is greater than or equal to a length which results from a predetermined optical resolution, b) Excitation of the activated molecules (1) to emit luminescence radiation, c) Detection of the luminescence radiation with the predetermined optical resolution, and d) Generating a single image from the luminescence radiation recorded in step c), wherein the geometric locations of the luminescence-emitting labeling molecules are determined with a spatial resolution increased above the predetermined optical resolution, where the steps are repeated several times and the resulting individual images are combined to form a complete image, characterized by the fact that an object is imaged via an imaging system (OB, TL) with a microscope objective (15, OB) onto an area detector (5, CCD) consisting of individual detector elements, wherein at least one microlens arrangement (ML, ZL), which is partially located in front of the detector elements, images a different object plane (OE1) onto the detector elements in the direction of light behind the microlenses than onto detector elements in front of which there are no microlenses.

Description

[0001] The invention relates in particular to a microscope for spatially high-resolution luminescence microscopy of a sample labeled with labeling molecules which can be activated by a signal such that they can only be excited to emit certain luminescence radiation in the activated state, wherein the method comprises the following steps: a) Introducing the signal to the sample in such a way that only a subset of the labeling molecules present in the sample are activated, wherein there are subregions in the sample in which activated labeling molecules have a distance to the nearest neighboring activated labeling molecules of at least a length greater than or equal to a length resulting from a predetermined optical resolution, b) Excitation of the activated molecules to emit luminescence radiation, c) Detection of the luminescence radiation with the predetermined optical resolution and d) Generating a single image from the luminescence radiation recorded in step c), wherein the geometric locations of the luminescence-emitting labeling molecules are determined with a spatial resolution increased above the predetermined optical resolution, the steps are repeated several times and the resulting individual images are combined to form a complete image.

[0002] A classic application of light microscopy for examining biological specimens is luminescence microscopy. Here, specific dyes (so-called phosphors or fluorophores) are used to label samples, such as cell parts. As mentioned, the sample is illuminated with excitation radiation, and the resulting luminescence is detected by suitable detectors. Typically, the microscope uses a dichroic beam splitter in combination with blocking filters, which separate the luminescence from the excitation radiation, allowing for separate observation. This method makes it possible to visualize individual, differently stained cell parts under the microscope. Of course, several parts of a specimen can also be stained simultaneously with different dyes that bind specifically to different structures within the specimen.

[0003] This method is called multiple luminescence. It is also possible to measure samples that luminesce on their own, i.e., without the addition of dye.

[0004] Luminescence is used here, as is generally the case, as an umbrella term for phosphorescence and fluorescence, thus encompassing both processes. Where fluorescence is mentioned here, it is meant to be a representative example and not a limitation.

[0005] For sample examination, laser scanning microscopes (LSMs) are also used. These microscopes image only the plane located in the focal plane of the objective lens from a three-dimensionally illuminated image using a confocal detection arrangement (then called a confocal LSM) or a nonlinear sample interaction (so-called multiphoton microscopy). An optical section is obtained, and the recording of several optical sections at different depths of the sample then allows a suitable data processing device to generate a three-dimensional image of the sample, composed of these various optical sections. Laser scanning microscopy is therefore suitable for examining thick specimens.

[0006] Of course, a combination of luminescence microscopy and laser scanning microscopy is also used, in which a luminescent sample is imaged at different depth levels using an LSM.

[0007] In principle, the optical resolution of a light microscope, including a light microscope (LSM), is diffraction-limited by the laws of physics. Special illumination configurations are known for achieving optimal resolution within these limits, such as 4π arrays or arrays with standing wave fields. These configurations can significantly improve the resolution, particularly in the axial direction, compared to a conventional LSM. Furthermore, non-linear depopulation processes can increase the resolution by a factor of up to 10 compared to a diffraction-limited confocal LSM. One such method is described, for example, in US 5,866,911 A. Various approaches for these depopulation processes are known, such as those described in DE 4416558 C2, US 6,633,432 B2, and DE 10325460 A1.

[0008] Another method for increasing resolution is addressed in EP 1157297 B1. This method utilizes structured illumination to exploit nonlinear processes. The document cites fluorescence saturation as the nonlinearity involved. The described method aims to shift the object's spatial spectrum relative to the optical system's transfer function through structured illumination. Specifically, this spectrum shift means that object spatial frequencies V0 are transmitted at a spatial frequency V0 - Vm, where Vm is the frequency of the structured illumination. Given the maximum spatial frequency the system can transmit, this allows the transmission of object spatial frequencies that lie above the maximum frequency of the transfer function by the shift frequency Vm. This approach requires a reconstruction algorithm for image generation and the processing of multiple images to create a single image.Another disadvantage of this method is that the sample is unnecessarily exposed to radiation in areas outside the detected focus, as the required structured illumination permeates the entire sample volume. Furthermore, this method cannot currently be used with thick samples because extrafocally excited fluorescence reaches the detector as a background signal, thus drastically reducing the dynamic range of the detected radiation.

[0009] A method that achieves a resolution beyond the diffraction limit, independent of laser scanning microscopy, is known from WO 2006 / 127 692 A2 and DE 10 2006 021 317 B3. This method, abbreviated as PALM (Photo Activated Light Microscopy), uses a labeling agent that can be activated by an optical activation signal. Only in the activated state can the labeling agent be excited by excitation radiation to emit specific fluorescence. Non-activated molecules of the labeling agent emit no or at least no noticeable fluorescence, even after being irradiated with excitation radiation. The activation radiation thus switches the labeling agent into a state in which it can be excited to fluorescence. Other activation methods, e.g., thermal activation, are also possible. Therefore, this is generally referred to as a switching signal.In the PALM method, the switching signal is applied in such a way that at least a certain proportion of the activated labeling molecules are spaced far enough apart from neighboring activated molecules to be separated, or subsequently separable, according to the optical resolution of the microscopy. The activated molecules are thus largely isolated. After recording the luminescence radiation, the center of the resolution-limited radiation distribution of these isolated molecules is determined, and from this, the position of the molecules is calculated with higher accuracy than is actually possible with optical imaging. This increased resolution through computational determination of the center of gravity of the diffraction distribution is also referred to as "superresolution" in English-language literature.It requires that at least some of the activated labeling molecules in the sample are distinguishable, i.e., isolated, at the optical resolution used to detect the luminescence radiation. Then, for such molecules, the location can be determined with increased resolution.

[0010] To isolate individual labeling molecules, the PALM method utilizes the fact that the probability of a labeling molecule being activated after receiving a switching signal of a given intensity, e.g., a photon of the activation radiation, is the same for all molecules. The intensity of the switching signal, and thus the number of photons incident on a unit area of ​​the sample, can therefore be adjusted to ensure that the probability of activating labeling molecules present in a given area of ​​the sample is so low that there are sufficient regions where, within the optical resolution, only distinguishable labeling molecules emit fluorescence. By appropriately selecting the intensity, e.g., the photon density, of the switching signal, it is ensured that, as far as possible, only labeling molecules that are isolated with respect to the optical resolution are activated and subsequently emit fluorescence.For these isolated molecules, the center of the diffraction-induced intensity distribution, and thus the position of the labeling molecule, is then calculated with increased resolution. To image the entire sample, the isolation of the labeling molecules from the subset is repeated by introducing activation radiation, subsequent excitation, and fluorescence imaging until as many labeling molecules as possible have been included in a subset and isolated within the resolution of the image.

[0011] The PALM method has the advantage that neither activation nor excitation requires high spatial resolution. Instead, both activation and excitation can be performed using wide-field illumination.

[0012] As a result, the labeling molecules are statistically activated in subsets by appropriately selecting the intensity of the activation radiation. Therefore, generating a complete image of a sample, in which the positions of all labeling molecules can be computationally determined with a resolution exceeding, for example, the diffraction limit, requires the evaluation of a large number of individual images. This can involve up to 10,000 individual images. Consequently, large amounts of data are processed, and the measurement takes a correspondingly long time. Even acquiring a single complete image requires several minutes, which is essentially determined by the readout rate of the camera used. The position determination of the molecules in the individual images is performed using complex computational procedures, such as those described in Egner et al., Biophysical Journal, pp. 3285-3290, Vol. 93, November 2007.The processing of all individual images and their assembly into a high-resolution overall image, i.e., an image in which the locations of the labeling molecules are specified with a resolution beyond the diffraction limit, typically takes one to two hours. Problem description (state of the art) Literature: [1] Betzig et al., Science 313, 1642-1645 (2006) [2] Hess et al., PNAS 104, 17370-17375 (2007) [3] Hess et al., Biophys J. 91, 4258-4272 (2006) [4] Shroff et al., PNAS 104, 20308-20313 (2007) [5] Rust et al., Nat Methods 3, 793-796 (2006) [6] Egner et al., Biophys J. 93, 3285-3290 (2007) [7] Toprak et al., Nano Lett. 7, 2043-2045 (2007) [8] Juette et al., Nature Methods 5, 527-529 (2008) and WO 2009 / 146016 A1 [9] Huang et al., Science 319, 810-813 (2008)

[10] Holst / Lomheim, CMOS / CCD sensors and camera systems, SPIE Press (2007)

[11] Lessard et al., Appl. Phys. Lett. 91, 224106 (2007)

[0013] The basic procedures as described above are also described in detail in the literature in various variations [1-6].

[0014] The variants (PALM, STORM, D-STORM etc.) differ mainly in the choice of fluorophores and the type of optical switching process.

[0015] However, all methods have in common the localization of the molecules by imaging them onto a highly sensitive camera (e.g., EMCDD).

[0016] The quasi-point-shaped light source (molecule) to be detected is mapped onto several camera pixels by the point image blur function (PSF) of the microscope.

[0017] The exact position of the molecule in the x / y plane can now be determined either by fitting the known PSF (Gaussian mask) or by determining the center of mass or by a mixture of both (Gaussian mask).

[0018] Typical localization accuracies (depending on experimental conditions) are 5-30nm; this is also approximately the achievable lateral resolution of this method.

[0019] In practical terms, the requirement for molecules that are not too close together on the one hand, and for the most complete possible representation of the structures under investigation on the other, means that many individual images (typically 20000) of the sample must be taken.

[0020] In each image, the positions of the molecules active at that moment are determined and stored. In addition to the already considerable image acquisition time for 20,000 images, there is also the (depending on the algorithm and computer system used) significantly longer processing or evaluation time before the actual high-resolution image is available.

[0021] However, the localization-based high-resolution method described above is limited to surfaces or 2 dimensions, since the localization of individual dye molecules in the third spatial direction (z-direction) is much more complex.

[0022] Several approaches to this are known from the literature, which will be briefly explained below.

[0023] Astigmatism / cylindrical lens ([9]):

[0024] In this approach, a weak cylindrical lens is introduced into the detection beam path, resulting in an astigmatic PSF. Consequently, the image of the molecule is distorted elliptically if the molecule is located above or below the PSF's symmetry point. Information about the molecule's z-position can then be extracted from the orientation and magnitude of the distortion.

[0025] One problem with this method is that the local environment and the orientation of the molecular dipole can also lead to a distortion of the molecule's spot.

[0026] Depending on their orientation, these molecules would then be assigned an incorrect z-value.

[0027] Detection in two planes: Bewersdorf et al., Toprak et al.([7,8]) : Here, a 50 / 50 beam splitter is introduced into the detection beam path, splitting (duplicating) the image into two partial images. These two images are either projected onto two identical cameras or side-by-side onto a single camera chip. An optical path length difference is introduced into one of the two partial beam paths such that the two beam paths result in two object planes separated by approximately half to one z-PSF (700 nm) in the z-direction. The z-position for molecules lying between these two planes is then determined, for example, by subtracting the two partial images of the molecule or by fitting a three-dimensional PSF.

[0028] This method requires two highly sensitive cameras, or both images must be arranged side-by-side on a single camera chip. The latter naturally leads to a limitation of the image field. Both variants also require precise adjustment of the optical paths or calibration measurements to ensure subpixel-accurate overlap of the two partial images. Alternatively, the images must be superimposed computationally (using software). Problem solving:

[0029] The solutions according to the invention are the subject of the independent patent claims.

[0030] Preferred further training is listed in the sub-requirements.

[0031] The invention will now be explained in more detail, for example with reference to the accompanying drawings, which also reveal essential features of the invention. They show: Fig. 1 a schematic representation of an activated labeling molecule in a resolution-limited volume; Fig. 2 a schematic representation of the mapping of various activated and non-activated labeling molecules onto a spatially resolved detector, Fig. 3 a flowchart for image generation in the PALM process, Fig. 4 to the flowchart of the Fig. 3 accompanying explanatory diagrams of the detector of the Fig. 2 illustrated labeling molecules, Fig. 5 a schematic representation of a microscope for PAL microscopy, Fig. 6: An embodiment of the invention with a microlens array Fig. 7: A distribution of detection elements Fig. 8: aggregated “superpixels” for three-dimensional evaluation Fig. 9: A combination of microlenses and cylindrical lenses array. Fig. 10. A representation using microlenses in or near the intermediate image plane.

[0032] Fig. Figure 1 schematically shows a labeling molecule 1 that has been excited to fluoresce. Fluorescence detection naturally requires a large number of excitations, since each excitation yields exactly one fluorescence photon, and radiation detection requires the integration of many fluorescence photons. Due to physical principles, the fluorescence radiation emitted by the labeling molecule 1 can only be detected with limited optical resolution in a microscope. Even when the microscope reaches the diffraction limit of the optical resolution, the photons from the fluorescent labeling molecule 1 are still scattered by diffraction and thus detected in a diffraction disk 2. Therefore, the microscope essentially displays the fluorescence of the labeling molecule 1 instead of its geometric extent, which is shown in the image. Fig. 1. A larger object, schematically drawn as a black circle, is shown again, which is in Fig. 1 is illustrated by the diffraction disk 2. The size of the diffraction disk 2 depends on the quality of the microscopy setup used and is defined by the full width at half maximum (FWHM) of the point spread function of the optical image. Of course, it is not actually a two-dimensional object, but rather a diffraction volume into which the fluorescence photons enter. In the two-dimensional representation of the Fig. However, this appears as a disk. The term diffraction disk is therefore used here in a general way to refer to the maximum resolution volume that the optics used can achieve. The optics used do not necessarily have to operate at the diffraction limit, although this is preferable.

[0033] To more precisely locate the labeling molecule 1 within the diffraction disk 2, the PALM method, described above in general terms, is used. This method activates individual labeling molecules. In this description, "activation" is understood to mean the activation of specific luminescence properties of the labeling molecules, i.e., both the switching on of luminescence excitability and a change in the luminescence emission spectrum, which corresponds to the activation of specific luminescence properties. In the embodiment described here, activation is effected by optical activation radiation. However, other non-optical activation mechanisms are also possible.

[0034] The activation now occurs in such a way that there are at least some activated molecules whose center of gravity does not lie in the diffraction disk of other activated molecules, i.e., which can at least just be distinguished within the optical resolution.

[0035] Fig. Figure 2 schematically shows an exemplary situation on a detector 5 that integrates the photons with spatial resolution. As can be seen, there are regions 3 where the diffraction disks of neighboring labeling molecules overlap. Here, as in the left region 3 of the Fig. 2 is visible, but only those labeling molecules that were previously activated are relevant. Unactivated labeling molecules 1' do not emit the specific fluorescence radiation that is detected on the matrix detector 5, and therefore play no role.

[0036] In regions 4, e.g., region 4 located in the center of matrix detector 5, label molecules 1 are positioned such that their diffraction pattern 2 does not overlap with any diffraction pattern of another activated label molecule 1. The right-hand region of matrix detector 5 shows that regions 3, where diffraction patterns of activated label molecules overlap, can indeed be adjacent to regions 4, where this is not the case. Furthermore, the right-hand region 4 illustrates that the proximity of an activated label molecule 1 to a non-activated label molecule 1' is irrelevant for detection, since such a label molecule 1' does not emit the fluorescence radiation detected by matrix detector 5; i.e., it does not fluoresce.

[0037] To capture an image with a level of detail exceeding the optical resolution of the device, which in this context is a high-resolution image, the following are now used: Fig. 3 schematically illustrated steps were used.

[0038] In a first step, S1, a subset of the labeling molecules is activated by means of a switching signal; they are thus switched from a first state, in which they cannot be excited to emit the specific fluorescence, to a second state, in which they can be excited to emit the specific fluorescence. Of course, the activation signal can also cause selective deactivation, meaning that an inverse procedure can also be used in step S1. The essential point is that after step S1, only a subset of the labeling molecules can be excited to emit the specific fluorescence. The activation or deactivation (for the sake of simplicity, only the case of activation will be described below) depends on the labeling molecules used. For a dye such as, for example,In DRONPA, PA-GFP or reversibly switchable synthetic dyes (such as Alexa / cyan constructs), activation occurs through optical radiation; the switching signal is therefore switching radiation.

[0039] The under the Fig. 3 shown Fig. Figure 4 shows the state after step S1 in subfigure a. Only a subset of the labeling molecules l_n is activated. The labeling molecules in this subset are represented by a fully highlighted black dot. The remaining labeling molecules were not activated in this step. They are shown in subfigure a of the Fig. 4 is denoted by l_n+1.

[0040] In a second step (S2), activated labeling molecules can then be excited to emit fluorescent radiation. Fluorescent proteins known from the art, such as PA-GFP or DRONPA, are preferably used as fluorescent dyes. Activation of such molecules occurs with radiation in the range of 405 nm, excitation to fluorescence occurs at a wavelength of approximately 488 nm, and the resulting fluorescence lies in the range above 490 nm.

[0041] In a third step S3, the emitted fluorescence radiation is detected, for example by integrating the recorded fluorescence photons, so that the underlying partial image b is displayed. Fig. Four situations are shown on the matrix detector 5. As can be seen, the diffraction disks of the activated labeling molecules l_n do not overlap. The size of the diffraction disks is determined by the optical resolution of the image onto the matrix detector 5. Additionally, in sub-image b of the Fig. Four (theoretical) diffraction patterns of fluorescent molecules belonging to the non-activated group l_n+1 are shown. Since these non-activated labeling molecules do not emit fluorescence, no fluorescence radiation contained in their (theoretical) diffraction patterns interferes with the detection of the fluorescence radiation of the subset l_n of activated labeling molecules.

[0042] To minimize the overlap of diffraction disks in the subset I_n to the point where the labeling molecules are no longer distinguishable, the activation energy is set so that the subset l_n represents only a comparatively small proportion of the total number of labeling molecules, thus ensuring that a statistically large number of labeling molecules are distinguishable relative to the volume resolvable with the optical arrangement.

[0043] In a fourth step S4, the position of the fluorescent labeling molecules is determined computationally from the diffraction distribution of the fluorescent disks, thereby improving the resolution with which the position of the activated labeling molecules is known beyond the resolution of the optical arrangement, as shown in partial image c of the Fig. 4 shows.

[0044] Alternatively, instead of calculating the gain, it is fundamentally possible to amplify the recorded fluorescence radiation nonlinearly and thus sharpen the resolution beyond the optical arrangement with reduced effort. The nonlinear amplification can be calculated, for example, according to the function S = A · F N (Equation 1) or S = A • exp F / w (with w = 10 -N (Equation 2)) can be described, where F is the amplitude of the fluorescence signal, A is a normalization factor and N is an integer greater than 1.

[0045] A strong nonlinear dependence of the parameter S on F is particularly advantageous, i.e., high values ​​for N in equations 1 or 2. Of course, other functions can also be used. Generally, the nonlinearity is preferably chosen such that the half-width of the diffraction disk corresponds to a desired spatial resolution for the location of the labeling molecules. In addition to nonlinear amplification, nonlinear attenuation can also be used. Here, fluorescence signals of low amplitude or intensity are attenuated, whereas strong signals remain largely undamped. Naturally, a combination of nonlinear amplification and attenuation can also be used.

[0046] A fifth step, S5, assembles the labeling molecules, whose positions have been precisely determined, into a single image whose spatial resolution is increased beyond the optical resolution. However, it only contains information about the previously activated subset of labeling molecules.

[0047] In a sixth step, S6, the individual image is combined into a composite image using a known method. The process then returns to step S1, at which point the previously fluorescent molecules must be deactivated. Deactivation can be achieved, depending on the type of label molecule, by separate radiation or by the decay of the activation state. It is also possible to bleach previously imaged label molecules using excitation radiation.

[0048] With each iteration, another individual image is obtained, contributing to the overall picture. In the next iteration, a different subset of the labeling molecules is activated, e.g., those in Fig. 4 shows subset l_n+1.

[0049] Through multiple iterations of steps S1 to S6, the overall image is built from individual images of each iteration. These images specify the locations of the labeling molecules with a spatial resolution that is sharper than that of the optical imaging. A high-resolution overall image is thus successively built up through a corresponding number of iterations. In this method, the reduction of the diffraction disk preferably occurs in all three spatial dimensions when multiple image stacks, spaced apart in the z-direction, are acquired. The overall image then contains the location of the labeling molecules with high resolution in all three spatial directions.

[0050] Fig. Figure 5 schematically shows a microscope 6 for high-resolution imaging of a sample 7. The sample is labeled, for example, with the dye DRONPA (see WO 2007009812 A1). For activation and fluorescence excitation, the microscope 6 has a radiation source 8, which has individual lasers 9 and 10, whose beams are combined by a beam combiner 11. The lasers 9 and 10 can emit radiation, for example, at 405 nm (activation radiation) and 488 nm (fluorescence excitation and deactivation). Dyes are also known (e.g., the dye called DENDRA (see Gurskaya et al., Nature Biotech., Vol. 24, pp. 461–465, 2006)) in which activation and fluorescence excitation can occur at one and the same wavelength. In this case, a single laser is sufficient.

[0051] An acoustic-optical filter 12 serves for wavelength selection and for rapidly switching or attenuating individual laser wavelengths. An optical system 13 focuses the radiation via a dichroic beam splitter 14 into a pupil of a lens 15, so that the radiation from the radiation source 8 falls on the sample 7 as wide-field illumination.

[0052] Fluorescence radiation generated in sample 7 is collected via the objective lens 15. The dichroic beam splitter 14 is designed to allow the fluorescence radiation to pass through a filter 16 to a tube lens 17, so that the fluorescent sample 7 is imaged onto the detector 5.

[0053] A control unit is provided for controlling the operation of the microscope 6, here designed as a computer 18 with display 19 and keyboard 20. The process steps S2 to S6 are carried out in the computer 18. The frame rate of the matrix detector is crucial for the overall measurement time, so a matrix detector 5 with the highest possible frame rate is advantageous in order to reduce the measurement time. The core ideas of the present invention are explained in more detail below: 1. Camera sensor with at least 2 integrated object planes:

[0054] The core idea here lies in generating at least two object planes separated in the vertical (z) direction using only one camera and without altering the microscope beam path. This is achieved not by splitting the image and introducing an optical path length difference between the sub-images as in [8], but by nesting two sub-images on one camera sensor ( Fig. (right). In principle, this could be achieved by arranging the nested pixels at different heights on the sensor. However, this is difficult to implement, as the required splitting of the two object planes in z of -700 nm would require a pixel height difference of -700 nm*^(100)^2 = 7 mm with a typically used 100x lens.

[0055] Quite apart from the production difficulties, a typical lateral pixel size of only 8-16 µm would result in an extremely unfavorable ratio; without microlenses, a pixel on the same sensor would have to be 7 millimeters higher than its neighboring pixel, with a pixel size of only 8-16 micrometers. Even if manufacturing were possible, only the pixels at the top would "see" an image. Instead, the z-offset of the two object planes can be achieved by a microlens (ML) array on the sensor ( Fig. left) or alternatively in the imaging beam path .

[0056] The use of ML arrays on CCD or CMOS sensors to increase the fill factor is a common method (e.g.

[10] and references therein).

[0057] In the proposed arrangements, the z-off can be advantageously adjusted by selecting the focal length of the linear modules (MLs). The MLs are not assigned to every pixel, but rather alternately, so that two sub-images result in a single exposure, corresponding to the two object planes separated in the z-direction.

[0058] The idea is not only limited to the in Fig. The pixel arrangement shown is limited.

[0059] In another embodiment, for example, three object planes can be introduced using two different microlens groups. Depending on the data analysis, this results in three support points for data fitting in the z-direction, and thus potentially higher accuracy. A possible high-density packing arrangement with three associated object planes, based on the hexagonal super-CCD structure (Fuji), is shown in Fig. sketched.

[0060] Fig. shows the generation of two object planes separated in the z-direction using microlens arrays.

[0061] In Fig. The CCD sensor plane is shown schematically, with microlenses ML arranged in an array in alternating (two-dimensional) arrangements with gaps in front of the individual sensor elements.

[0062] The object planes OE1 and OE2 are imaged onto the CCD sensor via the lens OB and a tube lens TL.

[0063] Due to the ML array used to map object plane OE1, object planes OE1 and OE2 do not lie in the same plane but are shifted relative to each other in the Z-direction. In the representation on the right, the viewer sees different Z-planes corresponding to different focal lengths, characterized here by black and gray.

[0064] In Fig. The sensor pixels assigned to object level OE2 are shown in lighter shades of gray, and the sensor pixels assigned to object level OE1 are shown in darker shades of gray.

[0065] They are arranged symmetrically here, due to the symmetrical arrangement of the ML array; however, deviations from this symmetry are also within the scope of this invention.

[0066] The advantages of this approach are, in particular, that only one camera and one beam path are required.

[0067] The alignment of the two sub-images relative to each other is automatically achieved through a constant and known 1-pixel offset.

[0068] Any microscope or any other optical system could be directly and advantageously retrofitted to such a 3D high-resolution system without modification of the beam path and without additional optical components.

[0069] Fig. shows a possible pixel and ML arrangement for generating three object planes separated in the z-direction using a microlens array with two different focal lengths.

[0070] The image planes OE2 and OE3 are created by ML arrays with different focal lengths; OE1 is as in Fig. 1 an object plane that is arranged on a detector element without a microlens.

[0071] However, three ML arrays with different focal lengths can also be provided.

[0072] Contributions from other object layers are perceived by individual pixels only as a diffuse background.

[0073] The lens used (see PALM method and the cited literature) has a very short focal length (high NA), which means that objects located outside the focal length are detected as very blurred (diffuse).

[0074] The elements can be confocal (corresponding to one Airy unit), but are usually larger or the imaging is performed in this way. 2.) ML array as in 1), but in the intermediate image e.g. as shown in Fig. 10.

[0075] Although an additional optical element (the ML array) has to be introduced into the beam path (into an intermediate image ZB), the advantages listed above (only one camera and beam path, automatic alignment, retrofit capability) are retained.

[0076] However, the ML array could be advantageously configured and manufactured separately, thus avoiding dependence on the camera chip manufacturer. The intermediate image would be appropriately enlarged, thereby reducing the tolerances of the ML array.

[0077] The microlenses can be positioned slightly in front of the ZB plane (or behind it as shown with a negative focal length) so that their focal plane lies in the ZB, or the position of the sensor is adjusted accordingly.

[0078] When using multiple ML arrangements with, for example, 2 different focal lengths as shown above, the intermediate image can advantageously lie between the two focal planes of the ML arrangements.

[0079] The detected spot then appears as a spot that lies exactly in the intermediate image plane, equally blurred for both detected receiver arrangements that are assigned to the respective ZL arrangements.

[0080] Differences in the detected blur would advantageously allow for precise position detection of the particle based on the detected blur within the detected depth range without Z-adjustment of the arrangement (e.g., 700 nm). 3.) At least 2 object layers with adapted “superpixels” with substructure:

[0081] Such a substructure is suitable for 2 or 3 object levels, as shown in the examples in Fig. , but also as in Fig. . 6 can be used advantageously.

[0082] Especially for CMOS cameras with random access pixels, suitable superpixels can be defined that allow the direct processing of the position information of each superpixel. This approach would be advantageous for real-time localization per superpixel in 3D and thus, for example, suitable for tracking individual objects by means of probe methods and rules based on an individual superpixel tracking signal. In effect, a PSD (Position Sensitive Detector) is implemented for each superpixel. The superpixel arrangement in Fig. The right side would be directly suitable for particle tracking as described in

[11] .

[0083] In contrast to

[11] , a wide-field detection method (camera) could be directly linked to 3D particle tracking without additional optical elements. This approach is not limited to high-resolution microscopy with highly sensitive cameras, but can also be implemented with cost-effective cameras for sufficiently light-sensitive objects. An example of this would be tracking beads or vesicles with many fluorophores, or even the implementation of an optical tweezer integrated into the wide-field microscope.

[0084] For example, in CMOS cameras, each pixel can be addressed individually.

[0085] As shown, several pixels (for example 9) are considered as a "superpixel" and are jointly assigned to an evaluation unit (storage unit) for evaluation purposes - see black frame in the image.

[0086] Such a superpixel would, for example, capture a single pixel.

[0087] A large number of such superpixels would be located side by side on the sensor. Their size can be adapted to the expected size of the spots (molecules) to be examined, for example by appropriate optical imaging or by dimensioning on the receiver.

[0088] Its substructure allows lateral and axial position information of the image point (molecule) to be obtained through summation and difference calculations. Fig. shows in detail in a) Possible “superpixels” (black rectangles) SP, consisting of several individual pixels with object layers as encoded in the legend (+ 0 - ) corresponding to different shades of gray

[0089] The individual object layers can in turn be realized by ML arrays on the camera chip and / or in the intermediate image layer as described in 6) and 7).

[0090] In Fig. Each pixel corresponding to a different object plane is connected to logic stages that either subtract the signals from, for example, two pixels receiving signals from the same object plane (C1, X, Y direction) or add them (C2 (+) and then subtract the added signals from each other (Z direction).

[0091] As in Fig. Figure 7 represents the different grayscale values ​​from different object layers. For example, if a dashed molecule M in Figure 8c moves to the left, the detected values ​​of the symmetrically arranged detector elements of the superpixel (e.g., the light gray to the right and left of the center pixel Pz) are unequal. By calculating the difference and drawing the corresponding values, the direction of movement can be determined, and the magnitude of the movement can be calculated from the absolute value of the inequality.

[0092] This can be achieved both through the pixels arranged horizontally (X) in the drawing and through the pixels arranged vertically (y), so that complete two-dimensional motion information is obtained.

[0093] In the Z direction, the intensity of the upper and lower focal planes of the ML, for example with 3 different focal planes, represented by different sensor elements as shown by the gray levels, is compared analogously, and in this way a movement is detected or the position of the investigated spot within the detected depth range is determined by comparing the detected intensity.

[0094] Figure 8d) shows the respective plus / minus + / - combination of pixels P1, P2 which capture different object levels and the subsequent subtraction to obtain the Z information regarding its magnitude and sign.

[0095] Figure 8c shows on the right a possible alternative wiring configuration of the detector elements of a superpixel, which is only partially symmetrical or has a different axis of symmetry. The above remarks regarding the determination of position and motion also apply here, analogously.

[0096] Each pixel / superpixel of the presented arrangements can function as a position-sensitive detector (comparable to a quadrant photodiode), as described above. This allows for the realization of optical tweezers using only the presented camera and high-resolution microscope. The fundamentals of optical tweezers can be found, for example, at: http: / / en.wikipedia.org / wiki / Optical-tweezers or in the references cited there. The actual tweezers, i.e., the focused holding laser, can be implemented using a laser integrated into the LSM (laser scanning module). This focused spot, which forms the actual optical trap, can be positioned arbitrarily via the LSM scanner. The fast scan-and-switch functionality (AOTF) also allows for the generation and positioning of multiple spots.

[0097] When a particle (vesicle, bead, depending on the experiment) is 'trapped', the superpixel located at the site of its image can be 'activated' as a PSD. If forces now act on the particle and it is slightly deflected from its equilibrium position, this movement can be detected.

[0098] For tracking: In conjunction with a 3D piezo sample stage, diffusion and transport processes of individual objects can also be tracked with one of the superpixels: again, the superpixel located at the particle's position in the image is activated, and the position signal is then kept at "zero" in all directions (x, y, z) via a feedback loop. The piezo control voltage required for this is directly proportional to the change in position of the tracked particle.

[0099] Advantages include in particular: - Combination of LSM / fluorescence microscopy / high-resolution microscopy / particle tracking / optical tweezer in one system. These methods are all of interest in molecular cell biology, but currently require their own dedicated systems. This also makes it cost-effective, as the expensive components (laser, camera) are only needed once. 4. Combination of microlenses and cylindrical lenses to obtain X / Y / Z information:

[0100] Fig. This demonstrates z-localization achieved by combining a microlens array ML and a microcylindrical lens array ZL with a suitable superpixel structure. The xy-localization can be obtained, as in PALM methods, from the intensity information of the central pixel or from the sum of the signals of all nine pixels of each superpixel.

[0101] Here, however, the z-information is obtained similarly to [9] – see section on the prior art – by introducing an astigmatic focus. Here too, in addition to a microlens array ML as described above, a cylindrical microlens array ZL is introduced such that the symmetrical circle of confusion K defines the central pixel of the array as described above. Fig. The sketched superpixel is just filled. This allows x / y localization for objects (molecules) in focus to be performed using only the central pixels without signal loss. For a point source located above or below the object plane associated with this circle of confusion, a sagittal or meridional ellipsoid E is then obtained, respectively.

[0102] The advantage of this arrangement is that the z-information of the observed point source is directly derived from the edge pixel signals of each superpixel. In particular, modern CMOS camera chip architecture, with its directly addressable and readable pixels (random access), would allow for direct processing of the intensity, and thus the z-information, of each superpixel without software fitting. The xy-localization can be obtained, as with PALM methods, from the intensity information of the central pixel or from the sum of the signals of all nine pixels of each superpixel.

[0103] The combination of cylindrical lens and microlenses essentially represents an anamorphic optic that is inserted into the converging beam path after the tube lens.

[0104] In front of the focal plane of the arrangement, the focusing effect of the microlens ML and the one-axis limited effect of the cylindrical lens ZJ each work together and produce a point-like focus K in the focal plane (left image, small circle) and in front of the focal plane (middle image, black ellipse) and after the focal plane (right image, black ellipse) an ellipsoid E which has a different (sagittal and meridional) orientation for a point source that lies above or below the object plane.

[0105] The appearance of the ellipsoid causes a signal to be detected in the elements adjacent to a central detector element, whereby the orientation of the ellipsoid (detection by vertical or horizontal neighboring elements in the example, as well as in Fig.8 described) and the detector elements (vertical or horizontal) that produce a signal characterize whether the object is located above or below the object plane.

[0106] In contrast to the invention, in printed document 9, the recorded image is evaluated by pixel-by-pixel image reading and software determination of the smearing.

[0107] Disadvantages include, in particular, - Transfer and evaluation of large data sets (a fundamental problem with Palm). The solution presented here offers the possibility of implementing position calculation directly on the camera (in an FPGA) and transferring only the already evaluated data. - The applications described under 3) such as Tweezer and / or tracking are not possible with this 'classic' approach.

Claims

[1] High-resolution microscopy method for the three-dimensional position determination of objects, in particular individual fluorophores, preferably for spatially high-resolution luminescence microscopy of a sample (7) which is labelled with labeling molecules (1) which can be activated or switched with a signal such that they can only be excited to emit certain luminescence radiation in the activated state, wherein the method comprises the following steps: a) Introducing the signal onto the sample (7) such that only a subset of the labeling molecules (1) present in the sample (7) are activated, wherein the sample (7) contains subregions in which activated labeling molecules (1) are at least as far apart from the nearest neighboring activated labeling molecules (1) as is greater than or equal to a length which results from a predetermined optical resolution, b) Excitation of the activated molecules (1) to emit luminescence radiation, c) Detection of the luminescence radiation with the predetermined optical resolution, and d) Generating a single image from the luminescence radiation recorded in step c), wherein the geometric locations of the luminescence-emitting labeling molecules are determined with a spatial resolution increased above the predetermined optical resolution, where the steps are repeated several times and the resulting individual images are combined to form a complete image, characterized by , that an object is imaged via an imaging system (OB, TL) with a microscope objective (15, OB) onto an area detector (5, CCD) consisting of individual detector elements, wherein at least one microlens arrangement (ML, ZL), which is partially located in front of the detector elements, images a different object plane (OE1) onto the detector elements in the direction of light behind the microlenses than onto detector elements in front of which there are no microlenses. [2] High-resolution microscopy method according to claim 1, characterized by a symmetrical distribution of microlenses (ML). [3] High-resolution microscopy method according to claim 1 or 2, wherein the microlenses (ML) are arranged in front of the detector (5, CCD) and / or in or near an intermediate image plane (ZB). [4] High-resolution microscopy method for the three-dimensional position determination of objects, in particular individual fluorophores, preferably for spatially high-resolution luminescence microscopy of a sample (7) which is labelled with labeling molecules (1) which can be activated or switched with a signal such that they can only be excited to emit certain luminescence radiation in the activated state, wherein the method comprises the following steps: a) Introducing the signal onto the sample (7) such that only a subset of the labeling molecules (1) present in the sample (7) are activated, wherein the sample (7) contains subregions in which activated labeling molecules (1) are at least as far apart from the nearest neighboring activated labeling molecules (1) as is greater than or equal to a length which results from a predetermined optical resolution, b) Excitation of the activated molecules to emit luminescence radiation, c) Detection of the luminescence radiation with the predetermined optical resolution, and d) Generating a single image from the luminescence radiation recorded in step c), wherein the geometric locations of the luminescence-emitting labeling molecules are determined with a spatial resolution increased above the predetermined optical resolution, where the steps are repeated several times and the resulting individual images are combined to form a complete image, characterized by , that An object is imaged via an imaging system (OB, TL) with a microscope objective (OB) onto a planar detector (5, CCD) consisting of individual detector elements, wherein at least one microlens arrangement (ML, ZL) is located in front of the detector elements and different, preferably adjacent, detector elements receive light from microlenses (ML) with different focal lengths and from different object planes (OE1, OE2). [5] High-resolution microscopy method according to claim 4, wherein the microlenses (ML) are arranged in front of the detector (5, CCD) and / or in or near an intermediate image plane (ZB). [6] Microscopic method according to one of the preceding claims, wherein several detector elements of an area detector (5, CCD) grouped around a central detector element are evaluated together in such a way that two- or three-dimensional position detection and / or object detection is carried out by computationally combining the signals of the detectors grouped around a central detector element and generating location and / or motion information from the computation result and / or its sign. [7] Method according to any of the preceding claims, characterized by its use for tracking one or more particle trajectories. [8] Method according to any of the preceding claims, characterized by its use in optical tweezers. [9] High-resolution microscope (6) for three-dimensional position determination of objects, in particular individual fluorophores, preferably for spatially high-resolution luminescence microscopy of a sample (7) which is labelled with labeling molecules (1) which can be activated or switched with a signal such that they can only be excited to emit certain luminescence radiation in the activated state, wherein an object is imaged via an imaging system (OB, TL) with a microscope objective (OB) onto an area detector (5, CCD) consisting of individual detector elements, characterized by at least one microlens arrangement (ML) that is located at least partially in front of the detector elements, such that a different object plane (OE1) is imaged onto the detector elements in the direction of light behind the microlenses (ML) than onto detector elements in front of which there are no microlenses (ML) or microlenses (ML) with a different focal length. [10] High-resolution microscope (6) according to claim 9, characterized by a symmetrical distribution of microlenses (ML). [11] High-resolution microscope (6) according to claim 9 or 10, wherein the microlenses (ML) are arranged in front of the detector (5, CCD) and / or in or near an intermediate image plane (ZB). [12] Use of a microscope (6) according to any one of claims 9 to 11 in an optical tweezer.

Images