Method and apparatus for accelerated three-dimensional microscopy with structured illumination

Abstract

The present invention relates to a method and apparatus for accelerating three-dimensional microscopy with structured illumination. In three-dimensional structured illumination microscopy (SIM), the focal plane of a sample is focused and each focal plane is sequentially illuminated with multiple phases with structured illumination light, and the sample light emitted by the sample is recorded in a corresponding individual image. A result image with improved resolution relative to the individual image is reconstructed from the individual images to produce a super-resolution image stack. The method is time-intensive and stresses the sample. By reconstructing a result image from individual images of two different focal planes by an approximate method, the result image represents a sample plane located between the focal planes, an image stack can be produced in a short time with less stress on the sample. The present invention also relates to fluorescence microscopy.

Description

Technical Field

[0001] The invention relates to a method and apparatus for imaging a sample using a microscope, wherein the microscope is used to focus a focal plane of the sample and sequentially illuminate the focal plane in multiple phases with structured illumination light, and a detector is used to record the sample light emitted and / or scattered by the sample in each phase in a corresponding (structured) individual image, so that a resulting image with enhanced (relative to the individual images) resolution can be reconstructed from the individual images. Structuring in this case means that at least one component of the illumination light in the sample has a constant spatial frequency. Background Art

[0002] The resolution of a microscope depends on the aperture of the microscope objective and the wavelength of the light due to diffraction of the light received by the sample in the microscope objective. Since the available wavelength range of visible light is limited, the resolution of a microscope is in principle limited (Abbe 1873). This means that the support of the optical transfer function to the microscope in frequency space is limited to a finite element region around the origin of the coordinates, relative to the spatial frequencies of the sample to be imaged. Therefore, the microscope can only image spatial frequencies that are located in the central interval where the support does not vanish.

[0003] If the excitation intensity of the illumination and the emission intensity of the sample have a linear relationship, the resolution can be improved approximately by a factor of two laterally by structured illumination of the sample (SIM—“structured illumination microscopy”). SIM is disclosed, for example, in DE 19908 883 A1 and in the paper “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy” by M. Gustafsson (Journal of Microscopy, Vol. 198, 2000, p. 82). This is based on the generation of a spatial light structure on the sample to be examined, for example by sinusoidal interference of the illumination light downstream of the grating. Due to the convolution of the sample response in real space with the point spread function (PSF) of the microscope objective, regions of the spatial frequency of the sample structure that are outside the branches of the OTF in the frequency domain are shifted into the central branch interval, where they are superimposed with the original spatial frequency intensities. The light structure is generated sequentially in a plurality of different phase positions, and a separate image is recorded in each phase position in each case.

[0004] According to the SIM method, a consistent result image can be reconstructed from such individual images by solving a set of equations describing the interaction, the individual image comprising a superposition of the shifted spatial frequency and the original spatial frequency, the result image comprising both the original spatial frequency of the support interval and the original higher spatial frequency that has been temporarily shifted into the support interval by structured illumination. Therefore, the result image has better lateral resolution than a conventional single recording with uniform illumination. This resolution is called super-resolution because it is finer than diffraction-limited resolution. However, multiple recordings with different phase positions and structured orientation require high stability of the optical arrangement and the sample throughout the duration of the measurement. Additionally, the number of individual images required reduces the effective frame rate.

[0005] In addition to improving the lateral resolution (lateral to the optical axis of the detection), SIM can also be used to generate optical sections through the sample with improved axial resolution (along the optical axis of the detection) from a collection of individual images of the focal plane (known as OS-SIM). The resulting image is then a quasi-confocal image, and the axial resolution also corresponds approximately to the axial resolution of a confocal microscope. If N focal planes are recorded in each case of a collection of images with structured illumination, a z-stack of quasi-confocal resulting images can thereby be reconstructed.

[0006] However, SIM can therefore be extended to multiple focal planes so that super-resolution can also be achieved in the axial direction (this is known as 3D-SIM). For this purpose, the illumination in the sample must also be structured axially and for each of the N focal planes, a dedicated set of individual images from the relevant focal plane is recorded. Based on the system of equations that describes the interactions in all N focal planes, a z-stack can be calculated from the N axial (especially also lateral) super-resolved resulting images. In order to obtain a complete z-stack, the focal planes must be arranged at a distance from each other that is at most half of the minimum distance that the microscope can optically resolve axially (that is, according to an axial resolution that is as fine as twice the optical resolution) due to the Nyquist-Shannon sampling theorem, that is, for example, approximately 110 nm for a high numerical aperture objective with NA=1.4.

[0007] Improvements in resolution over SIM can be obtained by means of a sample that is excited (using illumination or otherwise) in such a way that a nonlinear relationship is created between the intensity of the excitation and the intensity of the light emitted by the sample ("saturation mode excitation microscopy"; SPEM). SPEM is disclosed, for example, in DE 199 08 883 A1. In the case of fluorescence microscopy, nonlinear excitation is accomplished, for example, by high illumination intensities that partially saturate the excitation of the fluorescent dye in the region of the illuminated structure. As a result, even higher spatial frequencies of the object structure than in the case of SIM are shifted into the interval of the support of the OTF. These higher frequencies can also be reconstructed by taking into account nonlinear interactions in the system of equations to be solved. However, in the case of SPEM, smaller phase steps and therefore even more individual images are required than in the case of SIM. SPEM can also be performed in three dimensions. Summary of the invention

[0008] The invention is based on the object of improving a method and a microscope of the type mentioned in the introduction such that the measurement duration can be shortened and the sample load can be reduced.

[0009] This object is achieved by a method having the characteristics specified below, and by a microscope having the characteristics specified below.

[0010] Advantageous developments of the invention are also specified below.

[0011] The microscope according to the invention is arranged for focusing a second focal plane different from the first focal plane and for recording individual images with different illumination phases for said second focal plane and for reconstructing a result image from the individual images of the two focal planes, the result image representing a sample plane located between said focal planes. However, no individual image of this additional sample plane is recorded (the additional sample plane is not focused), or in any event, the individual image of the additional sample plane during which it was focused is not used in the reconstructed result image.

[0012] For example, different focal planes can be focused by moving the sample in the z direction, by moving the objective in the z direction or by adjusting the internal focus of the objective.

[0013] The principle of the invention is that, based on a (specified, measured, calculated or estimated) point spread function, information about other planes than the focal plane (that is to say the intensities from these other planes) can also be determined from a collection of individual images that have been recorded from a single focal plane under structured illumination, because the microscope also captures other sample planes than the immediate focal plane due to its limited depth of field. These other sample planes may not be imaged sharply, but this is exactly where the available depth information block is located. The depth information block is used in OS-SIM to distinguish out-of-focus light. However, the intensity detected from outside the focal plane is lost and is not used.

[0014] Alternatively, it can be advantageously used to compute the image of the out-of-focus sample plane. In the case of a typical PSF that is mirror-symmetric with respect to the focal plane (and typically rotationally symmetric about the detection optical axis), all that is needed is to resolve the ambiguity in the depth information due to the mirror symmetry. This is done by using the depth information from two different focal planes to reconstruct the sample plane located between them.

[0015] According to the invention, it has been found that the system of equations known from 3D-SIM (or 3D-SPEM) can be used to reconstruct a resulting image from such additional sample planes, although no separate images were recorded for said additional sample planes.

[0016] For this purpose, the number of focal planes necessary for recording a z-stack can advantageously be reduced by selecting a distance between the focal planes which is greater than half the smallest axial distance optically resolvable by the microscope (that is to say corresponds to an axial resolution which is less than twice as fine as the optical resolution) and thereby formally violates the Nyquist-Shannon sampling theorem in the first place. By incorporating additional (unfocused) sample planes into the approximate evaluation, the Nyquist-Shannon sampling theorem is effectively satisfied. The additional sample planes can be considered as "explicit" focal planes and are described as focal planes in the SIM equation system. Due to the invention, the recorded focal planes can, for example, be located at an axial distance of approximately 330 nm from one another, whereas the complete resulting image of the plane can be calculated with an axial distance of only approximately 110 nm. Thus, for each of the additional sample planes which have been included only by the calculation (explicit focal planes), in each case where the "real" focal plane can be omitted, the recording of individual images in a correspondingly large number of illumination phases is omitted. In this way, the measurement duration is significantly shortened and in the same way the stress on the sample is reduced, since this requires a lower frequency illumination and thus an overall shorter time.

[0017] Preferably, the two focal planes (between which the additional sample plane is situated) are at most spaced apart by a distance such that the depth regions determined by the depth of field of the microscope and already recorded in the individual images overlap one another along the optical axis. In other words, the two focal planes (between which the additional sample plane is situated) are preferably spaced apart by a distance which is smaller than the smallest axial distance optically resolvable by the microscope (that is to say corresponds to an axial resolution finer than the optical resolution), and preferably also larger than half the smallest optically resolvable distance, as described above. These distance conditions can also be approximately represented by the size of the PSF: the distance between the focal planes (between which the additional sample plane is situated) is preferably selected such that it is larger than one half-maximum width (FWHM) of the PSF in the axial direction. The distance between the focal planes (between which the additional sample plane is situated) is preferably selected such that it is smaller than two half-maximum widths of the PSF in the axial direction.

[0018] Preferably, at least one solution of a system of equations is approximately determined, which describes the interaction of the light structure with the optical properties of the sample and with the point spread function of the microscope in the focal plane and in the sample plane located between the focal planes. In particular, the following system of equations or a mathematically equivalent system of equations can be used:

[0019]

[0020] Particularly advantageous are embodiments in which, for the approximate solution, linear regression is used, in particular regularization, in particular a truncated singular value decomposition, in which only a real subset of all singular values ​​is determined.

[0021] The accuracy of the approximation can be improved by optically oversampling the individual images laterally, in particular recording them with a resolution as fine as twice the optical resolution of the optical system of the microscope. To this end, the following system of equations or a mathematically equivalent system of equations can be used:

[0022] Preferably, structured illumination is implemented such that the light structure generated in the sample is regularly structured, in particular with lateral and / or axial periodicity, wherein the light structure is in particular arranged at different positions and / or in different orientations in each of the illumination phases.

[0023] In a particularly advantageous embodiment, the resulting image representing the additional sample planes lying between the focal planes can be reconstructed with an increased lateral and / or axial resolution compared to the individual images of the focal planes. However, it is not necessary to achieve super-resolution. For example, it is also possible to create only optical sections of the focal planes and of the additional sample planes with quasi-confocal resolution (as in OS-SIM).

[0024] This is equivalent to solving a system of equations to deconvolute the individual images approximately based on a model of the point spread function of the microscope after separating the illumination phase by calculation.

[0025] The reconstruction of the additional planes that are not in focus is correspondingly done by means of the fact that in the deconvolution, (intensity) parts of one of the individual images, in particular (intensity) parts of exactly one pixel, are assigned to different result images. Preferably, in the deconvolution, (intensity) parts of one of the individual images, in particular (intensity) parts of exactly one pixel, are partly assigned to at least one of the result images representing one of the focal planes and partly assigned to the result image representing the sample plane situated between the focal planes. In particular, the assignment to the result image representing the sample plane situated between the focal planes can be done based on the intensities of the individual images of at least two closest focal planes.

[0026] Advantageously, a two-dimensional spatially resolving detector is used to record the individual images, wherein in particular no aperture is arranged in front of the detector to optically intercept the sample. Thus, a rapid recording of the individual images is achieved. Alternatively, a confocal scanning detection can be performed.

[0027] Particularly preferred are embodiments in which the point spread function of the microscope is not manipulated to produce axial asymmetry. This keeps the outlay on the device very low. In particular, complex and expensive phase masks can be omitted with this method.

[0028] Also particularly preferred is an embodiment in which only exactly one focal plane is focused during the recording of one of the individual images. In this way, complex optical systems for simultaneously imaging multiple focal planes can be omitted.

[0029] The invention also comprises a microscope having a control unit configured to carry out the method as described above. In addition, the microscope has a light source, means for focusing different focal planes in the sample (e.g. an adjustable objective in the detection beam path, an adjustable sample stage or a wavefront manipulator), a two-dimensional spatially resolved detector for recording individual images from the instantaneous focal plane, and means for generating structured illumination light in the sample, such as a diffraction grating, a spatial light modulator (SLM), a micromirror array (DMD) or a light guide, which provides a pattern of interference-forming light structures in the sample in the back focal plane of the microscope objective. The means for generating structured illumination light can also be implemented in the light source, for example a light source in the form of an array of light emitting diodes.

[0030] To the extent that method steps are described, a control unit of the microscope can be configured to perform these method steps.

[0031] The present invention can be implemented with fluorescent samples or non-fluorescent samples. In the case of fluorescent samples, the light source advantageously excites the fluorescent dye provided in the sample. The light source preferably comprises one or more lasers, in particular an adjustable intensity modulator, so that the variation of the excitation intensity can be relatively wide.

[0032] The present invention is explained in more detail below based on exemplary embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In the attached figure:

[0034] Figure 1 shows a microscope,

[0035] Figure 2 shows the principle of structured illumination in multiple phases,

[0036] Figure 3 shows the stack of focal planes to be recorded and the additional sample plane,

[0037] Figure 4 shows illumination with three interference levels, and

[0038] Figure 5 An illumination with five interference levels is shown.

[0039] In all the figures, corresponding parts have the same reference numerals. DETAILED DESCRIPTION

[0040] Figure 1 A microscope 1 is shown which is able to carry out both classical microscopy methods, that is to say microscopy methods with a resolution limited by diffraction, and super-resolution microscopy methods, that is to say microscopy methods with a resolution beyond the diffraction limit.

[0041] The microscope 1 captures a sample 2. In addition, it has an objective 3 through which radiation passes for all microscopy methods.

[0042] The objective 3 images the sample via a beam splitter 4 together with a tube lens 5 onto a CCD detector 6, which in the example is a generally possible area detector. In this regard, the microscope 1 has a conventional light microscope module 7, and the beam path from the sample 2 through the objective 3 and the tube lens 5 to the CCD detector 6 corresponds to a conventional widefield detection beam path 8. Figure 1 As indicated by the double-headed arrow in , the beam splitter 4 is exchangeable so as to be able to switch between beam splitters with different dichroic properties or an achromatic beam splitter, as in US 2008 / 0088920.

[0043] Furthermore, connected in the beam path to the objective 3 is a laser scanning module 9, whose LSM illumination and detection beam paths are coupled into the beam path to the objective 3 via a switching mirror 11, which likewise has a beam splitter function. The beam path from the switching mirror 11 through the beam splitter 4 to the objective 3 is therefore a beam path combining the illumination beam path and the detection beam path. This is true both with respect to the laser scanning module 9 and with respect to the widefield detection beam path 8, since, as will be explained below, the illumination radiation, which together with the widefield detection beam path 8 (i.e. the CCD detector 6) implements the microscopy method, is also coupled at the switching mirror 11.

[0044] The switching mirror 11 and the beam splitter 4 are combined to form a beam splitter module 12, so there is the possibility of replacing the switching mirror 11 and the beam splitter 4 depending on the application. This is also illustrated by the double-headed arrow. Also provided in the beam splitter module 12 is an emission filter 13 located in the wide-field detection beam path 8 and which appropriately filters the spectral components that can propagate through the wide-field detection beam path 8. The emission filter 13 in the beam splitter module 12 is of course likewise replaceable.

[0045] The laser scanning module 9 receives the laser radiation required for operation from the laser module 15 via the optical fiber 14 .

[0046] exist Figure 1 In the configuration illustrated in , a focused illumination beam path 16 is coupled in at the beam splitter module 12, more specifically at the switching mirror 14, through which the illumination radiation of various microscopy methods passes. Different illumination beam paths of separate illumination modules are coupled into the focused illumination beam path 16. For example, a widefield illumination module 17 couples the widefield illumination radiation into the focused illumination beam path 16 via a switching mirror 18, so that the sample 2 is illuminated in a widefield via a tube lens 27 and an objective lens 3. For example, the widefield illumination module 17 can have an HBO lamp. As provided by other illumination modules is a TIRF illumination module 19, which realizes TIRF illumination by appropriate positioning of the switching mirror 18. For this purpose, the TIRF illumination module 19 receives radiation from the laser module 15 via an optical fiber 20. The TIRF illumination module 19 has a longitudinally movable mirror 21. Due to the longitudinal displacement, the illumination beam emitted by the TIRF illumination module 19 is displaced perpendicularly to the main propagation direction of the emitted illumination beam, so that the TIRF illumination is incident at the objective lens 3 at an angle that is adjustable relative to the optical axis of the objective lens. In this way, the required angle for total internal reflection at the cover glass can be ensured. Of course, other components are also suitable for realizing this angle adjustment.

[0047] Further coupled to the focused illumination beam path is the illumination beam path of a manipulator module 22, which likewise receives radiation from the laser module 15 via optical fibers (not otherwise specified here) and directs a point-shaped or line-shaped beam distribution in a scanning manner over the sample 2. The manipulator module 22 thus essentially corresponds to the illumination module of a laser scanning microscope and can therefore also be operated in combination with the detector of the laser scanning module 9 or with the widefield detection of the CCD detector 6.

[0048] In the focused illumination beam path 16, a grating 23 is also provided, which has a grating constant below a cut-off frequency that can be transferred into the sample 2 with the microscope 1. The grating 23 can be arranged, for example, in the plane of the illumination beam path 16 imaged onto the sample (intermediate image of the sample). The grating 23 is movable transversely to the optical axis of the focused illumination beam path 16. For this purpose, a corresponding displacement drive 24 is provided.

[0049] An image field rotator 25, which is rotated by a rotator drive 26, is also arranged downstream of the grating in the illumination direction in the focused illumination beam path 16. The image field rotator may be, for example, an Abbe-Koenig prism.

[0050] The modules and drivers as well as the detectors of the microscope 1 are all connected via lines (not otherwise specified here) to a control unit 28. The connection can be realized, for example, via a data and control bus. The control unit 28 operates the microscope 1 in different operating modes.

[0051] The control unit 28 thus allows the microscope 1 to perform classical microscopy, that is to say wide-field microscopy (WF), laser scanning microscopy (LSM) and fluorescence microscopy with total internal reflection (TIRF).

[0052] Figure 1 The microscope of the embodiment of the present invention essentially has two modules suitable for laser scanner illumination, in particular a laser scanning module 9 and a manipulator module 22. Other combinations are of course possible. The modules are coupled to the sample 2 via the objective lens 3 via a tube lens. The manipulator module 22 only includes the excitation part of the laser scanning module, but not the detection part. Therefore, the sample can be illuminated in a point-like manner, and the illumination spot can be scanned on the sample 2.

[0053] Preferably, a switching unit (e.g. a switching lens or a cylindrical lens) is also located in the manipulator module 22, by which the switching between point illumination and line illumination is achieved. The line illumination is advantageous when the grating 23 is pivoted into and perpendicular to the line of the line illumination. Alternatively, the line illumination can be used to dynamically (sequentially) generate structured illumination in the sample 2.

[0054] A variably adjustable strip modulator or DMD or SLM can also be used as an alternative to the grating 23 to generate structured illumination in the sample 2. In this case, the ability of the grating 23 to pivot in and out and the displacement drive 24 are of course no longer required.

[0055] The image field rotator 25 allows the structured illumination generated by the grating 23 (or by an element replacing the grating) to be rotated about the optical axis of the focused illumination beam path 16 so that the structured illumination is located in the sample 2 at different angles.

[0056] In order to switch between the various operating types, the switching mirrors 18 and 11 and the beam splitter 4 are appropriately adjusted. In an implementation, folding or tilting mirrors can be used for this purpose so that switching between the operating types can be achieved sequentially. Alternatively, dichroic mirrors that allow simultaneous operation of the various modules are also possible.

[0057] The beam splitter 4 is preferably embodied as a dichroic beam splitter, wherein the spectral properties are adjustable so that the spectral components from the fluorescence emission of the marker molecules to be detected by means of the CCD detector 6 pass into the widefield detection beam path 8 and the remaining spectral components are transmitted if possible. In order to increase the flexibility with respect to the use of marker molecules with different emission properties, a plurality of different beam splitters 4 and emission filters 13 are arranged in the beam splitter module 12 in such a way that they are exchangeable, for example on a filter wheel.

[0058] The microscope as described above can be used to produce a super-resolution resulting image. To this end, the control unit 28 has a suitable configuration, for example by suitable programming.

[0059] Figure 2 Schematic diagram of the concept for producing super-resolution images with a single focal plane according to the SIM method. Figure 1 The sample under the microscope 1 is imaged repeatedly, wherein different illumination states are set.

[0060] Figure 2 A collection 30 of individual images 40 from a single focal plane is shown, the difference between these individual images 40 being the structured illumination light 41 they contain, which is applied to the sample using structured illumination via the illumination beam path 16. As can be seen, the transverse, e.g. periodic structured illumination light 41 produced using the grating 23 is different in the different images 40. Overall, the example includes nine individual images 40, which are composed of three different orientations of the structured illumination light 41 and three different displacement positions of the structured illumination light 41. The various orientations and displacement positions are grouped together under the phase term. More different phases are of course also possible, as known from the paper cited above on the principle of SIM.

[0061] However, the structured illumination light 41 shown is to be understood as being only an example. In particular, it is not necessarily a line structure. The schematically depicted line can also be further structured along the line. In addition to the structuring of the line type used in the initially cited SIM paper, it is equivalent to use a scanning confocal point or line illumination with confocal detection, as known from the paper "Image Scanning Microscope" published by C. Müller and J. Enderlein (Physical Review Letters, 104, 1989101, (2010). This principle is also called ISM. In this case, of course, there are not nine oriented structured illuminations but a suitable number of individual images obtained by scanning the sample. Each individual image 40 then corresponds to a specific scanning position, that is to say a specific scanning state during the scanning image.

[0062] The control unit 28 calculates a super-resolution result image 50 from the recorded individual images 40 in the manner described below.

[0063] Due to the interaction with sample 2, the illumination pattern I(x,y,z) is multiplied by the optical properties S(x,y,z) of sample 2 and mixed with the detection PSFH(x,y,z):

[0064] I em (x,y,z,z0)=∫dx′dy′dz′I(x′,y′,z′)S(x′,y′,z0-z′)H(xx′,yy′,z+z′) (1)

[0065] Where z0 indicates the displacement of the sample relative to the Talbot pattern of the structured illumination and relative to the instantaneous focal plane, and z indicates the position of the instantaneous focal plane. By setting z = 0:

[0066] I em (x,y,z0)=∫dx′dy′dz′I(x′,y′,z′)S(x′,y′,z0-z′)H(xx′,yy′,z′) (2)

[0067] The Fourier transform with respect to x, y, and z gives:

[0068]

[0069] Discrete equidistant sampling in the z direction can be represented by a Dirac comb function ( https: / / en.wikipedia.org / wiki / Dirac_comb ):

[0070]

[0071] Δz denotes the sampling interval in the z direction. The summation index k in the specific calculation formula extends over all axial sample planes of the result images to be determined, that is to say over a finite integer range instead of the infinite series given here for the sake of completeness.

[0072] The Fourier transform of equation (4) with respect to x, y and z yields:

[0073]

[0074] The Fourier transform of a Dirac comb of period Δz is again a Dirac comb, but with period 1 / Δz. However, it is necessary not to associate the summation index k with the component k of the wave vector in the frequency domain. x ,k y ,k z Confusion.

[0075] Inserting equation (3) into equation (5) yields the overall valid system of equations by which any Fourier transformed illumination pattern I can be unmixed f (k′ x ,k′ y ,k′ z ):

[0076]

[0077] As mentioned above, in specific calculations the infinite series should be replaced by a finite sum over the sample plane to be calculated.

[0078] The system of equations can be solved based on: Based on the illumination pattern I(x, y, z) specifically used in different illumination phases (more specifically based on its Fourier transform I f (k x ,k y ,k z ), the detection PSF H(x,y,z) of the microscope (more specifically based on its Fourier transform H f (x,y,z) - that is, OTF), and recorded individual images such as 40 or IS em (x, y, z) (more specifically based on the Fourier transform of the individual images 40

[0079] Solving the system of equations leads to the separation of the illumination arrangement and the shifting of the spatial frequency components, which themselves are not transferable by the OTF of the microscope but are contained in the separate images with the structured illumination in a frequency-shifted form, to their original positions in the frequency domain.

[0080] The solution is the Fourier transform of the optical properties of the sample Due to the interaction with the pattern of the illumination structure, these also contain higher frequency components than those optically transmitted by the microscope 1. The resulting lateral and / or axial super-resolution image can be calculated therefrom for each recorded focal plane by inverse Fourier transformation.

[0081] If for each plane described by the system of equations, a set 30 of individual images 40 is recorded with a different illumination phase, this corresponds to conventional 3D-SIM. Here the system of equations is completely determined or even overdetermined and is solved simultaneously for all planes described overall. Solving the system of equations (in conjunction with a subsequent inverse Fourier transformation) can be considered a deconvolution, since the convolution of the light emitted by the sample with the PSF of the microscope is essentially reversed.

[0082] However, it is not necessary at all to record a set 30 of individual images 40 for each plane described by the system of equations, and instead one or more planes may be omitted. The system of equations then appears to be underdetermined and the reconstruction problem is therefore ill-posed. However, the system of equations can be approximately solved so that even for omitted planes a reconstruction of their optical properties S(x,y,z) and thus of the resulting image can be achieved. The system of equations is also solved simultaneously across multiple focal planes.

[0083] The approximate solution is done, for example, by linear regression in the frequency domain (in particular regularization). In the simplest case, interpolation can be performed. Preferably, a truncated singular value decomposition is performed or deconvolution is implemented under SVD-based constraints.

[0084] Finer sampling transverse to the optical axis - lateral oversampling - produces additional independent measurements that allow for more stable regularization, by correspondingly small detector pixels. For ordinary SIM, lateral oversampling would be redundant. However, equation (6) can be extended with additional pixels for oversampling:

[0085]

[0086] where i, j specify additional pixels in the frequency domain and Δx, Δy indicate the sampling interval transverse to the optical axis. This improves the accuracy of the approximate solution. Alternatively or in addition to lateral oversampling, more illumination phases may also improve the accuracy of the solution. As above, in the specific calculations the infinite series should be replaced by a finite sum over the sample plane to be calculated.

[0087] Figure 3 By way of example, focal planes F1, F2 and F3 are shown, which are located in the sample 2. The focus of the objective lens 3 is set successively at these focal planes. Figure 2, the sample 2 is illuminated in a structured manner. In each of the nine illumination phases, the instantaneous focal plane is recorded in a diffraction-limited manner in a corresponding one of, for example, nine individual images. For three focal planes, then, 27 individual images occur.

[0088] The focal planes F1, F2 and F3 are arranged at a distance from each other that is greater than the distance allowed by the Nyquist criterion. The distance from each other is just such that the depth of field ranges of the adjacent focal planes F1 and F2 and the focal planes F2 and F3 still overlap. Between the focal planes F1, F2 and F3, the sample has ("additional") sample planes X1 and X2, in which no focused individual images are recorded. The intensities of these planes are only detected in the individual images from the focal planes F1, F2 and F3 when not focused. The additional sample planes X1 and X2 are arranged at intervals from their adjacent focal planes, so that the focal planes F1, F2 and F3 together with the additional sample planes X1 and X2 satisfy the Nyquist criterion, for example all five planes are arranged equidistantly.

[0089] By approximately solving the set of equations (6) with the corresponding illumination phases in the three focal planes F1, F2 and F3 and the additional sample planes X1 and X2, five super-resolution result images 50 can be calculated - one result image for each focal plane and for each additional sample plane. For the approximate solution, the set of equations (6) is first regularized using, for example, a truncated singular value decomposition and then solved.

[0090] Depend on Figure 4 and Figure 5 Two possible specific illumination patterns I(x,y,z) are shown. Figure 4 The three beams shown in generate a grid pattern of parallel stripes in the sample in the focal plane, which repeats axially in the other planes due to the Talbot effect:

[0091]

[0092] Therein a0 indicates the intensity of the central beam and a indicates the intensities of the other beams. are the x and y components of the resulting raster, Corresponding to the Talbot frequency, is the phase of the Talbot pattern (the displacement of the Talbot pattern with respect to the focal plane), and is the phase of the generated grating transverse to the optical axis, used for pairwise interference of the various beams.

[0093] Substituting into equation (3) we obtain:

[0094]

[0095] Among them A n , n = 0 ... 2 is the corresponding intensity of the series. The result is five intensity components to be separated in the frequency domain. For this purpose, five different illumination phases are required. Five separate images.

[0096] In this illumination pattern, equations (6) and (7) take the following form (as above, the infinite series should be replaced by a finite sum over the sample plane to be calculated in the specific calculation):

[0097]

[0098]

[0099] In the back focal plane of objective lens 3, Figure 5 The five beams shown in Figure 2 produce a two-dimensional grid pattern in the focal plane of the sample that is repeated axially in the other planes due to the Talbot effect:

[0100]

[0101] The variables have the same meanings as those in equation (8).

[0102] Substituting into equation (3) we obtain:

[0103]

[0104]

[0105] Among them A n , n = 0 ... 6 is the corresponding intensity of the series. The result is thirteen intensity components to be separated in the frequency domain. For this purpose, thirteen different illumination phases are required. Thirteen separate images.

[0106] In this illumination pattern, equations (6) and (7) take the following form (as above, the infinite series should be replaced by a finite sum over the sample plane to be calculated in the specific calculation):

[0107]

[0108]

[0109]

[0110]

[0111] Reference numerals list

[0112] 1 Microscope

[0113] 2 Samples

[0114] 3 Objective lens

[0115] 4 Beam splitters

[0116] 5 Tube lens

[0117] 6 CCD detector

[0118] 7 Light Microscope Module

[0119] 8 Widefield detection beam path

[0120] 9 Laser scanning module

[0121] 11 Switching mirrors

[0122] 12 Beam Splitter Module

[0123] 13 Emission filter

[0124] 14 Fiber Optic

[0125] 15 Laser Module

[0126] 16 Focused Illumination Beam Path

[0127] 17 Wide field lighting module

[0128] 18 Switching mirrors

[0129] 19TIRF Illumination Module

[0130] 20 Fiber Optic

[0131] 21 Reflector

[0132] 22 Manipulator Module

[0133] 23 Grating

[0134] 24 Displacement Driver

[0135] 25 Image field rotator

[0136] 26 Rotator drive

[0137] 27 Tube lens

[0138] 28 Control Unit

[0139] A collection of 30 individual images

[0140] 40 Single Image

[0141] 41 Structured Lighting

[0142] 50 results like

[0143] Fi Focal plane

[0144] X i Additional sample plane

Claims

1. A method for imaging a sample using a microscope (1), wherein: Using the microscope (1), focusing a first focal plane of the sample (2) and sequentially illuminating the first focal plane with structured illumination light (41) in a plurality of phases, and using a detector to record sample light emitted and / or scattered by the sample (2) in each phase in a corresponding individual image (40), so that a result image (50) with improved resolution can be reconstructed from the individual images (40), characterized in that the microscope (1) is used to focus a second focal plane different from the first focal plane, and to record individual images (40) in different illumination phases for the second focal plane, and in that a result image (50) is reconstructed from the individual images (40) of the two focal planes, the result image representing a sample plane located between the focal planes, wherein the distance between the two focal planes is selected to be greater than half of the smallest axial distance optically resolvable by the microscope, the additional sample plane being located between the two focal planes, wherein the distance between the two focal planes is selected to be smaller than the smallest axial distance optically resolvable by the microscope, the additional sample plane being located between the two focal planes, and Therein, at least one solution of a system of equations is approximately determined, which describes the interaction of the light structure in the focal planes and in a sample plane situated between the focal planes with the optical properties of the sample and with the point spread function of the microscope.

2. The method according to claim 1, wherein the following equations or mathematical equivalent equations are used for the system of equations: in, Δz indicates the sampling interval in the z direction, k is the summation index, and k x ,k y ,k z are the components of the wave vector in the frequency domain.

3. The method according to claim 1, wherein: For an approximate solution, linear regression is used.

4. The method according to claim 3, wherein: For approximate solutions, regularization is used.

5. The method according to claim 3, wherein: For an approximate solution, a truncated singular value decomposition is used, where only a real subset of all singular values ​​is determined.

6. A method according to any one of the preceding claims, wherein the individual images are optically laterally oversampled.

7. The method according to claim 6, wherein the individual images are recorded with a resolution that is twice as fine as the optical resolution of the optical system of the microscope.

8. The method of claim 6, wherein at least one solution of the following equation or a mathematically equivalent equation is determined: in, Δz indicates the sampling interval in the z direction, k is the summation index, k x ,k y ,k z are the components of the wave vector in the frequency domain, i, j specify additional pixels in the frequency domain, and Δx, Δy indicate the sampling interval transverse to the optical axis.

9. The method according to any one of claims 1 to 5, wherein the light structures generated in the sample are regularly structured.

10. The method according to claim 9, wherein the optical structure generated in the sample has a lateral periodicity and / or an axial periodicity.

11. The method according to claim 9, wherein the light structures generated in the sample are arranged at different positions and / or in different orientations in each of the illumination phases.

12. A method according to any one of claims 1 to 5, wherein a resulting image (50) representing a sample plane lying between the focal planes is reconstructed with an improved resolution compared to the individual images.

13. The method according to any one of claims 1 to 5, wherein the individual images or the resultant images are approximately deconvolved based on a model of the point spread function of the microscope.

14. The method according to claim 13, wherein after separating the illumination phase by calculation, the individual images or the resulting images are approximately deconvolved.

15. The method according to claim 9, wherein: In deconvolution, intensity portions of one of the individual images are assigned to different resulting images.

16. The method according to claim 15, wherein: Exactly parts of the intensity of a pixel are assigned to different result images.

17. The method according to claim 12, wherein: In the deconvolution, intensity portions of one of the individual images are partially assigned to at least one of the result images representing one of the focal planes and partially to a result image representing a sample plane situated between the focal planes.

18. The method according to claim 17, wherein: An intensity portion of exactly one pixel is partly assigned to at least one of the result images representing one of the focus planes and partly to a result image representing a sample plane situated between the focus planes.

19. The method of claim 12, wherein the assignment of the resulting image to a sample plane representing a plane lying between the focal planes is done based on the intensities of the individual images of at least two closest focal planes.

20. The method according to any one of claims 1 to 5, wherein a two-dimensionally spatially resolving detector is used to record the individual images.

21. The method of claim 20, wherein no aperture is arranged in front of the detector to optically intercept the sample.

22. The method of any one of claims 1 to 5, wherein the point spread function of the microscope is not manipulated to produce axial asymmetry.

23. The method according to any one of claims 1 to 5, wherein during the recording of each of the individual images only exactly one focal plane is focused using the microscope.

24. A microscope (1) comprising a control unit (28) configured to perform the method as claimed in any one of the preceding claims, and further comprising a light source, a component for focusing different focal planes in a sample (2), a two-dimensional spatially resolved detector for recording individual images (40) of instantaneous focal planes, and a component for generating structured illumination light (41) in the sample (2).

25. The microscope according to claim 24, wherein no aperture is arranged in front of the detector to optically intercept the sample (2).

26. The microscope of claim 24, wherein the point spread function of the microscope is not manipulated to produce axial asymmetry.

Images