Synchrotron radiation detection method, detection device, and laser scanning microscope

Abstract

The present invention relates to a method for detecting radiation, in particular fluorescence from at least one fluorescent dye, in a laser scanning microscope, in which radiation emitted from a specimen is guided by an imaging optical unit to a two-dimensional matrix sensor, in particular having a plurality of pixels and located in an image plane, and a detection point distribution function is detected by the matrix sensor in a spatially oversampled manner. The method is characterized in that the radiation emitted from the specimen is spectrally separated by a dispersion device, in particular along the dispersion direction, and the spectrally separated radiation is detected by the matrix sensor in a spectrally resolved manner, and the spectral separation is compensated for in at least some of the pixels constituting the pixel area during analysis of the intensities measured by these pixels. Additional aspects of the invention relate to a detection device for spectrally resolved detection of radiation in a laser scanning microscope and to the laser scanning microscope.

Description

[Technical Field]

[0001] A first aspect of the present invention relates to a method for detecting synchrotron radiation, particularly fluorescence from at least one fluorescent dye, using a laser scanning microscope, in accordance with the premise of claim 1. Further aspects of the present invention relate to a detection device for detecting synchrotron radiation using a laser scanning microscope, in accordance with the premise of claim 16, and to a laser scanning microscope according to the premise of claim 27. [Background technology]

[0002] A general method for detecting synchrotron radiation, particularly fluorescence from at least one fluorescent dye, in a laser scanning microscope is disclosed, for example, in Non-Patent Document 1 (hereinafter [Castello et al., 2019]). In this process, synchrotron radiation from a specimen is guided via an imaging optical unit to a two-dimensional matrix sensor having multiple pixels located in the image plane, and the detection point spread function is detected using the matrix sensor in a spatial oversampling manner.

[0003] A general detection device for detecting synchrotron radiation in a laser scanning microscope is also disclosed in [Castello et al., 2019]. The general detection device is a two-dimensional matrix sensor located in the image plane and having multiple pixels, equipped with a device for spatial oversampling detection of the detection point spread function of synchrotron radiation coming from a specimen, and equipped with an imaging optical unit that guides the synchrotron radiation to the two-dimensional matrix sensor.

[0004] A typical laser scanning microscope is also described in [Castello et al., 2019], and it comprises the following components: a light source that emits excitation light, particularly a laser; an excitation beam path accompanied by a microscope objective system that guides the excitation light onto or into the specimen under investigation; a scanning device located on the excitation beam path that operates to scan the specimen with at least one illumination spot; a detection beam path that guides the synchrotron radiation emitted by the specimen, particularly fluorescence, to a detection unit; a main color splitter that separates the excitation light and the synchrotron radiation; a detection unit that detects the synchrotron radiation; and a control and evaluation unit, particularly a PC, that controls the light source and evaluates the measurement data obtained by the detection unit.

[0005] In biomedical research, confocal laser scanning microscopes (LSMs) have become a powerful tool over the past few decades, supporting not only the pure imaging of fluorescent biological specimens but also numerous image-based correlational or statistical experiments and analyses. The primary reason for this power lies in the fact that LSMs allow for the simultaneous measurement of multiple dyes at a reasonable cost. Many solutions are known in relation to this. For example, dichroic filters can be used to split the beam into various partial beams, which can then be guided to separate photon multiplier tubes (PMTs). In contrast, a configuration that spectrally disperses the synchrotron radiation using a grating or prism and then detects the spectrum using a line sensor offers considerable flexibility. Unwanted spectral bands can be blocked in a flexible and dye-dependent manner using a movable stop upstream of the sensor. Furthermore, solutions are known in which spectral bands are defined using, for example, a mirroring stop, and then these spectral bands are sent to separate PMTs. In this and subsequent sections, the term "dye" is intended to encompass both synthetic dyes and fluorescent proteins. Furthermore, there is also an intention to include intrinsic luminescence structures. For example, the emission of light by many biological structures under laser irradiation at specific wavelengths is called autofluorescence.

[0006] Technological innovations in confocal microscopy systems include image scanning microscopy (ISM; Non-Patent Literature 2), which was theoretically described about 30 years ago but not commercialized until 2014 with the appearance of the ZEISS LSM880 (product name). Based on oversampling measurement of the detection point spread function using a camera-like sensor, it is possible to achieve the confocal resolution limit even with a pinhole open, and the system sensitivity is significantly increased because detection is inherently parallelized. However, the solution currently implemented in the LSM980 (product name) is, firstly, very expensive because the fiber-based (2D to 1D) image conversion is combined with a GaAsP PMT line. Secondly, this solution severely limits the number of pixels that can be used to oversample the point spread function (PSF). In this specific example, it is said to be exactly 32 pixels. Therefore, these sensors can only be advantageously used in configurations where the spectral channels are defined by dichroic filters, and they also require optical zoom to guide the PSF to the sensor. The PSF depends on both the selected objective system (more precisely, its etendue and thus NA / M ratio) and wavelength. Furthermore, due to the unit cost of this detector technology, it is almost impossible to implement multiple detectors in a single device.

[0007] On the sensor side, there has been rapid progress in recent years in the field of so-called SPAD arrays or SPAD cameras (SPAD = single-photon avalanche diode). In principle, in these cameras, one pixel is realized by one photon avalanche diode, so individual activation (driving) of each pixel is possible. Furthermore, since these pixels can be operated in so-called Geiger mode, photon counting is possible on the area sensor. That is, the readout signal becomes digital immediately, so an extremely high frame rate on the order of 1 MHz can be achieved. Moreover, SPAD cameras do not generate readout noise. Readout noise occurs especially in other sensors such as sCMOS sensors and CCDs, and increases significantly at high readout speeds. EM-CCDs use an amplification mechanism near the sensor to strengthen the signal more than the readout noise, so in principle, they can be single-photon sensitive. However, this also results in significant amplification noise (also called excess noise or multiplication noise), which halves the effective sensitivity of the sensor. Furthermore, when there are a relatively large number of pixels, the achievable speed is fundamentally limited. For sCMOS cameras, it's 0.3e. - Since extremely low readout noise is achieved, photon counting using such cameras is theoretically possible.

[0008] Because they have frame rates in the MHz range, SPAD cameras are already attracting attention as LSM sensors with pixel dwell times on the order of 1 μs. For example, using such a camera, 156 kfps (fps = frames per second) can be achieved with a sensor resolution of 512 × 128 pixels at a 1-bit data depth. This corresponds to a data rate of approximately 10 Gbit. This means that it may even be possible to achieve a frame rate of 1 Mfps at a 1-bit data depth using a line sensor with approximately 1000 × 6 pixels. The limit is not yet a fundamental constraint, but rather is technically determined by the enormous data rate. With a data depth of exactly 1 bit, it is possible to achieve a speed roughly equivalent to the reciprocal of the dead time, and therefore a speed on the order of 10 MHz. However, in the example above, this would correspond to a data rate of 100 Gbit / s. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] U.S. Patent Application Publication No. 2019 / 258041 [Non-patent literature]

[0010] [Non-Patent Document 1] M. Castello et al., “Image Scanning Microscopy with Single-Photon Detector Array”, bioRxiv, doi: http: / / dx.doi.org / 10.1101 / 335596 [Non-Patent Document 2] http: / / www.gattaquant.com / files / en_wp_airyscan-detector.pdf [Non-Patent Document 3] Nat Methods 16, 175-178 (2019). https: / / doi.org / 10.1038 / s41592-018-0291-9 [Overview of the project] [Problems that the invention aims to solve]

[0011] The objective of the present invention is to provide a method of the type described at the beginning that can be used in a particularly versatile manner. Furthermore, the intention is to specify suitable detection devices and laser scanning microscopes. This objective is achieved by the method having the features of claim 1, by the detection device having the features of claim 13, and by the laser scanning microscope having the features of claim 23.

[0012] Advantageous modifications of the method according to the present invention, as well as preferred exemplary embodiments of the detection device and microscope according to the present invention, will be described later, particularly in relation to the dependent claims and drawings. [Means for solving the problem]

[0013] According to the present invention, the previously described type of method is further developed in that the synchrotron radiation coming from the sample is spectrally decomposed using a dispersion device, particularly along the dispersion direction; the spectrally decomposed synchrotron radiation is detected in a spectrally decomposed manner using a matrix sensor; and when evaluating the intensity measured by the pixels constituting the pixel region, the spectral separation is reversed for at least some of those pixels.

[0014] According to the present invention, the detection device of the type previously described is further developed in that a dispersion device is provided for spectral separation of synchrotron radiation, a matrix sensor is configured and arranged for spectral resolution detection of the spectrally separated detection light, and an evaluation electronic circuit connected to the matrix sensor is provided and configured to invert the spectral separation with respect to the pixels within the evaluation range of the intensity measured by the pixels constituting the pixel region.

[0015] According to the present invention, the previously described type of laser scanning microscope is further developed in that its detection unit is equipped with the detection device according to the present invention.

[0016] The essential idea of ​​this invention is that the synchrotron radiation emanating from the sample is spectrally separated, and consequently, the spectral components of different fluorescent dyes are first incident on separate regions of the matrix sensor, that is, on spatially separated regions. These regions are identified and each is assigned to a fluorescent dye. Then, since the spectral separation is inverted for at least some of the pixels within the individual regions, the point spreading function can be determined for each fluorescent dye in principle, similar to the known ISM case. Therefore, the point spreading function for various fluorescent dyes can be measured. In any case, spectral separation is performed favorably for those pixels where significant intensity is measured.

[0017] The essential advantage of this invention is that it achieves both the benefits of resolution and sensitivity, as well as spectral flexibility, by oversampling the point spreading function.

[0018] Therefore, the method, detection device, and microscope according to the present invention also enable ISM with spectrally dispersed signal distribution. Furthermore, the detection device and microscope according to the present invention are outstanding in that they achieve exceptionally high optical efficiency and an overall stable configuration.

[0019] The detection device according to the present invention is particularly suitable for carrying out the method according to the present invention.

[0020] The term "detection point spread function" refers to the intensity distribution on the detector, which is generated by point-like luminescent objects within the sample plane.

[0021] In principle, pixel regions related to different dyes may overlap in the detector. The important thing is that there is at least some degree of spatial separation between pixel regions. Clearly, evaluation becomes easier and better when these pixel regions are more clearly separated and when the spectral signatures of the individual dyes are more distinct.

[0022] The laser scanning microscope according to the present invention, in particular its control and evaluation unit, can be configured in conjunction with its detection device to perform the method according to the present invention.

[0023] Several examples of how to perform information processing inversion in spectral separation will be explained later.

[0024] In a particularly preferred modified form of this method, at least one pixel region assigned to the emission of a dye is identified based on a spectrum measured using a matrix sensor.

[0025] Advantageously, the spectral intensity distribution of synchrotron radiation on the matrix sensor is determined first. Then, for example, those pixel regions can be identified based on their intensity distribution. Furthermore, this intensity distribution can be used for the information processing inversion of its spectral separation. In a particularly preferred modification of the method according to the present invention, the intensity value for a specific wavelength is determined by summing the measurement data of multiple pixels in a column of the matrix sensor along a direction perpendicular to the dispersion direction, particularly the measurement data of all pixels in the column of the matrix sensor, and this is used to determine the spectral intensity distribution of synchrotron radiation on the matrix sensor.

[0026] This data, specifically the data on the spectral intensity distribution of synchrotron radiation, can be automatically determined, for example, after a change in the measurement environment. Changes in the measurement environment are primarily thought to be changes in the sample under investigation, and optionally, changes in the location where the sample is located. Optionally, different dyes can be placed at the sample or at the location where the sample is located.

[0027] As a supplement or addition, means for continuously determining data relating to the spectral distribution can be provided, particularly means for automatically determining it based on a large number of prior measurement results, especially the most recent prior measurement result. This modified form is advantageous in that it does not require individual initiation of data recording.

[0028] As you can see, for any of the evaluation purposes described in this application, a better signal-to-noise ratio can be achieved by adding up multiple sets of measurement data. If necessary, the number of measurement results from which the average is calculated can be varied to obtain the desired data.

[0029] In a more preferred modification of the method according to the present invention, the maximum and minimum values ​​in the determined spectral distribution are automatically searched for, and the corresponding pixel regions are identified, so that the spectral limits for calculating the point spreading function of a specific dye can be presented to the user based on the found maximum and minimum values. Alternatively, the spectral limits can be automatically defined based on the found maximum and minimum values. Therefore, the control and evaluation unit in the microscope according to the present invention can be configured to search for the maximum and minimum values ​​in the determined spectral distribution and to present the spectral limits for calculating the point spreading function of a specific dye based on the found maximum and minimum values. Alternatively, the control and evaluation unit can be configured to independently define the spectral limits for calculating the point spreading function of a specific dye based on the found maximum and minimum values.

[0030] Specimens containing spectrally overlapping dyes or fluorescent proteins can also be imaged and measured using the method and microscope according to the present invention.

[0031] A preferred modification of the method according to the present invention is one in which pixel regions overlap on a matrix detector, and spectral unmixing of the intensity measured by the individual pixels is performed. Methods for unmixing spectra are known in principle.

[0032] In addition to the spectral bifurcation inversion method described in this application, such spectral unmixing methods may be further performed before or after the inversion of the spectral bifurcation.

[0033] For example, the dispersion inversion can first be performed on a freely selected number of spectral bands, typically two regions with pure emission and one region with overlapping emission in the case of a two-dye system, followed by scanning microscopy and then unmixing of those channels, in this example, unmixing of three channels.

[0034] This method can be used to determine the relative ratio of specific spectral components in a pixel. This means that, according to the rules for pixel reassignment (described later), only the weighted components of the pixel, not the total intensity, are shifted (displaced). Using the configuration according to the present invention, the corresponding reference spectrum can be recorded. In the next step, spectral unmixing can be performed, for example, line by line, to determine the individual spectral loads, i.e., the loads that generate spectrally weighted components based on them. However, there are other methods for separating spectral components besides spectral unmixing, such as PCS, SCA, and the use of "deep learning."

[0035] The method according to the present invention makes it possible to determine the detection point broadening function for at least one fluorescent dye. However, a notable advantage of the present invention is that it is also possible to determine the detection point broadening function for multiple dyes having different emission spectra based on measurement data from a single measurement.

[0036] If the size of the matrix sensor being used, specifically the number of pixels in that matrix sensor, is acceptable, a multipoint variation of the method according to the present invention can also be realized in principle. In this case, multiple points on the sample are simultaneously illuminated by excitation light, and the synchrotron radiation emitted by those points is simultaneously guided to and evaluated by the matrix sensor. In this case, the excitation beam path and the detection beam path must be configured for multi-spot excitation and detection.

[0037] In the first preferred modification form, in order to reverse the spectral separation with respect to individual pixels constituting a pixel region, the intensity values ​​measured by those pixels are combined by calculation, taking into account the spectral intensity distribution of the synchrotron radiation from the dye related to that pixel region, and also taking into account the spatial intensity distribution of the individual spectral components on the matrix sensor.

[0038] To address this and to reverse the spectral separation, the evaluation electronic circuit of the detection device according to the present invention can be configured to combine the intensity values ​​measured by the pixels constituting the pixel region by calculation, taking into account the spectral intensity distribution of the synchrotron radiation from the dye related to the pixel region, and also taking into account the spatial intensity distribution of the individual spectral components on the matrix sensor.

[0039] The spatial intensity distribution of individual spectral components determines the degree to which spectral components of the dye's point spreading function that are offset from each other on the matrix sensor spatially overlap along the dispersion direction.

[0040] In this case, the intensity distribution measured by pixels forming a column perpendicular to the dispersion direction, particularly the pixels forming the column where the highest intensity is measured within the individual pixel region, can be used as the spatial intensity distribution of individual spectral components. This is based on the assumption that the detection point spread function on the matrix sensor is rotationally symmetric and, so to speak, circular. This is a good assumption when a rotationally symmetric optical unit is used.

[0041] In another important group of method transformations, the inversion of spectral separation for individual pixels constituting a pixel region is achieved by pixel reassignment. For spectrally non-resolvable methods, see, for example, [Castello et al., 2019], as these are known based on scanning image microscopy (ISM).

[0042] Based on this, in order to invert the spectral separation with respect to individual pixels that make up a pixel region, the evaluation electronic circuit of the detection device according to the present invention can be configured to assign the intensity values ​​measured by those pixels to locations within the image plane that are shifted relative to those individual pixels, using a displacement (shift) vector that depends on the location of the individual pixel and the wavelength related to that location (pixel reassignment).

[0043] In these modified forms of the method, the intensity values ​​measured by each pixel are assigned to locations within the image plane that are shifted relative to the individual pixels (pixel reassignment), thereby inverting the spectral separation with respect to the individual pixels that make up the pixel region. In this case, as with known pixel reassignment, the displacement vector depends not only on the location of the individual pixel but also on the wavelength related to that location.

[0044] Specifically, a displacement vector is determined for each pixel involved in the pixel reassignment, and this displacement vector is made dependent on the location of the related pixel and the wavelength associated with that related pixel. Then, the intensity value measured for that related pixel is assigned to the location that is shifted relative to that related pixel, according to the displacement vector.

[0045] For example, the wavelength-independent component of the displacement vector can be obtained for a specific pixel by rescaling the vector component of the vector from the reference pixel to the related pixel using a reassignment factor. The reassignment factor -1 / 2 is obtained under the assumption that the point spreading functions are very similar for excitation and emission (Stokes shift is ignored), that is, the intensity value measured by a specific pixel will be assigned to an in-image-plane location exactly halfway along the path from the reference pixel to the related pixel.

[0046] Specifically, pixel reassignment is performed in such a manner that the resulting detection point spread function has substantially the same shape along the dispersion direction and along the direction perpendicular to that dispersion direction. This is based on the assumption that when a rotationally symmetric optical unit is used, the detection point spread function will always have a circularly symmetric intensity distribution.

[0047] However, there are other ways to obtain the corresponding displacement vector. For example, the displacement vector related to the wavelength range assigned to the sample structure can be determined by evaluating the phase correlation of multiple scan images.

[0048] In the detection device according to the present invention, an analog integrating type and / or photon counting type detector can be used as the matrix sensor. It is preferable to use an sCMOS camera, an EMCCD camera, and / or a charge integrating type sensor. It is particularly advantageous to have a SPAD camera as the matrix sensor, or to use a SPAD camera as the matrix sensor. It is especially advantageous to operate the matrix sensor, in particular the SPAD camera, in a photon counting mode that can count individual photons. This mode, also known as the "Geiger mode," stands out for its particularly favorable signal-to-noise ratio.

[0049] The significant decline in the cost of the sensors used in this invention is particularly advantageous to the present invention. In the configuration described herein, the use of conventional PMT technology involving fiber coupling is almost unthinkable.

[0050] In particular, optical diffraction and / or optical refraction devices can be used as dispersion devices in the detection device according to the present invention. It is especially desirable that the dispersion device comprises a grating, particularly a line grating, and / or a prism. In principle, a grism, i.e., a combination of a prism and a grating, can also be used. Adjustable or controllable elements, such as DMDs (Digital Micromirror Devices), MEMS (Microelectromechanical Systems), or SLMs (Spatial Light Modulators), can also be used in the same way.

[0051] In a particularly preferred modification of the detection device according to the present invention, the dispersion direction is the same as, or more particularly, parallel to, the coordinate direction of the matrix sensor. For example, a group of pixels along the same direction as the dispersion direction can be called a pixel row, and a group of pixels along a direction perpendicular to the dispersion direction can be called a pixel column. Performing the alignment of the matrix sensor in such a manner that the dispersion direction is parallel to the direction of the pixel row is advantageous because it ensures that the pixels within each pixel column are assigned to exactly the same wavelength range, or more precisely, exactly the same wavelength.

[0052] In relation to the dimensional setting of the matrix sensor and / or optical imaging on the matrix sensor, it is desirable to select the pixel pitch (in principle, the lattice constant) of the matrix sensor such that, across the spectral bandwidth of the dye, it is greater than the change in the width of the Airy disk of the detection point spreading function in the plane of the matrix sensor. In this context, it would also be desirable to dimensional the matrix sensor and / or the optical image on the matrix sensor such that the per-pixel spectral bandwidth of the matrix sensor along the dispersion direction is less than 0.5 nm, preferably less than 0.4 nm, and more preferably 0.3 nm. Considering the calculation of the point spreading function, specifically the implementation of discrete deconvolution, it is possible to make the assumption that the calculation is simplified in these configurations. More precisely, in this case, it can be assumed with good approximation that the width of the Airy disk is substantially wavelength-independent through the spectrum of the dye. Nevertheless, sufficient spectral resolution can be obtained.

[0053] Particularly desirable, the matrix sensor and / or the optical image on the matrix sensor can be further sized such that the diameter of the airy disk of the detection point spread function in the plane of the matrix sensor is less than 20 times the lattice constant of the matrix sensor, particularly preferably less than 7 times the lattice constant, and most preferably less than 5 times the lattice constant of the matrix sensor. Such dimensional settings allow the discrete deconvolution calculation to be advantageously limited to a relatively small number of wavelengths, specifically to a number corresponding to the number of pixels covered by each airy disk.

[0054] Preferably, the diameter of the airy disk on the matrix sensor can be made more than three times the grid constant, so that the final stage can ultimately perform a method equivalent to that of ISM.

[0055] SPAD array sensors are particularly well-suited for measuring fluorescence lifetime information using TCSPC (Time-Correlated Single-Photon Counting), histogramming, or time-windowed detection (FLIM = Fluorescence Lifetime Imaging Microscopy). This requires pulse excitation and a properly configured sensor. A combination of this and the Airyscan® method is described, for example, in Non-Patent Document 3.

[0056] Furthermore, this invention enables spectral decomposition. This combination is particularly attractive.

[0057] In one preferred modified form of the method according to the present invention, time-resolved measurement for determining the fluorescence lifetime of a dye is performed using a matrix sensor, for example, using several pixels of the matrix sensor, and in particular using each individual pixel of the matrix sensor.

[0058] One advantageous development of the detection device according to the present invention is particularly noteworthy in that, in order to determine the fluorescence lifetime of various dyes, the matrix sensor and evaluation electronic circuit are configured to perform time-resolved measurements, for example, using some of the pixels of the matrix sensor, preferably using each individual pixel of the matrix sensor.

[0059] Therefore, these matrix sensors and electronic circuits should preferably be designed in such a way that time-resolved measurements can be performed in this manner, and consequently, the fluorescence lifetime can be determined for each pixel. That is, in these modified forms of the present invention, spectrally resolved FLIM can be combined with image scanning. It is advantageous to use a pulsed laser for these measurements.

[0060] To improve the detection efficiency of the matrix sensor, a microlens can be placed in front of the matrix sensor, i.e., upstream of the matrix sensor.

[0061] The imaging optical unit of the detection device according to the present invention may be equipped with a zoom system for changing the image of synchrotron radiation, particularly one for scaling the spectral bandwidth of each pixel.

[0062] Furthermore, following fundamentally known procedures, the microscope according to the present invention may be equipped with means for blocking excitation light, in particular at least one radiation filter. A changer device having multiple radiation filters, such as a filter wheel, may also be preferably provided. However, the detection device according to the present invention may also be a modified method in which unwanted spectral components of synchrotron radiation are not evaluated. For such purposes, for example, the corresponding pixel group of the matrix sensor, in particular the pixel column, can be set to a passive or inactive state.

[0063] Further advantages and features of the method, detection apparatus, and laser scanning microscope according to the present invention will be described below with reference to the attached drawings. The drawings are as follows. [Brief explanation of the drawing]

[0064] [Figure 1] This figure shows a schematic appearance of the laser scanning microscope according to the present invention. [Figure 2] This figure shows a schematic appearance of the detection device according to the present invention. [Figure 3] This diagram clearly shows the diameter of the airy disk as a function of wavelength for various numerical apertures. [Figure 4] This figure clearly shows the dispersion angle curve in a diffraction grating as a function of wavelength. [Figure 5] This diagram shows the typical emission spectra of dyes. [Figure 6] This figure clearly shows the curve of the detection point spread function across multiple pixels of a matrix sensor, along a direction perpendicular to the dispersion direction. [Figure 7] Figure 7a shows a simulated image, Figure 7b shows the count rate plotted against the number of pixels in a pixel column, and Figure 7c shows the count rate plotted against the number of pixels along the dispersion direction. [Figure 8] Figure 8a shows an image accumulated from many simulated images, Figure 8b shows the average count rate plotted against the number of pixels in a pixel column, and Figure 8b shows the average count rate plotted against the number of pixels along the dispersion direction. [Figure 9] Figure 9a shows an image accumulated from many simulated images with two types of dyes present in the sample; Figure 9b shows the average count rate plotted against the number of pixels along the dispersion direction for those two types of dyes; Figure 9c shows the average count rate plotted against the number of pixels along the dispersion direction under spectral constraints on the emission region of those dyes; and Figure 9d shows the average count rate plotted against the number of pixels in multiple pixel columns associated with the maximum emission values ​​of those two types of dyes. [Figure 10] This figure shows a schematic representation of a matrix sensor, omitting spectral decomposition for the purpose of explaining the principle of pixel reallocation. [Figure 11] This figure shows a schematic representation of the matrix sensor of the detection device according to the present invention for the purpose of explaining the fundamental objectives and pixel area of ​​the present invention. [Figure 12] This figure shows a schematic representation of a matrix sensor for a detection device according to the present invention, for the purpose of explaining the principle of pixel reallocation in conjunction with spectral decomposition. [Modes for carrying out the invention]

[0065] Similar components and components that operate similarly will be shown in the figures by the same reference numerals.

[0066] Figure 1 shows a schematic representation of the laser scanning microscope 100 according to the present invention. Essentially, the microscope 100 comprises a light source 12, particularly a laser, that emits excitation light 14; an excitation beam path 10 with a microscope objective system 24 that guides the excitation light onto or into the specimen S under investigation; and a scanning device 22 located on the excitation beam path 10 that scans the specimen S with at least one illumination spot 27. A detection beam path 30, accompanied by a detection unit 32 that detects synchrotron radiation 28, is also provided to guide the synchrotron radiation 28, particularly to a fluorescence detection unit 32; this synchrotron radiation is emitted by the specimen S as a result of exposure to the excitation radiation 14. A main color splitter 18, provided according to the present invention, separates the excitation light 14 and the detection light 28. A control and evaluation unit 34, particularly a PC, is provided to control the light source 12 and to evaluate the measurement data obtained by the detection unit 32. According to the present invention, the detection unit 32 is equipped with the detection device 200 according to the present invention.

[0067] The excitation light 14 emitted by the light source 12 passes through the deflection mirror 16 to the main color splitter 18, where it is guided toward the scanning device 22; the scanning device can be positioned in a plane optically conjugate to the posterior pupil of the microscope objective system 24. The excitation light 14, having passed through the scanning device 22, passes through the scanning objective system and tube lens to the microscope objective system 24, which then focuses the excitation light 14 onto an illumination spot 27 on or within the specimen plane 26 of the specimen S. The scanning objective system and tube lens are schematically depicted as a single component 23 in Figure 1.

[0068] Subsequently, radiant radiation 28 is emitted from the area exposed to the excitation light 14 on or within the specimen S; this is typically fluorescence from a dye previously installed in the specimen S. The radiant radiation 28 then travels back along the same path as the excitation light 14 previously followed on the "descan" detection beam path 30 to the main color splitter 18, but this time it passes through the main color splitter and reaches the detection unit 32 equipped with the detection device 200 according to the present invention. The data obtained from the measurement by the detection unit 32 is evaluated by the control and evaluation device 34. The control and evaluation device 34 can also control the light source 12 and the scanning unit 22. The microscope 100 according to the present invention is, in particular, a confocal laser scanning microscope.

[0069] Figure 2a) shows a schematic representation of an exemplary embodiment of the detection device 200 according to the present invention. The detection device 200 for spectral resolution detection of synchrotron radiation 28 in a laser scanning microscope 100 comprises, as essential components, a dispersion device 40 for spectrally separating the synchrotron radiation 28 coming from the sample S under investigation, a two-dimensional matrix sensor 50 for spatially resolving detection of the spectrally resolved synchrotron radiation, and an imaging optical unit 48 for guiding the spectrally resolved synchrotron radiation onto the two-dimensional matrix sensor 50.

[0070] In the exemplary embodiment shown in Figure 2a), the dispersion device 40 is a line grid 43, more precisely a reflection grid, and the synchrotron radiation to be detected 28 is incident on the line grid 43 from below at a certain angle. A prism or grism can be used instead of the grid 43. Clear spectral smearing of the signal on the matrix sensor 50 is important; in principle, this spectral smearing can be either linear or nonlinear depending on the wavelength distribution at that location. However, it will always be monotonic. In the illustrated exemplary embodiment, spectral branching occurs along the dispersion direction 41 that extends vertically in Figure 2a) because the lines of the grid 43 extend perpendicular to the plane of the paper. Reference numeral 29 indicates the zero-order diffracted light. The zero-order diffracted light 29 is optional, but guiding it back onto the grid 43 along the incident light, optionally with deflection rotation, can improve detection efficiency.

[0071] The synchrotron radiation 28 is confocally filtered by a pinhole (not shown here), parallelized by an optical system (also not shown here), and then directed towards the dispersion device 40.

[0072] Subsequently, the individual spectral components 42, 44, 46, and 47 are incident on the imaging optical unit 48, that is, on the lens element schematically shown in Figure 2a). In principle, the imaging optical unit 48 can be equipped with multiple beam shaping members, and moreover, it can be equipped with a zoom system.

[0073] The imaging optical unit 48 focuses these separate spectral components 42, 44, 46, and 47 onto a matrix sensor 50 having multiple pixels 51. The matrix sensor 50 can be, for example, a SPAD multi-line camera, which can have, for example, 5 lines, and each line can have 800 pixels, equivalent to 800 columns.

[0074] Figure 2b) shows a schematic representation of the area of ​​the matrix sensor 50. The matrix sensor 50 is positioned relative to the grid 43 such that the dispersion direction 41 extends along the row direction, that is, parallel to the x-direction of the coordinate system shown in Figure 2b). The column direction extends parallel to the y-direction. For example, in Figure 2b), the blue spectral component can be located at the left edge of the spectrum, and the red spectral component can be located at the right edge of the spectrum. The indices i and j are used to label the column and row of the matrix sensor 50, respectively, so to speak, a pixel (i,j) becomes a pixel in the j-th row within the i-th column.

[0075] Figure 2a) schematically shows an evaluation electronic circuit 60 provided according to the present invention, which is connected to a matrix sensor 50 and measures the intensity I measured by the pixels (i,j) that make up the pixel region. i,j The system is configured to invert the spectral separation for those pixels (i,j) during the evaluation.

[0076] The evaluation electronic circuit 60, which may include a data grabber, can be implemented using, for example, an FPGA, a microcontroller, or other comparable components. Essentially, by achieving a reduction in data volume as close to the hardware as possible, that is, as close to the matrix sensor 50 as possible, it becomes possible to flow data as continuously as possible from the evaluation electronic circuit 60 to the control PC, that is, to the control and evaluation unit 34 of the microscope 100 according to the invention described in Figure 1.

[0077] The situation in Figure 2a) can also be described as a situation in which the dispersion element 40 causes large and clear chromatic aberration along the dispersion direction 41, separating different wavelengths and thus making them individually detectable. In the direction perpendicular to this, i.e., the y direction in Figure 2b), the point spreading function remains essentially unchanged. However, spectrally dispersing the synchrotron radiation 28 destroys this point spreading function to some extent, making the method of performing ISM imaging using this system less obvious.

[0078] To achieve this, the imaging optical unit 48 is first sized based on the dimensions of the matrix sensor 50, that is, in the illustrated example, based on the dimensions of the SPAD camera. For example, the pixel pitch of the matrix sensor 50, or what can also be called the lattice constant of the matrix sensor 50, can be set to a = 25 μm.

[0079] The diameter of the airy disk is known to depend on the wavelength of light and the numerical aperture, as described below. d Airy =1.22λ / NA (1)

[0080] To evaluate the signal related to ISM, the point spread function must be oversampled by at least 3-4 pixels in each spatial direction. To achieve this for the relevant wavelength range from 450 nm to 800 nm, the detection-side numerical aperture must be approximately NA = 0.01 when the sensor is illuminated. This is evident from the illustration in Figure 3, where the diameter of the airy disk is shown as a function of wavelength with respect to various numerical apertures. In this case, the grid formed by the y-axis in the illustration in Figure 3 corresponds to the assumed pixel size of the SPAD camera, which is 25 μm.

[0081] Figure 4 shows a curve of the dispersion angle in degrees for wavelengths in nm for a 1000 lines / mm grid. For example, if the focal length of the imaging optical unit 48 located between the grid 43 and the SPAD camera 50 is assumed to be f=50mm, this results in a spectral distribution covering approximately 730 pixels in the spectral range shown above. Therefore, the spectral bandwidth assigned to a single pixel 51 is approximately Δλ(pixel) = 0.5nm.

[0082] In principle, to reconstruct the point spreading function from a dispersed smeared signal distribution, various spectral components must be numerically integrated. Once the intensity of these pixels is obtained in this manner, the scanning image microscopy (ISM) method can be applied in the next step.

[0083] The incident intensity on each individual pixel 51 is I i,j It is represented by , and the exponent i represents the number of the column, i.e., extending along the x-direction and dispersion direction 41. The exponent j represents the number of the row, i.e., extending along the y-direction.

[0084] Strength I i,j It can be written as follows:

number

[0085] Here, Sp(λ) is the emission spectrum of the excited fluorescent dye, and Airy(x,λ) is the Airy function corresponding to the spatial intensity distribution of the point spreading function. The wavelength λ within the Airy function, in particular, represents the parameter that determines the width of the Airy function Airy(x,λ). x0 represents the center of the point spreading function. The center of the point spreading function, i.e., the point of maximum intensity, is therefore determined by the dispersion x0(λ) and by the adjustment made by optical means relative to the column center y0.

[0086] Typically, the spectral distribution of common fluorescent dyes (phosphors) has a spectral bandwidth of approximately δλ = 50 nm (see Figure 5). As shown in the graph in Figure 3, the width change of the Airy function over this bandwidth is considerably smaller than 10 μm. That is, if the width of the spectral emission centroid is set to λ0 with respect to its point spreading function, the error will be less than 1 / 4 of the pixel; this occurs especially in the weak emission edge area of ​​the spectrum and therefore contributes almost nothing. For this reason, Airy(y-y0,λ) can be replaced with Airy(y-y0,λ0) with good approximation. This makes the PSF term along the y coordinate independent of λ, so it can be taken out of the integral along the λ coordinate.

[0087] At this time, a one-dimensional deconvolution problem remains in the calculation of the unperturbed Airy function from spectral smearing along the x-direction when the spectrum Sp(λ) of the excited fluorescent dye is known. The information processing and hardware effort in this context should not be underestimated. After all, the signals from at least 4×730 pixels must be evaluated.

[0088] As will be described later, by introducing additional assumptions, the information processing load borne by the evaluation electronic circuit 60 can be further reduced. The first assumption introduced is that the unperturbed point spread function is radially symmetric. This means that the spatial spread of the point spread function along the dispersion direction 41, i.e., the x-direction, can be determined based on the readable intensity distribution I in the direction orthogonal thereto, i.e., the y-direction j (y) This means that it can be determined based on Furthermore, since the per-pixel spectral bandwidth at Δλ<0.5 nm is very narrow, the dispersive signal smearing caused by the spread of the pixel can be neglected with a good approximation, resulting in that the per-pixel spectrum can be assumed to be constant for each segment. Furthermore, the width of the integration range, that is, the number of pixels over which integration must be performed, can also be determined based on the intensity distribution determined along the orthogonal direction, i.e., the y-direction. As is clear from the illustration in FIG. 3, integration over 5 pixels is completely sufficient. This gives the following discretized version of the aforementioned problem. [Formula]

[0089] Here, sp n is the discretized intensity of the fluorescent dye at the spectral position of the relevant pixel. Accordingly, the problem is to determine the components of the Airy function centered on an adjacent pixel in accordance with the intensity I measured at pixel (i,j) i,j In this connection, an approximate value can also be obtained from data measured in the orthogonal direction, i.e., the y-direction.

[0090] First, it is advantageous to calculate the sum of the signals along the direction perpendicular to the dispersion direction, that is, across all pixels in a column. This yields a normalizable emission spectrum, resulting in the distribution shown in Figure 5. Based on this, the spectral distribution discretization value sp necessary for calculating the discrete integral value can be calculated. i This can be determined. Figure 5 shows, as an example, the first wavelength λ i Spectrum sp related to the first pixel (i,j) i The value of the wavelength λ located right next to it. i+1 Spectrum sp in the corresponding pixel (i+1,j) i+1 The values ​​of are shown. Once the spectral distribution is determined, the contributions of neighboring pixels i-2, i-1, i+1, i+2 to pixel i can be determined based on the intensity relationship along the y-direction. In the example of the spatial distribution of the Airy function shown in Figure 6, the spectral value sp i+1 The following proportional intensity distribution will be obtained for the five pixels. I cal i =(I(i-2),I(i-1),I(i),I(i+1),I(i+2))=(0.008,0.16,0.65,0.16,0.008)

[0091] Figure 6 shows the sp at spectral position i. i Regarding this, specifically for pixel i adjacent to pixel number i+1 in the SPAD camera 50, the value of a relatively low point spreading function is also shown, corresponding to the spectrum in Figure 5.

[0092] The value determined in this way is the last remaining integral.

number

number

[0093] Here, Pn,j This is the component determined at pixel (i,j) of the intensity of the Airy function centered on the neighboring pixel n. n,j for

number

number

[0094] When the photon counting matrix sensor is in Geiger mode, P i-n,j As a result, N photons are detected for each pixel of the matrix sensor. i-n,j However, the subpixels of the PSF to be determined are assigned according to the specific pre-calibrated distribution of the PSF at a given sensor position, as described below.

number

[0095] As a result, spectral branching inversion according to the present invention is performed with respect to the pixels in the pixel region related to the dye under consideration.

[0096] If the error resulting from ignoring dispersed signal smearing appears to be too large, countermeasures can be taken by using a larger sensor array or an imaging optical unit with a longer focal length. For example, by using a 4 × 1400 pixel array and setting the focal length to f = 100 mm, the pixel-by-pixel bandwidth immediately becomes Δλ < 0.25 nm. That is, a reasonable operating point can be found in any case. There is no need to limit the number of rows of the matrix sensor 50 to a few. If a high frame rate can be achieved, it is possible to have a considerably large number of image lines, thereby enabling measurements with multipoint excitation. In the method according to the present invention, the pixel dwell time can be extended to a factor of the parallelization at a given frame rate, so such parallelization is beneficial.

[0097] The limitation on the number of photons in an LSM presents problems related to the application of the principles described above. The matrix sensor 50 supplies a digital signal, i.e., photons and images for each sensor pixel. Subsequently, one image from the matrix sensor 50 is converted into one pixel in the entire LSM image. Since the photon flux incident on the matrix sensor 50 can be on the order of several megahertz, and the pixel dwell time of the LSM should be on the order of 1 μs, for example, only a few photons are distributed per pixel dwell time across 4 × 730 pixels. Consequently, most of the pixels in the matrix sensor 50 supply 0 as data, and only a few pixels supply 1. Therefore, a single direct readout from the matrix sensor 50 cannot even provide usable results. In addition, since the emission and detection of photons are statistical events, the long-term average photon distribution cannot even be derived from a single image with such a small number of photons. This is illustrated in Figure 7a), where a single image obtained from the SPAD camera 50, related to exposure to the dye spectrum described in Figure 5, is simulated with a photon flux of Iphot = 10 MHz with an exposure duration of 1 μs. The spectral bandwidth per pixel is 0.5 nm. Only a few events are visible, and they are clustered around the maximum value of dye emission at approximately 520 nm, on the central line of the 5-line SPAD camera 50. Figure 7b) shows the sum of the signals of each event along the image line (along the x-direction) of the SPAD camera 50, thereby reproducing the point spreading function. Figure 7c) shows the sum of the counting events of all pixels within the column, i.e., along the y-direction, and is very similar to the detection spectrum of that dye. As is obvious, the algorithm described earlier cannot even be meaningfully applied to the data in Figure 7.

[0098] Therefore, the system needs to be calibrated using an integrated image. However, in principle, this is a very fast process, and multiple image scans are generally not necessary. This is illustrated in Figure 8a) for the average of 1000 individual images with an average photon flux of 10 MHz. The integration time in this case is only 1 ms, which is roughly equivalent to scanning one image line. The point spreading function in Figure 8b) was determined based on the summation signal and reproduces the expected intensity distribution described above on the five pixels with roughly strict accuracy, and the spectrum in Figure 8c) can also be understood.

[0099] Based on the measurements in Figures 8b) and 8c), sp i-n and P i-n,j Data related to this can be obtained and stored, for example, in the memory of the evaluation electronic circuit 60. Then, the SPAD image for each scanned image pixel can be evaluated according to the rule (equation 5). The integer photon values ​​can now be converted to floating-point values ​​to obtain the luminance ratio.

[0100] A second option for spectral separation and inversion according to the present invention, which relates to the pixels in the pixel region assigned to the dye, is based on a pixel reassignment method. This will be described with reference to Figures 10 to 12.

[0101] First, known pixel reallocation methods will be explained based on Figure 10, using a self-directed description in accordance with the terminology used in [Castello et al., 2019].

[0102] Consider a scenario in which a sample is scanned using a point-type illumination source, and the inverse radiation intensity from the sample is measured in a spatially resolved manner using a confocal two-dimensional detector array fixed to the sample. Assume that this confocal matrix sensor has n × m pixels (i,j), and that the indices i and j range from 1 to n and m, respectively.

[0103] The intention is to evaluate the spectral measurement results on a rectangular array at a later stage, so for simplicity, a rectangular array is assumed here. However, this is not a constraint, and in principle, other geometrically shaped pixel sensors are also conceivable. In particular, hexagonal configurations are often used because they yield a good fill factor (packing efficiency).

[0104] Furthermore, for clarity, we assume that the magnification from the object plane to the intermediate image plane is 1. Also, for advantages, we assume that each pixel of the sensor is smaller than 1 Airy unit (AU). This example corresponds to the standard assumption in image scanning, namely that the PSF is detected in a spatial oversampling manner and that each pixel represents an effective pinhole smaller than 1 AU (or smaller than 0.8 AU, or even better, smaller than 0.3 AU). In principle, this configuration works well even with a PSF smaller than the pixel size. However, in that case, only the spectrum can be measured and spectral confocal images can be recorded. Consequently, image scanning cannot be used in a meaningful manner.

[0105] An image scanning microscope is a linear and spatially invariant system; it can be thought of as a linear mapping of the specimen plane onto the image plane 56 of the matrix sensor. The variable x represents the position obtained by back-projecting the position on the image plane 56, i.e., the matrix sensor plane, onto the specimen plane. When the scanning position at point x is within the specimen, that is, when the maximum illumination intensity is at position x within the specimen, the intensity g measured by pixel (i,j) is i,j (x) can be described as a convolution as follows:

number

[0106] Here, h i,j (x) = h exc (x)h em (xd i,j) represents the effective PSF for each sensor pixel under consideration. i,j (x) is the PSF of the excitation h exc (x) and the PSF of radiation h em (xd i,j It is the product of ). In principle, the PSF of the excitation is h exc (x) can be measured, so we can assume it is known. To obtain the correct detected PSF, if the size of individual pixels cannot be ignored, the aperture function that describes the geometry of that pixel is h em (xd i,j It will need to be folded into ). i,j is the in-plane vector of the detector array, and corresponds to the offset between the reference element, for example, the element at the center of the sensor, and the pixel (i,j). Therefore, d i,j This can be written as follows:

number

[0107] Simply put, the effective PSF is h i,j The maximum value of (x) is the function h exc (x) and h i,j (xd i,j It is located approximately midway between the maximum values ​​of ), i.e., approximately at position s i,j =d i,j It can be assumed that it is at / 2. This is likely to hold strictly in aberration-free systems when the excited PSF and emitted PSF are very similar, and in fluorescence when there is no Stokes shift. In the basic idea of ​​pixel reassignment, it is assumed that most of the intensity measured by pixel (i,j) originates from a sample position that does not correspond to the position coordinates of the related pixel on the image plane 56. Simply put, the maximum effective PSF is function h exc (x) and h i,j (xd i,j If it is assumed to be at the midpoint between the maximum values ​​of ), the location from which the intensity of the measured pixel (i,j) originates is at position s in the sample plane. i,j =d i,j It becomes / 2.

[0108] The fundamental idea behind pixel reassignment and image scanning is to shift signals that are misaligned relative to a reference pixel in the opposite direction along the direction of that reference pixel, and then to add those signals together. In principle, this is intuitively obvious because each pixel (i,j) of a matrix sensor in a confocal system operating in this manner supplies a displacement image. In addition to this inverse shift, these images can be simply aligned relative to each other, or signals from all pixels can be computationally combined using other forms of computation, such as multiview deconvolution.

[0109] Figure 10 shows the situation and labels when confocal image scanning is performed using a matrix sensor 54 having 5x5 pixels. The central pixel (3,3) acts as the reference pixel. Circle 57 represents the centroid of the effective PSF related to pixel (5,4). However, the extent of the PSF is generally considerably larger than circle 57, and it is shown in Figure 10 solely to indicate the position of the maximum value of the PSF.

[0110] Image scanning is known to achieve better optical resolution and improved signal-to-noise ratio (SNR). Color dependence can be realized through image scanning using filters. The use of strip grids allows for the measurement and evaluation of PSF (Photon Scaffolding) including two colors.

[0111] However, as is obvious, directly measuring the spectrum at multiple points provides a considerable amount of information. Furthermore, this information can be used for spectral unmixing of the data. In addition, the positive characteristics of image scanning can be used for such measurements, which would be advantageous.

[0112] By using the detection device according to the present invention, for example, measurements can be performed in such a manner that the spectral components of a specific radiation band corresponding to a pixel area on the matrix sensor 50 are spectrally combined, and consequently, a so-called image scanning method (also known as Airy scanning or optical reassignment) can be performed with respect to that spectral band. This combination is referred to as spectral bifurcation inversion in the terminology of independent claims.

[0113] To create combinations of pixels related to the radiation belt, it is necessary to invert the dispersed effects of the grid and other dispersed devices with respect to N adjacent pixels, that is, with respect to the pixels that make up the pixel region.

[0114] Ultimately, this is comparable to a situation where light can travel backward through the dispersion element in a manner that inverts its dispersion. This would result in a PSF containing only spatial information, in which the various spectral components also form a point spreading function and are spatially superimposed. Something similar is described, for example, in Patent Document 1. However, dispersion inversion by optical means is not always possible. For example, it is desirable to use a grating for spectral branching because it brings about linear dispersion. However, due to its limited efficiency, the use of multiple gratings, i.e., use in both the forward and return paths of light, is not advantageous. Furthermore, purely optical dispersion inversion devices are complex, expensive, and difficult to adjust.

[0115] In the case of linear dispersion induced by a grid, the relationship λ = kx applies to the wavelength assignment along the longitudinal x-direction of the matrix sensor. Here, k is a proportionality constant that depends on the intensity of the dispersion, and consequently on the line width of the grid. Therefore, its unit is nm / mm. However, in principle, this consideration is not limited to grids but can also be applied to prisms, which are dispersion elements. However, in that case, the relationship between the location on the sensor and the wavelength can no longer be described using a simple linear relationship. Consequently, the calibration and evaluation of measurement results will be somewhat different.

[0116] Furthermore, the use of a grating is desirable because linear dispersion leads to optimal sampling of the spectrum (pixels for each wavelength) across various wavelengths, and the position-wavelength relationship is maintained linearly. While the use of a prism is optically somewhat more efficient under certain circumstances, it results in better sampling of the blue portion of the spectrum, but the red wavelengths are spectrally "suppressed" and therefore not sampled well. Strictly speaking, this is unfortunately a disadvantage because excitation / detection becomes more difficult when longer wavelengths are selected, especially when imaging biological specimens. In this case, multiple staining methods should be made easily detectable, particularly with the apparatus according to the present invention.

[0117] Herein lies an exemplary embodiment. Figure 11 shows a matrix sensor along with two spectral bands schematically represented, and these spectral bands correspond to the spectral signature of a fluorescence radiation specimen having, for example, a blue-green emission region (pixel region 71) and a more orange-red emission region (pixel region 72). In principle, the working set for data evaluation involves combining specific spectral bands, and thus the signals of the pixels constituting pixel regions 71 and / or 72, in such a manner that one or two circular symmetric point spreading functions 80 (PSF) are generated. Using a known image scanning method, the image of the specimen can be obtained, either subsequently or simultaneously, based on the evaluation of the PSF obtained in this manner, so that the advantageous characteristics of the image scanning are linked to spectral resolution, i.e., both dyes can be represented within the image. In order to maintain transparency in this representation, it is first assumed that the dyes are spectrally separated to the extent that they overlap on separate pixels of the sensor. This situation is advantageous. However, this configuration can also be used to image and measure specimens containing spectrally overlapping dyes or fluorescent proteins. This has already been explained in the overview section of this specification.

[0118] Figure 12 shows details of the matrix sensor 50 of the detection device according to the present invention, along with the pixels (i,j) used to measure the dispersed synchrotron radiation 28. The synchrotron radiation 28 is dispersedly branched along direction i. This means that each column i is assigned to one wavelength. There is no dispersed wavelength branching along direction j, so only spatial information can be used there. In the situation shown in Figure 12, pixel (12,3) represents a reference pixel, and the reference wavelength associated with it is λ r It is stated that the signals of other pixels are associated with the reference pixel. This means that the signals of other pixels are pushed back towards the reference pixel (12,3) in a manner similar to that which has been performed in known image scanning microscopy techniques to date. In this case, "pushing back" means that the signals of individual pixels are assigned to numerically determined locations within the plane of the matrix sensor, and therefore within the sample plane. However, in contrast to known methods, the displacement in this case must be performed in a manner that correctly takes its variance into account.

[0119] In other words, the essential point of modification is the displacement vector d i,j The key point regarding this is that it is assumed that the displacement path is composed of two parts. d i,j =2s i,j +ξ i ξ=ξ(λ) (9)

[0120] Here, ξ(λ) is a function of wavelength. For example, the following are some possible options for this function: ξ i =κ(λ i / λ r -1) (10)

[0121] The proportionality constant k has units of length and is determined by the dispersion intensity of the dispersion element, and in particular by the lattice constant of the lattice 43 being used.

[0122] Therefore, the displacement path is represented as follows:

number

[0123] Therefore, in this example, the variance only affects the x-direction. Consequently, in Figure 12, the displacement vector for the plotted pixel (12,5) appears unlikely to change due to the variance, while the displacement vector for the plotted pixel (8,4) will have a variance component.

[0124] Based on this, the following can be obtained regarding the centroid of the effective PSF.

number

[0125] Thus, while there is no change along the direction perpendicular to the dispersion, comparable to conventional image scanning (Aery scanning) evaluation, there is expansion / compression along the dispersion direction. This is done in such a way that the further the wavelength is from the reference wavelength, the more correction is applied, resulting in the effective PSF's centroid shifting closer along the reference pixel direction. As a result, these centroids are no longer at half the pixel-reference pixel displacement length, but rather shift somewhat closer to the reference pixel. This, therefore, provides a rule for how the components of those pixels must be shifted so that the contributions of all pixels can ultimately be combined computationally. Optionally, further calibration can be used here to obtain the correct s i,j It is also possible to determine this. However, the displacement vector can be affected by sample-induced aberrations, as can occur especially in the case of thick specimens, which are often investigated using multiphoton microscopy (Castello et al., 2019; Figure 1c). One method for determining these pushback vectors based solely on the image data of individual pixels of the sensor is given in the next section.

[0126] [Determination of displacement vectors by phase correlation] Another data evaluation strategy for the case under consideration is based on the assumption that the wavelengths evaluated within the pigment band are collectively characterized by the very specific spatial structure of the specimen, and that since this structure is, of course, marked using the corresponding pigment, the structure is ultimately very similar across all spectral components. Furthermore, a second color, for example, the blue-green spectrum in Figure 11 (pixel region 71), is distinguished by another distinguishable structure of the specimen (e.g., cell nuclei labeled using DAPI). Therefore, it can be assumed that even if they emit slightly different colors, their images will ultimately all have roughly similar structural content. Phase correlation in this case is itself worthy of being considered another embodiment [Castello et al., 2019].

[0127] In this case, first, the pixels of the scanned image are n=(n x ,n y They are numbered and labeled according to ). In this way, N x ×N y An image composed of these image points is g i,j (n), however n x =1,…,N x and n y =1,…,N y It is represented as follows. Furthermore, the so-called correlogram (in this case, one relating to the reference pixel (3,3)) is defined as follows.

number

[0128] FFT and FFT in this case -1 These represent the (fast) Fourier transform and its inverse transform, respectively, according to a known principle. The maximum value of this correlogram yields individual displacement vectors, which in turn push the image content back.

number

[0129] The technical scope of this method is similar to that discussed in [Castello et al., 2019]. Ultimately, this procedure is analogous to what is known as image registration, which yields various pixel positions. The advantage of this evaluation is that, in principle, it is not even necessary to know the variance, and this algorithm can be used to handle variances using other function curves. Furthermore, this method is less dependent on image aberrations on the sensor. However, the information processing expenditure is relatively high.

[0130] In principle, further known methods based on image scanning can also be used for the present invention. Please refer again to [Castello et al., 2019]. What is known as multiview deconvolution in relation to data evaluation is also discussed there and can be used in the present invention. Furthermore, publications on Zeiss® Airyscan® can be helpful.

[0131] In other words, this indicates another way to evaluate data from a confocal spectral sensor using spatial oversampling of the PSF, and consequently another way to simultaneously determine a higher-resolution image with a higher SNR and examine its spectrum.

[0132] As mentioned above, the spectrum itself can always be obtained by summing the pixels within the column, that is, the pixels in the direction perpendicular to the dispersion direction (direction j in Figure 3 above).

[0133] [Application to multicolor excitation] The method according to the present invention can be applied very advantageously to the simultaneous detection of multiple dyes. To achieve this objective, it is advantageous to be able to flexibly define the integration limit, particularly in the dispersion direction 41. In this case, the integration limit means the limit within which the individual spectral contributions of the point spreading function must be summed up with respect to a specific dye. This makes it possible, for example, to individually calibrate the spreading of the point spreading function for each dye. This will be explained in more detail in relation to Figure 9.

[0134] Figure 9a) shows the sum of 1000 simulated recordings from the SPAD camera 50 when exposed to two different dyes, with each individual image having a photon flux of 10 MHz and an exposure duration of 1 μs. Figure 9b) shows the entire spectrum determined based on that image data. Figure 9c) shows the partial spectrum obtained when the integration bandwidth is constrained. And Figure 9d) shows the point spreading function determined in the subregion shown in Figure 9c).

[0135] [Automation Options] The calibration of this system is particularly dependent on the selected objective system and the dye being investigated, and therefore is always strictly applicable only to a fixed preset experiment. For this reason, it is advantageous to retrain the system for calibration after modifying the experiment by evaluating the averaged image data according to Figures 8 and 9 and writing the calibration data to the memory of, for example, the evaluation electronic circuit 60.

[0136] Continuously updating calibration data based on recent (several) LSM image scans is advantageous. This allows the system to respond independently to changes in the experimental environment.

[0137] A further advantage lies in the option of automatically setting the spectral channels by defining the integration limit. This can be achieved by high-resolution sampling of the spectral space at 1 nm or even finer increments. For example, using an algorithm to find the maximum and minimum values, the integration limit, or equivalent spectral channels, can be determined using the integrated signal shown in Figure 9b). That is, this system can efficiently detect radiation from unknown samples and preset advantageous dye separation. Optionally, the column along the y-direction of the matrix sensor 50, onto which light with the wavelength of excitation light 14 is incident, may not be evaluated or may be inactivated. [Explanation of symbols]

[0138] 10. Excitation beam path, 12 light source, (12,3) Reference pixels, 14 Excitation light, 16 deflector mirror, 18 Main color splitter, 22 Scanning device, 23 tube lenses, 24. Microscope objective system, 26 specimen plane, 27 lighting spots, 28 Synchrotron radiation, 29th order diffraction, 30 detection beam paths, 32 detection units, (3,3) Reference pixel, 34 Control and evaluation unit, especially PC, 40 Dispersion device, 41 Dispersion direction, 42. Spectral components of synchrotron radiation 28, 43 lattice, 44. Spectral components of synchrotron radiation 28, 46. ​​Spectral components of synchrotron radiation 28, 47. Spectral components of synchrotron radiation 28, 48. Optical unit for imaging, 50 two-dimensional matrix sensors, 51 Pixels of matrix sensor 50, 53 pixels in size, 54-matrix sensor, 56 Image plane (= plane of matrix sensor 50), 57 Function s 5,4 The center of gravity, 60 Electronic circuits for evaluation, 71 Pixel region of pigment, 72 Pixel regions of pigments, 80. Circularly symmetric point spread function, 100 laser scanning microscopes, 200 detection devices, a Grid constant of matrix sensor 50, d i,j Pixel reassignment displacement vector, d i,j x Displacement vector d i,j The x component of, d i,j y Displacement vector d i,j The y component of, g i,j (n) Image at position n, g i,j ,g i,j (x) Intensity values ​​measured by pixels i and j, g i,j Position vectors to pixels i and j within image plane 56, h em (x) PSF of radiation, h exc (x) Excitation PSF, h i,j (x) Effective PSF for pixels i and j, i Matrix sensor 50 column, (i,j) pixels, j Matrix sensor 50 low, m Number of lows in matrix sensor 50, m i Spectral distribution sp i The minimum value in mx1 spectral distribution sp ithe maximum value in, mx2 spectrum distribution sp i the maximum value in, the number of columns of matrix sensor 50, position vector leading to an image point, n x x-component of position vector n leading to an image point, n y y-component of position vector n leading to an image point, r i,j correlogram related to pixel i,j, s1 emission spectrum of dye, s2 emission spectrum of dye, s i,j effective PSF h i,j position vector leading to the maximum value of (x), sp(λ) spectral distribution (continuous), sp i spectral distribution (discrete), x a location in image plane 56 back-projected into the sample plane, x coordinate direction of matrix sensor 50 (=dispersion direction), y coordinate direction of matrix sensor 50 (perpendicular to the dispersion direction), Airy(x,λ) Airy function, I cal i spatial intensity distribution, I i,j intensity value measured by pixel i,j, N x the number of image points in the x-direction, N y the number of image points in the y-direction, FFT Fast Fourier Transform, FFT -1 Inverse Fast Fourier Transform, P i-n,j overlap data relating to spatial overlap on matrix sensor 50 of components that are spectral components of the dye point spread function and shifted along the dispersion direction 41, PSF Point Spread Function, S sample, SNR Signal-to-Noise Ratio, ξ(λ),ξi(λ) Displacement vector d i,j Wavelength-dependent component, δλ spectral bandwidth of the dye, λ wavelength, λ i Wavelength in column i of matrix sensor 50, λ r The location of the reference pixel or the wavelength in the column, k displacement vector d i,j A constant for modeling the variance related to [the variable].

Claims

1. This method involves detecting synchrotron radiation in a laser scanning microscope, where the synchrotron radiation from a specimen is guided via an imaging optical unit to a two-dimensional matrix sensor having multiple pixels located within the image plane, each of which outputs an intensity value. Using the two-dimensional matrix sensor, the point spreading function of the synchrotron radiation coming from the sample is spatially oversampled. The synchrotron radiation coming from the aforementioned sample is spectrally separated using a dispersion device. The spectrally separated synchrotron radiation is individually detected using the two-dimensional matrix sensor. The intensity values ​​measured by the pixels constituting the pixel region of the point spreading function are evaluated, and the spectral separation is inverted for at least some of those pixels. Includes, When inverting the spectral separation with respect to at least some pixels that constitute a pixel region, the intensity value measured by one of those pixels is assigned to a location within the image plane that is shifted relative to that individual pixel, using a displacement vector that depends on the location of that individual pixel and the wavelength related to that location. A method characterized by the following:

2. The method according to claim 1, further, Identifying at least one pixel region of the point spreading function assigned to the emission of the dye based on the spectrum measured using the two-dimensional matrix sensor, A method characterized by including the following.

3. The method according to claim 1, further, The spectral intensity distribution of the synchrotron radiation on the two-dimensional matrix sensor is determined, and in doing so, the intensity value for a specific wavelength is determined by summing the measurement data of multiple pixels in the column of the two-dimensional matrix sensor perpendicular to the dispersion direction of the spectrally separated synchrotron radiation. A method characterized by including the following.

4. The method according to claim 2, wherein when identifying at least one pixel region, To determine the spectral intensity distribution of the synchrotron radiation on the two-dimensional matrix sensor, Automatically search for the maximum and minimum values ​​in the determined spectral intensity distribution, and To present to the user the spectral limits for calculating the point spreading function for a specific dye, based on the maximum and minimum values ​​found, or, The spectral limits are automatically defined based on the maximum and minimum values ​​found. A method characterized by including the following.

5. The method according to claim 1, Multiple pixel regions overlap on the two-dimensional matrix sensor, and further, The intensity values ​​measured by the individual pixels are separated into component intensity values ​​for each of the pixel regions, and the weights of each component of the spectrum for each of the plurality of pixel regions are determined. A method characterized by including the following.

6. The method according to claim 1, When performing the aforementioned evaluation, the point spreading function is determined for at least one fluorescent dye. A method characterized by including the following.

7. The method according to claim 1, further, To simultaneously guide synchrotron radiation emitted by multiple points on a specimen that are simultaneously illuminated by excitation light to the two-dimensional matrix sensor and evaluate the synchrotron radiation, A method characterized by including the following.

8. The method according to claim 1, The two-dimensional matrix sensor operates in photon counting mode. A method characterized by the following:

9. The method according to claim 1, When inverting the spectral separation for at least some pixels forming a pixel region, the intensity values ​​measured by those pixels are combined based on the spectral intensity distribution of the synchrotron radiation related to the dye in that pixel region and the spatial intensity distribution of the spectral components of the individual dyes on the two-dimensional matrix sensor. A method characterized by including the following.

10. The method according to claim 9, The intensity distribution measured by pixels forming a column perpendicular to the dispersion direction of the spectrally separated synchrotron radiation is used as the spatial intensity distribution of the spectral components of the individual dyes. A method characterized by the following:

11. The method according to claim 1, further, By scaling the vector component of the vector from the reference pixel to its related pixel using a reallocation factor, the wavelength-independent component of the displacement vector is obtained with respect to a specific pixel. A method characterized by including the following.

12. The method according to claim 1, The point spreading function obtained after performing pixel reassignment has the same shape along the dispersion direction of the spectrally separated synchrotron radiation and along the direction perpendicular to that dispersion direction. A method characterized by the following:

13. The method according to claim 1, further, The displacement vector, which relates to the wavelength range assigned to the sample structure, is determined by evaluating the phase correlation of multiple scan images. A method characterized by including the following.

14. The method according to claim 1, Time-resolved measurements to determine the fluorescence lifetime of the dye are performed using the two-dimensional matrix sensor. A method characterized by the following:

15. This is a detection device that detects synchrotron radiation within a laser scanning microscope. A two-dimensional matrix sensor having a plurality of pixels located within the image plane, each of which outputs an intensity value, wherein the point spreading function of the synchrotron radiation coming from a sample is detected as spatial oversampling and the synchrotron radiation is guided to the two-dimensional matrix sensor by an imaging optical unit, A dispersion device for spectrally separating the synchrotron radiation, An evaluation electronic circuit connected to the two-dimensional matrix sensor, which evaluates the intensity values ​​measured by the pixels that make up the pixel region of the point spreading function, and in the process inverts the spectral separation for at least some of those pixels, Equipped with, The two-dimensional matrix sensor is configured and arranged to individually detect the spectrally separated synchrotron radiation. The evaluation electronic circuit is configured such that, in order to invert the spectral separation for at least some of the pixels that make up the pixel region, the intensity value measured by one of those pixels is assigned to a location within the image plane that is shifted relative to that individual pixel, using a displacement vector that depends on the location of that individual pixel and the wavelength related to that location. A detection device characterized by the following features.

16. A detection device according to claim 15, The evaluation electronic circuit is configured to invert the spectral separation for at least some of the pixels that make up the pixel region, by combining the intensity values ​​measured by those pixels based on the spectral intensity distribution of the synchrotron radiation relating to the dye in that pixel region and the spatial intensity distribution of the individual spectral components on the two-dimensional matrix sensor. A detection device characterized by the following features.

17. A detection device according to claim 15, The dispersion device comprises a light diffraction and / or light refraction device. A detection device characterized by the following features.

18. A detection device according to claim 15, The two-dimensional matrix sensor comprises an analog integrating detector and / or a photon counting detector. A detection device characterized by the following features.

19. A detection device according to claim 15, The dispersion direction of the spectrally separated synchrotron radiation is parallel to the two-dimensional matrix sensor. A detection device characterized by the following features.

20. A detection device according to claim 15, To improve detection efficiency, a microlens is positioned upstream of the two-dimensional matrix sensor. A detection device characterized by the following features.

21. A detection device according to claim 15, The diameter of the airy disk of the point spread function in the plane of the two-dimensional matrix sensor is less than 20 times the lattice constant of the two-dimensional matrix sensor. A detection device characterized by the following features.

22. A detection device according to claim 15, The spectral bandwidth per pixel of the two-dimensional matrix sensor in the dispersion direction of the spectrally separated synchrotron radiation is less than 0.5 nm. A detection device characterized by the following features.

23. A detection device according to claim 15, The aforementioned imaging optical unit includes a zoom system. A detection device characterized by the following features.

24. A detection device according to claim 15, The two-dimensional matrix sensor and the evaluation electronic circuit are configured to perform time-resolved measurements in order to determine the fluorescence lifetime of various dyes. A detection device characterized by the following features.

25. A light source that emits excitation light, An excitation beam path, accompanied by a microscope objective system, guides the excitation light onto or into the specimen under investigation, A scanning device located on the excitation beam path and operating to scan the specimen with at least one illumination spot, A detection beam path guides the synchrotron radiation emitted by the aforementioned sample to a detection unit, The detection unit for detecting the synchrotron radiation, A main color splitter separates the excitation light and the synchrotron radiation, A control and evaluation unit that controls the light source and evaluates the measurement data obtained by the detection unit, A laser scanning microscope equipped with, The detection unit comprises the detection device described in claim 15. A microscope characterized by the following features.

26. A microscope according to claim 25, The aforementioned control and evaluation unit It is configured to search for the maximum and minimum values ​​in the determined spectral distribution, and, To present to the user the spectral limits for calculating the point spreading function for a specific dye, based on the maximum and minimum values ​​found, or, The system is configured to independently define the spectral limits for calculating the point spreading function for a specific dye, based on the maximum and minimum values ​​found. A microscope characterized by the following features.

27. A microscope according to claim 25, The control and evaluation unit is configured to detect the synchrotron radiation using the detection device, and when detecting it, The synchrotron radiation coming from the aforementioned sample is spectrally separated using a dispersion device. The spectrally separated synchrotron radiation is individually detected using the two-dimensional matrix sensor. The intensity values ​​measured by the pixels constituting the pixel region are evaluated, and at that time, the spectral separation is inverted for at least some of those pixels. A microscope characterized by the following features.

28. This method involves detecting synchrotron radiation in a laser scanning microscope, where the synchrotron radiation from a specimen is guided via an imaging optical unit to a two-dimensional matrix sensor having multiple pixels located within the image plane, each of which outputs an intensity value. Using the two-dimensional matrix sensor, the point spreading function of the synchrotron radiation coming from the sample is spatially oversampled. The synchrotron radiation coming from the aforementioned sample is spectrally separated using a dispersion device. The spectrally separated synchrotron radiation is individually detected using the two-dimensional matrix sensor. The intensity values ​​measured by the pixels constituting the pixel region of the point spreading function are evaluated, and the spectral separation is inverted for at least some of those pixels. Includes, When inverting the spectral separation with respect to at least some pixels that make up the pixel region, the intensity value measured by one of those pixels is combined based on the spectral intensity distribution of the synchrotron radiation related to the dye in that pixel region and the spatial intensity distribution of the spectral components of the individual dyes on the two-dimensional matrix sensor. A method characterized by the following:

29. This is a detection device that detects synchrotron radiation within a laser scanning microscope. A two-dimensional matrix sensor having a plurality of pixels located within the image plane, each of which outputs an intensity value, wherein the point spreading function of the synchrotron radiation coming from a sample is detected as spatial oversampling and the synchrotron radiation is guided to the two-dimensional matrix sensor by an imaging optical unit, A dispersion device for spectrally separating the synchrotron radiation, An evaluation electronic circuit connected to the two-dimensional matrix sensor, which evaluates the intensity values ​​measured by the pixels that make up the pixel region of the point spreading function, and in the process inverts the spectral separation for at least some of those pixels, Equipped with, The two-dimensional matrix sensor is configured and arranged to individually detect the spectrally separated synchrotron radiation. In order to reverse the spectral separation, the evaluation electronic circuit is configured to combine the intensity values ​​measured by at least some of the pixels constituting the pixel region based on the spectral intensity distribution of the synchrotron radiation related to the dye in that pixel region and the spatial intensity distribution of the spectral components of the individual dyes on the two-dimensional matrix sensor. A detection device characterized by the following features.

Images