Optical microscope with reconfigurable sensor array

Abstract

To provide an optical microscope, an optical detector, and a method for imaging which can provide an image with an extremely high quality without causing an unreasonably high optical complicity and cost.SOLUTION: The optical microscope includes: a light source for radiating a sample; a sensor array 60 made of photon counting detector elements 61, 62, for measuring detection light from the sample; and a controller for controlling the sensor array 60. The controller is configured to flexibly bin a binning photon counting detector element 62 to at least one super pixel 65.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to an optical microscope according to the preamble of claim 1. The present disclosure also relates to an optical detector according to the preamble of claim 25 and to an imaging method according to the preamble of claim 28. [Background technology]

[0002] Optical microscopes are used in a wide range of applications, such as life sciences or materials testing. In particular, confocal scanning microscopy is a well-established technique. Current sensor technologies employed in confocal scanning microscopes include photomultiplier tubes with multialkali or GaAsP photocathodes, or hybrid detectors with GaAsP photocathodes and APD (avalanche photodiode) detectors.

[0003] Recent advances include single-photon avalanche diodes (SPADs), which offer high sensitivity, fast response times, and low dark count rates. SPAD arrays allow spatial sampling of point spread functions (PSFs) that can be used in image scanning techniques (Airyscan) based on the work of Sheppard et al. See Sheppard, CJ, Optic 80, 53-54 (1988), and Sheppard, CJ, Mehta, S.B. & Heintzmann, R. Opt Lett. 38, 2889-2892 (2013). In certain applications, the PSF is axially symmetric, but in general, any light distribution may be used.

[0004] A photon-counting detector element, or SPAD, comprises a junction with a depletion region lacking free charge carriers. A voltage Vop exceeding the breakdown voltage is applied to the junction. Photons absorbed by the SPAD may be able to inject carriers in the depletion region, resulting in impact ionization that can disperse to further regions of the diode and cause an avalanche that can be detected as photon counting.

[0005] To detect a subsequent photon, the avalanche is quenched. Once the diode is quenched (i.e., there is no current due to impact ionization and the absence of free carriers in the diode), the voltage across the diode is recharged, for example, by current flow through a quench resistor, and the diode is ready to detect another carrier.

[0006] A typical optical microscope includes a light source for illuminating a specimen and a sensor array of photon-counting detector elements for measuring detected light emanating from the specimen. A controller is configured to control the sensor array. Optical elements may be arranged to direct the illumination light from the light source to the specimen and the detection light from the specimen to the sensor array. Similarly, a typical optical detector includes a sensor array of photon-counting detector elements and a controller for controlling the sensor array. A typical imaging method includes illuminating a specimen with illumination light, measuring the detected light emanating from the specimen by the sensor array comprising photon-counting detector elements, and controlling the sensor array with the controller.

[0007] US2016 / 0131883A1 describes a laser scanning microscope with multiple avalanche photodiodes or PMTs. Optical fibers are used to direct the detection light to the individual photodiodes. The PSF is spatially resolved, thereby enabling the implementation of the image scanning techniques described above. As a result of the relatively small number of photodiodes and optical fibers used to direct the light to the photodiodes, zoom optics are typically used to adapt the size of the PSF to the limited number of optical fibers and sensor pixels. The zoom optics required to adapt the size of the PSF are quite complex, and the large number of required optical elements can adversely affect signal detection efficiency while increasing costs.

[0008] US 2017 / 0176250 A1 describes, among other things, a distance measurement sensor array that includes SPAD elements. In this case, the output lines of all SPAD elements of the sensor array are combined into a single output. Thus, the sensor array outputs a detection signal each time any one of the SPAD elements measures an impinging photon. While this design provides accurate results for certain applications, the binning of the output lines into a single output makes this sensor array unusable for many microscopy techniques, such as the PSF oversampling technique mentioned above.

[0009] It is an object of the present invention to provide an optical microscope, optical detector and imaging method that provides very high image quality without undue optical complexity and cost. Summary of the Invention

[0010] The above mentioned object is achieved by an optical microscope having the features of claim 1, an optical detector having the features of claim 25 and a method as described in claim 28.

[0011] Preferred embodiments are presented in the following description, in conjunction with the dependent claims and in particular the accompanying drawings.

[0012] According to the invention, an optical microscope of the above mentioned kind and an optical detector of the above mentioned kind are characterized in that the control device is configured for flexible binning of the photon-counting detector elements into one or more superpixels.

[0013] According to the present invention, the above-mentioned method is characterized by at least the step of flexibly binning the binned photon-counting detector elements into one or more superpixels, which method can be implemented in particular using the embodiments of the optical microscope according to the present invention described herein.

[0014] In particular, the preferred embodiments described below that describe the sensor array, the control device, the optical elements or the light source may be applied to both the optical microscope and the optical detector of the present invention.

[0015] Conventionally, complex zoom optics are used to adjust the size of the light beam impinging on the sensor array so that the light beam fully fills the sensor array. In stark contrast, the present invention allows the sensor array to adjust to the size and characteristics of the impinging light spot. In PSF oversampling mode, the size of the light beam (or light spot) impinging on the sensor array is defined by the PSF. The spatially resolved PSF enables the acquisition of information necessary for high-resolution techniques. While a sufficient number of detection points is necessary for the spatially resolved PSF, an unnecessarily large number of detection points will exceed reasonable data rate constraints. Flexible binning of the output lines of photon-counting detector elements into an indefinite number of superpixels solves this problem. The larger the PSF (light spot impinging on the detector array), the larger the superpixels that can be set. In the case of a small light spot, the detector elements impinging on the light can be grouped into small superpixels (or some detector elements may not be binned into any superpixels), while other detector elements impinging on low-intensity light or not impinging on light can be binned into large superpixels or disabled. The inventive concept allows for accurate measurement of light distribution, while reducing the requirements for zoom optics (e.g., zoom magnification), and at the same time, the inventive concept may offer advantages in maximum count rate and signal-to-noise ratio (SNR).

[0016] Specifically, based on CMOS SPAD technology, each detector element can be addressed individually, and the sensor array can be reconfigurable so that all individual pixels (i.e., output from the photon-counting detector elements) can be combined (binned) into any super-pixel. Binning is therefore, in principle, independent of the position of the detector element. Thus, the controller may be configured for flexible binning of consecutive (adjacent) and / or non-consecutive photon-counting detector elements into the same super-pixel.

[0017] In a preferred embodiment, event-driven effective recharge is used for photon-counting detector elements. Corresponding event-driven effective recharge (electronic) components may be considered part of the controller and / or sensor array. Event-driven effective recharge allows each detector element to be recharged individually. That is, not all detector elements are recharged simultaneously; instead, only the specific detector element that undergoes an avalanche event is recharged. This results in a "non-nullifying" sensor response to light, extending the dynamic range compared to passive, clock-driven recharge SPADs. Further explanation is provided in "Dynamic range extension for photon counting arrays" by Ivan Michel Antolovic et al., published August 20, 2018, Vol. 26, No. 17, OPTICS EXPRESS 22234.

[0018] Additional advantages are achieved when the flexibility of a reconfigurable sensor array relates not only to binning but also to the activation of individual detector elements. "Activated" or "activated" herein refers to a detector element that outputs a measurement signal (photon count rate) for further processing, while a disabled detector element is understood to not output a measurement signal for further use. Alternatively, "deactivated" may refer to a detector element whose output is not routed for further processing. When the light intensity impinging on some detector elements is low, enabling those detector elements may actually worsen the overall SNR due to background noise. In this case, disabling the associated detector elements may be preferable to combining them into one or more large superpixels. For low detected light power, it may be preferable to focus the detected light into a small spot and activate only the respective detector element. Accuracy can be improved by increasing the spot size and correspondingly increasing the active area as the intensity increases. In other words, the controller may be configured to control the optical elements to reduce the detected light spot size and the active area on the sensor array as the detected light power decreases. With increasing detected optical power, more detector elements may be enabled to increase the maximum count rate and SNR through a larger photon flux.

[0019] A superpixel is understood to be a combination of the outputs of one or more (or two or more) detector elements. For simplicity of explanation, this disclosure often refers to binning into superpixels, including the option of all detector elements being binned into superpixels, but may also include the option of some (effective) detector elements not being binned.

[0020] The controller may be configured to assign a respective multi-bit counter to each superpixel. The multi-bit counters reduce the data rate compared to transmitting measurement data from each detector element individually. Each multi-bit counter outputs a cumulative number indicative of the photons counted by all detector elements binned into the respective superpixel. Thus, the data rate can be reduced by a factor of X [bits / sec] to X N / (2 N −1) [bits / sec], where X denotes the superpixel detection speed and N denotes the number of bits in the multi-bit counter.

[0021] The controller (or parts of the controller) and the sensor array may be constructed on the same circuit board and / or in the same IC (integrated circuit). Specifically, parts of the controller described above may be constructed with the sensor array in the same IC, while other parts of the controller are formed external to the IC but on the same circuit board as the IC. For example, a multi-bit counter may be formed as an on-chip multi-bit counter. Each photon-counting detector element may be connected to the controller (i.e., to the multi-bit counter or to a pre-processing unit before reaching the multi-bit counter) via a respective signal line. This reduces the maximum required data bandwidth of the controller.

[0022] The control device may variably set the number of superpixels and / or the number of pixels binned into one superpixel. The different superpixels used simultaneously may also be of different sizes.

[0023] An exemplary electronic aspect of the control device is described below. The control device may be configured to flexibly configure a number of superpixels up to M, and for that purpose, the control device comprises the same number of processing units as M units. Each detector element is connected to each unit. If the number of detector elements is N, each unit may have N input lines, each unit receiving the photon count rate of one detector element (more generally, each unit may be connected to at least some, but not all, detector elements). Each unit comprises multiple switches (specifically, one switch per detector element) that flexibly control whether the photon count rate output by a particular detector element is forwarded or blocked through the switch. Each unit comprises a combining circuit that switches the connection. Depending on the state of the switch, it is possible to flexibly control which detector elements are connected to the same combining circuit. Each combining circuit outputs a data stream indicative of all photon count rates of those detector elements for which the respective switch connected to this combining circuit is closed.

[0024] Each switch may be controlled through a memory, specifically a respective one-bit memory, with two possible memory states defining whether the switch should be closed or open.

[0025] The control device may further comprise a superpixel counter unit having a plurality of individual multi-bit counters. Each combining circuit is connected to at least one of the multi-bit counters. The multi-bit counters thus output a number corresponding to the photon count rates of all detector elements connected to the respective combining circuit. In a further variation, each combining circuit is connected to a pair of multi-bit counters. Thus, the total number of multi-bit counters must be at least 2M when there are M combining circuits. The pair of multi-bit counters operate in alternating phases, i.e., one of the multi-bit counters counts the photon count rate incident from the combining circuit while the other multi-bit counter is read out, and vice versa. In this way, time delays due to readout are reduced or avoided.

[0026] To provide up to M superpixels, the control device may comprise a number M of individual combining circuits and at least a number M (preferably 2M) of multi-bit counters. Each combining circuit comprises up to N programmable switches connected to the detector elements, where N denotes the number of detector elements. M is less than the number of programmable switches and less than N. Thus, the total number of 1-bit memories associated with the programmable switches may be NM. In operation, fewer than M superpixels may be desired, in which case a correspondingly reduced number of multi-bit counters are read out.

[0027] The controller may set the binning and / or active area of ​​the sensor array according to the light spot size information, i.e., each detector element may be individually enabled and disabled. The light spot size information may generally be information or an assumption indicating the expected light distribution or size on the sensor array. The information may be based on past measurements or reference measurements. Additionally or alternatively, the light spot size information may be derived (at least in part) from the currently used illumination wavelength and / or the currently used objective lens. The illumination wavelength affects the PSF size; specifically, the PSF size may increase with increasing illumination wavelength. A detailed description of how the active area and binning are set according to the PSF size (and thus according to the wavelength) is provided below. The PSF size is also affected by the back aperture diameter of the objective lens. The smaller the back aperture diameter, the larger the PSF size. In some embodiments, with increasing illumination wavelength and / or when the objective lens is changed to one with a smaller back aperture diameter, the active area of ​​the sensor array is increased and / or the (average) number of detector elements per superpixel is increased, or vice versa. That is, as the illumination wavelength decreases and / or the objective lens is changed to one with a larger back aperture diameter, the active area and / or the (average) number of detector elements per superpixel are reduced. Because not only the wavelength but also the back aperture diameter affect the PSF size, it may be beneficial to adjust the active area and superpixels depending on these factors. Additionally or alternatively, the active area and / or the average number of detector elements per superpixel may be set depending on the numerical aperture at which the sample portion is imaged onto the sensor array. The numerical aperture also affects the detected light spot size on the sensor array. By adjusting the average number of detector elements associated with the same superpixel, the transmitted data stream remains manageable, specifically, constant, as described below.

[0028] The control device may be configured to provide a plurality of imaging modes to a user. The user may select one of the imaging modes, for example, via a computer. The imaging modes include at least two of a PSF oversampling mode, a line (or array) scanning mode, a field of view imaging mode, and a confocal detection mode, as described below. Binning and optionally an effective sensor area are set according to the selected imaging mode. Additionally, optical elements are adjusted according to the imaging mode, as further described below.

[0029] PSF oversampling mode (image scanning mode) The control device may be configured to implement a PSF oversampling mode (also referred to as an image scanning mode). Illumination light is focused on a sample point, and detection light (e.g., fluorescence) emanating from the sample point is focused onto a sensor array located in a plane conjugate to the illuminated sample plane. A pinhole may be located in the pupil plane. The detection light forms a detection light spot on the sensor array. The size and intensity distribution of the light spot represent the PSF. The diameter of the light spot can be understood as one Airy disk diameter. In the PSF oversampling mode, superpixels may be configured so that the center-to-center distance between adjacent superpixels is a maximum of 0.2 Airy disk diameters or less. This ensures a sufficient number of detection points for the spatially resolved PSF. Additionally, the effective area of ​​the sensor array is configured according to the PSF size. In some embodiments, the effective sensor area increases as the PSF size increases. Specifically, the effective area may be configured to be equal in size or diameter (within a 10% or 20% error) to the PSF size (i.e., the size of the PSF or light spot on the sensor array). Additionally or alternatively, the number of superpixels may be set according to the PSF size. The number of photon-counting detector elements binned into the same superpixel may also be set according to the PSF size. As the PSF size increases, the number of detector elements coupled to the same superpixel can be increased, particularly to avoid excessively high data rates. The number of coupled detector elements may be adjusted so that the data rate output by the control device (i.e., data including the acquired measurement data of the superpixels) does not exceed a predetermined maximum rate and / or is constant, i.e., strictly constant or constant within a predetermined range, e.g., 10% or 20%. This ensures a sufficient number of data points to sample / spatially resolve the PSF while avoiding excessive data rates. The number of detector elements coupled to the same superpixel may also be set according to the desired maximum count rate. The larger the desired maximum count rate, the more detector elements can be coupled to the same superpixel. If a lower limit on the count rate is not considered, the maximum count rate can be considered as the dynamic range of the sensor.However, the dark count rate may define a lower limit for the photon count rate of an individual detector element, so combining several detector elements increases the minimum count rate, increasing the maximum count rate but not the dynamic range.

[0030] The superpixels used together may be of different sizes and shapes: if the PSF is (or is predicted to be) largest at the center, one or more central superpixel(s) may be smaller than the outer superpixels.

[0031] Line scan mode and array scan mode The controller may also be configured to implement a line scan mode (or, more generally, an array scan mode). In line scan mode, the specimen is illuminated with an elongated illumination light spot, for example, by using a cylindrical lens to focus the illumination light onto the specimen. This illumination directs an elongated detection light spot onto the sensor array. The controller bins the photon-counting detector elements together so that elongated superpixels are formed transversely or perpendicularly to the elongated detection light spot. For example, if the elongated detection light spot forms a column on the sensor array, detector elements in one or more rows are binned into the same superpixel (rows and columns are orthogonal to each other). Detector elements not illuminated by the elongated detection light spot may be disabled. Thus, the binning pattern used in line scan mode may vary significantly from the binning pattern used in PSF oversampling mode.

[0032] In line scan mode, an elongated illumination light beam is scanned over the specimen, with the scan direction being transverse or perpendicular to the longitudinal dimension of the elongated illumination light spot, and several measurements are recorded successively during this scan using the sensor array, and an image of the specimen is calculated from these measurements.

[0033] In principle, illumination shapes other than elongated illumination light spots can be used while offering similar advantages as line scan modes in terms of reducing scanning and image acquisition times. Such array scan modes use illumination light patterns other than dots or lines. For example, a grid or multiple lines may be used as the illumination light pattern that is then scanned onto the specimen.

[0034] In array scanning mode, multiple illumination light spots are scanned simultaneously across the specimen. The illumination light spots may be circular, elongated, or have almost any other shape.

[0035] Field of view imaging mode The field-of-view imaging mode may be used, for example, to record an overall image with reduced resolution or SNR compared to specimen images recorded in other described imaging modes. In the field-of-view imaging mode, a wide-field image of the specimen is recorded using a sensor array. Thus, optical elements may be removed or inserted into the illumination and / or detection beam paths to ensure that the specimen plane (rather than the pupil plane) is imaged onto the sensor array, rather than simply illuminating a specimen point. The active area may span the entire sensor array or may be set according to the numerical aperture or zoom setting. Binning may be used to reduce the output data rate. In a variation of the field-of-view imaging mode, several wide-field images of different specimen portions are recorded consecutively and then stitched together to form the overall image.

[0036] Confocal detection mode The controller may also be configured to perform a confocal detection mode without PSF oversampling. In confocal detection mode, illumination light is focused onto a sample point, and detection light emanating from this sample point is directed to the sensor array. The same objective lens is used to focus illumination light onto the sample and receive detection light from the sample. A pinhole may be located at the pupil plane. It may be desirable to measure all detection light impinging on the detector array as precisely as possible without requiring spatial resolution. In this case, all enabled photon-counting detector elements may be binned into one superpixel. Because no PSF oversampling is performed in this mode of operation, one superpixel is sufficient. Nevertheless, using multiple photon-counting detector elements (SPADs) and binning them into one superpixel offers significant advantages over conventional detectors. SPADs are ideal for measuring low light intensities. At high optical powers, a single SPAD may saturate. During this dead time (when the voltage after photon detection has not yet recovered to exceed the diode breakdown voltage), a single SPAD cannot detect another photon. However, the maximum count rate can be increased by using a sensor array with multiple SPADs. Thus, the SPAD array can provide increased accuracy without significant drawbacks in dynamic range or maximum count rate.

[0037] Optical element adjustment The control device may also be configured to adjust optical elements, such as lenses and mirrors, depending on the selected imaging mode. In confocal detection mode, optical elements may be adjusted, e.g., moved or deformed, to homogenize the intensity distribution on the sensor array. Specifically, optical elements may be incorporated into the beam path of the detection light to image the pupil on the sensor array. Furthermore, optical elements may adjust or move to adjust the size of the detection light spot on the sensor array. Thus, the optical system is configured to switch between imaging the specimen plane and imaging the pupil on the same sensor array. The pupil image may have a more uniform intensity distribution than the specimen image (e.g., when only a specimen point is illuminated). Therefore, it may be preferable to image the pupil on the sensor array. Because a SPAD array is used, a uniform intensity distribution without excess intensity peaks on individual detector elements is more appropriate compared to conventional structures. When the impinging optical power is relatively high, the pupil image should uniformly fill the sensor area to optimize SNR. When the optical power of the impinging detection light is relatively low, it may be preferable to concentrate the light on a small number of detector elements to increase the intensity on the illuminated detector elements. Otherwise, weak signal intensities would be overwhelmed by background noise at the detector elements. Therefore, at least within a certain power range of the detected light, the controller may adjust the detected light spot size on the sensor array to increase with increasing beam power. The active area of ​​the sensor array is adjusted correspondingly with this spot size adjustment.

[0038] In confocal and / or PSF oversampling imaging modes, adjustment of the detected light spot size may also be performed to avoid saturation of the detector elements or to increase the SNR: if the intensity on the detector elements exceeds a predetermined threshold (e.g., a value above which the output photon count rate is no longer linearly proportional to the impingement intensity), the detected light spot size may be increased to reduce the impinging light power per detector element.

[0039] To adjust the size of the detection light spot (or pupil image), the imaging lens may be adjusted to focus the detection light near the sensor array. For example, the imaging lens may be moved to displace the focal plane relative to the sensor array. The further the sensor array is out of focus, the larger the detection light spot. Depending on the design of the imaging lens, the imaging lens may be deformed or otherwise adjusted instead of moved, for example in the case of an adaptive liquid lens.

[0040] In contrast to the confocal imaging mode, in the PSF oversampling mode, the optical elements are adjusted to image a sample point onto the sensor array and record spatial information about the PSF. Therefore, changing from the confocal detection mode to the PSF oversampling mode may imply a change from pupil imaging to sample plane imaging on the sensor array. Such a change can be effected by inserting or removing a Bertrand lens in the beam path of the detection light, for example, by moving the Bertrand lens or redirecting the detection light to the Bertrand lens. Alternatively, a phase mask for setting the phase distribution or an SLM (Spatial Light Modulator) may be placed in the illumination or detection pupil plane. The SLM may be formed, for example, by an array of adjustable microlenses, a diffractive optical element, an adjustable mirror such as a digital micromirror device, or a controllable liquid crystal array. The SLM may also adjust the spot size and / or intensity distribution.

[0041] Additionally, optical elements in the illumination beam path may be adjusted or inserted depending on the selected imaging mode: for example, in PSF oversampling or confocal imaging modes, the illumination light is focused onto a sample point, whereas wide-field imaging modes require a larger illuminated sample area, and line-scanning modes may require the introduction of a cylindrical lens.

[0042] FLIM regulation The microscope of the present invention is also suitable for FLIM (fluorescence lifetime imaging microscopy). The SPAD provides the high temporal resolution desired for FLIM measurements. To determine the fluorescence lifetime, the fluorophore is excited with repeated light pulses. After each pulse, the time it takes for a photon to reach the sensor array is determined. A histogram of photon arrival times is then generated, from which the fluorescence lifetime is obtained. It would be beneficial to use high-intensity pulses to measure a large number of photons and reduce the required measurement time. However, with increasing intensity, a second photon may arrive at the same SPAD shortly after it registers the first. Detection of the first photon is accompanied by a dead time during which the second photon is not noticed by the SPAD. Thus, the photon arrival time histogram is skewed or biased toward short time periods, known as the pile-up effect. This drawback can be avoided in the microscope of the present invention by homogenizing the intensity distribution and increasing the light spot diameter on the sensor array. These measures reduce the chance that a second photon will strike the SPAD during the dead time. The controller may be configured to provide FLIM as an additional imaging mode in which the detected light spot size increases with increasing detected light intensity, and optionally pupil imaging is performed to homogenize the intensity distribution. The superpixel may be connected to a time-to-digital converter to detect multiple photons per laser period.

[0043] Another embodiment The controller may be configured to prompt the user to input imaging parameters, particularly the desired resolution, frame rate, and / or SNR. In response to the user's input, the controller then adjusts the binning and effective area of ​​the sensor array and, optionally, the optical elements as described above. Specifically, the binning and effective area may be set to limit the resulting data rate depending on the desired frame rate. Because the bandwidth for transmitting data from the controller is known, the desired frame rate can be translated into an upper limit for the data rate output from the controller. The binning and effective area affect the amount of resulting data and can therefore be adjusted to comply with the data rate requirements.

[0044] An additional advantage is that adaptive binning balances spatial resolution and acquisition speed. To benefit from the increased resolution achievable with image scanning microscopes, a certain SNR is required. If the signal strength (e.g., the intensity on the sensor array) is too low, traditionally the acquisition speed must be reduced to increase the SNR. However, adaptive pixel binning can reduce the resolution in favor of a desired acquisition speed. The signal strength can be, for example, the photon count rate and can be obtained from a previous measurement (during the scanning of the specimen) or a reference measurement.

[0045] The controller may also be configured to keep the output data rate constant (i.e., strictly constant or constant within a predetermined limit, e.g., 10% or 20%) by adjusting the binning, particularly when the active area of ​​the sensor array is changed. For example, a change in the objective lens may lead to a change in the size of the detection light spot on the sensor array, and the active area may therefore be adjusted to the new light spot size, thereby adjusting the binning to the new active area size, particularly to keep the data rate constant. The number of superpixels may be kept constant, but different binning patterns may be used. As the number of bits in the multi-bit counter for the superpixels changes, the number of superpixels may also change to ensure a constant data rate.

[0046] As light intensity increases, the probability that photons will strike a detector element during the dead time and remain unnoticed increases. Thus, measurements from the detector element are adversely affected by extremely high intensities. To address this issue, the sensitivity of the detector element may be adjusted, for example, by adjusting the excess voltage above the breakdown voltage. A light spot impinging on the sensor array typically has a non-uniform light distribution. Specifically, in PSF oversampling mode, the impinging light spot has maximum intensity in the center and decreases in intensity toward the outer regions. Therefore, the sensitivity of the central detector element may be set to a different level than the sensitivity of the outer detector elements. Specifically, different sensitivities may be set between photon-counting detector elements of the same superpixel. The sensitivity may be set according to the expected intensity distribution on the sensor array, for example, based on assumptions or reference measurements about the PSF.

[0047] Different lengths of signal lines from the detector elements to the controller may introduce timing delays in the output data streams of different superpixels. The controller may be configured to adjust or compensate for these timing delays. A calibration matrix may be stored in the controller's storage and used for this purpose.

[0048] The control device may have multiple physical output lines, each associated with a respective superpixel. The number of output lines may then correspond to the maximum number of simultaneously usable superpixels. In one embodiment, the number of output lines may be 10% to 40% of the number of detector elements.

[0049] Two or more light spots can be measured simultaneously on a photon-counting detector array. Different PSFs can be applied to different light spots, and therefore the light spots can have different sizes on the sensor array. A superpixel for a large light spot can be configured with more detector elements than a superpixel for a small light spot.

[0050] A scanner may be provided and configured to scan the specimen with illumination light and / or direct detection light toward the sensor array. Binning may be adjusted during scanning, for example, depending on the amount of light emanating from a particular specimen region. While scanning continues, binning may be adjusted using information obtained from the scan. Such on-the-fly adaptation may also adjust other parameters, such as, for example, SPAD sensitivity and illumination light intensity.

[0051] The optical microscope described above may in particular be formed as a (laser) scanning microscope. Alternatively, or optionally, it may be configured as a wide-field microscope. The sensor array and control unit described above realize the above-mentioned advantages in many applications in which any object is illuminated and the detected light from the object is measured. The invention may also be applied to other sensor devices that do not necessarily require a microscope, such as material analysis, camera or surveillance systems, astronomy or production monitoring, etc.

[0052] An optical microscope may be defined by having a specimen holder and an objective lens, particularly an infinity objective lens that sets the image distance to infinity. An optical microscope may further include a tube with an imaging lens positioned in the beam path after the objective lens, i.e., between the objective lens and the detector array, to focus the light from the objective lens onto an (intermediate) image plane. Unlike other optical systems, an optical microscope generates at least one intermediate image plane. An optical microscope may also include an illumination port to which a light source is connected, e.g., one or more lasers. The detected light emitted from the specimen may be any type of light, such as fluorescence or phosphorescence, illumination light scattered or affected by the specimen through other mechanisms, or light emitted from the specimen for other reasons that may be at least partially caused by the illumination light. Generally, specimen light may also be emitted by processes unrelated to illumination.

[0053] The optical microscope may include an objective lens arranged to direct or focus illumination light onto the specimen. The objective lens may also be specifically arranged to receive detection light emitted from the specimen and direct the detection light toward the sensor array. Generally, separate illumination and detection objective lenses may be used. The optical microscope may further include a scanner arranged between the objective lens and the illumination source. The controller controls the scanner to perform a specimen scan on the specimen. Optionally, the scanner may also direct the detection light from the objective lens toward a photon-counting detector array (non-scanning configuration). The scanner may be understood as a device configured to adjustably deflect a light beam. The scanner may include one or more movable optical elements, such as a mirror, a lens, or a prism. Alternatively, the scanner may also adjustably deflect illumination light based on the acousto-optic effect.

[0054] The PSF can be understood to define how illumination and detection light are guided by an optical microscope or optical system. Specifically, the PSF can be considered to consist of an illumination PSF and a detection PSF, where the illumination PSF defines how a point of the light source is imaged onto the specimen plane, and the detection PSF defines how a point from the specimen plane is imaged onto the plane of the sensor array. As used herein, the detection spot size on the detector array can also be referred to as the PSF size in image scanning techniques or confocal imaging.

[0055] The photon-counting detector elements, also referred to as "detector elements" for simplicity, may in particular be SPADs (single-photon avalanche diodes) that operate in the so-called Geiger mode. In Geiger mode, a voltage V is applied to the diode of the SPAD by an excess bias voltage, exceeding the breakdown voltage of the diode. As a result, photon absorption can cause a charge avalanche and thus a countable event. Accordingly, the sensor array may be referred to as a SPAD array.

[0056] The controller may include electronic components such as an FPGA or processing unit that may be formed as a single unit or a distributed system. The functionality of the controller may be implemented as software and / or hardware. The controller or parts thereof may be specifically located as an on-chip unit adjacent to the sensor array. Portions of the controller may also be provided via a server or computer application that communicates with other components of the optical microscope over a network.

[0057] In scanning mode, a sample point may be defined as the illuminated portion of the specimen where the detector elements count photons during the pixel dwell time, after which the scanner illuminates another portion of the specimen, defined as the next sample point.

[0058] For clarity, the expression "light spot" is often used in this disclosure to refer to a light distribution on a specimen or a sensor array. More generally, a "light spot" may be understood as a light distribution, for example, a ring-shaped pattern, one or more lines, or a spot or ring.

[0059] The detectors of the present invention may be used in the optical microscopes described herein. Alternatively, the detectors may be part of other light measurement devices for distance measurement, quality control, and surveillance, for example, in portable / handheld devices, medical devices, or vehicle sensors. Optionally, the detectors may be used in combination with the optical elements described herein and a light source for illuminating an object, and the detector may be configured to measure light emitted from the object. The intended uses of various embodiments of the optical microscopes of the present invention result in variations of the methods of the present invention. Similarly, the optical microscopes of the present invention may be configured to perform the above-described exemplary methods of the present invention. Specifically, a controller may be configured to control the sensor array or other components of the optical microscope to perform the method steps described herein. [Brief explanation of the drawings]

[0060] A better understanding of the present invention and various other features and advantages thereof will be readily obtained from the following description taken in conjunction with the following schematic drawings, given by way of example only and not of limitation, in which like reference numerals may refer to similar or substantially similar components.

[0061] [Figure 1] 1 is a schematic diagram of an embodiment of an optical microscope according to the present invention; [Figure 2] FIG. 2 is a schematic diagram of the sensor array of the optical microscope of FIG. 1 in a first configuration using binning. [Figure 3] FIG. 2 is a schematic diagram of the sensor array of the optical microscope of FIG. 1 in a second configuration using binning. [Figure 4] FIG. 2 is a schematic diagram of the sensor array of the optical microscope of FIG. 1 in a third configuration using binning. [Figure 5] FIG. 2 is a schematic diagram of the sensor array of the optical microscope of FIG. 1 in a fourth configuration using binning. [Figure 6] FIG. 2 is a schematic diagram of the sensor array of the optical microscope of FIG. 1 in a fifth configuration using binning. [Figure 7] FIG. 2 is a schematic diagram of the sensor array of the optical microscope of FIG. 1 in a sixth configuration using binning. [Figure 8] FIG. 2 is a schematic diagram of the sensor array of the optical microscope of FIG. 1 in a seventh configuration using binning. [Figure 9] FIG. 2 is a schematic diagram of the sensor array of the optical microscope of FIG. 1 in an eighth configuration using binning. [Figure 10] FIG. 2 is a schematic diagram of the sensor array of the optical microscope of FIG. 1 in a ninth configuration using binning. [Figure 11] 1 is a schematic diagram of a sensor array and control unit of an optical microscope of the present invention; [Figure 12] 1 is a schematic flow chart illustrating the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0062] FIG. 1 shows a schematic diagram of one embodiment of an optical microscope 100 of the present invention.

[0063] The optical microscope 100 includes a light source 10 that emits illumination light 12. The light source 10 may include, for example, one or more lasers. The exemplary optical microscope 100 is configured as a laser scanning microscope. The optical microscope includes a scanner 25 having one or more movable mirrors or other movable optical elements that scan the illumination light 12 over a specimen 35. Optical elements 23, 24 may be used to guide the illumination light 12 from the light source 10 through the scanner 25 to an objective lens 30. The objective lens 30 focuses the illumination light 12 onto a specimen point, and different specimen points are successively illuminated by the scanning motion of the scanner 25. Two consecutive beam paths of the illumination light established using the scanner 25 are indicated by reference numerals 12A and 12B.

[0064] The specimen 35 emits detection light 15, which may be, for example, fluorescence or phosphorescence. The illumination light 12 may be pulsed, and in particular may result in multi-photon excitation of particles in the specimen 35. The detection light 15 thereby emits only from a small specimen spot and has a different (in particular, shorter) wavelength than the illumination light 12.

[0065] In the illustrated non-scanning configuration, the detection light 15 is directed through the objective lens 30, scanner 25, and optical elements 23 and 24 on the same beam path as the illumination light 12. A beam splitter 22 is used to spatially separate the detection light 15 and the illumination light 12. In one example, the beam splitter 22 may be configured to transmit or reflect the impinging light depending on its wavelength. The detection light 15 is then directed to the sensor array 60 by additional optical elements 31, 32, 33, and 34. An optional pinhole 18 for confocal designs may be placed at an intermediate image plane.

[0066] Controller 70 controls sensor array 60, optical elements 33 and 34, light source 10, and scanner 25, and may also be configured to control additional components of optical microscope 100. These components may collectively be referred to as an optical assembly that is part of optical microscope 100, but may also be used generally in other imaging systems.

[0067] 2-4 each show an enlarged view of the sensor array 60. Together with the controller, the sensor array forms a detector that can be implemented in an optical microscope or another light measurement device and can be considered an independent aspect of the present invention. Thus, one embodiment of the controller of the present invention may be formed by the sensor array 60 and controller 70 as shown in FIG. 1 , without the need for additional components of an optical microscope. Referring now to FIGS. 2-4, the sensor array 60 comprises a plurality of single-photon detector elements 61 arranged adjacent to one another in a two-dimensional array, e.g., a hexagonal or rectangular structure. The single-photon counting detector elements 61 may specifically be formed as an array of SPADs (single-photon avalanche diodes). Detected light impinging on the sensor array 60 forms a detected light spot 16, shown as a dotted circle. The controller is configured to individually enable or disable the detector elements 61. Disabled detector elements are depicted in white and designated by reference numeral 63. Only enabled detector elements output measurements (i.e., photon count rates) that are further processed, e.g., to calculate an image of the specimen or determine a PSF. The controller is further configured to flexibly bin or combine any number of detector elements 61 into superpixels. The superpixels comprise or are connected to multi-bit counters that output the accumulated photon count rate of the detector elements they comprise. The number of superpixels and the number of detector elements in each superpixel can be flexibly set by the controller.

[0068] In the example of Figure 2, detector elements 62 illuminated by detection light spot 16 are enabled and not binned, resulting in a photon count rate of 37. As shown in Figure 2, it is beneficial to disable a significant number of non-illuminated detector elements. The area of ​​sensor array 60 defined by enabled detector elements 62 is also referred to as the "active area."

[0069] 3 shows the case where several enabled detector elements 62 are binned into superpixels 65. Each superpixel 65 comprises seven detector elements 62. Superpixels 65 with different fill patterns (slash, check, diamond, etc.) are shown. Again, non-illuminated detector elements 63 are disabled. In FIG. 3, the detection light spot 16 is larger than in FIG. 2, so more detector elements 62 are enabled. However, binning into 19 superpixels 65 outputs only 19 photon count rates, and therefore the data rate is relatively low.

[0070] As another example, Figure 4 shows the case where all or nearly all detector elements 61 are illuminated, enabled, and binned into several superpixels 65. This allows the data rate to be reduced while still using the photon count rate of all or nearly all detector elements 61.

[0071] 5 shows the case where the detection light spot / distribution 16 illuminates only a portion of the detector array 60. The active area of ​​the detector array 60 is adjusted to fit the detection light spot 16. The binning pattern is set to include one central superpixel 65 and several surrounding ring-shaped superpixels 65. The ring-shaped superpixels 65 are particularly suitable when the PSF is rotationally invariant. The ring-shaped superpixels 65 have different diameters. In the illustrated embodiment, the thickness of each ring is defined by one layer of detector elements 61. However, the outer rings may alternatively be thicker and include two or more layers of detector elements 61.

[0072] 6, superpixels of various sizes are set, with the inner superpixels centered around a central point having a smaller size than the superpixels 65A-65D further from the central point.

[0073] In the embodiment of FIG. 7, the detector array 60 is divided into four quadrants that make up respective superpixels 65A-65D. The superpixels 65A-65D may be directly adjacent to each other or may be separated by inactive detector elements 63. FIG. 8 shows binning patterns using various superpixel shapes. The inner region is divided into several similarly sized superpixels 65C, 65D, e.g., several circular or hexagonal superpixels 65C, 65D. The outer region is divided into ring-shaped superpixels 65A, 65B. The ring-shaped superpixel 65A may be configured to have a greater thickness than the ring-shaped superpixel 65B that is closer to the ring center.

[0074] FIG. 9 shows a binning pattern that can be used with line-shaped illumination. When one or more elongated illumination spots are generated at the specimen plane using, for example, one or more cylindrical lenses, one or more elongated detection spots 16A, 16B may be formed on the detector array 60. In FIG. 9, multiple adjacent columns (or lines) of detector elements are binned into the same superpixel 65A or 65B. Thus, each superpixel 65A or 65B has a line shape, and the line thickness may be flexibly set. The superpixels 65A, 65B of FIG. 9 may also be used with other illumination patterns, for example, to select specific wavelengths when dispersed according to wavelength.

[0075] FIG. 10 further illustrates the flexibility of the binning pattern. It is not necessary for only adjacent detector elements to be binned into the same superpixel. Instead, superpixel 65B or 65C may consist of spaced detector elements. For example, first superpixel 65A may form a line, second superpixel 65B may be formed by two groups of spaced detector elements, and third superpixel 65C may be formed by several spaced individual detector elements. One or more of these superpixels may be used for calibration measurements, while other superpixels are assigned for actual sample measurements. In particular, for different sample positions, calibration measurements may be performed using superpixels including spaced detector elements.

[0076] The controller 70 is configured to set the binning pattern and active area to increase the SNR, keep the data rate within acceptable boundaries, and ensure the desired frame rate (requiring a limited amount of data per frame). Furthermore, the controller 70 can modify the size of the detection light spot 16 on the sensor array 60 and affect the intensity distribution within the detection light spot 16. For this purpose, the controller 70 adjusts the optics of the microscope. Before describing the effective binning and activation patterns, this adjustment will first be described.

[0077] As shown in FIG. 1, optical elements 33 and 34 in the beam path of the detection light 15 are adjustable to affect the detection light 15. The optical element 34 may be an imaging lens in front of the sensor array 60. The optical element focuses the detection light 15 onto the sensor array 60 to generate the detection light spot shown in FIGS. 2-4. By adjusting the optical element 34, the focus of the detection light 15 moves relative to the sensor array 60. As a result, the detection light spot becomes out of focus and enlarged (as shown in FIG. 1A). The optical element 34 may be shifted in the propagation direction of the detection light 15. Alternatively, the optical element 34 may be an adjustable (liquid) lens that deforms to change the focus position.

[0078] The optical element 34 may be a Bertrand lens 33. In the situation shown in Figures 1 and 1A, the Bertrand lens 33 is moved out of the beam path of the detection light 15. The controller may move the Bertrand lens 33 into the beam path, resulting in the case shown in Figure 1B. The Bertrand lens 33 images the pupil plane onto the plane of the sensor array 60. Adjusting the optical element 34 changes the size of the resulting detection light spot on the sensor array 60. The detection light spot of Figure 1B has a different, often more uniform, intensity distribution than the detection light spot of Figure 1. Furthermore, the size of the detection beam spot can be effectively changed, as in the case shown in Figure 1B.

[0079] FIG. 11 shows a schematic diagram of the structure of the sensor array 60 and control unit 70 of an optical microscope embodiment of the present invention. The sensor array 60 includes a number of N1-N2 detector elements and is therefore configured to output a number of N1-N2 photon count rates, for example, via respective signal lines to a processing unit 71. The processing unit 71 is part of the control device 70 and is formed together with the sensor array 60 in the same IC or on the same circuit board. The processing unit 71 includes a number of M individual units 71a. The design of one such unit 71a is shown schematically in FIG. 11. The remaining units 71a may be formed similarly. The units 71a include a combining circuit 71d to which all detector elements are connected via respective switches 71c. Thus, each unit 71a has a number of N1-N2 input lines and an equal number of switches 71c, providing flexible control over which detector elements are connected to the combining circuit 71d. The combining circuit 71d outputs a combined signal indicative of the received signal. The combining circuit 71d may have only one output line leading to the superpixel counter unit 72. The combining circuit 71d may be implemented as a shared bus with pull-up or pull-down drivers controlled by the detector element outputs, as an OR function, an XOR function, or any other combining function. Each switch 71c may be controlled via a respective memory 71b, which may have one bit (indicating the "on" or "off" state of the respective switch 71c). Thus, each combining circuit 71d may comprise a number of N1-N2 1-bit memories 71b. Each switch 71c may be implemented as an AND gate or a parallel PMOS+NMOS combination. The individual memory elements may be connected together to form a shift register. The output of the combining circuit 71d leads to the superpixel counter unit 72. Thus, there are a number M of output lines from the processing unit 71, where M is less than the product N1-N2. The superpixel counter unit 72 comprises at least one, preferably two, multi-bit counters per combining circuit 71d.In the case of at least one multi-bit counter per combining circuit 71d, the superpixel counter unit 72 can output a photon count rate indicative of the total photon count rate of all detector elements connected to each combining circuit 71d via the respective switch 71c.

[0080] If the superpixel counter unit 72 comprises two multi-bit counters per combining circuit 71d, parallel counting and readout during the dwell time is possible, i.e., while one multi-bit counter is read out (without counting further signals received and output by the respective combining circuit), another multi-bit counter connected to the same combining circuit is enabled to count signals received and output by that combining circuit, thus reducing the delay during the dwell time between superpixels due to sequential readout.

[0081] The reading of the multi-bit counters may be sequential so that only a fraction of all multi-bit counters (and therefore all superpixels) are read out, reducing the data rate.

[0082] The flexibility of the microscope described above and the beneficial effects of the preferred operational modes set using the controller are explained with reference to Figure 12, which is a flow chart illustrating steps of an exemplary method of the present invention. Unless otherwise specified, steps may be performed by the controller, specifically by adjusting the components described above (e.g., optics, light source, and sensor array).

[0083] In step S1, a selection of imaging modes and imaging parameters is provided or displayed to the user. Depending on the user's selection, the method continues with the selected imaging mode S2, S6, S11, or S15.

[0084] If the "PSF Oversampling Mode" S2 is selected, the method continues with steps S3-S5. The order of these steps may be changed, some or all steps may be performed simultaneously, or one or two of the steps may be omitted. In step S3, the optical elements 33 and 34 are adjusted to clearly image the sample plane onto the sensor array 60. As described above, the Bertrand lens for pupil imaging may be omitted, and an adjustable focusing lens may be configured to generate a clear image of the sample plane on the sensor array. The clear image forms a detected light spot having a size and intensity distribution indicative of the system's PSF. The optical elements 33 and 34 are further adjusted in step S3 to appropriately adjust the detected light spot size on the sensor array 60 depending on the intensity / power of the detected light beam. Excessive intensity leads to detector element saturation, i.e., the output photon count rate deviates from a linear relationship with the impinging photon velocity or intensity. The saturation threshold may be defined as the light intensity at which the output photon count rate deviates from a linear relationship with the impinging light intensity by more than 10%. To avoid saturation, the optical elements 33, 34 are controlled to keep the intensity of the detector elements below the saturation threshold by increasing the detection light spot size as the power of the detection light beam increases. In step S4, the active area of ​​the sensor array is set according to the detection spot size on the sensor array. Specifically, the active area may match the spot size, for example, with a deviation of no more than 20% or 30% of the PSF size. In step S5, the binning of the detector elements into superpixels is set according to selected imaging parameters, for example, the desired frame rate. The desired frame rate and maximum bandwidth for data transmission from the control unit affect or define the upper threshold for measurement data that can be output during a pixel dwell time. The number of superpixels is set to match (or not exceed) this upper threshold.

[0085] If the "confocal detection mode" S6 is selected, the method continues with steps S7-S10. Again, the order of steps S7-S10 may be changed, some or all steps may be performed simultaneously, or one or more of these steps may be omitted. In steps S7 and S8, the optical elements 33, 34 are adjusted to set the optical spot size depending on the power or energy of the detected light beam and homogenize the intensity distribution across the sensor array. Reducing the spot size means that fewer detector elements are illuminated with higher intensity or optical power. At low intensities, the background noise of the detector elements becomes significant, and therefore, reducing the spot size may increase the SNR. At high intensities, in contrast, the detector elements may saturate (the output photon count rate no longer linearly depends on the impingement intensity, further worsening the overall SNR). Therefore, at high intensities, the spot size is increased to avoid excessively high intensities on the detector elements. Thus, the controller may utilize a predefined function that calculates the spot size to be set depending on the power or energy of the light beam (calculated or previously measured). As the intensity increases, this function outputs a set spot size that increases. In confocal detection mode, only the total output from the detector elements is concerned; the intensity distribution across the detector elements is not analyzed. Therefore, the intensity distribution across the detector elements may be influenced to meet other criteria. Specifically, to avoid saturation of some detector elements due to excess intensity, the optical elements 33 and 34 are adjusted to make the intensity distribution more uniform. Because spatial information from individual detector elements is not used, a sharp image on the sensor array is not required. As mentioned above, the focusing lens 34 may be adjusted to change the spot size, and a Bertrand lens 33 may optionally be used to generate a pupil image (which may have a more uniform intensity distribution) and affect the spot size. In contrast to zoom optics, which can adjust the spot size while still providing a sharp image, adjusting using only two optical elements 33 and 34 significantly reduces complexity and potentially reduces light loss and cost. The use of such simplified optics is made possible by the flexibility provided by the SPAD sensor array 60.The above steps S7 and S8 may be performed as one step. In the following step S9, the active area is set to match the light spot size on the sensor array. Alternatively, the active area may be set to a size larger than the light spot size, for example, by an error of up to 20% of the light spot size. In a simpler variant, all detector elements may be enabled. In step S10, all enabled detector elements are binned into the same superpixel. The output value of the superpixel is therefore the total photon count rate of all enabled detector elements. This procedure reduces the data rate, but no information related to confocal imaging is lost.

[0086] If the "line scan mode" S11 is selected, the method continues with steps S12-S14. Again, the order of steps S12-S14 may be changed, some or all of the steps may be performed simultaneously, or one or more of these steps may be omitted. In line scan mode, optical components in the illumination beam path are adjusted to provide an illumination line on the specimen surface to be inspected. For example, a cylindrical lens may be inserted into the illumination beam path. In step S12, optical elements 33 and 34 are adjusted to clearly image the specimen surface onto the sensor array. When a line or elongated area of ​​the specimen is illuminated, detection light is emitted from this illuminated specimen area, directing an elongated or linear detection light spot on the sensor array. In step S13, an active area is set according to the elongated detection light spot so that at least some non-illuminated detector elements are disabled. Specifically, only illuminated detector elements are enabled. Step S14 sets a binning pattern. Detector elements that are adjacent to each other in a direction perpendicular to the longitudinal direction of the elongated detection light spot are binned into the same superpixel, in contrast to longitudinally adjacent detector elements that are not binned into the same superpixel (or, to further increase frame rate at the expense of resolution, only two or three longitudinally adjacent detector elements are binned).

[0087] If the "Field of View Scanning / Imaging Mode" S15 is selected, the method continues with steps S16-S19. In step S16, the optics are adjusted to clearly image the specimen plane onto the sensor array. In step S17, wide-field illumination is set, as opposed to, for example, PSF oversampling mode S2 or confocal detection mode S6, where illumination is set so that only the specimen point is illuminated. Next, in step S18, the active area is set to correspond to the illuminated area on the sensor array, which may depend on the currently inserted objective, wide-field illumination size, or zoom setting. In step S19, a binning pattern is set, which may be selected depending on, for example, the current zoom position.

[0088] In variations of the above method, optical elements other than optical elements 33, 34 may be provided and adjusted to change the spot size and / or light distribution. Further variations may provide additional imaging modes or may not provide all of the three above-mentioned imaging modes.

[0089] The method, detector and optical microscope of the present invention provide particularly good image quality and at the same time high flexibility without the need for complex and costly zoom optics. [Explanation of symbols]

[0090] 10 light source 12 Irradiation light 12A, 12B Beam path of the irradiated light 12 15 Detection light 16, 16A, 16B Detected light spot (size) / PSF on sensor array 60 18 Pinhole 22 Beam Splitter 23, 24 Optical elements 25 Scanner 30 objective lenses 31, 32 Optical elements 33 Optical elements, Bertrand lenses 34 Optical elements, adjustable lenses 35 specimens 60 Sensor Array 61 Photon counting detector element 62 Activated Photon Counting Detector Elements 63 Disabled photon counting detector element 65, 65A~65D Super Pixel 70 Control device 71 Processing unit of the control device 70 71a Processing unit 71 unit 71b Memory of unit 71a 71c Unit 71a switch 71d Coupling circuit of unit 71a 100 Optical microscope N1-N2 Number of detector elements 61; number of input lines to processing unit 71 M is the number of units 71a; the number of output lines of the processing unit 71 S1~S19 Method steps

Claims

1. 1. An optical microscope, comprising: a light source (10) for illuminating a specimen (35); a sensor array (60) of photon-counting detector elements (61, 62) for measuring detected light (15) emitted from the sample (35); a control unit (70) for controlling the sensor array (60); Equipped with the controller (70) is configured for flexible binning of the photon-counting detector elements (62) into one or more superpixels (65); An optical microscope, characterized in that the control device (70) is configured to input imaging parameters to adjust the binning and active area of ​​the sensor array (60) to limit the resulting data rate output by the control device (70) and including acquired measurement data of the one or more superpixels (65).

2. 2. The optical microscope of claim 1, wherein the control device (70) is configured to variably set the number of superpixels (65) and to variably set the number of photon-counting detector elements (62) binned into one superpixel (65).

3. the control device (70) or components of the control device (70) and the sensor array (60) are constructed on the same circuit board and / or in the same integrated circuit; each photon-counting detector element (61-63) is connected via a respective signal line to said control device (70); 3. The optical microscope of claim 1, wherein the control device (70) is configured to assign a respective multi-bit counter to each of the super-pixels (65).

4. further comprising adjustable optical elements (33, 34) for adjusting a spot size (16) of the detected light (15) on the sensor array (60); The control device (70) adjusting the active area of ​​the sensor array (60); controlling the optical elements (33, 34) to reduce a detection light spot size (16) on the sensor array (60) in accordance with the reduction in power of the detection light (15) and to reduce the effective area of ​​the sensor array (60); The optical microscope according to any one of claims 1 to 3, which is configured as follows.

5. 5. The optical microscope of claim 1, wherein the control device is configured to set the binning and active area of ​​the sensor array according to information about a detection light spot size.

6. 6. An optical microscope according to claim 1, wherein the control device (70) is configured to increase the active area of ​​the sensor array (60) and to increase the average number of photon-counting detector elements (61, 62) per superpixel (65) with increasing illumination wavelength and / or when a currently used objective lens is replaced with an objective lens with a smaller back aperture and / or depending on the numerical aperture at which the sample portion is imaged onto the sensor array.

7. the control unit (70) is configured for a PSF oversampling mode (S2); an effective area of ​​the sensor array (60) is set according to a PSF size (16) on the sensor array (60); the number of said superpixels (65) is set according to said PSF size (16), and / or 7. The optical microscope of claim 1, wherein the number of photon-counting detector elements (61, 62) binned into the same superpixel (65) is set according to a desired maximum count rate.

8. 8. The optical microscope of claim 7, wherein the control device (70) is configured to increase the number of photon-counting detector elements (61, 62) binned into the same superpixel (65) as the PSF size (16) increases, so that in the PSF oversampling mode (S2), the data rate output by the control device (70) does not exceed a predetermined maximum rate and / or is constant.

9. The optical microscope of claim 7 or 8, wherein the control device (70) is configured to set the superpixels (65) in the PSF oversampling mode (S2) so that the center-to-center distance between adjacent superpixels (65) is at most 0.2 Airy disk diameters.

10. the control device (70) is configured for a line scan mode (S11); the sample (35) is illuminated with an elongated illumination light beam that directs an elongated detection light spot on the sensor array (60); 10. The optical microscope of claim 1, wherein the control device (70) bins the photon-counting detector elements (61, 62) together such that elongated superpixels (65) are formed in a direction perpendicular to the elongated detection light spot.

11. 10. The optical microscope of claim 7, wherein the control device (70) is configured for a confocal detection mode (S6) and all enabled photon-counting detector elements (61, 62) are binned into one superpixel (65).

12. 12. The optical microscope of claim 1, wherein the control device (70) is configured for an array scanning mode, in which the specimen (35) is illuminated with a plurality of illumination spots, and the active area and the binning of the photon-counting detector elements (61, 62) are set according to the set illumination.

13. 12. The optical microscope of claim 11, wherein the control device (70) is configured to adjust the detection light spot size (16) on the detector array (60) based on the impinging light intensity in the PSF oversampling mode (S2) or the confocal detection mode (S6) to avoid saturation of the photon counting detector elements (61, 62) and increase SNR.

14. The control device (70) providing a user with a plurality of imaging modes (S2, S6, S11, S15) including at least two of the PSF oversampling mode (S2), the confocal detection mode (S6), a line or array scanning mode (S11), and a field of view imaging mode (S15); configured to set the binning of the photon-counting detector elements (61, 62) in response to the selected imaging mode (S2, S6, S11, S15); the control device (70) is configured to adjust optical elements (33, 34) of the optical microscope according to the selected imaging mode (S2, S6, S11); In the confocal detection mode (S6), the optical elements (33, 34) are adjusted to homogenize the intensity distribution on the sensor array (60), in particular by setting a pupil image on the sensor array (60) and adjusting the detection light spot size (16) on the sensor array (60), 12. The optical microscope of claim 11, wherein in the PSF oversampling mode (S2), the optical elements (33, 34) are adjusted to image sample points onto the sensor array (60) to record spatial information about the PSF.

15. one of the optical elements (33) is a Bertrand lens (33) and another of the optical elements (34) is an imaging lens (34); The control device (70) setting up pupil imaging by inserting the Bertrand lens (33) into the beam path of the detection light (15); 15. The optical microscope of claim 14, configured to adjust the detection light spot size (16) by adapting the focal length of the imaging lens (34) that focuses the detection light (15) near the sensor array (60).

16. An optical microscope as described in any one of claims 1 to 15, wherein the control device (70) is configured to prompt a user to input the imaging parameters, and the binning and active area of ​​the sensor array (60) are adjusted in response to the parameters to limit the resulting data rate.

17. The optical microscope according to any one of claims 1 to 16, wherein the imaging parameters are a desired resolution and / or a desired frame rate and / or a desired SNR.

18. 18. The optical microscope of claim 1, wherein the control device is configured to keep the output data rate constant by adjusting the binning when the active area of ​​the sensor array is changed.

19. 19. The optical microscope of claim 1, wherein the control device (70) comprises a processing unit (71) having a number M of units (71 a) for providing a maximum number M of flexibly configurable superpixels, each of the units (71 a) comprising a respective combining circuit (71 d) to which the photon-counting detector elements (61) are connected via programmable switches (71 c), each of the combining circuits (71 d) outputting a signal stream indicative of the combined photon count rate of all the detector elements (61), for which the respective switch (71 c) to the combining circuit (71 d) is closed.

20. the control device (70) comprises a superpixel counter unit (72) having a number M of input lines, each of which is connected to a corresponding output of the combining circuit (71d); 20. The optical microscope of claim 19, wherein each of the M input lines leads to at least one multi-bit counter of the superpixel counter unit (72).

21. 21. The optical microscope of claim 20, wherein the superpixel counter unit (72) comprises two multi-bit counters per combining circuit, each of the M input lines leading to a pair of multi-bit counters, one of the pair of multi-bit counters being read out and the other of the multi-bit counters counting the incident photon count rate.

22. the control device (70) is configured to adjust the timing delays between the output data streams of different super-pixels (65) based on a calibration matrix that indicates timing delays due to different lengths of signal lines from the photon-counting detector elements (61, 62) to the control device (70); and / or The optical microscope of any one of claims 1 to 21, wherein the control device (70) is configured to set different sensitivities between the photon-counting detector elements (61, 62) of the same superpixel (65).

23. a scanner (25) configured to scan the specimen (35) with the illumination light (12) and direct the detection light (15) toward the sensor array (60); The optical microscope of any one of claims 1 to 22, wherein the control device (70) is configured to adjust the binning during the scanning.

24. 1. An optical detector comprising: a sensor array (60) consisting of photon-counting detector elements (61, 62); a control unit (70) for controlling the sensor array (60); Equipped with the controller (70) is configured for flexible binning of the photon-counting detector elements (62) into one or more superpixels (65); 1. An optical detector, characterized in that the control device (70) is configured to input imaging parameters to adjust the binning and active area of ​​the sensor array (60) to limit the resulting data rate output by the control device (70) and including acquired measurement data of the one or more superpixels (65).

25. 25. The optical microscope of claim 24, wherein the control device (70) is configured to prompt a user to input the imaging parameters, and the binning and active area of ​​the sensor array (60) are adjusted in response to the parameters to limit the resulting data rate.

26. 26. The optical microscope of claim 24 or 25, wherein the imaging parameters are a desired resolution and / or a desired frame rate and / or a desired SNR.

27. further comprising an event-driven effective recharge component for each of said photon-counting detector elements (61, 62); 27. The optical detector of any one of claims 24 to 26, wherein the control device (70) is configured for flexible binning of continuous and / or discontinuous photon counting detector elements (61) into the same superpixel (65).

28. An imaging method, comprising: illuminating a specimen (35) with illumination light (12); measuring detected light (15) emitted from said sample (35) with a sensor array (60) of photon-counting detector elements (61, 62); controlling the sensor array (60) using a control device (70); flexibly binning said photon-counting detector elements (62) into one or more superpixels (65); 10. The imaging method, comprising: the control device (70) inputting imaging parameters to adjust the binning and active area of ​​the sensor array (60) to limit the resulting data rate output by the control device (70) and including acquired measurement data of the one or more superpixels (65).

29. 30. The imaging method of claim 28, further comprising prompting a user to input the imaging parameters, in response to which the binning and active area of ​​the sensor array (60) are adjusted to limit the resulting data rate.

30. 30. The imaging method according to claim 28 or 29, wherein the imaging parameters are a desired resolution and / or a desired frame rate and / or a desired SNR.

Images