Optical microscope with reconfigurable sensor array

By combining and reading the photon count detector elements into superpixels, adapting to the spot characteristics, solving the problems of optical complexity and high cost in optical microscopes, and achieving a microscope system with high image quality and high signal-to-noise ratio.

CN118962964APending Publication Date: 2024-11-15CARL ZEISS MICROSCOPY GMBH +1
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411227797.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2019-04-09
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

While providing high image quality, existing optical microscopes have problems of optical complexity and high cost, especially when adjusting the beam size using zoom optics, which affects signal detection efficiency and economy.

Method used

By flexibly combining photon count detector elements into one or more superpixels, adapting to the size and characteristics of impact spots, reducing the need for zoom optics, and improving dynamic range and signal-to-noise ratio through event activation active charging technology .

Benefits of technology

A microscope system with high image quality is realized, reducing optical complexity and cost, while improving signal-to-noise ratio and maximum counting rate, adapting to different light intensity conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118962964A_ABST
    Figure CN118962964A_ABST
Patent Text Reader

Abstract

An optical microscope comprises a light source (10) for illuminating a sample (35), a sensor array (60) consisting of photon counting detector elements (61, 62) for measuring detection light (15) from the sample (35), and a control device (70) for controlling the sensor array (60). The control device (70) is configured to flexibly merge the photon counting detector elements (62) to read out the one or more superpixels (65).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to optical microscopes. The present disclosure also relates to optical detectors and imaging methods. Background Art

[0002] Optical microscopy is used in various applications such as life sciences or material testing. In particular, confocal scanning microscopy is a mature technology. Current sensor technologies used in confocal scanning microscopy include photomultiplier tubes with polyaluminum or GaASP photocathodes, or hybrid detectors including GaASP photocathodes and APD (avalanche photodiode) detectors.

[0003] Recent advances can see the use of single-photon avalanche diodes (SPADs), which offer high sensitivity, fast response times, and low dark counts. Based on work by Sheppard et al., cf. Sheppard, CJ Optik 80, 53–54 (1988); and Sheppard, CJ, Mehta, SB & Heintzmann, R. Opt. Lett. 38, 2889–2892 (2013), SPAD arrays allow spatial sampling of point spread functions (PSFs) that can be used for image scanning techniques (Airyscan). In some applications, the PSF is rotationally symmetric, but generally any light distribution can be used.

[0004] A photon counting detector element or SPAD comprises a junction with a depletion region devoid of any free charge carriers. A voltage Vop, which is higher than the breakdown voltage, is applied at the junction. Photons absorbed by the SPAD may be able to inject carriers in the depletion region, and thus, impact ionization may result in an avalanche of carriers that spread to other regions of the diode and may be detected as photon counts.

[0005] To detect the following photon, the avalanche is quenched. When the diode is quenched (ie, there is no other current due to impact ionization and there are no free carriers in the diode), the voltage at the diode will be charged by current flowing through, for example, the quenching resistor, and the diode is ready to detect another carrier.

[0006] A general optical microscope comprises a light source for illuminating a sample and a sensor array comprised of photon counting detector elements for measuring detection light from the sample. A control device is configured for controlling the sensor array. The optical elements may be arranged for directing illumination light from the light source to the sample and for directing detection light from the sample to the sensor array. Similarly, a general optical detector comprises a sensor array comprised of photon counting detector elements, and a control device for controlling the sensor array. A general imaging method comprises the steps of illuminating a sample with illumination light, measuring detection light from the sample having a sensor array comprising photon counting detector elements, and controlling the sensor array with a control device.

[0007] US2016 / 0131883A1 describes a laser scanning microscope with multiple avalanche photodiodes or PMTs. Optical fibers are used to direct the detection light to the individual photodiodes. The PSF is spatially resolved, thus allowing the above-mentioned image scanning techniques to be performed. Due to the rather low number of photodiodes and optical fibers used to direct the light to the photodiodes, zoom optics are typically used to adapt the size of the PSF to the limited number of optical fibers and sensor pixels. The zoom optics required for adjusting the size of the PSF are rather complex and therefore increase the cost while potentially having a negative impact on the signal detection efficiency due to the large number of optical elements required.

[0008] US2017 / 0176250A1 describes a sensor array with SPAD elements, in particular for distance measurement. In this case, the output lines of all SPAD elements of the sensor array are combined and read out as a single output. Thus, the sensor array outputs a detection signal each time any one of the SPAD elements measures an impinging photon. This design provides accurate results for certain applications, but its combined output line readout of a single output does not allow the use of this sensor array in many microscopy techniques, such as the PSF oversampling technique mentioned above.

[0009] It is an object of the present invention to provide an optical microscope, an optical detector and an imaging method which provide a particularly high image quality without excessive optical complexity and costs. Summary of the invention

[0010] The above objects are achieved by an optical microscope, an optical detector and a method.

[0011] Preferred embodiments are given in the dependent claims and in the following description, particularly in conjunction with the drawings.

[0012] According to the invention, an optical microscope of the above-mentioned kind and an optical detector of the above-mentioned type are characterized in that the control device is configured for flexible combined readout of the photon counting detector elements into one or more superpixels.

[0013] According to the invention, the above method has at least one step of readout (binning) by flexibly combining photon counting detector elements into one or more superpixels. In particular, the method can be implemented using embodiments of the optical microscope described in the invention.

[0014] The preferred embodiments described below, in particular, describe sensor arrays, control devices, optical elements or light sources, which can be applied to the optical microscope and optical detector of the present invention.

[0015] Traditionally, complex zoom optics are used to adjust the size of the light beam impinging on the sensor array so that the light beam fully fills the sensor array. In contrast, the present invention allows the sensor array to be adapted to the size and characteristics of the impinging light spot. In the case of a PSF oversampling mode, the size of the light beam (or light spot) impinging on the sensor array is defined by the PSF. Spatially resolving the PSF allows the information required for high-resolution techniques to be obtained. A sufficient number of detection points are required when spatially resolving the PSF, without necessarily a large number of detection points that exceed reasonable constraints on the data rate. Flexible merging and reading out the output lines of the photon counting detector elements into a variable number of superpixels overcomes these problems. The larger the PSF (the light spot impinging on the detector array), the larger the superpixel that can be set. In the case of a smaller light spot, the detector elements impinging on the light can be combined and read out into smaller superpixels (or no superpixels are combined and read out on some detector elements), while other detector elements at lower light powers or without light impinging can be combined and read out into larger superpixels or deactivated. The concept of the present invention allows accurate measurement of light distribution with reduced requirements on the zoom optics (e.g., zoom factor). At the same time, the inventive concept may provide advantages in maximum count rate and signal-to-noise ratio (SNR).

[0016] In particular, based on CMOS SPAD technology, each detector element can be addressed individually and the sensor array can be reconfigured to enable combining (merging readout) each individual pixel (i.e., output from a photon counting detector element) into any superpixel. Thus, the merged readout is in principle independent of the detector element position. Thus, the control device can be configured to flexibly merge readout consecutive (adjacent) and / or non-consecutive photon counting detector elements into the same superpixel.

[0017] In a preferred embodiment, the photon counting detector elements employ event-activated active charging. The corresponding event-activated active charging (electronic) components may be considered as part of the control device and / or the sensor array. Event-activated active charging allows for individual charging of each detector element, i.e. not all detector elements are charged at the same time, but only specific detector elements that experience an avalanche event. Compared to passive and clock-driven charging of SPADs, this results in a "non-crippling" sensor response to light and provides a dynamic range extension. Further explanation is given by Ivan Michel Antolovic et al. in "Dynamic Range Extension of Photon Counting Arrays", Volume 26, Issue 17, Optics Express 22234, published on August 20, 2018.

[0018] Further advantages can be achieved if the flexibility of the reconfigurable sensor array is not only considered for the combined readout, but also for the activation of individual detector elements. Here, "activation" or "activated" is understood to indicate that the detector element outputs a measurement signal (photon count rate) for further processing, while an inactivated detector element does not output a measurement signal for other use. Alternatively, "inactivation" can indicate that the detection element has no route to further process its output. In the case of low-intensity light hitting some detector elements, activating these detector elements may indeed degrade the overall SNR due to dark noise. In this case, deactivating the relevant detector elements may preferably group them into one or more large superpixels. In the case of low detection light power, it may be preferred to focus the detection light on a smaller spot and activate only the corresponding detector elements. As the intensity increases, the accuracy can be improved by increasing the spot size and correspondingly increasing the activation area. In other words, the control device can be configured to control the optical element to reduce the detection spot size on the sensor array and reduce the activation area by reducing the detection light power. As the detection light power increases, more detector elements can be activated, which increases the maximum count rate and increases the SNR through a higher photon flux.

[0019] A superpixel is generally understood to be a combination of the outputs of one or more (or two or more) detector elements. To use shorter language, the present disclosure frequently refers to a merged readout superpixel, which should cover all detector elements to merge the readout superpixel for all detector elements, but may also include the option of unmerged readout of certain (activated) detector elements.

[0020] The control device may be configured to assign a corresponding multi-bit counter to each superpixel. The multi-bit counter reduces the data rate compared to transmitting measurement data from each detector element individually. Each multi-bit counter outputs an accumulated number indicating the counted photons of all detector elements in the respective superpixel that are combined for reading out. Thus, the data rate may be reduced from X [bits / second] to X·N / (2N-1) [bits / second], where X represents the superpixel detection rate and N represents the number of bits of the multi-bit counter.

[0021] The control device (or a component of the control device) and the sensor array may be built in the same circuit board and / or the same IC (integrated circuit). In particular, the components of the control device may be built together with the sensor array in the same IC, while other components of the control device are formed outside the IC but on the same circuit board as the IC. For example, the multi-bit counter may be formed as an on-chip multi-bit counter. Each photon counting detector element may be connected to the control device via a respective signal line (i.e., to the multi-bit counter or a pre-processing unit before reaching the multi-bit counter). This reduces the maximum necessary data bandwidth outside the control device.

[0022] The control device may variably set the number of superpixels and / or may variably combine the number of pixels read out of a superpixel. Different simultaneously used superpixels may also vary in their size.

[0023] An exemplary electronic implementation of a control device is described below. The control device can be configured to flexibly set up to M numbers of superpixels, while the control device includes a processing unit having the same number of M units. Each detector element is connected to each unit. If the number of detector elements is N, each unit can have n input lines, each input line receiving the photon count rate of a detector element. (More generally, each unit can be connected to at least several but not necessarily all detector elements.) Each unit includes a plurality of switches (particularly one switch for each detector element) to flexibly control whether the photon count rate output by a particular detector element is forwarded or blocked by the switch. Each unit includes a combination circuit having a switch connection. Therefore, depending on the state of the switch, it is possible to flexibly control which detector elements are connected to the same combination circuit. Each combination circuit outputs a data stream indicating all photon count rates for those detector elements for which the respective switches leading to the combination circuit are closed.

[0024] Each switch can be controlled by means of a memory, in particular a respective 1-bit memory, wherein the two possible states of the memory define whether the switch is closed or open.

[0025] The control device may also include a superpixel counter unit having a plurality of independent multi-bit counters. Each combination circuit is connected to at least one multi-bit counter. Thus, the multi-bit counter outputs a number corresponding to the photon count rate of all detector elements connected to each combination circuit. In another variant, each combination circuit is connected to a pair of multi-bit counters. If there are M combination circuits, the total number of multi-bit counters may be at least 2M. A pair of multi-bit counters operates in alternating phases, i.e., when one of the multi-bit counters calculates the photon count rate input from the combination circuit, the other multi-bit counter has been read out, and vice versa. In this way, time delays due to readout are reduced or avoided.

[0026] To provide up to M superpixels, the control device may include a number M of independent combination circuits and at least a number M (preferably 2M) of multi-bit counters. If N indicates the number of detector elements, each combination circuit includes up to N programmable switches connected to the detector elements. M is less than the number of programmable switches and less than N. The total number of programmable switches and associated 1-bit memories may be n·m. In operation, fewer than M superpixels may be required with a correspondingly reduced number of multi-bit counters read out.

[0027] The control device may set the merged readout and / or activation area of ​​the sensor array based on the spot size information, i.e., activate and deactivate each detector element individually. The spot size information may generally be any information or assumption indicating the expected light distribution or size on the sensor array. The information may be based on previous measurements or reference measurements. Additionally or alternatively, the spot size information may be derived (at least in part) from the currently used illumination wavelength and / or the currently used objective lens. The illumination wavelength affects the PSF size, in particular, it may increase with increasing illumination wavelength. A detailed description of how the activation area and the merged readout are set in dependence on the PSF size (and therefore depending on the wavelength) is described below. The PSF size is also affected by the return aperture of the objective lens. The smaller the return aperture, the larger the PSF size. In some embodiments, the activation area of ​​the sensor array increases and / or the (average) number of detector elements per superpixel increases with an increased illumination wavelength and / or when the objective lens is changed to an objective lens with a smaller return aperture, and vice versa, i.e. the activation area and / or the average number of detector elements per superpixel decreases with a decrease in wavelength and / or when the objective lens is changed to an objective lens with a larger return aperture. Since the wavelength as well as the return aperture affect the PSF size, it may be advantageous to adjust the activation area and the superpixels depending on these factors. Additionally or alternatively, the activation area and / or the average number of detector elements per superpixel may be set depending on the aperture ratio for imaging the sample portion on the sensor array. Furthermore, the aperture ratio affects the detection spot size on the sensor array. By adjusting the average number of detector elements combined in the same superpixel, the transmitted data stream remains manageable, in particular constant, as further explained below.

[0028] The control device can be configured to provide a plurality of imaging modes to the user. The user can select an imaging mode, for example, via a computer. The imaging modes include at least two of the following: a PSF oversampling mode, a line (or array) scan mode, a field imaging mode, and a confocal detection mode, as described below. A binned readout or an optional activation sensor area is set according to the selected imaging mode. In addition, the optical elements are adjusted according to the imaging mode, as further described below.

[0029] PSF oversampling mode (image scanning mode)

[0030] The control device may be configured to perform a PSF oversampling mode (also referred to as an image scanning mode). The illumination light is focused on a sample point, and the detection light (e.g., fluorescence) emitted from the sample point is focused on a sensor array, which is arranged in a conjugate plane of the illuminated sample plane. A pinhole may be provided in the pupil plane. The detection light forms a detection spot on the sensor array. The size and intensity distribution of the spot represent the PSF. The spot diameter may be understood as 1 Ery disk diameter. In the PSF oversampling mode, superpixels may be provided so that the center-to-center distance between adjacent superpixels is at most 0.2 Ery disk diameters. This ensures a sufficient number of detection points for spatially resolving the PSF. In addition, the activation area of ​​the sensor array is provided according to the PSF size. In some embodiments, the area of ​​the activated sensor area increases as the PSF size increases. In particular, the activation area may be provided to be equal to the size or diameter (within a 10% or 20% margin) of the PSF size (i.e., the PSF or spot size on the sensor array). Additionally or alternatively, the number of superpixels may be provided according to the PSF size. The number of photon counting detector elements that are combined and read out into the same superpixel can also be set according to the PSF size. As the PSF size increases, the number of detector elements that are combined and read out into the same superpixel can increase, in particular to avoid excessive data rates. The number of combined detector elements can be adjusted so that the data rate output by the control device (i.e., the data including the measurement data of the acquired superpixel) does not exceed a predefined maximum rate and / or is constant, i.e., exactly constant or constant within a predefined range (e.g., 10% or 20%). This ensures that a sufficient number of data points are used to sample / spatially resolve the PSF while avoiding excessive data rates. The number of detector elements to be combined and read out of the same superpixel can also be set according to the required maximum count rate. The larger the required maximum count rate, the more detector elements are combined and read out of the same superpixel. If the lower limit of the count rate is not taken into account, the maximum count rate can be regarded as the dynamic range of the sensor. However, since the dark count rate can define the lower limit of the photon count rate of a single detector element, combining multiple detector elements will result in an increase in the minimum count rate and may therefore not increase the dynamic range while still increasing the maximum count rate.

[0031] The size and shape of the superpixels used may vary. If the PSF has (or is expected to have) a maximum at its center, then one or more of the central superpixels may be smaller than the outer superpixels.

[0032] Line scan mode and array scan mode

[0033] The control device may also be configured to perform a line scan mode (or more generally: array scan mode). In the line scan mode, the sample is illuminated by an elongated illumination spot, for example by focusing the illumination light onto the sample using a cylindrical lens. The illumination results in an elongated detection spot on the sensor array. The control device merges the photon counting detector elements for readout together such that elongated superpixels are formed along a direction transverse to or perpendicular to the elongated detection spot. For example, if the elongated detection spots form columns on the sensor array, then one or more rows of detector elements are merged for readout into the same superpixel (wherein the rows and columns are perpendicular to each other). Detector elements that are not illuminated by the elongated detection spot may be activated. Thus, the merged readout mode used in the line scan mode has a significant change compared to the merged readout mode used in the PSF oversampling mode.

[0034] In line scan mode, an elongated illumination beam is scanned over the sample, with the scanning direction being transverse or perpendicular to the longitudinal direction of the elongated illumination spot. During this scan, multiple measurements are continuously recorded using the sensor array, and the sample image is calculated from these measurements.

[0035] In principle, other illumination shapes than elongated illumination spots can also be used, while providing similar advantages in line scanning mode with respect to reduced scanning time and image acquisition time. Such array scanning modes use illumination light patterns other than points or lines. For example, a grid or a plurality of lines can be used as the illumination light pattern to be scanned over the sample.

[0036] In array scanning mode, multiple illumination spots are scanned simultaneously on the sample. The multiple illumination spots can have circular, longitudinal or any other general shape.

[0037] Field Imaging Mode

[0038] For example, field imaging mode can be used to record an overview image with reduced resolution or SNR compared to sample images recorded using the other described imaging modes. In field imaging mode, a widefield image of the sample is recorded using a sensor array. Optical elements can therefore be removed or inserted into the illumination and / or detection beam paths to ensure that more than just one sample point is illuminated and that the sample plane (rather than the pupil plane) is imaged onto the sensor array. The activation area can span the entire sensor array or can be set depending on the aperture ratio or zoom setting. Binned readout can be used to reduce the output data rate. In different field imaging modes, multiple widefield images of different sample parts can be recorded consecutively and then stitched together to form an overview image.

[0039] Confocal detection mode

[0040] The control device can also be configured to perform a confocal detection mode without PSF oversampling. In the confocal detection mode, the illumination light is focused onto a sample point and the detection light emitted from the sample point is directed to the sensor array. The same purpose is used to focus the illumination light onto the sample and to receive the detection light from the sample. The pinhole can be arranged in the pupil plane. It may be necessary to measure all the detection light that strikes the detector array as accurately as possible, without spatial resolution. In this case, all activated photon counting detector elements can be combined to read out a superpixel. There is no PSF oversampling in this operating mode, and one superpixel is sufficient. Nevertheless, using multiple photon counting detector elements (SPADs) on a conventional detector and then combining them to read out into a superpixel has significant advantages: SPADs are unmatched in measuring low-intensity light. At higher light intensities, a single SPAD may saturate; during its idle time (when the voltage after a photon detection has not yet recovered to exceed the breakdown voltage of the diode), a single SPAD cannot detect another photon. However, by using a sensor array with multiple SPADs, the maximum count rate can be increased. Therefore, SPAD arrays can improve accuracy in dynamic range or maximum count rate without major disadvantages.

[0041] Adjusting the Optics

[0042] The control device can also be configured to adjust optical elements, such as lenses and reflectors, according to the selected imaging mode. In the confocal detection mode, the optical elements can be adjusted, for example, moved or deformed, to uniformize the intensity distribution on the sensor array. In particular, the optical element can enter the beam path of the detection light for pupil imaging on the sensor array. In addition, the optical element can be adjusted or moved to adjust the size of the detection spot on the sensor array. Therefore, the optical system is configured to switch between sample plane imaging and pupil imaging on the same sensor array. The pupil image can have a more uniform intensity distribution than the sample image (for example, when only the sample point is illuminated). Therefore, it may be preferred to image the pupil onto the sensor array. Compared with the conventional setting, due to the use of the SPAD array, the uniform intensity distribution without excessive intensity peaks on a single detector element has greater relevance. If the impinging light power is quite high, the pupil image should uniformly fill the sensor area to optimize the SNR. If the light power of the impinging detection light is quite low, it may be preferred to concentrate the light on a smaller number of detection elements, thereby increasing the intensity on the irradiated detection elements. Otherwise, the weak signal intensity will be masked by the dark noise of the detector element. Therefore, at least within a certain power range of the detection light, the control pen can adjust the detection spot size on the sensor array so that the spot size increases with the increase of the beam power. The activation area of ​​the sensor array is adjusted accordingly with the adjustment of the spot size.

[0043] In the confocal imaging mode and / or the PSF oversampling imaging mode, the detection spot size can also be adjusted to avoid saturation of the detector elements or to increase the SNR. If the intensity on the detector element exceeds a predetermined threshold (e.g., the output photon count rate is linearly proportional to the impinging intensity as described above), the detection spot size can be increased, and in turn the impinging light power per detector element is reduced.

[0044] In order to adjust the size of the detection spot (or pupil image), the imaging lens can be adjusted to focus the detection light near the sensor array. For example, the imaging lens can be moved to move the focal plane relative to the sensor array. The farther the sensor array is from the focal point, the larger the detection spot becomes. Depending on the design of the imaging lens, the imaging lens can also be adjusted by deformation or other means instead of moving, for example, in the case of an adaptive liquid lens.

[0045] In contrast to the confocal imaging mode, in the PSF oversampling mode the optical elements are adjusted to image the sample points onto the sensor array in order to record spatial information on the PSF. Thus, a change from the confocal detection mode to the PSF oversampling mode may imply a change from pupil imaging to sample plane imaging on the sensor array. This change is effected by inserting / moving a Bertrand lens into / out of the beam path of the detection light, for example by moving the Bertrand lens or redirecting the detection light to the Bertrand lens. Alternatively, a phase mask for setting the phase distribution or an SLM (spatial light modulator) may be arranged in the illumination or detection pupil plane. In particular, the SLM may be formed by an array of adjustable microlenses, diffractive optical elements, adjustable mirrors such as digital micromirror devices, or a controllable liquid crystal array. The SLM may similarly adjust the spot size and / or intensity distribution.

[0046] Furthermore, optical elements in the illumination beam path can be adjusted or inserted according to the selected imaging mode. For example, in PSF oversampling mode or confocal imaging mode, the illumination light should be focused onto the sample point, while widefield imaging mode requires a larger sample area to be illuminated, and line scanning mode may require the introduction of a cylindrical lens.

[0047] Adjustments for FLIM

[0048] The microscope of the present invention is also very suitable for FLIM (fluorescence lifetime imaging microscopy). SPAD provides a high resolution that is beneficial for FLIM measurements. To determine the fluorescence lifetime, the fluorophore is stimulated with repeated light pulses. After each pulse, the time span until the photon reaches the sensor array is determined. A histogram of the photon arrival times is thereby generated, from which the fluorescence lifetime is derived. It would be beneficial to use high-intensity pulses to measure a greater number of photons and reduce the required measurement time. However, with increasing intensity, a second photon may reach a SPAD shortly after the same SPAD has registered the first photon. The detection of the first photon requires a dead time during which the SPAD cannot notice the second photon. As a result, the histogram of the photon arrival times will tend or be biased towards shorter times, which is the so-called pile-up effect. This disadvantage can be avoided with the microscope of the present invention by homogenizing the intensity distribution and increasing the spot diameter on the sensor array. These measures reduce the probability that a second photon hits the SPAD during its dead time. The control device can be configured to provide FLIM as a further imaging mode, wherein with increasing detection light intensity, the detection spot size increases and pupil imaging is optionally implemented to homogenize the intensity distribution. The superpixel can be connected to a time-to-digital converter to detect multiple photons per laser cycle.

[0049] Other embodiments

[0050] The control device may be configured to prompt the user to input imaging parameters, in particular the desired resolution, frame rate and / or SNR. Based on the user input, the control device then adjusts the merged readout and activation regions of the sensor array, as well as optional optical elements as described above. In particular, the merged readout and activation regions may be set to limit the resulting data rate based on the desired frame rate. Since the bandwidth for transmitting data from the control device is known, the desired frame rate may be converted into an upper limit on the data rate output from the control device. The merged readout and activation regions may affect the amount of resulting data and may therefore be adjusted to meet the data rate requirement.

[0051] With adaptive binning readout, a further advantage is balancing spatial resolution and acquisition speed. In order to benefit from the increased resolution achieved by image scanning microscopy, a constant SNR is required. If the signal intensity (e.g., the intensity on the sensor array) is too low, the acquisition rate must generally be reduced to increase the SNR. However, adaptive pixel binning readout allows the resolution to be reduced to support the required acquisition speed. The signal intensity can be, for example, a photon count rate and can be derived from a previous measurement (during a scan of the sample) or a reference measurement.

[0052] The control device may also be configured to keep the output data rate constant (i.e. completely constant or constant within predefined limits, such as 10% or 20%) by adapting the binned readout, in particular when the activation area of ​​the sensor array changes. For example, a change of the objective lens may result in a change in the size of the detection spot on the sensor array, so the activation area may be adapted to the new spot size and in turn the binned readout may be adapted to the new activation area size, in particular keeping the data rate constant. The number of superpixels may be kept constant but different binned readout modes may be used. The number of bits of the multi-bit counter used for the superpixel may vary, but it is also possible that the number of superpixels varies to ensure a constant data rate.

[0053] As the intensity of light increases, the probability that a photon hits a detector element during its idle period increases and thus remains unnoticed. As a result, the measurement result of the detector element is negatively affected at very high intensities. In order to cope with this problem, the sensitivity of the detector element can be adjusted, for example, by adjusting its overvoltage above the breakdown voltage. The light spot that hits the sensor array is usually not a uniform light distribution. In particular in PSF oversampling mode, the impinging light spot can have a maximum intensity in its center and a decreasing intensity towards its outer areas. Therefore, the sensitivity of the central detector element can be set to a different level than the sensitivity of the outer detector elements. In particular, different sensitivities can be set between photon counting detector elements of the same superpixel. The sensitivity can be set depending on the desired intensity distribution over the sensor array, for example, based on assumptions about the PSF or based on reference measurements.

[0054] Different lengths of signal lines from the detector elements to the control device can result in timing delays between the output data streams of different superpixels. The control device can be configured to adjust or compensate for these timing delays. A calibration matrix can be saved in a memory of the control device and used to this effect.

[0055] The control device may comprise a plurality of physical output lines, wherein each output line is associated with a respective superpixel. The number of output lines may then correspond to the maximum number of superpixels that may be used simultaneously. As an example, the number of output lines may be between 10% and 40% of the number of detector elements.

[0056] Two or more spots can be measured simultaneously on a photon counting detector array. Different PSFs can be applied to different spots, so the spots can have different sizes on the sensor array. Superpixels for larger spots can be arranged to include more detector elements than superpixels for smaller spots.

[0057] A scanner may be provided and configured to scan the sample with illumination light and / or direct detection light toward the sensor array. For example, during scanning, a combined readout may be adjusted based on the amount of light emitted from a particular sample area. Information acquired during scanning may be used to adjust the combined readout as scanning continues. In such online adaptation, other parameters may also be adjusted, such as the sensitivity of the SPAD or the illumination light intensity.

[0058] In particular, the described optical microscope can be formed as a (laser) scanning microscope. Alternatively or additionally, it can be configured as a wide-field microscope. The described sensor array and the control unit produce the described advantages in many applications, in which any object is illuminated and the detected light from the object is measured. The invention can also be applied to other sensor devices, which do not necessarily require a microscope, for example in material analysis, cameras or monitoring systems, astronomy or production supervision.

[0059] An optical microscope may be defined by comprising a sample holder and an objective, in particular an infinity objective setting the image distance to infinity. It may further comprise a tube with a tube lens arranged in the beam path behind the objective, i.e. between the objective and the detector array, for focusing the light from the objective into an (intermediate) image plane. In contrast to other optical systems, an optical microscope produces at least one intermediate image plane. It may further comprise an illumination port to which a light source may be connected, such as one or more lasers. The detection light emitted from the sample may be of any kind, such as fluorescent or phosphorescent light, illumination light that is scattered or affected by the sample by other mechanisms, or light emitted from the sample for other reasons that are at least partially caused by the illumination light. In general, sample light may also be emitted due to effects that are not related to the illumination.

[0060] The optical microscope may include an objective lens arranged to direct or focus an illumination light onto the sample. In particular, the objective lens may be arranged to receive detection light emitted from the sample and direct the detection light toward the sensor array. In general, different illumination and detection objective lenses may be used. The optical microscope may also include a scanner arranged between the objective lens and the illumination source. The control device controls the scanner to perform a scan of the sample under the scanning of the illumination light on the sample. Optionally, the scanner may also direct the detection light from the objective lens to a photon counting detector array (de-scanning setting). The scanner may be understood as a device configured to adjustably deflect a light beam. It may include one or more movable optical elements, such as a reflector, a lens or a prism. Alternatively, the illumination light may also be adjustably deflected based on the acousto-optic effect.

[0061] The PSF can be understood as defining how an optical microscope or optical system directs the illumination light and the detection light. In particular, the PSF can be considered to consist of an illumination PSF and a detection PSF, where the illumination PSF defines how the points of the light source will be imaged in the sample plane, and the detection PSF defines how the points of the sample plane are imaged on the plane of the sensor array. As used herein, the detection spot size on the detector array can also be referred to as the PSF size in image scanning techniques or confocal imaging.

[0062] For simplicity, photon counting detector elements for use are also referred to as "detector elements", in particular SPADs (single photon avalanche diodes), in particular operated in the so-called Geiger mode. In Geiger mode, a voltage VOP is applied to the diode of the SPAD, which exceeds the breakdown voltage of the diode by an excess bias voltage. Thus, photon absorption may lead to a charge avalanche, resulting in countable events. The sensor array may accordingly be referred to as a SPAD array.

[0063] The control device may comprise electronic components such as an FPGA or a processing unit which may be formed as a single unit or as a distributed system. The functions of the control device may be implemented as software and / or hardware. In particular, the control device or its components may be arranged as an on-chip unit next to the sensor array. Components of the control device may also be provided by a server or computer application which communicates with the other components of the optical microscope via a network.

[0064] In scanning mode, a sample point can be defined as the illuminated portion of the sample for which the detector elements count photons during the pixel dwell time. After the pixel dwell time expires, the scanner illuminates another portion of the sample, which is defined as the next sample point.

[0065] For easier understanding, the expression "light spot" is often used in this disclosure to refer to the light distribution on a sample or sensor array. More generally, "light spot" can be understood as any light distribution, such as an annular pattern, one or more lines or multiple spots or rings.

[0066] The detector of the present invention can be used in the optical microscope described herein. Alternatively, the detector can be a component of other optical measuring devices, for example, for distance measurement, quality control, monitoring, in portable / handheld devices, in medical equipment or as a vehicle sensor. Optionally, the detector can be used in combination with the optical elements described herein and a light source for illuminating an object, and the detector can be arranged to measure light from the object. The intended use of different embodiments of the optical microscope of the present invention leads to variations of the method of the present invention. Similarly, the optical microscope of the present invention can be configured to perform the exemplary methods described in the present invention. In particular, the control device can be configured to control the sensor array or other components of the optical microscope to perform the method steps described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The present invention and various other features and advantages of the invention will be better understood from the following description in conjunction with the schematic drawings, which are illustrated by way of example only and not limitation, in which like reference numerals may refer to similar or substantially similar components:

[0068] Figure 1 A schematic diagram of an embodiment of an optical microscope according to the present invention is shown; Figure 1A Shows Figure 1 Schematic diagram of an optical microscope sensor array detecting that a light spot is out of focus and therefore magnified; Figure 1B Shows that with Figure 1 Schematic representation of a detection spot with a different and generally more uniform intensity distribution;

[0069] Figure 2 Shows Figure 1 Schematic diagram of the optical microscope sensor array in the first setup;

[0070] Figure 3 Shows Figure 1 Schematic diagram of an optical microscope sensor array in a second setup using binned readout;

[0071] Figure 4 Shows Figure 1 Schematic diagram of an optical microscope sensor array in a third setup using binned readout;

[0072] Figure 5 Shows Figure 1 Schematic diagram of an optical microscope sensor array in a fourth setup using binned readout;

[0073] Figure 6 Shows Figure 1 Schematic diagram of an optical microscope sensor array in a fifth setup using binned readout;

[0074] Figure 7 Shows Figure 1 Schematic diagram of an optical microscope sensor array in a sixth setup using binned readout;

[0075] Figure 8 Shows Figure 1 Schematic diagram of an optical microscope sensor array in the seventh setup using binned readout;

[0076] Fig. 9 Shows Figure 1 Schematic diagram of an optical microscope sensor array in an eighth setup using binned readout;

[0077] Fig.10 Shows Figure 1 Schematic diagram of an optical microscope sensor array in a ninth setup using binned readout;

[0078] Fig.11 A schematic diagram showing a sensor array and a control unit of a light microscope of the present invention; and

[0079] Fig.12 A schematic flow chart illustrating the method of the present invention is shown. DETAILED DESCRIPTION

[0080] Figure 1 An embodiment of an optical microscope 100 of the present invention is schematically shown.

[0081] The optical microscope 100 includes a light source 10 that emits an illumination light 12. The light source 10 may include, for example, one or more lasers. The exemplary optical microscope 100 is formed as a laser scanning microscope. It includes a scanner 25 having one or more movable mirrors or other movable optical elements to scan the illumination light 12 over a sample 35. Optical elements 23, 24 may be used to direct the illumination light 12 from the light source 10 to an objective lens 30 via the scanner 25. The objective lens 30 focuses the illumination light 12 onto a sample point, and due to the scanning motion caused by the scanner 25, different sample points are illuminated successively. Two consecutive beam paths of the illumination light set using the scanner 25 are indicated by reference numerals 12a and 12b.

[0082] The sample 35 emits detection light 15 which may be, for example, fluorescent or phosphorescent light. The illumination light 12 may be pulsed and may in particular lead to multiphoton excitation of particles in the sample 35. Thus, the detection light 15 is emitted only from a small sample point and has a different (in particular smaller) wavelength than the illumination light 12.

[0083] In the depicted descanning setup, the detection light 15 is directed via the objective 30, the scanner 25 and the optical elements 23, 24 onto the same beam path as the illumination light 12. A beam splitter 22 is used to spatially separate the detection light 15 from the illumination light 12, which can be configured, for example, to transmit or reflect the impinging light according to its wavelength. The detection light 15 is then directed to the sensor array 60 via other optical elements 31, 32, 33, 34. An optional pinhole 18 for confocal designs can be arranged in the intermediate image plane.

[0084] The control device 70 controls the sensor array 60, the optical elements 33 and 34, the light source 10 and the scanner 25, and may also be configured to control other components of the optical microscope 100. These components may be collectively referred to as an optical assembly, which is a component of the optical microscope 100 but may also be generally used in other imaging systems.

[0085] Figure 1-4 , which, together with the control device, forms a detector which can be regarded as an independent aspect of the present invention and can be implemented in an optical microscope or another light measuring device. Thus, an embodiment of the control device of the present invention can be formed by Figure 1 The sensor array 60 and control device 70 shown in FIG. 1 are formed without other components in the optical microscope. Figures 2 to 4 As shown, the sensor array 60 includes a plurality of single photon detection elements 61 arranged adjacent to each other in a two-dimensional array, such as a hexagonal or rectangular arrangement. In particular, the single photon counting detector elements 61 can form a SPAD array (single photon avalanche diode). The detection light impact forms a detection spot 16 on the sensor array 60, as shown by the dotted circle. The control device is configured to activate or deactivate the detection element 61 individually, and the deactivated detection element is displayed in white and represented by the reference mark 63. Only the activated detection element outputs a measurement value (i.e., photon count rate) for further processing, such as calculating an image of the sample or determining the PSF. The control device is further configured to flexibly merge, read out, or combine any number of detector elements 61 into a superpixel. A superpixel includes or is connected to a multi-bit counter that outputs the cumulative photon count rate of the detector element. The number of superpixels and the number of detector elements in each superpixel can be flexibly set by the control device.

[0086] exist Figure 2 In the example of , the detector elements 62 illuminated by the detection spot 16 are activated rather than being binned for readout, resulting in a photon count rate of thirty-seven. Figure 2As shown, it is advantageous to deactivate a substantial number of non-illuminated detector elements. The area of ​​the sensor array 60 defined by activated detector elements 62 is also referred to as the "active area".

[0087] Figure 3 It is shown that several activated detector elements 62 are combined to read out a superpixel 65. Each superpixel 65 includes 7 detector elements 62. The superpixels 65 are shown with different fill patterns (slashes, squares, diamonds, etc.). Likewise, the non-illuminated detector elements 63 are deactivated. Figure 3 The detection spot 16 is larger than Figure 2 Therefore, more detector elements 62 are activated; however, due to the combined readout into 19 superpixels 65, only 19 photon count rates are output, so the data rate is quite low.

[0088] As another example, Figure 4 A situation is shown where all or nearly all detector elements 61 are illuminated, activated and combined for readout of several superpixels 65. This allows the data rate to be reduced while still using the photon count rate of all or nearly all detector elements 61.

[0089] Figure 5 A situation is shown where the detection spot / distribution 16 illuminates parts of the detector array 60. The activation area of ​​the detector array 60 is adjusted to match the detection spot 16. The merged readout mode is set to include one central superpixel 65 and several circumferential annular superpixels 65. The annular superpixels 65 are particularly suitable if the PSF is rotationally invariant. The diameters of the annular superpixels 65 are different. In the example shown, the thickness of each ring is defined by a layer of detector elements 61; however, alternatively, the outer ring can be thicker and therefore include two or more layers of detector elements 61.

[0090] exist Figure 6 In another merged readout configuration shown in , superpixels of different sizes are set. The inner superpixels around the center point can have a smaller size than the superpixels 65a-65d away from the center point.

[0091] exist Figure 7 In the example shown, the detector array 60 is divided into four quadrants, forming individual superpixels 65a-65d. The superpixels 65a-65d may be directly adjacent to each other or may be separated by inactive detector elements 63. Figure 8 A merged readout pattern for superpixels using different shapes is shown. The inner region is divided into several similarly sized superpixels 65c, 65d, such as several circular or hexagonal superpixels 65c, 65d. The outer region is divided into annular superpixels 65a, 65b. The annular superpixel 65a can be set to have a greater thickness than the annular superpixel 65b closer to the center of the ring.

[0092] Fig. 9 A binning readout mode that can be used with linear illumination is shown. If one or more longitudinal illumination spots are generated in the sample plane, for example using one or more cylindrical lenses, one or more longitudinal detection spots 16A, 16B can be formed on the detector array 60. Fig. 9 In the embodiment of the present invention, multiple adjacent rows (or lines) of detector elements are combined and read out into the same super region. Therefore, each super pixel 65A and 65B has a line shape, where the thickness of the line can be flexibly set. Fig. 9 The superpixels 65A, 65B may also be used with other illumination modes, for example, to select specific wavelength portions where the detection light is dispersed according to its wavelength.

[0093] Fig.10 The flexibility of the merged readout mode is further illustrated. It is not necessary to merge only adjacent detector elements into the same superpixel. Instead, a superpixel 65B or 65C can be composed of distant detector elements. For example, a first superpixel 65A can form a line, a second superpixel 65B can be formed by two sets of detector elements spaced apart from each other, and a third superpixel 65C can be formed by several individual detectors spaced apart from each other. One or more such superpixels can be used for calibration measurements, while other superpixels are assigned to actual sample measurements. Correlation measurements, in particular for different sample positions, can be performed with superpixels comprising distant detector elements.

[0094] The control device 70 is configured to set the binned readout mode and the activation area to increase the SNR, keep the data rate within acceptable boundaries and ensure the desired frame rate (which requires a limited amount of data per frame). In addition, the control device 70 is able to modify the size of the detection spot 16 on the sensor array 60 and influence the intensity distribution within the detection spot 16. To this end, the control device 70 adjusts the optical elements of the microscope. This adjustment will first be described before explaining the advantageous binned readout and activation modes.

[0095] like Figure 1 As shown, the optical elements 33, 34 in the beam path of the detection light 15 are adjustable to influence the detection light 15. The optical element 34 may be an imaging lens in front of the sensor array 60. It focuses the detection light 15 onto the sensor array 60, generating Figure 2-4 By adjusting the optical element 34, the focus of the detection light 15 is moved relative to the sensor array 60. As a result, the detection light spot is out of focus and thus amplified (e.g. Figure 1A). The optical element 34 can be moved in the propagation direction of the detection light 15; however, the optical element 34 can be moved in the propagation direction of the detection light 15. Alternatively, the optical element 34 can be an adjustable (liquid) lens that is deformed to change the focal position.

[0096] The optical element 33 may be a Bertrand lens 33. Figure 1 and Figure 1A In the case shown, the Bertrand lens 33 is moved out of the light path of the detection light 15. The control device can move the Bertrand lens 33 into the light path of the detection light 15, resulting in Figure 1B The Bertrand lens 33 images the pupil plane into the plane of the sensor array 60. Adjusting the optical element 34 in turn causes the size of the detection spot generated on the sensor array 60 to change. Figure 1B With Figure 1 A different and usually more uniform intensity distribution of the detection spot. In addition, Figure 1B In the case shown, the size of the detection beam spot can effectively be varied.

[0097] Fig.11 The architecture of the sensor array 60 and the control unit 70 of an embodiment of the optical microscope of the present invention is schematically shown. The sensor array 60 comprises a plurality of N1·N2 detector elements and is therefore configured to output a plurality of N1·N2 photon count rates, for example via respective signal lines, to a processing unit 71. The processing unit 71 is a component of the control device 70 and is formed in the same IC or on the same circuit board with the sensor array 60. The processing unit 71 comprises a number M of independent units 71a. The design of one such unit 71a is Fig.11Schematically shown in . The remaining units 71a can be formed similarly. Unit 71a includes a combination circuit 71d, and each detector element is connected to the combination circuit 71d via a respective corresponding switch 71c. Therefore, each unit 71a has some N1·N2 input lines and the same number of switches 71c to flexibly control which detector elements are connected to the combination circuit 71d. The combination circuit 71d outputs a combination signal representing the received signal. The combination circuit 71d may have only one output line leading to the super pixel counter unit 72. The combination circuit 71d can be implemented as a shared bus with a pull-up or pull-down driver controlled by the detector element output, an OR function, an XOR function or any other combination function. Each switch 71c can be controlled by a respective memory 71b, which can have 1 bit (indicating the "on" and "off" states of each switch 71c). Therefore, each combination circuit 71d may include multiple N1·N2 1-bit memories 71b. Each switch 71c can be implemented as an AND gate or a parallel PMOS+NMOS combination. The individual memory elements may be connected together to form a shift register. The output of the combinatorial circuit 71 d leads to a superpixel counter unit 72. There are thus M output lines from the processing unit 71, where M is less than the N1·N2 product. The superpixel counter unit 72 comprises at least one, preferably two, multi-bit counters per combinatorial circuit 71 d. By means of at least one multi-bit counter for each combinatorial circuit 71 d, the superpixel counter unit 72 is able to output a photon count rate indicative of the combined photon count rate of all detector elements connected to the corresponding combinatorial circuit 71 d via the respective switch 71 c.

[0098] If the superpixel counter unit 72 includes two multi-bit counters for each combinational circuit 71d, parallel counting and readout during the dwell time becomes possible. This means that while one multi-bit counter is being read out (and not counting further signals received and output by the corresponding combinational circuit), another multi-bit counter connected to the same combinational circuit is activated to count the signals received and output by that combinational circuit. This reduces any delay in the dwell time between superpixels due to sequential readout.

[0099] The readout of the multi-bit counters may be sequential, where only a portion of all multi-bit counters (and hence all superpixels) are read out to reduce the data rate.

[0100] Reference Fig.12 The flexibility of the described microscope and the advantageous effects of the preferred operating mode set by means of a control device are explained. Fig.12 A flow chart illustrating the steps of an exemplary method of the invention is shown. Unless otherwise stated, these steps may be performed by a control device, in particular by adjusting the components described above (eg optical elements, light sources and sensor arrays).

[0101] In step S1, a selection of imaging modes and imaging parameters is provided or displayed to the user. Depending on the user's selection, the method continues with the selected imaging mode S2, S6, S11 or S15.

[0102] If "PSF oversampling mode" S2 is selected, the method continues with steps S3-S5. The order of these steps can be varied, some or all steps can be performed simultaneously, or one or two steps can be omitted. In step S3, the optical elements 33, 34 are adjusted to clearly image the sample plane onto the sensor array 60. As described above, the Bertrand lens used for pupil imaging can be removed and the adjustable focus lens can be set so that a clear image of the sample plane is produced on the sensor array. The clear image constitutes a detection spot with a size and intensity distribution indicating the PSF of the system. Depending on the intensity / power of the detection beam, the optical elements 33, 34 are further adjusted in step S3 to adjust the detection spot size on the sensor array 60 to an appropriate size. Excessive intensity can cause saturation of the detector elements, that is, their output photon count rate deviates from a linear relationship with the impact photon rate or intensity. The saturation threshold can be defined as a light intensity above which the output photon count rate deviates from a linear relationship with the impact light intensity by more than 10%. To avoid saturation, the optical elements 33, 34 are controlled to increase the detection spot size as the detection beam power increases, thereby keeping the intensity of each detector element below a saturation threshold. In step S4, the activation area of ​​the sensor array is set according to the detection spot size on the sensor array. In particular, the activation area can match the spot size with a deviation of no more than, for example, 20% or 30% of the PSF size. Now, according to the selected imaging parameters, such as the required frame rate, step S5, the merged readout of the detector elements into superpixels is set. The frame rate and the maximum bandwidth required for data transmission from the control unit affect or define an upper threshold of the measurement data that can be output during the dwell time. The number of superpixels is set to correspond to (or not exceed) the upper threshold.

[0103] If "confocal detection mode" S6 is selected, the method continues with steps S7-S10. Likewise, the order of steps S7-S10 may vary, some or all of the steps may be performed simultaneously, and one or more of these steps may be omitted. In steps S7 and S8, the optical elements 33, 34 are adjusted to set the spot size according to the power or energy of the detection beam and to make the intensity distribution on the sensor array uniform. Reducing the spot size means irradiating a smaller number of detector elements with a greater intensity or optical power. For low intensities, the dark noise of the detector elements is important, so if the spot size is reduced, it may increase the SNR. On the contrary, for large intensities, the detector elements may saturate (their output photon count rate no longer depends linearly on the impinging intensity; in addition, the overall SNR deteriorates). Therefore, for large intensities, the spot size is increased to avoid excessive intensities on the detector elements. Therefore, the control device can use a predefined function that calculates the spot size to be set according to the beam power or energy (which is calculated or previously measured). As the intensity increases, the function outputs an increase in the spot size to be set. In the confocal detection mode, only the total output of the detector elements is relevant, without analyzing the intensity distribution on the detector elements. Therefore, the intensity distribution on the detector elements may be affected to meet other criteria: in particular, in order to avoid saturation of some detector elements due to excessive intensity, the optical elements 33, 34 are adjusted to make the intensity distribution more uniform. Since the spatial information from the individual detector elements is not used, a sharp image on the sensor array is not required. Therefore, as described above, the focusing lens 34 can be adjusted to change the spot size, and the Bertrand lens 33 can be optionally used to produce a pupil image (which may have a more uniform intensity distribution) and affect the spot size. Compared with zoom optics that allow the spot size to be adjusted while maintaining sharp imaging, the adjustment using only two optical elements 33, 34 is significantly less complex, thereby potentially reducing light losses and reducing costs. The use of such simplified optics is made possible by the flexibility provided by the SPAD sensor array 60. The above steps S7 and S8 can be performed as one step. In the next step S9, the activation area is set to match the spot size on the sensor array. Alternatively, the activation area size can be set larger than the spot size, for example with a margin of up to 20% of the spot size. In a simpler variant, all detector elements can be activated. In step S10, all activated detector elements are combined and read out into the same superpixel. The output value of the superpixel is therefore the combined photon count rate of all activated detector elements. This process reduces the data rate without losing information relevant to confocal imaging.

[0104] If "line scan mode" S11 is selected, the method continues with steps S12-S14. Likewise, the order of steps S12-S14 may vary, some or all steps may be performed simultaneously, or one or more of these steps may be omitted. In line scan mode, the optics in the illumination beam path are adjusted to provide an illumination line in the sample plane to be inspected. For example, a cylindrical lens may be inserted into the illumination beam path. In step S12, the optical elements 33, 34 are adjusted to image the sample plane sharply onto the sensor array. When a line or elongated area of ​​the sample is illuminated, detection light is emitted from the illuminated sample area and results in an elongated or line-shaped detection spot on the sensor array. In step S13, an activation area is set according to the elongated detection spot so that at least some of the non-illuminated detector elements are deactivated. In particular, only the illuminated detector elements are activated. Step S14 sets a merged readout mode. Detector elements adjacent to each other in a direction perpendicular to the longitudinal direction of the elongated detection spot are respectively merged and read out into the same superpixel. In contrast, detector elements that are adjacent to each other in the vertical direction are not binned and read out into the same superpixel (or only two or three adjacent detector elements in the vertical direction are binned and read out together to further increase the frame rate at the expense of resolution).

[0105] If "field scan / imaging mode" S15 is selected, the method continues with steps S16-S19. In step S16, the optics are adjusted to sharply image the sample plane onto the sensor array. In step S17, the wide field illumination is set, in contrast to e.g. the PSF oversampling mode S2 or the confocal detection mode S6, where the illumination is set so that only the sample points are illuminated. Next, in step S18, the activation area is set to correspond to the illumination area on the sensor array, which may depend on the currently inserted objective, the wide field illumination size or the zoom setting. In step S19, a binning mode is set, which may be selected, for example, depending on the current zoom position. .

[0106] In variations of the above method, further optical elements besides the optical elements 33, 34 may be provided and adjusted to change the spot size and / or light distribution. Other variations may provide additional imaging modes or not all three described imaging modes.

[0107] The method, detector and optical microscope of the invention provide particularly good imaging quality while offering great flexibility without the need for complex and cost-intensive zoom optics.

[0108] Reference Symbols List

[0109] 10 Light Source

[0110] 12 irradiation light

[0111] 12A, 12B Light path of the irradiation light 12

[0112] 15 Detection Light

[0113] 16,16A,16B Detection light spot on sensor array (size) / PSF 60

[0114] 18 Pinhole

[0115] 22 Beam splitter

[0116] 23, 24 Optical elements

[0117] 25 Scanner

[0118] 30 Objective

[0119] 31,32 Optical elements

[0120] 33 Optical elements, Bertrand lenses

[0121] 34 Optical elements, tunable lenses

[0122] 35 samples

[0123] 60 sensor array

[0124] 61 Photon counting detector element

[0125] 62 activated photon counting detector elements

[0126] 63 Deactivated photon counting detector elements

[0127] 65,65A-65D Super Pixel

[0128] 70 Control Equipment

[0129] 71 Processing unit of control device 70

[0130] 71a Processing unit 71

[0131] 71b Memory of unit 71a

[0132] 71c Switch of unit 71a

[0133] 71d Combinatorial circuit of unit 71a

[0134] 100 Optical microscope

[0135] N1·N2 Number of detector elements 61; Number of input lines of processing unit 71

[0136] M Number of units 71a; number of output lines of processing unit 71

[0137] S1-S19 Method Steps

Claims

1. An optical microscope comprising - a light source (10) for illuminating the sample (35), - the sensor array (60) consists of photon counting detector elements (61, 62) for measuring the detection light (15) from the sample (35), and - a control device (70) for controlling the sensor array (60), The control device (70) is configured to flexibly combine the photon counting detector elements (61, 62) to read out one or more superpixels (65), characterized in that The control device (70) is configured to adjust the combined readout and activation area of ​​the sensor array (60) to limit the resulting data rate based on the imaging parameters.

2. The optical microscope according to claim 1, wherein The control device (70) is configured to variably set the number of superpixels (65) and to variably set the number of photon counting detector elements (61, 62) that are combined for readout into one superpixel (65).

3. The optical microscope according to claim 1, wherein The control device (70) or a component of the control device (70) and the sensor array (60) are constructed on the same circuit board and / or in the same integrated circuit, Each photon counting detector element (61, 62) is connected to a control device (70) via a respective signal line, and the control device (70) is configured to assign a corresponding multi-bit counter to each superpixel (65).

4. The optical microscope according to claim 1, further comprising an adjustable optical element (33, 34) for adjusting a detection spot size (16) of the detection light (15) on the sensor array (60), wherein the control device (70) is configured to: - adjust the activation area of ​​the sensor array (60), - controlling the optical elements (33, 34) to reduce the detection spot size (16) on the sensor array (60) and to reduce the active area of ​​the sensor array (60) as the power of the detection light (15) decreases.

5. The optical microscope according to claim 1, wherein The control device (70) is arranged to set the combined readout and activation area of ​​the sensor array (60) based on the information of the detection spot size (16).

6. The optical microscope according to claim 1, wherein The control device (70) is configured to increase the active area of ​​the sensor array (60) and the average number of photon counting detector elements (61, 62) per superpixel (65) as the illumination wavelength increases and / or when the currently used objective lens is replaced with an objective lens with a smaller return aperture, and vice versa, and / or depending on the numerical aperture.

7. The optical microscope according to claim 1, The control device (70) is configured for a PSF oversampling mode (S2), wherein - setting the activation area of ​​the sensor array (60) according to the detection spot size (16) on the sensor array (60), - setting the number of superpixels (65) according to the detection spot size (16), and / or - Setting the number of photon counting detector elements (61, 62) combined for readout into the same superpixel (65) according to the desired maximum count rate.

8. The optical microscope according to claim 7, The control device (70) is configured to combine the number of photon counting detector elements (61, 62) read out into the same superpixel (65) as the detection spot size (16) increases in a PSF oversampling mode (S2) so that the data rate output by the control device (70) does not exceed a predetermined maximum rate and / or is constant.

9. The optical microscope according to claim 7, The control device (70) is configured to set the superpixels (65) in the PSF oversampling mode (S2) so that the center-to-center distance between adjacent superpixels (65) is at most 0.2 Airy disk diameters.

10. The optical microscope according to claim 1, The control device (70) is configured in a line scan mode (S11), wherein - the sample (35) is illuminated by an elongated illumination beam, resulting in an elongated detection spot on the sensor array (60), and - The control device (70) combines the readout of the photon counting detector elements (61, 62) together so that an elongated superpixel (65) is formed in a direction perpendicular to the elongated detection spot.

11. The optical microscope according to claim 1, The control device (70) is configured for a confocal detection mode (S6), wherein all activated photon counting detector elements (61, 62) are combined and read out into one superpixel (65).

12. The optical microscope according to claim 1, Wherein the control device (70) is configured in an array scanning mode, wherein the sample (35) is illuminated with a plurality of illumination points and the activation area and the binning readout of the photon counting detector elements (61, 62) depend on the set illumination settings.

13. The optical microscope according to claim 7, in, The control device (70) is configured to adjust the detection spot size (16) on the detector array (60) based on the impinging light intensity in the PSF oversampling mode (S2) or the confocal detection mode (S6) to avoid saturation of the photon counting detector elements (61, 62) and increase the SNR.

14. The optical microscope according to claim 7, in, The control device (70) is configured to - providing a plurality of imaging modes (S2, S6, S11, S15) to a user, wherein the imaging modes (S2, S6, S11, S15) include at least two of a PSF oversampling mode (S2), a confocal detection mode (S6), a line or array scanning mode (S11), and a field imaging mode (S15), and - setting a combined readout of the photon counting detector elements (61, 62) according to the selected imaging mode (S2, S6, S11, S15); in The control device (70) is configured to adjust the optical elements (33, 34) of the optical microscope according to the selected imaging mode (S2, S6, S11): - in a confocal detection mode (S6), adjusting the optical elements (33, 34) to homogenize the intensity distribution on the sensor array (60) by setting pupil imaging on the sensor array (60) and adjusting the detection spot size (16) on the sensor array (60); - In the PSF oversampling mode (S2), the optical elements (33, 34) are adjusted to image the sample points onto the sensor array (60) to record spatial information on the PSF.

15. The optical microscope according to claim 14, wherein One optical element (33) is a Bertrand lens (33), the other optical element (34) is an imaging lens (34), and The control device (70) is configured to: - setting pupil imaging by inserting a Bertrand lens (33) into the light path of the detection light (15), and - adjusting the detection spot size (16) by adjusting the focal length of the imaging lens (34) that focuses the detection light (15) near the sensor array (60).

16. An optical microscope according to any one of claims 1 to 15, wherein The control device (70) is configured to prompt a user to input imaging parameters and adjust the combined readout and activation area of ​​the sensor array (60) based on the imaging parameters to limit the resulting data rate.

17. An optical microscope according to any one of claims 1 to 15, wherein Imaging parameters are desired resolution and / or desired frame rate and / or desired signal-to-noise ratio.

18. An optical microscope according to any one of claims 1 to 15, wherein The control device (70) is configured to maintain an output data rate constant by adjusting the binning readout as the activation area of ​​the sensor array (60) changes.

19. The optical microscope according to claim 1, wherein In order to provide a maximum number M of flexibly configurable superpixels, the control device (70) includes a processing unit (71) having M units (71a), wherein each unit (71a) includes a respective combination circuit (71d), the combination circuit (71d) being connected to the photon counting detector elements (61, 62) via a programmable switch (71c), wherein each combination circuit (71d) outputs a signal flow indicating the combined photon count rate of all photon counting detector elements (61, 62) whose corresponding switch (71c) is closed to the combination circuit (71d).

20. The optical microscope according to claim 19, wherein The control device (70) comprises a superpixel counter unit (72) having M input lines, each input line being connected to a respective output of the combining circuit (71d), and Each of the M input lines leads to at least one multi-bit counter of the superpixel counter unit (72).

21. The optical microscope according to claim 20, wherein Each combinational circuit of the superpixel counter unit (72) includes two multi-bit counters, wherein each of the M input lines leads to a pair of multi-bit counters, wherein one of the multi-bit counters of the pair is read out while the other multi-bit counter calculates the incoming photon count rate.

22. The optical microscope according to any one of claims 1 to 15, wherein The control device (70) is configured to adjust the time delay between output data streams of different superpixels (65) according to the calibration matrix; and / or. The control device (70) is configured to set different sensitivities between the photon counting detector elements (61, 62) of the same superpixel (65).

23. An optical microscope according to any one of claims 1 to 15, Further including: a scanner (25) configured to scan the sample (35) with illumination light (12) and to direct detection light (15) to the sensor array (60), Therein, the control device (70) is configured to adjust the binning readout during scanning.

24. An optical detector comprising a sensor array (60) consisting of photon counting detector elements (61, 62), and a control device (70) for controlling the sensor array (60), The control device (70) is configured to flexibly combine the photon counting detector elements (61, 62) to read out one or more superpixels (65), characterized in that The control device (70) is configured to adjust the combined readout and activation area of ​​the sensor array (60) to limit the resulting data rate based on the imaging parameters.

25. The light detector of claim 24, wherein The control device (70) is configured to prompt a user to input imaging parameters and adjust the combined readout and activation area of ​​the sensor array (60) based on the imaging parameters to limit the resulting data rate.

26. A light detector according to claim 24 or 25, wherein Imaging parameters are desired resolution and / or desired frame rate and / or desired signal-to-noise ratio.

27. The light detector according to claim 24 or 25, further comprising an event activated active charging component for each photon counting detector element (61, 62); The control device (70) is configured to flexibly combine and read out continuous and / or discontinuous photon counting detector elements (61, 62) into the same superpixel (65).

28. An imaging method comprising: - irradiating the sample (35) with irradiation light (12), - measuring the detection light (15) from the sample (35) using a sensor array (60) consisting of photon counting detector elements (61, 62), and - controlling the sensor array (60) by means of a control device (70), - flexible merging and readout of photon counting detector elements (61, 62) into one or more superpixels (65), It is characterized in that - Adjusting the combined readout and activation area of ​​the sensor array (60) to limit the resulting data rate based on imaging parameters.

29. The imaging method according to claim 28, further comprising The user is prompted to enter imaging parameters, and the combined readout and activation areas of the sensor array (60) are adjusted based on the imaging parameters to limit the resulting data rate.

30. The imaging method according to claim 28 or 29, wherein Imaging parameters are desired resolution and / or desired frame rate and / or desired signal-to-noise ratio.

Citation Information

Cited By

  • Single-photon kilohertz calcium imaging system

    CN119880865A

  • Single-photon kilohertz calcium imaging system

    CN119880865B