Imaging apparatus

Abstract

Imaging apparatus, comprising a laser illumination unit (10) configured to generate a laser beam; a spatial modulation system (20) configured to divide the laser beam into a plurality of illumination beams; an illumination optic (30) configured to focus the illumination beams towards respective points (P1,..., Pi) of a target (S; 100); a collection optic (40) configured to collect light emitted or scattered by the points (P1,..., Pi) of the target (S; 100); a two-dimensional array of single-photon detectors (50) optically coupled to the collection optic (40) such that the light emitted or scattered by each of the points (P1,..., Pi) of the target (S) is received by a respective detector of the two-dimensional array of single-photon detectors (50), each of the detectors acting as a pinhole; and a processing and control system (60) coupled to the array of detectors (50) and operable to produce images of the respective points (P1,..., Pi) of the target (S).

Description

[0001] Imaging apparatus

[0002] The present invention relates to optical imaging techniques.

[0003] Image scanning microscopy (ISM) is an advanced microscopy technique that overcomes certain limitations of confocal scanning microscopy in order to obtain high-resolution and high-contrast images. The main object is to overcome the trade-off between spatial resolution and signal -to-noise ratio. WO 2019 / 145889 Al describes a solution that uses a detector array, by means of which it is possible to double the spatial resolution. WO 2023 / 275777 Al describes a solution that uses a detector with single-photon resolution, by means of which it is not only possible to double the spatial resolution, but also to perform FLIM (fluorescence lifetime image microscopy) measurements, based on fluorescence lifetime. The combination of the two inventions allows, for the first time, to obtain superresolution FLIM images. These inventions have closed the gap that existed between superresolution microscopy and spectroscopy.

[0004] However, the solution described in WO 2019 / 145889 Al does not allow for multicolour imaging, except by simply multiplying the number of sensors. This solution is neither practical nor scalable, as it increases the costs and hardware resources required for a single measurement excessively. WO 2023 / 275777 Al attempts to address this limitation by introducing a spectral encoder / decoder and using the time channel to perform colour unmixing. However, it is a solution that “sacrifices” the acquired temporal information.

[0005] More generally, all scanning techniques require a long time to acquire a single image, as the image is collected one pixel / voxel at a time. The scanning time scales with the square of the number of pixels (in 2D) or with the cube (in 3D). This could be particularly disadvantageous in the case of ex-vivo or in-vivo applications.

[0006] In some applications, especially in-vivo ones, it may be necessary (possibly even within the same measurement) to have different resolutions for different areas of the image. For example, in tumour imaging, it is desirable to have resolution at the edges, to ensure that no area of infiltration into healthy tissue is missed, high resolution near blood vessels, while it is possible to favour acquisition speed in other areas. In other applications, resolution can instead be prioritised, accepting possibly longer scanning times. In yet other applications, speed can instead be prioritised, aiming to obtain the best possible image in the shortest possible time.

[0007] In some applications, multicolour imaging is desired, in others a single wavelength is used. In some cases, different wavelengths are used simultaneously, in others at successive times (per pixel, per line or per frame).

[0008] A necessary condition for super-resolution imaging (i.e., beyond the diffraction limit) is that the size, projected on the image plane, of each element of the detector array is smaller than 1 airy unit. It has been demonstrated that if 1AU is collected by at least 2x2 sensors, it is already possible to obtain a super-resolution image. However, the size of the airy disk is an arbitrary unit depending on the wavelength used, the physical magnification of the system, and the numerical aperture of the objective.

[0009] An object of the present invention is to make available a solution that allows high-speed imaging.

[0010] In view of this object, the subject of the invention is an imaging apparatus, comprising a laser illumination unit configured to generate a laser beam; a spatial modulation system configured to control at least one of intensity, phase and polarisation of the laser beam in a spatially variable manner such that the laser beam is divided into a plurality of illumination beams; an illumination optic configured to focus said illumination beams towards respective points of a target; a collection optic configured to collect light emitted or scattered by said points of the target; a two-dimensional array of single-photon detectors optically coupled to the collection optic such that light emitted or scattered by each of said points of the target is received by a respective detector of the two-dimensional array of single-photon detectors, each of said detectors acting as a pinhole; and a processing and control system coupled to the two-dimensional array of singlephoton detectors and operable to produce images of said respective points of the target from a signal provided by the two-dimensional array of single-photon detectors, wherein the collection optic comprises a magnification system such that the size of each detector of the two-dimensional array of single-photon detectors, projected on an image plane of the magnification system, is less than 1 airy unit.

[0011] The spatial modulation system allows the creation of multiple beams, ideally up to a number equal to the number of detectors in the detector array. In the case that all beams can be well focused and a perfect match with the scanning conditions is achieved, a parallel confocal with a plurality of beams would be obtained, confocal resolution / optical sectioning and speed increased by a number of times equal to the number of beams. If these conditions are not met in the current scan, high-speed imaging would still be achieved, and in any case, always time-resolved (FLIM).

[0012] Features and advantages of the proposed apparatus will be presented in the following detailed description, which refers to the accompanying drawings, provided solely by way of non-limiting example, wherein:

[0013] - figure 1 shows a block diagram of a microscope according to the invention;

[0014] - figure 2 shows a schematic representation of a detector of the microscope of figure 1 (left) and a graphical representation of the point spread function (PSF) relative to the detector according to three different configurations of the microscope (centre, upper right and lower right);

[0015] - figure 3 shows the characteristics of the single element of an example of a matrix sensor usable in the present invention: at the upper left the resolution curves (number of detected photons vs number of incident photons in the sensor) are shown for two different values of dead time (no dead time and dead time equal to 100ns); in the centre the fill factor as a function of the opening angle of the incident beam is shown; at the upper right the detection efficiency as a function of the photon wavelength is shown;

[0016] - figure 4 shows an example of a reconstruction of a high-resolution intensity image. At the top, from left to right: series of intensity images g obtained with a 7 arrays by 7 detectors SPAD, showing the nucleus of a cell; fingerprint image a calculated from g; drift matrix dxand dyestimated from g; PSFs hij calculated for each element of the SPAD array obtained from the excitation PSF hexc, the detection PSF hdet, and the displacement matrices sxand syestimated. At the bottom, from left to right: intensity image recorded by the central pixel gicjc,' low-resolution conventional CLSM intensity image; intensity image reconstructed by multi-image deconvolution;

[0017] - figure 5 shows an example of high-resolution intensity image reconstruction, in a multicolour configuration. At the top, from left to right: four series of intensity images g obtained with respective 3 by 3 subsets of a 7 by 7 SPAD detector array (each subset being exposed to photons having a wavelength different from the other subsets), representing a four-colour image with the nucleus of a cell at the centre, specifically: upper left, nuclear pore complex labelled with anti-nup 153 + AlexFluor 405; upper right, actin filaments labelled with Phalloidin + Alexafluor 488; lower left, tubulin labelled with anti-beta-tubulin + Al exaFluor 555; lower right, golgi labelled with golgi complex antibody + Al exaFluor 647; fingerprint images a calculated from g for each subset; drift matrices dxand dyestimated from g for each subset; PSFs hij calculated for each element of the SPAD array obtained from the excitation PSF hexc, the detection PSF hdet, and the displacement matrices sxand syestimated for each subset. At the bottom, from left to right: intensity image recorded by the central pixel g;c,7Cfor each subset; conventional low-resolution CLSM intensity image for each subset; pixel reassignment of the image relating to one of the subsets; intensity image reconstructed by multi-image deconvolution relating to one of the subsets;

[0018] - figure 6 shows an apparatus according to the present invention used for the improvement of optical sectioning;

[0019] - figures 7a and 7b respectively show an apparatus according to the invention configured for surgical microscopy, and an apparatus according to the invention configured for endoscopy;

[0020] - figure 8 is a diagram representing different configurations of the apparatus according to the invention.

[0021] With reference to figure 1, a laser scanning microscope apparatus is now described. In summary, the apparatus comprises a laser illumination unit 10 configured to generate a laser beam; a spatial modulation system 20 configured to control at least one of intensity, phase and polarisation of the laser beam in a spatially variable manner; an illumination optic 30; a collection optic 40; a two-dimensional array of single-photon detectors 50 optically coupled to the collection optic 40, each of said detectors acting as a pinhole; and a processing and control system 60 coupled to the detector array.

[0022] In the illustrated example, the illumination unit 10 comprises a tunable laser source 11 at visible wavelengths and one or more fixed-wavelength laser sources in the visible range 12, 13, 14. The laser sources may be continuous wave or pulsed, optionally amplitude- modulable. In the case of pulsed lasers, they may be capable of receiving an external trigger or providing a trigger themselves. Alternatively, a so-called "supercontinuum" laser source may be used, capable of generating photons in a very broad spectrum band. Associated with each source there is a mirror 1 la or a dichroic filter 12a, 13a, 14a which direct the respective beams to a modulator 15 such as an acousto-optic modulator (AOM) or a tunable acoustooptic filter (AOTF), optionally provided to select specific wavelengths. The illumination unit 10 may be coupled to the downstream optical part via optical fibre, waveguide or directly “in air”.

[0023] The illumination optic 30 comprises a dichroic filter 31 and an objective lens system 32 through which the laser beam or laser beams is / are focused on an object S to be analysed. 70 denotes a system for scanning the object to be analysed, made for example as a galvanometric mirror and / or with a movement mechanism that moves the sample holder 71 which supports the object S. For example, in the illustrated example the scanning along the x and y axes (parallel to the sample plane) is carried out by controllably deflecting the illumination beam / beams, while the scanning along the z axis (orthogonal to the sample plane) is carried out by controllably moving the sample holder 71. In some embodiments, the scanning system could be absent.

[0024] The fluorescence signal emitted by the object is collected by the same objective lens system 32 and transmitted through the dichroic filter 31 and a magnification system (zoom lens) 41 towards the array of single-photon detectors 50. The magnification system is capable of arbitrarily defining the size of the detector on the image plane (to comply with the physical condition of being sub-airy). This allows the system to be compatible with a wide range of experimental conditions: different objectives (magnification / numerical aperture), different wavelengths.

[0025] The detector array 50 is represented by a (two-dimensional) matrix of MxxMydetector elements, each of which is independent (fully parallel system), has single-photon level sensitivity and has a temporal resolution (low temporal jitter) such as to allow the measurement of the fluorescence lifetime of the most common fluorophores used in fluorescence microscopy. The operating parameters of the detector array (excess bias voltage, hold-off time and the number of active elements) can be adjusted before each measurement (alternatively, the system can be configured to autonomously decide what are the best conditions for the measurement in question). The detectors of the array 50 therefore provide photon-by-photon information resolved in time.

[0026] Each element of the detector array 23 has a circular active area (other shapes can be used, such as square or hexagonal), surrounded by an inactive frame (figure 2, left). Analogously to other pixel-based devices, the pixel pitch can be defined as the distance between (the centroids of) two adjacent pixels (if the pixel is square such pixels lie on the same row or on the same column). An important feature of the detector is the fill factor (see fig. 3), which can be calculated as the ratio between the active area and the square of the pixel pitch. The overall photon detection efficiency (PDE) of the detector, i.e., the probability that a photon reaching the detector is recorded, directly depends on the fill factor. To further improve the PDE, a microlens array 42 is used to direct the photons toward the centre of each detector element. In figure 2, the active area of the individual detector elements is represented by dashed circles, while the microlenses are represented by solid-line circles. Each element of the detector array signals the arrival of a photon with a TTL logic signal (transistor-transistor logic) on a dedicated digital channel. Specifically, the detector generates an LVDS (differential) signal, and the system converts it to TTL before supplying it to the FPGA board. Some FPGAs directly read the LVDS signal, so the conversion would not be necessary. Nothing would change with other formats, such as NIM. What is exploited is the fact that the detector output is a digital signal indicating the arrival of a photon: therefore photon-resolved and time-resolved. Three additional digital lines constitute a communication bus useful for the initial configuration of the entire array during the start-up of the measurement. The temporal information can be collected using two approaches: time domain and frequency domain. The time domain can be implemented on FPGA or through ASIC components. Both approaches are included in the present invention.

[0027] The processing and control system 60 may be developed with a field-programmable-gate- array (FPGA) processor. This allows all the photons collected during the dwell time on a single point n to be integrated (intensity mode) or the arrival times to be measured with respect to an external reference signal (e.g., the excitation laser pulse) thanks to the onboard integration of time-to-digital converters (TDC) (time-resolved mode or TCSPC).

[0028] It is important to underline that the detector and the data acquisition electronics are configured to operate in a completely asynchronous manner, i.e., when a photon is detected, it is either counted or its arrival time is measured, and the elements are independent from each other, without a limited frame-rate or the drawbacks of sequential reading.

[0029] Communication (synchronisation) with the microscope control system (denoted by 18 in figure 1) is carried out via digital lines for pixel / line / frame clocking, provided by the manufacturer.

[0030] The spatial modulation system 20 is associated with the illumination optic 30 and optionally also with the collection optic 40 and is configured to control at least one of intensity, phase and polarisation of the laser beam in a spatially variable manner, so as to engineer the illumination beam or beams incident on the sample, and to engineer the light emitted or scattered by the sample and collected by the collection optic 40. In the illustrated example, the spatial modulation system 20 comprises a spatial light modulator (SLM) which may be implemented with one of the technologies known in the field, for example with liquid crystals (LC-SLM), or with a digital micromirror device (DMD-SLM). A possible implementation of the spatial modulation system 20 is shown in the lower right box in figure 1, wherein 21 denotes the SLM. An incoming light beam IB is transmitted to the SLM 21 through an optical system and here manipulated to obtain an outgoing light beam / multiple outgoing light beams OB as required. The term “patterns” denotes a control signal through which a processing and control system 60 governs the spatial modulation system 20.

[0031] Specifically, the spatial modulation system is adjustable between a high-speed configuration (shown at the far right in figure 8) in which the spatial modulation system 20 divides the single laser beam coming from the illumination unit 10 into a plurality of illumination beams, and a high-resolution configuration (shown at the far left in figure 8) in which the single laser beam coming from the illumination unit 10 is undivided. Similarly, in the high-speed configuration the spatial modulation system 20 divides the light emitted or scattered by the object S into a plurality of signal beams, and in the high-resolution configuration the light emitted or scattered by the object S is collected in a single beam. In the high-speed configuration, the maximum speed of reconstruction of the image of the object S can be achieved, whereas in the high-resolution configuration, the maximum resolution of reconstruction of the image of the object S can be achieved.

[0032] In the high-speed configuration, there is one illumination beam for each sensitive element of the detector array 50, or for each sensitive element of a selection of detectors of said array. Thus, if said array is composed of 7 x 7 elements, up to 49 illumination beams may be obtained. In the case where the 49 beams can be well focused and a perfect match with the scanning conditions is achieved, a 49 beams parallel confocal would be obtained, confocal resolution / optical sectioning and speed 50 times higher. If these conditions are not met in the current scan, high-speed imaging would still be achieved, and in any case, always time- resolved (FLIM). The illumination optic 30 is configured to focus the illumination beams towards respective points Pl, . . ., Pi of the object S. The collection optic 40 is configured to collect the light emitted or scattered by said points Pl, ..., Pi of the target S. The singlephoton detectors 50 are optically coupled to the collection optic 40 such that the light emitted or scattered by each of said points Pl, . . . , Pi of the target is received by a respective detector of the two-dimensional array of single-photon detectors. Each of the detectors acts as a pinhole.

[0033] The processing and control system 60 is therefore capable of producing images of the respective points Pl, ..., Pi of the target and, in particular, through parallel scanning by means of the illumination beams, images of respective portions of the object S.

[0034] In the high-resolution configuration, the illumination optic 30 is configured to focus the single laser beam towards a point P of the object. The collection optic 40 is configured to collect the light emitted or scattered by the point P of the object. The single-photon detectors are optically coupled to the collection optic 40 such that the light emitted or scattered by the point P of the object is received by each of the detectors of the two-dimensional array of single-photon detectors. Each of the detectors acts as a pinhole. The processing and control system 60 is capable of producing at least one of a high-resolution image or a time-resolved image of said point of the target by processing a plurality of detection signals respectively provided by the single-photon detectors. In particular, with a scan through the above- mentioned laser beam, at least one of a high-resolution image or a time-resolved image of the object S is produced.

[0035] It is basically possible to sample at Nyquist rather than Nyquist / 2, gaining a factor of 4 in terms of speed [1], This without losing super-resolution, and with a single beam.

[0036] The process by which the high -resolution image or the time-resolved image of the object S is produced is described in WO 2019 / 145889 Al, but for convenience the description is reiterated below. In this regard, fig. 4 shows the various steps of image reconstruction.

[0037] Step 1. Record the time-resolved image series / matrix (or TCSPC) g' with the architecture described above.

[0038] For different positions of the focused excitation laser beam (single-photon or multi -photon) on the sample, the signals generated by the elements of the detector array 23 are read, and the time-resolved image series / matrix g' is obtained con Essentially, each image g-7(n, t)is a three-dimensional matrix wherein the temporal axis t reports the histogram of the arrival time of photons obtained through TCSPC measurement, i.e., the number of photons collected in a spatial pixel (nx,ny) and in a certain temporal window (temporal bin) t from the excitation event.

[0039] Step 2. Calculate the intensity image series g from g'

[0040] Given the TCSPC matrix of images g' collected with the apparatus described above in TCSPC mode, the matrix is integrated along the temporal dimension and the intensity image matrix g is obtained

[0041] Step 3. Calculate the “digital fingerprint” image a

[0042] Given the intensity image matrix g, the so-called “digital fingerprint” image a is calculated, from which the excitation PSF hexcand the detection PSF hrfe / are estimated.

[0043] The fingerprint image a is defined as follows. All photons collected by each detector element during an entire measurement are integrated, producing the fingerprint image a. In practice, during a single experiment MY / MVimages are obtained, and the fingerprint image a is produced by summing all the intensity values image by image:

[0044] To understand the properties of the fingerprint image a and how to obtain the PSFs from it, it is important to derive a in the continuous domain.

[0045] Considering a detector array composed of infinitesimal elements, the image gxliy, acquired by an element at the position (%', y') E IR2can be expressed as wherein / is the object / sample function, hx'y> denotes the PSF associated with the detector element at position (x',y'), and * denotes the convolution operator; the fingerprint image a(x',y'), defined with respect to the detector coordinates, is

[0046] Applying the integration property of convolution, the fingerprint image is wherein is the total photon flux from the sample. Note that a(x',y') is independent of the sample under the condition <I»0 (<E>»0), but it is strictly connected to the PSF of the microscope system.

[0047] Recalling that the PSF of each infinitesimal element is

[0048] = hexc(x, y) ■ hdet(x — x',y — y') and substituting into the previous equation, it is possible to obtain wherein * denotes the correlation operator. In summary, the fingerprint image is instrument-dependent and sample-independent. Moreover, it depends on both the excitation PSF and the detection PSF.

[0049] It is noted that the fingerprint image can also be used to align the system. In particular, to co-align the excitation PSF and the detection PSF on the central pixel (ic, jc) of the detector array. This procedure is very important for a detector with a small number of elements. A misalignment results in a loss of fluorescence photons. If the system is correctly aligned, the central pixel is the brightest and the pixel intensity values are distributed symmetrically and isotropically with respect to the centre. It is thus possible to implement a feedback control system that measures the fingerprint image and accordingly adjusts the xy position of the detector to maximise the central pixel intensity.

[0050] Step 4. Estimate the displacement matrices s ’ and s-r. the drift matrices dvand c , and the microscope magnification Magn

[0051] Given the intensity image matrix g, the displacement matrices sYand s’7are calculated.

[0052] As described in the pixel reassignment method, each image g£C;7Cis translated (in the image plane) with respect to giC,jc by half the distance between the element (z,j) and the element (zcjc), i.e., Therefore, the displacement matrices sYand s-Ycan be estimated by recovering the displacement between the different images g£;7. A phase correlation method is used for its noise resilience and greater speed compared to spatial domain algorithms. Phase correlation estimates the shift between two similar images based on a frequency domain representation of the data, which in this description is obtained via fast Fourier transforms (FFT).

[0053] To calculate the phase correlation between the two different sample images (g£;7and g£C;7C), the so-called correlogram r£ / 7is first defined: and subsequently the maximum of the correlogram is found, whose position denotes the drift between the two images:

[0054] The position of the maximum is obtained using a fitting algorithm or a centroid-based algorithm to obtain sub-pixel values, wherein d* / y(t,j) E IR.

[0055] Given the drift matrices dYand cP, the displacement matrices sYand s’7can be calculated as follows:

[0056] Other approaches for estimating the displacement matrices use: (i) a theoretical model, based on the physical distance between the detector elements and the system magnification, (ii) a calibration sample, e.g., beads. However, such approaches do not account for the peculiarities of each sample and the conditions of the specific measurement. Moreover, the sample magnification is not always easy to estimate. On the other hand, the above-described approach is sensitive to the assumption of a Gaussian shape for the excitation PSF and for the detection PSF. However, for (zj) within the first Airy disk centred at (ic,jc\ the assumption is solid and the estimation of is robust. An optimal approach could integrate into the estimation of the correlogram maximum some constraints based on knowledge related to the geometric shape of the detector and the system magnification.

[0057] The magnification Magn of the system can be determined using the estimated values for the first-order neighbours (A) of the element (ic,jc\ i.e., (zc+ ljc), (zc-l,jc), (zcjc+l) and (zc,jc-l), together with the detector pixel pitch (PP) and the image pixel size (DP): Magn

[0058] Step 5. Calculate the time-resolved object function f.

[0059] Given the matrix of time-resolved images g( the fingerprint image a, and the displacement matrices s\ s-v, an estimate of the object function f is calculated as described below (steps 5.1-5.3).

[0060] Step 5.1. Estimate the excitation PSF h and the detection PSF hdet

[0061] Based on the relation with the fingerprint image a described above, the excitation PSF hexcand the detection PSF hdetcan be estimated according to the minimisation problem or in the case of a PSF parametrisation. Here, the MSE functional is

[0062] The minimisation of the MSE function can be performed with numerical techniques according to known practices.

[0063] Step 5.2. Calculate the time-resolved object function using multi-image deconvolution (MID).

[0064] Since all the information regarding the PSFs (including the displacement values) has been previously estimated, the original problem can be solved using conventional multi-image deconvolution, in particular by minimising the Kullback-Leibler (KL) distance or the mean square error (MSE) distance

[0065] In the time-resolved case, the KL distance is

[0066] The minimisation of the MSE or KL functional can be performed with numerical techniques according to known practices.

[0067] Step 5.3. Calculate the time-resolved object function f using pixel reassignment (PR).

[0068] Following the pixel reassignment approach, a high-resolution time-resolved image can be obtained simply by summing all images after shifting back each image g ■ j by the estimated amount d / y(i,j):

[0069] Essentially, each 2D image associated with each temporal bin and each detector element is shifted independently. For this reason, both the FFT and the inverse FFT are performed in 2D.

[0070] Step 6. Calculate the intensity object function f.

[0071] Given: (i) the intensity image matrix g, the fingerprint image a, and the displacement matrices sx, s’7or (ii) the previously estimated time-resolved object function F, an estimate of the intensity object function f is calculated as described below (steps 6.1-6.3).

[0072] Step 6.1. Calculate the intensity object function f by temporal integration.

[0073] Given the high-resolution time-resolved image, it is possible to obtain a high-resolution intensity image by integrating the temporal dimension of the reconstructed time-resolved object function f

[0074] Step 6.2. Calculate the intensity object function f by multi-image deconvolution.

[0075] Given the excitation PSF hexc, the detection PSF hrfe / , and the displacement matrices s\ s’) the intensity object function f can be calculated directly from the intensity image series g by multi-image deconvolution and without estimating the time-resolved object function F, with a substantial reduction in computational effort.

[0076] In this case, it is necessary to minimise the KL or MSE distance only with respect to f The minimisation of the MSE or KL functional can be performed with numerical techniques according to known practices.

[0077] Step 6.3. Calculate the intensity object function f by pixel reassignment.

[0078] Given the displacement matrices sx, s-v, the intensity object function f can be calculated directly from the intensity image series g by pixel reassignment and without estimating the time-resolved object function f, with a substantial reduction in computational effort.

[0079] In this case, the pixel reassignment estimate is

[0080] If the microscope apparatus operates in intensity mode (i.e., without performing TCSPC measurements), only the intensity image matrix g is generated. In this case, only the methods proposed in steps 6.2 and 6.3 can be used. For a laser beam operating in continuous wave mode, the importance of recording the signal in TCSPC mode decreases.

[0081] Preferably, the spatial modulation system is also adjustable to at least one intermediate configuration between the high-speed configuration and the high-resolution configuration (two different intermediate configurations are shown in the centre of figure 8). In such intermediate configuration(s), the spatial modulation system divides the laser beam into a plurality of illumination beams (in a number of beams still lower than that of the high-speed configuration), but it is possible to reconstruct high-resolution or time-resolved images according to the procedure described above. The upper right box of figure 2 shows the PSF obtainable in the case of four parallel monochromatic beams, each of which affects a subset of nine detector elements of the array 50.

[0082] In the intermediate configuration, the illumination optic 30 is configured to focus the illumination beams towards respective points Pl, . . . , Pj of the object. The collection optic 40 is configured to collect the light emitted or scattered by said points Pl, . . . , Pj of the object. The single-photon detectors are optically coupled to the collection optic 40 such that the light emitted or scattered by each of the points Pl,. . Pj of the object is received by each of the detectors of a respective subset of detectors of the two-dimensional array of singlephoton detectors 50. Each of the detectors acts as a pinhole. The processing and control system 60 is capable of producing at least one of a high-resolution image or a time-resolved image of each of said points of the target by processing a plurality of detection signals respectively provided by the detectors of the respective subset of detectors of the two- dimensional array of single-photon detectors. With a parallel scan through said illumination beams, at least one of a high-resolution image or a time-resolved image of respective portions of the target is produced. For each portion of the target, the reconstruction procedure described above is applied.

[0083] It is also possible to realise a further type of intermediate configuration in which the illumination unit 10 and the spatial modulation system 20 are operated in such a way as to generate multiple illumination beams having different wavelengths, which are focused on a point P of the object. Indeed, different areas of the sensor can be used for imaging with different colours: not all sensors of the array 50 are necessary to achieve super-resolution; for example, 3x3 sub-areas are sufficient if the 1 AU condition is met. Thus, the beams are directed onto different areas of the sensor using the aiming optics of the system.

[0084] However, even in the case of sequential monochromatic / multicolour imaging, a problem is that different wavelengths may follow slightly different paths. The system described above allows for correction of this chromatic aberration by computing the centre of mass of the fingerprint digital and registering everything to it (as previously described and as disclosed in WO 2019 / 145889 Al).

[0085] In the multicolour configuration, the collection optic 40 is configured to collect the light emitted or scattered from point P of the object under analysis. The single-photon detectors are optically coupled to the collection optic 40 such that the light emitted or scattered from point P of the object and having a given wavelength is received by each of the detectors of a respective subset of detectors of the two-dimensional array of single-photon detectors 50. Each of the detectors acts as a pinhole. The lower right box of figure 2 shows the PSF obtainable in the case of four beams having different wavelengths in parallel, each of which affects a subset of nine detector elements of the array 50. The processing and control system 60 is capable of producing at least one of a high-resolution image or a time-resolved image of the point of the target, relative to the given wavelength, by processing a plurality of detection signals respectively provided by the detectors of the respective subset of detectors of the two-dimensional array of single-photon detectors. With a parallel scan through said illumination beams, at least one of a high-resolution image or a time-resolved image of respective portions of the target is produced, for each of the wavelengths used. For each wavelength, the reconstruction procedure described above is applied. In this regard, figure 5 shows the various steps of image reconstruction in a manner analogous to that used for the single beam in figure 4.

[0086] It is also possible to realise a high-speed multicolour configuration in which the illumination unit 10 and the spatial modulation system 20 are operated so as to generate multiple illumination beams having different wavelengths, which are focused on a point P of the object. In this case, the collection optic 40 is configured to collect the light emitted or scattered from point P of the object under analysis. The single-photon detectors are optically coupled to the collection optic 40 such that the light emitted or scattered from point P of the object and having a given wavelength is received by a respective detector of the two- dimensional array of single-photon detectors 50. Each of the detectors acts as a pinhole. The processing and control system 60 is operable to produce digital images of said point P of the target, each of said digital images being associated with a respective wavelength. In the case where a scanning system 70 is present and associated with at least one of said illumination optic 30, collection optic 40 and target S, the processing and control system 60 is coupled to the scanning system 70 and is operable to produce, through parallel scanning by means of said illumination beams, digital images of respective portions of the target, each of said digital images being associated with a respective wavelength.

[0087] With reference to figure 6, it is noted that the apparatus described above allows improvement of optical sectioning through an algorithm capable of discriminating out-of-focus light from in-focus light, such as that described in [2], The idea underlying this concept consists in observing the distribution of light on the detector array in order to distinguish the axial position of the emitters. The central element of the detector array mainly contains the infocus signal, whereas the out-of-focus light dominates the outer elements. The same information can be extracted by calculating axial fingerprints, that is, by integrating, at different depths z, the scanned 3D PSFs over the scanning coordinates (x, y). When the emitter is in focus (left image in figure 6), the central pixels contain the majority of the signal. The further away from the focal plane, the more the outer pixels of the fingerprint populate with photons (right image in figure 6). This trend can be quantified by calculating the ratio between the intensity of the outer pixels and the intensity collected by the central pixels.

[0088] Another possible application of the apparatus described above concerns single particle tracking (SPT). Currently, SPT is performed using confocal systems with a single detector. Therefore, scanning is necessary, for example through an orbit (“orbital scanning”), to determine where the particle (which starts at the centre of the orbit) moves and to follow it. With the architecture described above, orbital scanning is not necessary; it is sufficient to move in the direction of the centroid for scanning. There are two levels of computation: a rough level for real-time tracking to keep the particle always at the centre, to be implemented on FPGA or another high-speed architecture, and a finer level performed a posteriori on the raw data collected.

[0089] Figures 7a and 7b show two additional possible implementations of the invention. Figure 7a shows an implementation for surgical microscopy. Reference 100 denotes a target or region of interest for a medical procedure on a patient, for example a surgical cavity. Reference 200 denotes a microscopy apparatus having substantially the same elements shown in figure 1. The scanning system, here denoted by 70’, is configured to operate along the x and y directions parallel to the focal plane, while no scanning is provided along the z axis. Reference 32 denotes an objective lens system through which the laser beam or laser beams from the apparatus 200 is / are focused on the target 100, through an optical fibre bundle 33 and a distal lens system 34 optically coupled to the objective lens system 32. Conversely, the signal emitted or scattered by the region of interest 100 is transmitted to the apparatus 200 through the distal lens system 34, the optical fibre bundle 33, and the objective lens system 32. As for functionality, they are substantially the same as the architecture shown in figure 1, except for the sample holder movable along the z axis.

[0090] Figure 7b shows an implementation for an endoscopic probe. Reference 100 denotes a target or region of interest for a medical procedure on a patient, for example a surgical cavity. Reference 300 denotes an apparatus having substantially the same elements shown in figure 1, but equipped with a special objective system + optical fibre bundle + distal lens (for example, GRIN lens) allowing use of an optical fibre bundle and decoupling the apparatus from the probe. Reference 310 denotes a wide-field light source. A coupling optic 301 and a distribution optic 311 respectively provide a laser illumination beam and a wide-field illumination beam to the region of interest 100. The light emitted or scattered by the region of interest 100 is collected by the coupling optic 301. The collected light includes fluorescence light emitted by any fluorophores present in the field of view (illuminated by the excitation light). In addition, the collected light includes reflected light (in the visible spectrum) reflected from any objects present in the field of view (illuminated by the white light). A first beam splitter 321, for example a dichroic filter, splits the collected light into two channels. In the reflected channel of the first beam splitter 321, a photographic camera 325 (e.g., CCD type) receives the reflectance light and generates a corresponding (digital) reflectance image representing what is visible in the field of view. In the transmitted channel of the first beam splitter 321, a second beam splitter 322 is present. In the reflected channel of the second beam splitter 322, an emission filter 323 filters the fluorescence light to remove any excitation / white lights (which could be reflected by the field of view) and ambient light (which could be generated by intrinsic fluorescence). A fluorescence camera 324 (e.g., EMCCD type) receives the fluorescence light from the emission filter 323 and generates a corresponding (digital) fluorescence image representing the distribution of fluorophores in the field of view. In the transmitted channel of the second beam splitter 322, the light emitted or scattered by the region of interest 100 reaches the apparatus 300, where it can be processed as described above with reference to the previous embodiments.

[0091] Bibliographic References

[0092] 1. Reconstructing the image scanning microscopy dataset: an inverse problem. Zunino A et al, 2023, Inverse Problems 39, 064004. 2. Focus image scanning microscopy for sharp and gentle super-resolved microscopy. Tortarolo G et al, Nature Communications (2022), 13:7723.

Claims

CLAIMS1. Imaging apparatus, comprising a laser illumination unit (10) configured to generate a laser beam; a spatial modulation system (20) configured to control at least one of intensity, phase and polarisation of the laser beam in a spatially variable manner such that the laser beam is divided into a plurality of illumination beams; an illumination optic (30) configured to focus said illumination beams towards respective points (Pl, . . ., Pi) of a target (S; 100); a collection optic (40) configured to collect light emitted or scattered by said points (Pl, . . ., Pi) of said target (S; 100); a two-dimensional array of single-photon detectors (50) optically coupled to the collection optic (40) such that light emitted or scattered by each of said points (Pl, ..., Pi) of the target (S) is received by a respective detector of the two-dimensional array of singlephoton detectors (50), each of said detectors acting as a pinhole; and a processing and control system (60) coupled to the two-dimensional array of singlephoton detectors (50) and operable to produce digital images of said respective points (Pl, . . . , Pi) of the target (S) from a signal provided by the two-dimensional array of single-photon detectors (50), wherein the collection optic (40) comprises a magnification system (41) such that the size of each detector of the two-dimensional array of single-photon detectors (50), projected on an image plane of the magnification system (41), is less than 1 airy unit.

2. Apparatus according to claim 1, further comprising a scanning system (70) coupled to at least one of said illumination optic (30), collection optic (40) and target (S; 100), wherein said processing and control system (60) is coupled to the scanning system (70) and is operable to produce, by parallel scanning through said illumination beams, digital images of respective portions of the target (S) from a signal provided by the two-dimensional array of single-photon detectors (50).

Images