Imaging apparatus

Abstract

Imaging apparatus, comprising a laser illumination unit (10) configured to generate a laser beam; a spatial modulation system (20); an illumination optic (30); a collection optic (40); a single-photon detector array (50); a processing and control system (60) coupled to the detector array (50) and operable to produce digital images of a target (S; 100); wherein the spatial modulation system (20) is adjustable between a high-speed configuration in which the spatial modulation system (20) divides the laser beam into a plurality of illumination beams, and a high-resolution configuration in which the laser beam is undivided.

Description

[0001] Imaging apparatus

[0002] The present invention relates to optical imaging techniques.

[0003] Image Scanning Microscopy (ISM) is an advanced microscopy technique that overcomes certain limitations of confocal scanning microscopy in order to obtain high-resolution and high-contrast images. The main object is to overcome the trade-off between spatial resolution and signal-to-noise ratio. WO 2019 / 145889 Al describes a solution that uses a detector array, by means of which it is possible to double the spatial resolution. WO 2023 / 275777 Al describes a solution that uses a single-photon resolution detector, by means of which it is not only possible to double the spatial resolution, but also to perform FLIM (fluorescence lifetime image microscopy) measurements, based on fluorescence lifetime. The combination of the two inventions enables, for the first time, the acquisition of FLIM images in super-resolution. These inventions have closed the gap that existed between superresolution microscopy and spectroscopy.

[0004] However, the solution described in WO 2019 / 145889 Al does not allow the acquisition of multicolour images, except through the simple multiplication of the number of sensors. This solution is neither practical nor scalable, as it leads to an excessive increase in the cost and hardware resources required for a single measurement. WO 2023 / 275777 Al attempts to address this limitation by introducing a spectral encoder / decoder and using the temporal channel to perform colour unmixing. However, this is a solution that “sacrifices” the acquired temporal information.

[0005] More generally, all scanning techniques require a long time for the acquisition of a single image, because the image is acquired one pixel / voxel at a time. The scanning time scales with the square of the number of pixels (in 2D) or with the cube (in 3D). This could be particularly disadvantageous in the case of ex-vivo or in-vivo applications.

[0006] In some applications, especially in-vivo ones, it may be necessary (even within the same measurement) to have different resolutions for different areas of the image. For example, in tumour imaging one wishes to have resolution at the edges, in order to ensure that no area of infiltration into healthy tissue is missed, high resolution near blood vessels, while in other areas acquisition speed may be prioritised. In other applications, resolution may be prioritised, accepting possibly longer scanning times. In yet other applications, speed may be prioritised, trying to obtain the best possible image in the shortest possible time.

[0007] In some applications, multicolour imaging is desired, in others a single wavelength is used. In some cases, different wavelengths are used simultaneously, in others at successive times (per pixel, per line or per frame).

[0008] A necessary condition for performing super-resolution imaging (i.e. beyond the diffraction limit) is that the size, projected onto the image plane, of each element of the detector array is smaller than 1 airy unit. It has been shown that if 1AU is collected by at least 2x2 sensors, it is already possible to obtain a super-resolution image. However, the size of the airy disc is an arbitrary unit depending on the wavelength used, the physical magnification of the system, and the numerical aperture of the objective.

[0009] An object of the present invention is to make available a solution that allows to increase the versatility of the imaging apparatus.

[0010] With respect to this object, the invention relates to an imaging apparatus, comprising a laser illumination unit configured to generate a laser beam; a spatial modulation system; an illumination optic; a collection optic; a single-photon detector array; a processing and control system coupled to the detector array and operable to produce images of the target; characterised in that the spatial modulation system is configured to control at least one of intensity, phase and polarisation of the laser beam in a spatially variable manner, and is adjustable between a high-speed configuration in which the spatial modulation system divides the laser beam into a plurality of illumination beams, and a high-resolution configuration in which the spatial modulation system lets the laser beam be undivided; wherein in the high-speed configuration:

[0011] - the illumination optic is configured to focus said illumination beams towards respective points of a target;

[0012] - the collection optic is configured to collect the light emitted or scattered by said points of the target;

[0013] - the single-photon detectors are optically coupled to the collection optic such that the light emitted or scattered by each of said points of the target is captured by a respective detector of the single-photon detector array, each of said detectors acting as a pinhole;

[0014] - the processing and control system is operable to produce images of said respective points of the target; and wherein in the high-resolution configuration:

[0015] - the illumination optic is configured to focus said laser beam towards a point of a target;

[0016] - the collection optic is configured to collect the light emitted or scattered by said point of the target;

[0017] - the single-photon detectors are optically coupled to the collection optic such that the light emitted or scattered by said point of the target is captured by each of the detectors of the single-photon detector array, each of said detectors acting as a pinhole;

[0018] - the processing and control system is operable to produce at least one of a high- resolution image or a time-resolved image of said point of the target by processing a plurality of detection signals respectively provided by the single-photon detectors.

[0019] The apparatus described above allows the provision of time-resolved super-resolution spatial images while ensuring a relatively high frame rate, enabling the execution of time-resolved image scanning microscopy by scanning and simultaneously acquiring multiple image points or regions of interest (ROI) using a single matrix photosensor. This approach significantly improves imaging speed while maintaining high spatial and temporal resolution and the qualities associated with traditional image scanning microscopy.

[0020] Features and advantages of the proposed apparatus will be discussed in the following detailed description, which refers to the accompanying drawings, provided solely by way of non-limiting example, in which:

[0021] - figure 1 shows a block diagram of a microscope according to the invention;

[0022] - figure 2 shows a schematic representation of a detector of the microscope of figure 1 (left) and a graphical representation of the point spread function (PSF) relating to the detector according to three different configurations of the microscope (centre, top right and bottom right);

[0023] - figure 3 shows the characteristics of a single element of an example of a matrix sensor usable in the present invention: at the top left, resolution curves are shown (number of measured photons vs. number of photons incident on the sensor) for two different dead time values (no dead time and dead time equal to 100 ns); at the centre, the fill factor as a function of the opening angle of the incident beam is shown; at the top right, the detection efficiency as a function of the wavelength of the photons is shown;

[0024] - figure 4 shows an example of reconstruction of a high-resolution intensity image. At the top, from left to right: series of intensity images g obtained with a 7 by 7 SPAD detector array, showing the nucleus of a cell; fingerprint image a calculated from g; drift matrix d' and d' estimated from g; PSF hi>7calculated for each element of the SPAD array obtained from the excitation PSF hexc, the detection PSF hdet, and the shift matrices s and syestimated. At the bottom, from left to right: intensity image recorded by the central pixel gic,jC; conventional low-resolution CLSM intensity image; intensity image reconstructed by multi-image deconvolution;

[0025] - figure 5 shows an example of reconstruction of a high-resolution intensity image, in a multicolour configuration. At the top, from left to right: four series of intensity images g obtained with respective 3 by 3 subsets of a 7 by 7 SPAD detector array (each subset is struck by photons having a wavelength different from the other subsets), representing a four- colour image having at its centre the nucleus of a cell, specifically: top left, nuclear pore complex labelled with anti-nup 153 + AlexaFluor 405; top right, actin filaments labelled with Phalloidin + AlexaFluor 488; bottom left, tubulin labelled with anti-beta- tubulin + AlexaFluor 555; bottom right, Golgi labelled with Golgi complex antibody + AlexaFluor 647 ; fingerprint images a calculated from g for each subset; drift matrices d' and d' estimated from g for each subset; PSF hy calculated for each element of the SPAD array obtained from the excitation PSF hexc, the detection PSF hdet, and the shift matrices s and s' estimated for each subset. At the bottom, from left to right: intensity image recorded by the central pixel gicjc for each subset; conventional low-resolution CLSM intensity image for each subset; pixel reassignment of the image relating to one of the subsets; intensity image reconstructed by multi-image deconvolution relating to one of the subsets;

[0026] - figure 6 shows an apparatus according to the present invention used for improved optical sectioning;

[0027] - figures 7a and 7b respectively show an apparatus according to the invention configured for surgical microscopy, and an apparatus according to the invention configured for endoscopy;

[0028] - figure 8 is a diagram showing different configurations of the apparatus according to the invention.

[0029] With reference to figure 1, a laser scanning microscope apparatus is now described. In summary, the apparatus comprises a laser illumination unit 10 configured to generate a laser beam; a spatial modulation system 20 configured to control at least one of intensity, phase and polarisation of the laser beam in a spatially variable manner; an illumination optic 30; a collection optic 40; a single-photon detector array 50 optically coupled to the collection optic 40, each of said detectors acting as a pinhole; and a processing and control system 60 coupled to the detector array.

[0030] In the illustrated example, the illumination unit 10 comprises a tunable laser source 11 at visible wavelengths and one or more fixed-wavelength laser sources in the visible range 12, 13, 14. The laser sources may be continuous wave or pulsed, possibly amplitude modulated. In the case of pulsed lasers, they may be capable of receiving an external trigger, or of providing a trigger themselves. Alternatively, a so-called “supercontinuum” laser source may be used, capable of generating photons in a very broad bandwidth spectrum. Associated with each of the sources is a mirror I la or a dichroic filter 12a, 13a, 14a that directs the respective beams to a modulator 15 such as an acousto-optic modulator (AOM) or a tunable acousto-optic filter (AOTF), optionally present to select specific wavelengths. The illumination unit 10 may be coupled to the downstream optical part via optical fibre, waveguide or directly “in air”.

[0031] The illumination optic 30 comprises a dichroic filter 31 and an objective lens system 32 through which the laser beam or laser beams is / are focused onto an object to be analysed S. The reference numeral 70 denotes a system for performing the scanning of the object to be analysed, made for example as a galvanometric mirror and / or with a movement mechanism that moves the sample holder 71 which supports the object S. For example, in the illustrated example, scanning along the x and y axes (parallel to the sample plane) is performed by controllably deflecting the illumination beam / beams, while scanning along the z axis (orthogonal to the sample plane) is performed by controllably moving the sample holder 71. In some embodiments, the scanning system may be absent.

[0032] The fluorescence signal emitted by the object is collected by the same objective lens system 32 and transmitted through the dichroic filter 31 and a magnification system (zoom lens) 41 to the single-photon detector array 50. The magnification system is capable of arbitrarily defining the size of the detector on the image plane (to comply with the physical condition of being sub-airy). This allows the system to be compatible with a wide range of experimental conditions: different objectives (magnification / numerical aperture), different wavelengths.

[0033] The detector array 50 is represented by a (two-dimensional) matrix of M.txMvdetector elements, each of which is independent (fully parallel system), has single-photon sensitivity and temporal resolution (low timing jitter) such as to allow measurement of the fluorescence lifetime of the most common fluorophores used in fluorescence microscopy. The operating parameters of the detector array (excess bias voltage, hold-off time and the number of active elements) can be adjusted before each measurement (alternatively, the system may be configured to autonomously determine the optimal conditions for the given measurement). The detectors of the array 50 therefore provide photon-by-photon time-resolved information.

[0034] Each element of the detector array 23 has a circular active area (other shapes may be used, such as for example a square or hexagonal shape), surrounded by an inactive frame (figure 2, left). Analogously to other pixel-based devices, the pixel pitch can be defined as the distance between (the centroids of) two adjacent pixels (if the pixel is square, such pixels lie on the same row or the same column). An important feature of the detector is the fill factor (see fig. 3), which can be calculated as the ratio between the active area and the square of the pixel pitch. The overall photon detection efficiency (PDE) of the detector, i.e. the probability that a photon reaching the detector is recorded, has a direct dependence on the fill factor. To further improve the PDE, a microlens array 42 is used to direct photons towards the centre of each detector element. In figure 2, the active area of the individual detector elements is shown as circles drawn with dashed lines, while the microlenses are shown as circles drawn with solid lines. Each element of the detector array signals the arrival of a photon with a TTL logic signal (transistor-transistor logic) on a dedicated digital channel. Specifically, the detector generates an LVDS (differential) signal, and the system converts it to TTL before supplying it to the FPGA board. Some FPGAs read the LVDS signal directly, so conversion would not be necessary. Nothing would change with other formats, such as NIM. What is exploited is the fact that the output from the detector is a digital signal indicating the arrival of a photon: thus, photon-resolved and time-resolved. Three additional digital lines constitute a communication bus useful for the initial configuration of the entire array during the start-up of the measurement. The temporal information can be collected by following two approaches: time domain and frequency domain. Time domain approaches can be implemented on FPGA or by means of ASIC components. Both of the above approaches are included in the present invention.

[0035] The processing and control system 60 may be developed with a field-programmable gate- array-processor (FPGA). This allows to integrate all photons collected during the dwell time on the single point n (intensity mode) or to measure the arrival times with respect to an external reference signal (e.g. the excitation laser pulse) thanks to the onboard integration of time-to-digital converters (TDCs) (time-resolved or TCSPC mode).

[0036] It is important to emphasise that the detector and data acquisition electronics are configured to operate in a fully asynchronous mode, i.e. when a photon is detected, it is counted or its arrival time is measured, and the elements are independent of one another, without a limited frame rate or the drawbacks of sequential reading. Communication (synchronisation) with the microscope control system (indicated as 18 in figure 1) is carried out via digital lines for pixel / line / image clocking, provided by the manufacturer.

[0037] The spatial modulation system 20 is associated with the illumination optic 30 and possibly also with the collection optic 40 and is configured to control at least one of intensity, phase and polarisation of the laser beam in a spatially variable manner, so as to engineer the illumination beam or beams incident on the sample, and to engineer the light emitted or scattered by the sample and collected by the collection optic 40. In the illustrated example, the spatial modulation system 20 comprises a spatial light modulator (SLM) that may be implemented with one of the technologies known in the field, for example liquid crystalbased (LC-SLM), or a digital micromirror device-based micro-electromechanical system (DMD-SLM). A possible implementation of the spatial modulation system 20 is shown in the lower right box in figure 1, wherein 21 denotes the SLM. An incoming light beam IB is transmitted to the SLM 21 through an optical system, and here manipulated to obtain the required outgoing light beam(s) OB. The term “patterns” denotes a control signal by which a processing and control system 60 governs the spatial modulation system 20.

[0038] Specifically, the spatial modulation system is adjustable between a high-speed configuration (shown at the far right in figure 8) in which the spatial modulation system 20 divides the single laser beam from the illumination unit 10 into a plurality of illumination beams, and a high-resolution configuration (shown at the far left in figure 8) in which the single laser beam from the illumination unit 10 is undivided. Similarly, in the high-speed configuration the spatial modulation system 20 divides the emitted or scattered light from the object S into a plurality of signal beams, and in the high-resolution configuration the emitted or scattered light from the object S is collected as a single beam. In the high-speed configuration, the maximum speed of image reconstruction of the object S can be achieved, while in the high- resolution configuration, the maximum image reconstruction resolution of the object S can be achieved.

[0039] In the high-speed configuration, there is one illumination beam for each sensitive element of the detector array 50, or for each sensitive element of a selection of detectors of said array. Therefore, in the case where said array is composed of 7 x 7 elements, up to 49 illumination beams may be provided. If the 49 beams can be properly focused and a perfect match with the scanning conditions is achieved, a 49-beam parallel confocal system would result, with confocal resolution / optical sectioning and a speed 50 times higher. Should these conditions not be met in the current scan, very fast imaging would still be obtained and always time- resolved (FLIM). The illumination optic 30 is configured to focus the illumination beams towards respective points Pl, ..., Pi of the object S. The collection optic 40 is configured to collect the light emitted or scattered by said points Pl, ..., Pi of the target S. The singlephoton detectors 50 are optically coupled to the collection optic 40 such that the light emitted or scattered by each of said points Pl, ..., Pi of the target is captured by a respective detector of the single-photon detector array. Each of the detectors acts as a pinhole.

[0040] The processing and control system 60 is thus capable of producing images of the respective points Pl, ..., Pi of the target and, in particular, by parallel scanning through the illumination beams, images of respective portions of the object S.

[0041] In the high-resolution configuration, the illumination optic 30 is configured to focus the single laser beam towards a point P of the object. The collection optic 40 is configured to collect the light emitted or scattered by point P of the object. The single-photon detectors are optically coupled to the collection optic 40 such that the light emitted or scattered by point P of the object is captured by each of the detectors of the single-photon detector array. Each of the detectors acts as a pinhole. The processing and control system 60 is capable of producing at least one of a high-resolution image or a time-resolved image of said point of the target by processing a plurality of detection signals respectively provided by the singlephoton detectors. In particular, by scanning with the above-mentioned laser beam, at least one of a high-resolution image or a time-resolved image of the object S is produced.

[0042] It is basically possible to sample at Nyquist instead of Nyquist / 2, gaining a factor of 4 in terms of speed [1]. This without losing super-resolution, and with a single beam.

[0043] The process by which the high-resolution image or the time-resolved image of the object S is produced is described in WO 2019 / 145889 Al, but for convenience its description is repeated below. In this regard, fig. 4 shows the various steps of the image reconstruction.

[0044] Step 1. Record the time-resolved (or TCSPC) image series / matrix g' with the abovedescribed architecture.

[0045] For different positions of the focused excitation laser beam (single-photon or multi-photon) on the sample, the signals generated by the elements of the detector array 23 are read, and the time-resolved image series / matrix gfis obtained: gf= gL(n< 0 con

[0046] Essentially, each image g-;(n, t) is a three-dimensional matrix wherein the time axis t represents the histogram of the photon arrival time obtained by the TCSPC measurement, i.e. the number of photons collected in a spatial pixel (nx,ny) and in a certain temporal window (temporal bin) t from the excitation event.

[0047] Step 2. Calculate the intensity image series g from g1

[0048] Given the TCSPC image matrix g' collected with the above-described apparatus in TCSPC mode, the matrix is integrated along the temporal dimension to obtain the intensity image matrix g: gij(n) = ^ gi,7(a t) t

[0049] Step 3. Calculate the “fingerprint” image a

[0050] Given the intensity image matrix g, the so-called “fingerprint” image a is calculated, from which the excitation PSF hexcand the detection PSF hf / r / are estimated.

[0051] The fingerprint image a is defined as follows. All photons collected by each detector element during an entire measurement are integrated, producing the fingerprint image a. In practice, during a single experiment MtxMvimages are obtained, and the fingerprint image a is produced by summing all the intensity values image by image:

[0052] To understand the properties of the fingerprint image a and how to obtain the PSFs from it, it is important to derive a in the continuous domain.

[0053] Considering a detector array composed of infinitesimal elements, it is observed that the image gx>y< acquired by an element at position (x',y') E R2can be expressed as wherein f is the object / sample function, hx> y denotes the PSF associated with the detector element at position (x',y'), and * denotes the convolution operator; the fingerprint image a(x',y'), defined with respect to the detector coordinates, is

[0054] Applying the convolution integration property, the fingerprint image is wherein is the total photon flux from the sample. It should be noted that a(x',y') is independent of the sample under the condition > 0 ( » 0), but it is closely connected to the PSF of the microscope system.

[0055] Recalling that the PSF of each infinitesimal element is

[0056] = hexc(x, y) ■ hdet(x — x' ,y — y') and substituting into the previous equation, it is possible to obtain ^x,y

[0057] Wherein * denotes the correlation operator. In summary, the fingerprint image is instrumentdependent and sample-independent. Furthermore, it depends on both the excitation PSF and the detection PSF.

[0058] It should be noted that the fingerprint image can also be used to align the system. In particular, to coalign the excitation PSF and the detection PSF on the central pixel ic,jc of the detector array. This procedure is very important for a detector with a small number of elements. A misalignment results in a loss of fluorescence photons. If the system is properly aligned, the central pixel is the brightest, and the intensity values of the pixels are distributed symmetrically and isotropically with respect to the centre. It is thus possible to implement a feedback control system that measures the fingerprint image and accordingly adjusts the xy position of the detector to maximise the intensity of the central pixel.

[0059] Step 4. Estimate the shift matrices sxand sP', the drift matrices dxand d\ and the mcroscope magnification Magn

[0060] Given the intensity image matrix g, the shift matrices sxand s’" are calculated.

[0061] As described for the pixel reassignment method, each image gij is shifted (in the image plane) with respect to gicjcby half the distance between the element(i, j) and the element(ic,jc), that is d f j = Therefore, the shift matrices sxe sycan be estimated by recovering the displacement between the different images gij. A phase correlation method is used for its resilience to noise and higher speed compared to spatial domain algorithms. The phase correlation estimates the displacement between two similar images based on a frequency domain representation of the data, which in the present description is obtained by fast Fourier transforms (FFT).

[0062] To calculate the phase correlation between the two different images of the sample (gij and gicjc), the so-called correlogram ty-is first defined: and then the maximum of the correlogram is found, whose position denotes the drift between the two images:

[0063] The position of the maximum is obtained using a fitting algorithm or a centroid-based algorithm to obtain sub-pixel values, wherein

[0064] Given the drift matrices d' and d', the shift matrices sxand s' can be calculated as follows

[0065] Other approaches that estimate the shift matrices use: (i) a theoretical model based on the physical distance between detector elements and the system magnification, (ii) a calibration sample, e.g. spheres.

[0066] However, such approaches do not account for the peculiarities of each sample and the specific measurement conditions. Moreover, the sample magnification is not always easy to estimate. On the other hand, the above-described approach is sensitive to the Gaussian shape assumption for the excitation PSF and the detection PSF. Nevertheless, for (i,j) within the first Airy disc centred at (ic,jc) the assumption is solid and the estimate of is robust. An optimal approach could integrate into the estimate of the maximum of the correlogram some constraints based on knowledge related to the geometric shape of the detector and the system magnification.

[0067] The magnification Magn of the system can be determined using the values of estimated for the first-order neighbours (AQ of element (ic,jc), that is (ic + l,jc), (ic — 1, jc), (ic,jc + 1) and (ic,jc — 1), Together with the detector pixel pitch (PP) and the pixel size of the image (DP):

[0068] Magn

[0069] Step 5. Calculate the time-resolved object function ff.

[0070] Given the matrix of time -resolved images gf, the fingerprint image a and the shift matrices s' and s' an estimate of the function ffis calculated as described below (steps 5.1-5.3).

[0071] Step 5.1. Estimate the excitation PSF hexcand the detection PSF hdet

[0072] Based on the relationship with the fingerprint image a described above, the excitation PSF hexcand the detection PSF hdetcan be estimated according to the minimisation problem or

[0073] (<pexc, (pdet) = argminlf)exclf)detJMSE(hexc, hdetla) in the case of a parametrisation of the PSFs. Here the MSE functional is

[0074] Minimisation of the MSE function may be performed with numerical techniques according to known practices.

[0075] Step 5.2. Calculate the time-resolved object function via multi-image deconvolution (MID).

[0076] Since all information concerning the PSFs (including shift values) has already been estimated, the original problem can be solved using conventional multi-image deconvolution, in particular by minimising the Kullback-Leibler (KL) divergence or the mean squared error (MSE):

[0077] In the time -resolved case, the KL distance is: and the MSE distance is:

[0078] Minimisation of the MSE or KL functional can be performed using numerical techniques according to known practices.

[0079] Step 5.3. Calculate the time-resolved object function f* via pixel reassignment (PR).

[0080] Following the pixel reassignment approach, a high-resolution time-resolved image can be obtained simply by summing all images after shifting back each image g ■ j estimated amount

[0081] Essentially, each 2D image associated with each temporal bin and each detector element is shifted independently. For this reason, both the FFT and inverse FFT are performed in 2D.

[0082] Step 6. Calculate the intensity object function f.

[0083] Given: (i) the intensity image matrix g, the fingerprint image a, and the shift matrices sx, syo (ii) the time-resolved object function ffpreviously estimated, an estimate of the intensity object function f is calculated as described below (steps 6.1-6.3).

[0084] Step 6.1. Calculate the intensity object function f by temporal integration.

[0085] Given the high-resolution time-resolved image, a high-resolution intensity image can be obtained by integrating the temporal dimension of the reconstructed time-resolved object function f

[0086] Step 6.2. Calculate the intensity object function f via multi-image deconvolution.

[0087] Given the excitation PSF hexc. the detection PSF hdetand the shift matrices sx, sythe intensity object function f can be calculated directly from the intensity image series g via multi-image deconvolution and without estimating the time-resolved object function ft, with a substantial reduction in computational effort.

[0088] In this case, it is necessary to minimise the KL or MSE distance only with respect to f:

[0089] The minimisation of the MSE or KL functional can be performed using numerical techniques according to known practices.

[0090] Step 6.3. Calculate the intensity object function f by pixel reassignment.

[0091] Given the shift matrices sx, sy, he intensity object function f can be calculated directly from the intensity image series g by pixel reassignment and without estimating the time-resolved object function ff, with a substantial reduction in computational effort.

[0092] In this case, the pixel reassignment estimate is: idy(ij)ny)^

[0093] If the microscope apparatus operates in intensity mode (i.e. without performing TCSPC measurements), only the intensity image matrix g is generated. In this case, only the methods proposed in steps 6.2 and 6.3 can be used. For a laser beam operating in continuous wave mode, the importance of recording the signal in TCSPC mode is reduced.

[0094] Preferably, the spatial modulation system is also adjustable in at least one intermediate configuration between the high-speed configuration and the high-resolution configuration (two different intermediate configurations are shown at the centre of figure 8). In such intermediate configuration(s), the spatial modulation system divides the laser beam into a plurality of illumination beams (in a number of beams still lower than that of the high-speed configuration), but it is possible to reconstruct high-resolution or time-resolved images according to the procedure described above. The upper right box in figure 2 shows the PSF obtainable in the case of four monochromatic beams in parallel, each of which affects a subset of nine detector elements of the array 50.

[0095] In the intermediate configuration, the illumination optic 30 is configured to focus the illumination beams towards respective points Pi,.. Pj of the object. The collection optic 40 is configured to collect the light emitted or scattered by said points Pi,..., Pj of the object. The single-photon detectors are optically coupled to the collection optic 40 such that the light emitted or scattered by each of the points Pi,..., Pj of the object is captured by each of the detectors of a respective subset of detectors of the single-photon detector array 50. Each of the detectors acts as a pinhole. The processing and control system 60 is capable of producing at least one of a high-resolution image or a time-resolved image of each of said points of the target by processing a plurality of detection signals respectively provided by the detectors of the respective subset of detectors of the single-photon detector array. By parallel scanning by means of the aforementioned illumination beams, at least one of a high- resolution image or a time-resolved image of respective portions of the target is produced. For each portion of the target, the above described reconstruction procedure is applied.

[0096] It is also possible to implement a further type of intermediate configuration in which the illumination unit 10 and the spatial modulation system 20 are operated in such a way as to generate multiple illumination beams having different wavelengths, which are focused on a point P of the object. In fact, different areas of the sensor can be used for imaging with different colours: not all the detectors of the array 50 are needed to achieve super-resolution; for example, 3x3 sub-areas are sufficient if the 1AU condition is met. Thus, the beams are directed to different zones of the sensor using the system’s aiming optics.

[0097] However, even in the case of sequential monochromatic / multicolour imaging, a problem is that different wavelengths may follow slightly different paths. The above-described system allows correction of this chromatic aberration by calculating the centre of mass of the fingerprint and registering everything on it (as previously described and in WO 2019 / 145889 Al).

[0098] In the multicolour configuration, the collection optic 40 is configured to collect the light emitted or scattered by point P of the object under analysis. The single-photon detectors are optically coupled to the collection optic 40 such that the light emitted or scattered by point P of the object and having a given wavelength is captured by each of the detectors of a respective subset of detectors of the single-photon detector array 50. Each of the detectors acts as a pinhole. The lower right box of figure 2 shows the PSF obtainable in the case of four beams having different wavelengths in parallel, each of which affects a subset of nine detector elements of the array 50. The processing and control system 60 is capable of producing at least one of a high-resolution image or a time-resolved image of the point of the target, associated with the given wavelength, by processing a plurality of detection signals respectively provided by the detectors of the respective subset of detectors of the single-photon detector array. By parallel scanning through the aforementioned illumination beams, at least one of a high-resolution image or a time-resolved image of respective portions of the target is produced, for each of the wavelengths used. For each wavelength, the reconstruction procedure described above is applied. In this regard, figure 5 shows the various steps of image reconstruction in a manner analogous to that used for the single beam in figure 4.

[0099] It is also possible to implement a multicolour high-speed configuration in which the illumination unit 10 and the spatial modulation system 20 are operated in such a way as to generate multiple illumination beams having different wavelengths, which are focused on a point P of the object. In this case, the collection optic 40 is configured to collect the light emitted or scattered by point P of the object under analysis. The single-photon detectors are optically coupled to the collection optic 40 such that the light emitted or scattered by point P of the object and having a given wavelength is captured by a respective detector of the single-photon detector array 50. Each of the detectors acts as a pinhole. The processing and control system 60 is operable to produce digital images of said point P of the target, each of said digital images being associated with a respective wavelength. In the case where the scanning system 70 is associated with at least one of said illumination optic 30, collection optic 40 and target S, the processing and control system 60 is coupled to the scanning system 70 and is operable to produce, by parallel scanning through said illumination beams, digital images of respective portions of the target, each of said digital images being associated with a respective wavelength. With reference to figure 6, it is observed that the above-described apparatus allows the improvement of optical sectioning, by means of an algorithm capable of discriminating out- of-focus light from in-focus light, such as that described in [2]. The idea behind this concept is to observe the distribution of light on the detector array to distinguish the axial position of the emitters. The central element of the detector array mainly contains the in-focus signal, while out-of-focus light dominates the outer elements. The same information can be extracted by calculating axial fingerprints, i.e. by integrating, at different depths z, the 3D PSFs scanned over the scanning coordinates (x, y). When the emitter is in focus (left image in figure 6), the central pixels contain most of the signal. The more one moves away from the focal plane, the more the outer pixels of the fingerprint image become populated with photons (right image in figure 6). This tendency can be quantified by calculating the ratio between the intensity of the outer pixels and the intensity collected by the central pixels.

[0100] Another possible application of the above-described apparatus concerns single particle tracking. Currently, SPT is performed using confocal systems with a single detector. Therefore, scanning is necessary, for example an orbit ("orbital scanning"), to determine where the particle (starting at the centre of the orbit) moves and to follow it. With the abovedescribed architecture, orbital scanning is not necessary; it is sufficient to move towards the direction of the centroid to perform the scanning. Two levels of computation: rough, to perform real-time tracking and always keep the particle centred, to be implemented on FPGA or another fast architecture, and then more refined, a posteriori, on the raw data collected.

[0101] Figures 7a and 7b show two further possible implementations of the invention. Figure 7a shows an implementation for surgical microscopy. Reference numeral 100 denotes a target or region of interest for a medical procedure on a patient, for example a surgical cavity. Reference numeral 200 denotes a microscopy apparatus having substantially the same elements shown in figure 1. The scanning system, indicated here with 70’, is configured to operate along the x and y directions parallel to the focal plane, whereas no scanning is provided along the z axis. Reference numeral 32 denotes an objective lens system by means of which the laser beam or beams from the apparatus 200 is / are focused on the target 100 by means of an optical fibre bundle 33 and a distal lens system 34 optically coupled to the objective lens system 32. Conversely, the signal emitted or scattered by the region of interest 100 is transmitted to the apparatus 200 through the distal lens system 34, the optical fibre bundle 33 and the objective lens system 32. As for functionalities, they are substantially the same as those of the architecture shown in figure 1, except for the absence of a movable sample holder along the z axis.

[0102] Figure 7b shows an implementation for an endoscopic probe. Reference numeral 100 denotes a target or region of interest for a medical procedure on a patient, for example a surgical cavity. Reference numeral 300 denotes an apparatus having substantially the same elements shown in figure 1, but provided with a special objective system + optical fibre bundle + distal lens (for example GRIN lens) which allows the use of an optical fibre bundle and decouples the apparatus from the probe. Reference numeral 310 denotes a wide-field light source. A coupling optic 301 and a distribution optic 311 respectively provide a laser illumination beam and a wide-field illumination beam to the region of interest 100. The light emitted or scattered by the region of interest 100 is collected by the coupling optic 301. The collected light includes the fluorescence light emitted by any possible fluorophores present in the field of view (illuminated by the excitation light). In addition, the collected light includes reflected light (in the visible spectrum) which is reflected by any possible objects present in the field of view (illuminated by the white light). A first beam splitter 321, for example a dichroic filter, divides the collected light into two channels. In the reflected channel of the first beam splitter 321, a photographic camera 325 (for example, of the CCD type) receives the reflectance light and generates a corresponding (digital) reflectance image representing what is visible in the field of view. In the transmitted channel of the first beam splitter 321, a second beam splitter 322 is present. In the reflected channel of the second beam splitter 322, an emission filter 323 filters the fluorescence light to remove any excitation / white light (that may be reflected from the field of view) and ambient light (that may be generated by autofluorescence). A fluorescence camera 324 (for example, of the EMCCD type) receives the fluorescence light from the emission filter 323 and generates a corresponding (digital) fluorescence image representing the distribution of the fluorophores in the field of view. In the transmitted channel of the second beam splitter 322, the light emitted or scattered by the region of interest 100 reaches the apparatus 300, where it can be processed in the ways described above with reference to the previous embodiments. Bibliographic references

[0103] 1. Reconstructing the image scanning microscopy dataset: an inverse problem. Zunino A et al, 2023, Inverse Problems 39 064004. 2. Focus image scanning microscopy for sharp and gentle super-resolved microscopy. Tortarolo G et al, Nature Communications (2022), 13: 7723.

Claims

CLAIMS1. Imaging apparatus, comprising a laser illumination unit (10) configured to generate a laser beam; a spatial modulation system (20); an illumination optic (30); a collection optic (40); an array of single-photon detectors (50); a processing and control system (60) coupled to the detector array (50) and operable to produce digital images of a target (S; 100); characterised in that the spatial modulation system (20) is configured to control at least one of intensity, phase and polarisation of the laser beam in a spatially variable manner, and is adjustable between a high-speed configuration in which the spatial modulation system (20) divides the laser beam in-to a plurality of illumination beams, and a high-resolution configuration in which the spatial modulation system (20) lets the laser beam be undivided; wherein in the high-speed configuration:- the illumination optic (30) is configured to focus said illumination beams towards respective points (Pl, ..., Pi) of the target (S);- the collection optic (40) is configured to collect the light emitted or scattered by said points (Pi,.. Pj) of the target (S);- the single-photon detectors (50) are optically coupled to the collection optic (40) such that the light emitted or scattered by each of said points (Pi,.. Pj) of the target (S) is received by a respective detector of the single-photon detector array (50), each of said detectors acting as a pinhole;- the processing and control system (60) is operable to produce digital images of said respective points (Pi,..., Pj) of the target (S); and wherein in the high-resolution configuration:- the illumination optic (30) is configured to focus said laser beam towards a point (P) of the target (S);- the collection optic (40) is configured to collect the light emitted or scattered by said point (P) of the target (S);- the single-photon detectors (50) are optical-ly coupled to the collection optic (40) such that light emitted or scattered by said point (P) of said target (S) is received by each of the detectors of the array of single-photon detectors (50), each of said detectors acting as a pinhole;- the processing and control system (60) is operable to produce at least one of a high resolution image and a time-resolved image of said point (P) of the target (S) by processing a plurality of detection signals respectively provided by the single-photon detectors (50).

2. Apparatus according to claim 1, further comprising a scanning system (70; 70’) associated with at least one of said illumination optic (30), col-lection optic (40) and target (S; 100), wherein said processing and control system (60) is coupled to said scanning system (70; 70’), in the high-speed configuration, said pro-cessing and control system (60) being operable to produce, by parallel scanning by said illumination beams, digital images of respective portions of the target (S; 100); and in the high-resolution configuration, said processing and control system (60) being operable to produce, by scanning with said laser beam, at least one of a high resolution image and a time-resolved image of said target.

3. Apparatus according to claim 1 or 2, wherein the spatial modulation system (20) is further adjustable in at least one configuration intermediate between the high-speed configuration and the high-resolution configuration, in which the spatial modulation system (20) divides the laser beam into a plurality of illumination beams; wherein in said at least one intermediate configuration:- the illumination optic (30) is configured to focus said illumination beams towards respective points (Pi,..., Pj) of the target (S);- the collection optic (40) is configured to collect the light emitted or scattered by said points (Pi,..., Pj) of the target (S);- the single-photon detectors (50) are optically coupled to the collection optic (40) such that the light emitted or scattered by each of said points (Pi,..., Pj) of the target (S) is received by each of the detectors of a respective subset of detectors of the array of single-photon detectors (50), each of said detectors acting as a pinhole;- the processing and control system (60) is operable to produce at least one of a high- resolution image and a time-resolved image of each of said points (Pi,..., Pj) of the target (S) by processing a plurality of detection signals respectively provided by the detectors of the respective subset of detectors of the single -photon detector array (50).

4. Apparatus according to claim 3, wherein in the at least one intermediate configuration said processing and control system (60) is operable to produce, by parallel scanning by means of said illumination beams, at least one of a high-resolution image and a time-resolved image of respective portions of the target.

5. Apparatus according to claim 1 or 2, wherein the spatial modulation system (20) is further adjustable in at least one configuration intermediate between the high-speed configuration and the high-resolution configuration, in which the spatial modulation system (20) divides the laser beam into a plurality of illumination beams having different wavelengths; wherein in said at least one intermediate configuration:- the illumination optic (30) is configured to focus said illumination beams towards a point (P) of the target (S);- the collection optic (40) is configured to collect the light emitted or scattered by said point (P) of the target (S);- the single-photon detectors (50) are optically coupled to the collection optic (40) such that light emitted or scattered by said point (P) of the target (S) and having a given wavelength is received by each of the detectors of a respective subset of detectors of the array of single photon detectors (50), each of said detectors acting as a pinhole;- the processing and control system (60) is operable to produce at least one of a high- resolution image and a time-resolved image of said point (P) of said target (S), associated with said wavelength, by processing a plurality of detection signals respectively provided by the detectors of the respective subset of detectors of the single-photon detector array (50).

6. Apparatus according to claim 5, wherein in the at least one intermediate configuration said processing and control system (60) is operable to produce, by parallel scanning by means of said illumination beams, at least one of a high-resolution image and a time-resolvedimage of the target (S), for each of the given wavelengths.

7. Apparatus according to any of the preceding claims, wherein the collection optic (40) comprises a magnification system (41) such that the size of each detector of the array of single-photon detectors (50), projected on an image plane of the magnification system (41), is less than 1 airy unit.

Images