Image processing system with photon relocation

Abstract

The invention relates to a photon rearrangement imaging system, comprising: a polychromatic light source (SL) for generating an illumination light beam (FE); a first optical system (SO1) for dispersing said illumination light beam with a first angular dispersion DA1 and focusing it on a specimen (E) to form a first spectral line (LS1); and a first optical system (SO1) for collecting a light beam (FS) elastically scattered by said specimen, called signal beam, applying to it a second angular dispersion DA2 and focusing it on a focal plane to form a second spectral line (LS1) with a length longer than the first spectral line. - a second optical system (SO2) configured to form a first spectral line (LS2) at the image focal plane; - a matrix image sensor (CMI) arranged at the image focal plane; - means (S1) for inducing a relative displacement between the first spectral line and the specimen in a third direction perpendicular to the first direction; - means for forming an image of the specimen through photon relocation based on one or more images acquired by the matrix image sensor corresponding to a plurality of different positions taken by the first spectral line on the specimen.

Description

[Technical Field]

[0001] The present invention relates to the technical field of optical microscopy and more generally to the technical field of image processing. [Background technology]

[0002] Optical microscopes play an important role in biology, microtechnology, and nanotechnology because they allow rapid observation of specimens, unlike, for example, electron microscopes, which require complex preparation procedures. However, their resolution is inevitably limited by the diffraction of light. According to Abbe's theory, the maximum resolution d of a conventional optical microscope is given by the following equation:

[0003]

number

[0004] In the formula, λ is the wavelength of the light used (380 nm to 780 nm for visible light), and ON is the numerical aperture, which rarely exceeds 1.4 for biological specimens. As a result, the resolution of visible light cannot exceed 135 nm, making it unsuitable for observing very small structures such as virus particles. While shortening the wavelength can improve spatial resolution, it comes at the expense of considerable technical difficulties.

[0005] So-called "super-resolution" techniques make it possible to exceed the Abbe diffraction limit by using fluorescent markers and / or nonlinear effects. These techniques are complex to implement and are not suitable for all applications.

[0006] In addition, in biology as well as in micro- and nanotechnology, it is often necessary to achieve high spatial resolution in the axial direction (the Abbe limit relates to the lateral resolution in a plane perpendicular to the optical axis). Confocal microscopy allows the acquisition of images with a very shallow depth of field (on the order of a few hundred nanometers), thus "slicing" the specimen and allowing access to its three-dimensional structure. This technique most often involves the use of fluorescent markers, but it can also be used for reflectance without markers.

[0007] Confocal microscopes use a point source of illumination, the image of which is projected by an objective onto the specimen being observed. Light coming from the specimen (backscattered light in the case of a reflectance confocal microscope, or fluorescent light when a fluorescent marker is used) is focused onto a pinhole optically conjugate to the point source and detected, for example, by a photomultiplier tube. The function of the pinhole is to filter out radiation that does not come from the focal plane of the objective, thereby achieving optical sectioning. A point-by-point image of the specimen is obtained by scanning. More precisely, a two-dimensional scan in both directions perpendicular to the optical axis provides a slice of the specimen centered around the focal plane of the objective. An additional axial scan of this focal plane generates a three-dimensional image.

[0008] Smaller pinhole diameters result in thinner slices and therefore better axial resolution, but provide less gain below 1 Airy unit (AU). An Airy unit is the diameter of the microscope's Airy disk,

number

[0009] Confocal microscopes allow a theoretical gain in lateral resolution of up to 30% relative to the Abbe limit by using pinholes with diameters less than 1 AU, however this comes at the cost of a reduced signal-to-noise ratio.

[0010] The photon relocation technique was first proposed in (Non-Patent Document 1) and, in principle, is capable of doubling the lateral resolution of a confocal microscope. The idea behind this technique is to replace the photomultiplier tube, or more generally, the point-like radiation detector, with a matrix detector that acquires an elementary image for each acquisition point. The image is then resized (ideally reduced by a factor of two) by digital or optical means before proceeding to the next scanning point. The final image is obtained by integrating the different scan images acquired successively from a shift of one scan step relative to the previous image.

[0011] Purely optical implementations of the photon relocation technique are described in (Non-Patent Document 2), (Non-Patent Document 3), and (Non-Patent Document 4). It consists of applying a first angular scan to an illumination beam, applying an inverse angular scan to the beam coming from the specimen, and then applying a second angular scan to this same beam coming from the specimen, synchronized with the first scan. The second angular scan has a larger amplitude normalized to the beam cross-sectional area than the first scan, ideally twice that of the first scan. "Normalized to the beam cross-sectional area" means that the ratio of the amplitude of the second angular scan to the amplitude of the first angular scan is α, and the ratio of the cross-sectional area of ​​the beam coming from the specimen to the cross-sectional area of ​​the light beam is the quantity α / M, which should be greater than 1 and ideally equal to 2.

[0012] Acquisition can be parallelized using pinhole arrays and microlens arrays, similar to conventional confocal microscopes (see, for example, the aforementioned paper (Non-Patent Document 2)).

[0013] Photon remapping has been applied in particular to confocal fluorescence microscopy (see, for example, the aforementioned papers (Non-Patent Document 2), (Non-Patent Document 3), and (Non-Patent Document 4); in these papers, a 1.5-fold gain in lateral resolution was observed). To the inventors' knowledge, the application of photon remapping to reflection has only been described in (Non-Patent Document 5). However, this paper does not refer to a microscope, but rather to an ophthalmoscope in which the objective lens is replaced by the lens of the patient's eye, thus presenting a small numerical aperture and resulting resolution of about tens of micrometers.

[0014] US Pat. No. 5,629,499 and (Non-Patent Document 6) describe confocal microscope apparatus and methods in reflection or transmission using photon relocation with spatial resolution optimized by appropriate pinhole sizing.

[0015] Confocal microscopy with photon relocation is a point-by-point scanning imaging technique, which results in long acquisition times that are detrimental and sometimes prohibitive for certain applications, such as non-destructive in-situ characterization for rapid diagnostics in the biomedical, materials, microtechnology, and nanotechnology industries. [Prior art documents] [Patent documents]

[0016] [Patent Document 1] European Patent Application Publication No. 4012476 [Non-patent literature]

[0017] [Non-Patent Document 1] CJR Sheppard, "Super-resolution in Confocal Imaging," Optik 80, No. 2, pp. 53-54, 1988 [Non-patent document 2] AG York et al., "Instant super-resolution imaging in live cells and embryos via analog image processing," Nature Methods, November 2013, 10(11), pp. 1122-1126 [Non-patent document 3] G.M. R. De Luca, "Rescan confocal microscopy: scanning twice for better resolution," Biomedical Optics Express, Vol. 4, No. 11, November 2013 [Non-patent document 4] A. Curd et al., "Construction of an instant structured illumination microscope," Methods 88, pp. 37-47, 2015. [Non-patent document 5] T.B. DuBose et al., "Super-resolution retinal imaging using optically reassigned scanning laser ophthalmology," Nature Photonics, Vol. 13, pp. 257-262, April 2019. [Non-patent document 6] Aguilar, A., Boyreau, A., and Bon, P., "Label-free super-resolution imaging below 90-nm using photon-reassignment," 2021, Open Research Europe, 1(3), 3 [Non-Patent Document 7] D.R. Sandison et al., "Quantitative Comparison of Background Rejection, Signal-to-Noise Ratio, and Resolution in Confocal and Full-Field Laser Scanning Microscopes," July 1, 1995, Applied Optics, Vol. 34, No. 19, pp. 3576-3588 [Non-patent document 8] GJ Tearney et al., "Spectrally Encoded Confocal Microscopy," Optics Letters, Vol. 23, No. 15, August 1998. [Non-Patent Document 9] Hwang J. et al., "Frequency and spectrally-encoded confocal microscopy," March 9, 2015, Optics Express, Vol. 23, No. 5 Summary of the Invention [Problem to be solved by the invention]

[0018] The present invention aims to overcome this drawback of the prior art. [Means for solving the problem]

[0019] According to the invention, this object is achieved by dispersing a light beam in a first direction so as to form a "spectral line" that illuminates the specimen. Thus, several points on the specimen aligned in said first direction are simultaneously illuminated with different wavelengths. It is therefore possible to inspect a part of the surface of the specimen by only scanning in a second direction that is not parallel to the first direction (preferentially perpendicular).

[0020] Having to scan in only one direction instead of two significantly reduces image acquisition time. Additionally, in some embodiments of the present invention, this allows the use of resonant mirrors that oscillate at very high frequencies—on the order of tens of kHz—which further reduces acquisition time. The use of such mirrors would be difficult to contemplate if the two scans along the first and second directions had to be synchronized according to prior art techniques.

[0021] The replacement of scanning in the spatial direction with spectral lines has already been proposed, for example, in (Non-Patent Document 8) and (Non-Patent Document 9). More specifically, (Non-Patent Document 8) concerns a fiber-based confocal microscope intended for integration into an endoscope, while the microscope of (Non-Patent Document 9) replaces scanning in the second direction with spatially variable frequency modulation. None of these systems is capable of exceeding the Abbe limit and, in particular, is incompatible with the principle of photon relocation.

[0022] Therefore, one object of the present invention is a photon rearrangement imaging system, which comprises: a polychromatic light source configured to generate at least one spatially coherent illumination light beam over a range of illumination wavelengths; and a first optical system configured to disperse the illumination light beam with a first angular dispersion DA1 and focus it on the specimen to form a first spectral line directed in a first direction; a second optical system configured to collect the light beam elastically scattered by the specimen, called the signal beam, to apply a second angular dispersion DA2 to it and to focus it in a focal plane to form a second spectral line directed in a second direction; a matrix image sensor arranged in the image focal plane; - means for inducing a relative displacement between the first spectral line and the sample in a third direction perpendicular to the first direction; means for forming an image of the specimen through photon relocation based on one or more images acquired by the matrix image sensor corresponding to a plurality of different positions taken by a first spectral line on the specimen; It encompasses a first optical system including a first dispersive optical element for dispersing the illumination light beam with a first angular dispersion DA1, and a second optical system including a second dispersive optical element for applying the second angular dispersion DA2 to the illumination light beam; The length of the second spectral line in the second direction is longer than the length of the first spectral line in the first direction, and the ratio of the size of the light beam on the first dispersive optical element measured in the first direction to the size of the signal beam on the second dispersive optical element measured in the second direction is defined as M. G When A G =M G DA2 / DA1 times, magnification A G is greater than 1.

[0023] According to a specific embodiment of the present invention, the magnification AG can be set to 1.8 to 2.2.

[0024] The first optical system is a first collimator system configured to collimate the illumination light beam; said first dispersive optical element; - focusing optics configured to receive as input the illumination light beam dispersed by the first dispersive element and to focus it onto a specimen; can be included.

[0025] The second optical system is said or another focusing system for collecting the signal beam, collimating it and directing it to the first or another dispersive optical element so that the signal beam is spectrally recombined; the first dispersive optical element or the further dispersive optical element configured to spectrally recombine the signal beams; a second dispersive optical element having a second angular dispersion DA2 for spectrally dispersing the signal beam in the second direction; at least one optical element configured to focus the signal beam spectrally dispersed by the second dispersive optical element onto a focal plane, referred to as the image focal plane; can be included.

[0026] In that case, the second optical system may include a pinhole located in a plane conjugate with the focal plane of the focusing system of the second optical system to provide confocal filtering.

[0027] The means for forming an image of the specimen through photon relocation based on the plurality of images of the second spectral line acquired by the matrix image sensor can include a digital image processor, in which case the means for inducing a relative displacement between the first spectral line and the specimen can include a specimen translation stage or alternatively a deformable mirror common to the first and second optical systems configured to apply an angular scan in a third direction to the illumination light beam and an angular scan in an opposite direction to the spectrally recombined signal beam.

[0028] According to another embodiment, the means for inducing a relative displacement between the first spectral line and the specimen may comprise a first deformable mirror common to the first and second optical systems, configured to apply an angular scan in a third direction to the illumination light beam and to apply an angular scan in an opposite direction to the spectrally recombined signal beam; the means for forming an image of the specimen through photon relocation includes a second deformable mirror configured to apply an angular scan to the signal beam spectrally dispersed by the second dispersive optical element in a fourth direction perpendicular to the second direction; The angular scan applied by the second deformable mirror is synchronized with that applied by the first mirror and has an amplitude A S / M S has an amplitude equal to times A S is the magnification of 1.8 to 2.2, and M Sis the ratio of the dimension of the illumination light beam on the second deformable mirror measured in the fourth direction to the dimension of the spectrally dispersed signal beam on the first deformable mirror measured in the third direction, in which case the first and second deformable mirrors can be structurally independent and the first dispersive optical element is separated from the second dispersive optical element.

[0029] The first dispersive optical element can be a diffractive system having the angular dispersion DA1 in the first diffraction order and the angular dispersion DA2=2DA1 in the second diffraction order, and the second optical system is configured to collect light scattered in the second diffraction order by the diffractive system.

[0030] In other embodiments, the first and second variable-orientation mirrors can be formed by two separate reflective regions of the same variable-orientation support, e.g., said variable-orientation support. The first dispersive optical element can be separated from the second dispersive optical element.

[0031] The light source can include a first set of microlenses aligned in a third direction to generate a plurality of parallel illumination light beams, whereby the first optical system forms a plurality of first spectral lines on the specimen that are oriented in the first direction and arranged in the third direction, and the second optical system can include a second set of microlenses aligned in the third direction to form a plurality of first spectral lines on the matrix image sensor that are oriented in the first direction and arranged in a fourth direction perpendicular to the second direction.

[0032] It should be noted that, unlike what is taught, for example, in (NPL 8) and (NPL 9), the imaging system according to the present invention forms a second spectral line on the image sensor that is longer (ideally twice as long) than the first spectral line projected onto the specimen, and performs a second spatial dispersion operation, which is necessary to enable photon relocation.

[0033] Further features, details and advantages of the invention will become apparent from the following description, given by way of example only, with reference to the accompanying drawings in which: [Brief explanation of the drawings]

[0034] [Figure 1] 1 is a schematic diagram of a confocal microscope with photon relocation according to a first embodiment of the present invention; [Figure 2] FIG. 2 is a schematic diagram of a confocal microscope with photon relocation according to a second embodiment of the present invention. [Figure 3] FIG. 10 is a schematic diagram of a confocal microscope with photon relocation according to a third embodiment of the present invention. [Figure 4] FIG. 10 is a schematic diagram of a confocal microscope with photon relocation according to a fourth embodiment of the present invention. [Figure 5] FIG. 10 is a schematic diagram of a confocal microscope with photon relocation according to a fifth embodiment of the present invention. [Figure 6] FIG. 10 is a schematic diagram of a confocal microscope with photon relocation according to a sixth embodiment of the present invention. [Figure 7] FIG. 10 is a schematic diagram of a confocal microscope with photon relocation according to a seventh embodiment of the present invention. [Figure 8] FIG. 10 is a schematic diagram of a confocal microscope with photon relocation according to an eighth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0035] In these figures, corresponding elements are designated with the same reference symbols.

[0036] In the apparatus shown in Figure 1, a polychromatic light source SL generates a spatially coherent (but generally not temporally coherent) light beam FE. The source SL includes a broadband source SLS, such as an arc lamp, light-emitting diode, or pulsed laser. A spectral filter FSC selects a portion of the source SLS's spectrum. The broad wavelength range selected by the filter FSC, together with the angular dispersion DA1, determines the broadness of the first spectral line scanned across the specimen. Its sizing represents a compromise: a broad spectral range lowers the constraint on angular dispersion DA1 and allows for utilization of a larger portion of the total light intensity emitted by the source SLS, but makes correcting chromatic aberrations (axial and longitudinal) in the imaging system more difficult, especially when using an f-theta scan lens, whose allowable spectral width Δλ / λ is typically around 10%. The resolution across the field of view also varies over a wide spectral range (optical resolution is proportional to the local wavelength).

[0037] Converging lens L 110 A pinhole P1, placed in the focal plane of this lens, performs spectral filtering to ensure spatial coherence of the light beam FE.

[0038] The light beam FE emitted by the light source SL is reflected by a beam splitter BS, which directs it towards a first optical system SO1 intended to illuminate a specimen E. In addition to the beam splitter BS, said optical system SO1 also includes a first focusing lens L, which collimates the beam FE. 105 and a plane mirror M 102 (not required) and two other focusing lenses L that form an afocal system. 104 , L 103 The collimating lens L contains the first pinhole P1 and the second pinhole P2 in a common focal plane, which is therefore optically conjugate with the first pinhole P1 at the output of the light source SL. 103The light beam leaving the grating G1 is directed to a diffraction grating G1 having an angular dispersion DA1 in a first direction x. The beam diffracted by the grating G1 is then focused by two further focusing lenses L 101 , L 102 through an afocal system consisting of a plane mirror M 102 (optional) and then focused by a microscope objective MO onto the specimen E. Spectral dispersion by the grating G1 and focusing by the objective MO results in the formation of a spectral line LS1 on the specimen, i.e., an elongated focal spot oriented in the x-direction, at which the wavelength of light varies monotonically with position in the x-direction.

[0039] Lens L 101 and L 102 The main function of the afocal system formed by is to systematically position the diffraction grating G1 in a plane conjugate with the pupil of the microscope objective at the rear of the microscope. In addition, it can also be used to adjust the size of the light beam.

[0040] The specimen E is mounted on a translation stage PT that can be translated in the y direction, perpendicular to the x direction. In this way, the spectral line LS1 scans one surface of the specimen.

[0041] The second optical system SO2 collects the light beam FS ("signal beam") elastically scattered by the specimen (i.e., scattered without a change in wavelength) and forms a second spectral line LS2 on the matrix image sensor CMI. The imaging system shown in Figure 1 operates in a reflective mode, which allows for the sharing of optical elements between the first and second optical systems. More specifically, all elements of the first optical system SO1 also belong to the second optical system, which also includes additional components. The light backscattered by the specimen E is also collected and collimated by the microscope objective MO and spectrally recombined by the same diffraction grating G1 that caused scattering on the outward path. The lens L 103 and L 104The afocal system formed by allows for confocal filtering as well as modification of the size of the beam FS, just like in confocal microscopes according to the prior art.

[0042] The beam FS passes through a beam splitter BS, which reflects the light beam FE from the light source SL. Two focusing lenses L are placed on either side of the beam splitter. 105 and L 106 forms an afocal system that allows the placement of a second diffraction grating G2 in a plane conjugate with the pupil of the objective lens MO. This second diffraction grating disperses the light again in the x-direction. Lens L 107 forms a second spectral line LS2 on the matrix image sensor CMI, oriented in the x-direction.

[0043] The matrix image sensor can be manufactured using, for example, CMOS or CCD technology. The number of pixels in the x-direction is determined by the spatial spread of the spectral lines obtained by the optical system SO2 and is typically several hundred. The number of pixels in the y-direction can be smaller, such as about 10 or even several tens, since these pixels are only used for the implementation of the photon relocation algorithm by the digital image processor PNI. To implement this algorithm, the digital image processor PNI must be synchronized with the moving stage. The digital image processor PNI can be, for example, a suitably programmed microprocessor or a dedicated digital circuit.

[0044] For this algorithm to achieve an improvement in spatial resolution, the second spectral line LS2 must be longer in the x-direction, ideally by a factor of 2 or close to 2 (for example, 1.8-2.2). This is shown in the top left of the figure, where it can be clearly seen that the segment corresponding to the wavelength interval Δλ is longer in the spectral line LS2 than in LS1. This stretching is achieved by using a second grating G2 with an angular dispersion DA2 that is larger than the angular dispersion DA1 of the first grating, and by using a lens L103 , L 104 , L 105 , and L 006 The magnification M of the light beam produced by the assembly G Generally speaking, the diameter d of the beam incident on the grating G2 is G2 and the diameter d of the beam incident on the grating G1 G1 The ratio of G Then (M G =d G2 / d G1 ), M G DA2 / DA1 = length (LS2) / length (LS1) = A G A G is a magnification between 1.8 and 2.2, and ideally has a value of 2. For example, M G = 1, a second grating will be selected with (roughly) twice the angular dispersion of the first grating.

[0045] The embodiment shown in FIG. 2 is a mirror M 102 The embodiment shown in FIG. 1 differs from the embodiment shown in FIG. 1 only in that the first spectral line LS1 is replaced by an oscillating (or more generally, variable-orientation) mirror S1. More precisely, the change in the orientation of mirror S1 is controlled so as to induce a displacement in the y direction of a first spectral line LS1 on the specimen E, for this reason S1 will be referred to hereinafter as a "scanning mirror". In this way, the spectral line LS1 scans the specimen E, making its translation along the y direction unnecessary. The relocation of photons is performed digitally, as in the embodiment shown in FIG. 1, by an image processor PNI synchronized with the movable mirror S1.

[0046] In the embodiment shown in FIG. 3, the second optical system SO2 also includes a lens L for focusing the beam FS onto the matrix image sensor CMI. 107 2 in that it includes an oscillating scanning (or more generally orientable) mirror S2 positioned in front of the focusing lens L. 308and L 309 The afocal system formed by the lens can be conjugated to the pupil of the microscope objective MO.

[0047] The change in the orientation of mirror S2 is controlled to induce a displacement in the y direction of the second spectral line LS2 on the matrix image sensor, thus achieving a purely optical photon relocation. To achieve a higher resolution, this scan must be synchronized with the scan performed by the first spectral line LS1 on the specimen E and have a higher amplitude, i.e. the diameter d of the beam on mirror S2 S2 and the beam diameter d on mirror S1 s1 The ratio of S When this is done, the magnification A should be between 1.8 and 2.2, ideally 2. S Regarding A S / M S In the embodiment shown in FIG. G Lens L 103 , L 104 , L 105 , L 106 , L 308 , and L 309 is the magnification of the entire assembly.

[0048] Unlike the embodiments shown in Figures 1 and 2, the number of pixels in the y-direction of the matrix image sensor CMI is determined by the amplitude of the scan performed by the second spectral line LS2, and is typically an order of magnitude larger than the number of pixels in the y-direction.

[0049] In addition, if the scanning performed by the scanning mirrors S1 and S2 is fast enough, the image sensor CMI does not necessarily need to acquire a separate image for each position of the spectral lines LS1 and LS2, but a single image can correspond to multiple positions. In boundary cases, the acquisition of a single image may be sufficient, making synchronization between the scanning mirrors S1, S2 and the matrix image sensor unnecessary.

[0050] In the embodiments shown in Figures 1, 2 and 3, the scanning rate is limited by the need for synchronization, i.e., between the translation stage PT and the matrix image sensor CMI in the case of Figure 1, between S1 and the matrix image sensor CMI in the case of Figure 2, and at least between S1 and S2 in the case of Figure 3. This limitation can be overcome by the embodiment shown in Figure 4, in which the two scanning mirrors S1 and S2 are made on two opposite sides of the same oscillating support, or more generally, of an orientable support SPO. In the specific embodiment shown in the figures, this is achieved by the two steering mirrors M 402 and M 403 This eliminates any need for synchronization and makes it much easier to use high scanning frequencies in the kilohertz range, which can be achieved, for example, with a resonant support SPO or polygon scanner. Since the angular amplitude of the scans performed by mirrors S1 and S2 is the same due to the construction, the magnification M of the beam FS between the two scanning mirrors provided by the second optical system S must be approximately 2 (in other words, d S2 / d S1 =M S =A S In this case, at least approximately A S = 2). In the embodiment shown in FIG. 4, this is the lens L 105 , L 106 , L 408 , and L 409 This is achieved by an afocal system formed from

[0051] The embodiment shown in [Fig. 5] is similar to the embodiment shown in [Fig. 4], except that the functions of the two diffraction gratings G1 and G2 are performed by two separate regions of a single grating G, which is connected to a turning mirror M 502 , M 503 , and M 504This is possible by the use of , and in the example shown in Figure 5, by the fact that the beam splitter BS passes the light beam FE but reflects the light beam FS. Since the two separate gratings G1 and G2 are replaced by two regions of the same grating, DA1 = DA2, and the magnification M G is approximately 2, in other words, at least approximately d G2 =2d G1 This means that it will have to become...

[0052] The embodiments shown in Figures 4 and 5 require two diffraction gratings, or one grating large enough to provide two separate regions that can be illuminated independently of each other, whereas the embodiment shown in Figure 6 includes a single grating whose spatial extent need not be greater than the extent of the light beam FE.

[0053] The system shown in FIG. 6 includes a light source SL as described above with reference to FIG. 1. A light beam FE from this light source is collimated through a collimation lens L 105 and is collimated by the first redirecting mirror M 601 The beam is then deflected by a (optional) lens, G1, and reaches a diffraction grating G1 where it is spectrally dispersed in the first diffraction order. The spectrally dispersed beam is then passed through two lenses L, which form an afocal system. 601 and L 602 The spectral dispersion is then spatially filtered by a slit FR located in the common focal plane of the lens L. The length of the slit FR in the spectral dispersion direction x depends on the angular dispersion introduced by the grating, while its width in the direction y is determined in the same way as for a pinhole in a conventional confocal microscope. 602 and L 105 The main function of the afocal system formed by is to optically conjugate the slit FR with the pinhole P1 of the light source SL. Alternatively, the slit FR can be omitted.

[0054] The light beam dispersed and spatially filtered by the grating G1 is directed to the microscope objective MO by a scanning mirror S1 formed by the reflective surface of an oscillating or orientable support SPO, resulting in a spectral line LS1 that scans the specimen E, as previously described with reference to other embodiments. The light backscattered by the specimen is collected by the objective MO and reflected by S1, which compensates for the scanning effect introduced on the "outward journey", and is reflected by the assembly L 601 -FR-L 602 , which reaches the diffraction grating G1 again after being spatially filtered by the grating G2. The originality of this embodiment lies in the fact that instead of focusing the beam spectrally recombined by the first diffraction order of the grating, which propagates along the path of the light beam but in the opposite direction, the second diffracted order light (beam FS' in the figure) is used. Since the second order angular dispersion is twice that of the first order, there is no spectral recombination of the second order diffracted beam FS', but it has an angular dispersion twice the amplitude of the first order diffracted light beam FE (in other words, the spectral dispersion introduced on the "outward journey" is overcompensated). This beam FS' is then focused by the lens L 607 and L 608 The afocal system formed by S = 2 times larger (d S2 =2d S1 ), Miller M 602 , M 603 , M 604 , which is then redirected (optionally) by a scanning mirror S2 formed on the same orientable support SPO as S1, and finally by a lens L 609 , which forms a spectral line LS2 on the matrix image sensor CMI. In this embodiment, this configuration G1 =d G2 This becomes:

[0055] It should be noted that in the aforementioned embodiments, the first optical system SO1 and the second optical system for forming the first spectral line LS1 on the specimen E are substantially coincident, with the outgoing light (forming the first spectral line and illuminating the specimen) and the returning light (collecting and processing the signal beam) passing through the same optical components. More specifically, in the embodiments shown in FIGS. 1-5, the first optical system is a subset of the second optical system. This is less true for the embodiment shown in FIG. 6, where the use of second-order diffraction of the diffraction grating actually separates the optical path of the signal beam from the optical path of the light beam. In other embodiments, the first and second optical systems may be completely or substantially decoupled. This is particularly true for imaging systems operating in a transmission mode.

[0056] Such a system is shown in Figure 7. The operating principle is similar to that of Figure 3, except that it operates in transmission mode rather than reflection mode. Similarly, the embodiments shown in Figures 1, 2, 4, and 5 can be adapted to operate in transmission mode.

[0057] In the system shown in FIG. 7, a light beam FE emitted from a light source SL passes through a focusing lens L 702 The beam is collimated by the deformable mirror S, which scans it in the y direction. 10 The beam FE is reflected by the afocal system L 703 ~L 704 (Note that SO1 does not require a pinhole and does not have one) and passes through the (optional) mirror M 701 After being deflected by the grating G 10 and the afocal system L 703 ~L 704 and lens L 702 The beam is then focused onto the Fourier plane of the pinhole P1 of the light source SL by the second afocal system L. 705 ~L 706 and (optionally) a mirror M 702After being deflected again by the afocal system L, it is focused by the first microscope objective MO1 to form the spectral line LS1 on the specimen E. 706 L 705 The function of is to optically conjugate the rear pupil of the microscope objective with the grating G1. All of these elements make up the first SO1 optical system.

[0058] The light that passes through the specimen (signal beam FS) is collected and collimated by a second microscope objective MO2, located opposite the first microscope objective, and then passes through another diffraction grating G 11 (Note that the dispersion and spectral recombination functions performed by the grating G0 alone in the previous embodiment are now performed by these two gratings.) 10 G 10 is the angular dispersion of 11 G 11 In the case of angular dispersion of M G10~G11 DA 11 =DA 10 In this case, M G10~G11 L 705 , L 706 , MO1, MO2, L 707 , and L 708 The magnification formed by the grating G 11 Diameter d of the light beam above G11 and lattice G 10 Diameter d of the light beam above G10 The ratio of the turning mirror M 703 and afocal system L 707 ~L 708 allows this grating to be optically conjugated to the rear pupil of MO2. Similarly, L 705 , L 706 , MO1, MO2, L 707 , and L 708 The system formed by G 10 and G 11 can be considered as optically conjugates, and G 11 Dispersion DA imaged in the plane of 10 DA11 The spectrally recombined light beam is then transmitted through the assembly L 709 -P2-L 710 and the deformable mirror S 11 This mirror reaches S 10 so that the angular scan introduced by S is corrected 10 The operation is synchronized with S 10 and S 11 The total magnification between M S10~S11 (S 11 The diameter of the beam incident on S11 and S 10 The diameter of the beam incident on S10 with the vibration amplitude divided by the ratio of the lattice G 10 and G 11 As in the case of 10 and S 11 (In embodiments operating in reflective mode, these replace the single directing mirror S1). Similarly, S 10 and S 11 can be considered to be optically conjugate, and S 11 The image of S formed on the plane 10 The angular displacement of S 11 The beam FS is then spectrally dispersed by yet another diffraction grating G2, with an angular dispersion DA2 equal to or nearly equal to the angular displacement of M G2~G11 DA2 / DA 11 =A G = 2 (at least approximately), where M G2~G11 is d G2 (diameter of the beam incident on G2) and d 11 The spectrally redispersed beam is then scanned by another scanning mirror S2, i.e., S 10 and S 11 synchronized with the afocal L 711 , L 712 , L 713 , L 714 (in other words, the diameter d of the light beam on mirror S2) S2 and d S11 (ratio to MS2 When S2 M S10~S2 The lens L is rotated so as to form a spectral line LS2 oriented in the x direction, which is reflected by a mirror with an oscillation amplitude approximately equal to . 715 The afocal system L 711 , L 712 , and L 713 ~L 714 ensures optical conjugation between the various optical elements of system SO2. Note that this system, operating in transmission mode, has more components than the previous embodiment and requires synchronization of the three scanning mirrors. To relax these synchronization constraints, S 10 , S 11 , and S 12 It is possible to replace the scanning or oscillating system with a single scanning or oscillating system such as a resonant mirror or polygon scanner.

[0059] As explained above, it is possible to accelerate the angular velocity of the scanning mirror to shorten the image acquisition time, but this is technically challenging. An alternative approach, shown in [Figure 8], consists of replacing the single light beam with a multiple-beam MFE arranged in the y-direction.

[0060] The embodiment shown in [Fig. 7] is similar to the embodiment shown in [Fig. 3], and only the differences will be described in detail. First, the light source SL is replaced by a light source SLMF including a set of microlenses MML1 aligned in the y direction, which decomposes a single light beam emitted from a broadband spectrum light source SLS into multiple beams MFE. The spatial coherence of these beams is ensured by multiple pinholes MP1 arranged in the focal plane of each microlens, instead of a single pinhole P1. Similarly, the pinhole P2 used for confocal filtering is replaced by multiple pinholes MP2 aligned in the y direction. Also, the lens L in [Fig. 3] 105 is a lens L arranged at the output of the light source SLMF to collimate the multiple light beams MFE. 810The focusing lens L focuses the multiple signal beams MFS after passing through the beam splitter BS. 805 Note that the first and second microlenses MML1 and MML2 are replaced by a plurality of second microlenses aligned in the y direction. In this system, N>1 spectral lines PLS1 and PLS2 aligned in the y direction (oriented in the x direction) are generated on the specimen E and the matrix image sensor, respectively. This allows the angular scanning performed using the scanning mirrors S1 and S2 to image the same region of the specimen to be N times shorter than in the case of a single spectral line. Therefore, the time required to acquire an image of that region of the specimen is also reduced by a factor of N.

[0061] Although the present invention has been described with reference to a certain number of embodiments, variations are possible, such as:

[0062] The light beam does not necessarily have to be circular in cross section. In that case, the concept of diameter must be replaced by the concept of transverse dimension. More specifically, the concept of beam diameter on the grating must be replaced by the concept of beam dimension in the dispersion direction (x-direction in the above example), and the concept of beam diameter on the scanning mirror must be replaced by the concept of beam dimension in the scanning direction (y-direction in the above example).

[0063] The internal structure of the light source SL is only an example, other structures are possible.

[0064] All or part of the focusing lens can be replaced by other optical elements, for example, a concave mirror or a diffractive system such as a holographic lens.

[0065] The diffraction grating or parts thereof can be replaced by other dispersive systems, for example three-dimensional diffractive structures, or even refractive systems such as prisms (except in the case of the embodiment shown in Figure 6, which requires a diffractive system).

[0066] The microscope objective can be replaced by other types of focusing systems, such as a concave mirror or an F-theta lens, in which case the dispersive element does not necessarily have to be located in a plane conjugate with the pupil of the focusing system, but more generally in a plane conjugate with the Fourier plane of the specimen.

[0067] The -x and y directions do not necessarily have to be orthogonal, but it is advantageous if they are; they simply need not be parallel.

[0068] In embodiments in which the two scanning mirrors S1 and S2 are supported by the same orientable structure, they do not necessarily have to be located on opposite sides of that structure.

[0069] The concept of a "variable orientation" mirror or support covers both resonant vibrating structures and structures whose orientation changes in a controlled manner, either continuously or in discrete steps.

[0070] [Reference] (Sheppard 1988): CJR Sheppard, “Super-resolution in Confocal Imaging”, Optik 80, No.2, pages 53, 54. (York 2013): AG York et al. “Instant super-resolution imaging in live cells and embryos via analog image processing,” Nat. Methods 2013, November, 10(11), pages 1122-1126. (De Luca 2013): GMR De Luca “Re-scan confocal microscopy: scanning twice for better resolution” Biomedical Optics Express, Vol.4, No.11, November 2013. (Curd 2015): A. Curd et al. “Construction of an instant structured illumination microscope”, Methods 88 (2015) pages 37 47. (DuBose 2019): T. B. DuBose et al. “Super-resolution retinal imaging using optically reassigned scanning laser ophthalmology”, Nature Photonics, Vol.13, April 2019, pages 257-262. (Sandison 1995): D. R. Sandison et al. “Quantitative comparison of background rejection, signal-to-noise ratio, and resolution in confocal and full-field laser scanning microscopes” Applied Optics Vol.34, No.19, July 1, 1995, pages 3576-3588. (Aguilar 2020) Aguilar, A., Boyreau, A., Bon, P. “Label-free super-resolution imaging below 90-nm using photon-reassignment”. Open Research Europe, 1(3), 3 (2021). (Tearney 1998) G. J. Tearney et al. “Spectrally encoded confocal microscopy” Optics Letters, Vol.23, No.15, August 1998. (Hwang 2015) Hwang J. et al. “Frequency and spectrally-encoded confocal microscopy” Optics Express, Vol.23, No.5, March 9, 2015 [Explanation of symbols]

[0071] A G magnification A S magnification BS Beam Splitter CMI Matrix Image Sensor DA1~DA 11 angular dispersion d G1 ~d G10 Beam diameter E specimen FR Slit FSC Spectral Filter G0~G 11 lattice L 101 ~L 810 lens LS1 Spectral Line LS2 Second Spectral Line M 101 ~M 703 Flat mirror MFE Beam MML1-MML2 microlens set MO, MO1 to MO2 objective lenses P1, P2 pinholes PNI Digital Image Processor S1~S 12 mirror SL multicolor light source SLS Broad Spectrum Light Source SO1 First optical system SO2 Second optical system SPO support

Claims

1. 1. A photon rearrangement imaging system, comprising: a polychromatic light source (SL) configured to generate at least one spatially coherent illumination light beam (FE) over a range of illumination wavelengths; - directing said illumination light beam to a first angular dispersion DA １ and a first optical system (SO1) configured to disperse the light beam at a wavelength of 1000 nm and focus the light beam on the sample (E) to form a first spectral line (LS1) directed in a first direction (x); - collecting the light beam (FS) elastically scattered by the specimen, called the signal beam, and applying a second angular dispersion DA ２ and focusing it onto a focal plane to form a second spectral line (LS2) directed in a second direction (x); a matrix image sensor (CMI) arranged in said image focal plane; means (PT, S) for inducing a relative displacement between said first spectral line and said sample in a third direction (y) perpendicular to said first direction; １ )and, means for forming an image of the specimen through photon relocation based on one or more images acquired by the matrix image sensor corresponding to a plurality of different positions taken by the first spectral line on the specimen; It encompasses The first optical system distributes the illumination light beam with a first angular dispersion DA １ A first dispersion optical element (G １ , G １０ ) and the second optical system provides a second angular dispersion DA to the illumination light beam. ２ a second dispersive optical element (G ２ ) The length of the second spectral line in the second direction is longer than the length of the first spectral line in the first direction, and the ratio of the size of the light beam on the first dispersive optical element measured in the first direction to the size of the signal beam on the second dispersive optical element measured in the second direction is defined as M. Ｇ When this is done, it is A Ｇ = M Ｇ D.A. ２ / DA １ times, and the magnification A Ｇ is greater than 1, Photon rearrangement imaging system.

2. The magnification A Ｇ 2. The photon rearrangement imaging system of claim 1, wherein is 1.8 to 2.

2.

3. The first optical system (SO1) comprises: a first collimator system (L) configured to collimate said illumination light beam (FE); １０５ , L ２０５ , L ３０５ , L ４０５ , L ５０５ , L ６０５ )and, said first dispersive optical element (G １ , G １０ )and, a focusing optical system (MO, MO1) configured to receive as input the illumination light beam dispersed by the first dispersive element and to focus it onto the specimen (E); 3. The photon rearrangement image processing system according to claim 1, further comprising:

4. The second optical system (SO2) comprises: the focusing system or another focusing system (MO, MO2) for collecting the signal beam, collimating it and directing it towards the first dispersive optical element or another dispersive optical element so that the signal beam is spectrally recombined; the first dispersive optical element (G) configured to spectrally recombine the signal beams; １ ) or the other dispersive optical element (G １１ )and, said second angular dispersion DA ２ and a second dispersive optical element (G) for spectrally dispersing the signal beam in the second direction (x). ２ )and, - the second dispersive optical element (G ２ at least one optical element (L) configured to focus the signal beam, spectrally dispersed by the at least one optical element (L), onto a focal plane called the image focal plane; １０７ , L ２０７ , L ３０９ , L ４０９ , L ５０９ , L ６０９ )and, 4. The photon rearrangement imaging system of claim 3, comprising:

5. The second optical system includes a pinhole (P) disposed in a plane conjugate with the focal plane of the focusing system of the second optical system (MO) so as to perform confocal filtering. ２ 5. The photon rearrangement imaging system of claim 4, comprising:

6. 6. The photon rearrangement imaging system of claim 4, wherein the means for forming an image of the specimen through photon rearrangement based on a plurality of images of the second spectral line acquired by the matrix image sensor includes a digital image processor (PNI).

7. 7. The photon rearrangement imaging system of claim 6, wherein said means for inducing a relative displacement between said first spectral line and said specimen comprises a specimen translation stage (PT).

8. The means for inducing a relative displacement between the first spectral line and the sample comprises a deformable orientation mirror (S) common to the first and second optical systems, configured to apply an angular scan in the third direction to the illumination light beam and to apply an angular scan in the opposite direction to the spectrally recombined signal beam. １ 7. The photon rearrangement imaging system of claim 6, comprising:

9. the means for inducing a relative displacement between the first spectral line and the specimen comprises a first orientation-variable mirror (S ) common to the first and second optical systems, configured to apply an angular scan in the third direction to the illumination light beam and to apply an angular scan in the opposite direction to the spectrally recombined signal beam; １ ) The means for forming an image of the specimen through photon relocation comprises a second deformable orientation mirror (S) configured to apply an angular scan to the signal beam spectrally dispersed by the second dispersive optical element in a fourth direction perpendicular to the second direction. ２ ) The angular scan applied by the second deformable mirror is synchronized with that applied by the first mirror and has an amplitude A Ｓ / M Ｓ times the amplitude, where A Ｓ is the magnification of 1.8 to 2.2, and M Ｓ is the ratio of a dimension of the illumination light beam on the second deformable mirror measured in the fourth direction to a dimension of a spectrally dispersed signal beam on the first deformable mirror measured in the third direction; The photon rearrangement imaging system according to any one of claims 3 to 5.

10. The first variable orientation mirror (S １ ) is the second variable orientation mirror (S ２ ) and is structurally independent of said first dispersive optical element (G １ ) is the second dispersive optical element (G ２ 10. The photon rearrangement imaging system of claim 9, wherein the photon rearrangement imaging system is separated from the optical fiber.

11. The first dispersive optical element (G １ ) is the angular dispersion DA in the first order diffraction １ and in the second order of diffraction, the angular dispersion DA ２ =2DA １ 10. The photon rearrangement image processing system of claim 9 when dependent on any one of claims 1 to 3, or claim 3 alone, wherein the second optical system is a diffractive system exhibiting a second order of diffraction by the diffractive system, and the second optical system is configured to collect the light scattered in a second order of diffraction by the diffractive system.

12. 12. A photon rearrangement imaging system according to claim 9 or claim 11, wherein the first and second deformable mirrors are formed by two separate reflective areas of the same deformable support (SPO).

13. 13. The image processing system of claim 12, wherein the variable orientation support (SPO) is selected from a resonant mirror support and a polygon scanner.

14. 14. An image processing system according to claim 12 or claim 13, wherein the first dispersive optical element (G1) is separated from the second dispersive optical element (G2).

15. The light source (SLMF) is aligned in the third direction (y) with a first set of microlenses (MMLs) that generate a plurality of parallel illumination light beams. １ ), whereby the first optical system (SO1) forms on the specimen (E) a plurality of first spectral lines (LS1) oriented in a first direction (x) and arranged in the third direction (y), and the second optical system (SO2) forms a second set of a plurality of microlenses (MML) aligned in the third direction (y). ２ 15. The image processing system according to claim 1, further comprising a matrix image sensor (CMI) including a first spectral line (LS1) and a second spectral line (LS2) arranged in a fourth direction (y) perpendicular to the second direction (x).

Images