Hyperspectral imaging device and method

Abstract

A hyperspectral imaging device (1, 100, 200, 300, 400; 500; 600; 700; 800) and method are disclosed herein. The device and method for hyperspectral imaging disclosed herein overcome the technical problems of the prior art by replacing the intensity measurement performed by a single high-resolution 2D sensor of the prior art methods with a measurement of the intensity (fluctuation) correlation retrieved by two high-resolution 2D sensors: one is an imaging / space sensor (4, 204, 304, 404; 504; 604; 704; 804) dedicated to polychromatic image acquisition, the other is a spectral sensor (5, 205, 305, 405; 505; 605; 705; 805) dedicated to pure spectral measurement. In the hyperspectral correlation imaging according to the present application, the spectral information is encoded as intensity correlation without any need for spectral scanning. Even if multiple exposures (frames) are usually needed to reconstruct the light statistics and perform the correlation measurement, the exposure time is several orders of magnitude shorter than the time needed in the scanning methods; moreover, there is no need to change the device during such multiple exposures: this simplifies the optical / optomechanical mechanism of the device (1, 100, 200, 300, 400; 500; 600; 700; 800) and avoids further time consumption.

Description

TECHNICAL FIELD

[0001] The present invention relates to hyperspectral imaging. BACKGROUND

[0002] Current devices for hyperspectral imaging rely on intensity measurements performed by high-resolution 2D sensors and are essentially based on two complementary concepts: scanning and snapshot imaging. A general problem of such devices is that information which is actually three-dimensional, the third dimension being the frequency of the light, is encoded on a 2D sensor.

[0003] In scanning hyperspectral imaging techniques, a high-resolution 2D sensor acquires a sequence of monochromatic images, one for each frequency in the range of interest. Examples of scanning hyperspectral systems include point-scanning spectrometers, pushbroom spectrometers, tunable filter cameras, Fourier transform imaging spectrometers, computed tomography hyperspectral imaging spectrometers and coded aperture line imaging spectrometers. The drawback of this approach is obviously the amount of time required to acquire the sequence of monochromatic images.

[0004] On the other hand, in snapshot hyperspectral imaging techniques, a high-resolution 2D sensor is divided into multiple 2D blocks of pixels, each block containing information about all the desired frequencies, one for each pixel. Examples of snapshot hyperspectral systems include integral field spectroscopy with a multi-faceted mirror, coherent fiber bundles and lenslet arrays, multispectral beam splitting, computed tomography imaging spectroscopy, multi-aperture filter cameras, tunable stepwedge imagers, spectrally resolving detector arrays, image replication imaging spectrometers, coded aperture snapshot imagers, image mapping spectroscopy, snapshot hyperspectral imaging Fourier transform spectrometers, multispectral Sagnac interferometers. Here, the fast parallel acquisition of the required multispectral images is at the expense of image and spectral resolution.

[0005] Therefore, the scanning approach requires an extremely time-consuming procedure to achieve fine spectral resolution, while the snapshot approach is characterized by a strong trade-off between spatial and spectral resolution, which are inversely proportional for a given sensor.

[0006] OBJECT OF THE INVENTION

[0007] It is an object of the present invention to solve the above technical problems. More specifically, it is an object of the present invention to provide a high-resolution hyperspectral imaging device and method capable of achieving high image resolution at fast processing rates. SUMMARY

[0008] The object of the present invention is achieved by the device and method having the features of the appended claims, which form an integral part of the technical disclosure provided herein in relation to the present invention. BRIEF DESCRIPTION OF DRAWINGS

[0009] Further features and advantages of the present application will become apparent from the following description, given by way of non-limiting example only, with reference to the accompanying drawings, in which:

[0010] - Figure 1 is a schematic view of a device according to a further embodiment of the present application,

[0011] - Figure 2 is a schematic view of a device according to a further embodiment of the present application,

[0012] - Figure 3 is a schematic view of a device according to a further embodiment of the present application,

[0013] - Figure 4 is a schematic view of a device according to a further embodiment of the present application,

[0014] - Figure 5 is a schematic view of a device according to a further embodiment of the present application,

[0015] - Figure 6 is a schematic view of a device according to a further embodiment of the present application,

[0016] - Figure 7 is a schematic view of a device according to a further embodiment of the present application,

[0017] - Figure 8 is a schematic view of a device according to a further embodiment of the present application, and

[0018] - Figure 9 is a schematic view of a device according to a further embodiment of the present application. DETAILED DESCRIPTION

[0019] Figure 1 Reference number 1 in Fig. 1 generally designates a device for hyperspectral imaging according to an embodiment of the present application. The device 1 comprises a collection window 2, a beam splitter 3 configured for splitting a light beam B coming from an object OBJ and entering the collection window into a first secondary light beam B1 travelling along a first optical path S1 and comprising a plurality of first secondary light signals, and a second secondary light beam B2 travelling along a second optical path S2 and comprising a plurality of second secondary light signals. Generally, the object OBJ can be regarded as a light source for the device 1 according to the present application, whereby each single point of the volume of the object OBJ is capable of emitting a primary light signal of the light beam B.

[0020] The whole image of the object is a composition of all the light signals emitted from the object OBJ and collected through the collection window 2. Therefore, while the following functional description is based on the attached figures representing a single primary light signal emitted from the object OBJ and processed into a secondary light signal, the functional description applies to the full primary light beam B emitted from the object, which comprises all the primary, point-based light signals (and accordingly to the full secondary light beams B1, B2 comprising all the corresponding first and second secondary light signals). In the embodiment, the beam splitter 3 is arranged in the proximity of the collection window 2 downstream the light path of the primary light beam B, so as to be illuminated by the light beam B, to split the light beam B into the first secondary light beam B1 and the second secondary light beam B2. However, in other embodiments, the arrangement can vary due to, for example, space or mounting requirements.

[0021] In other embodiments, as will become apparent in the following, there is little or no access between the beam splitter 3 and the collection window, where the device relies on a more complex light path between the collection window and the beam splitter. In general - this applies to all the embodiments herein - the collection window can be a simple planar screen without light path modification capabilities, leading to a configuration where no further optical elements are arranged between the collection window and the beam splitter, or to a configuration where one or more optical elements (lenses) are arranged between the collection window and the beam splitter, or it can be a lens element or a multi-lens group to provide the desired degree of processing of the primary light beam B before incidence on the beam splitter 3.

[0022] The device 1 further comprises a first sensor 4 configured to be incident by the first secondary light beam B1 and a second sensor 5 configured to be incident by the second secondary light beam B2. Throughout the description, for reasons that will become apparent in the following, the sensor 4 can be referred to as a "spatial sensor" while the sensor 5 can be referred to as a "spectral sensor". A correlator C is operatively connected to the sensor 4 and to the sensor 5 to perform a correlation of the light intensity data / information retrieved by the sensors. A frequency splitter device 6 is arranged along the second light path S2 and is configured to split the second secondary light beam B2 into its frequency components before incidence on the second sensor 5. In the present embodiment, the frequency splitter device comprises a prism.

[0023] According to the application, the first sensor 4 is configured to retrieve from the first secondary light beam B1 an image of the object OBJ, which image (2-D) comprises a plurality of spatial positions, each spatial position being defined by the incidence of a respective first light signal of the secondary light beam B1 onto the first sensor 4. In other words, the first sensor 4 is an imaging sensor, configured to reconstruct, upon incidence of the light signals coming from the object OBJ, by interaction between the primary light signals constituting the primary light beam B (the secondary signals constituting the secondary light beam B1) and the sensor elements thereof, an image of the object OBJ. The image (2-D) of the object OBJ is formed on the sensor 4 by means of focusing elements, for example comprising one of a lens (simple, compound, lens group), a curved mirror (spherical or parabolic), a dioptric reflecting system or a system comprising a lens and a mirror. For example, when the acquisition window 2 comprises a focusing lens (or when a focusing lens is arranged at the position illuminated by the primary light beam B or by the first secondary light beam B1), the relationship holds, so as to acquire on the first sensor 4 a focused image of the object, where f is the focal length of the lens, S OBJ is the distance between the focal plane of the object OBJ and the main plane of the lens, S I is the distance between the main plane of the lens and the sensor 4.

[0024] On the other hand, the second sensor 5 is configured to retrieve, from the second secondary light beam B2 deriving from the light beam B coming from the object OBJ, and for each spatial position of the image of the object OBJ, spectral information. In other words, since each light signal constituting the light beam B2 is paired with a respective light signal constituting the light beam B1 by means of the beam splitter 3, the light signal coming from the light beam B2 intrinsically carries the spatial information of the light signal coming from the light beam B1, but it is used to encode the information in the frequency domain. In other words, while the detection by the sensor 4 of the light signals coming from the light beam B1 returns an image of the point of the object OBJ from which the individual light signals of the light beam B come (the secondary light beam B1 deriving exactly from the light beam B), the detection by the sensor 5 of the paired light signals of the light beam B2 returns spectral data of the same point of the object OBJ.

[0025] The "physical" dimension of the two data sets (image and spectrum) can not be the same, and generally is not the same: while the image data are based on a one-to-one correspondence between each pixel (or generally sensor element) of the sensor 4 and a specific region on the object, as far as the spectral data are concerned, the information on the same frequency or band can be stored in a plurality of pixels (for example, in a pixel array). As mentioned above, the pairing of the two data sets, representative of the physical pairing quantity, which is intrinsic to the generation of the secondary light beams B1 and B2 by the beam splitter, taking into account the chaotic nature of the light coming from the object OBJ, is implemented by means of the correlator C.

[0026] In some embodiments, one or more wavelength converters (e.g. coatings) can be associated with the sensor 5 to convert the wavelength of the incident light to a wavelength at which the sensor 5 operates with higher sensitivity. The "actual" spectral information can then be reconstructed by the control system by taking into account the conversion ratio of the converters.

[0027] Moreover, in embodiments, the frequency divider device can be provided as a set of dichroic mirrors configured to divert each frequency band to a respective sensor 5: thus, in such embodiments, the spectral sensor can be more than one.

[0028] Moreover, with reference to Figures 2 to 9 , the device according to the application can be susceptible to structural variations, while still retaining the functional features described above. For this reason, Figures 2 to 5 other embodiments are provided, designated by reference numbers 100, 200, 300, 400, 500, 600, 700, 800, and in which the same components as those already combined Figure 1 in the device disclosed are marked with the same reference numbers, while the components that vary compared to those of the device Figure 1 are marked with reference numbers in the same range as the main reference number of the embodiment being designated, and deviate accordingly from the corresponding features or numbers of the embodiment Figure 1 .

[0029] With reference to Figure 2 , the device 100 according to the application is identical to the device 1, except that the frequency divider device is designated as 106 and comprises a diffraction grating. As with the device 1, the correlator C is operatively connected to the sensors 4 and 5 to perform the correlation of the light intensity data / information retrieved by the sensors.

[0030] Figure 3 The embodiment of figure 2 (in which the device according to the application is designated by reference number 200) is an example of an embodiment in which the first (or imaging) sensor and the second (or spectral) sensor are provided on a single sensor element 245 and correspond to different sensitive areas or elements of the sensor element 245. In this particular embodiment, the imaging sensor is designated by reference number 204 and corresponds to a first area of the sensor element 245, while the spectral sensor is designated by reference number 205 and corresponds to a second area of the sensor element 245 adjacent to the first area. The sensor areas are disjoint, i.e. the optical arrangement of the device 200 is such that there is no intersection between the secondary light beams Bl and B2 impinging on the sensors 204, 205. As with the previous embodiments, the correlator C is operatively connected to the sensors 205 and 204 to perform the correlation of the light intensity data / information retrieved by the sensors.

[0031] The device 200 also comprises additional components that cause a partially modified optical path. This can be an example of an alternative arrangement in which the optical path is adapted to certain requirements or constraints that can be dictated by the relevant application.

[0032] The device 200 comprises a collection window 202, unlike the device 1 or the device 100, which is not in the field of view of the beam splitter 3. On the contrary, the collection window 202 is arranged facing a parabolic mirror 207 configured to reflect the incident (through the collection window 2) light beam B coming from the object OBJ to a deviating mirror 208 (preferably a plane mirror) from which the light beam B is focused (209) before or after the beam splitter 3. The first secondary light beam B1 has an optical path S1 that passes through an imaging lens 210, which can in itself be optional, while the second secondary light beam B2 has an optical path S2 that passes through a collimating lens 211 arranged upstream of the frequency separator 6 (preferably a prism, but a diffractive grating can also be used).

[0033] A second far-field mirror (preferably a parabolic mirror) 212 is arranged downstream of the frequency separator 6 to transfer the split frequency components of the secondary light beam B2 to the spectral sensor 205.

[0034] Reference Figure 4 Yet another embodiment of a device according to the present application is indicated by reference number 300. In the device 300, the same functional layout is implemented as in the embodiment of Figure 2 , but the imaging sensor (reference number 304) and the spectral sensor (reference number 305) are again provided on a single sensor element 345 and correspond to different sensitive areas or elements of the sensor element 345. In this particular embodiment, the imaging sensor 304 corresponds to a first area of the sensor element 345, while the spectral sensor is indicated by reference number 305 and corresponds to a second area of the sensor element 345 adjacent to the first area. As in the previously disclosed embodiments, the sensor areas are disjoint, i.e. the optical arrangement of the device 300 is such that there is no intersection between the secondary light beams B1 and B2 impinging on the sensors 304, 305. As in the previous embodiments, the correlator C is operatively connected to the sensors 304 and 305 to perform the correlation of the light intensity data / information retrieved by the sensors.

[0035] Like the device 200, the device 300 also comprises additional components that, compared to Figure 1 , cause a partially modified optical path. This can also be an example of an alternative arrangement in which the optical path is adapted to certain requirements or constraints that can be dictated by the relevant application.

[0036] The device 300 comprises a collection window 302 which, like the device 1, is in the field of view of the beam splitter 3, preferably through an imaging lens interposed between the collection window 302 and the beam splitter 3.

[0037] The primary light beam B coming from the object OBJ is split into a first secondary light beam B1 and a second secondary light beam B2, the light path S1 of the first secondary light beam eventually reaching the imaging sensor 304, the light path S2 of the second secondary light beam impinging on a collimating mirror 308 (preferably parabolic) which is configured to reflect the light beam B2 to the frequency splitter 6. From the latter, the light path S2 is diverted to the spectral sensor 305, also passing through a far field lens 309.

[0038] Reference Figure 5 Yet another embodiment of a device according to the present application is indicated by reference number 400. In the device 400, Figure 3 The functional layout of the embodiment of 400 is slightly changed, as described below. In any case, as far as the sensor part is concerned, the device 400 has the same layout as the device 200, wherein the imaging sensor 404 and the spectral sensor 405 are arranged on a single sensor element 445 and correspond to different sensitive areas or elements of the sensor element 445. In this particular embodiment, the imaging sensor 404 corresponds to a first area of the sensor element 445, while the spectral sensor 405 corresponds to a second area of the sensor element 445 adjacent to the first area. As in the previously disclosed embodiments, the sensor areas are disjoint, i.e. the optical arrangement of the device 400 is such that there is no intersection between the secondary light beams B1 and B2 impinging on the sensors 304, 305. As in the previous embodiments, the correlator C is operatively connected to the sensors 405 and 404 to perform the correlation of the light intensity data / information retrieved by the sensors.

[0039] The device 400 comprises a collection window 402 which comprises a lens or a set of lenses. The window 402 is in the field of view of the beam splitter 3 so that the primary light beam B coming from the object OBJ is split into a first secondary light beam B1 and a second secondary light beam B2, the first secondary light beam B1 having a light path S1 which passes through a second lens 407 and eventually reaches the imaging sensor 404, the second secondary light beam B2 having a light path S2 which directly impinges on a blazed reflective grating 6 which is the frequency splitter device. The split frequency components of the light beam component B2 impinge on a far field mirror 408 which in turn is configured to divert the frequency components to the spectral sensor 405.

[0040] Reference Figure 6 Yet another embodiment of a device according to the present application is indicated by reference number 500. In the device 500, Figure 3The functional layout of embodiments of the application is again slightly varied, as described below. In any event, as far as the sensor part is concerned, the device 500 has a similar layout as the device 200, with the imaging sensor 504 and the spectral sensor 505 being disposed on a single sensor element 545, and corresponding to different sensitive areas or elements of the sensor element 545. In this particular embodiment, the imaging sensor 504 corresponds to a first area of the sensor element 545, while the spectral sensor 505 corresponds to a second area of the sensor element 545 adjacent to the first area. As with the previously disclosed embodiments, the sensor areas are disjoint, i.e. the optical arrangement of the device 500 is such that there is no intersection between the secondary light beams Bl and B2 impinging on the sensors 504, 505. As with the previous embodiments, a correlator C is operatively connected to the sensors 405 and 505 to perform correlation of the light intensity data / information retrieved by the sensors.

[0041] The device 500 comprises a collection window 502 comprising a lens or a set of lenses. The window 502 is in the field of view of the beam splitter 3, such that the primary light beam B from the object OBJ is split into a first secondary light beam Bl having a light path SI through a second lens 507 and eventually to the imaging sensor 504, and a second secondary light beam B2 having a light path S2 impinging on a collimating mirror 508, which is preferably parabolic, which turns the second secondary light beam B2 towards the reflective diffraction grating 6, which operates as a frequency demultiplexer. The reflective diffraction grating 6 turns the second secondary light beam B2 towards a focusing mirror 509, which in turn focuses (and turns) the second secondary light beam B2, having been split into its frequency components, onto a cylindrical lens 510. The cylindrical lens focuses each frequency band onto a line, thereby acquiring the spectral information onto the spectral sensor 505 as an array of frequency-dependent pixels (i.e. one row of pixels / sensing elements per frequency band).

[0042] In a variant, the mirror 509 and the lens 510 can be provided as a collection aimed at focusing the spectral information onto a single pixel instead of a band of pixels, to reduce the area of the sensor dedicated to spectral acquisition. In this way, the area of the sensor can be made available and "traded" to the spatial sensor 504 to obtain additional resolution or for full-optical imaging capabilities. Moreover, with pixel focusing of the spectral signal, a linear spectral sensor can be used instead of a spectral sensor as a 2D sensor.

[0043] In more detail, the arrangement of the mirror 509 and the lens 510 providing pixel-focused spectral data onto the sensor 505 comprises one of:

[0044] i) the focusing mirror is a concave mirror exhibiting focusing properties only along an axis orthogonal to the axis of the lens 510;

[0045] ii) the mirror 509 is a circularly symmetric concave mirror and the lens 510 is a pair of cylindrical lenses whose axes are orthogonal to each other;

[0046] ii) the mirror 509 is a plane mirror and the lens 510 is a pair of cylindrical lenses.

[0047] Reference Figure 7 Yet another embodiment of the apparatus according to the present application is indicated by reference number 600. The apparatus 600 differs from all the embodiments disclosed so far in that it does not comprise a separate frequency divider apparatus along the light path of the secondary light beam upstream of the spectral sensor. Instead, as will be disclosed in the following, the frequency related information is retrieved directly by the spectral sensor.

[0048] The apparatus 600 comprises a collection window 602 in the field of view of a focusing lens 603 (which can be itself the collection window 602) and a beam splitter 3 downstream of the light path of the light beam B emitted from the object OBJ, through which the primary light beam B from the object OBJ is split into a first secondary light beam B1 having a light path S1 that impinges onto an imaging sensor 604 and a second secondary light beam B2 having a light path S2 that impinges directly onto a spectral sensor 605. As in the previous embodiments, a correlator C is operatively connected to the sensors 605 and 604 to perform the correlation of the light intensity data / information retrieved by the sensors 604, 605. The imaging sensor 604 and the spectral sensor 605 are preferably provided as physically distinct elements.

[0049] While the imaging sensor 604 is not distinguished from the imaging sensors already disclosed with respect to the other embodiments, the sensor 605 has a matrix of pixels (or, more generally, of sensing elements), wherein each pixel (or sensing element) has a spectral sensitivity peaking at a different frequency. This can be achieved by a frequency selective coating of the sensor area, or by a hybrid structure of the sensor, making it include sensor elements that physically peak at a certain frequency - in terms of spectral sensitivity. This obviates the use of a separate frequency divider apparatus, since it is the sensor 605 that performs the frequency detection in the form of a specific retrieval of the signal associated with the secondary light beam B2 at the frequency thereof. The light signal impinging the sensor 605 should represent a signal contribution from the entire object: this requires verifying the following condition:

[0050]

[0051] where O is the distance between the object OBJ and the plane of the lens 603, i1 is the distance between the plane of the lens 603 and the imaging sensor 604; f’ is the focal length of the lens 603, i2 is the distance between the focal plane of the lens 603 and the sensor 605. In order for the sensor 605 to collect the primary light signal from the whole object OBJ, the sensor should not be placed on the plane where the object OBJ is focused. One possible solution to this can be to arrange a lens, possibly a cylindrical lens, between the beam splitter 3 and the sensor 605 in order to have an image of the focusing lens 603 on the sensor 605. Otherwise, if one wants to avoid the arrangement of additional components, one can make the distance i2 very short compared to i1, which can be achieved by positioning the beam splitter 3 very close to the focusing lens 603.

[0052] Reference Figure 8 Yet another embodiment of the device according to the present application is denoted by reference number 700. The device 700 comprises a collection window 702 comprising a lens or a set of lenses. The window 702 is in the field of view of the beam splitter 3 so that the primary light beam B from the object OBJ is split into a first secondary light beam B1 and a second secondary light beam B2. As for the first secondary light beam B1, it has an optical path S1 that goes through a lens 707 and eventually reaches a second beam splitter 3’ that is configured to split the secondary light beam B1 into a third secondary light beam B1’ and a fourth secondary light beam B1”, the former being illuminated onto a primary imaging sensor 704 and the latter being illuminated onto a secondary imaging sensor 7040. As for the second secondary light beam S2, it has an optical path S2 that goes through a collimating lens 708, is diverted by a plane mirror 709 to a diffraction grating 710 that is a frequency splitter device, and further goes through a focusing lens 711 before being illuminated onto a spectral sensor 705. As in the previous embodiments, a correlator C is operatively connected to the sensors 704 and 705 to perform the correlation of the light intensity data / information retrieved by the sensors, but it is also operatively connected to the sensor 7040.

[0053] In the device 700, thanks to the pairing of the spatial sensor 704 with the secondary spatial sensor 7040, the latter can be used to retrieve focused images of the object OBJ on planes different from the one on which the "main" spatial sensor 704 is focused. This variant allows to perform 1) hyperspectral imaging of two focal planes by correlating the intensity on the main spatial sensor 704 or on the secondary sensor 7040 with the intensity on the spectral sensor 705; 2) 3D imaging by correlating the intensity on the two spatial sensors, exploiting the "correlation between arbitrary planes" principle, according to which the correlation between intensity fluctuations on the two sensors focusing images of two different planes allows to reconstruct the light directions in the scene, providing the possibility to refocus out-of-focus images, to change the viewpoint on the scene and to perform three-dimensional image reconstruction; 3) hyperspectral 3D imaging by correlating the signals on all three sensors. In the last case, according to the correlation between arbitrary planes principle, the product of the intensity fluctuations in each pair of points on the spatial sensors 704 and 7040 is correlated with the pixel of the spectral sensor corresponding to a given wavelength to obtain a 3D image of the scene corresponding to that particular wavelength.

[0054] In general, the correlator C is configured to operate the correlation of data coming from the sensors 704, 7040 and 705, 704 and 7040 (correlation of plenoptic), 704 and 705 (as in the previous embodiment), or 7040 and 705.

[0055] With reference to Figure 9 Yet another embodiment of a device according to the present application is indicated by reference number 800. The device 800 has some similarities with the device 600 in that it does not comprise a separate frequency divider device along the optical path of the secondary light beam upstream of the spectral sensor. Instead, as will be disclosed hereinafter, the frequency-related information is retrieved directly by the spectral sensor.

[0056] The device 800 comprises a collection window 802 in the field of view of a focusing lens 803 (which can itself be the collection window 802) and a beamsplitter 3 downstream of the optical path of the light beam B emitted from the object OBJ. By the beamsplitter 3, the primary light beam B coming from the object OBJ is split into a first secondary light beam B1 having an optical path S1 which is imaged onto an imaging sensor 804 and a second secondary light beam B2 having an optical path S2 which is imaged onto a mirror 810 configured to divert the optical path S2 through a lens 811 and eventually onto a spectral sensor 805. The lens 803 focuses the image of the object OBJ onto the beamsplitter 3 and onto the sensor 804 (secondary light beam B1), while the combination of the mirror 810 and the lens 811 processes the second secondary light beam B2 to return the image of the lens 803 (i.e. the image produced by the primary light beam imaged onto the lens 803) onto the spectral sensor 805, where the image is de-focused (i.e. out of focus) to provide the sensor 805 with a secondary light beam (or secondary light signal) representative of the light emitted from the entire object OBJ, as disclosed with respect to the imaging system 600. To this end, the following combinations can be envisaged:

[0057] - a plane mirror 810 and a spherical lens 811 (two-dimensional image of the lens 803 on the sensor 805)

[0058] - a concave mirror 810 and a cylindrical lens 811 having an axis arranged perpendicular to the drawing (linear image of the lens 803 on the sensor 805), i.e. parallel to the plane of the second sensor 805.

[0059] The imaging sensor 804 and the spectral sensor 805 are provided on a single sensor element 845 and correspond to different sensitive areas or elements of the sensor element 845. In this particular embodiment, the imaging sensor 804 corresponds to a first area of the sensor element 845, while the spectral sensor 805 corresponds to a second area of the sensor element 845 adjacent to the first area. As with the previously disclosed embodiments, the sensor areas are disjoint, i.e. the optical arrangement of the device 800 is such that there is no intersection between the secondary light beams B1 and B2 imaged onto the sensors 804, 805. As with the previous embodiments, a correlator C is operatively connected to the sensors 804 and 805 to perform the correlation of the light intensity data / information retrieved by the sensors.

[0060] In this case, the sensor area dedicated to the spectral measurement (sensor area 805) has a matrix of pixels (or more generally, a matrix of sensing elements), where each pixel (or sensing element) has a spectral sensitivity that peaks at a different frequency. Again, this can be achieved by a frequency-selective coating of the sensor area, but also by a hybrid structure of the sensor, such that it comprises sensor elements that physically peak at a certain frequency - in terms of spectral sensitivity. This spares the use of a separate frequency divider device, since it is the sensor (area) 805 that performs the frequency detection in the form of a frequency-specific retrieval of the signal associated with the secondary light beam B2.

[0061] In all embodiments 1, 100, 200, 300, 400, 500, 600, 700, 800 disclosed herein, the spatial sensor 4, 204, 304, 404, 504, 604, 704, 7040, 804 can be an image acquisition sensor, such as a two-dimensional CCD, CMOS or s-CMOS camera, or even a photodiode array, or a SPAD array, while the spectral sensor 5, 205, 305, 405, 505, 605, 705, 805 can be of the same type as the spatial sensor, or a line-scan camera.

[0062] According to the present application, in all embodiments herein, the device 1, 100, 200, 300, 400, 500, 600, 700, 800 comprises a processing unit configured to retrieve a hyperspectral image of the object OBJ by a correlation measure between:

[0063] - a first light intensity information retrieved at a spatial position on a first sensor 4, 204, 304, 404, 504, 604, 704 (and 7040, where applicable), 804 associated with a first light signal of a first secondary light beam B1, the first light intensity carrying information about an image of the object OBJ, and

[0064] - a second light intensity retrieved at a position on a second sensor 5, 205, 305, 405, 505, 605, 705, 805 associated with a second light signal of a second secondary light beam B2 paired with the first light signal of the first secondary light beam B1, the second light intensity information being provided for each frequency in the spectral information retrieved by the second sensor 5, 205, 305, 405, 505, 605, 705, 805,

[0065] The correlation measure is provided over an exposure time.

[0066] Therefore, a full hyperspectral image is obtained by performing a correlation measure for each spatial position of the image of the object OBJ, as described above, each spatial position is defined by the illumination of the first sensors 4, 204, 304, 404, 504, 604, 704 (and 7040, where applicable), 804 by a corresponding first secondary light signal of the first secondary light beam B1.

[0067] The following description is intended to provide a comprehensive disclosure of the operation of the apparatus 1 , 100 , 200 , 300 , 400 , 500 , 600 , 700 , 800 and the general method for hyperspectral imaging according to the present invention, as implemented by a hyperspectral apparatus according to the present invention in any of the previously disclosed embodiments.

[0068] The method according to the invention is based on measuring the intensity and The correlation between and At each pair of points ρ a and ρ b The subsequent simultaneous observations (frames) on the register, a (ρ a ) on the imaging sensors 4, 204, 304, 404, 504, 604, 704, 804, one (ρ b ) on spectral sensors 5, 205, 305, 405, 505, 605, 705, and 805. Point ρ a and ρ b exist Figures 1-5 The correlation is operatively measured by a correlator C that is part of or connected to each hyperspectral imaging device 1 , 100 , 200 , 300 , 400 , 500 , 600 , 700 , 800 .

[0069] quantity and They correspond to the first light intensity information and the second light intensity information mentioned above, and represent the time average value of the light intensity within the exposure time Δt. More precisely, and is the time average of the light intensity during the exposure time Δt, corresponding to the first and second light intensities measured at the ends of the secondary beams B1 and B2, respectively. Assuming the average intensity (where j = a; b) is obtained by measuring each point ρ within a time window of duration Δt. j The light intensity at is obtained. For the convenience of calculation, considering Gauss, the following applies:

[0070]

[0071] The hyperspectral image of the object OBJ under consideration is thus encoded in the quantity

[0072]

[0073] This quantity corresponds to the correlation of the intensity fluctuations, where <. > denotes the average over the randomness of the light coming from the object OBJ. If the object OBJ can be modeled as a traversing light source, this average is essentially identical to the average over the considered quantity on the collected frames.

[0074] Under the further assumption that

[0075] - the light signal emitted by the object OBJ is stationary, i.e. the expectation value of the physical quantity involved in the light signal does not depend on the reference time,

[0076] - the light emitted by the object OBJ has negligible transverse coherence and its fluctuations approximately follow a Gaussian distribution,

[0077] The correlation of the intensity fluctuations defined in equation (2) is

[0078]

[0079] where

[0080]

[0081] where τ c is the coherence time of the observed light corresponding to the inverse frequency bandwidth, ω0is the central frequency of the observed light, g j (ρ j ,ρ OBJ ; ω) is the function that propagates the electromagnetic field component corresponding to the frequency ω from the point p OBJ of the object to the point p j on the sensor / detector D j (where j = a, b; in this case, D a = sensor 4, 204, 304, 404, 504, 604, 704, 804, D b = sensor 5, 205, 305, 405, 505, 605, 705, 805). The product

[0082]

[0083] is the light intensity component corresponding to the frequency ω at the point p OBJ of the object, which is conveniently separated into a Gaussian envelope of width and a slowly varying function I OBJ. The goal of hyperspectral imaging is to detect the function I OBJ (ρ OBJ , ω) of all points on the object plane with fine frequency resolution.

[0084] A key assumption for performing hyperspectral related imaging is that the propagator p a determining the propagation of the secondary beam B1 needs to know the geometric point-to-point correspondence between points on the object plane and points on the detector / sensor D a while the propagator p b which determines the propagation of the beam B2 is essentially independent of the object point and reaches a peak near the frequency in the emission bandwidth:

[0085]

[0086] S OBJ as the distance between the object and the first principal plane of the imaging system, the constant C b determines the intensity of the field emitted towards p b and

[0087] τ cut (ρ b ) = τ c (6).

[0088] The frequency filtering by the inverse width is operated by g b at a certain point.

[0089] Taking into account the assumptions discussed earlier and approximating the slowly varying function of the intensity with its value on the central filter frequency one obtains the following result

[0090]

[0091] where

[0092]

[0093] The image of the expected frequency component of the intensity distribution is suitably encoded. If the imaging system on the optical path a (i.e. the optical path a (reference Figure 1 , Figure 2 )) is characterized by a focal length f and an effective aperture P(ρ), then the function (9) reads

[0094]

[0095] The above function has the structure of a quasi-coherent image of the component of the intensity distribution I OBJ which corresponds to a frequency with a coherent point spread function determined by the planar Fourier transform of the lens aperture.

[0096] Typically, a group of different pixels on a sensor can correspond to the same frequency This redundancy can be exploited to increase the probability of the frequency component by integrating the measured correlation over all pixels corresponding to that frequency. Finally, note that the devices 1, 100, 200, 300, 400, 500, 600, 700, 800 can be used for multispectral imaging, rather than hyperspectral imaging, by integrating the correlations over a set of pixels on the spectral sensor / detector Db, corresponding to wavelengths belonging to a finite interval (typically ≥ 20 nm in width).

[0097] In the most general case, embodiment 700 can be used to measure the correlation between all three sensors 704, 7040, and 705 in order to obtain simultaneous information about: 1) the spatial distribution of light from an object (i.e., an image), 2) the propagation direction of the light from the object, and 3) the spectrum of the light from the object. This measure of the correlation between the three sensors 704, 7040, and 705 is described by the following quantity:

[0098]

[0099] in and is the coordinate ρ on the sensor 704 a , coordinate ρ′ on sensor 7040 a and the coordinate ρ on sensor 705 b The intensity fluctuation measured for a pixel is calculated as the difference between the intensity obtained during the exposure time Δt and the average intensity.

[0100] Γ (3) (ρ a ,ρ′ a ,ρ b ), with varying ρ a and ρ′ a , and for the frequency corresponding to a specific ρ b Each fixed value of contains the plenoptic information on the scene corresponding to that particular frequency (i.e., encoding the spatial distribution and direction of light, as needed for refocusing the image in post-processing and performing 3D imaging).

[0101] Therefore, all the results shown with reference to the embodiments 1, 100, 200, 300, 400, 500, 600 and 700 (only for the sensor 705 in combination with 704 or 7040) are also applicable to the most general use of the embodiment 700, which is further enriched by the full light information. The devices and methods for hyperspectral imaging disclosed herein overcome the technical problems of the prior art by replacing the intensity measurement performed by a single high-resolution 2D sensor in the prior art methods with the measurement of the intensity (fluctuation) correlation retrieved with the following two high-resolution 2D sensors: one is the imaging / space sensor 4, 204, 304, 404, 504, 604, 704, 804 - dedicated to polychromatic image acquisition, the other is the spectral sensor 5, 205, 305, 405, 505, 605, 705, 805 - dedicated to pure spectral measurement.

[0102] In hyperspectral correlation imaging, the spectral information is encoded as intensity correlation without any need for spectral scanning. Even if multiple exposures (frames) are usually needed to reconstruct the light statistics and perform the correlation measurement, the exposure time is several orders of magnitude shorter than the time needed in the scanning methods; moreover, there is no need to change the device during such multiple exposures: this simplifies the optical / optomechanical mechanism of the device 1, 100, 200, 300, 400, 500, 600, 700, 800 and avoids further time consumption.

[0103] On the other hand, since the spatial and spectral information is retrieved by two different sensors, there is no trade-off between spatial and spectral resolution of the (intensity-based) snapshot hyperspectral imaging typical of the prior art in the devices and methods according to the present application.

[0104] It is worth emphasizing that, in addition to the ability to obtain hyperspectral images through correlation measurement, the processes and devices of the present application also retrieve full-resolution panchromatic images of the object by means of the intensity measurement performed by the imaging (spatial) sensor 4, 204, 304, 404, 504, 604, 704, 804.

[0105] Still further, all the embodiments disclosed herein can be modified to have full optical imaging capability and, accordingly, to increase the depth of field.

[0106] Naturally, while the principles of the application remain unchanged, the details of construction and embodiments can vary widely from what has been described purely by way of example, without thereby departing from the scope of the application.

Claims

1. A device for hyperspectral imaging, comprising: - an acquisition window, - a beam splitter configured for splitting a primary light beam coming from an object and entering the acquisition window into a first secondary light beam having a first optical path and comprising a plurality of first secondary light signals, and a second secondary light beam having a second optical path and comprising a plurality of second secondary light signals, - a first sensor configured to be incident by the first secondary light beam, - a second sensor configured to be incident by the second secondary light beam, wherein: - the first sensor is configured to retrieve an image of the object from the first secondary light beam, the image comprising a plurality of spatial locations, each spatial location being defined by the incidence of a first secondary light signal of the first secondary light beam on the first sensor, - the second sensor is configured to retrieve spectral information from the second secondary light beam and for each spatial location of the image, the device further comprises a processing unit configured to retrieve a hyperspectral image of the object by a correlation measure between, within an exposure time: - first light intensity information at the spatial locations on the first sensor associated with the first secondary light signals of the first secondary light beam, and - second light intensity information at the locations on the second sensor associated with the second secondary light signals of the second secondary light beam, the second secondary light signals of the second secondary light beam being paired with the first secondary light signals of the first secondary light beam, the second light intensity information being provided for each frequency in the spectral information retrieved by the second sensor.

2. The apparatus of claim 1, wherein, The first sensor and the second sensor are different sensors.

3. The apparatus of claim 1, wherein, The first sensor and the second sensor are disposed on a single sensor element and correspond to different sensitive areas or elements of the sensor element.

4. The device according to any of the preceding claims, further comprising a frequency divider device arranged along the second optical path and configured to process the second secondary light beam to retrieve a frequency component thereof upon interaction with the second sensor, wherein, The frequency splitter device is configured to split the second secondary light beam into its frequency components before the second secondary light beam is incident on the second sensor.

5. The apparatus of any one of claims 1 to 3, wherein, The second sensor comprises a matrix of sensing elements, wherein each sensing element has a spectral sensitivity peaking on a different frequency.

6. The apparatus of any one of claims 1 to 3, wherein, The acquisition window is arranged facing a first mirror configured to reflect an incident light beam coming from the object to a shift mirror configured to divert the light beam to the beam splitter, wherein the first secondary light beam has a first optical path through an imaging lens, while the second secondary light beam has a second optical path through a collimating lens arranged upstream of a frequency splitter, and wherein a second mirror is arranged downstream of the frequency splitter to divert the frequency components of the second secondary light beam to the second sensor.

7. The apparatus of claim 6, wherein, The first mirror is a parabolic mirror.

8. The apparatus of claim 4, wherein, The acquisition window is in the field of view of the beam splitter, and wherein the first optical path passes through a lens to terminate on the first sensor, and wherein the second optical path passes through the frequency splitter device and is incident on a far field mirror, in turn configured to divert the frequency components to the second sensor.

9. The apparatus of claim 4, wherein, The second secondary light beam comprises an optical path incident on a collimating mirror configured to divert the second secondary light beam to the frequency divider device, in turn configured to divert the second secondary light beam to a focusing mirror, in turn focusing the second secondary light beam to a cylindrical lens, wherein the cylindrical lens is configured to focus each frequency band on a line.

10. The apparatus of claim 9, wherein, The focusing mirror is a concave mirror exhibiting focusing properties only along an axis orthogonal to the axis of the lens.

11. The apparatus of claim 5, wherein, The beam splitter is configured to split the primary light beam into a first secondary light beam having an optical path S1 incident on an imaging sensor and a second secondary light beam having an optical path directly incident on a spectral sensor.

12. The device of claim 1 or 2, comprising a second beam splitter configured to split the first secondary light beam into a third secondary light beam and a fourth secondary light beam, the former being configured to be incident on the first sensor as a primary imaging sensor, the latter being configured to be incident on a secondary imaging sensor configured to retrieve a focused image of the object on a plane different from the plane on which the primary imaging sensor is focused.

13. The apparatus of any one of claims 1-3, comprising a first focusing lens configured to be incident by the primary light beam, wherein, The beam splitter is configured to split the primary light beam focused by the focusing lens into a focused first secondary light beam incident on the first sensor and a focused second secondary light beam configured to be defocused before being incident on the second sensor by one of the following combinations: - a plane mirror and a spherical lens; - a concave mirror and a cylindrical lens having an axis arranged parallel to the plane of the second sensor.

14. A method for hyperspectral imaging, comprising: - splitting a primary light beam into a first secondary light beam comprising a plurality of first secondary light signals and a second secondary light beam comprising a plurality of second secondary light signals, - directing the first secondary light beam to a first sensor, - directing the second secondary light beam to a second sensor, - retrieving an image of an object by means of the first sensor and from the first secondary light beam, the image comprising a plurality of spatial positions each defined by the incidence of a respective first secondary light signal of the first secondary light beam on the first sensor; - retrieving spectral information of each spatial position of the image by means of the second sensor and from the second secondary light beam, - retrieving a hyperspectral image of the object by means of a correlation measure between: - first light intensity information at a spatial position on the first sensor associated with a first secondary light signal of the first secondary light beam, and - second light intensity information at a position on the second sensor associated with a second secondary light signal of the second secondary light beam, the second secondary light signal of the second secondary light beam being paired with the first secondary light signal of the first secondary light beam, the second light intensity information being provided for each frequency in the spectral information retrieved by the second sensor.

15. The method of claim 14, wherein, The first sensor and the second sensor are provided on a single sensor element and correspond to different sensitive areas or elements of the sensor element.

16. The method according to claim 14 or claim 15, comprising splitting the second secondary light beam into its frequency components before being incident on the second sensor.

17. The method of claim 14, wherein, The second sensor comprises a matrix of sensing elements, wherein each sensing element has a spectral sensitivity peaking at a different frequency.

18. The method according to claim 14, further comprising: - splitting the first secondary light beam into a third secondary light beam and a fourth secondary light beam; - directing the third secondary light beam to the first sensor and the fourth secondary light beam to another sensor, the other sensor being configured for retrieving an image of the object, and - retrieving, by the other sensor, an image of the object on a plane different from the plane from which the first sensor retrieves data.

19. The method according to claim 18, wherein the other sensor is a camera.

Images