Multi-zone hyperspectral sensor

Abstract

Multi-zone hyperspectral sensor (2), comprising a processor (14) and a detector array (10) with zones (114, 140, 144) of detectors (56) coupled to the processor (14) and configured to detect light in distinct spectral bands, and comprising a telecentric optical system (6) in front of said detector array (10), whereby said processor (14) is configured to integrate light from said zones (114, 140, 114), whereby that said optical system (6) is configured to deliver a defocussed image on said detector array (10).

Description

[0001] MULTI -ZONE HYPERSPECTRAL SENSOR

[0002] DESCRIPTION

[0003] TECHNICAL FIELD

[0004] The invention relates to multi-zone spectral sensors. It more particularly relates to multi-zone spectral sensors with a detector array and an optical system.

[0005] TECHNICAL BACKGROUND

[0006] In many fields of applications, it is advantageous to obtain in addition to a picture with spatial resolution of a scene also a corresponding hyperspectral picture with spectral resolution. The spectrum of these pictures can comprise visible light but also light frequencies in the IR or UV range. Such spectral information can help to detect features which are not detectable in a regular picture. In other camera applications, the hyperspectral image can be used in automatic white balancing algorithms.

[0007] In optical systems, single zone and multi zone sensors can be used. A typical single zone sensor detects irradiance [lx, lm / m2] => illuminance [W / m2] . A diffuser on top of the sensor mixes all the light within the field of view (FOV, which can be up to 180°) into a near lambertian distribution or constant angular distribution towards the photodiode array of the sensor. Diffused light which arrives at the sensor is independent from the granularity of the picture scene. The single zone sensor in this way detects the sum of light sources .

[0008] A multi zone sensor detects luminous intensity [cd, Im / sr] => radiant flux [W / sr] . Light is integrated zone-wise (similar to a camera) . The detected light depends on the granularity of picture scene, on the FOV which can be up, to but also greater than 120°, and / or the directionality of each zone. The multi- zone sensor can zone wise detect the hue of di f ferent ambient and arti ficial light sources .

[0009] A common approach to ultra-wide-angle spectral sensing is to use a di f fuser for achieving a uni form detector response regardless of the angle of incident light . To achieve the same in multi- zone spectral sensing, use of a di f fuser is impractical because this will signi ficantly increase device si ze and complexity . This would need each angular zone to be optically shielded from each other to avoid the scattered light from one angular zone hitting the detector space corresponding to another angular zone after interacting with di f fuser .

[0010] A further common approach to obtain a multispectral picture of a scene is to create separate optical copies of the scene that are then imaged onto di f ferent sensor / camera elements , each group with a common wavelength selectivity .

[0011] Another known approach is to use a camera with spectral filters placed over a sensor as it is the case of a color camera, but this approach is limited to the use of a limited number of spectral filters only and with the use of multiple such cameras with varying filters and overlaying the resultant images a hyper-spectral image can be generated .

[0012] Alternatively, a spectral filter wheel can be placed in front of a camera to capture multiple spectral images that can then be overlaid to generate a hyperspectral image . This approach requires use of multiple camera systems or filter wheels that are not compact and add cost and complexity to the system while also overlaying images captured at di f ferent timeframes in some geometries .

[0013] The US 11 , 134 , 848 B2 discloses a mobile hyperspectral camera system . The mobile hyperspectral camera system comprises a mobile host device comprising a processor and a display . A plurality of cameras is coupled to the processor, configured to capture images in distinct spectral bands . A hyperspectral flash array is provided which is coupled to the processor and configured to provide illumination to the distinct spectral bands .

[0014] The DE 10 2019 008 472 Al describes a multi-lens camera system with a lens matrix with a plurality of single lenses .

[0015] SUMMARY

[0016] The obj ect of the invention is therefore to provide a, especially miniaturi zed, multi- zone hyperspectral sensing solution at an economical cost of manufacturing without the use of di f fuser and complex transverse multi-lens , multiaperture optics . Another obj ect is to provide a corresponding method .

[0017] The obj ect is solved according to the invention by multi- zone hyperspectral sensor with the features of claim 1 .

[0018] To this end, the multi- zone hyperspectral sensor comprises a processor and a detector array of detectors coupled to the processor and configured to detect light in distinct spectral bands , and comprising a telecentric optical system in front of the detector array, whereby the processor is configured to integrate light from these zones , and whereby the optical system is configured to deliver a defocussed image on said detector array .

[0019] Preferred embodiments are subj ect of the dependent claims .

[0020] The invention is based on the consideration that in multi- aperture-optics the use of a di f fuser is impractical due to the complex configuration and the installation space needed .

[0021] Applicant has found that a defocussed telecentric imaging system does not need the complex multi-aperture optics .

[0022] The core of the invention therefore is the use of defocus in a telecentric imaging system . A telecentric imaging system is used to separate out light coming in at di f ferent angles into a separated image on the detector plane with the position on the detector plane being a function of the angle of light entering the system . The optical system is designed to ensure that the angle of light incident on the detector plane is quite narrow for a given range of angle of incidence on the aperture of the imaging system . Use of defocus then spreads the narrow angular light rays on to a spot containing a multiple of spectrally selective detectors . This methodology ensures detector response independence to the angle of light incident on the system .

[0023] Preferably the optical system comprises two lenses arranged in spatial sequence . The preferred embodiments show two lenses . One or more lenses and combination of such lenses and combination of such lenses mounted as one lens are also possible .

[0024] The lenses of the optical system are preferably configured with low profile , telecentric, strong zone overlap, uni form spot . In a di f ferent preferred embodiment , the lenses are configured with high profile , telecentric, minimum zone overlap, non-uni form spots .

[0025] In a preferred embodiment , the respective detector of the detector array is built as a photodiode . Preferably, all detectors of the detector array are built as photodiodes .

[0026] In another preferred embodiment , the respective detector of the detector array is built as a camera . Preferably, all detectors of the detector array are built as cameras .

[0027] The detectors of the detector array are preferably arranged in a rectangular, especially square , array .

[0028] The detectors of the detector array advantageously are arranged as a grid of zones , whereby in each zone each detector has a spectral sensitivity di f ferent from all other detectors in this zone . This means that , the spectral sensitivity of all detectors in one zone is pairwise di f ferent . These zones can be called physical zones as they are based on spectral properties of the detectors in each zone , namely that in each zone , a plurality of detectors are provided with respective sensitivity di f ferent from all other detectors in this zone .

[0029] In a preferred embodiment , each zone comprises 16 detectors .

[0030] Advantageously, the detector array comprises 16 zones .

[0031] In a further preferred embodiment , the detectors are arranged in an array of hexagonal zones .

[0032] Advantageously, each zone comprises between 10 and 25 , especially 19 detectors .

[0033] In case of a rotationally symmetric system, higher measurement accuracy can be achieved by using a circular unit detector or a polygon shaped detector that better approximates a circle than a square / rectangular detector . This can then be extended to circular or polygon shaped zones that better map a rotationally symmetric system .

[0034] In a preferred embodiment , from the detectors of the detector array a plurality of virtual zones is built , and whereby the processor integrates light from said virtual zones .

[0035] In this way, an ultra-compact multi- zone hyperspectral sensing solution is provided at an economical cost of manufacturing while expanding beyond the limitations of physical angular zone speci fications .

[0036] An approach according to the prior art for multi- zone spectral sensing is to create optical copies of the scene using a complex multi-aperture imaging system . The proposed approach according to the invention using defocussed telecentric imaging system does not need complex multi- aperture optics . But approaches are limited to the actual physical angular resolution af forded by the number of optical copies or number of zones .

[0037] With the use of virtual zones in the defocussed telecentric imaging system, the concept of virtual zones increases the angular resolution of measurement multi- fold over results achievable by simple physical angular zones .

[0038] In the concept of virtual zone in each permutation of a shi fted collection of spectral detector channels can be treated as a zone in itsel f and all overlaps of such virtual zones can be used to signi ficantly narrow down the incident angular beam extent for spectral measurement . This requires combining the simulated optical angular response characteristic of each spectrally similar channel to generate an angular spectral map .

[0039] The captured scene is divided into multiple angular zones that are then imaged onto the detector pixels with all spectral channels being covered within a single angular image zone . As such, optical copies of the scene are not created . Instead, the scene is divided into parts each of which forms an independent spectral analysis building block .

[0040] Additionally, as one example a single aperture two element lens system can be used which is common for the entire detector array that no multi aperture optical elements are used for obj ect scene duplication as opposed to what is observed in prior art .

[0041] The simulated optical angular response characteristics of each spectrally similar channel are advantageously combined to generate an angular spectral map .

[0042] Preferably, at least two of the virtual zones overlap . Overlapping is needed to have the same light information on di f ferent detectors . In a preferred embodiment , all virtual zones overlap .

[0043] The obj ect is also solved by a method for obtaining hyperspectral information of light hitting a multispectral detector array, whereby the light is defocussed by an optical system before hitting the detector array .

[0044] The advantages of the invention are especially as follows . Ultra-wide-angle , multi- zone , hyperspectral measurements are reali zed in a miniature form factor, space saving through removal of di f fuser . Cost of manufacturing is signi ficantly lower than competing alternatives . A multi- fold higher angular resolution compared to a physical angular zoning approach based hyperspectral sensing can be achieved .

[0045] The angular spectral map obtained has a multi- fold higher angular resolution than what is achievable by simply concentrating on the spectral measurements of each physical angular zone .

[0046] The invention can be used in light sensors , especially for use in smartphones , digital still cameras , surveillance cameras , computers , and augmented reality and machine vision devices . It can be used for applications such as camera / display white balance , hyperspectral sensing, multiple-source identi fication, and spectral reconstruction .

[0047] The use of virtual zones allows to further increase the angular resolution compared to systems which only employ physical zones . A multi zone sensor capable of angular resolution greater than the limitation imposed by individual zone si zes is reali zed . An ideal channel distribution would be repeating channel positioning azimuthally which can be approximated by using varying sensor geometries .

[0048] BRIEF DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0049] A preferred embodiment of the invention is described in connection with a drawing . In this drawing schematically : FIG. 1 shows a multi-zone hyperspectral sensor in a preferred embodiment;

[0050] FIG. 2 shows an optical sensor with optical image zones;

[0051] FIG. 3 shows a multi-zone hyperspectral sensor in a preferred embodiment;

[0052] FIG. 4 shows a multi-zone hyperspectral sensor in a preferred embodiment;

[0053] FIG. 5 shows a diagram showing angular response;

[0054] FIG. 6 shows a detector array of multi-zone hyperspectral sensor in a first preferred embodiment;

[0055] FIG. 7 shows a detector array of a multi-zone hyperspectral sensor in a second preferred embodiment;

[0056] FIG. 8 shows a diagram showing the fraction of rays detected per field angle vs channel position;

[0057] FIG. 9 shows a diagram showing the normalized probability of ray detection versus angle of incidence at different positions of virtual zone centres;

[0058] FIG. 10 shows a detector array with virtual zones, and

[0059] FIG. 11 shows a schematic diagram of a multi-zone hyperspectral sensor.

[0060] Identical parts are labelled in all figures with the same reference numerals.

[0061] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0062] In FIG. 1, a multi-zone hyperspectral sensor 2 is shown which comprises a telecentric optical system 6, a detector array 10 and a processor 14 electronically coupled to the detector array 10 .

[0063] The detector array 10 coupled to the processor 14 is configured to detect light in distinct spectral bands . The detector array 10 to this end comprises a plurality of spectrally selective detectors 56 built as photodiodes , see FIG . 2 .

[0064] A telecentric optical system 6 or imaging system is used to separate out light coming in at di f ferent angles into a separated image zone on the detector plane with the position on the detector plane being a function of the angle of light entering the system . The optical system 6 is designed to ensure that the angle of light incident on the detector plane is quite narrow for a given range of angle of incidence on the aperture 12 of the imaging system .

[0065] As can be seen in FIG . 1 a first group 20 of light rays impinges on the optical system in a direction perpendicular to the detector array 10 and parallel to an optical axis 36 , i . e . , under an angle of 0 ° . The light is scattered onto a certain region of the detector array 10 .

[0066] A second group 26 of light rays impinges on the optical system 6 with an angle of 26 ° with respect to the optical axis 36 a third group 32 of light rays impinges on the optical system 6 with an angle of 60 ° with respect to the optical axis 36 .

[0067] The optical system 6 is configured to deliver a defocussed image on the detector array 10 . Use of defocus spreads the narrow angular light rays on to a spot containing a multiple of spectrally selective detectors . This methodology ensures detector response independence to the angle of light incident on the system .

[0068] The detector array 10 comprises a plurality of optical zones of which three optical zones 40 , 46 , 52 are indicated in FIG. 1. A strong defocus as shown in this FIG. leads to covering the entire zone. Angular overlap is provided for a wide range of FOV.

[0069] In FIG. 2, a detector array 10 in a preferred embodiment is shown. The detector array 10 comprises a plurality of spectrally selective detectors 56 of which only one is labelled in FIG. 2. The detectors 56 are arranged in square detector regions 60, whereby each detector region 60 comprises 9 detectors 56. Also indicated in FIG. 2 by circles are optical zones of which two optical zones 66, 72 are labelled. The two optical zones 66, 72 overlap in an overlap region 78. The respective detector 56 is built as photodiode 58.

[0070] A multi-zone hyperspectral sensor 2 with field of view (FOV) overlap in a preferred embodiment is shown in FIG. 3. The optical stack of the optical system 6, i.e., its components, comprises of a first lens 80 and a second lens 86. Light to be detected by detector array 10 first passes first lens 80 and then second lens 86. Total optical stack length L is ca. 2.8 mm. The entrance pupil diameter is 400 microns, the f-number is 2.25.

[0071] In this preferred embodiment, a strong defocus covering the entire zone is introduced. Introduction of a strong defocus covering the entire zone overlap possible only for higher angles of incidence.

[0072] A multi-zone hyperspectral sensor 2 with minimal overlap in a preferred embodiment is shown in FIG. 4. The optical stack of the optical system 6, i.e., its components, comprises of a first lens 80 and a second lens 86. Light to be detected by detector array 10 first passes first lens 80 and then second lens 86. The total optical stack length L is ca. 3.5 mm. Th entrance pupil diameter is 400 microns, the f-number is 2.4.

[0073] This embodiment leads to a non-uniform spot size across field angles of incidence. A strong defocus covering the entire zone is introduced in this embodiment. The overlap is reduced to minimal level.

[0074] In FIG. 5, the angular response of the embodiment of FIG. 4 is shown. On an x-axis 100, the image height in millimeters, on y-axis 104 the incident angle in degrees is plotted. As can be seen in this figure, the angle of incidence on the f ilters / sensors varies up to 14 degrees across the field.

[0075] In FIG. 6, a detector array 10 in a preferred embodiment is shown which is built as a square photodiode array 110. Each detector in detector array 10 is built as a photodiode. The photodiode array 110 comprises a plurality of identical zones 114 of which only one zone 114 is labelled in FIG. 6.

[0076] Each zone 114 is built as an identical 4x4 square grid of photodiodes with four rows and four columns, whereby each photodiode in a zone 114 is sensitive to a different radiation wavelength compared to all other photodiodes in this zone 114. In this way, each zone 114 comprises 16 channels. In the present example, a 4x4 grid of zones 114 is realized, i.e., in total 16 zones are provided. The processor 14 is configured to integrate light from zone 114, i.e. summing up the amount of light over a specific exposure time.

[0077] A detector array 10 in a further preferred embodiment is shown in FIG. 7 which is built as photodiode array 110. Each detector in detector 10 is built as a photodiode. The photodiode array 110 comprises a plurality of identical zones 114 of which only two zones 114 are labelled in FIG. 7.

[0078] Each zone 114 is built as a hexagonal array of 19 photodiodes. Each photodiode in a zone 114 is sensitive to a different radiation wavelength compared to all other photodiodes in this zone 114. In this way, each zone 114 comprises 19 channels. In the present example, in total 13 zones are provided. The hexagonal optical zones 114 are approximations to circular diodes and consistent with optical zones 66, 73, see FIG. 2. In FIG. 8, in a diagram on an x-axis 100 a field angle of incidence (position of an individual detector) and on an y- axis 104 a fraction is plotted for different curves 120, 124, 128, whereby cure 132 represents the multiplication of curve 120 and curve 124.

[0079] A first curve 120 represents the value 0.98, a second curve 124 represents the value 0.8, a third curve 128 represents the value 0.65, and a fourth curve 132 represents the product 0, 95x0.8.

[0080] The FIG. 8 demonstrates how depending on channel density and positioning, virtual zones can be created. Channel measurements can be adjusted for measurement probability at a certain angle of incidence. Data from multiple channels of the same type can be used to further narrow down the angle of incidence estimation. As such, it is possible to go much beyond the restriction on angular measurement resolution placed by number of zones. The ideal shape of the sensor is circular with azimuthal repetition of channels at higher radius values.

[0081] In FIG. 9, in a diagram on an x-axis 100 the angle of incidence is plotted, and on a y-axis 104, the probability is plotted. The graph shows the normalized probability of ray detection versus the angle of incidence (AOI)at different positions of zone centers. The various curves shown correspond to different zone centers. FIG. 9 thereby shows a total of 13 virtual zones along the sensor diagonal for a sensor with 4 x 4 physical zones. Two of these virtual zones are indicated in FIG. 10.

[0082] The square detector array 10 according to FIG. 6 is shown in FIG. 10 with the difference, that two virtual zones 140, 144 are indicated by squares. The virtual zone 140 corresponds to the zone 114 indicated in FIG. 6. The virtual zone 144 overlaps with the virtual zone 140. In FIG. 11, a multi sensor 2 is schematically shown. The sensor 2 comprises an optical system, a detector array and a processor .

[0083] LIST OF REFERENCE SIGNS

[0084] 2 multi- zone hyperspectral sensor

[0085] 6 optical system

[0086] 10 detector array

[0087] 12 aperture

[0088] 14 processor

[0089] 20 first group

[0090] 26 second group

[0091] 32 third group

[0092] 36 optical axis

[0093] 40 optical zone

[0094] 46 optical zone

[0095] 52 optical zone

[0096] 56 detector

[0097] 58 photodiode

[0098] 60 detector regions

[0099] 66 optical zone

[0100] 72 optical zone

[0101] 78 overlap region

[0102] 80 lens

[0103] 86 lens

[0104] 100 x-axis

[0105] 104 y-axis

[0106] 110 photodiode array

[0107] 114 zone

[0108] 120 curve

[0109] 124 curve

[0110] 128 curve

[0111] 132 curve

[0112] 140 virtual zone

[0113] 144 virtual zone

[0114] L optical stack length

Claims

CLAIMS1. Multi-zone hyperspectral sensor (2) , comprising a processor (14) and a detector array (10) with zones (114, 140, 144) of detectors (56) coupled to the processor (14) and configured to detect light in distinct spectral bands, and comprising a telecentric optical system (6) in front of said detector array (10) , whereby said processor (14) is configured to integrate light from said zones (114,140, 114) , characterized in that said optical system (6) is configured to deliver a defocussed image on said detector array (10) .

2. Multi-zone hyperspectral sensor (2) according to claim 1, whereby said optical system (6) comprises two lenses (80, 86) arranged in spatial sequence.

3. Multi-zone hyperspectral sensor (2) according to claim 2, whereby said lenses (80, 86) of the optical system (6) are configured with low profile, telecentric, strong zone overlap, uniform spot.

4. Multi-zone hyperspectral sensor (2) according to claim 2, whereby said lenses (80, 86) of the optical system (6) are configured with high profile, telecentric, minimum overlap, non-uniform spots.

5. Multi-zone hyperspectral sensor (2) according to one of the claims 1 to 4, whereby said respective detector (56) of said detector array (10) is built as a photodiode (58) .

6. Multi-zone hyperspectral sensor according to , whereby said respective detector of (56) said detector array (10) is built as a camera.

7. Multi-zone hyperspectral sensor (2) according to one of claims 1 to 6, whereby said detectors (56) of said detector array (10) are arranged in a rectangular, especially square, array .

8. Multi-zone hyperspectral sensor (2) according to claim 7, whereby said detectors (56) of said detector array (10) are arranged as a grid of zones (114) , whereby in each zone (114) each detector (56) has a spectral sensitivity different from all other detectors (56) in this zone (114) .

9. Multi-zone hyperspectral sensor (2) according to claim 8, whereby each zone (114) comprises a plurality, especially 16 detectors, (56) .

10. Multi-zone hyperspectral sensor (2) according to claim 9, whereby said detector array (10) comprises a plurality, especially 16, zones (114) .

11. Multi-zone hyperspectral sensor (2) according to claim 8, whereby said detectors are arranged in an array of hexagonal zones ( 114 ) .

12. Multi-zone hyperspectral sensor (2) according to claim 11, whereby each zone (114) comprises between 10 and 25, especially 19, detectors.

13. Multi-zone hyperspectral sensor (2) according to one of the previous claims, whereby from said detectors (56) of said detector array (10) a plurality of virtual zones (140, 144) is built, and whereby said processor (14) integrates light from said virtual zones (140, 144) .

14. Multi-zone hyperspectral sensor (2) according to claim 13, whereby the simulated optical angular response characteristics of each spectrally similar channel are combined to generate an angular spectral map.

15. Multi-zone hyperspectral sensor (2) according to claim 13 or 14, whereby at least two of said virtual zones (140, 144) overlap .

16. Multi-zone hyperspectral sensor (2) according to claim15, whereby all virtual zones (140,144) overlap.

17. Method for obtaining hyperspectral information of light hitting a multispectral detector array (10) , characterized in that the light is defocussed by an optical system (6) before hitting the detector array (10) .

Images