Multi-region hyperspectral sensor

Abstract

A multi-region hyperspectral sensor (2) comprising a processor (14) and a detector array (10) of regions (114, 140, 144) having detectors (56), the detector array (10) coupled to the processor (14) and configured to detect light in different spectral bands, and the multi-region hyperspectral sensor (2) comprising a telecentric optical system (6) in front of the detector array (10), wherein the processor (14) is configured to integrate light from the regions (114, 140, 114), wherein the optical system (6) is configured to deliver a defocused image onto the detector array (10).

Description

Technical Field

[0001] This invention relates to multi-region spectral sensors. More particularly, this invention relates to multi-region spectral sensors having detector arrays and optical systems. Background Technology

[0002] In many applications, it is advantageous to obtain corresponding hyperspectral images with spectral resolution, in addition to images with spatial resolution of the scene. The spectrum of these images can include visible light, as well as light frequencies in the IR or UV range. Such spectral information can help detect features that are undetectable in conventional images. In other camera applications, hyperspectral images can be used in automatic white balance algorithms.

[0003] In optical systems, both single-area and multi-area sensors can be used. A typical single-area sensor detects irradiance [lx, ]=> Illuminance[ The diffuser above the sensor will expand the field of view (FOV, which can be up to 180). All light within the sensor is mixed into a near-Lambertian distribution or a constant-angle distribution oriented towards the photodiode array. The diffused light reaching the sensor is independent of the graininess of the image scene. A single-area sensor detects the sum of light sources in this way.

[0004] Multi-area sensor detects luminous intensity [cd, ]=>Radiation flux[ The light is integrated over regions (similar to a camera device). The detected light depends on the granularity of the image scene, and can be as high as 120. But it can also be greater than 120. The field of view (FOV) and / or the directionality of each area. Multi-area sensors can detect the hues of different ambient and artificial light sources in different areas.

[0005] A common approach to ultra-wide-angle spectral sensing is to use diffusers to achieve a uniform detector response regardless of the angle of the incident light. However, using diffusers is impractical in multi-region spectral sensing because it would significantly increase device size and complexity. This would require each angular region to be optically shielded from each other to prevent scattered light from one angular region from interacting with the diffuser and then striking the detector space corresponding to the other angular region.

[0006] Another common method for obtaining multispectral images of a scene is to create separate optical copies of the scene and then image them onto different sensor / camera elements, each with a common wavelength selectivity.

[0007] Another known method is to use a camera device with spectral filters placed on the sensor, such as in the case of a color camera device. However, this method is limited to using only a limited number of spectral filters, and hyperspectral images can be generated by using multiple such camera devices with different filters and superimposing the resulting images.

[0008] Alternatively, a spectral filter wheel can be placed in front of the camera to capture multiple spectral images, which can then be superimposed to generate a hyperspectral image. This method requires multiple camera systems or filter wheels, which are not compact and increase the cost and complexity of the system, while also superimposing images captured at different time frames in some geometry.

[0009] US 11,134,848 B2 discloses a mobile hyperspectral camera system. This mobile hyperspectral camera system includes a mobile host device comprising a processor and a display. Multiple cameras are coupled to the processor and configured to capture images in different spectral bands. A hyperspectral flash array is provided, coupled to the processor and configured to provide illumination to the different spectral bands.

[0010] DE 10 2019 008 472 A1 describes a multi-lens camera system with a lens matrix having multiple individual lenses. Summary of the Invention

[0011] Therefore, the object of this invention is to provide a particularly miniaturized multi-region hyperspectral sensing solution at an economical manufacturing cost, without using diffusers and complex lateral multi-lens, multi-aperture optics. Another object is to provide a corresponding method.

[0012] According to the present invention, this objective is achieved by a multi-region hyperspectral sensor having the features described in claim 1.

[0013] To this end, a multi-region hyperspectral sensor includes a processor and a detector array of detectors, the detector array being coupled to the processor and configured to detect light in different spectral bands, and the multi-region hyperspectral sensor includes a telecentric optical system in front of the detector array, wherein the processor is configured to integrate light from these regions, and wherein the optical system is configured to transmit a defocused image to the detector array.

[0014] The preferred embodiments are the subject of the dependent claims.

[0015] The present invention is based on the following consideration: In multi-aperture optical devices, the use of diffusers is impractical due to the complex configuration and required installation space.

[0016] The applicant has discovered that defocused telecentric imaging systems do not require complex multi-aperture optics.

[0017] Therefore, the core of this invention is the use of defocus in a telecentric imaging system. A telecentric imaging system is used to separate light entering at different angles into separate images on a detector plane, where the position on the detector plane is a function of the angle at which the light enters the system. The optical system is designed to ensure that, for a given range of angles incident on the aperture of the imaging system, the angle at which light is incident on the detector plane is relatively narrow. The use of defocus then diffuses the narrow-angle light rays onto a spot containing multiple spectrally selective detectors. This method ensures that the detector response is independent of the angle at which the light is incident on the system.

[0018] Preferably, the optical system comprises two lenses arranged in spatial order. A preferred embodiment shows two lenses. One or more lenses, combinations of such lenses, and combinations of such lenses mounted as a single lens are also possible.

[0019] The lens of the optical system is preferably configured with a low profile, telecentricity, strong area overlap, and uniform spot. In different preferred embodiments, the lens is configured with a high profile, telecentricity, minimal area overlap, and non-uniform spot.

[0020] In a preferred embodiment, the respective detectors of the detector array are constructed as photodiodes. Preferably, all detectors of the detector array are constructed as photodiodes.

[0021] In another preferred embodiment, the respective detectors of the detector array are configured as camera devices. Preferably, all detectors of the detector array are configured as camera devices.

[0022] The detectors in the detector array are preferably arranged in a rectangular array, especially a square array.

[0023] The detectors of the detector array are advantageously arranged as a grid of regions, wherein in each region, each detector has a spectral sensitivity different from all other detectors in that region. This means that the spectral sensitivities of all detectors in a region are paired and different. These regions can be referred to as physical regions because they are based on the spectral characteristics of the detectors in each region, that is, in each region, multiple detectors are set with corresponding sensitivities different from all other detectors in that region.

[0024] In a preferred embodiment, each region includes 16 detectors.

[0025] Advantageously, the detector array comprises 16 regions.

[0026] In another preferred embodiment, the detectors are arranged as an array of hexagonal regions.

[0027] Advantageously, each region includes between 10 and 25 detectors, especially 19 detectors.

[0028] In the case of rotationally symmetric systems, higher measurement accuracy can be achieved by using circular unit detectors or detectors with polygonal shapes that better approximate a circle than square / rectangular detectors. This can then be extended to regions that better map the circular or polygonal shapes of the rotationally symmetric system.

[0029] In a preferred embodiment, a plurality of virtual regions are constructed from the detectors of the detector array, and wherein the processor integrates the light from the virtual regions.

[0030] In this way, an ultra-compact multi-region hyperspectral sensing solution is provided at an economical manufacturing cost, while extending beyond the limitations of physical angular region specifications.

[0031] Existing methods for multi-region spectral sensing utilize complex multi-aperture imaging systems to create optical copies of a scene. The method proposed according to the invention, using a defocused telecentric imaging system, does not require complex multi-aperture optics. However, the method is limited by the actual physical angular resolution provided by the number of optical copies or regions.

[0032] In defocused telecentric imaging systems, the concept of a virtual region increases the angular resolution of measurements by several times compared to results achievable with simple physical angular regions.

[0033] In the concept of virtual regions, each permutation of the shifted set of spectral detector channels can be considered a region in itself, and all overlaps of such virtual regions can be used to significantly reduce the incident angle beam range for spectral measurements. This requires combining simulated optical angular response characteristics of each spectrally similar channel to generate an angular spectrogram.

[0034] The captured scene is divided into multiple angular regions, which are then imaged onto detector pixels, with all spectral channels covered within a single angular image region. Therefore, no optical copy of the scene is created. Instead, the scene is divided into sections, each forming an independent spectral analysis building block.

[0035] Additionally, as an example, a single-aperture dual-element lens system shared by the entire detector array can be used, which is contrary to what has been observed in the prior art, where multi-aperture optics are not used for object scene replication.

[0036] Advantageously, the simulated optical angle response characteristics of each spectrally similar channel are combined to generate an angular spectrum.

[0037] Preferably, at least two of the virtual regions overlap. This overlap is necessary to ensure identical optical information on different detectors.

[0038] In a preferred embodiment, all virtual regions overlap.

[0039] This objective is also achieved through a method for obtaining hyperspectral information of light directed toward a multispectral detector array, wherein the light is defocused by an optical system before being directed toward the detector array.

[0040] The advantages of this invention are particularly as follows: Ultra-wide-angle, multi-region, hyperspectral measurements are achieved with a miniature shape factor, saving space by removing the diffuser. Manufacturing costs are significantly lower than competing alternatives. Angular resolution can be achieved many times higher than that of hyperspectral sensing based on physical angle partitioning methods.

[0041] The obtained angular spectra have a resolution many times higher than that achievable by simply focusing spectral measurements on each physical angular region.

[0042] This invention can be used in optical sensors, particularly in smartphones, digital still cameras, surveillance cameras, computers, and augmented reality and machine vision devices. It can be used in applications such as camera / display white balance, hyperspectral sensing, multi-source identification, and spectral reconstruction.

[0043] Compared to systems that use only physical regions, the use of virtual regions allows for a further increase in angular resolution. A multi-region sensor was implemented that can achieve a greater angular resolution than is limited by the size of each region. The ideal channel distribution would be repeating channel localization in the azimuth, which can be approximated by using different sensor geometries. Attached Figure Description

[0044] Preferred embodiments of the present invention are described in conjunction with the accompanying drawings. In the drawings, schematically:

[0045] Figure 1 A multi-region hyperspectral sensor is shown in a preferred embodiment;

[0046] Figure 2 An optical sensor with an optical image area is shown;

[0047] Figure 3 A multi-region hyperspectral sensor is shown in a preferred embodiment;

[0048] Figure 4 A multi-region hyperspectral sensor is shown in a preferred embodiment;

[0049] Figure 5A graph showing the angle response is provided.

[0050] Figure 6 The detector array of the multi-region hyperspectral sensor in the first preferred embodiment is shown;

[0051] Figure 7 The detector array of the multi-region hyperspectral sensor in the second preferred embodiment is shown;

[0052] Figure 8 A graph showing the fraction of light detected at each field angle versus the channel position is presented;

[0053] Figure 9 The graph shows the normalized probability of ray detection versus incident angle at different virtual region center locations;

[0054] Figure 10 An array of detectors with virtual regions is shown, and

[0055] Figure 11 A schematic diagram of a multi-region hyperspectral sensor is shown.

[0056] In all the accompanying drawings, the same parts are labeled with the same reference numerals. Detailed Implementation

[0057] exist Figure 1 The image shows a multi-region hyperspectral sensor 2, which includes a telecentric optical system 6, a detector array 10, and a processor 14 electrically coupled to the detector array 10.

[0058] The detector array 10, coupled to the processor 14, is configured to detect light in different spectral bands. For this purpose, the detector array 10 includes multiple spectrally selective detectors 56 constructed as photodiodes, see [link to relevant documentation]. Figure 2 .

[0059] The telecentric optical system 6, or imaging system, is used to separate light entering at different angles into separate image regions on the detector plane, where the position on the detector plane is a function of the angle at which the light enters the system. The optical system 6 is designed to ensure that, for a given range of angles incident on the aperture 12 of the imaging system, the angle at which light is incident on the detector plane is relatively narrow.

[0060] like Figure 1 As can be seen, the first group 20 of light rays is in a direction perpendicular to the detector array 10 and parallel to the optical axis 36, that is, at 0... The light is incident on the optical system at an angle. The light is scattered onto a specific area of ​​the detector array 10.

[0061] The second group of 26 rays is 26 degrees relative to the optical axis 36. The rays from the third group 32 are incident on the optical system 6 at an angle of 60 degrees relative to the optical axis 36. The light is incident on the optical system 6 at an angle.

[0062] Optical system 6 is configured to transmit a defocused image to detector array 10. The use of defocus diffuses narrow-angle light rays onto a spot containing multiple spectrally selective detectors. This method ensures that the detector response is independent of the angle at which light is incident on the system.

[0063] The detector array 10 includes multiple optical regions, in Figure 1 The figure indicates three optical zones: 40, 46, and 52. The strong defocusing shown in the figure allows coverage of the entire area, providing angular overlap for a wide field of view (FOV).

[0064] exist Figure 2 The image shows a detector array 10 in a preferred embodiment. The detector array 10 includes a plurality of spectrally selective detectors 56, wherein only one of the spectrally selective detectors 56 is in... Figure 2 The detectors are marked. Detectors 56 are arranged in square detector areas 60, where each detector area 60 includes nine detectors 56. Figure 2 The optical regions are also indicated by circles, with two optical regions 66 and 72 marked. The two optical regions 66 and 72 overlap in the overlapping region 78. The corresponding detector 56 is constructed as a photodiode 58.

[0065] Figure 3 The image shows a preferred embodiment of a multi-region hyperspectral sensor 2 with overlapping fields of view (FOV). The optical system 6 comprises an optical stack (i.e., its components) including a first lens 80 and a second lens 86. Light to be detected by the detector array 10 first passes through the first lens 80 and then through the second lens 86. The total optical stack length L is approximately 2.8 mm. The entrance pupil diameter is 400 micrometers, and the f-number is 2.25.

[0066] In this preferred embodiment, strong defocus covering the entire area is introduced. The introduction of strong defocus covering the entire area may only overlap at larger angles of incidence.

[0067] Figure 4 The image shows a preferred embodiment of a multi-region hyperspectral sensor 2 with minimal overlap. The optical system 6 comprises an optical stack (i.e., its components) including a first lens 80 and a second lens 86. Light to be detected by the detector array 10 first passes through the first lens 80 and then through the second lens 86. The total optical stack length L is approximately 3.5 mm. The entrance pupil diameter is 400 micrometers, and the f-number is 2.4.

[0068] This implementation produces a non-uniform spot size across the field of incidence. Strong defocusing covering the entire area is introduced in this implementation. Overlap is reduced to a minimum.

[0069] exist Figure 5 In, it is shown Figure 4 The angular response of the implementation method is shown. The image height in millimeters is plotted on the x-axis 100, and the angle of incidence in degrees is plotted on the y-axis 104. As can be seen in this figure, the angle of incidence on the filter / sensor varies by up to 14 degrees throughout the field.

[0070] exist Figure 6 The image shows a detector array 10 in a preferred embodiment, which is constructed as a square photodiode array 110. Each detector in the detector array 10 is constructed as a photodiode. The photodiode array 110 includes a plurality of identical regions 114, wherein only one region 114 is in Figure 6 It is marked in the middle.

[0071] Each region 114 is constructed as a photodiode with four rows and four columns of identical 4 A 4-square grid is used, where each photodiode in region 114 is sensitive to a different radiation wavelength compared to all other photodiodes in that region 114. In this way, each region 114 includes 16 channels. In this example, 4 square grids are implemented in region 114. A 4-grid design provides a total of 16 regions. The processor 14 is configured to integrate the light from the regions 114, that is, to sum the amount of light over a given exposure time.

[0072] Figure 7 The image shows a detector array 10 in another preferred embodiment, which is constructed as a photodiode array 110. Each detector in detector 10 is constructed as a photodiode. The photodiode array 110 includes a plurality of identical regions 114, wherein only two regions 114 are located in... Figure 7 It is marked in the middle.

[0073] Each region 114 is constructed as a hexagonal array with 19 photodiodes. Each photodiode in region 114 is sensitive to a different wavelength of radiation compared to all other photodiodes in that region 114. In this way, each region 114 includes 19 channels. In this example, a total of 13 regions are provided. The hexagonal optical region 114 is an approximation of a circular diode and corresponds to optical regions 66, 73, see [link to relevant documentation]. Figure 2 .

[0074] exist Figure 8In the figure, the field incident angle (position of a single detector) is plotted on the x-axis 100 for different curves 120, 124, and 128, and the fraction is plotted on the y-axis 104, where curve 132 represents the multiplication of curves 120 and 124.

[0075] The first curve (120) represents a value of 0.98, the second curve (124) represents a value of 0.8, the third curve (128) represents a value of 0.65, and the fourth curve (132) represents a product of 0.95. 0.8.

[0076] Figure 8 This demonstrates how virtual regions can be created based on channel density and positioning. Channel measurements can be adjusted for the measurement probability at a given incident angle. Data from multiple channels of the same type can be used to further narrow down the range of incident angle estimates. Therefore, the limitations imposed on angle measurement resolution by the number of regions can be significantly exceeded. The ideal sensor shape is circular, with azimuthally repeating channels at larger radius values.

[0077] exist Figure 9 In the graph, the angle of incidence is plotted on the x-axis at 100, and the probability is plotted on the y-axis at 104. This graph shows the normalized probability of ray detection versus the angle of incidence (AOI) at different region center locations. The various curves shown correspond to different region centers. Therefore, Figure 9 It shows that for a 4 The sensor has 4 physical regions and a total of 13 virtual regions along its diagonal. Two of these virtual regions are located within... Figure 10 As indicated in the document.

[0078] according to Figure 6 Square detector array 10 in Figure 10 As shown, the difference lies in the fact that two virtual regions 140 and 144 are indicated by squares. Virtual region 140 and... Figure 6 This corresponds to area 114 indicated in the diagram. Virtual area 144 overlaps with virtual area 140.

[0079] exist Figure 11 The diagram schematically illustrates a multi-sensor 2. Sensor 2 includes an optical system, a detector array, and a processor.

[0080] List of reference numerals

[0081] 2 Multi-region hyperspectral sensors

[0082] 6 Optical System

[0083] 10 detector array

[0084] 12 aperture

[0085] 14 processors

[0086] Group 1 of 20

[0087] Group 26

[0088] Group 32

[0089] 36 optical axes

[0090] 40 optical zones

[0091] 46 optical zones

[0092] 52 optical areas

[0093] 56 detectors

[0094] 58 photodiode

[0095] 60 detector area

[0096] 66 optical zones

[0097] 72 optical areas

[0098] 78 overlapping areas

[0099] 80 lens

[0100] 86 lens

[0101] 100x axis

[0102] 104y axis

[0103] 110 photodiode array

[0104] Area 114

[0105] 120 curve

[0106] 124 curve

[0107] 128 curve

[0108] 132 curve

[0109] 140 Virtual Areas

[0110] 144 Virtual Areas

[0111] L Optical stacking length

Claims

1. A multi-region hyperspectral sensor (2) comprising a processor (14) and a detector array (10) having detectors (56) in regions (114, 140, 144), the detector array (10) being coupled to the processor (14) and configured to detect light in different spectral bands, and the multi-region hyperspectral sensor (2) comprising a telecentric optical system (6) in front of the detector array (10), wherein, The processor (14) is configured to integrate the light from the regions (114, 140, 114). Its features are, The optical system (6) is configured to transmit the defocused image to the detector array (10).

2. The multi-region hyperspectral sensor (2) according to claim 1, wherein, The optical system (6) includes two lenses (80, 86) arranged in spatial order.

3. The multi-region hyperspectral sensor (2) according to claim 2, wherein, The lenses (80, 86) of the optical system (6) are configured with low profile, telecentric, strong area overlap, and uniform spot.

4. The multi-region hyperspectral sensor (2) according to claim 2, wherein, The lenses (80, 86) of the optical system (6) are configured with high profile, telecentric, minimal overlap, and non-uniform spot.

5. The multi-region hyperspectral sensor (2) according to any one of claims 1 to 4, wherein, The corresponding detectors (56) of the detector array (10) are constructed as photodiodes (58).

6. According to the aforementioned multi-region hyperspectral sensor, wherein, The corresponding detectors (56) of the detector array (10) are constructed as camera devices.

7. The multi-region hyperspectral sensor (2) according to any one of claims 1 to 6, wherein, The detectors (56) of the detector array (10) are arranged in a rectangular array, especially a square array.

8. The multi-region hyperspectral sensor (2) according to claim 7, wherein, The detectors (56) of the detector array (10) are arranged in a grid of regions (114), wherein each detector (56) in each region (114) has a different spectral sensitivity than all other detectors (56) in that region (114).

9. The multi-region hyperspectral sensor (2) according to claim 8, wherein, Each region (114) includes multiple detectors (56), specifically 16 detectors (56).

10. The multi-region hyperspectral sensor (2) according to claim 9, wherein, The detector array (10) includes multiple regions (114), particularly 16 regions (114).

11. The multi-region hyperspectral sensor (2) according to claim 8, wherein, The detectors are arranged in an array of hexagonal regions (114).

12. The multi-region hyperspectral sensor (2) according to claim 11, wherein, Each region (114) includes between 10 and 25 detectors, with a particular 19 detectors.

13. The multi-region hyperspectral sensor (2) according to any one of the preceding claims, wherein, Multiple virtual regions (140, 144) are constructed from the detectors (56) of the detector array (10), and the processor (14) integrates the light from the virtual regions (140, 144).

14. The multi-region hyperspectral sensor (2) according to claim 13, wherein, The simulated optical angle response features of each spectrally similar channel are combined to generate an angle spectrum.

15. The multi-region hyperspectral sensor (2) according to claim 13 or 14, wherein, At least two of the virtual regions (140, 144) overlap.

16. The multi-region hyperspectral sensor (2) according to claim 15, wherein, All virtual regions (140, 144) overlap.

17. A method for obtaining hyperspectral information of light directed toward a multispectral detector array (10), Its features are, The light is defocused by the optical system (6) before it is directed toward the detector array (10).

Images