Device and method for hyperspectral image acquisition

Abstract

The invention relates to a device (1) for hyperspectral image acquisition, comprising at least one image sensor (3) for recording camera images, at least one first pinhole aperture arrangement (7), a lens arrangement (6) and at least one MEMS actuator (2), wherein by means of the at least one MEMS actuator (2), the image sensor (3) and / or a pinhole aperture arrangement (7) can be moved in such a way that spectral scanning, a higher spectral resolution and / or a higher spatial resolution is made possible by the movement of the image sensor (3) and / or of the pinhole aperture arrangement (7).

Description

[0001] 202406849

[0002] 1

[0003] Device and method for hyperspectral image acquisition

[0004] The invention relates to a device and a method for hyperspectral image acquisition.

[0005] For example, systems that use hyperspectral cameras to inspect objects are known from the prior art. These systems can be used, for instance, in industrial applications to inspect goods.

[0006] To capture images, cameras typically include an image sensor that detects incoming light. These sensors usually consist of an array of photosensitive sensor segments or pixels capable of receiving light in the visible, infrared, and / or other regions of the electromagnetic spectrum. To allow the image sensor to be moved relative to other camera components, some camera types use an actuator to couple the sensor to it.

[0007] For example, from US patent 11982616B2, a method for determining the quality state of an agricultural product is known, comprising: receiving light at a light detector, wherein the received light includes reflected, scattered, refracted, and / or deflected light from the agricultural product; transmitting the received light to a spectrometer; generating agricultural product spectral data (AP) of the received light using the spectrometer; comparing the AP spectral data with reference spectral data using a computer electrically connected to the spectrometer to determine whether the agricultural product meets the quality state, wherein the reference spectral data correspond to known quality states of the agricultural product; and generating an output signal using the computer that corresponds to the quality state of the agricultural product.

[0008] 2

[0009] Furthermore, a multi-lens camera system for hyperspectral image acquisition is known from publication DE102019101324B4, comprising a planar image sensor, a position-sensitive spectral filter element, and an imaging system, wherein the imaging system comprises a planar lens matrix with a plurality of individual lenses arranged such that they generate a plurality of raster-arranged first images of a subject in a first area on the image sensor at a first acquisition time, and additionally comprises a cylindrical lens as a further lens, which is arranged in the plane of the lens matrix or separated from the plane of the lens matrix and is designed to generate a second image of the subject on the image sensor in a second area of ​​the image sensor and / or at a second acquisition time, wherein the further lens has a focal line and is positioned such thatthat their second image differs from the first images at least in one area dimension of the image sensor with regard to the size and / or with regard to the image information of the subject present in front of the filter element.

[0010] A disadvantage of these known systems is their high cost and the large size of the sensors they use. Furthermore, these sensors typically have a limited spectral bandwidth and lateral resolution. To obtain an image of the object under investigation, it is usually necessary to move either the object itself or the camera.

[0011] Furthermore, MEMS (Micro Electro-Mechanical Systems) technology is well known from the state of the art.

[0012] It is therefore an object of the present invention to provide a device and a method by which hyperspectral image acquisition is made possible in a cost-effective and precise manner.

[0013] This problem is advantageously solved by the subject matter of independent claims 1 and 7. 202406849

[0014] 3

[0015] Further advantageous embodiments and configurations are the subject of the dependent claims.

[0016] According to the invention, a device for hyperspectral image acquisition is proposed, comprising at least one image sensor for recording camera images, at least one first pinhole arrangement, a lens arrangement and at least one MEMS actuator, wherein the image sensor or the pinhole arrangement can be moved by means of the at least one MEMS actuator in such a way that the movement of the image sensor or the pinhole arrangement enables spectral scanning, a higher spectral resolution and / or a higher spatial resolution.

[0017] The device is a camera which, through the described configuration, enables hyperspectral imaging. By moving the pinhole array, its position in the plane is changed, and an image can be captured at each position, depicting the spectral response of a sample or object. The device is preferably used for inspecting samples or products, for example, to monitor cleanliness, quality, or chemical classification. The use of a MEMS actuator is advantageous because the entire object or sample can be scanned without moving the entire sensor or the object itself.

[0018] In a preferred embodiment, the image sensor has a monochrome color filter or a multi-color filter. Depending on the design of the color filter used, the device is adapted accordingly with the appropriate components. If the incoming light is not spectrally refracted or dispersed beforehand, it is necessary to use a multi-color filter. In other cases, the use of a monochrome color filter is also possible, and a multi-color filter can also be used even if the light is spectrally dispersed.

[0019] 4

[0020] In a preferred embodiment, object classification in the camera images is enabled by means of the movement of the image sensor and / or the pinhole arrangement. For example, the chemical classification of objects or, more generally, object classification would be conceivable.

[0021] In a further preferred embodiment, the image sensor or the pinhole arrangement can be moved in the x-direction and / or y-direction and / or z-direction by means of the MEMS actuator. Depending on the device configuration, hyperspectral image acquisition can be achieved either by the pinhole arrangement on the image sensor or by prior spectral splitting. As already described, the x / y movement of the image sensor or the pinhole arrangement allows an image to be acquired in any position, depicting the spectral response of a sample or object at higher lateral resolution without requiring any movement of the sample or sensor.

[0022] In a particularly preferred embodiment, the device further comprises a spectrometer optic using a prism or an optical diffraction grating. In such an embodiment, it is possible to use an image sensor with a monochrome color filter, since the incident light is spectrally dispersed by the prism or grating of the spectrometer optic.

[0023] In a further preferred embodiment, the device features a hyperchromatic lens or hyperchromat in the optical beam path upstream of the image sensor and an associated pinhole arrangement. The hyperchromat provides the spectral splitting of the light reflected from the object. In contrast to a classical spectrometer, however, the spectral splitting occurs along the optical axis by utilizing longitudinal chromatic aberration. The pinhole is used to focus only a specific wavelength at each Z-position. By moving the image sensor along the z-axis, the light reflected from the object can be spectrally analyzed. If the x / y position of the image sensor and pinhole combination is also adjusted, the spectral analysis can be performed.

[0024] 5

[0025] By varying the intensity, a higher lateral hyperspectral sampling of the object can be achieved.

[0026] Furthermore, a method for hyperspectral image acquisition is proposed according to the invention, comprising the following steps:

[0027] - Providing a device according to an embodiment of the invention; - Providing an object to be examined;

[0028] - Scanning the object to be examined, wherein during scanning the image sensor and / or the pinhole arrangement of the device is moved by means of a MEMS actuator in such a way that hyperspectral scanning of the entire object to be examined is enabled.

[0029] In a further advantageous embodiment of the method, the image sensor and / or the pinhole array is moved to a multitude of individual positions. The image sensor and / or the pinhole array can be moved in the x or y direction. The movement can, for example, be performed continuously, pixel by pixel. It would also be conceivable, particularly when moving in the x and y directions, to perform the movement row by row and / or column by column. This is advantageous because it enables improved lateral scanning of the object / sample.

[0030] Furthermore, preferably an image is recorded at each position of the multitude of individual positions to depict the spectral response of the object.

[0031] Accordingly, each pixel would represent an individual position of the image sensor, which can be adjusted using the MEMS actuator.

[0032] Further advantageous designs are shown in the drawings. These show:

[0033] Fig. 1: a schematic representation of a device according to an embodiment of the invention; 202406849

[0034] 6

[0035] Fig. 2: a further schematic representation of a device according to a further embodiment of the invention;

[0036] Fig. 3: a third schematic representation of a device according to a further embodiment of the invention

[0037] Fig. 4: a schematic flowchart of a method according to one embodiment of the invention.

[0038] Figure 1 shows a schematic representation of a device according to an embodiment of the invention. The device 1 shown here comprises a MEMS actuator 2, an image sensor 3, and a spectrometer optic consisting of several lenses 6, a pinhole arrangement 7, and a prism 5. The incident light from the object 4 is dispersed into individual wavelengths λ1 and λ2 by the prism 5. These individual beams with the corresponding wavelengths λ1 and λ2 strike the image sensor 3. The pinhole arrangement 7 is connected to a MEMS actuator 2. In this embodiment, the pinhole arrangement 7 can be moved in the x / y direction by means of the MEMS actuator 2. The movement can be performed continuously, pixel by pixel. Accordingly, each pixel is an adjustable individual position in which an image is recorded to depict the spectral response of the object.

[0039] Figure 2 shows a further schematic representation of a device according to a further embodiment of the invention. In this embodiment, only one lens 6 is arranged in the device 1. The image sensor 2 is displaceable in the x and y directions via the MEMS actuator. The displacement can, for example, be stepless, pixel by pixel, row by row, and / or column by column. In this embodiment, only one lens 6 or lens arrangement 6 is used, and no prism or grating is used for spectral dispersion of the incident light. Furthermore, the image sensor used preferably has a multi-color filter.

[0040] 7

[0041] Figure 3 shows a further schematic representation of a device according to a further embodiment of the invention. In this embodiment, only a hyperchromatic lens 6 is arranged in the device 1. The image sensor 3 is mounted on a MEMS actuator 2. In this embodiment, the image sensor 2 can be moved in the x, y, and z directions via the MEMS actuator. The movement can, for example, be stepless, pixel by pixel, row by row, and / or column by column. In this embodiment, only one lens 6 or lens arrangement 6 is used, and no prism or grating is required for spectral dispersion of the incident light. Furthermore, the image sensor 3 preferably has a multi-color filter.

[0042] Figure 4 shows a schematic flowchart of a method according to an embodiment of the invention. In step S1, a device according to an embodiment of the invention is provided. In step S2, an object to be examined is provided. In step S3, the object to be examined is scanned, wherein, during scanning, the image sensor and / or the aperture arrangement of the device is moved by means of a MEMS actuator such that hyperspectral scanning of the entire object to be examined is enabled.

[0043] 8

[0044] Reference symbol list

[0045] 1 Device

[0046] 2 MEMS actuator

[0047] 3 Image sensor

[0048] 4 objects

[0049] 5 prisms

[0050] 6 Lens / Lens Arrangement

[0051] 7-hole aperture

[0052] λ1 Light beam of wavelength λ1

[0053] λ2 Light beam of wavelength λ2

[0054] λ unbroken light ray

[0055] S1-S3 process steps

[0056] x x-direction

[0057] y y-direction

Claims

202406849 9 Patent claims 1. Device (1) for hyperspectral image acquisition comprising at least one image sensor (3) for recording camera images, at least one first pinhole arrangement (7), a lens arrangement (6) and at least one MEMS actuator (2), characterized by the fact that by means of at least one MEMS actuator (2) the image sensor (3) and / or a pinhole arrangement (7) can be moved in such a way that the movement of the image sensor (3) and / or the pinhole arrangement (7) enables spectral scanning, a higher spectral resolution and / or a higher spatial resolution.

2. Device (1) according to claim 1, characterized by the fact that the image sensor (3) has a monochrome color filter or a multi-color filter.

3. Device according to claim 1, characterized by the fact that by means of the movement of the image sensor (3) and / or the pinhole arrangement (7) a classification of objects (4) in the camera images is made possible.

4. Device (1) according to one of the preceding claims, characterized by the fact that the image sensor (3) and / or the pinhole arrangement (7) can be moved in the x-direction and / or y-direction and / or z-direction by means of the MEMS actuator (2).

5. Device (1) according to any one of the preceding claims, characterized by the fact that 202406849 10 the device further comprises a spectrometer optic using a prism (5) or grating.

6. Device (1) according to any one of the preceding claims, characterized by the fact that The device further features a hyperchromatic lens with longitudinal chromatic aberration.

7. Hyperspectral imaging procedures comprise the following steps: - Providing (S1) a device (1) according to any one of claims 1 to 6; - Providing (S2) an object to be examined (4); - Scanning (S3) of the object (4) to be examined, wherein during scanning the image sensor (3) and / or the pinhole arrangement (7) of the device (1) is moved by means of a MEMS actuator (2) such that hyperspectral scanning of the entire object (4) to be examined is made possible.

8. Method according to claim 7, characterized by the fact that the image sensor (3) and / or the pinhole arrangement (7) is moved into a multitude of individual positions.

9. Method according to claim 8, characterized by the fact that In each position of the multitude of individual positions, an image is recorded to depict the spectral response of the object (4).

Images