Hyperspectral Imaging with Spectrally Varying 2D Sensor for 3D Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hyperspectral cameras cannot be used for 3D structure estimation as they observe only one line at a time, lacking the necessary multi-angle views required for passive 3D methods, and current solutions are heavy, costly, and require significant processing power.
Innovation Solution
A method and device utilizing a two-dimensional image sensing unit with a spectrally varying optical filter that captures images at different wavelengths as the camera moves, allowing for the creation of spectral vectors and applicability vectors to efficiently represent spectral information, enabling 3D reconstruction and hyperspectral mapping with reduced processing and storage requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a hyperspectral camera observes one line at a time to capture spectral information, then spectral measurement capability is improved, but the ability to capture multi-angle views for 3D structure estimation deteriorates
Solution Approach 1:
The patent transitions from one-dimensional line scanning to two-dimensional area imaging by using a 2D detector array. Each column of the 2D detector captures spectral information at a different wavelength, while each row captures spatial information, enabling simultaneous acquisition of spectral data and multi-angle views for 3D reconstruction.
Solution Approach 2:
The imaging device performs multiple functions simultaneously: it captures spectral signatures for material identification and acquires multi-angle views for 3D structure estimation using the same 2D detector and optical system, eliminating the need for separate sensors.
2Measurement precision
If a laser scanner is combined with hyperspectral camera for 3D mapping, then 3D mapping capability is improved, but device weight and manufacturing cost increase
Solution Approach 1:
The patent merges the functions of the hyperspectral camera and laser scanner into a single integrated device. The 2D detector array simultaneously captures spectral information and geometric information needed for 3D mapping, eliminating the need for a separate laser scanner and reducing overall device weight.
Solution Approach 2:
The same optical system and 2D detector used for hyperspectral imaging are also utilized for 3D mapping by analyzing the geometric relationships of captured images from different angles, making the device multi-functional without additional heavy components.
3Power
If passive 3D methods are used with standard cameras, then processing power requirements are reduced, but compatibility with existing hyperspectral cameras deteriorates
Solution Approach 1:
The patent applies local quality by using different processing strategies for different parts of the 2D detector output. Spectral processing is applied column-wise while geometric processing for 3D reconstruction is applied row-wise, enabling compatibility with both hyperspectral analysis and passive 3D methods.
4Loss of information
If spectral content varies along the direction in the image, then spectral information is captured, but data processing complexity increases
Solution Approach 1:
The patent segments the 2D detector output into columns representing different wavelengths and rows representing spatial positions. This segmentation allows independent processing of spectral information along columns and geometric information along rows, simplifying the overall data processing complexity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables lightweight, cost-effective hyperspectral imaging and 3D mapping with simultaneous spectral signature and structure measurement, suitable for use in small drones and providing high spatial resolution.
Implementation Method 1
providing an imaging device comprising a two-dimensional image sensing unit having a spectral characteristic, which varies along at least one direction in a plane parallel with an image sensor
Data Source
Figure 1
Figure 2
Figure 3a~3c
AI summary
The present disclosure provides a method of providing a hyperspectral image of a scene comprising a point (p). The method comprises providing an imaging device (10, 10') comprising a two-dimensional image sensing unit (12, 13, 14) having a spectral characteristic, which varies along at least one direction in a plane parallel with an image sensor (13), acquiring a first two-dimensional image (11) of the scene, the first image comprising the point (p), wherein the spectral content of the scene varies along the direction as a consequence of the varying spectral characteristic of the image sensing unit (12, 13, 14), moving the imaging device (10, 10') and the scene relative to each other, acquiring a second two-dimensional image (I2) of the scene, the second image comprising the point (p), wherein the spectral content of the scene varies along the direction as a consequence of the varying spectral characteristic of the image sensing unit (12, 13, 14), identifying the point (p) in the second image (I2) as a second pixel at a position (x2, y2) depicting the point (p), providing a spectral vector (S(p)) of the point, and providing an updated spectral vector (S'(p)) of the point based on an applicability vector (A(p)) of the point (p), the spectral vector (S(p)) of the point, a spectral value (z2(p)) of the second pixel and an applicability vector (B(p)) of the second pixel.