Hyperspectral Light Field Imaging Stereo Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hyperspectral light field imaging systems face challenges in generating complete hyperspectral data-cubes from captured images due to the complexity of processing multi-view and multi-spectral data, particularly in accurately matching correspondence points across different spectral bands and handling spectral inconsistencies.
Innovation Solution
A method and system that calculate spectral-invariant feature descriptors using magnitude, direction, and overlapping histograms of oriented gradients, perform hyperspectral stereo matching to obtain disparity maps, and synthesize RGB color values to generate complete hyperspectral data-cubes, incorporating a data processing unit with preprocessing, stereo matching, and data-cube reconstruction units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If traditional stereo matching methods are used on hyperspectral light field images, then the processing is simpler, but the spectral inconsistencies and matching accuracy deteriorate
Solution Approach 1:
The patent introduces spectral-invariant feature descriptors as an intermediary representation that bridges different spectral bands. These descriptors capture geometric and textural information that remains consistent across spectral variations, enabling accurate correspondence matching without being affected by spectral inconsistencies. The descriptors act as a mediator that translates multi-spectral data into a unified feature space for reliable stereo matching.
Solution Approach 2:
The patent transforms the matching problem from direct intensity comparison in spectral space to feature descriptor comparison in a spectral-invariant feature space. By changing the parameter space from raw spectral intensities to histograms of oriented gradients and other invariant features, the system achieves robust matching across different spectral bands while maintaining processing feasibility.
2Measurement precision
If spectral-invariant feature descriptors are calculated using histograms of oriented gradients, then the matching accuracy improves, but the computational complexity increases
Solution Approach 1:
The patent segments the feature extraction process into distinct components: magnitude histogram calculation, direction histogram calculation, and overlapping histograms of oriented gradients. Each component processes specific aspects of the image data independently, allowing for optimized computation of each segment. This segmentation enables parallel processing and reduces the overall computational burden while maintaining high matching accuracy.
Solution Approach 2:
The patent computes only the essential spectral-invariant features needed for matching rather than processing all possible image characteristics. By focusing on magnitude histograms, direction histograms, and overlapping HoGs specifically for correspondence matching, the system achieves sufficient accuracy without the excessive computational cost of exhaustive feature analysis.
3Loss of information
If complete hyperspectral data-cubes are generated from multi-view images, then the information completeness improves, but the data processing complexity increases
Solution Approach 1:
The patent performs preliminary stereo matching and disparity map generation on individual spectral bands before reconstructing the complete hyperspectral data-cube. By pre-processing the multi-view images to establish correspondences and depth information in advance, the system simplifies the subsequent data-cube reconstruction process. The preliminary disparity maps serve as guides that reduce the complexity of integrating multi-view data into the final hyperspectral cube.
Solution Approach 2:
The patent transforms the problem from processing a large volume of multi-view hyperspectral data simultaneously to processing sequences of 2D disparity maps and spectral images. By working in the disparity dimension and reconstructing the data-cube through ordered integration of processed slices, the system manages data complexity through dimensional transformation and sequential processing.
Data Source
AI summary
A method for generating hyperspectral data-cubes based on a plurality of hyperspectral light field (H-LF) images is disclosed. Each H-LF image may have a different view and a different spectral band. The method may include calculating a magnitude histogram, a direction histogram, and an overlapping histogram of oriented gradient for a plurality of pixels; developing a spectral-invariant feature descriptor by combining the magnitude histogram, the direction histogram, and the overlapping histogram of oriented gradient; obtaining a correspondence cost of the H-LF images based on the spectral-invariable feature descriptor; performing H-LF stereo matching on the H-LF images to obtain a disparity map of a reference view; and generating hyperspectral data-cubes by using the disparity map of the reference view. A bin in the overlapping histogram of oriented gradient may comprise overlapping ranges of directions.