Deep Learning Spectral Mapping for Instant Hemodynamic Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional hyperspectral imaging systems face limitations such as bulky instruments, slow data acquisition rates, low detection efficacy, motion artifacts, and an intrinsic tradeoff between spatial and spectral resolutions, hindering practical and widespread utilization, especially in dynamic imaging applications.
Innovation Solution
A deep learning-based approach that utilizes a dual-channel imaging setup with a trichromatic camera and spectrograph to recover spectral information from RGB values, enabling high spectral and temporal resolutions without mechanical scanning, using a smartphone camera for instantaneous spatiospectral imaging.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If mechanical scanning elements (spectral filters or dispersive optical components) are used in hyperspectral imaging systems, then spectral resolution is improved, but device complexity and acquisition time increase
Solution Approach 1:
The patent segments the spectral acquisition process by using a pushbroom scanner that sequentially captures spectral information from different spatial positions. Instead of using complex mechanical scanning elements for the entire field of view, the system divides the imaging into multiple line scans that are later reconstructed into a full hyperspectral cube, thereby reducing device complexity while maintaining spectral resolution
Solution Approach 2:
The patent replaces complex mechanical scanning elements (such as spectral filters or dispersive optical components requiring high-precision translation) with a simpler pushbroom scanning approach combined with computational reconstruction. This substitution reduces mechanical complexity while achieving comparable or superior spectral resolution through the combination of line scanning and mathematical algorithms
2Measurement precision
If mechanical scanning elements with high precision are used, then spectral resolution is improved, but data acquisition rate decreases
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing the point spread function (PSF) characteristics of the optical system before actual imaging. This pre-characterization allows the system to use simpler scanning mechanisms with faster acquisition rates, as the computational reconstruction can compensate for the reduced mechanical precision, thereby improving data acquisition rate while maintaining spectral resolution
Solution Approach 2:
The patent substitutes mechanical scanning precision requirements with computational methods. By using pushbroom scanning combined with PSF-based reconstruction algorithms, the system achieves high spectral resolution without requiring high-precision mechanical scanning elements, thus significantly improving the data acquisition rate
3Productivity
If snapshot imaging with large-area image sensor is used, then data acquisition rate is improved, but spectral resolution deteriorates
Solution Approach 1:
The patent segments the spectral information acquisition by using a pushbroom scanner that captures one line at a time across the field of view. This segmentation allows the use of a linear array detector instead of a large-area image sensor, enabling sequential spectral sampling that maintains high spectral resolution while achieving fast snapshot-like acquisition through the efficiency of the scanning and reconstruction process
4Loss of information
If hyperspectral filter arrays or coded apertures are used, then spectral information is improved, but manufacturing precision requirements increase
Solution Approach 1:
The patent substitutes complex optical elements (hyperspectral filter arrays or coded apertures requiring nanofabrication and precision alignment) with a pushbroom scanning mechanism combined with computational reconstruction. This replacement uses a simple linear array detector and scanning motion, eliminating the need for difficult-to-manufacture optical components while preserving complete spectral information through sequential sampling and PSF-based reconstruction
Data Source
AI summary
A method of generating an image or video of a field of interest of a sample which includes obtaining i) a first RGB image from a field of interest of a sample, and ii) hyperspectral data from a subarea of the field of interest, extracting an RGB image of the subarea from the first RGB image of the field of interest, applying the hyperspectral data of the subarea to conduct a spectroscopic analysis of a sample thereby generating spectral parameters, inputting i) the spectral parameters, and ii) the first RGB image, collectively as training input data to a deep learning model (DLM), training the DLM with the training input data thus generating a trained DLM, obtaining and inputting a second RGB image of about the field of interest to the trained DLM, and outputting from the trained DLM a spectral map for the field of interest.