Simulating Light-In-Flight Imaging via Depth Map Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional camera systems are unable to capture 'light-in-flight' images or videos due to the requirement for expensive and complex time-tagging hardware capable of operating at trillion frames per second, making them unsuitable for commercialization.
Innovation Solution
A method and system that simulate light scattering effects by generating a three-dimensional data structure from a depth map, convolving it with a convolutional operator, and adjusting pixel brightnesses to produce a light-in-flight image, utilizing a mobile device with an optical camera and depth camera to create a simulated wavefront propagation effect.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional camera systems use time-tagging hardware to capture light-in-flight images, then measurement precision of light packet paths is improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent creates a simulated copy of the light-in-flight effect through computational processing rather than direct physical measurement. By generating a three-dimensional data structure from depth maps and convolving it with a convolutional operator, the system produces an optical simulation output that replicates the appearance of captured light packets, eliminating the need for complex time-tagging hardware while achieving the desired visual effect
Solution Approach 2:
The patent replaces the mechanical/optical time-tagging hardware system with a computational processing system. Instead of using ultra-sensitive cameras operating at trillion frames per second to physically capture light packets, the system uses standard camera hardware combined with computational algorithms that process depth maps and generate simulated light-in-flight images through convolution operations
2Productivity
If ultra-sensitive cameras operating at trillion frames per second are used, then light-in-flight imaging capability is improved, but ease of manufacture deteriorates
Solution Approach 1:
The patent employs standard, readily available camera hardware and processing equipment instead of expensive, specialized ultra-sensitive cameras. The computational processing uses conventional computing resources to generate the light-in-flight effect, making the system affordable and manufacturable with off-the-shelf components rather than requiring custom-built trillion-frame-per-second camera systems
Solution Approach 2:
The system creates a computational replica of the light-in-flight imaging capability through software processing rather than relying on expensive specialized hardware. By convolving the three-dimensional data structure with a convolutional operator that defines a Gaussian sphere, the system generates simulated light packet paths using standard computing resources, making the technology accessible and manufacturable
3Device complexity
If computational processing is used to simulate light scattering effects, then device complexity is reduced, but manufacturing precision of the simulation output may deteriorate
Solution Approach 1:
The patent transforms the two-dimensional depth map into a three-dimensional data structure by incorporating depth information as a third dimension. This dimensional transformation allows the computational processing to account for spatial relationships and light scattering effects in three-dimensional space, maintaining simulation accuracy while using standard hardware. The convolution operation then processes this three-dimensional structure to generate the optical simulation output
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Systems and methods related to simulated light-in-flight imaging are described. A computing device may execute a light-in-flight engine to process an optical image and a corresponding depth map to produce a light-in-flight image or video that simulates propagation of a wavefront across a scene. The light-in-flight engine may generate an optical simulation output by transforming a depth map to a three-dimensional lift domain to produce a three-dimensional data structure, then convolving the three-dimensional data structure with a convolutional operator, which may define a Gaussian sphere, to produce a filtered three-dimensional data structure. The light-in-flight engine then affine transforms the optical image with an identified slice of the filtered three-dimensional data structure to produce a light-in-flight image.