Light-Field and Active Depth Sensor Fusion for High Resolution Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth map generation techniques, whether using light-field data or active depth sensors, face limitations in accuracy and resolution, especially in regions lacking texture or features, which restricts their application in fields like computer vision and virtual/augmented reality.
Innovation Solution
Combining light-field data with active depth sensor data, such as LiDAR and Time of Flight technologies, using sensor fusion techniques to enhance depth map resolution and accuracy by calibrating the sensors and applying depth map generation algorithms that incorporate multi-view geometry and Markov Random Field solutions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If light-field data is used for depth map generation, then depth map resolution can be improved, but depth map accuracy deteriorates in regions lacking texture or features
Solution Approach 1:
The patent combines light-field data with active depth sensor data (such as ToF or LiDAR) to create a fused depth map. The light-field data provides high-resolution spatial information, while the active depth sensor provides accurate depth measurements. By merging these complementary data sources, the system achieves both high resolution and high accuracy in the final depth map, resolving the contradiction between the two parameters.
2Measurement precision
If active depth sensors are used for depth map generation, then depth map accuracy can be improved, but depth map resolution deteriorates to relatively low resolution
Solution Approach 1:
The patent merges active depth sensor data (which provides accurate depth measurements) with light-field data (which provides high spatial resolution). The active depth sensor ensures measurement precision, while the light-field component maintains manufacturing precision in the form of depth map resolution. This combination resolves the contradiction by allowing both parameters to be satisfied simultaneously through data fusion.
3Device complexity
If standalone light-field or active depth sensor technologies are used, then device complexity can be reduced, but depth information reliability deteriorates
Solution Approach 1:
The patent combines two separate sensing systems (light-field camera and active depth sensor) into an integrated system. While this increases device complexity compared to standalone solutions, it significantly improves the reliability of depth information by providing multiple independent measurements that can be cross-validated and fused. The complementary nature of the two technologies ensures more robust and reliable depth maps across various scene conditions.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach significantly improves the accuracy and resolution of depth maps, addressing the limitations of standalone light-field or active depth sensor technologies by integrating their strengths, resulting in more reliable depth information for various applications.
Implementation Method 1
the active depth sensor may include LiDAR and/or Time of Flight technologies. The active depth sensor may emit electromagnetic energy generally radially at the mirror(s), and the mirrors may reflect the electromagnetic energy toward the scene. The electromagnetic energy may reflect off of the scene, back toward the mirror(s).
Implementation Method 2
Time of Flight (ToF) cameras such as the Kinect sensor available from Microsoft Corporation of Redmond, Wash.
Data Source
AI summary
Depths of one or more objects in a scene may be measured with enhanced accuracy through the use of a light-field camera and a depth sensor. The light-field camera may capture a light-field image of the scene. The depth sensor may capture depth sensor data of the scene. Light-field depth data may be extracted from the light-field image and used, in combination with the sensor depth data, to generate a depth map indicative of distance between the light-field camera and one or more objects in the scene. The depth sensor may be an active depth sensor that transmits electromagnetic energy toward the scene; the electromagnetic energy may be reflected off of the scene and detected by the active depth sensor. The active depth sensor may have a 360° field of view; accordingly, one or more mirrors may be used to direct the electromagnetic energy between the active depth sensor and the scene.