Augmenting Depth Maps with Reflectivity for 3D Scene Fidelity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D representations of objects in Virtual Reality systems using only depth map information lack detailed surface information, degrading user experience due to insufficient surface detail rendering.
Innovation Solution
A system that augments depth map representations with reflectivity map representations, using structured light to capture images, analyzing pattern features to derive both depth and reflectivity information, and combining these to generate enhanced 3D representations that include both surface details and depth information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If only depth map representation is used to create 3D representations, then the system complexity is reduced, but the surface detail information is insufficient
Solution Approach 1:
The patent combines depth map representation and reflectivity map representation into a unified 3D representation. The depth map provides geometric information while the reflectivity map provides surface material properties, and their integration creates a more complete 3D model that preserves both shape and surface detail information without requiring separate processing systems.
Solution Approach 2:
The structured light projection system serves multiple functions: it captures both depth information through geometric deformation analysis and reflectivity information through intensity analysis of the same pattern features. This multi-functional approach extracts multiple types of data from a single illumination event, reducing the need for separate sensing systems.
2Reliability
If depth map representation is used alone, then the processing time is reduced, but the user experience quality deteriorates
Solution Approach 1:
The system performs preliminary processing by capturing both depth and reflectivity information simultaneously during the structured light scanning process. By extracting both types of information from the same captured images and pattern features, the system avoids the need for separate scanning operations, thereby improving user experience quality without proportionally increasing total processing time.
3Loss of information
If reflectivity map representation is added to depth map representation, then the surface detail information is improved, but the device complexity increases
Solution Approach 1:
The patent segments the information extraction process into distinct modules: depth map generation from geometric deformation of pattern features, reflectivity map generation from intensity characteristics of pattern features, and final integration. This segmentation allows each module to handle specific types of data processing independently, making the overall complex system more manageable and maintainable while preserving comprehensive surface detail information.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution provides more realistic and detailed 3D representations of objects, enhancing user experience in Virtual Reality systems by incorporating both surface details and depth information, thereby improving the fidelity of virtual scenes.
Implementation Method 1
capturing an image of an object in a scene that is illuminated by structured light. The captured image includes pattern features (e.g., stripes in one dimension) that result from the illumination pattern
Implementation Method 2
The pattern features are analyzed to derive a depth map representation of the captured object in the scene by modelling information derived from geometric deformation of the pattern features
Implementation Method 3
The pattern features are also analyzed to derive a reflectivity map representation by obtaining information about reflectivity characteristics of the object
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
A solution for generating a 3D representation of an object in a scene is provided. A depth map representation of the object is combined with a reflectivity map representation of the object to generate the 3D representation of the object. The 3D representation of the object provides more complete and accurate information of the object. An image of the object is illuminated by structured light and is captured. Pattern features rendered in the captured image of the object are analyzed to derive a depth map representation and a reflectivity map representation of the illuminated object. The depth map representation provides depth information while the reflectivity map representation provides surface information (e.g., reflectivity) of the illuminated object. The 3D representation of the object can be enhanced with additional illumination projected onto the object and additional images of the object.