Foveated Depth Mapping for AR Power Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning depth systems in AR/VR systems consume excessive power and computing resources due to the need for high-resolution depth maps across the entire field of view, which affects performance and user experience.
Innovation Solution
The system generates dense depth maps by determining regions of interest using eye-tracking data and implementing a foveated rendering approach, where a high-resolution image is focused on the center (foveated region) and lower resolution images are used on the periphery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution depth maps are generated across the entire field of view, then depth accuracy is improved, but power consumption and computational load increase excessively
Solution Approach 1:
The patent applies local quality by generating high-resolution depth maps only in the foveated region (center of visual attention) while using lower resolution in peripheral regions. This is achieved through eye-tracking data to identify the region of interest, then applying super-resolution algorithms selectively to that area, thereby maintaining depth accuracy where needed while reducing overall computational load and power consumption.
Solution Approach 2:
The patent segments the field of view into distinct regions: a central foveated region requiring high depth accuracy and peripheral regions with lower requirements. By dividing the processing into these segments, the system applies different resolution levels to different areas, optimizing the balance between depth accuracy and computational resources.
2Measurement precision
If high-resolution depth maps are generated across the entire field of view, then depth accuracy is improved, but computational load increases excessively
Solution Approach 1:
The system applies local quality by concentrating computational resources on generating high-resolution depth maps only in the foveated region identified through eye-tracking, while using lower-resolution processing for peripheral areas. This selective approach maintains depth accuracy in critical regions while significantly reducing overall computational load.
Solution Approach 2:
The patent implements partial action by applying super-resolution algorithms only to the necessary foveated region rather than the entire field of view. This partial processing approach provides sufficient depth accuracy for the region of interest while avoiding the excessive computational requirements of processing the complete scene at high resolution.
3Speed
If depth maps are refreshed at 60 Hz for fluid experience, then display refresh rate is improved, but power consumption and computational resources are excessive
Solution Approach 1:
The patent applies local quality by refreshing high-resolution depth maps at 60 Hz only in the foveated region while using lower refresh rates or resolutions for peripheral areas. This selective refresh approach maintains fluid visual experience in the region of interest while reducing overall power consumption and computational requirements compared to refreshing the entire field of view at the same rate.
Data Source
AI summary
The subject disclosure provides for systems and methods for generating a dense depth map. The method may include determining a first region of interest via input from a first sensor device configured to capture supplemental pixel images. The method may include determining a second region of interest via input from a second sensor device configured to capture supplemental pixel images. The method may include receiving depth measurements from a third sensor device configured to capture depth measurements. The method may include determining a plurality of depth maps from the pixel images from the first sensor device, the supplemental pixel images from the second sensor device, and the depth measurements from the third sensor device. The method may include generating from the plurality of depth maps a foveated region comprising a high-resolution image and a low-resolution image oriented on the periphery of the foveated region.