AR Hand Occlusion via Depth Map and Light Emitter Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR) technologies face challenges in efficiently rendering occlusion of physical objects, such as hands, within virtual environments, leading to immersion issues and increased computational resources, which results in battery drain and discomfort for users.
Innovation Solution
The method involves generating a virtual object representation of the user's hand using a three-dimensional mesh and height map, allowing for accurate occlusion by selectively disabling light emitters in the head-mounted display (HMD) to allow real-world light to pass through, thereby enhancing immersion and reducing computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional rendering methods are used to render occlusion of physical objects in AR, then occlusion accuracy is improved, but computational resources increase and battery drain worsens
Solution Approach 1:
The patent extracts and processes only the critical components needed for occlusion rendering - specifically generating virtual representations of hands and selective depth testing - rather than rendering the entire scene with full occlusion calculations. This extraction approach maintains occlusion accuracy for relevant objects while reducing overall computational load and battery consumption.
Solution Approach 2:
The system applies partial occlusion rendering by selectively rendering only certain objects (hands and selected interactive objects) with full occlusion accuracy, while using simplified or no occlusion rendering for other objects in the scene. This partial application of the occlusion effect maintains immersion where needed while conserving battery power.
2Reliability
If full occlusion rendering is applied to all objects, then immersion is improved, but device complexity and computational load increase
Solution Approach 1:
The patent applies different rendering qualities to different regions and objects - full occlusion rendering is applied locally to hands and selected interactive objects where immersion is most critical, while other areas use simplified or no occlusion rendering. This local differentiation maintains immersion in key areas without the computational burden of applying it universally.
Solution Approach 2:
Instead of applying occlusion rendering to all objects, the system selectively applies it only to specific objects (hands and selected interactive objects) that contribute most to user immersion. This partial application strategy maintains overall immersion while significantly reducing device complexity and computational requirements.
3Measurement precision
If hand tracking and virtual hand representation are implemented, then user interaction accuracy is improved, but processing time and energy consumption increase
Solution Approach 1:
The system extracts only the essential hand geometry information needed for occlusion - generating a virtual hand representation from captured images - rather than processing complete hand models or performing complex hand pose analysis. This extraction approach maintains hand tracking accuracy for occlusion purposes while minimizing processing time and energy consumption.
Data Source
AI summary
Systems and methods for occluding virtual objects in an augmented reality environment are described. A method includes generating a distance map associated with a physical object, determining a location of a virtual object in an artificial-reality environment, determining, based at least in part upon the distance map and the location of the virtual object, a first portion of the physical object that is closer than a first portion of the virtual object from the viewpoint of the user, determining a second portion of the virtual object that is not obscured by the physical object from the viewpoint of the user, and rendering a display image. The display image comprises a first set of pixels corresponding to the first portion of the physical object in a first state, and the display image comprises a second set of pixels corresponding to the second portion of the virtual object in a second state.