Natural XR Hand Rendering Using Clipping Masks for Depth Conflicts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The issue in XR systems is the disconcerting visual effect when a user's hand appears to be behind a virtual object due to the binocular display, causing confusion as the hand is closer to the user than the virtual object, which is displayed further away.
Innovation Solution
The XR system generates a clipping mask based on hand tracking data to create a clipped virtual object, ensuring the hand appears to overlap the virtual object by applying a 3D or 2D clipping mask to the virtual object's model data, adjusting the display to align with the user's hand position.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a binocular display is used to present virtual objects at a distance, then the virtual object appears to be located in the real-world scene at the correct apparent distance, but the user's hand appears to be behind the virtual object causing visual dissonance
Solution Approach 1:
The patent applies different rendering strategies to different spatial regions: virtual objects are rendered with full occlusion when behind the hand, while regions corresponding to the hand's position use clipping masks to allow hand visibility. This local differentiation resolves the visual dissonance by making the display behavior adaptive to the specific spatial relationship between hand and virtual object.
Solution Approach 2:
Instead of making the hand appear behind the virtual object (conventional approach), the patent inverts the rendering logic by making the virtual object transparent or clipped in regions where the hand should be visible. This inversion prioritizes the real-world hand's visibility over the virtual object's complete occlusion, resolving the depth perception conflict.
2Shape
If the virtual object is displayed to occlude the hand, then the virtual object appears complete, but the hand position and distance information becomes ambiguous
Solution Approach 1:
The patent segments the virtual object's rendering into multiple regions: fully opaque regions where the hand is not present, and clipped/transparent regions where the hand is located. This segmentation allows the virtual object to maintain completeness in most areas while selectively revealing hand position information in specific regions, thus preserving both object integrity and spatial information.
Solution Approach 2:
The clipping mask acts as an intermediary between the virtual object and the hand. Instead of direct occlusion where the virtual object completely blocks the hand, the clipping mask creates a semi-transparent or selectively visible region that mediates the interaction, allowing both the virtual object and hand position to be perceived simultaneously.
3Ease of operation
If hand tracking is implemented to detect hand position, then natural interaction is enabled, but additional processing complexity is introduced
Solution Approach 1:
The system uses the existing camera infrastructure already present for other XR functions to perform hand tracking. Rather than adding dedicated complex sensing hardware, the system repurposes existing optical components and processing pipelines to detect hand position, thereby enabling natural interaction while minimizing additional system complexity.
Data Source
AI summary
An extended Reality (XR) system provides methodologies for clipping a virtual object displayed to a user. The XR system provides an XR user interface that includes a virtual object. The XR system captures tracking video frame data of a hand of the user and generates a clipping mask based on the tracking video frame data. The XR system generates a clipped virtual object by applying the clipping mask to the virtual object and displays the clipped virtual object in the XR user interface.


