Natural XR Hand Rendering Using Clipping Masks for Depth Conflicts

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

The issue in XR systems is the disconcerting visual effect when a user's hand appears to be behind a virtual object due to the binocular display, causing confusion as the hand is closer to the user than the virtual object, which is displayed further away.

Innovation Solution

The XR system generates a clipping mask based on hand tracking data to create a clipped virtual object, ensuring the hand appears to overlap the virtual object by applying a 3D or 2D clipping mask to the virtual object's model data, adjusting the display to align with the user's hand position.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a binocular display is used to present virtual objects at a distance, then the virtual object appears to be located in the real-world scene at the correct apparent distance, but the user's hand appears to be behind the virtual object causing visual dissonance

Engineering Contradiction:
Improvedepth perception accuracyVSAvoidvisual dissonance
Core Design Contradiction:
Measurement precisionVSObject-generated harmful factors

Solution Approach 1:

The patent applies different rendering strategies to different spatial regions: virtual objects are rendered with full occlusion when behind the hand, while regions corresponding to the hand's position use clipping masks to allow hand visibility. This local differentiation resolves the visual dissonance by making the display behavior adaptive to the specific spatial relationship between hand and virtual object.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

Instead of making the hand appear behind the virtual object (conventional approach), the patent inverts the rendering logic by making the virtual object transparent or clipped in regions where the hand should be visible. This inversion prioritizes the real-world hand's visibility over the virtual object's complete occlusion, resolving the depth perception conflict.

Inventive Principle:
Principle #13The other way round (Inversion)

2Shape

If the virtual object is displayed to occlude the hand, then the virtual object appears complete, but the hand position and distance information becomes ambiguous

Engineering Contradiction:
Improvevirtual object completenessVSAvoidhand position information
Core Design Contradiction:
ShapeVSLoss of information

Solution Approach 1:

The patent segments the virtual object's rendering into multiple regions: fully opaque regions where the hand is not present, and clipped/transparent regions where the hand is located. This segmentation allows the virtual object to maintain completeness in most areas while selectively revealing hand position information in specific regions, thus preserving both object integrity and spatial information.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The clipping mask acts as an intermediary between the virtual object and the hand. Instead of direct occlusion where the virtual object completely blocks the hand, the clipping mask creates a semi-transparent or selectively visible region that mediates the interaction, allowing both the virtual object and hand position to be perceived simultaneously.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Ease of operation

If hand tracking is implemented to detect hand position, then natural interaction is enabled, but additional processing complexity is introduced

Engineering Contradiction:
Improveinteraction naturalnessVSAvoidsystem processing complexity
Core Design Contradiction:
Ease of operationVSDevice complexity

Solution Approach 1:

The system uses the existing camera infrastructure already present for other XR functions to perform hand tracking. Rather than adding dedicated complex sensing hardware, the system repurposes existing optical components and processing pipelines to detect hand position, thereby enabling natural interaction while minimizing additional system complexity.

Inventive Principle:
Principle #25Self-service

Data Source

PatentUS12482171B2Natural hand rendering in XR systems
Publication Date: 2025.11.25 SNAP INC
  • US12482171B2 patent drawing
  • US12482171B2 patent drawing
  • US12482171B2 patent drawing

AI summary

An extended Reality (XR) system provides methodologies for clipping a virtual object displayed to a user. The XR system provides an XR user interface that includes a virtual object. The XR system captures tracking video frame data of a hand of the user and generates a clipping mask based on the tracking video frame data. The XR system generates a clipped virtual object by applying the clipping mask to the virtual object and displays the clipped virtual object in the XR user interface.