Hand Pose Clustering for Ambiguity Resolution in XR Object Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Extended reality (XR) systems face challenges in ambiguity resolution for object selection and faster application loading, particularly in cluttered scenarios where multiple virtual objects are present, leading to confusion and slower interaction times.
Innovation Solution
The system identifies a cluster of virtual objects based on hand pose information, matches the hand posture to expected postures, and preloads applications associated with the initial object selection, allowing for faster application loading and reduced ambiguity by distinguishing between objects through initial and final hand postures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If hand tracking is used to select virtual objects in cluttered scenarios, then user interaction capability is improved, but ambiguity in object selection increases
Solution Approach 1:
The system performs preliminary action by preloading applications associated with virtual objects before the user actually selects them. When the hand approaches a cluster of objects, the system identifies candidate objects and preloads their associated applications in the background, so that when selection occurs, the application loads instantly without visible delay. This resolves the contradiction by preparing in advance to eliminate selection ambiguity delays.
Solution Approach 2:
The system applies dynamics by transitioning from static object selection to dynamic hand pose-based selection. It tracks hand movement through clusters of objects and uses pose classification (grasping, pointing, scrolling) to dynamically determine intent. This dynamic approach resolves ambiguity by continuously updating selection based on evolving hand posture rather than relying on fixed selection criteria.
2Measurement precision
If the system waits for final hand posture to identify object selection, then selection accuracy is improved, but application loading time increases
Solution Approach 1:
The system performs preliminary action by preloading applications before final selection is confirmed. When hand pose indicates intent to select (even before final grasp), the system begins loading the application in the background. This maintains selection accuracy by waiting for confirmed intent while eliminating perceived loading time through advance preparation.
Solution Approach 2:
The system applies skipping by bypassing the traditional sequential process of selection-then-loading. It skips the waiting period by overlapping the selection determination phase with the application loading phase, allowing both to occur concurrently. This reduces total time while maintaining accuracy by using pose-based early indication of selection intent.
3Speed
If the system preloads applications early, then application loading speed is improved, but energy consumption increases
Solution Approach 1:
The system applies partial action by selectively preloading only those applications associated with objects in the current field of view or recently interacted with, rather than preloading all possible applications. This partial preloading provides noticeable speed improvement for likely selections while consuming significantly less energy than comprehensive preloading of all applications.
Data Source
AI summary
Techniques and systems are provided for identifying a virtual object. For instance, a process can include a method for identifying an object. The method may include: receiving pose information associated with a pose of a hand; identifying a cluster of virtual objects the hand is moving toward based on the received pose information, wherein the cluster of virtual objects includes a plurality of independent virtual objects; obtaining a set of expected hand postures associated with virtual objects of the cluster of virtual objects; determining a hand posture based on the received pose information; matching the hand posture to an expected hand posture of the set of expected hand postures to determine an initial hand posture of the hand; and identifying a first initial object of the cluster of virtual objects based on the initial hand posture.