Projection-Based Hand Interaction Stabilization in XR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR systems are inaccurate, imprecise, and provide limited functionality for interacting with objects in artificial reality environments, often requiring separate controller devices and failing to correctly interpret user gestures, leading to frustrating and time-consuming operations.
Innovation Solution
The technology introduces projection-based interaction methods using a combination of origin and control points to stabilize hand interactions, allowing for various projection types such as ray, sphere, cylinder, cone, and pyramid projections, and incorporates bimanual gestures for disambiguating objects and switching between global and local interaction modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional controller-based interaction is used, then device functionality is provided, but device complexity increases and ease of operation decreases
Solution Approach 1:
The patent extracts the interaction control function from separate controller devices and relocates it to the user's hands themselves. By using hand tracking technology, the system eliminates the need for external controllers while maintaining full interaction functionality, directly resolving the contradiction between ease of operation and device complexity
Solution Approach 2:
The user's hands serve dual purposes: they are both the body parts being tracked and the interaction controllers. The hands perform the self-service function of controlling virtual objects without requiring separate controller devices, thereby simplifying the overall system while improving operational ease
2Measurement precision
If hand tracking is used for direct object interaction, then controller devices are eliminated, but measurement precision and reliability decrease
Solution Approach 1:
The patent introduces projection casting as an intermediary mechanism between hand tracking and object interaction. Instead of directly mapping hand position to object position, the system casts a projection from the hand to the target object, allowing for more precise and reliable interaction while maintaining natural hand-based controls
Solution Approach 2:
The system adds a dimensional transformation by casting projections in 3D space from the hand to target objects. This dimensional approach allows the system to maintain natural hand movements while achieving precise object selection and interaction through the projection mechanism
3Length of stationary object
If projection casting is implemented for distant object interaction, then interaction distance is extended, but device complexity increases
Solution Approach 1:
The hand tracking system serves multiple functions: it tracks hand position for proximity interactions and simultaneously casts projections for distant object interactions. This multi-functionality allows the system to extend interaction distance without adding separate dedicated systems, thereby avoiding increased device complexity
Data Source
AI summary
The present technology relates to artificial reality systems. Such systems provide projections a user can create to specify object interactions. For example, when a user wishes to interact with an object outside her immediate reach, she can use a projection to select, move, or otherwise interact with the distant object. The present technology also includes object selection techniques for identifying and disambiguating between objects, allowing a user to select objects both near and distant from the user. Yet further aspects of the present technology include techniques for interpreting various bimanual (two-handed) gestures for interacting with objects. The present technology further includes a model for differentiating between global and local modes for, e.g., providing different input modalities or interpretations of user gestures.