Multi-mode two-hand gesture tracking in XR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing user interaction detection systems are inadequate in determining interaction types based on user activities, particularly when two hands are involved, as they fail to accurately differentiate between direct and indirect interaction modes in extended reality environments.
Innovation Solution
The system interprets user activity involving two hands by determining an interaction mode based on user motion, context, and UI element positioning with respect to gaze direction, allowing for a combination of direct and indirect interactions, where one hand can operate in a direct mode and the other in an indirect mode, or both hands can use a combination of modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single interaction mode is used for both hands, then the system is simpler to implement, but it cannot accurately differentiate between direct and indirect interaction modes when two hands are involved
Solution Approach 1:
The patent segments the interaction detection system by hand, assigning independent interaction modes to each hand. The processor determines a first interaction mode for a first hand and a second interaction mode for a second hand separately, allowing accurate differentiation of direct and indirect interactions for each hand independently while managing complexity through modular processing
Solution Approach 2:
The system dynamically assigns interaction modes to each hand based on real-time gesture detection and analysis. The processor evaluates gestures from multiple sensors and dynamically determines whether each hand is performing direct or indirect interaction, allowing the interaction modes to adapt flexibly to different usage scenarios without fixed constraints
2Measurement precision
If multi-sensor data processing is implemented for each hand, then gesture recognition accuracy improves, but computational complexity and processing time increase
Solution Approach 1:
The patent processes sensor data separately for each hand independently. The processor receives sensor data corresponding to gestures of the first hand and second hand separately, determines interaction modes for each hand independently, and generates interaction data for each hand separately. This segmentation reduces the computational complexity of processing all data simultaneously while maintaining accurate gesture recognition for each hand
Solution Approach 2:
The system extracts and processes only the relevant interaction mode information for each hand from the full sensor dataset. By focusing on determining specific interaction modes (direct or indirect) for each hand rather than processing all possible gesture parameters equally, the system reduces unnecessary computational overhead while maintaining recognition accuracy
Data Source
AI summary
Various implementations disclosed herein include devices, systems, and methods that interpret user activity involving both hands based on determining an interaction mode associated with each hand of a user. For example, a process may receive data corresponding to user activity involving two hands of a user in a 3-dimensional (3D) coordinate system. The process may further identify actions performed by the two hands based on the data corresponding to the user activity. Each of the two hands performs one of the identified actions. The process may further determine whether the identified actions satisfy a criterion based on the data corresponding to the user activity and in accordance, determine to associate each of the actions performed by each of the two hands with a direct interaction mode or an indirect interaction mode.