Multi-Finger Gesture Tracking via Wearable and Extremity Sensors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current input modalities for engaging with computer-generated objects suffer from inaccuracies due to physical obstructions and depth complexities, leading to reduced tracking accuracy and user inefficiency.
Innovation Solution
An electronic device equipped with an extremity tracking system and a communication interface for a finger-wearable device, which obtains finger manipulation data to determine multi-finger gestures and register engagement events with computer-generated objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current input modality is used to engage with computer-generated object, then user interaction is enabled, but tracking accuracy is reduced due to physical obstructions and depth complexities
Solution Approach 1:
The system segments the input tracking into multiple independent data sources: extremity tracking system data and finger-wearable device data. By dividing the tracking function across multiple sensors and modalities, the system can compensate for obstructions affecting any single sensor, thereby maintaining high tracking accuracy even when physical obstructions or depth complexities interfere with one data source.
2Measurement precision
If single-finger input is used, then simple interaction is achieved, but user control precision is limited
Solution Approach 1:
The system merges data from multiple fingers (extremity tracking and wearable sensor data) to determine multi-finger gestures. By combining information from several input points, the system achieves higher engagement accuracy for complex interactions while maintaining ease of operation through natural hand gestures that users perform instinctively.
Solution Approach 2:
The system adds temporal and spatial dimensions to gesture recognition by tracking multiple fingers across time. Instead of relying on single-point instantaneous input, the system analyzes the trajectories, relative positions, and temporal sequences of multiple fingers, thereby enhancing precision without complicating the user's interaction model.
3Reliability
If extremity tracking system alone is used, then gesture recognition is enabled, but accuracy is reduced due to occlusions and depth issues
Solution Approach 1:
The finger-wearable device acts as an intermediary between the user's finger and the extremity tracking system. The wearable device provides direct tactile and positional data from the finger tip, serving as a mediator that compensates for occlusions and depth limitations of the external tracking system. This intermediary data source ensures reliable gesture recognition even when the finger is partially obscured or at varying depths from the display.
Data Source
AI summary
A method is performed at an electronic device with one or more processors, a non-transitory memory, a display, an extremity tracking system, and a communication interface provided to communicate with a finger-wearable device. The method includes displaying a computer-generated object on the display. The method includes obtaining finger manipulation data from the finger-wearable device via the communication interface. The method includes determining a multi-finger gesture based on extremity tracking data from the extremity tracking system and the finger manipulation data. The method includes registering an engagement event with respect to the computer-generated object according to the multi-finger gesture.