Head-Mounted Display Gesture Sensing With Prep-Triggered Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hand gesture detection in VR, AR, SR, and MR devices using head-mounted displays is prone to inaccuracies due to reliance on either streaming images or inertial measurement data alone, leading to potential false triggers or reduced accuracy.
Innovation Solution
A head-mounted display device combines camera-based streaming images and inertial measurement data from wearable devices to track hand gestures and movements, using preparation and command patterns to ensure accurate command execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hand gesture detection relies on streaming images alone, then device complexity is reduced, but measurement precision deteriorates due to false triggers or reduced accuracy
Solution Approach 1:
The patent combines streaming images from the camera unit with inertial measurement data from wearable devices to track hand gestures. The processor integrates both data sources to determine hand gestures and movements, using the inertial measurement data as a reference to verify and correct hand gesture recognition, thereby improving accuracy without requiring a completely new system architecture.
Solution Approach 2:
The system uses inertial measurement data as feedback to verify and correct hand gesture recognition from streaming images. The processor compares the hand gesture tracking results with the inertial measurement data and adjusts the recognition accordingly, creating a feedback loop that improves measurement precision while maintaining manageable system complexity.
2Measurement precision
If hand gesture detection relies on inertial measurement data alone, then measurement precision improves, but reliability deteriorates due to potential false triggers
Solution Approach 1:
The patent merges streaming image data with inertial measurement data to create a more reliable hand gesture recognition system. The processor uses both data sources together, with the inertial measurement data providing reference information to verify hand gesture recognition, thereby reducing false triggers while maintaining precision.
Solution Approach 2:
The system implements feedback by using inertial measurement data to verify and correct hand gesture recognition from streaming images. The processor continuously compares the two data sources and adjusts the recognition results, creating a feedback mechanism that improves both precision and reliability by cross-validating measurements.
3Measurement precision
If the processor continuously tracks hand movements, then measurement precision improves, but use of energy deteriorates due to increased computational load
Solution Approach 1:
The patent implements periodic action by having the processor track hand gestures and movements only during specific sensing periods rather than continuously. The processor is configured to track hand movements during sensing periods and execute operations only when hand movements match predetermined patterns, reducing power consumption while maintaining precision during active use.
Solution Approach 2:
The system dynamically adjusts its tracking behavior based on detected hand gestures and movements. The processor activates full hand movement tracking only when necessary (when hand gestures match preparation patterns) and can reduce or pause tracking during idle periods, making power consumption dynamic rather than static.
Data Source
AI summary
A command sensing method, for a head-mounted display device, includes following steps. Streaming images are captured. A hand gesture is tracked according to the streaming images. Whether the hand gesture matches with a preparation pattern is monitored. In response to the hand gesture matching with the preparation pattern at a first time point, a tracking of a hand movement is activated during a sensing period started from the first time point until a second time point. During the sensing period, whether the hand movement matches with a command pattern corresponding to the preparation pattern is monitored. In response to the hand movement matching with the command pattern, an operation corresponding to the command pattern is executed.