Hand Gesture Detection With External Providers Beyond HMD Field of View
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing head-mounted displays (HMDs) struggle to track hand gestures when the hand leaves the field of view, leading to a loss of image availability and inability to continue tracking the hand pose.
Innovation Solution
A method and system that utilize external gesture information providers to supplement hand gesture detection by determining the sufficiency of available information, receiving and correcting hand gesture information, and predicting hand gestures when the hand is outside the field of view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the HMD uses its own tracking camera to detect hand gestures, then the tracking is accurate when the hand is in the field of view, but the tracking fails when the hand leaves the field of view
Solution Approach 1:
The patent combines multiple gesture information providers (external cameras, sensors, or devices) with the HMD's own tracking camera to create a unified tracking system. When the hand leaves the HMD's field of view, external providers continue to track the hand, ensuring continuous and accurate gesture recognition across extended spatial ranges.
Solution Approach 2:
The system implements multi-functionality by enabling the gesture detection system to operate effectively both when the hand is within the HMD's field of view and when it is outside. Multiple external gesture information providers are configured to take over or supplement tracking in different spatial conditions, making the system universally applicable across various hand positions.
2Device complexity
If the HMD relies solely on its own tracking data, then the processing is simple, but the hand gesture detection fails when information is insufficient
Solution Approach 1:
The system performs preliminary actions by pre-configuring multiple gesture information providers and establishing data fusion mechanisms in advance. When the HMD's own tracking data becomes insufficient, the system can immediately activate external providers without complex real-time decision-making, maintaining reliability while keeping processing manageable.
Solution Approach 2:
The patent introduces an intermediary data fusion layer that combines tracking data from the HMD and external gesture information providers. This intermediary layer intelligently selects and integrates data sources based on availability and quality, improving detection reliability while managing system complexity through structured data processing.
Data Source
AI summary
The embodiments of the disclosure provide a method and a system for detecting a hand gesture, and a computer readable storage medium. The method includes: determining whether information of a hand is enough for identifying a hand gesture of the hand; in response to determining that the information of the hand is enough for identifying the hand gesture, identifying the hand gesture, receiving first hand gesture information from at least one external gesture information provider, and correcting the hand gesture based on the first hand gesture information; in response to determining that the information of the hand is not enough for identifying the hand gesture, receiving second hand gesture information from the at least one external gesture information provider, obtaining a predicted hand gesture, and obtaining the hand gesture based on the predicted hand gesture and the second hand gesture information.