Head-Mounted Display Tap Input Using Vision and Inertial Sensing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing head-mounted displays with inside-out tracking positioning face inaccuracies in identifying tap input signals due to image occlusion and difficulty in detecting smaller hand parts, leading to misjudgment of tap input signals.
Innovation Solution
A head-mounted display system that combines an image capturing device and inertial sensing data from wearable devices on the fingers to determine contact mode and generate accurate tap input signals by analyzing real-time images and inertial sensing data, using a processor to generate a target finger trajectory and corresponding input signals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If computer vision is used to identify hand movements, then the system can directly identify input signals, but the accuracy of tap input signal identification deteriorates due to image occlusion and difficulty in detecting smaller hand parts
Solution Approach 1:
The patent combines computer vision technology with inertial sensing data from wearable devices to create a hybrid identification system. The image capturing device captures hand movement images while wearable devices on fingers provide inertial sensing data, and the processor integrates both data sources to determine contact modes and generate tap input signals, thereby improving identification accuracy while maintaining automation
Solution Approach 2:
The patent introduces inertial sensing data as an intermediary element to bridge the gap between computer vision identification and accurate tap input detection. The wearable devices with inertial sensors act as mediators that provide additional information about finger movements, helping to resolve ambiguities in visual data and improve overall identification precision
2Device complexity
If only computer vision identification is used, then the system structure remains simple, but the reliability of tap input signal generation deteriorates due to misjudgment from image occlusion
Solution Approach 1:
The patent merges computer vision identification with inertial sensing technology to create a more reliable tap input signal generation system. By combining data from image capturing devices and wearable inertial sensors, the system achieves higher reliability in determining contact modes and generating tap input signals, while the integrated processing architecture maintains reasonable system complexity
3Ease of operation
If computer vision alone is used to detect hand movements, then the identification process is straightforward, but the precision of detecting smaller hand parts (e.g., fingers) deteriorates
Solution Approach 1:
The patent segments the identification process into two complementary parts: computer vision for capturing overall hand movement and inertial sensing for detecting specific finger movements. The wearable devices are placed on individual fingers to capture localized inertial data, enabling precise detection of smaller hand parts while the overall process remains operationally simple through integrated processing
Data Source
AI summary
A head-mounted display, tap input signal generating method, and non-transitory computer readable storage medium thereof are provided. The method determines whether a plurality of fingers conform to a contact mode corresponding to an entity plane based on a plurality of real-time images including the fingers of a user and an inertial sensing data. The method generates a target finger trajectory based on the real-time images and the inertial sensing data in response to the fingers conforming to the contact mode corresponding to the entity plane. The method generates a tap input signal corresponding to the fingers based on a target input type and the target finger trajectory.