Head-Mounted Display Tapping Input via Hand Tracking and Wearable Sensing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing head-mounted display systems with inside-out tracking and positioning face challenges in accurately generating tapping input signals, as users must operate physical buttons on controllers or rely on hand tracking, which can be cumbersome and prone to misjudgment.
Innovation Solution
A head-mounted display system equipped with an image capturing device and processor that captures real-time images of a user's fingers to determine tapping patterns and generates input signals based on matching sensing signals from a wearable device, eliminating the need to hold a controller and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If users operate physical buttons on a handheld controller to input data, then input accuracy can be maintained, but the user cannot perform other additional operations and input speed is slow
Solution Approach 1:
The patent replaces the mechanical button-pressing system with an optical hand-tracking system. The head-mounted display captures images of the user's hand and fingers, processes these images to detect tapping gestures, and generates corresponding input signals. This substitution eliminates the need for physical controllers while enabling both fast input and additional operational flexibility through natural hand movements.
2Ease of operation
If users use hand tracking to input data, then operational flexibility is improved, but it is difficult to accurately determine clear input actions and misjudgment occurs
Solution Approach 1:
The patent implements feedback by capturing real-time images of the user's hand, processing these images to detect finger tapping patterns, and generating corresponding input signals only when clear tapping patterns are identified. The system continuously monitors hand gestures and provides feedback through generated input signals, ensuring accurate recognition while maintaining operational flexibility.
Solution Approach 2:
The patent changes the detection parameters by focusing on specific finger tapping patterns rather than attempting to recognize all possible hand gestures. By monitoring particular parameters such as finger position, movement trajectory, and tapping rhythm in real-time images, the system achieves accurate gesture recognition while maintaining ease of operation.
3Measurement precision
If the system requires users to hold a controller, then input accuracy can be maintained, but user mobility and ease of operation are reduced
Solution Approach 1:
The patent extracts the input function from the physical controller and relocates it to the user's hand gestures. The head-mounted display's image capturing device directly tracks hand movements and generates input signals, eliminating the need to hold a controller. This extraction maintains input accuracy through precise gesture recognition while significantly improving user mobility and ease of operation.
Data Source
AI summary
A head mounted display, tapping input signal generating method, and non-transitory computer readable storage medium thereof are provided. Based on a plurality of real-time images, the head-mounted display determines whether a first finger among the plurality of fingers corresponds to a tapping pattern. The head-mounted display determines whether a sensing signal received from a wearable device matches a tapping gesture of the first finger in response to having the first finger among the fingers corresponding to the tapping pattern. The head-mounted display generates a tapping input signal corresponding to the first finger in response to the sensing signal matching the tapping gesture of the first finger.