Wearable Head-Mounted Display Pointer Control via Hand Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing screen display control systems require large calculation amounts and long processing times when using sensors and cameras to track finger positions for remote operation, and they can be cumbersome when objects are displayed congestedly, leading to potential erroneous operations.
Innovation Solution
A wearable terminal attached to the operator's head captures images of a display with a rectangular frame and viewport, extracts vertex positions and pointed positions, and uses a computer to determine if a corresponding position in the viewport is present, displaying a pointer accordingly, thereby reducing calculation load and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a sensor and camera are used to track fingertip position with head pose correction, then measurement precision is improved, but calculation amount increases and processing time lengthens
Solution Approach 1:
The patent extracts and processes only the essential information needed for pointer control - specifically the fingertip position and hand orientation - rather than processing all sensor data. This selective extraction reduces calculation complexity while maintaining tracking precision.
Solution Approach 2:
The system segments the tracking task into distinct components: hand detection, fingertip position extraction, and pointer control mapping. This segmentation allows each component to be optimized independently, reducing overall computational burden.
2Measurement precision
If many local features are handled for spatial positioning, then measurement precision is improved, but processing time increases
Solution Approach 1:
The patent extracts only the critical local features necessary for hand and fingertip positioning, rather than processing all local features in the camera image. This selective feature extraction maintains spatial positioning accuracy while significantly reducing processing time.
3Ease of operation
If a wearable terminal is attached to the head to capture images, then ease of operation is improved, but device complexity increases
Solution Approach 1:
The wearable terminal serves multiple functions: capturing images for hand tracking, determining head pose for coordinate transformation, and providing visual feedback. This multi-functionality reduces the need for separate devices while improving operational ease.
Data Source
AI summary
A screen display control method includes: by using a wearable terminal to be attached to a head of an operator to capture an image, capturing, with an image of a display having a rectangular frame and a viewport, an image of a hand portion with which the operator points to the display; by using a computer, extracting positions of vertexes of the rectangular frame in the captured image by the wearable terminal and a position pointed by the hand portion; determining, based on a position relation of the extracted positions of the vertexes and the extracted pointed position, whether a position in the viewport corresponding to the hand portion is present; and causing a pointer to be displayed at the position if present.