Hand Tracking Camera Module for Head Mounted Display
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hand tracking technologies in 3D user interaction applications, such as VR and AR, are ineffective in tracking all fingers' motion naturally and require users to constantly hold controllers, leading to unnatural user experiences and limited accuracy, especially in free-hand interactions.
Innovation Solution
A computer-vision-based hand tracking system using a head-mounted display with stereo or depth cameras, capable of tracking 26 degrees-of-freedom hand skeleton pose without external hardware, allowing for free-hand interaction and seamless switching between interaction modes without recalibration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional hand tracking methods are used, then users can interact with virtual objects, but users must constantly hold controllers which leads to unnatural user experience and limited accuracy
Solution Approach 1:
The patent extracts the hand tracking functionality from traditional controller-based systems and implements it using only the head-mounted display's integrated camera and processor. This eliminates the need for separate controllers while maintaining interaction capability, directly addressing the contradiction between natural operation and device complexity.
Solution Approach 2:
The head-mounted display system performs hand tracking autonomously using its own integrated camera and processor without requiring external hardware assistance. The system serves itself by utilizing its built-in components to achieve the tracking function that previously required separate devices, thereby improving ease of operation while reducing overall system complexity.
2Ease of operation
If hand tracking is implemented without external hardware, then ease of operation improves, but measurement precision and tracking accuracy deteriorate
Solution Approach 1:
The patent merges the camera, processor, and hand tracking algorithms into a single integrated system within the head-mounted display. This consolidation allows the system to process visual data and compute precise hand pose (26 degrees-of-freedom) using only its own components, achieving both ease of operation and measurement precision without external hardware assistance.
Solution Approach 2:
The system tracks 26 degrees-of-freedom hand skeleton pose parameters, which is a comprehensive set of parameters that captures detailed hand geometry and orientation. By changing from traditional 6-DOF controller tracking to 26-DOF skeleton joint tracking, the system achieves high measurement precision while maintaining the simplicity of wearables without external hardware.
3Measurement precision
If stereo or depth cameras are used for hand tracking, then measurement precision improves, but device complexity and manufacturing cost increase
Solution Approach 1:
The head-mounted display's camera serves multiple functions: it captures images for hand tracking, provides visual feedback to users, and enables 3D rendering. By making the camera multi-functional, the system achieves high measurement precision for hand tracking without adding separate dedicated cameras, thereby reducing device complexity and manufacturing cost while maintaining 3D tracking capability.
Data Source
AI summary
A system for hand tracking may comprise a head mounted display wearable by a user and a hand tracking camera module attached to the head mounted display and comprising at least one of a pair of stereo cameras or a depth camera. The hand tracking camera module may be configured to capture images of at least one physical hand of the user. The head mounted display may be configured to render a virtual hand resembling the physical hand in a virtual environment for viewing by the user based at least on the images.