3D Hand Pose Estimation via Database Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods of hand tracking for computer interaction are inefficient and require users to wear cumbersome instrumented gloves, which are impractical for applications like computer-aided design and entertainment, as they reduce dexterity and are uncomfortable for extended use.
Innovation Solution
A system using imaging devices, such as cameras, to record and process hand gestures, segmenting hand regions from the background, and querying a precomputed database to interpret 3D hand poses, allowing for gesture-based interaction without the need for gloves, utilizing techniques like locality sensitive hashing and boosting to recognize hand gestures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional hand tracking methods are used, then hand pose can be tracked, but users must wear cumbersome instrumented gloves that reduce dexterity and comfort
Solution Approach 1:
The patent creates a virtual copy of the hand by projecting a 3D hand model onto the image plane and matching it with the captured hand image. This allows the system to track hand pose accurately without requiring physical instrumentation on the user's hands, thus maintaining both tracking reliability and user comfort.
Solution Approach 2:
The patent replaces the mechanical instrumented glove system with an optical imaging and computer vision system. By using cameras to capture hand images and algorithms to estimate 3D pose from 2D images, the system eliminates the need for mechanical sensors worn by the user, resolving the contradiction between tracking accuracy and user comfort.
2Productivity
If instrumented gloves are used for hand tracking, then real-time tracking is achieved, but the gloves are cumbersome and reduce dexterity
Solution Approach 1:
The patent extracts the tracking functionality from the glove itself and relocates it to the computer system. By processing images captured by external cameras and performing 3D pose estimation algorithmically, the system removes the complex instrumentation from the glove while maintaining real-time tracking capability through software-based solutions.
Solution Approach 2:
The patent substitutes the mechanical sensor system in instrumented gloves with an optical-computational system. Using cameras to capture hand images and sophisticated algorithms to estimate 3D pose in real-time replaces the need for mechanical sensors, wires, and power sources in gloves, thereby reducing device complexity while maintaining productivity.
3Measurement precision
If special gloves are required for hand tracking, then accurate pose estimation is possible, but the system becomes impractical for extended use
Solution Approach 1:
The patent creates a virtual representation of the hand through 3D model projection and image matching, allowing continuous tracking without physical contact or wearable devices. This virtual copying approach maintains measurement precision while enabling indefinite usage duration since no physical glove is required.
Solution Approach 2:
The system uses the user's own hand as the tracking target without requiring external instrumentation. By leveraging natural hand geometry and appearance captured by cameras, the hand itself provides all necessary information for pose estimation, eliminating the need for gloves and enabling extended use without discomfort or fatigue.
4Productivity
If traditional image processing methods are used, then hand tracking can be performed, but the system is not efficient or robust enough for real-time control
Solution Approach 1:
The patent pre-computes and stores a database of projected hand images for various poses before actual tracking begins. During real-time operation, the system quickly matches captured images against this pre-computed database using efficient algorithms like LSH, enabling both real-time responsiveness and robust tracking by leveraging prepared reference data rather than computing everything from scratch.
Solution Approach 2:
The patent replaces traditional iterative image processing methods with a database-matching approach using locality sensitive hashing. This substitution allows the system to achieve real-time performance by hashing and comparing feature vectors against pre-computed database entries, significantly improving both speed and robustness compared to conventional real-time image processing.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables users to interact with computers using natural hand gestures in a comfortable and unencumbered manner, improving dexterity and reducing fatigue, with robust recognition of gestures like pinching, suitable for various applications including computer-aided design and gaming.
Implementation Method 1
A computer can generate a display that responds to these gestures. The generated display can include objects or shapes that can be moved, modified or otherwise manipulated by a user's hands.
Data Source
AI summary
A method and system for human computer interaction using hand gestures is presented. The system permits a person to precisely control a computer system without wearing an instrumented glove or any other tracking device. In one embodiment, two cameras observe and record images of a user's hands. The hand images are processed by querying a database relating hand image features to the 3D configuration of the hands and fingers (i.e. the 3D hand poses). The 3D hand poses are interpreted as gestures. Each gesture can be interpreted as a command by the computer system. Uses for such a system include, but are not limited to, computer aided design for architecture, mechanical engineering and scientific visualization. Computer-generated 3D virtual objects can be efficiently explored, modeled and assembled using direct 3D manipulation by the user's hands.