3D Hand Skeleton Reconstruction from 2D Keypoints Using Lookup Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer systems often lack 3D imaging capabilities, limiting the ability to accurately represent and interact with human hands in applications such as virtual reality, augmented reality, and security systems, despite the importance of 3D representations for enhanced interaction and detection.
Innovation Solution
A method to generate a 3D representation of a hand from a 2D image by identifying keypoints and using lookup tables to determine 3D poses, incorporating denoising techniques to refine the 3D model from noisy inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D imaging devices are used to capture hand images, then measurement precision and 3D representation accuracy are improved, but device complexity and cost increase
Solution Approach 1:
The patent creates a 3D digital copy (skeleton model) of the hand from a 2D image using lookup tables that map 2D keypoint coordinates to 3D pose parameters. This virtual copy enables 3D interaction without requiring physical 3D imaging hardware, resolving the contradiction between measurement precision and device complexity.
Solution Approach 2:
The patent introduces lookup tables as an intermediary component that bridges 2D image data and 3D hand pose representation. These tables pre-compute the mapping relationships, allowing accurate 3D reconstruction from 2D images without complex real-time computation or specialized imaging devices.
2Ease of operation
If 2D cameras are used in mobile devices, then device portability and ease of operation are improved, but measurement precision and 3D representation capability deteriorate
Solution Approach 1:
The patent transforms 2D image data into 3D hand pose information by introducing a third dimension through lookup tables that map 2D keypoint coordinates to 3D skeletal parameters. This dimensionality transformation enables 3D representation capabilities in devices that only have 2D cameras, maintaining portability while improving measurement precision.
3Productivity
If lookup tables are used to determine 3D poses from 2D keypoints, then processing speed and productivity are improved, but manufacturing precision and accuracy may be compromised
Solution Approach 1:
The patent pre-computes and stores mapping relationships in lookup tables during an offline training phase, where 3D hand models are rendered from multiple angles and keypoint coordinates are recorded. This preliminary action enables fast online processing while maintaining accuracy, as the computationally intensive 3D reconstruction work is done in advance and stored for rapid retrieval.
Data Source
AI summary
A processor identifies keypoints on a hand in a two-dimensional image that is captured by a camera. A three-dimensional pose of the hand is determined using locations of the keypoints to access lookup tables (LUTs) that represent potential poses of the hand as a function of the locations of the keypoints. In some embodiments, the keypoints include locations of tips of fingers and a thumb, joints that connect phalanxes of the fingers and the thumb, palm knuckles that represent a point of attachment of the fingers and the thumb to a palm, and a wrist location that indicates a point of attachment of the hand to a forearm. Some embodiments of the LUTs represent 2D coordinates of the fingers and the thumb in corresponding finger pose planes as a function of the locations of the tips of the fingers or thumb relative to the corresponding palm knuckles.