3D Pose Detection via 2D Signature Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Determining the pose of a three-dimensional object based on two-dimensional image data is challenging due to similarities in appearance of different three-dimensional poses, especially for deformable objects like a human hand, which are difficult to distinguish in two-dimensional images, and requires significant computing resources and specialized hardware.
Innovation Solution
A method using a signature database where two-dimensional signatures of objects are associated with known three-dimensional poses, utilizing machine learning to determine key-points and compute signatures for both computer-generated and actual three-dimensional objects, allowing for accurate pose determination through signature matching without requiring three-dimensional data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If three-dimensional or stereo image data is used to determine pose, then measurement precision is improved, but device complexity increases due to specialized hardware requirements
Solution Approach 1:
The patent creates a signature database using computer-generated three-dimensional models that represent poses of deformable objects. These synthetic signatures are stored and matched against signatures extracted from two-dimensional images, eliminating the need for three-dimensional cameras or stereo imaging hardware while maintaining pose determination accuracy
Solution Approach 2:
The patent replaces the mechanical/optical system of three-dimensional or stereo imaging with a computational approach using machine learning models. The system uses two-dimensional image processing combined with pre-computed signatures from three-dimensional models to determine pose, substituting hardware complexity with software-based solutions
2Measurement precision
If three-dimensional or stereo image data is used to determine pose, then measurement precision is improved, but use of energy increases due to large computing resources required
Solution Approach 1:
The patent pre-computes and stores signatures for various poses of three-dimensional models in a signature database before runtime. During actual pose determination, the system only needs to extract features from the two-dimensional image and match them against the pre-computed signatures, significantly reducing the computational energy required compared to processing raw three-dimensional or stereo image data in real-time
Solution Approach 2:
The patent extracts only the essential signature features from the three-dimensional models and stores them in a compact database. This extraction process removes redundant information while retaining the key characteristics needed for pose determination, reducing the amount of data that needs to be processed and the energy required for computation
3Device complexity
If two-dimensional image data is used to determine pose of deformable objects, then device complexity is reduced, but measurement precision deteriorates due to indistinguishable poses
Solution Approach 1:
The patent introduces an intermediary signature database that contains pre-computed signatures from three-dimensional models. This database acts as a bridge between simple two-dimensional image input and accurate three-dimensional pose output, enabling the system to achieve high measurement precision using only a single camera without specialized hardware
Solution Approach 2:
The patent transforms the problem from directly interpreting two-dimensional image coordinates to matching extracted signature features against a database of known pose signatures. This parameter transformation allows the system to leverage the structural information embedded in the synthetic signatures to disambiguate poses that would otherwise be indistinguishable from two-dimensional images alone
Data Source
AI summary
Certain aspects of the present disclosure provide techniques for determining a pose of a three-dimensional deformable object. Embodiments include providing one or more inputs to a machine learning model based on a computer-generated three-dimensional deformable object that has a known pose. Embodiments include determining, based on one or more outputs from the machine learning model in response to the one or more inputs, a two-dimensional signature of the computer-generated three-dimensional deformable object. Embodiments include associating the two-dimensional signature with the known pose of the computer-generated three-dimensional deformable object. Embodiments include determining a respective pose of an actual three-dimensional deformable object based on an image of the actual three-dimensional deformable object and the associating.