Dense 6-DoF Pose Object Detection via Virtual Depth Map
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for object detection and pose estimation in computer vision, particularly using RGB images, face challenges due to perspective ambiguities and the need for precise 6-DoF pose estimation, which is often imprecise and requires expensive depth cameras or extensive annotation efforts.
Innovation Solution
A computer-implemented method using a trained encoder-decoder neural network with ID mask and correspondence color channel decoder heads to generate a dense 2D-to-3D correspondence map, allowing for accurate 6-DoF pose estimation without relying on bounding boxes or region-of-interest layers, and utilizing a Perspective-n-Point algorithm with Random Sample Consensus for robust pose determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If depth cameras are used for 6DoF pose estimation, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent creates a virtual depth map through correspondence matching between 2D image points and 3D model points, copying the depth information functionally without requiring physical depth sensing hardware. This virtual depth map serves as a substitute for actual depth camera data, achieving pose estimation accuracy while avoiding expensive depth sensing devices
Solution Approach 2:
The patent replaces the mechanical/optical depth sensing system (depth cameras) with a computational approach using correspondence matching algorithms. Instead of using physical sensors to capture depth information, the system computationally derives depth relationships by matching 2D image features with 3D model correspondences, substituting hardware-based depth sensing with software-based computational geometry
2Device complexity
If conventional RGB images are used for pose estimation, then device complexity is reduced, but measurement precision deteriorates due to perspective ambiguities
Solution Approach 1:
The patent transitions from 2D image coordinate matching to 3D spatial correspondence by introducing depth information through virtual depth maps. By establishing correspondences in 3D space rather than just 2D image space, the system resolves perspective ambiguities that plague conventional 2D-based pose estimation, achieving accurate 6DoF pose from standard RGB images
Solution Approach 2:
The patent introduces a virtual depth map as an intermediary data structure that bridges 2D image observations and 3D pose estimation. This intermediate representation encodes depth relationships derived from correspondence matching, allowing the system to leverage simple RGB input while achieving precision comparable to dedicated depth sensing approaches
3Measurement precision
If dense correspondence matching is performed, then measurement precision is improved, but loss of time increases due to computational complexity
Solution Approach 1:
The patent performs preliminary feature detection and description computation before the actual correspondence matching stage. By pre-computing robust image features (such as SIFT, SURF, or ORB descriptors) and organizing them in efficient data structures, the system reduces the computational burden during the dense matching phase, enabling high-precision correspondence establishment without excessive computation time
Solution Approach 2:
The patent segments the correspondence matching process into multiple stages: coarse matching using feature descriptors, refinement through geometric constraints, and final dense correspondence establishment. This segmented approach processes correspondences hierarchically, achieving dense matching precision while reducing overall computational time by eliminating unnecessary computations in later refinement stages
Data Source
AI summary
A method and a system for object detection and pose estimation within an input image. A 6-degree-of-freedom object detection and pose estimation is performed using a trained encoder-decoder convolutional artificial neural network including an encoder head, an ID mask decoder head, a first correspondence color channel decoder head and a second correspondence color channel decoder head. The ID mask decoder head creates an ID mask for identifying objects, and the color channel decoder heads are used to create a 2D-to-3D-correspondence map. For at least one object identified by the ID mask, a pose estimation based on the generated 2D-to-3D-correspondence map and on a pre-generated bijective association of points of the object with unique value combinations in the first and the second correspondence color channels is generated.