6D Pose Estimation via Pointwise 2D-3D Feature Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 6D pose estimation methods face challenges in handling the domain gap between synthetic and real data, scale variance, and noisy segmentation, particularly in accurately determining the location and orientation of objects in images.
Innovation Solution
The method involves processing input images through a two-dimensional graphical neural network (GNN 2D) to extract pointwise 2D features, which are then matched with pointwise 3D features from a corresponding CAD model using a 3D graphical neural network (GNN 3D) to determine the 6D pose, thereby overcoming the domain gap and scale variance issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional 6D pose estimation methods are used, then the process is simpler, but accuracy deteriorates in handling occlusions and textureless cases
Solution Approach 1:
The method segments the pose estimation problem into two independent modules: 2D feature extraction from images and 3D feature extraction from CAD models. Each module processes features separately before matching, which improves accuracy in challenging cases while keeping each module's complexity manageable
Solution Approach 2:
The method transitions from traditional 2D image processing to 3D-2D correspondence by extracting features from both 2D images and 3D CAD models, then matching them across dimensions. This dimensional transformation enables accurate pose estimation in occlusion and textureless cases
2Productivity
If methods using synthetic data are used, then training is more efficient, but domain gap with real data increases
Solution Approach 1:
The method introduces 3D CAD models as an intermediary that bridges synthetic and real domains. By extracting features from both 2D images and 3D CAD models and matching them, the system achieves domain generalization while maintaining training efficiency
Solution Approach 2:
The method changes the feature representation parameters by using pointwise features from both 2D and 3D domains. This parameter transformation enables the system to work effectively across different data domains
3Measurement precision
If dense feature matching is used, then correspondence accuracy improves, but computational cost increases
Solution Approach 1:
The method segments the dense feature matching into 2D feature extraction, 3D feature extraction, and matching phases. This segmentation allows for optimized computation at each stage while maintaining high correspondence accuracy
Data Source
AI summary
Methods and systems for determining a 6D pose of an object in an image are disclosed. In embodiments, an input image is received from a sensor, wherein the input image includes an object in the image. A trained image encoder transforms the input image into a normal map and an instance segmentation map. The normal map is encoded with pointwise 2D features. A 3D CAD model is selected from memory that resembles the object in the image. The 3D CAD model is encoded with pointwise 3D features. The pointwise 2D features are matched with the pointwise 3D features to obtain correspondences between the 2D features and the 3D features. The 6D pose of the object is then determined based on the correspondences.