Semantic SLAM Framework for Object Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 6DoF object pose estimation methods face challenges with textureless or symmetric objects, as their semantic keypoints detection is inconsistent across frames, leading to invalid poses under the world coordinate system.
Innovation Solution
A computer-implemented method and system that generates 2D keypoints for objects, uses camera pose data to create keypoint heatmaps, and tracks objects in world coordinates, employing a keypoint network to predict consistent keypoint coordinates and uncertainty, especially for symmetric objects by utilizing prior knowledge from SLAM data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If semantic keypoints are detected for textureless or symmetric objects, then object pose estimation can be performed, but the keypoint detection becomes inconsistent across frames leading to invalid poses
Solution Approach 1:
The system performs preliminary action by detecting and tracking keypoints on reference objects first to establish camera pose, then uses this camera pose information to guide the detection of keypoints on target objects. This preliminary establishment of camera pose provides a stable coordinate system that prevents keypoint interchange issues in subsequent tracking
Solution Approach 2:
The camera pose data serves as an intermediary that connects the detection of reference object keypoints to the detection of target object keypoints. By using camera pose as a mediator, the system ensures that keypoints on target objects are detected in a consistent coordinate system, preventing the interchange problem that occurs when detecting keypoints on symmetric or textureless objects directly
2Productivity
If keypoint-based pose estimation is used for real-time performance, then processing speed improves, but accuracy decreases for symmetric and textureless objects
Solution Approach 1:
The system segments the pose estimation process into two distinct phases: first estimating camera pose using reference objects, then estimating target object poses using the established camera pose. This segmentation allows each phase to be optimized independently, maintaining real-time performance while improving accuracy for challenging objects through the two-stage approach
3Ease of operation
If traditional keypoint detection is applied to symmetric objects, then detection can proceed, but keypoints become interchanged causing inconsistent tracking
Solution Approach 1:
Camera pose data acts as an intermediary that provides external reference information to disambiguate keypoints on symmetric objects. By using the camera's position and orientation as a mediator, the system can consistently identify which keypoint corresponds to which physical location on a symmetric object, preventing keypoint interchange and ensuring stable tracking
Data Source
AI summary
A computer-implemented system and method for semantic localization of various objects includes obtaining an image from a camera. The image displays a scene with a first object and a second object. A first set of 2D keypoints are generated with respect to the first object. First object pose data is generated based on the first set of 2D keypoints. Camera pose data is generated based on the first object pose data. A keypoint heatmap is generated using the camera pose data. A second set of 2D keypoints is generated with respect to the second object based on the keypoint heatmap. Second object pose data is generated based on the second set of 2D keypoints. First coordinate data of the first object is generated in world coordinates using the first object pose data and the camera pose data. Second coordinate data of the second object is generated in the world coordinates using the second object pose data and the camera pose data. The first object is tracked based on the first coordinate data. The second object is tracked based on the second coordinate data.