Object Pose Estimation Using Point Cloud Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object pose estimation methods, particularly in augmented reality, face challenges with high computational time due to the reliance on clustering schemes for instance segmentation and keypoint detection, which can be inefficient and prone to errors in 3D space processing.
Innovation Solution
The method employs a processor-implemented approach that determines image features from point cloud data, combining depth and color information to estimate object pose through semantic segmentation, instance mask, and keypoint information, reducing the need for clustering and enhancing efficiency by using a multi-layer perceptron network for instance mask segmentation and regression-based keypoint detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If clustering schemes are used for instance segmentation and keypoint detection, then object pose estimation can be performed, but computational time increases significantly
Solution Approach 1:
The patent segments the object into multiple instances using instance segmentation, and further segments key points on the object surface for detection. This segmentation approach enables parallel processing of different object regions and key points, reducing overall computational time while maintaining estimation accuracy
Solution Approach 2:
The patent transitions from 2D image space to 3D point cloud space by extracting three-dimensional information from depth images. This dimensional change enables more accurate pose estimation by capturing spatial relationships in 3D, while the point cloud representation facilitates efficient processing through geometric feature extraction
2Reliability
If clustering schemes are used for instance segmentation and keypoint detection, then object pose estimation can be performed, but processing efficiency decreases
Solution Approach 1:
The patent replaces traditional mechanical clustering schemes with a neural network-based approach. The neural network directly processes point cloud data and predicts instance masks and keypoint locations through learned features, eliminating the need for iterative clustering operations and significantly improving processing efficiency while maintaining reliability
3Measurement precision
If depth images are processed to extract point cloud features, then 3D location information is obtained, but processing complexity increases
Solution Approach 1:
The patent performs preliminary processing of depth images to extract point cloud data and compute geometric features (normals, curvatures) before main processing. This preliminary action organizes the data into a structured point cloud format with pre-computed features, reducing the complexity of subsequent pose estimation operations
Data Source
AI summary
A processor-implemented method with object pose estimation includes: determining an image feature corresponding to a point cloud of an input image; determining semantic segmentation information, instance mask information, and keypoint information of an object, based on the image feature; and estimating a pose of the object based on the semantic segmentation information, the instance mask information, and the keypoint information.