3D Radial Voting for Object Pose Keypoint Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing pose estimation methods struggle with background clutter, occlusions, sensor noise, varying lighting conditions, and object symmetries, leading to reduced accuracy in keypoint localization and 6 DoF pose estimation, particularly in hybrid deep learning-based approaches that regress directional quantities for keypoint voting.
Innovation Solution
Employ a machine learning algorithm, such as a convolutional neural network, to estimate radial distances between keypoints and 3D scene locations, using an accumulator space to determine keypoint locations by identifying peaks, thereby improving keypoint localization accuracy and enabling 6 DoF pose estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hybrid deep learning-based approaches regress directional quantities for keypoint voting, then pose estimation can be performed, but keypoint localization accuracy deteriorates due to background clutter, occlusions, sensor noise, and varying lighting conditions
Solution Approach 1:
The patent changes the parameter being regressed from directional quantities (2D offsets) to radial distances (3D Euclidean distances). This parameter transformation allows the voting mechanism to operate in 3D space with spherical symmetry, making the keypoint detection more robust to occlusions, lighting variations, and background clutter while maintaining accuracy.
Solution Approach 2:
The patent transitions from 2D offset voting to 3D radial distance voting. By estimating radial distances in 3D space and using spherical voting in an accumulator space, the method adds a dimensional aspect that improves robustness against occlusions and varying lighting conditions while maintaining keypoint localization accuracy.
2Measurement precision
If conventional voting-based methods are used, then computational simplicity is maintained, but keypoint localization accuracy deteriorates in the presence of object symmetries and occlusions
Solution Approach 1:
The patent changes the voting parameter from 2D directional offsets to 3D radial distances. This simple parameter transformation enables the voting mechanism to leverage spherical symmetry in 3D space, improving accuracy for symmetric objects and occluded keypoints without significantly increasing computational complexity.
Solution Approach 2:
The patent introduces a machine learning model (CNN or transformer) as an intermediary to predict radial distances. This intermediary processes the input image and provides accurate radial distance estimates that feed into the simplified spherical voting mechanism, achieving high accuracy without complex voting logic.
3Productivity
If end-to-end deep learning networks directly regress 6 DoF pose, then processing speed is improved, but generalization accuracy deteriorates due to treating each object as a global entity
Solution Approach 1:
The patent segments the pose estimation problem into two independent parts: (1) keypoint detection through radial distance voting, and (2) pose calculation from detected keypoints. This segmentation allows each part to be optimized independently, achieving both speed and accuracy.
Solution Approach 2:
The patent performs preliminary keypoint detection through radial voting before final pose calculation. By pre-identifying accurate keypoint locations using the radial voting mechanism, the subsequent pose estimation becomes more accurate and can use simpler, faster algorithms.
Data Source
AI summary
Systems and methods are disclosed that facilitate object pose determination via keypoint detection. A machine learning algorithm is employed to facilitate voting-based estimation of the locations of at least three keypoints. In some example embodiments, the machine learning algorithm is trained, based on reference intensity-depth images of an object, to determine a radial distance between a keypoint and a 3D scene location associated with each pixel. At inference, the algorithm is employed to process an intensity-depth image to determine radial distance estimates for each pixel, and these radial distances are employed to increment an accumulator space, such that a sphere is generated in the accumulator space for each pixel, centered on the 3D scene location, and a keypoint location is determined by a peak in the accumulator space, with multiple keypoints enabling pose determination. The methods are adapted to other imaging modalities, including non-depth images and point cloud datasets.