Robotic Pick Pose Estimation for Arbitrarily Sized End Effectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic picking systems struggle to estimate optimal pick poses for arbitrarily sized end effectors with oblong footprints, especially in chaotic or occluded scenes, often relying on deep learning methods that require extensive training data and prior object model knowledge, leading to suboptimal and error-prone results.
Innovation Solution
A method that models the end effector as a 2D shape with specified dimensions, estimates a pick point on an object mask using instance segmentation, determines a pick surface from neighboring points, reprojects these points to create a planar representation, and computes a yaw-oriented pick pose based on alignment with the end effector's dimensions, reducing computational cost and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If deep learning methods are used to estimate pick poses, then flexibility in handling various objects is improved, but computational cost and training data requirements increase
Solution Approach 1:
The patent replaces complex deep learning computational systems with a geometric-based algorithmic approach. Instead of using neural networks that require extensive training data, the system uses instance segmentation to obtain object masks and applies geometric transformations (reprojection, surface normal calculation, yaw angle computation) to estimate pick poses. This substitution reduces computational overhead while maintaining adaptability to various objects.
Solution Approach 2:
The patent creates a simplified 2D geometric representation (object mask) from the 3D scene through instance segmentation, and then works with this copied representation to estimate pick poses. By operating on the mask rather than the full complex scene, the system reduces computational requirements while preserving essential geometric information needed for pose estimation.
2Adaptability or versatility
If arbitrarily sized end effectors are supported, then versatility of the picking system is improved, but accuracy of pick pose estimation deteriorates
Solution Approach 1:
The patent adapts the pick pose estimation to the local geometry of each object by computing surface normals and yaw angles specific to the object's orientation. The end effector dimensions are incorporated locally into the pose calculation, allowing the system to accurately estimate poses for arbitrarily sized effectors by tailoring the estimation to each specific object-effector combination rather than using a one-size-fits-all approach.
3Ease of operation
If traditional teach-based approaches are used, then programming simplicity is improved, but flexibility in handling random object configurations deteriorates
Solution Approach 1:
The patent enables the robotic system to automatically determine pick poses without requiring manual programming or teaching. The instance segmentation algorithm automatically identifies objects and their geometries, and the subsequent geometric processing automatically computes appropriate pick poses. This self-service approach eliminates the need for operators to predefine pick and drop positions while maintaining flexibility in handling random object configurations.
Data Source
AI summary
A methodology for estimating a pick pose for an arbitrarily sized robotic end effector. The end effector is modeled as a 2D shape with specified dimensions indicative of its footprint on an object being picked. A pick point is first estimated on an object mask of a selected object, produced by performing instance segmentation on one or more input images of a scene. A pick surface is determined utilizing neighboring points around the pick point in the object mask. A set of points in the object mask, which define an extent of the pick surface, are reprojected with respect to a normal of the pick surface, to create a planar representation of the pick surface. A yaw-oriented pick pose is computed based on alignment of a longer dimension of the end effector model with a longer dimension of the planar representation of the pick surface.