Task-Aware Robot Grasp Estimation for 6DoF Pick-and-Place
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robot motion control systems fail to effectively integrate object picking and placing tasks, often resulting in infeasible grasps due to independent treatment of these skills, limiting suitability for 6DoF pick-and-place tasks and novel objects/scenes.
Innovation Solution
A task-aware grasp planning system that determines 3D geometry and affordance information using neural networks to guide robot manipulators in grasping and placing objects, leveraging synergies between picking and placing actions to optimize for task constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If object picking and object placing are explored as independent problems, then algorithm robustness is improved and action search space is reduced, but grasp feasibility for downstream tasks deteriorates
Solution Approach 1:
The patent merges object picking and object placing into a unified task-aware grasp planning framework. The system jointly optimizes grasp selection and placement outcomes by considering both tasks simultaneously, using a combined cost function that evaluates grasp quality and placement feasibility together. This integration allows the robot to select grasps that are not only stable for picking but also enable successful downstream placement tasks.
2Device complexity
If grasp selection is made without considering downstream placement tasks, then computational complexity is reduced, but task feasibility deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-computing placement affordances and evaluating potential downstream tasks before finalizing grasp selection. The system performs preliminary assessment of placement feasibility for each candidate grasp, filtering out grasps that would not enable successful downstream tasks. This preliminary evaluation prevents wasted computation on infeasible grasps while maintaining task feasibility.
3Ease of manufacture
If conventional independent approaches are used for picking and placing, then ease of implementation is improved, but suitability for 6DoF pick-and-place tasks deteriorates
Solution Approach 1:
The patent segments the 6DoF pick-and-place task into distinct but coordinated components: grasp pose estimation, placement pose estimation, and trajectory planning. Each component is handled by specialized modules that can be independently developed and tested, yet work together through a unified optimization framework. This segmentation maintains ease of implementation while enabling full 6DoF task capability.
Data Source
AI summary
Systems, methods, and apparatuses for controlling a robot including a manipulator, including: determining three-dimensional (3D) geometry information about a target object based on an image of the target object; determining 3D geometry information about a scene in which the target object is to be placed based on at least one image of the scene; obtaining affordance information by providing the 3D geometry information about the target object and the 3D geometry information about the scene to at least one neural network model; commanding the robot to grasp the target object using the manipulator according to a grasp orientation corresponding to the affordance information; and commanding the robot to position the manipulator according to a placement direction corresponding to the affordance information in order to place the target object at a location in the scene.