Robotic Grasp Planning Using Multi-View Imaging and 3D Point Clouds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic systems face challenges in grasping objects in dynamic environments with varying 3-D geometries due to occlusions and fixed camera viewpoints, leading to limited information and reduced grasp success rates, especially in flexible production facilities and home automation settings where objects are unknown during algorithm design.
Innovation Solution
A robotic manipulator equipped with an imaging sensor that captures 3-D point clouds or multiple 2-D images from different viewpoints, using convolutional neural networks for real-time grasp quality assessment and stochastic optimization to select optimal grasp locations and trajectories, allowing continuous motion and refinement of grasp planning without returning to initial positions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a fixed camera viewpoint is used for object detection, then the system structure is simple, but the information about objects is incomplete due to occlusions and limited viewing angles
Solution Approach 1:
The system transitions from 2-D image data to 3-D point cloud representation, adding spatial depth information. This dimensional enhancement allows the system to perceive object geometry from multiple virtual viewpoints simultaneously, resolving occlusions and providing complete object information without adding physical cameras from multiple angles.
Solution Approach 2:
The system creates virtual copies of the object by generating point cloud representations from the single camera view. These digital copies contain enriched 3-D geometric information that compensates for the limitations of the fixed camera viewpoint, allowing grasp planning as if multiple physical cameras were present.
2Loss of time
If the robotic manipulator returns to initial position after failed grasp attempts, then the system is simple to control, but time consumption increases significantly
Solution Approach 1:
The system performs preliminary grasp quality assessment using the 3-D point cloud data before executing the actual grasp motion. By evaluating multiple candidate grasp locations and trajectories in advance, the system identifies optimal paths that avoid failed attempts, preventing time-wasting return-to-origin movements.
Solution Approach 2:
The system implements feedback through continuous evaluation of grasp quality metrics during motion planning. The grasp quality assessment provides real-time information about likely success probability, allowing the system to adjust trajectories and select alternative grasp locations dynamically, rather than following fixed predetermined paths.
3Manufacturing precision
If detailed programming of each robot position is performed a-priori, then grasp precision is high, but adaptability to new objects is lost
Solution Approach 1:
The system changes from fixed programmed positions to dynamically calculated grasp parameters based on 3-D point cloud analysis. By extracting geometric features from the point cloud data, the system adapts grasp locations, orientations, and trajectories to each unique object geometry while maintaining precise positioning through computational optimization.
Solution Approach 2:
The system transitions from static predetermined trajectories to dynamic motion planning that adapts to observed object geometry. The grasp planning process continuously adjusts trajectories based on the 3-D point cloud representation, enabling precise positioning for each object while maintaining versatility across different object types.
4Measurement precision
If multiple images from different viewpoints are captured during motion, then grasp location accuracy is improved, but the imaging system becomes more complex
Solution Approach 1:
The system achieves multi-viewpoint information from a single camera by transforming 2-D image data into 3-D point cloud representation. This dimensional transformation effectively creates virtual viewpoints in three-dimensional space, providing accurate grasp location information without requiring multiple physical cameras or complex multi-sensor arrangements.
Data Source
AI summary
Computerized system and method are provided. A robotic manipulator (12) is arranged to grasp objects (20). A gripper (16) is attached to robotic manipulator (12), which includes an imaging sensor (14). During motion of robotic manipulator (12), imaging sensor (14) is arranged to capture images providing different views of objects in the environment of the robotic manipulator. A processor (18) is configured to find, based on the different views, candidate grasp locations and trajectories to perform a grasp of a respective object in the environment of the robotic manipulator. Processor (18) is configured to calculate respective values indicative of grasp quality for the candidate grasp locations, and, based on the calculated respective values indicative of grasp quality for the candidate grasp locations, processor (18) is configured to select a grasp location likely to result in a successful grasp of the respective object.