Robotic Grasp Planning With Multi-View Imaging in Dense Clutter
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic systems face challenges in high precision grasp pose detection in dense clutter environments due to limited 3-D geometry information caused by occlusions and fixed camera viewpoints, leading to reduced grasp success rates and increased time consumption in flexible and dynamic settings.
Innovation Solution
A computerized system with an imaging sensor on a robotic manipulator generates 3-D point clouds and uses convolutional neural networks for real-time grasp quality assessment, allowing movement and continuous evaluation of alternative grasp candidates through stochastic optimization and active perception, decoupling environment reconstruction from grasping tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a fixed camera viewpoint is used for grasp detection, then the system structure is simple, but the grasp success rate decreases due to occlusions and limited 3-D geometry information
Solution Approach 1:
The patent applies dynamics by transitioning from a fixed camera viewpoint to a mobile robotic manipulator that can actively move to different positions and orientations. The manipulator equipped with imaging sensors dynamically adjusts its viewing angles and distances to capture multiple views of objects, thereby obtaining complete 3-D geometry information and eliminating occlusion problems while maintaining reasonable system complexity through integrated sensor-manipulator design
Solution Approach 2:
The patent implements dimensionality change by moving from 2-D fixed viewpoint images to 3-D point cloud data acquired from multiple viewpoints. The system reconstructs comprehensive 3-D object models by integrating imaging data from different spatial positions, enabling accurate grasp pose detection in dense clutter environments where single-view 2-D images fail due to occlusions
2Measurement precision
If precise positioning and detailed programming of each robot position is used, then grasp precision is improved, but the adaptability to new objects decreases and requires rewriting control algorithms
Solution Approach 1:
The patent applies self-service by enabling the robotic system to automatically acquire 3-D geometry information of new objects through active movement and multi-view imaging, then autonomously perform grasp pose detection and trajectory planning without requiring human intervention for programming or control algorithm rewriting. The system serves itself by learning object characteristics directly from sensory data
Solution Approach 2:
The patent implements parameter changes by transitioning from pre-programmed fixed-position control to adaptive control that dynamically adjusts grasp parameters based on real-time 3-D object geometry. The system modifies grasp pose, trajectory, and manipulation parameters according to the detected object characteristics, enabling precise and adaptable grasp execution for diverse objects without reprogramming
3Productivity
If CAD files or model information for each object is used, then automated grasp location finding is achieved, but feasibility in flexible dynamically changing environments with unknown objects is reduced
Solution Approach 1:
The patent replaces the mechanical approach of using pre-existing CAD files and models with an information-based sensing and processing system. The robotic manipulator equipped with imaging sensors actively acquires 3-D geometry information of unknown objects in real-time, and the system processes this sensory data to automatically determine grasp locations, eliminating the need for pre-programmed object models
Solution Approach 2:
The patent applies preliminary action by performing 3-D object reconstruction and grasp pose detection before actual grasp execution. The system proactively gathers comprehensive geometric information through multi-view imaging and preprocessing, then plans optimal grasp trajectories in advance, enabling automated and efficient grasp execution for unknown objects in dynamic environments
4Reliability
If multiple views from different perspectives are captured, then complete 3-D geometry information is obtained, but time consumption increases
Solution Approach 1:
The patent implements continuity of useful action by integrating multi-view imaging and 3-D reconstruction processes into the continuous motion of the robotic manipulator. Instead of stopping to capture images from multiple fixed positions, the system continuously acquires imaging data during manipulator movement, seamlessly combining exploration and data collection to reduce time consumption while maintaining complete 3-D geometry information acquisition
Solution Approach 2:
The patent applies preliminary action by performing 3-D object reconstruction and grasp pose detection before actual grasp execution. The system proactively gathers comprehensive geometric information through multi-view imaging and preprocessing, then plans optimal grasp trajectories in advance, enabling automated and efficient grasp execution for unknown objects in dynamic environments
Data Source
Figure 1
Figure 2
Figure 3~5
AI summary
Computerized system and method are provided. A robotic manipulator (12) is arranged to grasp objects (20). A gripper (16) is attached to robotic manipulator (12), which includes an imaging sensor (14). During motion of robotic manipulator (12), imaging sensor (14) is arranged to capture images providing different views of objects in the environment of the robotic manipulator. A processor (18) is configured to find, based on the different views, candidate grasp locations and trajectories to perform a grasp of a respective object in the environment of the robotic manipulator. Processor (18) is configured to calculate respective values indicative of grasp quality for the candidate grasp locations, and, based on the calculated respective values indicative of grasp quality for the candidate grasp locations, processor (18) is configured to select a grasp location likely to result in a successful grasp of the respective object.