Robot Grasp Position Detection Using RGB-Depth Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for autonomous grasping in robots, particularly in complex scenarios, face challenges in accurately segmenting target objects from complex backgrounds and efficiently computing grasping positions using depth images.
Innovation Solution
A method involving the collection of RGB and Depth images from different angles, where pixel points in RGB images correspond to pixel points in Depth images, utilizing convolutional neural networks for object segmentation, optimal grasping position generation, and quality evaluation to determine the highest scoring grasping position for robotic grasping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If automatic segmentation of the target object is based on the Depth image, then the segmentation can be performed, but the object in a complex background may not be segmented accurately and the efficiency is low
Solution Approach 1:
The patent combines RGB images and Depth images into a unified processing framework. The RGB pixel region from object segmentation is mapped to corresponding Depth pixel regions, allowing the system to leverage the strengths of both image types - the texture and color information from RGB images for accurate object boundaries and the depth information from Depth images for spatial understanding, thereby improving both segmentation accuracy and efficiency
Solution Approach 2:
The patent introduces a pixel region mapping mechanism as an intermediary between RGB and Depth images. This mapping relationship serves as a bridge that connects the segmentation results from RGB images with the depth information, enabling accurate object segmentation in complex backgrounds while maintaining high processing efficiency through coordinated processing of both image types
2Extent of automation
If traditional geometric analysis methods are used for autonomous grasping, then grasping can be achieved, but the methods are quite complex
Solution Approach 1:
The patent replaces traditional geometric analysis methods with a deep learning-based approach. Instead of using complex geometric calculations and mechanical reasoning for grasping analysis, the system employs neural networks to automatically learn and identify grasping positions from image data, significantly simplifying the method while maintaining or improving autonomous grasping capability
Solution Approach 2:
The patent transforms the grasping detection problem from a geometric parameter-based approach to a data-driven approach. By changing from explicit geometric analysis to learning from training data, the system achieves autonomous grasping with simpler implementation, as the neural network automatically adapts to different object geometries without requiring complex geometric reasoning
3Extent of automation
If data-driven inference methods are used for autonomous grasping, then grasping can be achieved, but the methods exhibit poor effects in complex scenarios
Solution Approach 1:
The patent merges RGB and Depth image processing to enhance data-driven inference in complex scenarios. By combining the rich texture and color information from RGB images with the spatial depth information from Depth images, the system achieves more reliable autonomous grasping in complex backgrounds where single-modality approaches fail
Solution Approach 2:
The patent adds the depth dimension to the traditional 2D RGB image processing by incorporating Depth images. This dimensional enhancement provides the system with three-dimensional spatial understanding, enabling more reliable data-driven inference for autonomous grasping in complex scenarios by considering both visual appearance and spatial structure
Data Source
AI summary
A method for detecting a grasping position of a robot in grasping a target object includes: collecting a target RGB image and a target Depth image of the target object at different view angles; inputting each of the target RGB image to a target object segmentation network for calculation to obtain an RGB pixel region of the target object in the target RGB image and a Depth pixel region of the target object; inputting the RGB pixel region to an optimal grasping position generation network to obtain an optimal grasping position for grasping the target object; inputting the Depth pixel region of the target object and the optimal grasping position to a grasping position quality evaluation network to calculate a score of the optimal grasping position; and selecting an optimal grasping position corresponding to a highest score as a global optimal grasping position of the robot.