Robotic Gripper Grip Detection Using Multimodal Non-Contact Sensing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robotic grippers with underactuated digits face challenges in determining grasp success and quality due to limited control inputs and the reliance on force-based sensors, which can be unreliable, especially when interacting with varied objects like paper or plastic cups.
Innovation Solution
Incorporating a combination of non-contact sensors, such as a time-of-flight sensor and an infrared camera on the palm of the gripper, to generate multiple sensing modalities like distance and image data, which are used by an object-in-hand classifier to assess grasp success and quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If force-based sensors are used to determine grasp success, then grasp quality can be measured, but the system becomes unreliable when interacting with varied objects like paper or plastic cups
Solution Approach 1:
The patent replaces force-based mechanical sensors with optical sensing systems (time-of-flight sensors and infrared cameras) to detect grasp success. This substitution eliminates the reliability issues of force sensors with compliant objects while maintaining the ability to evaluate grasp quality through optical measurements of object presence and position between gripper digits.
Solution Approach 2:
The patent introduces an intermediary optical measurement system that indirectly detects grasp success by measuring the presence and position of objects in the space between gripper digits, rather than directly measuring contact forces. This intermediary approach allows reliable detection across diverse object types including paper and plastic cups.
2Measurement precision
If multiple non-contact sensors are added to the gripper palm, then sensing accuracy improves, but device complexity increases
Solution Approach 1:
The patent combines multiple sensing modalities (time-of-flight distance sensing and infrared imaging) into a single integrated sensor module mounted on the gripper palm. This merging approach achieves high measurement precision through multimodal data fusion while minimizing the increase in device complexity by consolidating sensors rather than adding them separately.
Solution Approach 2:
The patent implements a multi-functional sensor system where the time-of-flight and infrared sensors serve multiple purposes: detecting object presence, measuring distance, evaluating grasp quality, and identifying object properties. This universality allows accurate grasp detection without proportionally increasing system complexity, as the same sensors perform multiple measurement functions.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach provides more accurate and reliable grasp evaluation, especially for underactuated grippers, by minimizing the complexity of sensor placement and avoiding the limitations of force-based solutions, while maintaining low compute requirements and compactness.
Implementation Method 1
a time-of-flight sensor arranged on the palm such that the time-of-flight sensor is configured to generate time-of-flight distance data in a direction between the plurality of digits
Implementation Method 2
an infrared camera, comprising an infrared illumination source, where the infrared camera is arranged on the palm such that the infrared camera is configured to generate grayscale image data in the direction between the plurality of digits
Data Source
AI summary
A method is provided that includes controlling a robotic gripping device to cause a plurality of digits of the robotic gripping device to move towards each other in an attempt to grasp an object. The method also includes receiving, from at least one non-contact sensor on the robotic gripping device, first sensor data indicative of a region between the plurality of digits of the robotic gripping device. The method further includes receiving, from the at least one non-contact sensor on the robotic gripping device, second sensor data indicative of the region between the plurality of digits of the robotic gripping device, where the second sensor data is based on a different sensing modality than the first sensor data. The method additionally includes determining, using an object-in-hand classifier that takes as input the first sensor data and the second sensor data, a result of the attempt to grasp the object.