Robotic Grasping With Fixed and Onboard Sensor Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object picking systems face challenges such as exhaustive and biased manual labeling of grasp locations, lack of scalability to different object types, and over-fitting due to weak supervision in grasping performance, leading to inefficient and inflexible grasping strategies.
Innovation Solution
A multi-sensor system integrated with a robotic arm that uses a combination of fixed and onboard sensors, including cameras, force/torque sensors, and tactile sensors, to automatically generate and optimize grasp labels through real-time sensor feedback, allowing for adaptive grasping of various objects by evaluating grasp quality and adjusting grasping positions iteratively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual labeling of grasp locations is used, then grasp stability can be improved, but the process becomes exhaustive and time-consuming
Solution Approach 1:
The system uses autonomous agents that automatically explore the environment and label grasp locations without human intervention. The agents independently navigate, perceive objects, and generate grasp labels through trial-and-error interactions, making the system self-sufficient in the labeling process.
Solution Approach 2:
The patent replaces manual human labeling with an automated multi-agent system using reinforcement learning. The mechanical process of human annotation is substituted with autonomous software agents that learn optimal grasp locations through environmental interaction and reward-based feedback.
2Loss of information
If human labeling is used to determine grasp locations, then semantic understanding is improved, but bias is introduced in the labeling process
Solution Approach 1:
Autonomous agents independently discover and label grasp locations based on physical interaction outcomes rather than human semantic interpretation. The labeling process is driven by the agents' own exploration and learning, eliminating human semantic bias while maintaining object understanding through environmental feedback.
3Reliability
If calibration is performed for a particular object, then grasping performance is improved, but the solution is not scalable to different object types
Solution Approach 1:
The multi-agent system performs calibration for multiple object types simultaneously through parallel exploration. Different agents can work on different objects or the same agent adapts its behavior across various object types, making the calibration process universal and scalable rather than object-specific.
Solution Approach 2:
The system dynamically adapts its calibration approach based on the object being encountered. Agents modify their exploration strategies and grasp attempts according to real-time observations of object properties, enabling flexible calibration that scales across diverse object types rather than requiring static pre-calibration for each object category.
4Extent of automation
If trial-and-error experiments are conducted to generate grasp labels, then automatic labeling is achieved, but the learner becomes prone to over-fitting and provides weak supervision
Solution Approach 1:
The system implements feedback mechanisms where agents receive reward signals based on successful grasp outcomes. This feedback guides the learning process, preventing over-fitting by reinforcing generalizable grasp patterns rather than memorizing specific trial outcomes. The feedback loop ensures strong supervision through objective success criteria.
Solution Approach 2:
The labeling process is segmented into multiple independent agent experiences rather than a single monolithic trial-and-error process. Each agent contributes independent labeled data from its own exploration, providing diverse supervision signals that reduce over-fitting and strengthen the overall learning through multiple perspectives.
Data Source
AI summary
A grasping system includes a robotic arm having a gripper. A fixed sensor monitors a grasp area and an onboard sensor moves with the gripper also monitors the area. A controller receives information indicative of a position of an object to be grasped and operates the robotic arm to bring the gripper into a grasp position adjacent the object based on information provided by the fixed sensor. The controller is also programmed to operate the gripper to grasp the object in response to information provided by the first onboard sensor.