Robot Control Using RF and Vision for Occluded Object Grasping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic and mechanical systems are unable to locate and retrieve target objects in partially or fully occluded environments, leading to inefficiencies and increased costs, as they rely solely on line-of-sight vision and are limited to single piles or pick-up bins.
Innovation Solution
A control system that combines radio frequency (RF) signals with visual information from image sensors to determine the location of tagged objects, generate trajectories for robot grasping, and declutter environments, using reinforcement learning and deep convolutional neural networks to overcome occlusions and achieve successful grasping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If vision sensors are used to locate target objects, then the robot can identify objects in structured environments, but the robot cannot locate objects in partially or fully occluded environments
Solution Approach 1:
The patent combines vision sensor data with RF signal data to create a hybrid localization system. The vision sensor provides visual information about the environment and visible objects, while the RF signals (from RFID tags) provide location information for occluded objects. By merging these two data sources, the system overcomes the limitation of vision-only systems and can locate objects regardless of occlusion status.
Solution Approach 2:
The patent introduces RF signals as an intermediary mechanism to bridge the gap between the robot and occluded objects. The RFID tags attached to objects act as intermediaries that emit RF signals detectable by the robot's RF sensor, enabling the robot to locate objects even when they are not visible to the vision sensor.
2Device complexity
If the robot operates in single piles or pick-up bins, then the system is simpler to implement, but the system cannot handle multiple piles or cluttered spaces
Solution Approach 1:
The patent creates a universal localization system that can handle multiple piles and cluttered spaces by equipping objects with RFID tags. The RF-based localization approach is not limited to single-pile environments but can scale to multiple piles and complex cluttered spaces, making the system versatile and adaptable to various warehouse configurations.
3Reliability
If human intervention is used to resolve occluded object problems, then the robot can retrieve occluded objects, but efficiency decreases and costs increase
Solution Approach 1:
The patent enables the robot to autonomously locate and retrieve occluded objects using the combined vision-RF localization system. The robot independently processes RF signals to determine object locations even when occluded, and autonomously plans trajectories to retrieve objects without human assistance, making the system self-sufficient and eliminating the need for human intervention.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables robots to locate and retrieve objects in complex, occluded environments by 'seeing through' obstructions, improving efficiency and reducing human intervention, and allowing operation in multiple piles or cluttered spaces.
Implementation Method 1
determine a location of a tagged target object in an area of interest based on a radio frequency (RF) signal
Data Source
AI summary
A control system and method locates a partially or fully occluded target object in an area of interest. The location of the occluded object may be determined using visual information from a vision sensor and RF-based location information. Determining the location of the target object in this manner may effectively allow the control system to “see through” obstructions that are occluding the object. Model-based and/or deep-learning techniques may then be employed to move a robot into range relative to the target object to perform a predetermined (e.g., grasping) operation. This operation may be performed while the object is still in the occluded state or after a decluttering operation has been performed to remove one or more obstructions that are occluding light-of-sight vision to the object.