RF-Visual Robotic Grasping for Fully Occluded RFID Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic systems struggle to efficiently grasp fully occluded objects due to restrictive assumptions about object shape and orientation, requiring separate infrastructure and calibration, and are inefficient in locating objects outside the camera's field of view.
Innovation Solution
A robotic system integrating RF localization and imaging on a single end-effector, utilizing dense RF-visual geometric fusion and RF-visual reinforcement learning to efficiently localize and grasp objects in line-of-sight, non-line-of-sight, and fully-occluded settings without additional infrastructure or calibration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a separate dedicated infrastructure for RF localization is used, then localization capability is improved, but device complexity and ease of operation deteriorate due to required calibration and environment instrumentation
Solution Approach 1:
The patent combines the RF localization antenna and imaging device into a single integrated end-effector unit. This merging eliminates the need for separate dedicated RF localization infrastructure and calibration processes, while maintaining accurate localization capability through the unified sensor suite.
Solution Approach 2:
The integrated end-effector serves multiple functions: imaging for visual perception, RF localization for positioning, and grasping for manipulation. This multi-functionality allows the robot to operate in uninstrumented environments without requiring separate specialized infrastructure for each function.
2Measurement precision
If the robot scans the 3D environment to accurately localize RFID, then localization accuracy is improved, but productivity deteriorates due to time-consuming scanning
Solution Approach 1:
The system performs preliminary localization using RF signals to identify the target object's position before visual scanning begins. This preliminary action provides prior knowledge that guides the subsequent visual search, reducing the time required for environment scanning while maintaining accurate localization.
Solution Approach 2:
The RF localization signal acts as an intermediary that bridges the gap between the robot and the occluded target object. By providing intermediate positioning information, it enables the robot to efficiently navigate toward the target without exhaustive environmental scanning, thus improving both accuracy and efficiency.
3Difficulty of detecting and measuring
If camera-based visual perception is used, then object perception capability is improved, but ability to detect occluded objects deteriorates
Solution Approach 1:
The patent merges RF sensing capability with visual perception in a single integrated end-effector. The RF signals can penetrate occlusions to detect target objects, while the camera provides visual confirmation, creating a complementary sensing system that overcomes the limitations of either modality alone.
Solution Approach 2:
RF signals serve as an intermediary sensing modality that can detect objects through occlusions where visual perception fails. This intermediary capability allows the system to maintain object detection ability even when the target is hidden from the camera's view.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system achieves centimeter-scale localization accuracy and 96% success rate in grasping complex objects, reducing travel distance and time compared to baseline systems.
Implementation Method 1
Because RF signals can traverse many occlusions, these systems can identify and locate items of interest through occlusions
Implementation Method 2
Remarkable progress in vision systems has enabled robots to perceive, locate, and grasp items in unstructured environments
Data Source
AI summary
Described is the design, implementation, and evaluation of a robotic system configured to search for and retrieve RFID-tagged items in line-of-sight, non-line-of-sight, and fully-occluded settings. The robotic system comprises a robotic arm having a camera and antenna strapped around a portion thereof (e.g. a gripper) and a controller configured to receive information from the camera and (radio frequency) RF information via the antenna and configured to use the information provided thereto to implement a method that geometrically fuses at least RF and visual information. This technique reduces uncertainty about the location of a target object even when the object is fully occluded. Also described is a reinforcement-learning network that uses fused RF-visual information to efficiently localize, maneuver toward, and grasp a target object. The systems and techniques described herein find use in many applications including robotic retrieval tasks in complex environments such as warehouses, manufacturing plants, and smart homes.