Robot Object State Estimation via Multi-Source Data Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic devices face challenges in accurately estimating the state of objects due to limitations in using single sources of data, such as vision or kinematic models, which can be affected by self-occlusions, incomplete views, and time delays.
Innovation Solution
The system employs multiple sources and types of object data, including vision, kinematic, and force feedback, to determine the state of objects using a state function that evolves in time, ensuring high accuracy and reliability by prioritizing data sources and discarding unreliable data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If vision-based machine learning algorithms are used to estimate object state, then object state estimation can be performed, but self-occlusions, incomplete views, and time delays reduce reliability
Solution Approach 1:
The patent combines multiple data sources (vision data, kinematic data, force feedback data) into a unified object state estimation system. The computing system integrates these diverse data types to compensate for the limitations of individual sources, particularly addressing self-occlusions and time delays in vision-based estimation by incorporating real-time kinematic and force feedback information.
Solution Approach 2:
The patent introduces an intermediate software layer that acts as a mediator between perception (vision) and control functions. This layer fuses multiple data sources and provides a unified object state representation, resolving the contradiction by creating an intermediary processing stage that harmonizes conflicting or incomplete information from different sensors.
2Ease of operation
If kinematic models are used to estimate object state, then manipulation planning can be supported, but object slip and unsensed gripper compliance make determination difficult or impossible
Solution Approach 1:
The patent implements a feedback mechanism where force feedback data from the end effector is continuously monitored and used to update object state estimation. This feedback loop detects object slip and gripper compliance by measuring forces and torques, allowing the system to correct kinematic model predictions and maintain accurate object state determination despite challenging grasping conditions.
3Reliability
If multiple sources and types of object data are integrated, then more reliable object state estimates can be determined, but system complexity increases
Solution Approach 1:
The patent creates a universal data fusion framework that handles multiple data types (vision, kinematic, force feedback) through a single integrated computing system. This multi-functional system uses unified processing algorithms that can accommodate different sensor types and data formats, reducing the effective complexity by providing a single entry point for diverse data integration rather than separate processing paths for each sensor type.
Data Source
AI summary
A computing system of a robot receives robot data reflecting at least a portion of the robot, and object data reflecting at least a portion of an object, the object data determined based on information from at least two sources. The computing system determines, based on the robot data and the object data, a set of states of the object, each state in the set of states associated with a distinct time at which the object is at least partially supported by the robot. The set of states includes at least three states associated with three distinct times. The computing system instructs the robot to perform a manipulation of the object based, at least in part, on at least one state in the set of states.