Pose Fusion Estimation via Graph Neural Network Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current pose estimation techniques face challenges in accurately determining the position and orientation of objects or people using limited data from single sensory inputs, such as visual or tactile sensors, which can be insufficient for reliable predictions.
Innovation Solution
A system that combines data from different sensor types, including visual and tactile sensors, using a graph-based neural network to generate geometric features, which are then processed through a pose fusion network with rotation, translation, and confidence layers to produce accurate pose estimates, leveraging the strengths of both sensory inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data from multiple sensor types is fused, then pose estimation accuracy is improved, but device complexity increases
Solution Approach 1:
The patent combines data from multiple sensor types (visual sensors and tactile sensors) into a unified pose estimation system. The visual sensor data and tactile sensor data are processed through separate neural network branches and then fused to produce a final pose estimate, leveraging the complementary strengths of different sensing modalities to improve accuracy while managing system complexity through structured integration.
2Reliability
If geometric features are extracted through graph-based neural network, then pose prediction reliability is improved, but processing time increases
Solution Approach 1:
The patent performs preliminary extraction of geometric features from sensor data using a graph-based neural network before the final pose estimation step. By pre-processing the data to extract meaningful geometric representations (such as point cloud features and spatial relationships), the system prepares the data in advance, making the subsequent pose prediction more reliable and efficient.
3Measurement precision
If semantic segmentation is performed on sensor data, then feature extraction accuracy is improved, but computational load increases
Solution Approach 1:
The patent applies semantic segmentation to divide the sensor data into meaningful regions or segments that correspond to different objects or parts of interest. This segmentation allows the system to focus computational resources on extracting features from relevant portions of the data, improving feature extraction accuracy while managing computational load through selective processing.
Data Source
AI summary
Pose fusion estimation may be achieved via a first and second set of sensors receiving a first and second set of data, passing the first and second set of data through a graph-based neural network to generate a set of geometric features to be passed through a pose fusion network to generate a first and second pose estimate. A second portion of the pose fusion network may receive the set of geometric features and generate a second set of geometric features and the second pose estimate based on the set of geometric features. A first portion of the pose fusion network may receive the first set of data and the second set of geometric features and generate the first pose estimate based on a fusion of the first set of data and the second set of geometric features.