Robot State Estimation Using Wireless-Visual Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile robots face challenges in accurately estimating their state in environments where GPS is unreliable or unavailable, such as extraterrestrial and indoor settings, due to the sensitivity of visual odometry and fiducial tag-based methods to noise and limited field-of-view, leading to positioning errors and navigation issues.
Innovation Solution
Fusing wireless features, such as Wi-Fi fingerprinting information, with visual input to enhance state estimation by incorporating Channel State Information (CSI), Received Signal Strength Indicator (RSSI), and Fine Time Measurement (FTM), using a system that integrates sensors like Wi-Fi, Ultra-wideband, and Bluetooth to provide accurate positioning and orientation estimates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If visual odometry and fiducial tag-based methods are used for state estimation, then positioning can be achieved in GPS-denied environments, but the system becomes sensitive to noise and limited field-of-view, leading to positioning errors
Solution Approach 1:
The patent combines multiple sensor modalities (visual sensors, wireless sensors, inertial sensors) into a unified state estimation system. The visual system captures images while wireless sensors (Wi-Fi, Ultra-wideband, Bluetooth) provide additional positioning data through fingerprinting and signal strength measurements. This merging of complementary sensor types resolves the contradiction by maintaining reliability in GPS-denied environments while improving positioning accuracy through multi-modal data fusion that compensates for individual sensor limitations.
Solution Approach 2:
The system creates a composite sensing architecture that integrates heterogeneous sensor types with different operational characteristics. Visual cameras provide spatial context while wireless sensors offer signal-based positioning cues and inertial sensors provide motion data. This composite approach allows the system to maintain reliable state estimation even when individual sensor modalities experience noise or field-of-view limitations, as other sensors can compensate.
2Measurement precision
If multiple sensor modalities are integrated for state estimation, then positioning accuracy improves, but system complexity increases
Solution Approach 1:
The patent divides the state estimation system into modular sensor components (visual sensors, wireless sensors, inertial sensors) that can be independently processed and then fused. Each sensor type operates semi-independently, with its data processed through dedicated pipelines before being integrated. This segmentation manages complexity by allowing parallel processing of multiple sensor streams while maintaining overall system coordination through the fusion architecture.
Solution Approach 2:
The system employs a universal state estimation framework that handles multiple sensor modalities through a common processing architecture. The same computational infrastructure processes visual, wireless, and inertial data, applying unified algorithms for data fusion and state estimation. This multi-functional approach improves positioning accuracy while managing complexity by avoiding separate dedicated systems for each sensor type.
3Reliability
If wireless features are fused with visual input, then state estimation robustness improves in unreliable GPS environments, but computational requirements increase
Solution Approach 1:
The patent implements selective data fusion where wireless features (Channel State Information, Received Signal Strength Indicator, Fine Time Measurement) are combined with visual input only when GPS is unavailable or unreliable. The system dynamically activates fusion based on environmental conditions, using partial computation rather than continuous full-scale processing. This approach improves robustness in GPS-denied environments while managing computational energy consumption by avoiding unnecessary processing when GPS provides sufficient data.
Data Source
AI summary
A computer-implemented system and method relate to operating a mobile robot with respect to a reference location. First state data is generated using sensor data obtained from a first set of sensors of a first sensor modality. Second state data is generated using second obtained from a second set of sensors. The second set of sensors provide wireless sensing. The second state data is generated from wireless features of the second sensor data. A first distribution of the first state data is generated. A second distribution of the second state data is generated. A posterior distribution is computed by fusing the first distribution and the second distribution. Optimal state data and associated uncertainty data is generated using the posterior distribution. The optimal state data including a position estimate of the mobile robot. The mobile robot is controlled using at least the optimal state data.