Multi-Modal SLAM Sensor Fusion for Drift-Resistant Robot Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current SLAM solutions face challenges in error accumulation, sensitivity to environmental conditions, and lack of redundancy, particularly due to independent operation of visual and LIDAR sensors, which affects the reliability and safety of robot navigation in varying environments.
Innovation Solution
The multi-modal SLAM (MM-SLAM) system dynamically adjusts the weighting of data from LIDAR and camera sensors based on environmental conditions, using a probability density function and historical data to reduce uncertainty and error accumulation, allowing for joint utilization of sensor data from both sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If current SLAM solutions use only cameras or only LIDAR systems as sensors independently, then the system complexity is reduced, but the reliability and functional safety are insufficient due to lack of redundancy
Solution Approach 1:
The patent combines LIDAR and camera sensors into a unified multi-modal SLAM system where both sensors operate simultaneously and their data are integrated through a common optimization framework. This merging provides redundancy and improves reliability while maintaining manageable system complexity through unified processing.
Solution Approach 2:
The system design allows the same processing framework to handle data from multiple sensor types (LIDAR and camera), making the system multi-functional. The unified optimization approach can process various sensor inputs universally, improving reliability without proportionally increasing complexity.
2Device complexity
If visual odometry and LIDAR odometry are estimated independently in VLOAM, then the computational processing is simplified, but error accumulation reaches arbitrarily high values
Solution Approach 1:
The patent merges visual odometry and LIDAR odometry into a unified optimization framework where both data sources are processed together rather than independently. This joint estimation reduces error accumulation by leveraging complementary information from both sensors simultaneously, while the modular optimization structure keeps processing complexity manageable.
3Device complexity
If current SLAM solutions do not dynamically adjust the weighting factor between visual odometry and LIDAR odometry, then the algorithm complexity is reduced, but the system cannot adapt to environmental conditions such as rain or bad lighting
Solution Approach 1:
The patent implements dynamic adjustment of weighting factors between visual and LIDAR odometry based on real-time environmental conditions. The system adapts its sensor fusion strategy dynamically, increasing reliance on LIDAR in poor lighting or rain, and on visual data in favorable conditions, thereby improving environmental adaptability while keeping algorithm complexity manageable through structured optimization.
4Device complexity
If LIDAR and camera sensors operate independently in current SLAM solutions, then the data processing is simplified, but local drift increases due to uncertainties in sensors
Solution Approach 1:
The patent merges LIDAR and camera data processing into a unified optimization framework that jointly processes both sensor inputs. This combined approach reduces local drift by leveraging the complementary strengths of both sensors and their uncertainties simultaneously, while the structured optimization methodology keeps data processing complexity manageable.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
MM-SLAM enhances the reliability and safety of robot navigation by reducing local and global drift, improving absolute trajectory error, and providing redundancy for sensor malfunctions, enabling more accurate mapping and localization in diverse conditions.
Implementation Method 1
receive second data, wherein the second data include a second frame of a second scene of an environment detected by a light detection and ranging (LIDAR) sensor
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A method for motion tracking is provided including receive first data, receive second data, transform the second data to generate transformed second data corresponding to the first frame; determine a first weighting factor for the first data and a second weighting factor for the transformed second data; weight the first data using the first weighting factor to generate first weighted data; weight the transformed second data using the second weighting factor to generate second weighted data; and combine the weighted first data and the weighted second data to generate combined image data. The first data include a first frame of a first scene of an environment detected by a camera or image sensor. The second data include a second frame of a second scene of an environment detected by a light detection and ranging (LIDAR) sensor. At least a subset of the second scene corresponds to the first scene.