Walking Robot Pose Estimation Using LiDAR-Visual-Kinematic Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robot pose estimation technologies face challenges in accurately estimating the pose of walking robots, especially over long distances, due to limitations in sensor fusion and the quality of visual features.
Innovation Solution
A pose estimation method that fuses data from LiDAR, image, inertial, and joint sensors using a strong coupling approach, incorporating kinematic factors, point cloud optimization, and visual-inertial-kinematic odometry to improve accuracy and stability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If camera and IMU are used for pose estimation, then the system structure is simple, but the scale estimation accuracy is low due to camera structural limitations
Solution Approach 1:
The patent combines camera, IMU, and LiDAR sensors into a unified pose estimation system. The LiDAR sensor captures 3D point cloud data while the camera captures 2D image data, and both are fused with IMU data through a factor graph optimization framework to achieve accurate scale estimation and pose estimation that overcomes the limitations of camera-only approaches.
Solution Approach 2:
The patent introduces LiDAR as an intermediary sensor that provides accurate 3D geometric information to bridge the scale estimation gap. The LiDAR point cloud data serves as a mediator between the 2D camera observations and the 3D pose estimation, enabling accurate scale recovery through 3D-2D point correspondence.
2Ease of manufacture
If visual features are extracted from camera and IMU data, then the extraction process is simple, but the quantity and quality of features are limited leading to low pose estimation accuracy
Solution Approach 1:
The patent merges visual features from the camera with geometric features from LiDAR point cloud data. The system extracts features from both 2D images and 3D point clouds, then fuses them in a factor graph optimization framework to create a richer feature representation that significantly improves pose estimation accuracy compared to using camera data alone.
3Adaptability or versatility
If exteroceptive and proprioceptive sensors are fused, then the estimation coverage is improved, but estimation errors accumulate due to continuous accumulation of incorrect features in unsuitable environments
Solution Approach 1:
The patent implements a factor graph optimization framework that continuously optimizes pose estimates by minimizing reprojection errors and 3D-2D correspondence errors. This feedback mechanism corrects accumulation errors by constantly refining the pose estimates based on the consistency between LiDAR point cloud data, camera image data, and IMU measurements, preventing error drift over time.
Solution Approach 2:
The patent replaces simple sensor fusion with a factor graph optimization system that uses probabilistic graphical models. This substitution transforms the error accumulation problem into an optimization problem where the system finds the most likely pose sequence that minimizes overall error, providing mathematical guarantees against error accumulation.
4Measurement precision
If 3D LiDAR mapping is used for long-distance pose estimation, then the position estimation capability is improved, but non-linear movements such as hard impact or foot slippage limit the generation of reliable 3D maps
Solution Approach 1:
The patent uses factor graph optimization to continuously correct pose estimates by minimizing reprojection errors between LiDAR points and their corresponding 2D image projections. This feedback mechanism compensates for errors introduced by non-linear movements like foot slippage, maintaining reliable position estimation even when the robot experiences dynamic disturbances during walking.
5Adaptability or versatility
If heterogeneous sensor fusion is used for pose estimation, then the sensor utilization is maximized, but stability issues arise due to pose differences between visual and 3D distance sensors
Solution Approach 1:
The patent replaces ad-hoc sensor fusion methods with a rigorous factor graph optimization framework. This substitution provides a unified probabilistic model that handles heterogeneous sensors (LiDAR, camera, IMU) consistently, optimizing pose estimates by minimizing a combined error metric that accounts for the different characteristics and noise models of each sensor type, thereby ensuring stability.
Data Source
AI summary
A walking robot pose estimation method is provided. The walking robot pose estimation method according to an embodiment of the present disclosure includes the calculation a kinematic factor considering data measured by an inertial measurement unit (IMU) mounted on the walking robot and joint kinematics while the walking robot is in motion, obtaining point cloud data with respect to the environment where the walking robot is located by using the LiDAR sensor mounted on it, obtaining image data with respect to the environment where the walking robot is located by using the image sensor mounted on it, data fusion operation of the kinematic factor, the point cloud data, and the image data, and pose estimation of the walking robot based on the fused data.