Stereo Visual Odometry via Gradient Joint Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional visual odometry methods face challenges in accurately estimating camera pose and position, especially in complex environments with large displacements and varying lighting conditions, due to limitations in image alignment and stereo vision matching, leading to suboptimal performance in applications like autonomous driving and robotics.
Innovation Solution
A stereo visual odometry method based on image gradient joint optimization, which preprocesses images, calculates disparity maps, and uses a Kalman filter and gradient-based stereo navigation to estimate camera pose, incorporating a pyramid model and Tukey's loss function for robustness and convergence, avoiding the need for affine illumination models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If direct image alignment is used to obtain dense 3D reconstruction, then the information of the entire image is utilized, but the method converges to local optimum and performs weaker in KITTI visual odometry test
Solution Approach 1:
The patent segments the image alignment process into multiple scales using a pyramid model. Instead of aligning the entire image at once, it divides the problem into coarse-to-fine stages, where each scale level handles a portion of the alignment task. This segmentation prevents local optimum convergence by allowing the algorithm to escape from local minima at coarser scales before refining at finer scales.
Solution Approach 2:
The patent introduces a new dimension by transforming the 2D image alignment problem into a multi-scale pyramid structure. By adding the scale dimension, the algorithm can optimize camera pose across different resolution levels, effectively expanding the search space and avoiding local optima that plague single-scale direct methods.
2Device complexity
If Lucas-Kanade framework is used for image alignment, then the alignment process is simplified, but the theoretical convergence domain is limited and unable to cope with large displacement scenes
Solution Approach 1:
The patent makes the alignment process dynamic by implementing a multi-scale pyramid model that adapts to different displacement magnitudes. At coarse scales, the system handles large displacements efficiently, while at fine scales, it refines the alignment for small residuals. This dynamic multi-scale approach extends the convergence domain beyond what a fixed single-scale Lucas-Kanade method can achieve.
3Ease of manufacture
If pixel grayscale values and affine illumination models are used, then the image alignment is straightforward, but the method cannot cope with complex ambient lighting environments
Solution Approach 1:
The patent extracts and removes the illumination model component from the optimization process entirely. Instead of incorporating complex affine illumination models that struggle with real-world lighting variations, it focuses on gradient-based alignment that is inherently more robust to illumination changes. This extraction of the problematic illumination modeling step simplifies the approach while improving reliability in complex lighting environments.
4Area of stationary object
If conventional stereoscopic matching is used to evaluate disparity estimation accuracy, then the coverage area is improved, but the point-to-point disparity estimation accuracy required for visual navigation is not achieved
Solution Approach 1:
The patent merges stereo disparity estimation with direct gradient-based image alignment in a unified optimization framework. By combining these two approaches, the system achieves both the coverage benefits of stereo matching and the precision of gradient alignment, as the joint optimization simultaneously leverages depth information from stereo and gradient information from image alignment for accurate point-to-point correspondence.
Data Source
AI summary
A stereo visual odometry method based on image gradient joint optimization includes steps of: pre-processing each frame of images of a stereo video input; calculating a stereo image disparity map of a stereo video frame, and obtaining a metric distance; scaling to form scale images with different scales for forming a pyramid model, and calculating features to obtain a series of feature images; processing the images with a Kalman filter according to the metric distance, so to predict and estimate a motion process of a camera pose, wherein camera motion models is built in the Kalman filter; accurately calculating a camera pose of a current frame by using a gradient-based stereo visual navigation method; and updating the camera motion models in the Kalman filter with the camera pose of the current frame.