Stereo Visual Odometry Using Point-Line Matching in Low Light
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing stereo video odometry techniques struggle to accurately determine feature points in low contrast and low light conditions, which affects the system's ability to determine distance and location reliably.
Innovation Solution
The method combines point features with line features using neural networks and image segmentation to enhance feature detection, employing Superpoint and SOLD2 algorithms, and utilizes attention/graph neural networks (attn/GNNs) and differentiable optimal transport processors to improve matching and triangulation in low contrast and low light environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional stereo video odometry techniques are used, then the system can determine location and orientation, but the accuracy deteriorates in low contrast and low light conditions
Solution Approach 1:
The patent combines point features and line features into a unified feature detection framework. By merging these two types of features, the system leverages the complementary strengths of each: point features provide precise location markers while line features provide structural context and orientation information. This combination enables reliable feature detection and matching even when individual feature types become indistinct due to low contrast or poor lighting conditions.
Solution Approach 2:
The patent transitions from traditional 2D image space analysis to 3D spatial reasoning by extracting and utilizing line features that extend into the third dimension. By detecting lines in 3D space and using them for triangulation and pose estimation, the system adds a dimensional aspect that provides additional geometric constraints and information, improving robustness against degraded visual conditions in the 2D image plane.
2Device complexity
If only point features are used for feature detection, then the system is simpler to implement, but the reliability of distance determination deteriorates in challenging lighting conditions
Solution Approach 1:
The patent merges point feature detection and line feature detection into an integrated system. The complexity increase is justified by the significant improvement in reliability: line features provide additional geometric constraints for triangulation and pose estimation, making distance determination more robust when point features alone become unreliable due to low contrast or poor lighting.
Solution Approach 2:
The feature detection system uses a composite approach by combining two different feature types (points and lines) with complementary properties. Just as composite materials combine different substances to achieve superior properties, this composite feature system combines point and line features to achieve detection reliability that exceeds what either feature type can provide alone in challenging visual conditions.
3Measurement precision
If the system uses advanced neural network algorithms for feature matching, then the measurement precision improves, but the computational complexity increases
Solution Approach 1:
The patent replaces traditional mechanical or geometric feature matching algorithms with neural network-based approaches. The neural networks (including attention mechanisms and graph neural networks) learn optimal feature correspondence patterns from data, providing superior matching accuracy. This substitution of computational paradigms achieves higher precision in feature matching and triangulation, enabling reliable 3D reconstruction and pose estimation even in difficult visual conditions.
Data Source
AI summary
A computer that includes a processor and a memory, the memory including instructions executable by the processor to select first side stereo images of first and second pairs of stereo images acquired at first and second time steps, respectively, and mask first and second first side stereo images, determine point features in masked first and second first side stereo images and determine line features in masked first and second first side stereo images. Matching point features in the masked first and second first side stereo images can be determined using a first attention/graph neural network (attn/GNN) based on keypoints determined based on the line features. Matching line features in the masked first and second first side stereo images can be determined using a second attn/GNN based on keypoints determined based on the line features. Three-dimensional (3D) locations in a scene can be determined by determining stereo disparity based on the matched point features included in the first first side stereo image and point features determined in a first second side stereo image of first and second stereo pairs of images and a 3D stereo camera pose can be determined by determining a perspective-n-point and line algorithm on the 3D locations.