Pose Estimation Using Depth and Sensor-Scaled Visual Odometry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current visual odometry methods, such as monocular and stereo visual odometry, face challenges in accurately estimating the pose of a camera or vehicle using only image sequences without labeled data, leading to imprecise pose information due to the lack of absolute scale in training networks.
Innovation Solution
A processor-implemented method that uses a depth network and a pose network to generate depth images and image pose transformation matrices, calculates a composite loss value by comparing image and sensor position information, and updates the network training based on a total loss value to improve the accuracy of pose estimation, incorporating sensor position information from GPS or indoor sensors to apply an absolute scale.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If visual odometry methods use only image sequences without labeled data, then the training network can operate with unlabeled data, but the pose estimation accuracy deteriorates due to lack of absolute scale
Solution Approach 1:
The patent introduces sensor position information (GPS, indoor position sensors) as an intermediary to bridge the gap between unlabeled image data and absolute scale requirements. The sensor data serves as a mediator that provides ground truth position information without requiring manually labeled training data, enabling the network to learn accurate pose estimation while maintaining adaptability to unlabeled data operation
Solution Approach 2:
The training network is designed to process both image sequences and sensor position information simultaneously, making it multi-functional. The network can operate in modes with or without labeled data by selectively utilizing available inputs, thus achieving universality that resolves the contradiction between adaptability to unlabeled data and maintenance of precision through sensor fusion
2Ease of manufacture
If the network is trained offline, then training can be completed before deployment, but the pose estimation cannot improve during vehicle operation
Solution Approach 1:
The patent implements dynamic training capability that allows the network to transition from static offline training to continuous online training during vehicle operation. The training process is made adaptive by enabling real-time updates using sensor position information as ground truth, allowing the system to evolve and improve reliability while maintaining the option for initial offline training
3Measurement precision
If sensor position information is incorporated to provide absolute scale, then pose estimation accuracy is improved, but the device complexity increases
Solution Approach 1:
The patent replaces complex mechanical or manual positioning systems with sensor-based position information (GPS, indoor sensors) that provides absolute scale electronically. This substitution achieves high measurement precision through software-based sensor fusion rather than complex hardware mechanisms, thereby improving accuracy while minimizing the increase in device complexity
Data Source
AI summary
A method and apparatus with pose estimation, where the method may include obtaining, using a depth network, a respective depth image for each of a plurality of successive input images, obtaining, using a pose network, respective image pose transformation matrices between images, of the successive input images, at adjacent time points, obtaining, based on initial pose information and the respective image pose transformation matrices, image pose information for each of the adjacent times, estimating final pose information dependent on the obtained image pose information, accumulating the image pose transformation matrices, calculating a pose loss value based on a result of comparing image position information, obtained from a result of the accumulating, and sensor position information obtained from a sensor. The pose and depth networks may be updated based on the pose loss value and a composite loss value dependent on the image pose transformation matrices and the input images.