Vehicle Image Localization Using Embeddings and Relative Displacement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in accurately determining their localized state due to sensor outages or imprecision, leading to difficulties in navigating their environment effectively.
Innovation Solution
A method involving a machine-learned image embedding model and feature extraction model processes query images to generate embeddings and feature representations, which are compared with pre-computed embeddings and feature representations from a database to determine relative displacements, enabling accurate localization of the vehicle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image data processing is performed in real-time without pre-computed embeddings, then localization accuracy can be maintained, but computational resources and processing time increase significantly
Solution Approach 1:
The patent pre-computes image embeddings and feature representations for reference images and stores them in a database before actual localization tasks. This preliminary processing allows the system to quickly retrieve and compare pre-computed data during real-time operation, maintaining localization accuracy while significantly reducing computational resources and processing time required at runtime.
2Adaptability or versatility
If all reference images are processed and stored in the database, then comprehensive coverage of the environment is achieved, but storage requirements and data processing complexity increase
Solution Approach 1:
The patent extracts only the essential features from reference images by computing image embeddings and feature representations, storing these compact numerical representations instead of the full high-resolution images. This extraction process maintains the ability to identify and localize features in the environment while dramatically reducing the storage volume and processing complexity compared to storing complete reference images.
3Device complexity
If sensor data is used alone for localization, then system simplicity is maintained, but reliability decreases due to sensor outages or imprecision
Solution Approach 1:
The patent introduces image-based localization as an intermediary system that works alongside sensor-based localization. The image processing system compares query images against pre-computed reference images to determine vehicle position, providing an independent verification and correction mechanism that enhances reliability without completely replacing the existing sensor system, thus maintaining a reasonable level of system complexity.
Data Source
AI summary
Systems and methods for determining a location based on image data are provided. A method can include receiving, by a computing system, a query image depicting a surrounding environment of a vehicle. The query image can be input into a machine-learned image embedding model and a machine-learned feature extraction model to obtain a query embedding and a query feature representation, respectively. The method can include identifying a subset of candidate embeddings that have embeddings similar to the query embedding. The method can include obtaining a respective feature representation for each image associated with the subset of candidate embeddings. The method can include determining a set of relative displacements between each image associated with the subset of candidate embeddings and the query image and determining a localized state of a vehicle based at least in part on the set of relative displacements.