Vehicle Visual Localization Using Learned Appearance Transforms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing visual localization systems struggle with large changes in appearance due to variations in lighting conditions or seasonal differences, leading to poor localization performance and the need for extensive mapping under various conditions.
Innovation Solution
A method involving trainable transforms that modify input images to generate representations suitable for localization, using a two-stage training process to enhance feature detection and description, allowing for localization under different conditions without requiring extensive mapping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If standard keypoint feature matching (SURF, SIFT, BRIEF, ORB) is used for visual localization, then localization works well under similar lighting conditions, but performance breaks down when appearance differs due to illumination changes or seasonal differences
Solution Approach 1:
The patent applies parameter changes by transforming images through learned transformation functions that modify appearance parameters (illumination, season, time of day) while preserving geometric structure. This allows the same feature matching algorithm to work across different conditions by normalizing the appearance parameters of query and map images before matching.
Solution Approach 2:
The patent introduces an intermediary transformation function (learned via Cycle-Consistent Adversarial Networks) that acts as a mediator between images of different appearances. This intermediary transform converts images from different lighting/seasonal conditions into a common reference appearance, enabling standard feature matching to succeed where it would otherwise fail.
2Reliability
If mapping is performed for each category of appearance (sunny, snow, dawn, dusk, rain, night) to improve localization robustness, then localization reliability improves, but system complexity and mapping requirements increase
Solution Approach 1:
The patent implements universality by creating a single learned transformation function that handles multiple appearance conditions (different lighting, seasons, times of day) simultaneously. Instead of requiring separate mapping procedures for each condition, one universal transform can normalize any condition to a reference appearance, greatly simplifying the system while maintaining reliability.
Solution Approach 2:
The transformation function learns to adjust multiple appearance parameters (illumination level, seasonal characteristics, time of day effects) in a unified manner, allowing a single system to handle diverse conditions that would otherwise require separate specialized mappings.
3Measurement precision
If mapping is performed for each category of appearance to ensure coverage, then localization accuracy is maintained, but the time and resources required for mapping increase
Solution Approach 1:
The patent applies preliminary action by pre-training the transformation function on diverse appearance conditions before deployment. This preliminary training allows the system to handle unseen conditions during operation without requiring additional mapping time, as the transform has already learned to normalize various appearances to the reference condition.
Solution Approach 2:
A single universally trained transformation function replaces the need for multiple condition-specific mappings, reducing the total mapping time and resources required while maintaining localization accuracy across all conditions.
Data Source
AI summary
A computerised method of generating a first trainable transform arranged to be used in localisation of an entity, the transform being arranged to transform a first representation of an environment to a second, different, representation of the environment, the method comprising processing a plurality of first training representations of an environment using the first trainable transform to generate a transformed first training representation; performing at least one of: i)) running at least one known process on the first training representation and a modified version of the first training representation to generate an error signal where in the process is selected such that the first trainable transform is arranged to enhance features within the first training representation; and ii) running at least one known process on a second training representation, corresponding to the first training representation, but under a different lighting condition, and on the modified version of the first training representation to generate an error signal wherein the process is selected such that the first trainable transform is arranged to enhance features within the first training representation; and; c) using the error signal to train the first transform.