Static Feature Localization via Neural Network 3D Geometry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current localization methods for autonomous driving, such as GPS and inertial measurement units, fail to provide sufficient accuracy in complex urban environments, with desired accuracy being around 10 cm for effective route planning, while existing algorithms like MonoSLAM offer limited improvements.
Innovation Solution
A trained neural network is used to identify and generate static feature information for vehicle localization, which is embedded within a digital map, enabling precise pose determination by extracting three-dimensional geometry of invariant features like building facades and lane lines, and storing this information in a static feature library for real-time localization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If GPS is used for vehicle localization, then real-time location information is provided, but localization accuracy deteriorates to 7.8 meters with 95% confidence interval and up to 30 meters error in urban environments
Solution Approach 1:
The patent segments the localization task into multiple independent feature detection processes (lane lines, buildings, trees, signs) that operate in parallel. Each feature type is detected separately using specialized neural network branches, and their results are combined to achieve high-precision localization. This segmentation allows the system to overcome GPS limitations by using multiple independent feature sources rather than relying on a single signal source.
Solution Approach 2:
The patent introduces static features (lane lines, buildings, trees, signs) as intermediary elements between the vehicle and the final localization result. These features serve as stable reference points that mediate the localization process, allowing the system to achieve centimeter-level accuracy by matching detected features against pre-stored map data, thereby overcoming direct GPS signal limitations.
2Measurement precision
If inertial measurement units (IMUs) are used to increase localization accuracy by accounting for vehicle movement, then movement compensation is achieved, but accuracy deteriorates due to sensor drift
Solution Approach 1:
The patent implements feedback by continuously detecting static features in the environment and comparing them against pre-stored map data. This continuous feedback loop allows the system to correct drift accumulation from IMUs in real-time. The detected feature positions provide ongoing reference information that feeds back to the localization algorithm, enabling the system to maintain high accuracy over extended periods despite sensor drift.
Solution Approach 2:
The patent creates a digital copy of the static environment (buildings, trees, signs, lane lines) and stores it in advance in map data. This copied representation serves as a stable reference that does not suffer from drift. By comparing real-time sensor data against this static copy, the system can identify and correct drift accumulation, thereby maintaining long-term localization accuracy.
3Productivity
If MonoSLAM algorithm is used for real-time single camera SLAM, then sparse map creation is achieved, but localization accuracy remains insufficient for autonomous driving requirements
Solution Approach 1:
The patent merges multiple feature detection capabilities (lane line detection, building detection, tree detection, sign detection) into a unified neural network system. By combining these different detection branches and integrating their results with pre-stored map data, the system achieves both real-time processing and high localization accuracy, overcoming the limitations of MonoSLAM's sparse mapping approach.
Solution Approach 2:
The patent transitions from traditional 2D image processing to 3D spatial reasoning by detecting and matching features across multiple dimensions. The system extracts three-dimensional information from two-dimensional images by detecting features like building heights, tree positions, and lane line geometries, then matches these 3D features against stored map data, thereby achieving higher precision than conventional 2D SLAM methods.
4Measurement precision
If static features are used for high-precision localization, then localization accuracy improves to around 10 cm, but system complexity increases due to need for multiple sensors and processing algorithms
Solution Approach 1:
The patent creates a universal neural network system that handles multiple feature types (lane lines, buildings, trees, signs) and multiple detection tasks (detection, recognition, localization) within a single integrated architecture. This multi-functional system reduces overall complexity by consolidating what would otherwise require separate algorithms and processing pipelines, while still achieving 10 cm localization accuracy through static feature matching.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A first image and a second image are provided to a trained neural network. The first image comprises one or more static features and the second image comprises at least one of the one or more static features. A static feature is identified in both the first and second images by a branch of the trained neural network. A three dimensional image comprising the identified static feature is generated and three dimensional geometric information/data related to the static feature is extracted and stored in association with a tile of a digital map. A set of training images may be used to train the trained neural network comprises training image subsets comprising two or more images that substantially overlap that were (a) captured at different times; (b) captured under different (i) weather conditions, (ii) lighting conditions, or (iii) weather and lighting conditions; or both a and b.