Autonomous Vehicle Localization via Semantic Segmentation Object Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image localization systems for autonomous vehicles face challenges in accuracy due to varying extraneous objects and suboptimal sensor data, making precise vehicle positioning and navigation difficult, especially when using semantic segmentation labeling.
Innovation Solution
A system and method utilizing semantic parsing with two phases: a mapping phase where baseline semantic label images are created by removing extraneous dynamic objects, and a localization phase where live image data is processed to compare with the baseline map for accurate vehicle positioning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If semantic segmentation labeling is used for vehicle localization, then object identification capability is improved, but localization accuracy deteriorates due to varying extraneous objects in images
Solution Approach 1:
The patent extracts and removes extraneous dynamic objects from semantic label images by comparing multiple images captured at different time points. Objects that appear only in certain time frames are identified as extraneous and removed, leaving only static objects that provide reliable localization information. This extraction process resolves the contradiction by eliminating the harmful variable (extraneous objects) while preserving the useful capability (semantic segmentation for object identification).
Solution Approach 2:
The patent performs preliminary processing by capturing multiple images over time and pre-identifying extraneous objects before final localization. By anticipating and removing dynamic objects in advance through temporal comparison, the system ensures that only stable semantic features remain for localization, thus improving accuracy while maintaining the benefits of semantic segmentation.
2Speed
If sensor data from autonomous vehicle is used for localization, then real-time positioning is achieved, but data accuracy deteriorates due to suboptimal sensor performance
Solution Approach 1:
The patent merges semantic segmentation results from multiple image frames captured at different time points to create a more accurate representation of the environment. By combining temporal information and performing logical operations on semantic labels across frames, the system enhances data accuracy while maintaining real-time processing capability. This merging approach allows the system to filter out transient errors and strengthen reliable signals.
3Loss of information
If all objects in semantic label image are used for localization, then comprehensive environmental information is obtained, but localization precision deteriorates due to inclusion of extraneous dynamic objects
Solution Approach 1:
The patent segments objects in the semantic label image into two categories: static objects and dynamic extraneous objects. By dividing the set of all detected objects into these distinct groups based on temporal consistency analysis, the system can selectively use only static objects for localization. This segmentation maintains comprehensive environmental awareness while improving localization precision by excluding the harmful dynamic subset.
Data Source
AI summary
A system and method for image localization based on semantic segmentation are disclosed. A particular embodiment includes: receiving image data from an image generating device mounted on an autonomous vehicle; performing semantic segmentation or other object detection on the received image data to identify and label objects in the image data and produce semantic label image data; identifying extraneous objects in the semantic label image data; removing the extraneous objects from the semantic label image data; comparing the semantic label image data to a baseline semantic label map; and determining a vehicle location of the autonomous vehicle based on information in a matching baseline semantic label map.