Dynamic Object Masking for Robust VSLAM Feature Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VSLAM systems face errors in mapping and localization due to dynamic objects that change location or appearance over time, which are incorrectly assumed to be static.
Innovation Solution
The system processes images by identifying dynamic objects, generating a masked image by masking these objects, and using features from the masked image for feature tracking, mapping, localization, and relocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If VSLAM systems use all detected objects for feature tracking and mapping, then the quantity of features available for tracking increases, but the accuracy of mapping and localization deteriorates due to dynamic objects being incorrectly assumed to be static
Solution Approach 1:
The system segments the image into static and dynamic portions by detecting dynamic objects and generating a mask. Feature tracking is then performed separately on the static portion (masked image) to exclude dynamic objects. This segmentation resolves the contradiction by dividing the feature set into tracking-worthy static features and non-tracking dynamic features, maintaining both sufficient feature quantity and high tracking accuracy.
2Adaptability or versatility
If VSLAM systems include dynamic objects in the environment map, then the map completeness improves, but the reliability of device location determination deteriorates due to location-changing objects
Solution Approach 1:
The system segments the environment model into static environment data and dynamic object data. The masked image containing only static features is used for reliable device localization and map construction, while dynamic objects are separately tracked and overlaid during rendering. This ensures reliable location determination from static features while maintaining complete environmental awareness.
Solution Approach 2:
The system introduces an intermediary masking operation between image capture and feature processing. The mask acts as a mediator that separates static and dynamic components, allowing the VSLAM system to process only static features for localization while still being aware of dynamic objects for complete environmental modeling and augmented reality overlay purposes.
3Productivity
If VSLAM systems process all image features without differentiation, then the processing speed is maintained, but the productivity of accurate mapping and localization deteriorates due to errors from dynamic objects
Solution Approach 1:
The system performs preliminary action by generating the dynamic object mask before feature extraction and tracking. This pre-processing step identifies and masks dynamic objects in advance, so that subsequent feature detection and tracking operations automatically exclude them. This preliminary segmentation maintains processing efficiency while ensuring high accuracy in the actual mapping and localization tasks.
Data Source
AI summary
Examples are described for processing images to mask dynamic objects out of images to improve feature tracking between images. A device receives an image of an environment captured by an image sensor. The image depicts at least a static portion of the environment and a dynamic object in the environment. The device identifies a portion of the image that includes a depiction of the dynamic object. For example, the device can detect a bounding box around the dynamic object, or can detect which pixels in the image correspond to the dynamic object. The device generates a masked image at least by masking the portion of the image. The device identifies features in the masked image, and uses the features from the masked image for feature tracking from other images of the environment, masked or otherwise. The device can use this feature tracking for mapping, localization, and/or relocation.